How to Blend Without Blitting Transparent Pixels

[Dianna Houx]

I am really confused by this problem.  This has got to be a simple fix and I am just missing how to do it.

EXAMPLE:
I have a heart and a snowman, each on their own layer.  With normal blending, I have this:

 

What I want to do is composite the heart on the snowman so that transparent pixels on the layers below the heart act as a bitmask/hardmask so that the snowman essentially acts as a clipping mask for the heart.  I want to achieve this:

 

Some folks would say "Well you just need to create a mask object/layer using the transparency data from the snowman as a source."
Except that's not an actual solution, because it has serious implications.
What happens if I want to paint directly on the snowman or erase part of the snowman?  I would be forced to recreate the mask all over again.
Or what If I'm trying to create text effects and the snowman is word?  I want to edit the text without having to completely recreate the mask.

This really should be a solved problem.  Compositing mathematics with alpha channels is well understood and I can easily use ImageMagick or even javascript with an HTML5 Canvas to get the compositing effect described.  Either this is completely missing from Affinity or its so well buried and hidden that I just can't find the feature.

Basically I should be able to configure a layers blending operation to say that it should not draw anything ontop of transparent pixels.
How can I do this?

 

[thomaso]

Hi @Dianna Houx, welcome to the Affinity forums!

You can simply use a layer as parent clipping layer (your snowman) for a nested child layer (your heart), like so:

To avoid painting on transparent areas activate the Brush option "Protect Alpha" in the Context Toolbar.

[Dianna Houx]

Oof! Apparently, I need to reframe my question.

I tried to come up with the simplest possible example so I restricted this to two layers.  I see that its a non-issue when there are two simple layers.

What if the snowman is a composite resulting from a group of other layers??

I want to achieve the same effect but have the heart only composite onto the result of all the snowman group.  I cannot seem to figure out how to do this.

[Dianna Houx]

Here's an example.  The effect is wrong.

[thomaso]

Yes, unfortunately it doesn't work with a Group. There may be various ways to achieve this, one would be using a separate mask for the clipped object, for instance with a copy of the group as a nested masking object. To move or scale the masked object without changing the mask you can activate the option "Lock Children" in the Context Toolbar.

If you have Affinity Designer, you could create a "Symbol" of the group to automatically propagate later changes to the group to the mask.

[Dianna Houx]

Ugh.  It feels quite lackluster that something as simple as compositing with ignore-alpha is missing. 

The symbol solution seems half-way reasonable work-around.  I do have both Designer and Photo.  How would I setup the symbol work around?

I greatly appreciate your help so far. thank you!

[Dianna Houx]

I've thought more about this.  And this really does seem like a major design flaw in Affinity that is correctable.

The blend modes work quite unintuitively with transparency.  If you multiply a color with 100% transparency, the result should *not* be an addition of fully opaque color.  The result should be no effect.  The underlying 100% transparent pixel should be the resulting output.

Changing this would make it far easier to build compositions that blend on top of transparent/clipped imagery.

This isn't even a tall request.  It should amount to a few lines of code in the primary compositing stack.

In Photoshop and every single other compositor I've ever worked in, you have the ability to composite while retaining transparency.  I suspect many experienced users who try Affinity and see this feature missing just go back to their existing tools.  For some people who leverage the power of blending modes and composition stack, Affinity makes compositing unnecessarily difficult.

[Dianna Houx]

Example.  This is just such a weird default behavior:

 

Multiplying the red/orange tones of the heart with a fully-transparent/non-existing/undefined background produces "Add heart color to zero".
What???????
It should produce undefined/sustained-full-transparency.

 

[Dianna Houx]

Btw, turning the Group into a Symbol and then using that Symbol as a mask directly on the heart worked.  So I greatly appreciate that!!

But I insist that the default math for the blending/compositing in Affinity is *broken/ill-designed* and really should be fixed

Edited January 8 by Dianna Houx

[Dianna Houx]

I'm beginning to think Affinity's compositing stack is broken in multiple ways.

Observe the following.  I have a letter G composed entirely of greyscale values.
I add a gradient map on top of it to tone map the greyscale values to gold colors.
You can see that the stroke around the letter is treated differently than the fill area.
In no world would this be expected behavior.
What you would expect is that all of the FX on the layers in the symbol would be composited together and the symbol would produce a final output for for the gradient map.
Instead, the elements from the layers within the symbols are somehow getting combined independently with the gradient map.  Totally strange.




Here, I've moved the gradient Map outside the nested groups and the result is now correct (ignoring the side-effect that the grey background is also impacted):

[Dianna Houx]

Putting the gradient map on top of the nested group but inside the outer group results in still-broken "grey" edge that is improperly mapped by the gradient map.  There's a blend operation happening that should not happen.

This is also wrong:

[Dianna Houx]

Nevermind on the G stroke behavior.  Turns out that the layer FX had an outline that was partially transparent.  So this is not an issue.

But the original snowman demonstration still demonstrates a compositing behavior flaw.

[Medical Officer Bones]

3 hours ago, Dianna Houx said:

Example.  This is just such a weird default behavior:

 

Multiplying the red/orange tones of the heart with a fully-transparent/non-existing/undefined background produces "Add heart color to zero".
What???????
It should produce undefined/sustained-full-transparency.

 

That's how it also works in Photoshop, Krita, PhotoLine, ClipStudio, Pro Motion NG, VectorStyler, Illustrator, ... To name but a few. Are you saying they all got it wrong?

One of the few exceptions to this behaviour is GIMP. GIMP will multiply a foreground's layer pixels with the underlying transparent background to full transparency. But 90% of other design software behaves the opposite.

That said, a provision to ensure the opposite behaviour should be made available, of course. Other design software resolves this through a layer specific setting such as Protect UnderlyingTransparency (PhotoLine), or Clip to Layer Below (ClipStudio Paint), Clipping Masks (Photoshop), or other layer blend settings.

I agree that it would be very useful if groups in Affinity Photo could be used or set to clip their contents without the use of extra layer masks.

[thomaso]

7 hours ago, Dianna Houx said:

The blend modes work quite unintuitively with transparency.  If you multiply a color with 100% transparency, the result should *not* be an addition of fully opaque color.  The result should be no effect.  The underlying 100% transparent pixel should be the resulting output.

If I understand your idea correctly, a "Shadow" effect in blend mode "Normal" on a transparent background (e.g. an empty page in APub) would appear like on a white background – whereas a "Shadow" effect in blend mode "Multiply" on a transparent background would not be visible?

Maybe this description in the "PDF Reference, version 1.6" (2006) helps: Acrobat PDF spec _blend modes.pdf

Quote

Blend mode "Multiply": The result color is always at least as dark as either of the two constituent colors. Multiplying any color with black produces black; multiplying with white leaves the original color unchanged. Painting successive overlapping objects with a color other than black or white produces progressively darker colors.

Nevertheless, there are known issues/bugs in Affinity with semi-transparent colour on transparency (vs. white background) and/or certain blend modes. For instance discussed in this thread (above and below the video) and other threads:

[R C-R]

25 minutes ago, thomaso said:

Maybe this description in the "PDF Reference, version 1.6" (2006) helps:

Somewhat similarly, from the AP  Layer blend mode help topic:

Quote

• Multiply—The blending result is a combination of the top and bottom color at each pixel position, always producing a darker value.

So it isn't quite as helpful as the reference but does at least suggest that this mode is working as it should.