Invisible/Non-Refinable/Immovable Fill Layer Mask

[Mr. Doodlezz]

Hello Community!

Question regarding fill layers, selection marquee and masks (in all applications of the suite, but especially in Photo due to my workflow in that app):

I often find myself in a situation where I already have an active selection marquee and need to create a fill layer to which I want to apply this selection as a mask.

Way to replicate this:

1. With an active selection marquee, navigate to Layer → New Fill Layer.
2. Click on New Fill Layer – the active selection marquee is now gone and an invisible, inaccessible, non-editable mask (?) is created/applied to the fill layer. (Unless I'm missing something).

Question: How can I access this mask/delete the automatically generated invisible »mask«? Is there any way to change this behaviour? The Affinity Help mentions nothing about it.

And yes, I know there are a few workarounds for this. 😉
For example, plan ahead and create the fill layer first, or select the newly generated fill layer with the invisible mask and convert it into a regular mask, and so on.
But this is not always so easy and costs at least a few extra steps that I would like to save myself.

Expected behaviour:

I would rather have the selection converted and applied as a normal mask to the fill layer or remain an active selection.

Cheers and happy weekend
Dennis

 

Edited April 30, 2023 by Mr. Doodlezz
Clarified what I mean by »selection« – I'm talking about a marquee/marching-ant-selection, not a selected layer or similar. // Expanded title to reflect what this is all about.

[walt.farrell]

If you have a pixel selection active when you create a Fill Layer, the layer is restricted by the selection rather than filling the entire canvas. Is that what you're referring to?

I am a bit confused by your description, because (for me) the selection still exists, and the marching ants can be seen.

In any case, you might try this workflow instead:

• Click somewhere to dismiss the selection, or Select > Deselect.
• Create the Fill Layer.
• Select > Reselect.

Edit: Also, if you have created a Fill Layer that is restricted to the pixel selection, you can dismiss the pixel selection (Select > Deselect, for example) and then paint using a White brush to show more of the Fill Layer. Whenever you paint on a Fill Layer you are painting on its mask.

[Mr. Doodlezz]

25 minutes ago, walt.farrell said:

If you have a pixel selection active when you create a Fill Layer, the layer is restricted by the selection rather than filling the entire canvas. Is that what you're referring to?

I am a bit confused by your description, because (for me) the selection still exists, and the marching ants can be seen.

In any case, you might try this workflow instead:

• Click somewhere to dismiss the selection, or Select > Deselect.
• Create the Fill Layer.
• Select > Reselect.

Edit: Also, if you have created a Fill Layer that is restricted to the pixel selection, you can dismiss the pixel selection (Select > Deselect, for example) and then paint using a White brush to show more of the Fill Layer. Whenever you paint on a Fill Layer you are painting on its mask.

Oh, I never realised there's a Reselect option, interesting.

Anyway, according to this workaround, and with an active selection, that's 8 steps/clicks (Selection → Deselect; Layer → New Fill Layer; Selection → Reselect; Layer → New Mask Layer) instead of just 2 clicks via »Layer → New Fill Layer«, where (according to my previous habit, expected behaviour and the way it works in Photoshop), the new fill layer would get a regular mask instead of a hidden mask.

I'm not sure what else you'd call it than »hidden mask«, but here's an example file. It contains a single fill layer that was created with an active selection (marching ants). There's – to my knowledge – no way to remove or edit the »inivsible mask« for that fill layer.

Fill_Layer_Invisible_Mask.afphoto

[walt.farrell]

45 minutes ago, Mr. Doodlezz said:

There's – to my knowledge – no way to remove or edit the »inivsible mask« for that fill layer.

Again, paint on the Fill Layer with a White brush, while no pixel selection is active, just as you could with an Adjustment Layer.

[NathanC]

As far as I'm aware Fill layers don't have masks applied to them in the traditional sense like you find with adjustments, filters etc which creates an editable mask visible on the layers panel when applied with an active selection.

As Walt mentioned, after creating the fill layer with a pixel selection that restricts the fill layer to the selection bounds on the canvas. You can still select that fill layer, and then with the paint brush tool subtract from the layer with the paint brush set to black and add back to the fill layer with white, which is essentially editing the layer mask.

I've attached a quick recording demonstrating this behaviour. It's worth noting that when I switch between my black/white brush colours to mask via the X shortcut, the second colour spot automatically changes to my fill layer's colour to facilitate the content being added back, whilst with a traditional mask layer editing with the brush is black/white/grey only. If you Deselect your active selection you can also remove/add  anywhere else on the canvas area with your fill layer selected.

 

 

[Mr. Doodlezz]

Yes, you're both sort of right, and yes, I know you can edit the »mask« that way – still, that's not really the answer to my question (or maybe it wasn't that clear) why there isn't a regular mask like there is with all the other layers/filters. By the way, I'm not new to Affinity Suite at all, if that's what I came across. 😄

Also the way this mask works comes with several downsides:

1. You can never see the actual mask without changing the colour to pure black and adding yet another pure white layer as a background. With any other mask, you can simply Alt/Option-click on the mask in the Layers panel to get a view of the actual mask, which you can then edit in black and white.
2. You can't refine the »invisible mask« of fill layers like you can refine any other mask in the traditional sense. (I don't mean just draw/erase on it, but actually use the refine mask option, see the attached video at around 00:00:10).
3. It behaves differently when applying a live adjustment/filter – here the marquee selection actually gets converted into a traditional mask instead of being applied invisibly to the adjustment/filter, like it does with fill layers. Interestingly, if you try to draw directly on a live filter, a common mask is added.If you try to draw directly onto a live filter it results into a general mask being added.

For me, the way a marquee selection works with a subsequent fill layer is just a logical break compared to how it works with other »special layers«.

If I want a layer that does not use a real mask and is pure colour, what is the advantage of a fill layer over a pixel layer in this situation? (Being able to change its colour or add a gradient isn't an advantage, because I could do that with a bucket tool or other filters just a quickly on a pixel layer.)

[walt.farrell]

46 minutes ago, Mr. Doodlezz said:

If I want a layer that does not use a real mask and is pure colour, what is the advantage of a fill layer over a pixel layer in this situation? (Being able to change its colour or add a gradient isn't an advantage, because I could do that with a bucket tool or other filters just a quickly on a pixel layer.)

It's faster to use the Context Toolbar to change its color, than it is to use the Flood Fill Tool.

Also, a gradient on a Fill Layer is editable, but a gradient on a pixel layer isn't. You have to start over to change it on a pixel layer.

[Mr. Doodlezz]

5 minutes ago, walt.farrell said:

It's faster to use the Context Toolbar to change its color, than it is to use the Flood Fill Tool.

Also, a gradient on a Fill Layer is editable, but a gradient on a pixel layer isn't. You have to start over to change it on a pixel layer.

(Appreciated, but that's not the original topic of this thread, Walt.

Layer effects such as Gradient Overlay also retain the editability of the gradient, even on a pixel layer.)

[walt.farrell]

12 minutes ago, Mr. Doodlezz said:

(Appreciated, but that's not the original topic of this thread, Walt.

Layer effects such as Gradient Overlay also retain the editability of the gradient, even on a pixel layer.)

But you specifically asked why one would use Fill layers, not pixel layers, and gave gradients as an example. Gradients are not the same as Gradient Overlays.

[Old Bruce]

Hi @Mr. Doodlezz,

Have you considered using the Channels panel to save the Selection? Then you can apply that to adjustment layers or other layers as a mask.

[walt.farrell]

By the way, if you do want a "real" mask for a Fill Layer, you can just click the Mask button at the bottom of the Layers panel and add it.

[Mr. Doodlezz]

53 minutes ago, walt.farrell said:

But you specifically asked why one would use Fill layers, not pixel layers, and gave gradients as an example. Gradients are not the same as Gradient Overlays.

Yes, it was more of a rhetorical question. I mean that's why I originally wanted to use a fill layer in the first place. But, again, not what this thread is initially about.
It's about the way it get's masked.

29 minutes ago, walt.farrell said:

By the way, if you do want a "real" mask for a Fill Layer, you can just click the Mask button at the bottom of the Layers panel and add it.

Please also read my original workflow. That's not particularly helpful in this matter, to be quite honest.

52 minutes ago, Old Bruce said:

Hi @Mr. Doodlezz,

Have you considered using the Channels panel to save the Selection? Then you can apply that to adjustment layers or other layers as a mask.

Hey @Old Bruce, unfortunately hat would make things even more tedious.

–––

Let me rephrase the question/s, to make it perfectly clear:
A) Why doesn't Affinity handle marquee selection conversion for fill layers like it does with any other layer, what makes a fill layer the exception?
B) Why doesn't Affinity handle marquee selection conversion for fill layers like Photoshop handles them – creating a fill layer with a regular layer mask of the current marquee selection in one go?

 

[Old Bruce]

4 hours ago, Mr. Doodlezz said:

I often find myself in a situation where I already have an active selection and need to create a fill layer to which I want to apply this selection as a mask.

If I have a selection made (I assume we are talking about a marquee or marching ants selection, not a "selected Layer" selection) and then hit the New Fill layer I get a fill layer with the colour filling only the selection.

If I want a mask I use the mask button at the bottom of the layer's panel. and then use the Layer > Refine Mask command or a right click on the mask and choose Refine mask from that.

[walt.farrell]

21 minutes ago, Mr. Doodlezz said:

A) Why doesn't Affinity handle marquee selection conversion for fill layers like it does with any other layer, what makes a fill layer the exception?

Because it's more convenient to treat them similarly to Adjustment layers, as (like Adjustment layers) they have a built-in Mask.

 

 

[Mr. Doodlezz]

Just now, Old Bruce said:

If I have a selection made (I assume we are talking about a marquee or marching ants selection, not a "selected Layer" selection) and then hit the New Fill layer I get a fill layer with the colour filling only the selection.

If I want a mask I use the mask button at the bottom of the layer's panel. and then use the Layer > Refine Mask command or a right click on the mask and choose Refine mask from that.

Now you have repeated Walt's first comment almost 1:1. 🙈

[Mr. Doodlezz]

10 minutes ago, walt.farrell said:

Because it's more convenient to treat them similarly to Adjustment layers, as (like Adjustment layers) they have a built-in Mask.

 

 

Okay, so let's turn the question around: Why doesn't this (arguable) convenience apply to all Live Filters?

And how convenient is it really to start with an »invisible mask« that can only be edited by drawing and erasing, instead of a real mask from the very beginning?

[walt.farrell]

2 minutes ago, Mr. Doodlezz said:

Okay, so let's turn the question around: Why doesn't this (arguable) convenience apply to Live Filters?

What "arguable convenience"? Having a built-in mask? It does apply.

Or, omitting the display of the mask for Live Filters as is done for Fill Layers? I can only guess, but if I had to guess it's because the content of a Fill Layer is much more known/static than the content of a Pixel layer that you might be applying a Live Filter to. And therefore, there is more need to be able to see what the mask contains when dealing with a pixel layer than when dealing with a Fill layer, usually.

[Old Bruce]

I guess my problem is that I honestly don't understand why you want to use a Fill layer if you are wanting a Mask. Are you actually wanting a Mask? Are you needing the fill layer? I truly don't understand what it is you are trying to achieve here. 

4 hours ago, Mr. Doodlezz said:

Way to replicate this:

1. With an active selection, navigate to Layer → New Fill Layer.
2. Click on New Fill Layer – the active selection is now gone and an invisible, inaccessible, non-editable mask (?) is created/applied to the fill layer. (Unless I'm missing something).

Question: How can I access this mask/delete the automatically generated invisible »mask«? Is there any way to change this behaviour? The Affinity Help mentions nothing about it.

"How can I access this mask"

You could try using the Channels panel. One of the alphas may help you.

[Mr. Doodlezz]

55 minutes ago, Old Bruce said:

I guess my problem is that I honestly don't understand why you want to use a Fill layer if you are wanting a Mask. Are you actually wanting a Mask? Are you needing the fill layer? I truly don't understand what it is you are trying to achieve here. 

"How can I access this mask"

You could try using the Channels panel. One of the alphas may help you.

Yeah, I feel like I'm repeating myself over and over. 🥴

One last time, as a clip with my timestamps as follows:

00:00 - 00:26 – Expected behaviour when clicking on Live Filter or (Live) Adjustment. With an active marquee selection, the selection instantly gets translated into a separate mask layer. That mask is accessible, movable and refinable through »Refine Mask« and also viewable with Alt/Option-click on the mask in the layers panel (Alt/Option-click not demonstrated).

00:26 - 00:47 – Unexpected behaviour when clicking on New Fill Layer. With an active marquee selection, the selection gets instantly translated into an invisible mask. That mask is inaccessible/immovable and cannot be refined through »Refine Mask« and is not viewable with Alt/Option-click on the mask (because obviously there is no mask) in the layers panel (Alt/Option-click not demonstrated).

00:47 - end – This is what I would expect, manually recreated through unnecessary extra clicks/steps and not always feasible pre-planning: A normal mask added to the fill layer that you can access, refine or move between layers, not just draw/erase on.

I hope this helps to clarify once and for all what this is all about.

[Old Bruce]

15 minutes ago, Mr. Doodlezz said:

This is what I would expect, ...: A normal mask 

Try this, once you have your selection made make a mask. Then after deselecting everything make the fill layer. 

[NotMyFault]

6 hours ago, Mr. Doodlezz said:

Question: How can I access this mask/delete the automatically generated invisible »mask«? Is there any way to change this behaviour? The Affinity Help mentions nothing about it.

Use channels panel, fill alpha. And it’s gone.

 

[NotMyFault]

4 hours ago, NathanC said:

As far as I'm aware Fill layers don't have masks applied to them in the traditional sense like you find with adjustments, filters etc which creates an editable mask visible on the layers panel when applied with an active selection.

As far as I tested, they behave exactly like any other adjustment or filter layer.

• fill layers have an inherent mask
• you can use brush or erase brush to edit the mask
• you can use channels panel to fill/clear/invert the mask, or use create spare spare channels etc

[NotMyFault]

4 hours ago, Mr. Doodlezz said:

You can never see the actual mask without …

Just use the channels panel and activate alpha channel only - this works consistently for all layer types.

what you describe as unexpected is in simply consistent behavior. I understand that this seems unexpected in case of fill layers, but it isn’t. Fill layers have inherent masks like any adjustment or filter layer. They work identical. 
the only surprises come from color panel. If you have no brush active, changing colors will change the fill layer color directly.

but if you activate a brush, it will paint on the inherent mask (again, 100% consistent to any other adjustment layer).

i struggled to understand this for a long time, too. But it is completely as designed and makes sense once you get used to it.

in case you have any remaining doubts: 

instead of a fill layer, add a channels mixer layer. Then set the offset of the RGB channels to any suitable value. The result is similar to a fill layer, and behaves with respect to the inherent mask exactly like a fill layer.

[Mr. Doodlezz]

I still don't feel understood and that you claim there is no difference in behaviour. All the while it's pretty obvious, and my last video is proof, that it does in fact behave differently.

I'm not interested in the detours – I want to work efficient.
It's quite clear to me that I can somehow get a normal mask to fill layer by other means. As stated in my opening post. 😑
But again, I am not interested in detours. I want a one (well, two) click solution that creates a normal layer mask that's accesible without a detour to alpha masks or reselection shenanigans, just like any other layer mask. If someone were to follow the workflow as seen in my clast clip, they'd get what I'm after.

Why am I forced to go through these detours or mind games about what to do first when it comes to working exclusively with marquees and fill layers in the first place, why (is Affinity forcing) a different behaviour exclusively for the fill layer than for all the other special layers?

@NathanC, since you have already replied: I'd be happy if you'd take another look at the last clip I posted, maybe you can follow my train of thought? 🤔

[Mr. Doodlezz]

17 hours ago, NotMyFault said:

what you describe as unexpected is in simply consistent behavior.

As you can see in my clip, it is – at least according to my definition – exactly the opposite of consistent behaviour. 🤔

The Fill Layer is the only exception to this workflow. And the workflow (again) is: 1. Have a marquee, 2. create a new layer (Adjustments, Live Filter, Fill Layer or really any other layer), 3. marquee gets converted into a visible, accessible, movable layer mask.

This workflow isn't something new that I've come up with, as you can see when working with marquees and other layers in Photo (last clip above), or when working in Photoshop (also seen in a clip above). It's the way it's actually designed to work, and it's very likely what people switching from Adobe to Affinity expect and are used to. Even the Photopea web application converts an active marquee into a real layer mask when a new fill layer is added. I haven't tried GIMP or other software, but I strongly suspect that they also convert it to a regular layer mask, rather than an embedded, invisible mask.