Editing a mask?

[Trev Raymond]

Hello, I am new to masks and have a question I am really hoping has an easy answer.   I have created a document imported an image and then created  a mask.  I am wondering what are the steps I must follow to edit the mask to make further adjustments as no matter what I try I cannot get into it. Apologies for the beginner question.

[toltec]

13 hours ago, Trev Raymond said:

Hello, I am new to masks and have a question I am really hoping has an easy answer.   I have created a document imported an image and then created  a mask.  I am wondering what are the steps I must follow to edit the mask to make further adjustments as no matter what I try I cannot get into it. Apologies for the beginner question.

You can just paint on the mask layer. Make sure just the mask layer is selected by clicking on the mask layer thumbnail (see below), choose a Paint Brush and either black paint (to add to the mask and erase the image) or white paint (to remove the mask and reveal the image) and paint on the mask layer. As you do, you can see the effect on the image.

You can also paint in grey for a semi transparent mask.

 

If you  right click on the mask, there are also some Edit options

Edit Mask displays the mask as black and white.

Refine Mask loads in into the Refine Selection panel.

Release Mask just moves it outside the image layer. So it is no longer nested.

P.S. If you are using Affinity Designer, you will need to be in Pixel Persona to paint on it.

 

[Dan C]

Hi Trev Raymond,

Welcome to the forums  
To edit a mask layer, simply select it in the layers panel. To focus the document only on the mask layer (or any layer for that matter) you can hold the alt key when selecting the layer, see below for a quick GIF showing this and a tutorial video showing a few more uses!
 

 

[Haitch]

I would like to be able to edit / revise a mask after creating it, using curves or levels .. ideally as an adjustment layer specifically for the mask, but I'd settle for simple adustment of the mask, as in Photoshop. I routinely use the image itself as a mask, for example to protect highlights, and it's essential to be able to adjust the tonality of the mask while editing.

[duncang]

It's 2021 and I am right clicking on a Mask Layer and see no Edit or Refine menu options - what gives here ?

[Dan C]

2 minutes ago, duncang said:

I am right clicking on a Mask Layer and see no Edit or Refine menu options - what gives here ?

These options will only show for layers that have been clipped and masked to a layer, currently your Mask is only clipped to the layer, which means the behaviour is slightly different and the context menu reflects this.

Please select the mask layer, then drag a drop it over the thumbnail of your layer, this should look as follows -

You should now find the right-click context menu provides the expected options. I hope this helps!

[Haitch]

45 minutes ago, duncang said:

It's 2021 and

Welcome to Affinity .. there's a bunch of us have been asking for some time for Affinity to give us the option to work with masks as we would in Photoshop, where the mask is a raster layer that you can just go in and edit like any other image. At the moment, most stuff you would do with masks in Photoshop can be achieved in Affinity, but the additional steps / hassle mitigate against efficient workflow. Affinity does a lot of things brilliantly, but a lot of things incredibly clumsily. I'm sticking with it, sort of, for some stuff ..
Please, Affinity, give us an option: 'Default to using raster masks like Photoshop does' in our basic settings. We don't mind the alpha masks being there for those who like them. This surely can't be difficult to implement.

[duncang]

18 hours ago, Dan C said:

These options will only show for layers that have been clipped and masked to a layer, currently your Mask is only clipped to the layer, which means the behaviour is slightly different and the context menu reflects this.

Please select the mask layer, then drag a drop it over the thumbnail of your layer, this should look as follows -

You should now find the right-click context menu provides the expected options. I hope this helps!

Sorry I don't understand what you are trying to show here - you're showing an animated image - what state should it look like ?  My mask has been dropped on the image layer (background) as can be seen by the indented level and the down arrow on the background layer.  It's not really clear to me how that is any different to what you are showing - assuming end state is the indented mask icon.

[Dan C]

Apologies for the confusion!

When nesting layers into one another in Affinity applications, there are multiple 'hit zones' that you can drag and drop a layer to, depending on the action you want.

When drag and dropping a mask layer to an image layer etc, if you see the highlight along the bottom of the layer, this means that the mask will be clipped to the image layer -

When a Mask is clipped to a layer, you will not see the Mask options when right clicking it, as the layer type is not correct for this context menu.

If we drag and drop the mask layer over the thumbnail of the image layer, then this will clip and mask the mask layer to the image - 

Once a mask has been clipped and masked to your image layer, by drag and dropping the mask into the Thumbnail of the image layer, then the right click Mask menu will appear as expected.

I hope this clears things up!

[kirkt]

If you want to use a grayscale representation of an image as a mask (for example, using the luminosity of the image to modulate an effect), you need to create that grayscale image and then apply it as a mask to the layer you want to modulate.  For example, open an image as the Background layer.  Make a Recolor adjustment layer above the Background - it will recolor the Background image as it should.  Let's say you want the recoloring to take place only in the shadows (the darker portions of the Background image).  In the Channels palette, right click on the Red, Green, or Blue channel of the Background layer (whichever gives you the proper contrast and luminosity you want) and select "Create mask layer."  A new layer appears in the layer stack that is a mask.  Drag it onto the Recolor adjustment layer and it will modulate the recolor adjustment - in this case, because you wanted to recolor the dark portions of the image, you would need to INVERT the mask layer - select it and use the keyboard shortcut CMD-I (capital letter i).  Done.

If you do not want to use a channel as above as the mask, you can make a grayscale copy of the image with normal tools and then right-click the resulting grayscale image layer and choose "Rasterize to mask" and drop that on the Recolor adjustment layer.

If you then ALT-click on the mask layer, it will be displayed as a grayscale image, just like in PS.  In this view, you can paint on it, etc.  You can also modify it will an appropriate adjustment layer, like a curves adjustment layer - add a Curves adjustment layer to the layer stack and drag it onto the Mask that is already nested in the Recolor adjustment.  It should appear below the Mask.  Then, and here is the KEY to making this work, in the Curves adjustment dialog, you need to select the RGB ALPHA curve to modulate the mask.  Manipulating the Master curve, or the R, G, or B curves will do nothing to a mask.  Now you can change the contrast of the mask with a non-destructive adjustment layer, which you cannot do in PS (that is, when you adjust a mask in PS with a Curve, it is a destructive change to the mask).

Kirk

[3DJ]

On 6/9/2021 at 4:22 PM, kirkt said:

If you want to use a grayscale representation of an image as a mask (for example, using the luminosity of the image to modulate an effect), you need to create that grayscale image and then apply it as a mask to the layer you want to modulate.  For example, open an image as the Background layer.  Make a Recolor adjustment layer above the Background - it will recolor the Background image as it should.  Let's say you want the recoloring to take place only in the shadows (the darker portions of the Background image).  In the Channels palette, right click on the Red, Green, or Blue channel of the Background layer (whichever gives you the proper contrast and luminosity you want) and select "Create mask layer."  A new layer appears in the layer stack that is a mask.  Drag it onto the Recolor adjustment layer and it will modulate the recolor adjustment - in this case, because you wanted to recolor the dark portions of the image, you would need to INVERT the mask layer - select it and use the keyboard shortcut CMD-I (capital letter i).  Done.

If you do not want to use a channel as above as the mask, you can make a grayscale copy of the image with normal tools and then right-click the resulting grayscale image layer and choose "Rasterize to mask" and drop that on the Recolor adjustment layer.

If you then ALT-click on the mask layer, it will be displayed as a grayscale image, just like in PS.  In this view, you can paint on it, etc.  You can also modify it will an appropriate adjustment layer, like a curves adjustment layer - add a Curves adjustment layer to the layer stack and drag it onto the Mask that is already nested in the Recolor adjustment.  It should appear below the Mask.  Then, and here is the KEY to making this work, in the Curves adjustment dialog, you need to select the RGB ALPHA curve to modulate the mask.  Manipulating the Master curve, or the R, G, or B curves will do nothing to a mask.  Now you can change the contrast of the mask with a non-destructive adjustment layer, which you cannot do in PS (that is, when you adjust a mask in PS with a Curve, it is a destructive change to the mask).

Kirk

This is what I am trying to do, I gave up on my iPad and am now on Desktop, miserable experience so far... I have an ambient occlusion pass for my 3D render (grayscale) that I converted to a mask. Ideally I would like to have curves to adjust the mask, and that mask would mask another set of adjustments curves etc. If I option click on the mask, I do see it, but I cant select a curves layer that is nested with the mask and adjust it, the preview goes away of the mask. I would take destructive workflows at  this point, just so damn frustrating. 

I may have it working some-what, I put a solid fill within the group to see whats happening, it looks like it is doing something but it doesn't effect the layers beneath and the preview when switching on and off works sometimes- other times not. This is feeling really buggy to me.

 

Any ideas from anyone would be appreciated.

 

-Jim

 

[3DJ]

I got it working, really odd, I have tried and tried walked away and it is now working - logically at that too. 

[kirkt]

19 hours ago, 3DJ said:

I got it working, really odd, I have tried and tried walked away and it is now working - logically at that too. 

LOL.  Glad it is working.

kirk

[PhilOsborne]

I just can’t get masks to work as you would expect. While the mask layer is selected, eraser acts like painting black, it masks the image. Painting black does the same thing. But painting white does nothing at all … surely that should reveal the image again? The official instructions tell you to use a white paintbrush but it just does nothing! Black, and any level of grayscale right down to 1%, mask the image out to various levels but there is no way to remove parts of the mask or reveal the image because totally white doesn’t do anything at all and the eraser acts like a 100% black paintbrush! (On iPad by the way!)

[Ron P.]

4 hours ago, PhilOsborne said:

(On iPad by the way!)

Try asking in the iPad forum...

https://forum.affinity.serif.com/index.php?/forum/43-affinity-on-ipad-questions/

[jdvoracek]

5 hours ago, PhilOsborne said:

I just can’t get masks to work as you would expect. While the mask layer is selected, eraser acts like painting black, it masks the image. Painting black does the same thing. But painting white does nothing at all … surely that should reveal the image again? The official instructions tell you to use a white paintbrush but it just does nothing! Black, and any level of grayscale right down to 1%, mask the image out to various levels but there is no way to remove parts of the mask or reveal the image because totally white doesn’t do anything at all and the eraser acts like a 100% black paintbrush! (On iPad by the way!)

@PhilOsborneyour observations are correct. Here are some things that might help. I think this is largely the same logic on an iPad.

1) Masks can be external or internal (there may be better diction, but I don't know it). Adjustment and live filter layers come with internal masks. When you paint on such a layer, AP assumes you want to paint on the associated internal mask. This internal mask passes or blocks or partially passes/blocks the adjustment/filter effect.

2) Layers that are marked by AP as masks work by the grayscale value of each pixel. Black blocks, white passes and gray is some degree of translucent. Another property that is not much advertised is that the alpha channel of a mask also has an effect: alpha of 0% blocks and alpha 100% passes. This is confusing, not too useful, but does explain why erasing on a mask causes it to block what is below (or to what it is nested) or what adjustment or filter effect you are applying.

3) You can invert an external mask with Layer>Invert. If you select an adjustment or live filter layer, this command inverts the internal mask. You can see the state of the internal mask by holding opt/alt and clicking on either the adjustment/live filter layer (case of internal mask) or on the ICON area of an external mask. A grayscale render of the mask will be shown. You can paint on this render with black/white or gray to change the mask. To get out of this state, click on any other layer. In the case of an external mask nested to another layer, be sure only the external mask is selected or you will invert the parent layer instead.

4) Unless you specify otherwise, masks are created all white so painting white does indeed do nothing. Painting white will reverse the effects of painting black. Don't alter a mask by erasing unless you really know what you are doing because you can't reverse that except (maybe) with an undue. You can create an all black external mask instead of the default white if you hold alt/opt when creating it.

5) In summary, masks behave like varying degrees of opacity but masks are altered by grayscale painting, and masks are rendered for you to see (when you request) as a grayscale image.

There's lots and lots more to masks (and channels), but start with the above.

[Arthur King]

I just cannot edit a layer mask other than by hand-painting on it. I really would like to use curves/levels to accomplish this editing; the mask is almost perfect as it is; I just want to correct some overall sharpness-issues. In PH this is easy. For clarity: I don't want an extra mask-layer, I want to edit an existing one. I get the impression from this discussion that this is possible, just a little awkward. I can accept this, I appreciate AF, but I cannot find my way. Can anyone guide me through the advice given here?

Here are some details:

There are supposed to be two different menus concerning mask editing. Dan C writes: "Which one you get depends on whether you drop a mask between existing layers or onto a layer icon itself."

Dan C also writes: "These options <...which I believe are the ones relevant for me...> will only show for layers that have been clipped and masked to a layer, currently your Mask is only clipped to the layer, which means the behavior is slightly different and the context menu reflects this."

I have tried both ways of dropping masks: Only the same menu ...the irrelevant one... appears. This is the same menu that duncan has posted a screenshot of.

Later in the tread kirkt suggests a rather cumbersome way out which includes the following step:  "In the Channels palette, right click on the Red, Green, or Blue channel of the Background layer (whichever gives you the proper contrast and luminosity you want) and select "Create mask layer."

Even this menu I cannot make appear. All the comotion tells me that editing layer masks in AF is not straight forward, but do I also have a bug to fight against?  Or is it me that just cannot cope? I am new to AF, version 1.10.4 for mac. AF was so beautiful before I hit this wall. Please make me see the light again!

[Old Bruce]

1 hour ago, Arthur King said:

I don't want an extra mask-layer, I want to edit an existing one. I get the impression from this discussion that this is possible, just a little awkward.

You have the wrong impression then.

As far as I know there is no way can you apply a Curves adjustment to a mask. Or any adjustment layer.

You have to make a greyscale layer of the mask, apply your adjustment layer(s) to that. Flatten/rasterize the layers into one greyscale layer. Then load any one of the greyscale layer's channels that into the original mask alpha. Easier to do than explain.

[NotMyFault]

40 minutes ago, Old Bruce said:

As far as I know there is no way can you apply a Curves adjustment to a mask. Or any adjustment layer.

You can apply any adjustment layer to a mask layer. Unfortunately the UI is quite ..hm.. tricky on how to achieve this.

There was even an official legacy tutorial explaining the process:

https://player.vimeo.com/video/242758748

https://player.vimeo.com/video/143990072

Bug reports concerning nested alpha sensitive layers are broken

 

[Old Bruce]

Hey, @Arthur King,

Check out the tutorials @NotMyFault posted.

So cool, thanks for showing me I was completely wrong.

So any Adjustment with an Output channel choice. Meaning Levels, Curves and Channel Mixer. Set it to Alpha and Nest the Adjustment layer with the Mask layer. Super cool. Now if only I needed to use this....

[jdvoracek]

3 hours ago, Old Bruce said:

You have the wrong impression then.

As far as I know there is no way can you apply a Curves adjustment to a mask. Or any adjustment layer.

You have to make a greyscale layer of the mask, apply your adjustment layer(s) to that. Flatten/rasterize the layers into one greyscale layer. Then load any one of the greyscale layer's channels that into the original mask alpha. Easier to do than explain.

I rarely offer an opinion contrary to @Old Bruce, but after digesting the post of @kirkt earlier in the thread, under certain circumstances it seems you can apply some adjustments to mask layers. I don't have a compact, well-defined explanation of what AP is doing, but I ramble on.

I usually think of a mask layer as a grayscale image where AP uses the gray tone of each MASK pixel to determine the ALPHA channel values of the render of the layer(s) to which the mask is applied (clipped). White mask pixels pass the underlying layer(s) by multiplying their pixel alpha(s) by 1. Black mask pixels block the underlying layer(s) by multiplying their pixel alpha(s) by 0. Gray mask pixels partially pass the underlying layer(s) by multiplying their alpha(s) by some gray tone (luminosity, intensity? - formula?) value between 0 and 1.

Normal masks (Layer > New Mask Layer) seem to be white pixel layers with alpha =1. When you invert, you get a black pixel layer also with alpha = 1. As you paint on this mask, you change the gray tone value, but the alpha remains 1. However, you can ALSO affect the alpha of each mask pixel, and that has an ADDITIONAL (multiplicative?) effect on how the mask affects the render. Adjustments which can affect the alpha channel can thus be applied to mask layers, and they will have an effect on how the mask affect the render. It seems that the mask pixel alpha is combined (multiplies?) the gray tone value of the mask to determine the final mask effect on the render.

Here is an example. Open a pixel image. Apply a (white) mask. Paint with 50% gray on that mask. Add a levels adjustment - it won't clip directly to the mask and will probably open at the top of the layer stack. Just move it to the mask icon to clip it to the mask. BE SURE TO ADJUST THE ALPHA CHANNEL. Move the gamma slider to each end and observer the effect on the mask (via effect on the render).

P.S. @NotMyFault I saw your reply after I composed mine. Those are very helpful tutorials. When I searched in the past, I couldn't find anything so I guess my terms were incorrect. I sure would like to know how this works computationally, but I guess that is Affinity proprietary.

 

Edited January 23, 2022 by jdvoracek
added P.S.

[NotMyFault]

19 minutes ago, jdvoracek said:

P.S. @NotMyFault I saw your reply after I composed mine. Those are very helpful tutorials. When I searched in the past, I couldn't find anything so I guess my terms were incorrect. I sure would like to know how this works computationally, but I guess that is Affinity proprietary.

Not at all. It is all common knowledge on wikipedia, and 30+ years old standards established by photoshop, w3c, svg and others.

The main issue with Affinity apps is that since version 1.9 and introducing OpenCL in ~2021 almost all alpha related functions with nested or grouped layers are broken and still unfixed in 1.10.4 and current betas.

I think forum users can answer quite a few alpha related question including references to open standards or Adobe reference documents. The only problem is that Affinity apps mostly adhere to these standards unless their don’t- and hide behind questionable  „design decisions“. 

[telemax]

On 1/23/2022 at 6:02 PM, Arthur King said:

I just cannot edit a layer mask other than by hand-painting on it. I really would like to use curves/levels to accomplish this editing; the mask is almost perfect as it is; I just want to correct some overall sharpness-issues. In PH this is easy. For clarity: I don't want an extra mask-layer, I want to edit an existing one. I get the impression from this discussion that this is possible, just a little awkward. I can accept this, I appreciate AF, but I cannot find my way. Can anyone guide me through the advice given here?

You can use a normal layer as a mask, and do with it everything you can do with a normal layer.

1. Select the layer that should be the mask
2. Change its curve in Blend Ranges
3. Change its blend mode to Erase 

Non-destructive_Mask.afphoto

[Arthur King]

Thanks for advice, this inspires a lot. Still, for me there remains a task that I cannot figure out a smart workflow for. Old Bruce comments: “Super cool. Now if only I needed to use this…” I agree, need and love are not always very synchronized. I ask for patience when describing what I would love/need to achieve. It’s about creating patterns & textures. Here is a picture to take a load of my words:

 

For me this is a nice way to seperate the colouring from the drawing: I can adjust the two aspects one at a time. Often I will want to edit and re-edit the alpha masks until the overall composition fits in with neighboring textures—they are for use in a game. The editing of a mask will include filtering the channel, scaling it, contrast-adjusting it, roughing or smoothing it any way I can think of. In musical terms: I think of these adjustments as getting the “timbre” of the texture right. I have just learned, and this will surely come in handy: In AF(finity Photo) it is easy to set up a sequence of modifying child-layers. But sadly this is not how I have made a habit of working with textures. Making adjustments to a grayscale image and then applying it as a mask is a possible way out, but is it speedy enough when struggling with creative impatience? I have my doubts. Any ideas for a workaround will be appreciated!

(Is there some kind of “spooky masking at a distance” that may be used? I’m sorry, I have seen so many instructional videos for different tools that I get a bit disoriented.)

[jdvoracek]

5 hours ago, Arthur King said:

Often I will want to edit and re-edit the alpha masks until the overall composition fits in with neighboring textures—they are for use in a game. The editing of a mask will include filtering the channel, scaling it, contrast-adjusting it, roughing or smoothing it any way I can think of. In musical terms: I think of these adjustments as getting the “timbre” of the texture right. I have just learned, and this will surely come in handy: In AF(finity Photo) it is easy to set up a sequence of modifying child-layers. But sadly this is not how I have made a habit of working with textures.

Hi, not much help, but how are you working now? Photoshop? Sadly, my experience is Affinity Photo does not offer much straight-forward, intuitive alpha channel or mask manipulation beyond painting grayscale onto a mask layer. As in prior posts, you can do some operations albeit in rather obtuse and probably inefficient workflows. GL!