Affinity Photo white edges when applying color mask

[epenny]

Hi!

I'm trying to change the color of a white object that is masked - a person in a white shirt. I remove the background with a mask. Great. Then I mask the white shirt and change the colour. By default, if I use the same selection for the mask as I used for the original removal of background, I get a white edge showing up. (See image 1 - it's noticable in an image 3000 x 2000 px).

 

 

If I try to add the colour to the white (grey) edge, I end up with a 'black' edge (See image 2 with the mask 'cut out' to show the white underneath, the 'grey' edge that causes the issue, and the 'fix' of more masked color that makes the edge 'black'. 

 

I've tried removing pixels on the other side of the selection, and Any ideas on how to fix this?

[RichardMH]

Are you using Refine to play with the mask edges?

[Dan C]

Hi @epenny,

Welcome to the Affinity Forums

Are you able to attach a copy of this document here for me so I can investigate further?

Many thanks in advance!

[epenny]

@RichardMH Thanks for replying. Yes, I tried that. Improved, but couldn't get rid of it completely. 

@Dan C That would be awesome, thank you. Hopefully it's attached.

affinity_test.afphoto

Edited May 16, 2022 by epenny

[firstdefence]

Did you use the Pen tool to make a selection or one of the selection tools?

[epenny]

@firstdefence Thanks for your reply. I've tried several different ways with the selection brush etc, but not the pen tool. (I used my own cludge technique, then tried following https://www.youtube.com/watch?v=xhpTwl4Tifk then https://www.youtube.com/watch?v=bmNiU_gqiWo then https://www.youtube.com/watch?v=1mDUoYxzijg  and even https://www.youtube.com/watch?v=PDXbdT0ZS8w (which doesn't make the shirt dark enough) but I can't get rid of it.) But if there's a way with the pen tool that might be better, I'm all 👀

 

 

[Dan C]

Thanks for your file provided @epenny!

I can confirm you've unfortunately incidentally encountered a few bugs within the file, firstly there is a known issue causing mask blending to not be correctly applied when the Mask is an 'orphan' layer within a Group (ie not directly applied to a layer), which causes the mask to ignore semi transparency and therefore the mask is slightly smaller here, leaving the grey line around the shirt of your subject.

I was going to suggest duplicating the mask twice, so you had 3 separate mask layers, and applied each of these to your adjustment layers - however I can also see your Curves Adjustment layer has the Underlying Composition Range edited in the Blend Ranges. For the same reason as above, when masking this adjustment, the semi-transparent blend is not correctly applied, meaning the colour of the shirt changes as though you had not set this Underlying Composition Range for the Curves layer.

Both of these issues are already logged with our development team, though for slightly more complex documents - so I'm going to update these logs with your file now, to better demonstrate the bug & this will hopefully help our team to fix this issue asap.

Therefore I've restructured your document to better work with the layer setup you have, please find this file attached.
I duplicated the subject image and nested the mask & adjustment layers directly to this - to workaround both the mask group & layer blending bugs.

I hope this helps

affinity_test_f.afphoto

[JimmyJack]

If I may, I made one more change to @Dan C’s file….

A) I agree with the reordering. Without it the issues are harder to manage.

B) I added an additional element to each mask. A levels adjustment set to Alpha. This helps, to a limited degree, to push/pull/adjust the mask edge fidelity.
In combination they address both the overall shirt edge and the shirt to pant edge. One alone didn’t do both (at least in my quick attempt, ymmv).
They do, of course also change the edges around the head…. so I just painted those back in directly on the adjustments themselves.

 

 

affinity_test_f_jj.afphoto

[epenny]

Hi @Dan C and @JimmyJack,

Thank you both so much for looking at this. Those are the tips I needed. I really appreciate your time. Thank you.