Black outline after background removal refine tool

[BHP]

Hello everyone,

Not sure if this is a bug or if I'm doing something wrong: AP-AD 1.7.1

When I remove a subject from a background I get a weird black outline after refine tool to mask. If I try to remove some of the black outline it just erases and keeps black outline, almost like a drop shadow. In 1.6 when I exported to TIFF I would see this but not PNG. Now in 1.7 I see it in the program after I refine and turn to mask.

For example, If I remove background of someone with blonde hair i will see a black (almost like drop shadow look) around my selection. light hair with dark outline.

I can try to recreate this and post an image if no one else is seeing this.

Thank you!

Brett

 

[Murfee]

Hi @BHP 

Can you post a screenshot that shows the refine selection toolbar as well as one that is showing what your result with the mask is.

I have just done a very quick test on an image from Unsplash, I thought there were problems with the Affinity selection, but on closer inspection of the original image...it had previously been cut out and placed on a new background...there were plenty of artefacts  The Affinity selection output to a mask was fine.

[BHP]

5 hours ago, Murfee said:

Hi @BHP 

Can you post a screenshot that shows the refine selection toolbar as well as one that is showing what your result with the mask is. 

I have just done a very quick test on an image from Unsplash, I thought there were problems with the Affinity selection, but on closer inspection of the original image...it had previously been cut out and placed on a new background...there were plenty of artefacts  The Affinity selection output to a mask was fine.

HI Murfee,

I tried to reproduce it and of course it worked fine. I do a lot of masking so the next time I notice it I will post back to this forum.

Brett

[elk]

It is maybe the problem that I saw in early versions of AP as well in the newest. 

When making a selection refine and choose layer w/ mask it cuts out the slection with a really ugly pixeled outline which is masked. But if you want to alter the mask you have a big problem adding something again. See screenshot where the mask is off.

Carsten

[elk]

Just fiddling around a bit. The result depends on the amount of Smooth. But anyway, the result isn't what it should be in my opinion.

See three screenshots:

1 Original

2 New layer with mask

3 same layer as 2 with mask off

[BHP]

On 7/1/2019 at 7:42 AM, elk said:

Just fiddling around a bit. The result depends on the amount of Smooth. But anyway, the result isn't what it should be in my opinion.

See three screenshots:

1 Original

2 New layer with mask

3 same layer as 2 with mask off

I get that too, that is another issue for me. This problem I can't tell if it's a jpeg problem or a affinity problem. I just make sure my subject  is 100% selected  and let the background bleed and clean that up otherwise it looks terrible.

[BHP]

OK I think I have figured out what is going on with my transparencies. So if I remove the background from the subject it looks fine, proper, and a clean edge. As soon as I place a background behind it I get a weird blackish outline. Then I try to keep removing the edge via mask to clean up the black but the black just stays. I am uploading images so you can see hopefully what I'm talking about.

Really focus on the line in the arm area, with and without a background to see the blackish outline. I'm really not sure why it does this and that is why I am wondering if it's me or the program?

(I am using the latest 1.7 version) I first noticed this in version 1.6. I was designing a book cover. I removed a person from the background in Affinity Photo 1.6, it looked great. Then I placed the transparency into Design 1.6. When I was getting close to finished I was just fine tuning the edge in Designer and noticed the black outline thinking I did a shatty job in Photo removing the background. On a side note, the tiff placed in designer created the blackish outline, then I tried a PNG transparency and that seemed to work.

Thank you,

Brett

[Murfee]

39 minutes ago, BHP said:

Really focus on the line in the arm area, with and without a background to see the blackish outline. I'm really not sure why it does this and that is why I am wondering if it's me or the program?

Hi BHP, that is really strange, not seen that before. How did you select the area in the arm? If I zoom in really close there are areas that have not been erased, they are really faint. If you used a brush to remove from the mask, check the brush panel and select a basic plain round brush. It’s possible that a textured or effects brush has been used. The paintbrush remembers the last brush used, that might explain why it doesn’t always happen when you are masking. 

The area on the arm is not what you would expect to see. The edges are partially transparent but that darkening around the edge is not normal. Are you using any blend modes on any of the layers? It might be possible if you are that the effect is only coming through the partially transparent areas of the mask. Try using a hard edge brush to see if the effect still happens.

[BHP]

58 minutes ago, Murfee said:

Hi BHP, that is really strange, not seen that before. How did you select the area in the arm? If I zoom in really close there are areas that have not been erased, they are really faint. If you used a brush to remove from the mask, check the brush panel and select a basic plain round brush. It’s possible that a textured or effects brush has been used. The paintbrush remembers the last brush used, that might explain why it doesn’t always happen when you are masking. 

The area on the arm is not what you would expect to see. The edges are partially transparent but that darkening around the edge is not normal. Are you using any blend modes on any of the layers? It might be possible if you are that the effect is only coming through the partially transparent areas of the mask. Try using a hard edge brush to see if the effect still happens.

Hey Murfee, sorry hard to type all the steps. I missed some important ones.

In the above I used the selection brush tool, (photo is AdobeRGB from NEF raw file). I select the subject, then fine tune it, then save the fine tune with mask so I can clean any bad edges. That is what you are seeing above. I am using the basic brushes, I usually pick the middle soft brush to give the edges a softer more realisitc cutout. But in these new ones I use crisp, middle and super soft brush to mask out samples.

Now in these new photos, I opened the same photo, masked the background out by hand with NO refine tool. Then I changed the color and guess what. It appears that certain colors cause the black outline. Samples below. This must be why sometimes I see the black outline and other times I do not.

Weird!!!! Something to add to the Affinty X-FIles LOL

Thank you!

Brett

 

[Murfee]

5 hours ago, BHP said:

Now in these new photos, I opened the same photo, masked the background out by hand with NO refine tool. Then I changed the color and guess what. It appears that certain colors cause the black outline. Samples below. This must be why sometimes I see the black outline and other times I do not.

 

Hi Brett, you have indeed found a bug, I just ran some quick tests. NEF file developed in 32bit HDR ProPhoto colour space. It seems to be some strange behaviour with partially transparent pixels. I used the selection brush, no refinement just added a mask while selected. rectangle placed below, painted on the mask with a hard edged brush, couldn't see  any dark edges, painted on the mask with a softer brush, the darkening could be seen around the edges. Repeated the test by using refine selection, output to mask, dark edges were visible when I painted on the mask with a soft brush. Repeated it again, no selections just a mask painted with a soft edge brush the darkening was visible.

I have not seen this before, probably because I don't usually place my cutouts on a brightly coloured background. I usually check I have got a good selection by placing on either black or white. I did notice that the effect is more obvious with a pure hue, as you move around changing the background colour and get towards the lighter tints and darker tones the effect is not as visible.

I am running a Mac with Metal off, so I don't think it is connected to Metal compute.

Edited: I have just filed a bug report (they will get to see it earlier) and linked to this thread to give the Devs a bit more info. Well spotted 

Edited July 8, 2019 by Murfee
Added comment

[MEB]

Hi BHP, Murfee,
Thanks for reporting this. Issue logged/passed to the dev team for inspection/comments.

Hi elk,
We are aware of this issue. Don't use the New layer with mask output from the Refile dialog. Duplicate the original image first, refine it as usual but select Mask as the output in the Refine dialog instead. You will have the mask as intended and the image will not be "corrupted" since you duplicated it manually rather than allowed the Refine dialog generate it.

[elk]

44 minutes ago, MEB said:

Hi BHP, Murfee,
Thanks for reporting this. Issue logged/passed to the dev team for inspection/comments.

Hi elk,
We are aware of this issue. Don't use the New layer with mask output from the Refile dialog. Duplicate the original image first, refine it as usual but select Mask as the output in the Refine dialog instead. You will have the mask as intended and the image will not be "corrupted" since you duplicated it manually rather than allowed the Refine dialog generate it.

Hi MEB,

okay, thanks, layer with mask should finally work like the workaround I do now. That is -- as you say --- making a duplicate of the original layer, creating a refined selection,  and then create a mask from the section and assigning the mask to the layer duplicate. (Btw, using layer with mask comes from working with PSH where you get the desired result. But I am now mostly on Affinity and like it very much.)

[BHP]

5 hours ago, MEB said:

Hi BHP, Murfee,
Thanks for reporting this. Issue logged/passed to the dev team for inspection/comments.

Hi elk,
We are aware of this issue. Don't use the New layer with mask output from the Refile dialog. Duplicate the original image first, refine it as usual but select Mask as the output in the Refine dialog instead. You will have the mask as intended and the image will not be "corrupted" since you duplicated it manually rather than allowed the Refine dialog generate it.

Thanks Murfee for all your help! Added bonus fix from elk, which was happening to me as well.

Thanks to the Affinity Team! Keep up the great work!

Brett