Subject halos when blurring backgrounds

[Derek R]

Hi,

Newbie to pixel editing here. I've been trying out blurring a slightly distracting background using the following steps.

1) Select subject using brush

2) Refining selection

3) Inverting Selection to select the background

4) Applying Gaussian blur

Using this process anything beyond a <2 pixel blur produces a distinct halo-blur of the subject edge beyond the edge of the actual subject.

I've tried checking and unchecking preserve alpha with no obvious effect and trying different types of blur.

Does anyone have any advice?

Thanks,

Derek

[carl123]

1) Select subject using brush

2) Refine selection

3) Copy subject to new layer

4) Select original layer

5) Apply Gaussian blur

 

Note in some case it can help to Inpaint out the edges of the original subject after step 4 and before doing step 5

[Derek R]

Hi,

Sorry for being so late to get back to you on this. Do you mean to inpaint inside what would be the subject selection or outside? Also what attributes (approximately) would you give the inpainting brush?

Thanks for your help it is much appreciated.

[carl123]

After you copy the subject to a new layer you can temporarily switch of that new layer so you are working on the background layer

On the background layer clear any active selection you may have then use the Inpainting tool around the edges of the selection you just cut out, this should effectively bleed the background into the selection area a little bit which will help when you finally blur the background layer as it prevents the original subject from giving a halo type effect around your unblurred (sic) subject matter.

Not always necessary but useful if you see the effect I am trying to describe

As regards the attributes for the Inpainting brush it's hard to say without seeing the image and what you are trying to do but start with everything set to 100% and an appropriate brush size

 

[Alfred]

31 minutes ago, carl123 said:

As regards the attributes for the Inpainting brush it's hard to say without seeing the image and what you are trying to do but start with everything set to 100% and an appropriate brush size

 

Is there a useful rule for determining the appropriate brush size for a given selection size?

 

[Wosven]

44 minutes ago, Alfred said:

Is there a useful rule for determining the appropriate brush size for a given selection size?

The same as in Art : use tools depending of/appropriate for your canvas (i.e. no pencil for a 4 × 3 m… unless you've got a lot of time to draw )

[John Rostron]

I would suggest using a brush size no more than 5% of the size of your cut-out. So if the hole is about 200 by 300px, then make your brush about 10-12px. No need to be too precise, but it does need to be bigger than your blur.

John

[Derek R]

Thanks for the info on brushes, and everyone's input in this.

I've been experimenting and I've found the following to work best at suppressing the halo effect I was seeing. As I stated in my OP, I'm a newbie so there may be issues with this methodology but for what it's worth I did the following.

1) Make selection of subject

2) Refine selection

3) Create a layer copy of the selection

4) Invert original selection to select background

5) Create a layer copy of the background selection

6) Place background selection layer below the subject selection layer

7) Apply Gaussian blur to the background selection layer

Following Carl123's steps didn't seem to have an effect on the halos on my test image, but this might have been due to me not communicating what halos I meant effectively.

[IanSG]

I found this problem when I first started using AP.  My solution was to cut out the subject, clone a few pixels from the outside edge into the inside edge of the cutout area, apply the blur and then paste the subject back in.  The fundamental problem is that AP selects pixels outside the selected area to include in the blur - replacing them with pixels from the background means there's no halo.

[R C-R]

15 hours ago, IanSG said:

 The fundamental problem is that AP selects pixels outside the selected area to include in the blur - replacing them with pixels from the background means there's no halo.

In my tests, the blur will not include 100% transparent areas, so what works for me is this:

 

1. Duplicate the photo pixel layer.

2. Select the subject area to remain un-blurred & refine as needed in the usual way.

3. On the top copy, invert the selection & erase the background. Alternately, in the Refine Selection dialog, set the output to Mask instead of Selection.

4. On the bottom copy, use the non-inverted selection to erase the subject area. If step 3 outputted a mask, CMD-clicking (Mac) on the mask layer will bring back the selection, or use the Reselect command, or whatever.

5. On the bottom copy, apply a Live Gaussian (or whatever) blur, making sure Preserve Alpha is ticked.

 

If a mask layer was created, the file size can be reduced by rasterizing the top layer as a final step but I like to leave it as a masked layer so there is an unaltered version of the photo in the file in case I want to do something else with it later.

 

The attached Flower with bg blur.afphoto (5.8 MB) is an example of how this works. The Live Gaussian Blur can be set to any radius without creating a halo.

[IanSG]

46 minutes ago, R C-R said:

In my tests, the blur will not include 100% transparent areas,

That's the exact opposite of what I get!  How are you testing this?

[Fixx]

If there is a hole in blur target layer, Preserve Alpha keeps it preserved, transparency does not leak to opaque area.

If you have the original subject area instead of hole, original colour will leak to selected area when applying blur.

[R C-R]

29 minutes ago, IanSG said:

That's the exact opposite of what I get!  How are you testing this?

Basically, by doing what I described & demonstrated in the afphoto file I attached to my post. The key is to use a live filter that gives you the option to preserve alpha. If you download & open that file, you can hide the top "Flower photo" layer so only the "no flower background" layer shows. Double-click on the live Gaussian blur layer so you can see the difference ticking & not ticking "preserve alpha" makes.

 

EDIT: There is a tiny amount of an edge effect in that file, but that is because it is a 1/4 sized resampled version of the original. I did that to keep the download size reasonable.

[IanSG]

2 hours ago, Fixx said:

If there is a hole in blur target layer, Preserve Alpha keeps it preserved, transparency does not leak to opaque area.

If you have the original subject area instead of hole, original colour will leak to selected area when applying blur.

Damn - I'd completely forgotten about that!

 

2 hours ago, R C-R said:

The key is to use a live filter that gives you the option to preserve alpha.

Thanks for that!  It's amazing how quickly I forget these things!