Jump to content
You must now use your email address to sign in [click for more info] ×

[APh] Impainting tool improvement suggestions.


Recommended Posts

Impainting tool improvement suggestions.

 

Hi all

 

I am new to AP. The #1 reason I bought it is for what I call garbage collection (lines, posts, signs, TV aerials, etc.): the impainting tool. OK, I am interested in others tools as well. ;-)

I could do quick and excellent jobs but I also have been struggling in some situations. I was impressed by its performance yet I think it can be significantly improved.

Here are some of my suggestions. I have been thinking about them long before getting AP.

 

1. Create specific tools for specific objects to be removed, for example wires.

I had to remove tons of them with backgrounds of either foliage or buildings. If the line is just above or just under the top of the foliage or at the edge the result is often a blurred mixture of green and blue. It works well only if the background is plain blue sky or plain foliage.

If the background is a building and the line passes across or along such features as windows, cornices, brick chimneys and other reliefs, the tool often doesn't reproduce the right pattern. Often you then have to work by little chunks and often undo and redo until you get the right result.

 

1.1. Wires

A wire practically always follows a very predictable curb, has a regular width (or thickness) and a more or less consistent color (usually brown to black) or several colors when it shines under the sun light (black and whitish). It usually contrasts well with the background but can mingle with other patterns.

The procedure would consist in:

1.1.a. Clicking on both ends of the wire.

1.1.b. Clicking on several spots in between so the program can calculate the curb and evaluate the width.

1.1.c. The program would enhance the wire and allow the user to adjust the width.

 

1.2. Posts

I took pictures of a street art mural with posts in the foreground. Similarly they were straight, homogeneously colored (dark brown). The tool had the tendency to copy-paste the surrounding patterns, like it would do if it was grass for example. It gave a strange mirror effect with some duplicated features.

A specific “post” tool would be able to better recognize what's to be removed and how to stitch the background pattern.

 

1.3. Aerials

Aerials come in many different sizes and layout but have a common pattern. Their colors are also rather typical.

The procedure would consist in creating a polygon around the aerial.

 

1.4. Lattice: electricity pylons, cranes, bridges, scaffoldings, etc.

Like the aerials, they come in different shapes, patterns, colors and sizes and, depending on their background, can be very tricky to dispose of. What's special about them is that they can be very large and yet have a low opacity-to-transparency ratio: the actual total surface of the beams is small compared to the see-through "holes". The landscape that's behind is perfectly visible. A “lattice” tool would probably do the job more easily:

1.4.a. Select the Lattice tool.

1.4.b Click on a number of the beams and/or draw around it.

1.4.c Adjust if necessary.

 

1.5. Branches

I often choose to take certain shots (of buildings for example) in winter when there are no leaves blocking the view. Yet barren branches can be rather obstructing anyway. Therefore a branch tool similar to the lattice tool would be nice.

 

1.6. Signs: One way street, No parking, etc.

Specialized tools would be more efficient in certain cases.

 

2. Stitching

I shot another mural that had two road signs in the foreground. The impainting tool did a “credible” job for the eyes of someone who can't see the original. I would say: “Well tried”. Let's say it's creative. Of course AP couldn't guess the actual features that were behind. It will do, it's minor features and people are not going to check.

Whereas this shot was taken from right in front of the wall and therefore has only a slight vertical perspective deformation I also took another one from an offset position to capture the hidden parts and this shot has both horizontal and vertical deformations.

What I have in mind is a tool that would allow to choose a region from one shot and to paste it in another shot while trying to make the perspectives match.

In Hugin there is a Mask tool that allows to chose a region in the overlapping zones to be either included in or excluded from the final image. It's handy when a person, an animal or a car moved between two shots.

I tried to do the job with Hugin and it was a disaster. It's obviously not its domain of competence.

Maybe this tool would be rather tricky to develop but I know plenty of shots that I could save with such a feature.

 

Just my 2cts.

 

Nick

Link to comment
Share on other sites

Impainting tool improvement suggestions.

 

Hi all

 

I am new to AP. The #1 reason I bought it is for what I call garbage collection (lines, posts, signs, TV aerials, etc.): the impainting tool. OK, I am interested in others tools as well. ;-)

I could do quick and excellent jobs but I also have been struggling in some situations. I was impressed by its performance yet I think it can be significantly improved.

Here are some of my suggestions. I have been thinking about them long before getting AP.

 

1. Create specific tools for specific objects to be removed, for example wires.

I had to remove tons of them with backgrounds of either foliage or buildings. If the line is just above or just under the top of the foliage or at the edge the result is often a blurred mixture of green and blue. It works well only if the background is plain blue sky or plain foliage.

If the background is a building and the line passes across or along such features as windows, cornices, brick chimneys and other reliefs, the tool often doesn't reproduce the right pattern. Often you then have to work by little chunks and often undo and redo until you get the right result.

 

1.1. Wires

A wire practically always follows a very predictable curb, has a regular width (or thickness) and a more or less consistent color (usually brown to black) or several colors when it shines under the sun light (black and whitish). It usually contrasts well with the background but can mingle with other patterns.

The procedure would consist in:

1.1.a. Clicking on both ends of the wire.

1.1.b. Clicking on several spots in between so the program can calculate the curb and evaluate the width.

1.1.c. The program would enhance the wire and allow the user to adjust the width.

 

1.2. Posts

I took pictures of a street art mural with posts in the foreground. Similarly they were straight, homogeneously colored (dark brown). The tool had the tendency to copy-paste the surrounding patterns, like it would do if it was grass for example. It gave a strange mirror effect with some duplicated features.

A specific “post” tool would be able to better recognize what's to be removed and how to stitch the background pattern.

 

1.3. Aerials

Aerials come in many different sizes and layout but have a common pattern. Their colors are also rather typical.

The procedure would consist in creating a polygon around the aerial.

 

1.4. Lattice electricity pylon

Like the aerials, they come in different shapes, patterns and sizes and, depending on their background, can be very tricky to dispose of. A “lattice” tool would probably do it more easily. In particular, instead of impainting the whole region it would be able to take into account and keep what's seen through the lattice.

 

1.5. Branches

I often choose to take certain shots (of buildings for example) in winter when there are no leaves blocking the view. Yet barren branches can be rather obstructing anyway. Therefore a branch tool similar to the lattice tool would be nice.

 

1.6. Signs: One way street, No parking, etc.

Same as for the others.

 

2. Stitching

I shot another mural that had two road signs in the foreground. The impainting tool did a “credible” job for the eyes of someone who can't see the original. I would say: “Well tried”. Let's say it's creative. Of course AP couldn't guess the actual features that were behind. It will do, it's minor features and people are not going to check.

Whereas this shot was taken from right in front of the wall and therefore has only a vertical perspective deformation I also took another one from an offset position to capture the hidden parts and this shot has both horizontal and vertical deformations.

What I have in mind is a tool that would allow to choose a region from one shot and to paste it in another shot while trying to make the perspectives match.

In Hugin there is a Mask tool that allows to chose a region in the overlapping zones to be either included in or excluded from the final image. It's handy when a person, an animal or a car moved between two shots.

I tried to do the job with Hugin and it was a disaster. It's obviously not its domain of competence.

Maybe this tool would be rather tricky to develop but I know plenty of shots that I could save with such a feature.

 

Just my 2cts.

 

Nick

Its called the clone tool ..

Link to comment
Share on other sites

Its called the clone tool ..

Right. I use that one too. Yet, in many circumstances you have to work chunk by chunk and it's time consuming. In other cases it's just useless.

In the attached image each line passes accross many different types of background that need a different approach, either the impainting tool or the clone tool.

My suggestions would allow AP to exactly know what's to be removed and then make a more educated guess about the surrounding patterns to apply in the void. For example when it comes across a window it would know in advance it's the black line and not the white wooden part that's to erase.

Let's see what the Affinity guys think about it.

Nick

EDIT: it won't take my image: "Error 500". :-(

Link to comment
Share on other sites

1.1. Wires

 

- Use the pen tool to trace down the wire

- adjust the line thickness to the wire thickness, you can even choose a pressure curve to follow the wire closely

 

cmd + klick on the pen tool path in the layer panel, you can then uncheck the path to make it invisible and you´ll see that a selection has been created from the path

(do not use the selection button in the top toolbar cause this will give you a different result)

 

- Edit > Fill 

​> choose "Inpainting"

 

 

actually I just gave i another try and didn´t get the fill > inpaint option to work properly and it´s also not possible to do the inpainting non destructively on a new layer that way but you can however paint with the standard inpainting brush in that selection and that gives you full control.

 

 

Link to comment
Share on other sites

 

I couldn't get any result. The Fill > Impainting does nothing. I must have missed something or you omitted some steps too obvious to you to mention. I am not yet familiar with such tools so I need to train on it.

Anyway this procedure consists in applying another object over the target.

The purpose of my suggestion is to make the procedure much quicker by first choosing in a list the type of object we want to impaint (wires, posts, etc.) so that AP knows what it has to look for, then clicking on it. What about doing the job in half a dozen clicks?

 

Nick

Link to comment
Share on other sites

I really think implementing so many variations for the inpainting tool would be very complex. Unless I am completing misunderstanding it is the area of the photo surrounding the object to be removed that is important, not the object itself. Removing a post in a thicket of leaves is entirely different than removing one against a grassy field. There are other tools designed to be used in conjunction with inpainting to clean up difficult, complex areas. And sometimes you just have to use a paint brush and do it manually.

Link to comment
Share on other sites

+1 for nickdaum's suggestion, particularly for a tool that uses secondary shots, taken from different points of view, to build (and not to fake) what's behind an object that is to be "inpainted" in the shot taken from the main perspective.

take care,

stefano

Link to comment
Share on other sites

I really think implementing so many variations for the inpainting tool would be very complex. Unless I am completing misunderstanding it is the area of the photo surrounding the object to be removed that is important, not the object itself. Removing a post in a thicket of leaves is entirely different than removing one against a grassy field. There are other tools designed to be used in conjunction with inpainting to clean up difficult, complex areas. And sometimes you just have to use a paint brush and do it manually.

 

 

You didn't get my point. In order for the program to restore the part that's masked by the line, the post, the sign or whatever, instead of trying to make an educated guess about what to remove and what to imitate, the program would be able to identify exactly which thing to remove according to specific pattern.

I had once to struggle removing a line that was passing in front of a couple of trees that were in front of a building with parts across the sky. There is no way you can pass the brush all along. For each stretch you need to apply a different strategy with different brush sizes. That's because the app has to guess for each and every pixel what to do. If it knew in advance:

  • where to start and where to end.
  • the curve of the line.
  • its average width.
  • its color(s).

it wouldn't need to make an educated guess but to follow a precise pattern.

I have no doubt the algorithms currently used are pretty complicated.

 

Once I wanted to remove a one-way sign and got at first a kind of squashed strawberry stain. The second try was the right one. If there was a one-way sign tool in a library of signs, a simple click would allow the program to exactly identify the size, shape and color to treat. On could add new types of signs to the library.

It would prove perfect for latticed objects such a pylons, cranes, etc., which can be large but with a lot of air in the middle.

I think identifying stuff according to a library would be simpler than the current program (which would still be pertinent in many cases).

 

Nick

Link to comment
Share on other sites

+1 for nickdaum's suggestion, particularly for a tool that uses secondary shots, taken from different points of view, to build (and not to fake) what's behind an object that is to be "inpainted" in the shot taken from the main perspective.

 

 

For that purpose the program would follow a procedure rather similar to panoramas with the difference that, instead of putting images side by side, it would superimpose them on top of each other and most of all correct the perspective. Here how I see things:

  • Choose the master image, the one that was shot from the best spot with the best angle of view.
  • Choose the slave image(s), the one(s) that contain the parts missing in the master and shot from an imperfect spot.
  • Draw selection polygons around the stuff blocking the view and to be removed in each image.
  • If necessary draw a selection polygon around the part(s) to be restored.
  • Launch the stitching.
  • In Hugin, on each pic, the program identifies so-called matching Control Points by spotting characteristic features (angles, points of a roof or of a window for example). Yet it sometimes messes them up and anyway lets the user choose them by a click (1). This would help to do a very precise job especially if there is a strong perspective effect in the slave shots.

 

As you say it would be a killer because it could be very fast to apply and in case of a complicated and sophisticated background (like paintings, sculptures, etc.) the result would be perfectly genuine. It would allow to remove large objects like cars or trees for example or artifacts like reflexions you can't prevent (a light or the photograph in a glass).

 

I didn't try but I guess this could be done manually as well with layers and the perspective tool but one would need some patience to get a good result. I'll give it a try.

 

Hey, the Affinity guys, what do you think?

 

Nick

 

1) BTW this feature should be implemented in the Panorama Persona.

Link to comment
Share on other sites

 

Hi all, that's me again

 

I have yet another addition to my suggestions, #1.7, for something else that spoils so many otherwise nice landscapes:

 

1.7 Fences, chicken wire, mesh, railings.

Pretty much like lattice objects the overall size of such an object can be very large whereas its actual surface is comparatively very small and very see-through. It leaves plenty of the background to be seen that's therefore easy to restore. You can't imagine using the Impainting brush along all the wires. I am sure a smart tool could parse a given area and, once provided with the right clues, determine what's to remove in a breeze.

I shot recently an interesting building that has at mid-level a terrace with an additional high fence (probably to protect children). Terrible but impossible to impaint manually.

 

My 2cts.

 

Nick

Link to comment
Share on other sites

My wish for inpainting tool is that I could recreate absolutely same inpainting algorithm/pattern  on another layer.     For example I have a selection on first "color" layer , do inpainting, then go to another layer  representing depth information  and do one more inpainting with the exactly same pattern I have just did on "color" layer.

Link to comment
Share on other sites

My wish for inpainting tool is that I could recreate absolutely same inpainting algorithm/pattern  on another layer.     For example I have a selection on first "color" layer , do inpainting, then go to another layer  representing depth information  and do one more inpainting with the exactly same pattern I have just did on "color" layer.

 

Well, Kirk, if you have new suggestions you'd better create your own thread. It would give your propositions more visibility and it would allow this one focusing on its initial topic.

BTW, tell me what you think about my ideas?

Nick

Link to comment
Share on other sites

Sorry for heading in a bit different direction.    Having slightly different algorithms for different matters would be nice  indeed.   Although I doubt a software could recognise the subjects automatically.   Perhaps would took them  decades of Artificial Intelligence researches and a few Ph.D degrees in between.

 

Would be nice to just have a dropdown  list of slightly varying approaches to manually alternate. Maybe just a set of predefined copy/paste patterns . Would solve my problem too probably. 

Link to comment
Share on other sites

Having slightly different algorithms for different matters would be nice  indeed.   Although I doubt a software could recognise the subjects automatically.   Perhaps would took them  decades of Artificial Intelligence researches and a few Ph.D degrees in between.

That's why it should rather use a library of typical objects such as road signs and patterns such as lattice or mesh stuff.

 

Nick

Link to comment
Share on other sites

I find this very confusing. What difference does it make what shape the pixels take in the area you want to remove, or more accurately, fill in after removal? All of the information used to fill in those now missing pixels comes from the areas surrounding them. I don't understand how having a big library of all the things you might one day want to remove from an image has any impact on how well the inpainting tool works to fill in that shape with information from other parts of the image. You can make a precise selection of the offending object and still not get a completely satisfactory result because of the complexity of the surrounding pixels.

Link to comment
Share on other sites

 

Well, I explained that at length in my posts. In short, it seems to me my suggestions would significantly improve the productivity of the tool. By identifying exactly what's to remove the user wouldn't need in many situations to work stretch by stretch and struggle. When a wire passes in front of a rather homogeneous and/or simple, random background (sky, clouds, grass, foliage, etc.) it's a breeze. But when it comes to structured stuff with a mixture of trees, building facade, etc., it's time-consuming. It's even worse for latticed or meshed objects that are made of many very thin parts while being very large as a whole.

Read more carefully my explanations.

Moreover if you don't find it useful to you it doesn't mean it's useless for others.

 

Nick

Link to comment
Share on other sites

I was reminded at a Google project, they developed code based on a neural network that recalculates the pixels between two pictures.

 

You've got a couple of pictures shot in a burst and the algorithm then interpolates more pictures in between them an creates a fluid video. It therefore has to "guess" what lies behind certain things and how the 3D world is structured, although only the few pictures are available.

 

They also developed some code that takes all pictures taken around a significant building (for example Eiffel Tower) and then applies extrem perspective corrections on them to match them all together and then play them as a timeline hyperlapse, they got some really amazing results with that.

 

 

Link to comment
Share on other sites

I simply don't understand your explanations. If it is possible to make a tool like you describe it would truly be amazing. 

 

I did some programming in the past but not in the image processing technology.

Yet I suppose the technology I suggest would be simpler to implement than the current program which is very smart. The core idea is to make what's to be removed simpler to identify by the program and simpler and faster to point out by the user.

It would make the tool much more efficient for certain objects on top of being faster.

It would also make possible the removal of stuff like a large crane running across a building or a chicken wire around a garden. In some cases I didn't even try.

Such a tool, or set of tools, would be a fabulous marketing argument for Affinity.

Nick

Link to comment
Share on other sites

 

I think there is a need for a tool dedicated to flares. I have some shots of people dressed in Renaissance, damask outfits… with a few flares on them.

I tried different tools (impainting, clone, etc.) and none could reproduce correctly the underlying pattern. And yet the pattern is visible, it's just too pale.

With flares you don't need to replace something like you do with wires or signs by using the impainting tool. You need to enhance and generally darken the tone to match the surrounding.

OK guys, I know, one should keep his lens clean and shouldn't have flares in the first place. It serves me right. ;-)

Nick

Link to comment
Share on other sites

  • 10 months later...

How about an option for inpainting (for panas) that implements 'boundary warping' to stretch the image to fill in the missing areas? Sometimes that's all that's needed -- other times, of course, it's no good -- that's why an option would be nice...

I concur, well… so much that I created a thread about it: How to fill or impaint transparent gaps? ;)

Please add your support to it. :)

It doesn't seem to be in the pipe. There is such a tool in the Panorama Persona but it's only for panoramas you just did with AP. You can't even reopen a finished one and complete the job.

Nick

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...

Important Information

Terms of Use | Privacy Policy | Guidelines | We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.