Focus merge feedback

[TheLostVertex]

Hello, I do a lot of focus stacking, mostly with Zerene Stacker. So I would like to give my impressions of the focus merge functionality.

 

First, the stacking method looks a lot like Zerene's pmax method, or Helicon's Method C. But I do notice that Affinity is generating a depth map during the stacking sequence. Also there appears to be less noise and less increase in contrast that Zerene's pmax. Can the Affinity team elaborate on how they are stacking images? (Zerene method documentation: http://www.zerenesystems.com/cms/stacker/docs/howtouseit#dmap_versus_pmax HeliconDocumentation: http://www.heliconsoft.com/helicon-focus-main-parameters/ ) On to testing!

 

Running on a 27" 3.4ghz i7 iMac with 24gb ram.

 

First test: 53 images, 6000x4000 16 bit uncompressed tiffs (each 144mb)

Zerene Dmap: Completed 0:08:30

Zerene Pmax: Completed 0:03:51(ran directly after Dmap, so I believe alignment was cached). Fresh project with no previous alignment: Completed 0:05:32

Affinity Photo: Completed 0:05:50

 

So I stepped up the next test to see how it scaled(not well).

 

Second test:  283 images, 6000x4000 16 bit uncompressed tiffs (each 144mb)

Zerene Pmax: Completed in 0:28:00 Zerene used a peak of 12.8gb of memory, 0gb swap.

Affinity Photo: Completed in 1:35:00 Affinity Photo used 27gb memory and 8.7gb swap during image load and alignment, and 4gb memory with 3gb swap used during all of the"focus merge" progress bar, according to activity monitor.

 

During stacking, Zerene would make full use all CPU cores. Affinity photo would use very little cpu during image loading and alignment, and about 50-60% utilization during the "focus merge" progress bar. I think the way Affinity Photo handles loading images and aligning them can be optimized a lot more. Most of the time for Affinity Photo was spent loading the images(before the "aligning" progress bar appeared). 

 

Retouching in Affinity Photo feels much slower than Zerene. Switching source images seems to take a longer time. But much worse than that, scrolling the sources windows hangs, and gives a spinning beach ball for several seconds each time you attempt to scroll through it. Having the ability to paint from source images onto layers is great, but the performance is truly terrible. When scrolling through a list takes 10-20 second PER ATTEMPT, I dont think this feature is practically usable in beta 4. Speeding up loading of source files and fixing the scroll behavior seems like the most needed improvement for the Focus Merging feature.

 

Comparing images from the 283 image stack, there are some noticeable differences.  I choose this set of images because I did not feel like Zerene did a great job with it, and my camera sensor was particularly dirty. Affinity photo shows halos around edges some, and the alignment looks like it is warped compared to Zerene. Dust patterns are very different between Zerene's Pmax and Affinity Photo, so it does seem like Affinity is stacking with some other method. Noise is noticeably lower with Affinity Photo. Contrast and color is also generally a little better than Zerene's Pmax. However in multiple test images Affinity photo renders spots that should be the darkest parts of the image as being much brighter than it should be. (See the attached images, and notice the dark section in the Zerene image vs the pinkish section in the Affinity image.) Overall I think Zerene renders the subject truer to life, mainly due to the much better alignment. The warping in the Affinity image is very noticeable in some parts of the image. 

 

Finally, a word on nomenclature and me being an idiot. The first time I went to run the 283 image stack, I had accidentally used the option "New Stack" rather than "New Focus Merge". This lead to about an hour and a half of nothing happening before I decided to abort the operation. I only realized what I had done later, because the "Stack" dialog has a dropdown menu for how to align the images, and the "Focus Merge" dialog does not. Normally I, and I think most people, refer to this operation as focus stacking, so it was a mistake of habit and not paying attention. But it would be nice if the dialogs were a little more distinct to help prevent such user error. 

 

Overall, Focus merge works pretty well I think. It certainly has a lot of potential. I would summarize my suggested improvements as such:

 

-Add some UI feedback while images are loading into memory(it just looks like it is hanging until it gets to alignment)

-Add alignment options in the Focus Merge dialog. It looks like it is doing perspective alignment? I believe a lot of images would benefit from only scale/rotate/translate

-Improve memory management and loading for images, I am assuming this is the biggest performance slow down

-Fix long load times for selecting source images

-Fix hang when scrolling in the source panel

-Fix issue with darkest part of of image being light and washed out(if this is actually a bug, and not a limitation of the stacking method)

-Make Focus Merge dialog more distinct from the Stack dialog...to help with people being stupid and tired like me :)

 

I hope this feed back helps you guys. I look forward to seeing any improvements made. Also, if larger test stacks would help you guys in development, I would be happy to provide test images if needed. 

 

EDIT: I just tested selecting source images again, and the delay for selecting the image wasnt very bad at all, ~1sec. So either it was worse when testing it before, or I was exaggerating in my mind how it felt for some reason. Scrolling the source images with the scroll bar is still as described though. From clicking to scroll, till the source panel becomes responsive again, it takes no less than 10 seconds.

Edited August 28, 2016 by TheLostVertex

[Justin]

Thank you for the feedback, it's very helpful.

 

I know what some of the speed/memory issues are so hopefully that side of things will improve. The artefact in your result involving the pink area that should be black is also something that I am aware of but still need to think about. I don't know why the alignment would be poor, so if you could let me have an example image set via dropbox that would be great. 

 

Cheers

Justin

[TheLostVertex]

Im happy to provide some image sequences here. All images are jpeg. 

 

Here is the image with the pink background(a 20x magnification photo of a knife tip)  utilityknife.zip (1.68GB) zerene comparision images

Attached also are the stack outputs of Zerene Pmax with the provided jpegs. One was stack from image 1 to 283, and the other from 283 to 1. The automatic alignment option(where the software chooses the order) goes 283 to 1 for some reason, and so it produces the edge streaking. Both images the alignment looks straighter and less distorted to me. The sensor dust on the left side of the image provides some cues to the difference in alignment between Affinity photo, and the two different Zerene images. 

 

Also here is a 3 part image of paracord, meant for stacking and then stitching into panorama. paracord1.zip(705MB) paracord2.zip(644MB) paracord3.zip(760MB)

On the right side of the image the alignment seemed to be slightly distorted compared to Zerene's output as well. Also where the green weaves meet other weaves displays the artifact where dark sections appear lighter than they should, and in this case a lack of detail too. There are also a small alignment issue on the right side when creating a panorama, which I think is partially result of the alignment during stacking:

 

Affinity Photo Stacks

  

 

 

Zerene Stacks

 

Both images do not line up correctly, but its worse with the AP stacks. It is also pretty hard to correct fully with the pano tools. 

 

If anybody here on the forum wants to test this feature with the images above, feel free too. 

 

Hope this helps some. And thank you for the reply. 

 

-Steve

[TheLostVertex]

I have also had a chance to test some smaller image sequences, between 10-30 images. The scrolling behavior of the sources panel does not seem as bad with fewer images, but is still an obvious issue. With 20 images, it takes about 3 seconds for it to scroll and become responsive again. 

 

Overall I have been pretty happy with the results I have seen from testing. Its pretty close to what I would see in other software, in most cases. And of course MUCH better than the extremely poor

implementation of focus stacking that is in photoshop. 

 

Here are some the the stacks I had a chance to test out last night

 

Paracord stack and stitch from my previous post

 

Stale bread crust

 

RF blocking weave from a cell phone PCB

 

Knife edge @ 20x. Having all the editing tools standing by was very helpful for this, because there was heavy purple fringing on the edge. Being able to select and clone it away immediately is nice.

 

Good job with this feature guys. I look forward to seeing its development.

[TheLostVertex]

A quick update since I just saw beta 5 was released. 

 

Using the large stack I describe in the first post here are the beta 5 results(and a recap of the previous test):

 

Zerene Pmax: Completed in 0:28:00 Zerene used a peak of 12.8gb of memory, 0gb swap.

 

Affinity Photo Beta 4: Completed in 1:35:00 Affinity Photo used 27gb memory and 8.7gb swap during image load and alignment, and 4gb memory with 3gb swap used during all of the"focus merge" progress bar, according to activity monitor.

 

Affinity Photo Beta 5: Completed in 0:34:30 with a peak memory usage of 3.2gb(awesome!) and 66mb swap(I am not sure if this was due to Affinity photo or not, so we can assume no swap used)

 

That is a nearly a 280% speed improvement from the previous beta. 

 

The sources panel is fixed now and does not lag when scrolling or when activating the source preview button. There is feedback for when a source image is selected and loaded now(I do not believe there was a progress bar before?). Much improved.

 

The weird image highlights appear to be improved a lot. Also as important, image alignment seems better. Beta 4 alignment seems a bit distorted, beta 5 alignment is much closer to reality(though I do see a tiny bit of warping, its much improved). Halos in this particular image seem improved. 

 

Really good job with the update everyone! Pretty much everything I mentioned appears to have been addressed. I will plan on looking into things more in a few days when i have the time.

 

Attached is the beta 4 output and the beta 5 output of the previously tested stack. The changes in washed out areas and alignment should be obvious. 

[TheLostVertex]

Some bugs to report, and then some more performance test results. First the bugs I have found.

 

All of these are in reference to the sources panel:

-No progress bar when switching images in the source panel when "toggle source preview" button is enabled

 

-When "toggle source preview" is not enabled, a newly selected source will not load until the mouse cursor hovers over the canvas

 

-Selected sources will not load unless a layer is selected. 

 

-Source image will not show on screen when "toggle source preview" is enabled unless a layer is selected

 

---------------------

 

I got a chance to test on the performance and ram usage of beta 5 as well. All images used we 16bit 6000x4000 tiffs

 

Focus merge 1

Initial load of 6.5gb ram. Once alignment starts ram usage drops to a max of 3.6gb. (it fluctuates as I assume it is reading and dispatching images)

237 images in 42:29

 

Focus merge 2

Previous projected and left AP using 4gb ram with no documents open, for what ever reason. New stack started, initial loading peaked at 8.2gb. Once alignment started it dropped to a max of 6.5gb.

236 images in 41:43

 

Focus merge 3

Previous project closed and left AP using 6.8gb ram with no documents open. New stack started, initial loading peaked at 8.9gb. Once alignment started it dropped to a max of 7.9gb.

357images in 1:01:28

 

Final document closed, leaving AP using 7gb ram with no open documents.

 

I think what I gather from this, is either AP appears to be using a lot of ram from how OSX handles active/inactive memory OR affinity photo has a memory leak when closing documents. Perhaps specifically related to focus merging? At the very least it appears like ram usage climbs with each use. 

 

Other than that, memory usage is very good now it looks like. Over all speed is just slightly slower than Zerene(what I normally use) now as well. For no reason what so ever, I also timed each "portion" of the first test. Because graphs are fun? -_- 

[anon1]

A German video about focus stacking in AP VS PS CC

https://youtu.be/lgj5fihZXvY

[Puffin]

Thanks for a detailed explanation TheLostVertex, this for me will take a lot of practice but it is a good starting point and i'm grateful, thanks again.

[Asha]

Link to another thread on this topic:

 

https://forum.affinity.serif.com/index.php?/topic/29566-focus-stack-feedback/

 

Mods, please merge the threads, if possible.