HDR Merge not retaining highlight detail

[Corgi]

Maybe I'm not understanding HDR Merge completely, but with the images I'm working with, the default settings are routinely blowing out highlights, even though there is detail in at least one of the merged images.

Using 1.10.5.1342 on Windows 10. Hardware acceleration is disabled.

For example, here are two RGB/8 images that, when HDR Merged (with tone mapping), produce clipped highlights on the kid's shoulder with the default tone mapping settings (see cropped Result.tiff image). However, if (in Tone Mapping) I lower the Highlights slider in the Shadows & Highlights section and apply the image (Result-lower.tiff), there is clearly detail in the same area (as there is, of course, in the original 2.jpg).

I tried playing with the Clamp to SDR checkbox, but it seemingly had no effect. Similarly, deselecting Noise Reduction prior to the merge still resulted in the same clipped highlights.

Seems to me that merging two RGB/8 images (just a few stops apart) should produce an RGB/32 image without any blown highlights, at least when using the default Tone Mapping settings. Am I doing something wrong?

Result.tiff Result-lower.tiff

[NotMyFault]

Well the tone mapping defaults are just defaults, and would say i never used that default settings except to immediately change them.

Affinity gives you full control over dozens of sliders and direct real time preview of how they affect your document.

It does not analyze the image to create an image-specific settings for highlights and shadows. You can reduce dynamic range if you like, or just increase local contrast. 
 

Denoise is almost independent from HDR, it is just a comfort default settings (which i change to 0 in most cases).

Actually, creating HDR from RGB/8 (effectively grey, only 256 color values) is severely limited by this source material. These image live from the film grain, nothing much to gain with hdr. 

It is possible and might give you the results you like, but scanned B&W images have not enough dynamic range at all to just a HDR workflow.

Proper scanning with a 16 bit capable scanner will probably give you better results and save lot of editing time. Just my 2 cents.

 

[NotMyFault]

The issue with your images: an hdr merge creates a file which needs to be stretched instead of reduced. 
if you check the histogram (before tone mapping, or with compression to 0) you can spot this.

So the default tries to create a punchy image. But just reduce the slider a bit and everything is ok.

you can save this as preset for convenience, and activate this with one click.

Again, there is no dynamic range required any tone mapping in the source files. A simple curves adjustment will do a good job.

[Corgi]

Thank you for your thoughtful replies, @NotMyFault.

Quote

Well the tone mapping defaults are just defaults, and would say i never used that default settings except to immediately change them. ... So the default tries to create a punchy image. 

I have about 500 images to process, and wasn't thrilled by the idea of needing to adjust each merge to avoid clipping highlights. I've been using a new tone mapping preset to avoid clipped highlights as you suggest, so it's not a big problem. But it still seemed to me that, in principle, the default process for doing an HDR merge would always try to retain as much info as possible. It feels like a bug to me -- in other words, when doing something called an "HDR merge," the application's priority should be to maximize dynamic range, with creating a "punchy" image as lower priority.

Quote

It does not analyze the image to create an image-specific settings for highlights and shadows. 

Well, as above, that feels like it should be an option, if not standard behavior. Automation, you know?

So the main reason for my post was to find out whether I'm using HDR Merge improperly. If I understand your replies, the answer is that I'm using HDR Merge correctly, but the default HDR Merge (and tone mapping) settings shouldn't be relied upon to capture all of the available dynamic range in the source files. Which is a shame.

What follows is a bit of info explaining why I'm doing things this way.

I do agree with you that scanning with a 16-bit scanner would give superior results. However, from my experience, it would consume a bit more time. I actually tried a 16-bit scanner first but had *major* issues with dust (the negatives had been stored improperly, and are noisy Tri-X), and spent a great deal of time tweaking Vuescan for each scene. Very frustrating. Since the vast majority of these images aren't worth a huge time investment I decided to use a cheap "instant" scanner, which felt far faster than the more expensive scanner. After a few tests, I concluded that taking several different exposures with this cheap scanner (to compensate for its limited dynamic range) and merging them produced images that were weren't drastically inferior to those created using the 16-bit scanner.

So my goal was to produce "fair" images quickly, with a bit more dynamic range than this cheap scanner could produce with a single capture. I'm not trying to create a true HDR image, but am using the HDR Merge workflow so I end up with exported RGB/8 images that are a bit better than I'd get by doing a single RGB/8 scan and then, say, Auto-Levels.

Just to quantify this a bit: with this cheap scanner, capturing three different exposure levels per negative, I averaged about 45 seconds per negative. Then to merge, tweak, and export each negative presently takes about another 45 seconds per negative. 

Having to click on my custom preset to avoid blown highlights only takes a second: no big deal. It just seemed to me that this step should be unnecessary.
 

[Old Bruce]

You don't need to use the Tone Mapping. Just uncheck the box.

[Corgi]

44 minutes ago, Old Bruce said:

You don't need to use the Tone Mapping. Just uncheck the box.

When I try that with the two source images I provided, the default result is even worse (even more pronounced highlight clipping)

[Old Bruce]

3 hours ago, Corgi said:

When I try that with the two source images I provided, the default result is even worse (even more pronounced highlight clipping)

Not here. A simple Levels or Curves adjustment layer gave me a quite satisfactory result after unchecking that box. We may have different concepts of blown highlights.

[NotMyFault]

There are a few things worth commenting  

• As you found out, the tone mapping persona does not fit perfectly to your intended workflow. This is no bug.
• this workflow you describing has been tried by others before. Other users tried a similar workflow (3 scans with different settings) for the same reason (save time, stay with 8 bit scanner) and found out it does not work.
• Film slides normally show a very low dynamic range, and do not require HDR processing at all. A single scan even in 8 bit will deliver all DR contained in the slides / images.
• It does not make any sense to do 3 passes of extremely low DR and combine them via tone mapping to about 6 bit DR. It will not reduce noise or scratches (which are part of the source image, and not created by the image sensor), as hdr stacking will disclose these imperfections just in higher quality.

You can continue to try your workflow, but i wouldn’t be surprised if this never gives the results you’re expecting, no matter which hdr mapping app you try out.

 

 

[Corgi]

7 hours ago, Old Bruce said:

Not here. A simple Levels or Curves adjustment layer gave me a quite satisfactory result after unchecking that box. We may have different concepts of blown highlights.

Odd. When I HDR Merge those two images, there is no recovering the detail on the right-hand side of the kid. It is pure white. I can use Curves, or use Levels and put the Gamma all the way up to 2, and same result.

[NotMyFault]

HDR Stack results in RGB/32 files. The 32bit preview then controls if an ICC conversion gets applied (gamma curve for rendering only) or not (linear mode). 

Which settings do you use for that? (Screenshot).

It is best practice to use ICC mode, otherwise rendering will change (unexpectedly) when converting or exporting.

To be on the safe side, what happens if you convert the document to RGB/16 (and sRGB profile)?

Using curves after conversion is much easier, to avoid the gamma/linear complexity of RGB/32 documents.

HDR preview relies on 100% correct settings, including that the display correctly states its nits (max brightness) via DDI channel to the OS. Unfortunately those metadata APIs evolved slightly and what worked good since VGA sometimes breaks with modern HW/SW

 

[Corgi]

Quote

• As you found out, the tone mapping persona does not fit perfectly to your intended workflow. This is no bug.

Perhaps it isn't a bug; I don't know the intent of the coders. But it is too bad that there isn't a one-click tool to combine stacked exposures such that the combined dynamic range is retained. It still seems strange that a tool that is intended to create HDR images would discard chunks of dynamic range like this. I also briefly tried the Stack tool but it seemingly offered less control, and also produced an image that needed significant adjusting to get what I wanted.

Quote

• ...Other users tried a similar workflow (3 scans with different settings) for the same reason (save time, stay with 8 bit scanner) and found out it does not work.

Based on my preliminary results, I'm finding that it does indeed work for me. After tweaking the tone mapping and the resulting combined image, I end up with an 8-bit image that contains details both in the highlights and the shadows that simply are missing in any single capture from my scanner. I presume that it depends highly on the quality of the scanner, and presumably if my scanner wasn't so contrasty, or if it was Gray/16 instead of RGB/8, then I'd be in better shape.

Quote

• ...A single scan even in 8 bit will deliver all DR contained in the slides / images.

I'm scanning negatives, and as above, my single 8-bit scans seem to be clipping either shadows or highlights.

Quote

• It does not make any sense to do 3 passes of extremely low DR and combine them via tone mapping to about 6 bit DR. It will not reduce noise or scratches (which are part of the source image, and not created by the image sensor), as hdr stacking will disclose these imperfections just in higher quality.

You are correct; the noise and scratches are being amplified by the tone mapping. But I am also seeing greater detail at the extremes.

I confess to not understanding the complexities of the RGB/32, tone mapping, and ICC profiling. I've not profiled any of my equipment, and for decades, I've lived with the results which have been plenty good enough for my casual, non-professional tinkering (at least up till now :-))

Quote

Which settings do you use for that?

If I select "Unmanaged" then the result looks a whole lot better on the screen, though highlight detail is still blown in the resulting image unless I tweak the tone map.

Quote

To be on the safe side, what happens if you convert the document to RGB/16 (and sRGB profile)?

Not clear on what you're suggesting. Do you mean to convert my source images before merging, or after merging?

Thank you for all of your replies. You've more than answered my original question!

What I've learned is that I'm using a pretty sophisticated tool in a questionable manner. That being said, I'm reasonably satisfied with the result after I ensure that the tone mapping doesn't clip highlights, and then apply curves and/or levels on the RGB/32 result prior to exporting to RGB/8. I end up with an image that, while dusty, doesn't have severely blocked shadows or blown highlights.

 

 

 

[NotMyFault]

Thanks for your summery and (kind of) lessons learned. Good to know that you finally found a workflow working for you.