Help please issue with RAW files

[ianhg]

Hi

I have an issue with some landscape RAW images. I use a Canon 7D Mk2 with canon lens. If I import RAW image into Affinity Photo on my iMac the sky sometimes has blown out area that are pink in colour, as below example.

If I use Canon software this does not appear. Do I need to set something up in preferences affinity photo to avoid this issue. Thanks for any suggestions.

 

[Old Bruce]

I am just guessing that the Canon Software has something which may be automatically 'correcting' the overexposed clouds. 

[DWright]

I have tested a RAW sample file for your camera and it is opening correctly for me, which MacOS are you running and is it possible for you to send me a copy of your RAW file to test.

 

[ianhg]

3 hours ago, Old Bruce said:

I am just guessing that the Canon Software has something which may be automatically 'correcting' the overexposed clouds. 

Thanks yes I think you are correct as jpeg images come out OK from the Camera. But not too sure if its a 'bug' in Affinity Photo or me needing to make some changes in the preferences settings.

[ianhg]

1 hour ago, DWright said:

I have tested a RAW sample file for your camera and it is opening correctly for me, which MacOS are you running and is it possible for you to send me a copy of your RAW file to test.

 

Thank you for the reply, it does not always happen. MacOS is Catalina 10.15.6 Jpeg images opened directly from the camera are OK. Not all RAW files have the issue, but enough to be an issue. Thanks, attached image 

7H1A1683.CR2

[Old Bruce]

32 minutes ago, ianhg said:

Thank you for the reply, it does not always happen. MacOS is Catalina 10.15.6 Jpeg images opened directly from the camera are OK. Not all RAW files have the issue, but enough to be an issue. Thanks, attached image 

I just downloaded your file and there is something wrong with it. What I see is a magenta colour on what could be blown highlights. They are not blown out though, when I turn off the highlights warning nothing changes. When I lower the exposure or highlights nothing changes. 

I am unfamiliar with the Canon 7D mk2 could there be a setting or settings for the camera which is causing the highlights to get corrupted? Have you checked for a firmware update for the camera?

 

EDIT: I should add that I am on OS 10.14.6 so it is not a Catalina problem.

[smadell]

I also downloaded your raw file. I opened it into both the retail version (1.8.3) and the new beta version (1.8.4 GM) and the results are the same. If I use either the Apple Engine or the Serif Engine (in the Assistant settings) and turn OFF the automatic Tone Curve, there is a magenta cast in the brightest areas of the clouds. If I turn the Automatic Tone Curve ON, that cast goes away. However, it promptly returns when I do anything that lowers the highlight values (lower exposure, lower highlights, and so forth). I also opened the photo in Iridient Developer (turning off the tone curve there, too) and found the same result. [I cannot open the image in my current copy of DxO PhotoLab, and I have a Nikon only copy of Capture One, so can't adequately evaluate it there.] However, my best guess is the same as that voiced by Old Bruce – this is not an Affinity problem, but perhaps a problem with the camera or its settings.

[smadell]

OK, I have a question. I wanted to know why my copy of DxO PhotoLab wouldn't open your image. Their website says that the Canon EOS 7D Mark II is supported, but that there is no support for mRaw and sRaw file types. Both of these file types have a diminished pixel count and/or resolution. Could that be the problem?

Perhaps you can change the settings on your camera to straight Raw instead of mRaw or sRaw and see if the problem resolves itself?

[ianhg]

1 hour ago, Old Bruce said:

I just downloaded your file and there is something wrong with it. What I see is a magenta colour on what could be blown highlights. They are not blown out though, when I turn off the highlights warning nothing changes. When I lower the exposure or highlights nothing changes. 

I am unfamiliar with the Canon 7D mk2 could there be a setting or settings for the camera which is causing the highlights to get corrupted? Have you checked for a firmware update for the camera?

 

EDIT: I should add that I am on OS 10.14.6 so it is not a Catalina problem.

Thanks, yes firmware is up to date. I also tried adjusting the exposure and highlights which have no effect. So, I am thinking it might be camera settings. Appreciate your input.

[ianhg]

51 minutes ago, smadell said:

OK, I have a question. I wanted to know why my copy of DxO PhotoLab wouldn't open your image. Their website says that the Canon EOS 7D Mark II is supported, but that there is no support for mRaw and sRaw file types. Both of these file types have a diminished pixel count and/or resolution. Could that be the problem?

Perhaps you can change the settings on your camera to straight Raw instead of mRaw or sRaw and see if the problem resolves itself?

Thank you,  yes i have had it mRAW, mainly to save file sizes. I will change back to RAW and see if that makes a difference. Can you think of any other settings in camera that might need to changed. The canon software does not show the issue, so there presumably is a software fix. Hopefully, using RAW will resolve the issue as I prefer Affinity Photo.

[Old Bruce]

I tried using the Desaturation brush on the Magenta colour in the clouds and those areas do look like they are blown out highlights so it may well be that the mRAW setting is dealing poorly with overexposure when it is resampled.

[ianhg]

12 minutes ago, Old Bruce said:

I tried using the Desaturation brush on the Magenta colour in the clouds and those areas do look like they are blown out highlights so it may well be that the mRAW setting is dealing poorly with overexposure when it is resampled.

Yes I think you are right I have changed to RAW and will see how it works out. But the fact that Canon software 'fixes' the issue maybe its something for Affinity Photo to look into in the future. Will post back again once I have checked it out with mRAW and RAW. Thanks for the help and guidance much appreciated. Just imported the offending RAW image into Apple Photo for editing and although I can still see the magenta areas they are very faint and smaller.

[lepr]

Capture One does not produce the magenta cast. Here are its results with no adjustment and then with exposure reduced by 2 stops and then 3 stops to show that the cloud detail was captured.

 

[Old Bruce]

I just did a quick test with my Canon 60D and can confirm that the mRAW setting does uck-fup the blown highlights. Oddly the extremely blown out areas are rendered as pure white with no indication from the histogram using the Show Clipped Highlights button of being overexposed.

[lepr]

3 minutes ago, Old Bruce said:

I just did a quick test with my Canon 60D and can confirm that the mRAW setting does uck-fup the blown highlights. Oddly the extremely blown out areas are rendered as pure white with no indication from the histogram of being overexposed.

The highlights aren't blown in @ianhg's raw file. Look at my images above from Capture One.

[thomaso]

On 8/1/2020 at 9:51 PM, ianhg said:

sky sometimes has blown out area that are pink in colour

This article appears to be related:  https://www.darktable.org/2012/07/magenta-highlights/

In APh your file occurs to me rather underexposed, compared to Lightroom and Canon DPP.

One reason might be that APh ignores a camera-internal correction written to the CR2, as for instance Canon camera's picture styles "standard", "landscape", "natural" etc. 
APh in my experience ignores for instance a wide-angle fisheye lens correction written directly into the raw file (CR3) of a Canon GX5 mkII (opened in APh as DNG), whereas macOS preview and Lightroom show the correction automatically and can't even not show it. For such an image, APh shows additional pixel content at the edges of the image, unlike other apps, including Canon DPP, which only display the rectified and cropped image. That makes me assume that APh opens more of a kind of "pure" raw data.

What me irritates with your CR2 is when run through the Adobe DNG converter its file size increases from 22 MB up to 32,7 MB (no jpg preview, no raw embedded). My experiences with CR2 and CR3 files are reductions only, of ~ 10-20 %, I never noticed an increase with a DNG conversion. (but I also never used mRaw or sRaw, don't know whether this matters – however it shouldn't increase file size in my understanding, too.)

[smadell]

33 minutes ago, thomaso said:

What me irritates with your CR2 is when run through the Adobe DNG converter its file size increases from 22 MB up to 32,7 MB (no jpg preview, no raw embedded). My experiences with CR2 and CR3 files are reductions only, of ~ 10-20 %, I never noticed an increase with a DNG conversion. (but I also never used mRaw or sRaw, don't know whether this matters – however it shouldn't increase file size in my understanding, too.)

Evidently, running sRaw and mRaw files through Adobe’s DNG Converter does indeed increase file sizes considerably. See this article:  https://protogtech.com/adobe-lightroom/canons-mraw-sraw-formats-and-dng/

 

[thomaso]

6 hours ago, smadell said:

DxO PhotoLab wouldn't open your image. Their website says that the Canon EOS 7D Mark II is supported, but that there is no support for mRaw and sRaw file types. Both of these file types have a diminished pixel count and/or resolution. Could that be the problem?

4 hours ago, Old Bruce said:

I just did a quick test with my Canon 60D and can confirm that the mRAW setting does uck-fup the blown highlights.

Interesting. Though I never used them I always expected mRaw & sRaw would only reduce pixel dimensions and maintain raw – but they also seem to be a different quality, here described for sRaw:

Quote

4) sRAW is NOT RAW

An uncompressed RAW / NEF file contains 14-bits of data per filtered pixel, so color and luminance information is demosaiced by software to form RGB pixel data. When you open a RAW file in Camera RAW or Lightroom, the software reconstructs the image in color by using a demosaicing algorithm on the bayer pattern. An sRAW file is already demosaiced and reconstructed by manufacturer’s camera firmware, so it does not contain most of the information from the RAW file.

https://photographylife.com/sraw-format-explained

 

[kirkt]

Magenta highlights In clipped areas occur when the green channel clips (red+blue = magenta) and the raw converter assumes an incorrect sensor saturation value. This is a problem on the raw converter side. Try changing the raw converter used, in the Develop Assistant. 
 

kirk

[thomaso]

5 minutes ago, kirkt said:

Try changing the raw converter used, in the Develop Assistant. 

There is "Serif Labs" and "Apple" in the menu for me. With both the magenta parts in the OP's image appear. Are there different options in APh for Windows?

[kirkt]

6 minutes ago, thomaso said:

There is "Serif Labs" and "Apple" in the menu for me. With both the magenta parts in the OP's image appear. Are there different options in APh for Windows?

I use a Mac. I do not know. sounds like the raw converter cannot reconcile the clipping properly for the quasi-raw file (ie, the mRAW is not supported). 
 

kirk
 

 

[Jowday]

On 8/1/2020 at 9:51 PM, ianhg said:

Hi

I have an issue with some landscape RAW images. I use a Canon 7D Mk2 with canon lens. If I import RAW image into Affinity Photo on my iMac the sky sometimes has blown out area that are pink in colour, as below example.

If I use Canon software this does not appear. Do I need to set something up in preferences affinity photo to avoid this issue. Thanks for any suggestions.

 

First of all this is a classic issue with exposure and cameras. Ground and clouds. But the magenta-purple area is extreme.

It is areas of blown out highlights with a lot of magenta instead that is clearly visible in the RAW file if I view it in programs that only displays the RAW image without any alterations besides basic demosaic processing.

It is a pretty bad example of what I would call magenta fill rather than chromatic aberration - I have never in my own images from non-Canon cameras seen so bad handling of light and overblown areas and so much magenta. Lens or sensor issue? I don't know. Perhaps use a filter on the lens. Or better take the image in the morning or evening.

Anyway, this is is real test of RAW converters - can they identify and handle this much magenta automatically? Photoshop 2020 did an okay job in the initial processing but pull down exposure and saturation up and its there. If I only reduce highlights it is beautifully suppressed. But it could still resurceface if I continued editing. So...

Well, shit in, shit out (overblown area). Unless you repair and count-adjust in post-processing. But the magenta is from the source, the RAW file. Shoot early or late (when the family eats breakfast or dinner, your popularity will not improve).

[Jowday]

Examples of filters to use to avoid these issues:

https://en.wikipedia.org/wiki/Graduated_neutral-density_filter

https://en.wikipedia.org/wiki/Polarizing_filter_(photography)

[ianhg]

7 hours ago, kirkt said:

Magenta highlights In clipped areas occur when the green channel clips (red+blue = magenta) and the raw converter assumes an incorrect sensor saturation value. This is a problem on the raw converter side. Try changing the raw converter used, in the Develop Assistant. 
 

kirk

Thanks Kirk, tried your suggestion and changed from Serif Labs to Apple (core raw image) the result was more blown out areas now in red when I reduced the highlights it removed the larger red areas and left behind the magenta/pink areas as my image showed. But thanks for the suggestion.

[ianhg]

7 hours ago, Jowday said:

Examples of filters to use to avoid these issues:

https://en.wikipedia.org/wiki/Graduated_neutral-density_filter

https://en.wikipedia.org/wiki/Polarizing_filter_(photography)

Thanks for your reply, it's difficult to know when to use the filters as it does not always happen, depends on the light even on the same day. So maybe I will try the filters but for now I will shoot RAW and not mRAW and see if that makes a difference. Strange that Canon software resolves the issue, guess they are aware of it. So it appears that there is nothing I can change in AP preferences or set up to improve the issue. I will test shooting in RAW and mRAW on the same day and see the results. Will report back. Image attached after using Canon Software