Some questions on the RAW processing step...

[hooty]

I can understand that you are doing non-destructive edits to a RAW but then destructive edits on what is displayed in Photo Persona or Photoshop so I'm trying to understand what these two different objects are that you are working on in these two different steps.

• 1) What happens in RAW editor (Adobe Camera Raw, Lightroom, Capture One, Affinity Develop, Darktable) to get from the RAW values (the 14-bit values in the RAW file in my case) to the initially displayed image on the screen? When I open a RAW file, I can see a color image. One of the steps must be interpolation/demosaicing but what else happens in the time between opening the RAW file and seeing it on the screen and ready for you to start moving sliders and changing values?
• 2a) What is the end goal of editing the RAW file? What work should be done in the RAW editor before it is completed? I see that you can change colors and exposure and a lot of other things but you can also do this after the export. If you adjust your colors, contrast, exposure, and whatever else while in the RAW editor, should you try to avoid doing another round of those in the exported file? I'm trying to determine what my job should be while in RAW editing vs. whatever you do after you export it for photoshopping or something else.
• 2b) If the input for the RAW editor is a raw file, what is the output? I know that you can just export as a jpg or tiff or something but if you just "develop" it and go to the next step (like Affinity's "Photo Persona" or Adobe's Photoshop) then what is that image on the screen with the controls for Photoshop or Develop Persona?

 

[Old Bruce]

I use the Develop Persona (the "raw editor") to get my .CR2 files close to what I think I need. I'll use a bit of noise reduction and some chromatic aberration correction plus defringeing. Then I'll apply a curves adjustment. 

Now I save the file as a 16 bit .afphoto document.

I will continue to do some more work on this file and then export as a .tiff, .jpeg, .png, etc.

[Ldina]

36 minutes ago, hooty said:

1) What happens in RAW editor (Adobe Camera Raw, Lightroom, Capture One, Affinity Develop, Darktable) to get from the RAW values (the 14-bit values in the RAW file in my case) to the initially displayed image on the screen? When I open a RAW file, I can see a color image. One of the steps must be interpolation/demosaicing but what else happens in the time between opening the RAW file and seeing it on the screen and ready for you to start moving sliders and changing values?

It's a deep subject and can't be covered adequately here, but I'll make a few comments. First, the sensor captures three B&W images using red, green and blue filtered sensors. The RAW processor has algorithms that convert these light values (voltages) to intensity levels. Most RAW processors use ProPhoto RGB as their native color space because it has a HUGE color gamut and won't prematurely clip colors. So, the RAW processor converts the voltages at each pixel location and encodes it to fit within ProPhoto RGB (Affinity uses ROMM RGB, which is really ProPhoto). So, you have three B&W images representing the red, green and blue channels. 

These are offset into a mosaic on your camera sensor, so the RAW processor "de-mosaics" them and lays them one on top of the other, so they are aligned. Now you have the basis for a full RGB image, but it's still in Linear gamma. So, another algorithm converts from linear to a tone curve (unless you choose not to do so). This distributes to tones to mimic human vision. Now, you have an RGB image. 

The RAW converter needs some sort of color profile to generate accurate colors, and this varies with each camera. Some RAW processors allow you to create custom camera profiles, but LIB-RAW (which Affinity uses) does not support custom camera profiles, so it uses a generic profile in the background. Anyway, this profile is responsible for maintaining accurate relationships between colors (we hope). When you change various sliders and settings in RAW, you are manipulating color and tone values, etc. This will be displayed on your monitor Live as you make changes to the RAW data in the file. 

Once you are pleased with the look, you export to Photo. This involves a conversion from 32-bit linear ProPhoto to your chosen color format (eg., sRGB, 16 bit). The colors will be converted to fit into your chosen color space and opened in Photo as a pixel based image. If you set the export from the Develop Persona to "Pixel Layer", your RAW edits will be baked into the file. Affinity also allows you to set the export to RAW layer linked or Raw layer embedded, which allows you to re-edit in RAW, so it is non destructive. 

What you see on your monitor depends on your monitor's color gamut. If your monitor can display only sRGB gamut, anything outside of its gamut will be clipped to on display to fit into your monitor's color gamut. The profile of your monitor will control how colors get mapped to the screen in real time. 

 

56 minutes ago, hooty said:

2a) What is the end goal of editing the RAW file? What work should be done in the RAW editor before it is completed? I see that you can change colors and exposure and a lot of other things but you can also do this after the export. If you adjust your colors, contrast, exposure, and whatever else while in the RAW editor, should you try to avoid doing another round of those in the exported file? I'm trying to determine what my job should be while in RAW editing vs. whatever you do after you export it for photoshopping or something else.

That depends on you. I typically prefer to do most of my editing (non-destructively) in Photo, so my edits in the Develop Persona get me into the ballpark, without clobbering highlights, shadows, blacks, whites, etc. I want to be reasonably close and get a conservative pleasing image, which I will adjust (sometimes considerably) in Photo. Others may want to do everything they can in RAW and do little or no editing in Photo. One size does not fit all. 

Once converted to RGB pixels, you can do another round of editing in Photo...no problem. But if your edits are extreme, you will risk banding, artifacts and your image can break down if too excessive. For this reason, 16 bit is preferable when editing (usually). You have more flexibility to make MAJOR changes in RAW without destroying your image. I just try to get "in the ballpark" in RAW in terms of color, contrast, brightness, saturation, etc. Once again, for me, I keep my RAW edits somewhat conservative and leave some headroom for further edits in Photo. So, I don't usually 'take it to the max" in RAW.

1 hour ago, hooty said:

2b) If the input for the RAW editor is a raw file, what is the output? I know that you can just export as a jpg or tiff or something but if you just "develop" it and go to the next step (like Affinity's "Photo Persona" or Adobe's Photoshop) then what is that image on the screen with the controls for Photoshop or Develop Persona?

The output is a pixel based, RGB AfPhoto file in your chosen color space (e.g., Adobe RGB, 16 bit). If you save in the native afPhoto format, you will retain all your layers, edits, etc. To share or post images, you will normally export to JPG, TIFF, PNG, etc. You can downsample (reduce size), upsample (increase size), choose a different color space (sRGB is safest for general use, Adobe RGB, P3, ProPhoto RGB, etc). Only people with AfPhoto or an AfPhoto capable reader will be able to see you Affinity files, so save to the format of your choice for sharing with others. For sharing, I usually choose JPG, 85% quality, 1600 pixels on the long side. For me, that is a good compromise between filesize and quality, and suitable for email. Some prefer full sized JPGs, or reduce the pixel dimensions even smaller. It depends on how you will use that image. 

I didn't understand the last part of your last question. This ought to get you started. It's not complete by any means and there's more to learn. 

I hope this helps point you in the right direction. If most of your work is for email, posting on the web, etc, then I'd probably use sRGB. It will clip some super bright, saturated colors, but it works well for a large percentage of images and will be the safest bet when sharing with others. 

This is very long because you seem to really want to know (having read some of your other posts). I think I've said all I have to say on the subject. If you want more information, Google your questions. Good luck.

 

[hooty]

29 minutes ago, Ldina said:

It's a deep subject and can't be covered adequately here, but I'll make a few comments. First, the sensor captures three B&W images using red, green and blue filtered sensors. The RAW processor has algorithms that convert these light values (voltages) to intensity levels. Most RAW processors use ProPhoto RGB as their native color space because it has a HUGE color gamut and won't prematurely clip colors. So, the RAW processor converts the voltages at each pixel location and encodes it to fit within ProPhoto RGB (Affinity uses ROMM RGB, which is really ProPhoto). So, you have three B&W images representing the red, green and blue channels. 

These are offset into a mosaic on your camera sensor, so the RAW processor "de-mosaics" them and lays them one on top of the other, so they are aligned. Now you have the basis for a full RGB image, but it's still in Linear gamma. So, another algorithm converts from linear to a tone curve (unless you choose not to do so). This distributes to tones to mimic human vision. Now, you have an RGB image. 

The RAW converter needs some sort of color profile to generate accurate colors, and this varies with each camera. Some RAW processors allow you to create custom camera profiles, but LIB-RAW (which Affinity uses) does not support custom camera profiles, so it uses a generic profile in the background. Anyway, this profile is responsible for maintaining accurate relationships between colors (we hope). When you change various sliders and settings in RAW, you are manipulating color and tone values, etc. This will be displayed on your monitor Live as you make changes to the RAW data in the file. 

Once you are pleased with the look, you export to Photo. This involves a conversion from 32-bit linear ProPhoto to your chosen color format (eg., sRGB, 16 bit). The colors will be converted to fit into your chosen color space and opened in Photo as a pixel based image. If you set the export from the Develop Persona to "Pixel Layer", your RAW edits will be baked into the file. Affinity also allows you to set the export to RAW layer linked or Raw layer embedded, which allows you to re-edit in RAW, so it is non destructive. 

What you see on your monitor depends on your monitor's color gamut. If your monitor can display only sRGB gamut, anything outside of its gamut will be clipped to on display to fit into your monitor's color gamut. The profile of your monitor will control how colors get mapped to the screen in real time. 

 

That depends on you. I typically prefer to do most of my editing (non-destructively) in Photo, so my edits in the Develop Persona get me into the ballpark, without clobbering highlights, shadows, blacks, whites, etc. I want to be reasonably close and get a conservative pleasing image, which I will adjust (sometimes considerably) in Photo. Others may want to do everything they can in RAW and do little or no editing in Photo. One size does not fit all. 

Once converted to RGB pixels, you can do another round of editing in Photo...no problem. But if your edits are extreme, you will risk banding, artifacts and your image can break down if too excessive. For this reason, 16 bit is preferable when editing (usually). You have more flexibility to make MAJOR changes in RAW without destroying your image. I just try to get "in the ballpark" in RAW in terms of color, contrast, brightness, saturation, etc. Once again, for me, I keep my RAW edits somewhat conservative and leave some headroom for further edits in Photo. So, I don't usually 'take it to the max" in RAW.

The output is a pixel based, RGB AfPhoto file in your chosen color space (e.g., Adobe RGB, 16 bit). If you save in the native afPhoto format, you will retain all your layers, edits, etc. To share or post images, you will normally export to JPG, TIFF, PNG, etc. You can downsample (reduce size), upsample (increase size), choose a different color space (sRGB is safest for general use, Adobe RGB, P3, ProPhoto RGB, etc). Only people with AfPhoto or an AfPhoto capable reader will be able to see you Affinity files, so save to the format of your choice for sharing with others. For sharing, I usually choose JPG, 85% quality, 1600 pixels on the long side. For me, that is a good compromise between filesize and quality, and suitable for email. Some prefer full sized JPGs, or reduce the pixel dimensions even smaller. It depends on how you will use that image. 

I didn't understand the last part of your last question. This ought to get you started. It's not complete by any means and there's more to learn. 

I hope this helps point you in the right direction. If most of your work is for email, posting on the web, etc, then I'd probably use sRGB. It will clip some super bright, saturated colors, but it works well for a large percentage of images and will be the safest bet when sharing with others. 

This is very long because you seem to really want to know (having read some of your other posts). I think I've said all I have to say on the subject. If you want more information, Google your questions. Good luck.

 

I appreciate all this. I have given it a read and will re-read it multiple times to let it all sink in and it will get me where I need to go in my little journey to understand it to the best of my ability.

[Ldina]

Recently (as mentioned), Affinity added a feature that allows you to re-edit a RAW file, without having to start from scratch all over again. If you use RAW Layer Linked, or RAW Layer Embedded, you can go back and forth from Photo to the Develop Persona, which is a very nice feature. With these features, your RAW settings and sliders are remembered, so you can make some quick tweaks if you need or want to. White Balance, for example, works better with RAW images, so it can make sense to do it there.

When exporting to a Pixel Layer (which was the only option previously, and what most RAW converters do), your edits in the RAW processor are "baked in" and delivered to Photo, as a single, flattened pixel layer (like a camera created TIFF or JPG). So, RAW development to a pixel layer is, in essence, a destructive process since all those RAW adjustments are delivered in a flat bitmap file in your chosen file format (color space, bit depth, etc). Yes, you CAN start over with your RAW file, so in that sense it is a non destructive process, but it's not as convenient and is more time consuming. When editing in the Pixel layer Export mode, it was probably a bit more important to be conservative in the Develop Persona, otherwise you'd have to start over again from scratch. The Linked and Embedded options make it easier to go back and forth, using each persona for its strengths. 

So...where to do the edits? My choice is usually to remain somewhat conservative in the Develop Persona and do most of my editing in Photo, where I can use non-destructive layers, filters, etc. By conservative, I mean delivering a file to Photo that leaves headroom in whites and blacks (little or no channel clipping), has plenty of detail in the highlights and shadows, isn't overly saturated, super noisy, etc. The nature of RAW file data allows more aggressive adjustments to be done in the Develop Persona, without doing damage (or as much damage) to the file. So, if you have a very under exposed, low contrast image you shot as RAW, it makes sense to get it "about right" or "in the ballpark" in the Develop Persona before moving to Photo.

Once in Photo, I can use non-destructive, layer based edits to fine tune my image, tweak the blacks, whites, highlights, shadows, saturation, noise reduction, sharpening, or whatever. 

Hope that helps. Start by using the conservative approach in the Develop Persona and see how that works. As you gain experience, you'll learn what to do where for optimal results. 

[hooty]

I have tons of questions and I'm currently fixated on the step of taking the 14-bit (in my case) data in the RAW file and applying ProPhoto RGB and I'll get there before I move on thanks to your earlier post.

But to flesh out the intent of my question in the separation of duties between RAW editing with the tools available and then editing that output with a whole other range of tools, it begs the question of trying to determine the strengths and weaknesses of each step in the processing. To me, there are unique tools in each step but there is some overlap such as exposure. I am okay with it all being subjective and based on taste per person but when it comes to overlapping tools like exposure changes, should I be asking "is it better to handle exposure while editing the RAW or the exported photo?" 

I suppose it's possible that that is also subjective and is a question that doesn't need to be asked. (picture for reference)

[Ldina]

@hooty It's part reading/theory, and a LOT of doing. My goal in the Develop Persona is to 'eliminate problems' (blown out skies, plugged shadows, bad white balance, terrible brightness and contrast, etc) and deliver a file to Photo that is easy to edit. Here's a tutorial I wrote a while back that explains many of the tonal correction tools in both the Develop and Photo Personas. It's a start.

My overriding recommendation is this:

1. Use the Develop Persona to get your RAW file reasonably close and make any BIG moves there, if possible. Set white, black, highlights, shadows, white balance, etc and get them 'reasonably close'. Be sure to leave some headroom, especially on the white and black ends, avoid clipping, and don't go crazy with adding saturation or too much contrast. Once in Photo, it's easier to edit a slightly flatter, lower contrast image than it is to deal with excessive contrast/saturation/clipped highlights and shadows. That's my recommendation for the Develop Persona.

2. Do the rest in the Photo Persona. Do your fine tuning (and even some larger moves, if necessary), set your black and white points, fine-tune highlights and shadows, contrast, brightness, saturation, sharpening, etc. You can go back to the Develop Persona if needed by Developing your files using RAW Layer Linked, or RAW Layer Embedded. If you embed the RAW it will inflate your AfPhoto file sizes. I usually use RAW Layer Linked, since I keep all my RAW and AfPhoto files in the same folders. Linking results in MUCH smaller file sizes. If I am sharing an AfPhoto file with somebody else, which is rare, I will sometimes embed the RAW file so they have it available. 

Not everyone works this way, but I usually do. Some people like to do everything (or almost everything) in the Develop Persona. If I'm just banging out proofs, starting with RAW files, or don't want to spend too much time on an image, I occasionally do that, but I usually use steps 1 & 2 above.