Pixelated alpha on groups // Octane EXR

[tobecg]

Hey everyone,

thanks to the great OCIO support and the performance, I´m using affinty photo more and more to process my still linear exr renderings out of octane for cinema 4D. Unfortunately I'm having an issue with alpha masking of groups with this workflow. These are the steps I take:

1. I'm rendering a half float linear exr and use the "object layer color" for alpha masking [because it's faster to render inside of octane].

2. I take the exr into affinty [1.8.0.585]

3. Use OCIO to convert the Image to sRGB

4. Below that, I adjust my exposure if necessary

5. On Top, I have a LUT to convert the Image to sRGB ICC

5. I right-click my black and white (but still RGB) object layer color and rasterize to mask

6. I drag a drop the mask on my grouped beauty [regular beauty and denoised beauty]

7. As a result, the mask is very pixelated and unusable.

Masking a group seems to cause the problems. Without the group everything works with and without OCIO. The same mask and grouping works fine in Photoshop. The Photoshop files looks different because its without OCIO but adjusted exposure, otherwise it would be way too bright.

Thanks in advance!

 

[Chris B]

Hey tobecg, welcome to the Affinity Forums! 

Could you upload the original EXR and your saved afphoto file (please save this with its history) to this private Dropbox folder please:
https://www.dropbox.com/request/9t3nl4hcdZqSEoWBQhff

[tobecg]

Hi Chris,

thank you and thanks for the reply!

I uploaded the EXR, affinity file and the LUT into the dropbox. I had this problem on all my EXRs so far, unfortunately I can`t share the others because of NDA but let me know if you need another example.

 

[James Ritson]

Hi @tobecg, I've had a look at the files you provided to try and puzzle out what's happening. I can't replicate your exact setup as I'm not sure which OCIO configuration you're using? Unless you're using a filmic transform from a blender configuration?

It looks like there are two separate issues working in tandem, both related to groups and how group nodes render. You've got a mask on your beauty pass group, and also a separate group of adjustment layers. This is what was throwing me, because with your workflow both of these are contributing to the rendering issue.

There are a couple of quick workarounds you can try whilst we investigate this further: the first is to simply drag your OCIO, Lut, Gain group and clip it into the Beauty group—so drag and offer it to the text (not the thumbnail) of the Beauty group. In terms of node positioning, the mask in the Beauty group will now render above the adjustments, so all should be OK there.

Alternatively, Ctrl/CMD-select both the OCIO, Lut, Gain and Beauty groups and group them together, then drag the mask out from the Beauty group and offer it to the thumbnail of your newly-created master group. This will basically achieve the same thing as the above approach, but just means you can have child groups and stay more organised.

It should look like this:

I've tried reproducing the issue with my own renders, and I can see a distinct difference in alpha rendering when the mask is rendering on a group as opposed to a pixel layer. I have however been unable to reproduce the pixellation you're experiencing. I'm not familiar with Octane—is it very expensive for render time to create a straight alpha pass? I can't help but wonder about the object colour pass and whether it's intended for use with alpha masking, because it seems to contain erroneous pixel values. For example, if you open the original EXR document and push the exposure of the object colour pass, you will see this:

Whereas if we look at the same Exposure adjustment on a genuine straight alpha pass we will see:

 

I'm certainly not trying to absolve Affinity Photo of responsibility here 😉 there's definitely an issue with group nodes that needs looking into, and thank you for bringing it to our attention. Would it be worth trying a straight alpha pass on your end however to see if the issue can be mitigated that way?

Also, I noticed with your document file that you've chosen Unmanaged in the 32-bit preview panel, so you're seeing scene-referred linear values rather than managed display referred values. It's worth asking what your intended delivery is: are you planning to export to nonlinear 8-bit/16-bit delivery formats? If so, you have to use ICC Display Transform otherwise you'll get a nasty shock when you look at your exported images. ICC Display Transform exists so you can accurately preview what a linear 32-bit document will look like when exported to a non-linear, display-referred delivery format.

This causes an issue if you try and use certain OCIO transforms. Case in point, blender's Filmic transform, which is designed to go from linear to nonlinear gamma. You effectively double-up the nonlinear gamma transform which makes the resulting image look too bright and washed out. An easy solution to this is to add a live Procedural Texture filter (Layers>New Live Filter Layer>Colour>Procedural Texture). Click the plus icon three times to add red, green and blue channel entries, then raise each channel value to the power of 2.2 (linear transform) like so:

Given that most OCIO transforms (that I know of) are bounded and so will clip values outside 0-1, you will probably want to put this Procedural Texture filter underneath your OCIO adjustment layer.

If you're using Affinity Photo as an intermediary in a compositing pipeline (and so will be exporting back to EXR), you'll want to use the OCIO transform option instead of ICC Display Transform.

Hope the above helps!

[tobecg]

Hi James,

thank you for looking into this and the detailed write-up. The workaround works fine. It also worked by assigning the mask to each layer but that's not very efficient. So thanks for the workaround!

I'm fairly new to Affinity and all the OCIO and ACES workflow. I'm working in Cinema 4D with Octane and I usually produce more animations than stills, which worked pretty good so far with linear. Stills however are different because of the ICC system.

Regarding the object layer color: it can be expensive to render out straight alphas for animations but I also got to say that I don't use the object color layer workaround “correct” here, but so far it always worked in PS. If you're interested in this workaround, there is a talk from Drew Nelson who explains this workflow [at 10:10]:

For stills however it's absolutely manageable to render out clean alphas and I will just do this now or probably switch to cryptomatte in general.

With the exr stills I want to completely finish my frame in Affinity so the output will be 8/16 bit. I usually work with the ICC display transform turned on. I quickly rebuild the image in an extra Affinity file as I didn't save the history before, so I probably just didn't pay attention to this. But thanks for the explanation and the procedural texture solution!

My workflow with this rendering was using the OCIO with source color: ACES — ACEScg, output — sRGB and the LUT is used to convert the image from linear to ICC srgb, so I can assign the sRGB IEC61966-2.1 (Linear) ICC Profile and get a correct export. But as I'm said, I'm new to this so its totally possible that I'm also making a mistake here. So far, the final export looked like it should. 🤔

 

Thanks for all the help!

[James Ritson]

On 3/13/2020 at 12:03 AM, tobecg said:

My workflow with this rendering was using the OCIO with source color: ACES — ACEScg, output — sRGB and the LUT is used to convert the image from linear to ICC srgb, so I can assign the sRGB IEC61966-2.1 (Linear) ICC Profile and get a correct export. But as I'm said, I'm new to this so its totally possible that I'm also making a mistake here. So far, the final export looked like it should. 🤔

 

Thanks for all the help!

Hi again @tobecg, a couple of suggestions based on what you've said here:

When you say using the OCIO with source colour, are you referring to the OCIO adjustment layer (and not the OCIO transform on the 32-bit preview panel)? If so, one thing you should be aware of is that EXR documents are converted to scene linear upon import. Since EXR doesn't have the concept of tagged colour spaces, you can specify an input colour space by affixing the document file name with a valid colour space name from the configuration.

For example, if you are using the ACES OCIO configuration and you know that your linear colour values in the EXR file are in ACES colour space, you can rename your file to "filename aces.exr". When you import it into Affinity Photo, you'll get a toast in the top right saying it has converted from that ACES colour space to scene linear—the colour values in your document should now look correct without having to do that initial OCIO adjustment layer transform.

However, if it's too late for that, you could also achieve the same transform by adding your OCIO adjustment layer and on Source Colour Space choosing "ACES - ACES2065-1". Leave the Destination Colour Space as "role_scene_linear" since that's what you want to be converting to.

You could then theoretically add another OCIO adjustment layer going from "role_scene_linear" to "out_srgb", which I think is what you were referring to in your post? That would give your document a gamma-corrected nonlinear appearance.

A further complication is that in Affinity Photo EXR documents are (by default) given a linear sRGB colour profile, since Affinity Photo needs something to colour manage with. If you are working in Unmanaged (linear) or OCIO transform modes then this colour profile is completely arbitrary. It does however factor in if you use ICC Display Transform, since the colour values will be bounded to sRGB space during the document to screen profile conversion. Colour values outside sRGB are still there since you're working in unbounded floats, but I believe you won't see them as long as your document profile is linear sRGB.

Hope that helps!

[Edit] Forgot to add—I don't think this will affect you since you've mentioned there's no need to export back to EXR, but you can also affix the export filename with a colour space and Affinity Photo will convert colour values from scene linear to that colour space during export (e.g. "export filename acescg.exr"). Useful if you're taking your EXR into other software and need it to be in a certain colour space.

[tobecg]

Hi James,

yes, I was talking about the OCIO adjustment layer. Thanks again for the explanation, very interesting and very useful especially with the naming convention!