How to Simulate a Bayer Matrix

[Didge]

Hi,

I'm doing a uni project on image quality, and one of the topics relates to Bayer Matrices.

I'm hoping to use Affinity Photo to produce a simulated Bayer matrix from a standard RGB image - that is, to separate the original image into 3 layers (R,G,B), where each layer has the opposing pixels deleted - just like a normal RGGB image.

I've been exploring Procedural Textures for this, but can't see any way to selectively delete individual pixels (or more correctly, to set their brightness to zero) in a matrix pattern.

Can anyone suggest a way to do this ?

Many thanks

[RichardMH]

I think you want dcraw. What I would try anyway.

[NotMyFault]

you can use these formulas to selectively show pixels based on x/y position within document.

Activate only one of the 3 formulas (for alpha channel as target) per color-separated layer:

 

[NotMyFault]

basic workflow to create a bayer pattern image from a source image:

1. open source image
2. Resize (scale-up) source image by factor 2 using nearest neighbour 
3. nest PT filter (in masking position) to source image
4. duplicate parent layer 2 times
5. activate 1st, 2nd, 3rd formula to get R/G/B colors moved into bayer pattern (visible when zoomed 800% or larger
6. for top layer, use 0 as formula for G and B channel
7. for middle layer, use 0 as formula for R and B channel
8. for lower layer, use 0 as formula for R and G channel
9. May add curves adjustment to correct brightness of color channels

 

[Didge]

Hi NMF,

Thanks so much for your post - that was extremely helpful.

I've used your suggestion of osci filters to create 3 Bayer masks - that was actually very straightforward.

Then I used channel filters to separate the original image into 3 layers for RGB, applied the Bayer masks to the respective channel, used a procedural filter to remove the colour information from each layer, added the layers together to create a faked OSC RGGB image, saved that as TIFF then converted that to FITS.

Then I opened the FITS file using AP, set the deBayering method to RGGB and bingo !

Even at full resolution, the reconstituted image is very difficult to distinguish from the original image, which was rather surprising given that 75% of the R&B channels and 50% of the G channel is thrown away during the process.

Example attached - from a HST image of the Pillars of Creation in the Eagle Nebula.

Its a shame AP doesn't support different delayering methods, so I cant test it using anything other than Bilinear.

The only remaining problem is that the Bayer masks don't go all the way to the edge of the image ...
as in the pic below, for some reason it refuses to go to the last 3 columns and the last 2 rows - if you have any ideas on how to fix that ?

Thanks again for your help - the osci filters certainly did the trick !

Didge

[NotMyFault]

1 hour ago, Didge said:

Hi NMF,

Thanks so much for your post - that was extremely helpful.

Glad that it helped. My pleasure.

 

[NotMyFault]

2 hours ago, Didge said:

Example attached - from a HST image of the Pillars of Creation in the Eagle Nebula.

Its a shame AP doesn't support different delayering methods, so I cant test it using anything other than Bilinear.

The only remaining problem is that the Bayer masks don't go all the way to the edge of the image ...
as in the pic below, for some reason it refuses to go to the last 3 columns and the last 2 rows - if you have any ideas on how to fix that ?

Thanks again for your help - the osci filters certainly did the trick !

 

Great image.

regarding edge pixels:

• what is the exact size of the image before / after duplication?
• the PT filter has an inherent masks. This should be „all white“.

I assume all layers don’t use any inherent mask, or separate mask layers. The PT filter then should work on all pixels. Check with move tool active, inspect transform panel. All values must be grey/zero.

If possible, can you upload the actual afphoto file so I can inspect?

[NotMyFault]

attached you find my "single PT filter" version to create a simulated bayer pattern from any image (upscaled 2x).

The PT filter formula does all steps at once (color separation, filtering to RGGB pattern)

• m1 is the "masking" vector (0 or 1, for each color channel, depending on x/y position)
• pp is the old color values at pixel position. green reduced to 1/2 to compensate for double green pixels in bayer pattern
• lerp is used as workaround to multiply both vectors

var m1=vec3(osci(x)*osci(y),osci(x+1,y),osci(x+1)*osci(y+1)); var pp=vec3(R,G/2,B);lerp(vec3(0),m1,pp)

The test document uses RGB/8, leading to some loss of lower bits color information and color banding when de-bayering. I added a group which simply uses gaussian blur to delayer, and levels adjustment to re-gain original brightness.

 

Edit: the best way to de-bayer is to merge visible, the Filter->Distortion->pixelate with radius of 2. This gives back the original image, when using RGB/16 to avoid loss of precision.

bayer pattern V2.afphoto

Edited March 8, 2023 by NotMyFault

[Didge]

22 hours ago, NotMyFault said:

Great image.

regarding edge pixels:

• what is the exact size of the image before / after duplication?
• the PT filter has an inherent masks. This should be „all white“.

I assume all layers don’t use any inherent mask, or separate mask layers. The PT filter then should work on all pixels. Check with move tool active, inspect transform panel. All values must be grey/zero.

If possible, can you upload the actual afphoto file so I can inspect?

Hi NMF,

Thanks for your help again

The original image is 6,248 x 4,172, the same size before / after duplication.

Interestingly, the aspect ratio of the image is 3:2 ...
The same as the number of empty columns/rows on the right/lower margin of the image - 3 columns and 2 rows - probably related.

I haven't applied any masks to the PT filters, so they should be all white.

The AP file uses linked images, so it probably won't work unless you can link it to an image of the same resolution - but here it is attached.
FYI - this was edited using AP 2Beta 2.1.0.1713, so might not open in a lower version.

Any advice greatly appreciated:)

Also, 3 images attached - original, layered + reconstituted using bilinear interpolation.

I'm actually amazed at how good it is!

Gary

RGB Experiment 4 - Bayered.afphoto

NGC 3324 @ 6,248 x 4,176.tiff RGB Experiment 4 - Bayered.tiff RGB Experiment 4 - deBayered.tiff

[NotMyFault]

Hi,

 

both the image inside the group, and the group itself have been resized or moved.

My advice is to inspect all these parameters immediately after placing any image / layer, and lock the layers in layer stack. This protects against unintended movement. Set the visible digits to max (6) in preferences, otherwise values get rounded.

It is unclear at what editing action the layer position / size got wrong. Normally this happens when having the move tool active.

 

 

[Didge]

Hi NMF,

Thanks for that info - very helpful !

I suspect there is something else going on though ...

I initially placed the background image very carefully - making certain the 1st pixel was in the top left corner.
I made the background image exactly the same size as the canvas - 6,248 x 4,176 - so I didn't need to resize it.

If you look at the image at the top and turn all the bayered layers OFF, then the image fills the entire canvas, corner to corner.

BUT - if you select any of the Bayered layers and turn OFF all the PT and channel layers, the embedded image fills the entire canvas except for the right-most 3 columns, and the bottom 2 rows.

Here's the interesting bit ... something I just tried ...
If you switch ALL the layers in a set to OFF, the background is entirely black, except for the right-most 3 columns, and the bottom 2 rows - which are white.

It looks like my initial mask was missing those rows/columns - I filled those to black, now everything is OK !!

Thanks again for your help and suggestions - they really helped a lot