Jump to content
You must now use your email address to sign in [click for more info] ×

RAW FILES - SOMETHING BIG IS COMING


Recommended Posts

...and yes, I did use IMPACT for the font

 

something big is coming.jpg

To save time I am currently using an automated AI to reply to some posts on this forum. If any of "my" posts are wrong or appear to be total b*ll*cks they are the ones generated by the AI. If correct they were probably mine. I apologise for any mistakes made by my AI - I'm sure it will improve with time.

Link to comment
Share on other sites

Me, not him, he's just more handsome, than me, so I used his picture

To save time I am currently using an automated AI to reply to some posts on this forum. If any of "my" posts are wrong or appear to be total b*ll*cks they are the ones generated by the AI. If correct they were probably mine. I apologise for any mistakes made by my AI - I'm sure it will improve with time.

Link to comment
Share on other sites

SOMETHING BIG HAPPENNED WITH RAW FILES

The big discovery with (non-destructive) RAW file Layers was that double-clicking on one in the Pixel Persona allowed a Macro being recorded to continue recording in the Develop Persona which has always been impossible. This could now open up a whole host of possibilities for using Macros with RAW files.

Detailed below is just one of them we tried - to batch process RAW files. I HAVE NO IDEA IF IT WORKS!!!

Note I don’t use RAW files (just a few test ones) so you will need to test this yourself to see if it is actually processing your RAW files correctly. It appears to work, for us, but I can’t tell for sure  

 

BATCH PROCESSING OF RAW FILES IN AFFINITY PHOTO

STEP 1: Capture RAW settings in a Macro
 

1. Open a RAW file
2. Make your adjustments
3. Select Embedded or Linked in the context Toolbar (NOT Pixel)
4. Develop image
5. Start Macro recording
6. Double click thumbnail of Layer (in Layers panel) to re-enter Develop persona
7. Make no changes, just Develop the image (again)
8. Macro panel should show just one entry called Develop
9. Stop Macro recording and save macro to Library studio (call it what you like)
10. 
Note that there is no need to save this Affinity document if you don’t want to. It is no longer needed.

 

STEP 2:  Batch process additional RAW files with same settings

  1. Start new batch job (File > New Batch Job)
  2. Select the RAW files to process & the output folder to save them in (Suggest you test this first with just, say, 5 to 10 RAW files)
  3. Select the file format to output results to. E.g. JPGs, TIFFs etc  - (See note 1 below)
  4. IMPORTANT: Select the macro you saved earlier and click the Apply button (Not the OK button)
  5. Tick or untick parallel processing (See Note: 2 below)
  6. Click OK to start the batch process

 

The Batch Job Studio Panel should pop up on your screen showing you the progress of each file so you can gauge how long it takes. If the tab is hidden you can move it to the front or enable the Batch studio if disabled. Doing this does not stop the batch processing.

Once completed you can evaluate the results.

 

Note 1: You can output to multiple file formats at the same time but it will take longer and use more disk space. If you also output to AFPhoto format your AFPhoto file will contain the non-destructive developed RAW file which can be tweaked if required (for that image). Be advised that AFPhoto files are big and will slow down processing so only output to that additional format if you have a specific need to.

Note 2: Parallel processing should be faster but some users (in the past) have reported crashing when using parallel processing. So, unticking this is safer but will take longer.

Note 3: RAW processing is slow. A Batch job is not going to speed up that part of the process; it just automates it for multiple files

Note 4: I only use Widows some things may not work at all on Macs or they may be different.

Note 5: It looks like the Batch job honours the File > Placement Policy setting for Embedded or Linked files if the output is to AFPhoto files. So, set this how you want it (I mostly tested with embedded). Note that this may also be settable within the Macro (not tested) or afterwards in Resource Manager.

Note 6: I have no idea if any Assistant Manager settings affects the results in the batch processing. This was not tested. Other than, Mac users must use Serif Labs raw engine not Apple Raw

Note 7: Not interested in any spelling errors or grammar mistakes in the above. Just if it works or not and if it does work how does the batch processing speeds compare to any other batch RAW processing software you may use (Parallel processing set and not set). But bear in mind there is an acknowledged bug currently with Developing RAW files taking longer in V2 than V1 (this is scheduled to be fixed)

Note 8: We were working on another way to batch process RAW files which was more complex but seemed to work. If the above does not work correctly we may resurrect that idea to see if it does.

To save time I am currently using an automated AI to reply to some posts on this forum. If any of "my" posts are wrong or appear to be total b*ll*cks they are the ones generated by the AI. If correct they were probably mine. I apologise for any mistakes made by my AI - I'm sure it will improve with time.

Link to comment
Share on other sites

Shouldn't this be in Tutorials?

-- Walt
Designer, Photo, and Publisher V1 and V2 at latest retail and beta releases
PC:
    Desktop:  Windows 11 Pro, version 23H2, 64GB memory, AMD Ryzen 9 5900 12-Core @ 3.00 GHz, NVIDIA GeForce RTX 3090 

    Laptop:  Windows 11 Pro, version 23H2, 32GB memory, Intel Core i7-10750H @ 2.60GHz, Intel UHD Graphics Comet Lake GT2 and NVIDIA GeForce RTX 3070 Laptop GPU.
iPad:  iPad Pro M1, 12.9": iPadOS 17.4.1, Apple Pencil 2, Magic Keyboard 
Mac:  2023 M2 MacBook Air 15", 16GB memory, macOS Sonoma 14.4.1

Link to comment
Share on other sites

On 11/18/2022 at 5:54 AM, carl123 said:

Widows

Small typo there

-- Walt
Designer, Photo, and Publisher V1 and V2 at latest retail and beta releases
PC:
    Desktop:  Windows 11 Pro, version 23H2, 64GB memory, AMD Ryzen 9 5900 12-Core @ 3.00 GHz, NVIDIA GeForce RTX 3090 

    Laptop:  Windows 11 Pro, version 23H2, 32GB memory, Intel Core i7-10750H @ 2.60GHz, Intel UHD Graphics Comet Lake GT2 and NVIDIA GeForce RTX 3070 Laptop GPU.
iPad:  iPad Pro M1, 12.9": iPadOS 17.4.1, Apple Pencil 2, Magic Keyboard 
Mac:  2023 M2 MacBook Air 15", 16GB memory, macOS Sonoma 14.4.1

Link to comment
Share on other sites

On 11/18/2022 at 11:54 AM, carl123 said:

Once completed you can evaluate the results.

Carl, this is actually the way to go! So all cudos to you.

But with my PC: Windows 10 64 bit and Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz   2.81 GHz with 32 GB RAM, it takes eons to process a decent stack batch of RAWs. I have tried both to save as jpeg and tiff and I have a pretty small quantity of RAWs (only 24 images). It takes time like if I am doing it manually in Photoshop image by image. There's no point in comparing to Photoshops image processor that handles it in a few minutes. This takes like half an hour.
I have tried out both parallell processing and without parallell processing and I don't see any big difference in the time, it takes around 54-55 s for each photo so imagine a batch of 100 photos is not so hesitating. It can still be worth it if you want to have a cup of coffee or take stroll outdoors while the computer works... but I must say I am pretty disappointed of the performance overall with AP

It's strange that it demands so much from the PC...

Link to comment
Share on other sites

  • Staff

@carl123 this is absolutely fantastic. I found out about this just after launch but simply have not found the time to write something as comprehensive as this. Dan C just made me aware of the post and I think we need to get this pinned somewhere.

Link to comment
Share on other sites

@Chris B

Sure, you can just copy the relevant parts, rework it as required on a new post and place it wherever you think best.

To save time I am currently using an automated AI to reply to some posts on this forum. If any of "my" posts are wrong or appear to be total b*ll*cks they are the ones generated by the AI. If correct they were probably mine. I apologise for any mistakes made by my AI - I'm sure it will improve with time.

Link to comment
Share on other sites

19 hours ago, LAMF64 said:

Carl, this is actually the way to go! So all cudos to you.

But with my PC: Windows 10 64 bit and Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz   2.81 GHz with 32 GB RAM, it takes eons to process a decent stack batch of RAWs. I have tried both to save as jpeg and tiff and I have a pretty small quantity of RAWs (only 24 images). It takes time like if I am doing it manually in Photoshop image by image. There's no point in comparing to Photoshops image processor that handles it in a few minutes. This takes like half an hour.
I have tried out both parallell processing and without parallell processing and I don't see any big difference in the time, it takes around 54-55 s for each photo so imagine a batch of 100 photos is not so hesitating. It can still be worth it if you want to have a cup of coffee or take stroll outdoors while the computer works... but I must say I am pretty disappointed of the performance overall with AP

It's strange that it demands so much from the PC...

This morning I ran some tests with 11 RAW files (using different drives and settings) nothing improved too much with the timing of the batch job but one interesting thing I encountered was that with Parallel processing active my total time was 3 minutes 47 second which was 27 seconds slower than with Parallel processing off.

I think the reason for this was because I was maximising out my available RAM (16GB) and the task was having to swap to disk

Still testing other stuff

To save time I am currently using an automated AI to reply to some posts on this forum. If any of "my" posts are wrong or appear to be total b*ll*cks they are the ones generated by the AI. If correct they were probably mine. I apologise for any mistakes made by my AI - I'm sure it will improve with time.

Link to comment
Share on other sites

1 hour ago, carl123 said:

I encountered was that with Parallel processing active my total time was 3 minutes 47 second which was 27 seconds slower than with Parallel processing off.

I think the reason for this was because I was maximising out my available RAM (16GB) and the task was having to swap to disk

Still testing other stuff

I have also tested both parallell on and off and found out it might be a wee bit faster with it turned off. 

It owuld of course be a great thing if it could do the job faster with RAWs as well, as it is of a decent good speed when using macros without RAWs involved. Still this function is great and makes the software usable and not necessary any more with Adobe PS which is superb. I'm gonna go digging deeper now andsee what else it has hidden for me to use.

Link to comment
Share on other sites

  • 1 month later...

Hi All

I just tried the procedure and I must be doing something wrong

When I double click the developed embedded layer after starting the macro recording I get the attached window. By selecting the develop persona button at the top of the window the develop module activates. Once it is open and the macro is running I select the basic, lens and details presets and then press develop. Once in the photo window I stop the macro and save it in Library.

The next issue I have yet to solve is to get the develop macro to show in the available macros window of the Batch Processing panel. It is currently showing the macro categories I have and not the actual macros in the library. Do you know how to see the just the applicable macros to select from?

Please help, What AM I doing wrong?

screenshot_56.png

screenshot_59.png

Link to comment
Share on other sites

Hi Again

You're Right it doesn't work as described above. I create the macro as I have described perviously but when I ope Batch processing window the the avaialble Macros box only shows the categories not the actual macros, So I can't apply the newly created Develop macro.

Any suggestions how I can show the macros and not the categories?

Thanks,

Link to comment
Share on other sites

1 hour ago, spiderpod7d said:

When I double click the developed embedded layer after starting the macro recording I get the attached window. By selecting the develop persona button at the top of the window the develop module activates. Once it is open and the macro is running I select the basic, lens and details presets and then press develop. Once in the photo window I stop the macro and save it in Library.

It sounds like you skipped something at the beginning. Note that you start by opening the RAW file, making your adjustments, and clicking Develop. Once you've done that, you start recording the macro, and double-click on the RAW layer's thumbnail to get into the Develop Persona again. You do not make any adjustments at that point, as you made them all previously. You just click Develop, then stop recording.

1 hour ago, spiderpod7d said:

The next issue I have yet to solve is to get the develop macro to show in the available macros window of the Batch Processing panel. It is currently showing the macro categories I have and not the actual macros in the library. Do you know how to see the just the applicable macros to select from?

Once you've selected a category, the macros in that category should be displayed. Perhaps you didn't save your macro in the Library category you think you did? Can you find it in the Library panel?

-- Walt
Designer, Photo, and Publisher V1 and V2 at latest retail and beta releases
PC:
    Desktop:  Windows 11 Pro, version 23H2, 64GB memory, AMD Ryzen 9 5900 12-Core @ 3.00 GHz, NVIDIA GeForce RTX 3090 

    Laptop:  Windows 11 Pro, version 23H2, 32GB memory, Intel Core i7-10750H @ 2.60GHz, Intel UHD Graphics Comet Lake GT2 and NVIDIA GeForce RTX 3070 Laptop GPU.
iPad:  iPad Pro M1, 12.9": iPadOS 17.4.1, Apple Pencil 2, Magic Keyboard 
Mac:  2023 M2 MacBook Air 15", 16GB memory, macOS Sonoma 14.4.1

Link to comment
Share on other sites

Hi Walt

Yes files are saved in correct category. Even tried saving it in Default library folder.

When in Batch persona Available Macros window Only shows Library categories not actual Macros as per previous image I sent. Can't select the contents of the shown category. Obviously a bug in MAC

I will report it but a fix will have to wait til new year I guess.

Screen Shot 2022-12-22 at 5.01.10 PM (2).png

Link to comment
Share on other sites

On 11/18/2022 at 2:54 AM, carl123 said:

SOMETHING BIG HAPPENNED WITH RAW FILES

The big discovery with (non-destructive) RAW file Layers was that double-clicking on one in the Pixel Persona allowed a Macro being recorded to continue recording in the Develop Persona which has always been impossible. This could now open up a whole host of possibilities for using Macros with RAW files.

Detailed below is just one of them we tried - to batch process RAW files. I HAVE NO IDEA IF IT WORKS!!!

Note I don’t use RAW files (just a few test ones) so you will need to test this yourself to see if it is actually processing your RAW files correctly. It appears to work, for us, but I can’t tell for sure  

 

BATCH PROCESSING OF RAW FILES IN AFFINITY PHOTO

STEP 1: Capture RAW settings in a Macro
 

1. Open a RAW file
2. Make your adjustments
3. Select Embedded or Linked in the context Toolbar (NOT Pixel)
4. Develop image
5. Start Macro recording
6. Double click thumbnail of Layer (in Layers panel) to re-enter Develop persona
7. Make no changes, just Develop the image (again)
8. Macro panel should show just one entry called Develop
9. Stop Macro recording and save macro to Library studio (call it what you like)
10. 
Note that there is no need to save this Affinity document if you don’t want to. It is no longer needed.

 

STEP 2:  Batch process additional RAW files with same settings

  1. Start new batch job (File > New Batch Job)
  2. Select the RAW files to process & the output folder to save them in (Suggest you test this first with just, say, 5 to 10 RAW files)
  3. Select the file format to output results to. E.g. JPGs, TIFFs etc  - (See note 1 below)
  4. IMPORTANT: Select the macro you saved earlier and click the Apply button (Not the OK button)
  5. Tick or untick parallel processing (See Note: 2 below)
  6. Click OK to start the batch process

 

The Batch Job Studio Panel should pop up on your screen showing you the progress of each file so you can gauge how long it takes. If the tab is hidden you can move it to the front or enable the Batch studio if disabled. Doing this does not stop the batch processing.

Once completed you can evaluate the results.

 

 

Well I've tried several workarounds for batch processing raw files for use in Panos and focus merges but this method is outstanding.  Thank you and well done!

Link to comment
Share on other sites

  • 4 weeks later...

carl123 this is great and really works - thanks a lot!

One observation from me though - I've the Placement Policy set to "linked" and same thing I choose in Develop persona when recording the macro but everytime I run the batch process on set of RAW files, it always does them as "Embedded". It may not be a thing for someone but for me it makes more sense to rather have super small .afphoto file and NEF of size around 30MB vs .afphoto of size 150MB+ when the RAW file is embedded.

Link to comment
Share on other sites

7 hours ago, ghostek said:

carl123 this is great and really works - thanks a lot!

One observation from me though - I've the Placement Policy set to "linked" and same thing I choose in Develop persona when recording the macro but everytime I run the batch process on set of RAW files, it always does them as "Embedded". It may not be a thing for someone but for me it makes more sense to rather have super small .afphoto file and NEF of size around 30MB vs .afphoto of size 150MB+ when the RAW file is embedded.

I am seeing the same (not sure if it changed n 2.0.3 or I did not test that initially)

I can get around the issue by once I have started the macro and developed the image (using embedded as the output) I go into Resource Manager and select Make Linked then save the macro.

Note: If you selected Linked as the output and then developed the image, Make Linked will be grayed out in Resource Manager so you need to first select Embed then Make Linked which also seems to work.

 

Unfortunately, I can't seem to find a way to batch process and change existing .afphoto files from Embedded to Linked (overwriting the original file) but this may be due to a bug/limitation, which I will investigate further

 

To save time I am currently using an automated AI to reply to some posts on this forum. If any of "my" posts are wrong or appear to be total b*ll*cks they are the ones generated by the AI. If correct they were probably mine. I apologise for any mistakes made by my AI - I'm sure it will improve with time.

Link to comment
Share on other sites

11 hours ago, carl123 said:

Unfortunately, I can't seem to find a way to batch process and change existing .afphoto files from Embedded to Linked (overwriting the original file) but this may be due to a bug/limitation, which I will investigate further

 

Well, you can batch process and change existing .afphoto files from Embedded to Linked using a macro but not to the same directory (i.e. overwriting the original .afphoto file). You have to specify a different folder for the output files.

This actually has nothing to do with developing RAW files & macros.

Any .afphoto file cannot overwrite itself - even using one of the built-in macros, like the "Black & White" macro when batch processing

I initially thought this had to be a bug but after testing on an old copy of AP (1.8.5) it looks like it's always been this way.

So maybe it's just a limitation?

(Windows 11)

Is this the same on Macs? 

 

PS Crazy fact number two, you can't' even batch process .afphoto files, to the same folder, just to change their size (W & H) (i.e. no macros involved) But it works when outputting to a different folder !!!

 

To save time I am currently using an automated AI to reply to some posts on this forum. If any of "my" posts are wrong or appear to be total b*ll*cks they are the ones generated by the AI. If correct they were probably mine. I apologise for any mistakes made by my AI - I'm sure it will improve with time.

Link to comment
Share on other sites

4 minutes ago, carl123 said:

Any .afphoto file cannot overwrite itself - even using one of the built-in macros, like the "Black & White" macro when batch processing

I initially thought this had to be a bug but after testing on an old copy of AP (1.8.5) it looks like it's always been this way.

So maybe it's just a limitation? Is this the same on Macs? 

It is the same on Mac and I am of the opinion that it is a bug.

Mac Pro (Late 2013) Mac OS 12.7.4 
Affinity Designer 2.4.0 | Affinity Photo 2.4.0 | Affinity Publisher 2.4.0 | Beta versions as they appear.

I have never mastered color management, period, so I cannot help with that.

Link to comment
Share on other sites

  • Staff
2 hours ago, carl123 said:

Any .afphoto file cannot overwrite itself - even using one of the built-in macros, like the "Black & White" macro when batch processing

I initially thought this had to be a bug but after testing on an old copy of AP (1.8.5) it looks like it's always been this way.

So maybe it's just a limitation?

I can confirm this is a bug, which has been recently logged with the team - I'll be sure to 'bump' that log with this thread now :)

2 hours ago, carl123 said:

PS Crazy fact number two, you can't' even batch process .afphoto files, to the same folder, just to change their size (W & H) (i.e. no macros involved) !!!

I believe this is a continuation of the above issue, as this works for me when changing file dimensions with no macro applied to a new location, the known bug is that the Batch Processing dialog simply can't overwrite .af[photo/design/pub] files in the same location, regardless of batch settings.

21 hours ago, ghostek said:

I've the Placement Policy set to "linked" and same thing I choose in Develop persona when recording the macro but everytime I run the batch process on set of RAW files, it always does them as "Embedded".

However the above report doesn't appear to already be logged and I would consider this a bug, so I'll get this logged now - many thanks!

Please Note: I am now out of the office until Tuesday 2nd April on annual leave.

If you require urgent assistance, please create a new thread and a member of our team will be sure to assist asap.

Many thanks :)

Link to comment
Share on other sites

  • 4 months later...

OK, that method worked for me, thanks. This method is very unintuitive though, feels somewhat hacky. Why not just have makros available directly in the develop persona when you open the raw file? Everyone would get that right away.

So do I understand correctly that this awkward method of batch processing raw files is meant to be "THE BIG THING" for 2.0?

EDIT:
Wait, no. So the ability to do "non-destructive" raw editing is "THE BIG THING", right? Being able to make makros from it simply derives from that.

Link to comment
Share on other sites

  • 6 months later...
Hello
Affinty was probably busy: 
the development time for RAW, which was criticized, has now shrunk significantly (using Affinity 2.3). 
For me, the development of 5 Raw (Sony ARW 20MPixel) only took a total of 1-2 minutes on a 12 year old PC i7, 16GB, at least Nvidia RTX 2060super). 

Sounds good to me.

EW

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...

Important Information

Terms of Use | Privacy Policy | Guidelines | We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.