Batch Processing Very Unreliable

[ashers]

Current implementation of while very promising but very beta.

Often getting failed processes, but once the queue is finished it disappears. So no way to work out which images processed correctly and which didn't.

Also there is no way to stop or pause the current queue without force quitting the application.

Is there a log file anywhere that shows why it failed? Would love to get this feature working properly.

J.

 

 

[carl123]

22 minutes ago, ashers said:

Is there a log file anywhere that shows why it failed? Would love to get this feature working properly.

No, I do not believe so

Assuming the same files fail every time then have a look in the batch file output folder to see which files are missing

Then have a look at those original files to see if you can see a pattern as to why they fail

 

If you can't find a reason then you can always upload a failing file here for further analysis

[ashers]

It's completely random, sometimes the images fail, sometimes they don't. I am not processing this in parallel.

"Assuming the same files fail every time then have a look in the batch file output folder to see which files are missing"

This is not a particularly practical solution for a batch process of 2500 images.

[carl123]

1 minute ago, ashers said:

This is not a particularly practical solution for a batch process of 2500 images.

No, but I was hoping you could analyse a few failed files to see if there was a pattern which could lead to a solution

But if as you say "It's completely random" then that would not work

[Chris B]

Hey ashers,

I have asked in the past if we can find a way to keep the dialog open (or open it at will) in order to find problematic files. 

If you dump the images into this folder and let me know exactly what your settings are (screenshot might be quicker) I can have a go at reproducing it and see if we can figure out what's going on.

https://www.dropbox.com/request/xJazCvOnjjPrCRIeCNWh

[Spaceman Spiff]

I'm running into the same issue.  I have ~2200 files (~50 gigs) that I'm trying to run through.  Eventually, Affinity Photo locks up.  I kill Affinity photo, restart it, requeue the files that haven't been processed yet, and restart the batch processing with the unprocessed files.  Eventually...it hangs again.

Details:

• Affinity Photo 1.8.3.641
• Parallel processing is OFF
• Batch process is simply to open an NEF (Nikon Raw), apply auto levels, auto contrast, auto colors, auto white balance, then export to jpeg resizing to 1/2 of the original size on its longest axis.  Each NEF is 5600x3728, 14 bit lossless compressed raw, typically 20-25 megs.  JPEG settings are resampling: Lanczos 3 Non-separable, quality 85, embed metadata is not checked.
• Typically somewhere between 200 and 300 files will complete before Affinity Photo becomes unresponsive and I need to end the task.
• Running Windows 10 Pro 64 bit, on an i9-9900K with 64 gigs of RAM and 1 gig of swap.

Additional information:

• crashpad_handler.exe seems to stop working simultaneously with Affinity Photo going down.

Description
Faulting Application Path:    C:\Program Files\Affinity\Photo\crashpad_handler.exe

Problem signature
Problem Event Name:    BEX64
Application Name:    crashpad_handler.exe
Application Version:    0.0.0.0
Application Timestamp:    5e8324cd
Fault Module Name:    ucrtbase.dll
Fault Module Version:    10.0.18362.815
Fault Module Timestamp:    32a6df9a
Exception Offset:    000000000006db9e
Exception Code:    c0000409
Exception Data:    0000000000000007
OS Version:    10.0.18363.2.0.0.256.48
Locale ID:    1033
Additional Information 1:    c9aa
Additional Information 2:    c9aa7345478c2e0fac34248b68b6b065
Additional Information 3:    4cc9
Additional Information 4:    4cc9b239491bd086e60de990da5864e2

 

Edited June 24, 2020 by Spaceman Spiff
Corrected word choice

[Chris B]

Hey Spaceman Spiff,

Welcome to the Affinity Forums.

I'm not about to ask you to upload ~50GB of files but I think a dump file would help. When Affinity locks up, can you go to Task Manager > Right-click Affinity > Create dump file and attach it?

I suspect it's a file that's causing the crash as AFAIK we impose no limits to number of files or size etc. 

[Spaceman Spiff]

Will do.  Although I just retried and this time it hard locked my computer.  At this point I'm beginning to suspect a driver or hardware problem, and Affinity may just be exposing the issue with how much memory/CPU/bus pressure it puts on the system when running through the batch processing.

[Spaceman Spiff]

Ran a batch of 1000 photos, failed around photo 960 this time.  Grabbed the dump, but it's 55 gigs (yikes!).  I've held onto it for now, but not sure if I want to try uploading that.

Did a second batch of 1000 photos, this time lowering the maximum RAM I allowed Photo to use to 32767 megs (out of 64 gigs).  This time all 1000 photos finished processing despite me forgetting and leaving the parallel processing box ticked.

I think that's consistent with a hardware problem (maybe RAM being flaky?), which makes it my problem and not Affinity's problem--and probably unrelated to the problems ashers is reporting.

Edited June 24, 2020 by Spaceman Spiff

[Chris B]

That's interesting. I plan on doing a few tests (not on the same scale as you as I simply do not have the space nor that much RAM). I would delete that dump to be honest, it's huge and if I can reproduce it on a smaller scale, my dump file will be more palatable! 

Thanks so much for updating me