Jump to content
ashers

Batch Processing Very Unreliable

Recommended Posts

Current implementation of while very promising but very beta.

Often getting failed processes, but once the queue is finished it disappears. So no way to work out which images processed correctly and which didn't.

Also there is no way to stop or pause the current queue without force quitting the application.

Is there a log file anywhere that shows why it failed? Would love to get this feature working properly.

J.

 

 

Share this post


Link to post
Share on other sites
22 minutes ago, ashers said:

Is there a log file anywhere that shows why it failed? Would love to get this feature working properly.

No, I do not believe so

Assuming the same files fail every time then have a look in the batch file output folder to see which files are missing

Then have a look at those original files to see if you can see a pattern as to why they fail

 

If you can't find a reason then you can always upload a failing file here for further analysis


Due to the fact that Boris Johnson is now our Prime Minister, punctuation, spelling and grammar will never be worried about ever again.  We now have far bigger problems to be worried about.

Share this post


Link to post
Share on other sites

It's completely random, sometimes the images fail, sometimes they don't. I am not processing this in parallel.

"Assuming the same files fail every time then have a look in the batch file output folder to see which files are missing"

This is not a particularly practical solution for a batch process of 2500 images.

Share this post


Link to post
Share on other sites
1 minute ago, ashers said:

This is not a particularly practical solution for a batch process of 2500 images.

No, but I was hoping you could analyse a few failed files to see if there was a pattern which could lead to a solution

But if as you say "It's completely random" then that would not work


Due to the fact that Boris Johnson is now our Prime Minister, punctuation, spelling and grammar will never be worried about ever again.  We now have far bigger problems to be worried about.

Share this post


Link to post
Share on other sites

Hey ashers,

I have asked in the past if we can find a way to keep the dialog open (or open it at will) in order to find problematic files. 

If you dump the images into this folder and let me know exactly what your settings are (screenshot might be quicker) I can have a go at reproducing it and see if we can figure out what's going on.

https://www.dropbox.com/request/xJazCvOnjjPrCRIeCNWh

Share this post


Link to post
Share on other sites
Posted (edited)

I'm running into the same issue.  I have ~2200 files (~50 gigs) that I'm trying to run through.  Eventually, Affinity Photo locks up.  I kill Affinity photo, restart it, requeue the files that haven't been processed yet, and restart the batch processing with the unprocessed files.  Eventually...it hangs again.

Details:

  • Affinity Photo 1.8.3.641
  • Parallel processing is OFF
  • Batch process is simply to open an NEF (Nikon Raw), apply auto levels, auto contrast, auto colors, auto white balance, then export to jpeg resizing to 1/2 of the original size on its longest axis.  Each NEF is 5600x3728, 14 bit lossless compressed raw, typically 20-25 megs.  JPEG settings are resampling: Lanczos 3 Non-separable, quality 85, embed metadata is not checked.
  • Typically somewhere between 200 and 300 files will complete before Affinity Photo becomes unresponsive and I need to end the task.
  • Running Windows 10 Pro 64 bit, on an i9-9900K with 64 gigs of RAM and 1 gig of swap.

Additional information:

  • crashpad_handler.exe seems to stop working simultaneously with Affinity Photo going down.

Description
Faulting Application Path:    C:\Program Files\Affinity\Photo\crashpad_handler.exe

Problem signature
Problem Event Name:    BEX64
Application Name:    crashpad_handler.exe
Application Version:    0.0.0.0
Application Timestamp:    5e8324cd
Fault Module Name:    ucrtbase.dll
Fault Module Version:    10.0.18362.815
Fault Module Timestamp:    32a6df9a
Exception Offset:    000000000006db9e
Exception Code:    c0000409
Exception Data:    0000000000000007
OS Version:    10.0.18363.2.0.0.256.48
Locale ID:    1033
Additional Information 1:    c9aa
Additional Information 2:    c9aa7345478c2e0fac34248b68b6b065
Additional Information 3:    4cc9
Additional Information 4:    4cc9b239491bd086e60de990da5864e2

 

Edited by Spaceman Spiff
Corrected word choice

Share this post


Link to post
Share on other sites

Hey Spaceman Spiff,

Welcome to the Affinity Forums.

I'm not about to ask you to upload ~50GB of files but I think a dump file would help. When Affinity locks up, can you go to Task Manager > Right-click Affinity > Create dump file and attach it?

I suspect it's a file that's causing the crash as AFAIK we impose no limits to number of files or size etc. 

Share this post


Link to post
Share on other sites

Will do.  Although I just retried and this time it hard locked my computer.  At this point I'm beginning to suspect a driver or hardware problem, and Affinity may just be exposing the issue with how much memory/CPU/bus pressure it puts on the system when running through the batch processing.

Share this post


Link to post
Share on other sites
Posted (edited)

Ran a batch of 1000 photos, failed around photo 960 this time.  Grabbed the dump, but it's 55 gigs (yikes!).  I've held onto it for now, but not sure if I want to try uploading that.

Did a second batch of 1000 photos, this time lowering the maximum RAM I allowed Photo to use to 32767 megs (out of 64 gigs).  This time all 1000 photos finished processing despite me forgetting and leaving the parallel processing box ticked.

I think that's consistent with a hardware problem (maybe RAM being flaky?), which makes it my problem and not Affinity's problem--and probably unrelated to the problems ashers is reporting.

Edited by Spaceman Spiff

Share this post


Link to post
Share on other sites

That's interesting. I plan on doing a few tests (not on the same scale as you as I simply do not have the space nor that much RAM). I would delete that dump to be honest, it's huge and if I can reproduce it on a smaller scale, my dump file will be more palatable! 

Thanks so much for updating me :) 

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


×
×
  • Create New...

Important Information

Please note the Annual Company Closure section in the Terms of Use. These are the Terms of Use you will be asked to agree to if you join the forum. | Privacy Policy | Guidelines | We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.