Jump to content
thomasp

Beta 122: Batch processing issues

Recommended Posts

Running a batch on file sequences I can observe these behaviours:

  • Running a batch sometimes - not always - produces zero-byte files (one or two out of around 200 images in my case). The exact files that ends up as zero bytes do not stay the same when I run the same batch several times over.
  • With the 'Parallel processing' option ticked there are usually several files missing on longer batches, also zero byte files may happen.

These issues have been in the beta for some time now and I believe at least the parallel processing leaving out files was brought up before.

 

Share this post


Link to post
Share on other sites

In Beta 125 parallel processing no longer omits files, yay!

However, zero-byte files are still a thing for me.

Share this post


Link to post
Share on other sites

Hi thomasp,
Can you provide a few more details about the batch job settings and original files you are processing? What format(s) are you starting with, what settings are you using in the Batch Job dialog (screenshot). I'm having trouble reproducing the zero-byte issue on a batch job with 528 files. Thank you.

Share this post


Link to post
Share on other sites

Certainly!

I'm seeing these issues when running batches on

  • -TGA files, typically 2k-4k video resolution out of 3D rendering packages, no alpha channel
  • -JPG files, up to 24 Mpix resolution, straight out of my DSLR (Pentax K3)

The actions I'm using are just simple rescale and export operations - these are typical settings:

ap_settings.thumb.jpg.fb7210b149bb382a620872c024cb66d7.jpg

 

I've just run with these settings multiple times over the same list of 400 TGA files in 2560x1440 (14,7 MB per file). Results -

  • 1st run: 400 files generated, 1 zero byte file: frame 211
  • 2nd run: 399 files generated (!), missing file: frame 260
  • 3rd run: 400 files generated, all ok
  • 4th run: 400 files generated, 2 zero byte files: frame 196, frame 255
  • 5th run: 400 files generated, 3 zero byte files: frame 66, frame 114, frame 185

This was on AP Beta 125, macOS 10.13.6, 6-core Mac Pro 6,1 (aka 'Trashcan'), 64 GB RAM. Source and target in this test was the internal NVMe drive (Samsung 970 Pro, APFS). I had the zero-bytes issue happen on external USB3 spinning disks (FAT) as well.

Share this post


Link to post
Share on other sites

Just discovered the same problem with my batch files. What the Hell!? I had already processed a bunch of files before I found this error, so I've lost a lot of work. >:(

Usually I would be missing 1 or 2 files when I would do a batch over 100 or so files. When I run the same batch different files would be missing. I did not get the zero byte file error.

Affinity Photo 1.6.7, macOS Sierra 10.12.6, 2.3 GHz Intel Core i7, 16 GB RAM

1.jpg

Share this post


Link to post
Share on other sites

I wonder if you found the time to investigate?

The behaviour is still there in release 1.7.2 - random missing files and/or creation of zero byte files when running with the batch settings described above.

 

Share this post


Link to post
Share on other sites

Hi thomasp,
Sorry the delay getting back to you. I did check it with more than 400 files (even from a Pentax K3) and other sources as well as 700 TGA's of different sizes (some duplicates) but I'm still unable to make it fail. A colleague also reported trouble replicating it with even more files. Do you mind uploading/share a folder with the whole set of 400 TGA's (2560x1440 ) files you are using to perform your tests so we can try reproduce this in similar conditions and with same sources?

We would like to get to the bottom of this issue. Thanks for your support.

Share this post


Link to post
Share on other sites

Hi MEB,

Thanks for your efforts thus far.

I've looked into this a bit more and it may be dependant on my setup - I have AP installed on my (NVMe->adaptor->Mac Pro 2013) system drive and I'm batch-outputting images to an external (modern, fast, FAT-formatted) USB3 spinning disk. After error checking filesystems and updating OS 10.13.6 to the latest security update I can not currently reproduce the zero-byte file creation issue.

Still testing to see if now all files are being written whenever I run a batch with parallel processing turned off and after that will check if parallel processing behaves. Perhaps it's an issue with AP not waiting around for a comparably slower output device during batch export? Testing it over a slow network might get some results, too.

What I can rule out is that it only affects files of a certain format or image dimension/file size threshhold.

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×

Important Information

These are the Terms of Use you will be asked to agree to if you join the forum. | Privacy Policy | Guidelines | We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.