Batch processing with Parallel processing Causing infinite hang.

[ROfu]

Hey folks, 

So I've been using the batch job to simply convert a few raw files to tiffs - only option ticked is "convert to tiff"

if i choose more than 10 files at a time to batch, the software gets stuck saying saving tiff.. but not actually doing anything. 

If i deselect parallel processing it all works fine. -  This seems like your parallel processing is getting stuck somewhere.

For help with any debugs im running a Ryzen 3970x (incase its a thread count issue). 

[Chris B]

Hey ROfu,

We are seeing a pattern with these Ryzen CPU's with them either crashing or freezing during batch jobs, panorama stitching etc.

Could you please go to Preferences > Performance and change the Renderer from your GPU to WARP. You didn't mention which GPU you are using but I am assuming you have a dedicated GPU. 

This is not a fix but a workaround. WARP will generally be slower for general day-to-day processing of your work but I'd like to see what happens when we switch to WARP.

Also, if you can provide a dump file that would be fab! 

[ROfu]

Hey there, Same thing using WARP - so definitely linked to the CPU.

For reference the GPU is a Nvidia 2080ti

With regards to a dump file - if you can tell me where this is then i can find it (if were talking about the error code for it failing, then it doesn't fail it just never completes so i need to kill the task in the task manager

[walt.farrell]

4 minutes ago, ROfu said:

With regards to a dump file - if you can tell me where this is then i can find it (if were talking about the error code for it failing, then it doesn't fail it just never completes so i need to kill the task in the task manager

Rather than killing the task, if you right-click on it in the Task Manager one of your choices will be to create a dump file. If you do that it will also give you a popup telling you what file it created and where it put it.

I'm not sure if you'll get a .dmp from an Affinity application if you Kill it. You might, but I'm only familiar with getting them when Affinity crashes.

[ROfu]

excuse the Windows ignorance im used to Linux and MacOS.

So sadly the dump file is huge, so by the looks of it, it dumps the entire application state from memory - so the file is 15gb 

Is this expected? 

[walt.farrell]

1 hour ago, ROfu said:

Is this expected? 

I don't have enough experience to say; sorry.

You can also look for the dump file that Photo might have created. Press the Windows key and R, and then type the following into the box and press OK: %AppData%\Affinity\Photo\1.0\CrashReports\Reports

[ROfu]

Sadly no files exist .. as its never properly crashed only been killed manually. cheers for the suggestions. 

[Chris B]

Hey ROFu,

You can try uploading the dump file here:
https://www.dropbox.com/request/HW6OjJaqXTJXJON63I68

I can ask a developer to look at it once it has completed. Please give me a nudge once it uploads. Fingers crossed you have no issues uploading it. Thank you! 

[ROfu]

Hi Chris,

Like i mentioned to Wait, the file is 15gb, uploading that would take a while. 

Do you have any other suggestions or files i could send that are a little more manageable.

 

[Chris B]

Hey ROfu,

The large file size will be directly related to how much memory is in use when you get the hang. You can enable crash dumps but I don't know how daunting or how much time you're prepared to spend setting it up.

https://helpx.adobe.com/in/xd/kb/how-to-generate-crash-dump-on-windows-machine.html

We suspect it is directly related to the amount of cores as we've seen a few similar reports with i9 processors.

We're currently discussing this as I suspect we will need to get a threadripper to reproduce.

[HeinzG]

Hey I would like to bring this back up, I'm running into the exact same issue. Funny enough on a very equatable system, 2080ti and a 3960x in my case.

Another observation:

1) CPU goes to ~0
2) memory stays high.
3) In the batch viewer all tasks are noted as "Saving AFPhoto"
4) It repeatedly only happens with more then 11 itesm (as in batches of 0,1,2,3,4,5,6,7,8,9,10,11 always work, batches of 12 or more always hang)
5) It happens with both WARP and GPU rendering
6) The task group includes 2 executables:

* Affinity Photo
* crashpad_handler.exe

Given the issue is 3 month old some fix or even workaround would be very appriciated.

 

Note: I got a 14GB (compressed) dump file I'm happy to upload.

Edited October 31, 2020 by HeinzG

[Chris B]

Hey HeinzG, 

If you want to upload the dump, I've made you a Dropbox link:
https://www.dropbox.com/request/iouUaFRpZvFSPHvXkKI6

Can you confirm if this happens with Paralell Processing disabled?

[HeinzG]

Hi Chris,

thanks for the quick feedback! I'll upload the data and verify if non paralell processing works this evening.

 

Cheers,
Heinz

[HeinzG]

Hi,

the file should be uploaded.

Edit: I also ran the entire batch job (75 photos) in non paralell and it finished without problems.

Edited November 2, 2020 by HeinzG

[Chris B]

Thank you. I've passed the dump file over to the developers.

I would for now leave Parallel Processing disabled. In theory it is supposed to be more efficient on new/high-end machines like yours. Leaving it off will not affect the finished job, it might just take longer than it should if it was working.

[HeinzG]

Hi Chris,

any updates on this?

 

Cheers,
Heinz

[sb101]

here it worked very good, but as a feature req. I'd want to turn off the thumbnail generation - that takes too long for many big files