riveryeti
-
Posts
10 -
Joined
-
Last visited
Posts posted by riveryeti
-
-
I should also point out that this is larger than the stated sensor size by the manufacturer, so is probably an artifact of how affinity is processing the DNG format.
-
Hi Affinity devs!
I bought your software this week after testing it out a year or so ago against a$#%e pho#@$##p and illu$##%@!r and getting frustrated that they wanted a Credit Card# for me to even test raw development.
Anyway, tried out the batch for processing raws to jpegs for aerial mapping using structure-from-motion photogrammetry in comparison with a bunch of other softwares, and was surprised and impressed when quality-wise it outshone the others! By that, I mean that your raw conversion algorithms gave me the "best" images (more between-image matches), even relative to RAW. So whatever you added, it's doing great stuff. I have to strongly disagree with James Ritson (@JamesR_Affinity) that Affinity Photo "..is not suited for bulk/batch development of RAW images" - at least for our case, it is well-suited for bulk-development, especially if you can get the multithreaded workflow working on my threadripper!
Hoping this inspires you to fix the broken multithreaded batch and add scripting and give us more info on exactly what you're doing in your raw processing 🙂
Also FYI I would love the option (maybe basic/advanced?) to explicitly control JPEG DCT (ifast, integer, float) and chroma subsampling as well as compression quality (similar to GIMP) and maybe even support lossless jpeg like OpenImageIO does with q100 = lossless
more details on some of the technical specifics in a sort-of-crosspost here
- Chris B and PaulAffinity
-
2
-
Attached a DNG and the ARW I used to make it. I used Adobe DNG converter to make it, I think the most recent profile (whatever it defaulted to). The only thing I changed was to make it not generate embedded thumbnails (or to do the smallest one, whatever my choice was).
BTW I have gigabit symmetrical internets and you seem to have an upload bottleneck somewheres. my speed test is running at 357mbps up and I'm only getting about 1mbps to your forum upload.
Just a note - with all the image software that I am testing, the inability to reliably understand where the default sensor crop of 7360x4912 is cropping is frustrating. I dunno if I'm supposed to start at UL,LR, or the center to crop (though these bad pix on the right of the DNG imply that I should be way left). But the 2 extra "x" pixels and 8 extra "y" pixels in ARW mode are more difficult to figure out.
Since we are doing scientific imaging, the less resampling and manipulation that we do the better, so cropping without resampling would be ideal, and cropping to the "standard" sensor output or at least knowing what the standard sensor crop is, would be nice so we can duplicate the principal point of the lens and apply the same lens distortion model to images converted from ARW with different softwares. Not sure how I can find the default sensor crop extent though. Hope this isn't too off-topic for beta.
Andy
2018-10-08_17_14.10-DSC04712-N7251F.dng 2018-10-08_17_14.10-DSC04712-N7251F.ARW
-
10 hours ago, carl123 said:
Out of interest what is the total file size of the 1000 .ARW files and the 1000 .DNG files?
The 1000 .ARW files total 34.7GB and the 1000 DNGs total 36.2GB. I generated the DNGs with Adobe DNG converter and the "no thumbnails" option.
-
I'm experimenting with file export right now, and I was unimpressed with how long it took to load a thousand .ARW files into the batch dialogue queue, and the fact that the CPU was at ~1% and disk (NVMe) was at 0.4% the whole time, which makes me think there's an inefficiency in code somewhere?
So I figured I'd be a good beta tester and I did a few tests on the batch dialogue input file queue and found a couple interesting things. I tested with 1000 images in .ARW and .DNG (converted from the same .ARW)
To clarify, by "batch dialogue queue" I mean opening the "New Batch Job" dialogue from within Affinity Photo via File/New Batch Job/ and selecting the [Add] button to add files to the processing queue.
1. It took 2.25 minutes to load 1000 .ARW into the batch dialogue queue.
2 It took 15 SECONDS to load DNGs into the dialogue queue (9x faster than ARW)
3 It took 1.2 minutes to REMOVE the .ARW OR the .DNG from the batch dialogue queue.
4. Most of the times (I think all but the first) The dialogue queue disappeared because focus shifted to the main affinity photo window and the batch dialogue became hidden behind it.
Platform was Windows 10/Threadripper 3960x with 256GB RAM and NVMe drive.
Andy
-
This is a feature I would love too. I really like how darktable uses darktable-cli for scripting, but they don't auto-apply levels/curves adjustments to raw (at least for arw) as well as affinity does. I do projects with thousands of images at a time that would be way easier to manage with this feature.
-
This is a feature that would be very useful to me too. Regularly processing tens of thousands of aerial images, and affinity does good things with raws pretty quickly (would be nice to see batch working more reliably and more use of GPU too, but parallel processing is very effective on my system)
-
I was experimenting with batch converting arw to jpg on about 10 files with different parameters (with or without "convert to sRGB macro, pixel format default, rgb, cmyk, embedded sRGB profile). Anyway, I decided to output around 118 images that I'd already successfully output with sRGB macro/embedded profile, and I think rgb pixel format (maybe left at default). Importing in the same dir where I recently deleted a 10-file batch. parallel checked.
The batch job spun up and got to "saving" on the first 24 images but never made it past that. CPU hung out high for about how long it would take to save them then went to idle but the gui still says "saving". I was writing to a NVMe SSD. The drive is not full.
under task manager Affinity photo beta is using 29GB (I have 256GB) and barely any (less than 1/2%) of CPU (712 threads). There's another process - crashpad_handler.exe at 0% cpu (6 threads). Been this way for maybe 20 minutes now as I waited, then started writing this. Nothing in event logs. From looking at the thread handles in Resource Monitor it looks like maybe it's trying to write crash info. I attached a txt file with the copy/paste of the handles from Affinity Photo and Crash Handler.
System Info:
Win 10 Pro 1909 build 18363.836, 256 GB RAM, 2x EVGA RTX 2080 Super GPU, AMD Ryzen Threadripper 3960X 24-Core (SMT off), ASRock TRX40 Creator mobo
Affinity_beta_CPU_Handle_Dump.txt
---EDIT---
As a followup, I can't even reproduce what I did to successfully output images in batch. I can do little batches now, but haven't been able to reproduce the 118 image batch - I keep hanging. Have tried all sorts of color profiles and combos of clicking JPG/macro options. have also tried DNG/ARW. Haven't reset my pc yet but that's next. Also tried to do batch in non-beta and had the same hang.
--EDIT-- rebooted and still cannot find settings that successfully generate a 118 image batch (except turning off parallel processing). The first one I did worked in parallel (accidentally left box checked to generate affinity format photos and had to delete them) but I tried that too and it didn't work.
-EDIT- I did some more tests and things work fine with parallel processing up to 20 images, break at 22. When it breaks the CPUs stop running sooner than when it works (with message saving jpeg... and spinning wheel). I have no idea why I was able to make it work for a whole run my first time, but I wish I could repeat that luck. I know I included the sRGB macro, which I don't need, and I know I ran it from the developer persona, and I know I accidentally created the affinity-format files too, and I was working on a sRGB workflow, but I've tried all the permutations of that my patience can handle, and I haven't been able to find that magic that makes me get the whole batch done with all 24 cores. Which means I'm stuck with one. Unless I can set processor affinity to 20.. maybe I can do that..
And in case you're wondering I regularly peg out all my cores (and GPU cores) on other things, so I don't have some hidden system instability.
incidentally it would be nice if the batch settings (output dir/source dir/output type checkboxes/file type options) were persistent instead of resetting to default every time. And I'd love a ctrl-shortcut for batch or at least an alt-F-something shortcut. let me know if I can help debug. also a CLI for batch like darktable-cli would be super-amazing. I'd buy you a case of beer for that
Oh how I miss having beers with friends...
-
Hi Affinity! Just bought photo and designer and excited to join the community. I am trying the beta specifically for raw export and I noticed working with DNGs that Affinity doesn't honor the crop value. Capture One/darktable/adobe all say that the image (Sony ARW to Adobe DNG from Adobe DNG creator) is 7360x4912 (same as I get from RAW with sony software) but affinity says 7392x4920 (same as I get with OpenImageIO) and the rightmost pixels look funky (pic attached).
I'm using the images for mapping so this is a showstopper for me.
Incidentally if I open the ARW I get 7362x4920 with no artifacts (same as I get with OpenImageIO) and darktable gives me 7366x4920. Capture One and Sony and Adobe give me 7360x4912.
It would be nice if the "standard crop" was honored. If my center pixels are inconsistent it makes it hard to evaluate different softwares.
I can provide ARW and DNG if needed.
Oh how I miss having beers with friends...
Beta 1.8.4.650 batch convert crash bug?
in [ARCHIVE] Photo beta on Windows threads
Posted
I sent you a PM. I uploaded the .ARW and .DNG. Also just confirmed that I have the batch bug issue with both formats.