V2.1.0 -- Metadata could not be embedded

[plennon]

PSL_1576.DNGPSL_1575.DNG

 

Hi,

The other day I installed 2.1.0 and began processing my images. The editing process was fine, Develop and Photo personas worked as expected. However, when I went to export my images as JPEGs I repeatedly saw error messages stating my "Metadata could not be embedded". Sometimes an export or two would proceed without issue and then the following images would take repeated exports. I have attached a video of two DNG images that I tried to process. In the video you will see that the export process did not go well. I use the metadata to help me learn why a photo looks the way it does. Not including metadata is unacceptable.

Please reach out to me with any questions. I am happy to provide more videos and/or sample images. I am willing to test potential solutions.

Paul

 

Let me describe my workflow

I shoot images with a Pentax K-5 and save the images as DNGs. I use Affinity Photo to beautify the images and persist them as JPEGs

1. Open FileExplorer to the directory on my SD card where the source DNG images are
2. Launch Photo
3. Select an image on the SD and drag it into Photo
4. Adjust the RAW image as necessary
   1. In order to minimize variables I chose not to edit the images in the video
5. Click the Develop button
6. Final tweaks in the Photo persona
   1. Again, for this video I chose not to tweak the image
7. When satisfied, I click File/Export
8. I then wait for the Export dialog to display the image preview
9. Once the 'export' is ready, I click the Export button
10. I select where I want the image to be preserved to
11. Possibly adjust the filename
12. Click the Save button
13. Often, but not always a dialog will appear showing the export progress. As the progress bar reaches the end an error dialog often appears stating that my metadata was not embedded
    1. It's more than the metadata was not embedded, the JPEG was not created at all (nor should if been without the metadata)
14. I then retry the  export operation with the expectation that the next will surely succeed; it often does not
15. I will retry several times
    1. After three or four tries I will walk away and try again 10, 20 an hour later
    2. Sometimes waiting a long time works
16. In desperation I have saved the  image as an afphoto, closed the image and then loaded the AFPHOTO and tried to export it
    1. This works sometimes, more often than not, but not always

My 2cents

I suspect there is a threading issue. Photo uses many threads (150+). It is possible that a thread did not release its locks in the correct sequence

Machine

OS Name Microsoft Windows 10 Pro
Version 10.0.19044 Build 19044
Other OS Description    Not Available
OS Manufacturer Microsoft Corporation
System Manufacturer LENOVO
System Model    43539S7
System Type x64-based PC
System SKU  LENOVO_BI_A3
Processor   Intel(R) Xeon(R) CPU E5-2643 v2 @ 3.50GHz, 3501 Mhz, 6 Core(s), 12 Logical Processor(s)
Processor   Intel(R) Xeon(R) CPU E5-2643 v2 @ 3.50GHz, 3501 Mhz, 6 Core(s), 12 Logical Processor(s)
BIOS Version/Date   LENOVO A3KT45AUS, 5/21/2014
SMBIOS Version  2.7
Embedded Controller Version 255.255
BIOS Mode   UEFI
BaseBoard Manufacturer  LENOVO
BaseBoard Product   LENOVO
BaseBoard Version   0B98401 WIN
Platform Role   Workstation
Secure Boot State   Off
PCR7 Configuration  Elevation Required to View
Windows Directory   C:\windows
System Directory    C:\windows\system32
Locale  United States
Hardware Abstraction Layer  Version = "10.0.19041.2728"
Time Zone   Eastern Daylight Time
Installed Physical Memory (RAM) 16.0 GB
Total Physical Memory   15.9 GB
Available Physical Memory   5.88 GB
Total Virtual Memory    43.9 GB
Available Virtual Memory    22.7 GB
Page File Space 28.0 GB
Page File   C:\pagefile.sys
Kernel DMA Protection   Off
Virtualization-based security   Running
Virtualization-based security Required Security Properties  
Virtualization-based security Available Security Properties Base Virtualization Support
Virtualization-based security Services Configured   
Virtualization-based security Services Running  
Device Encryption Support   Elevation Required to View
A hypervisor has been detected. Features required for Hyper-V will not be displayed.   

 

Edited May 22, 2023 by plennon

[stokerg]

Hi @plennon,

I've tried exporting your files and they are exporting fine for myself.  This is from an SD card and also with the files on my Desktop.

Just to rule out the SD card being part of the issue, can you try copying the images to your desktop and loading them from there and see if you are able to get the error again.

I'll keep trying the export but so far, over 20 exports of each file and i've not seen the error once.

[walt.farrell]

There have been a number of reports of this over the years, mostly unresolved except that they stopped occurring for the users.

There was one I've found that determined the issue was with saving to Dropbox, and my guess is that the saving happens in two phases, with a Close and Reopen, and Dropbox was interfering with the Reopen. (That's just a guess.)

Anyway, I've linked that one below, where a Serif Developer said they'd also seen it with Google Drive, and thought they had a fix:

 

[plennon]

Hi @stokerg,

Thank you for looking into this. Yes, the example I posted used an SD as the source. Prior to that I used to empty the SD to my C drive and then process them. This scenario also experienced the metadata issue. I will give your scenario a try this week. When I do this, is there a way to activate logging? I too am a developer and I think this is a threading/locking issue. I think we can define this if we can capture what the code is doing.

Paul

[plennon]

Hi @stokerg

Tonight I again experienced the metadata embedding failure. This time I used Photo to open DNG files from my C drive and export them to my NAS. While I was using Photo I also had SysInternals Process Monitor running and it captured the file I/O during the export operations. I will say the Photo exports better when ProcessMonitor is running. I think the monitor slows file I/O enough that the timing is better (this is not a suitable solution).

Attached are two captures

1. ProcMon--APhoto-Stuck in export loop before button is active.Logfile.PML
   This captured I/O that was going on while the export dialog was preparing. The image was not yet displayed and the wait icon was spinning
   1. Near the end of the  log I can see that Photo tried to open a temporary file 12 times, failed each and then gave up. The wait icon continued to spin with zero file activity
2. ProcMon--APhoto-Export failed - possibly due to a sharing violation.Logfile.PML
   This captured the file I/O when a file that was not exported.
   1. Near the end of the file I found a sharing violation. I believe this occurred at the same time I saw the export failed dialog

I hope these logs can shed some light onto the metadata embedding issue. Please tell me how to turn on debug logging and I think you will find that information more useful

ProcMon--APhoto-Export failed - possibly due to a sharing violation.Logfile.PML ProcMon--APhoto-Stuck in export loop before button is active.Logfile.PML

[plennon]

An observation:

• It seems that when in the export dialog, if I pause for a count of ten before clicking Export, the process is less likely to fail
  • This is not a solution, but it helps me and could be a clue for you

[plennon]

Because of the export problem that I described here, this weekend when I attempted to perform a batch job that exported a collection of images to JPEG, most failed to export. Of the 114 images that I tried to export, only 37 were exported successfully. I as only able to complete the export jobs when I unchecked parallel processing. When checked, only four completed and the whole application froze. My machine has 24 cores, so 24 jobs were running at the same time. I really think you have a thread synchronization issue.