Lens EXIF data deleted?

[wps]

8 minutes ago, R C-R said:

So it seems pretty obvious that if we do dive into the big picture it becomes very clear that there is nothing simple about this, & that the issue is in fact a very difficult one to solve.

Then, once again, how is it that the other programs I have tried, including the free ones, have solved it? You keep ignoring the fact that other editing programs do not strip out the Makernote and export it out apparently properly enough for programs that make use of the tag to use it.

[R C-R]

1 hour ago, wps said:

Then, once again, how is it that the other programs I have tried, including the free ones, have solved it?

The problem isn't solved unless writing to metadata tags does not break the Makernote tag or cause any other problem with tag offsets, etc.

[wps]

55 minutes ago, R C-R said:

The problem isn't solved unless writing to metadata tags does not break the Makernote tag or cause any other problem with tag offsets, etc.

The other programs solve it for my purposes which was the point of this post, AP does not.  Many of the programs actually do write to the metadata tag to indicate which program has edited the file.  This doesn't seem to break the metadata tag. At any rate, its not that big of a deal as I just have to edit with DXO optics first. I just find it strange that those other programs that I mentioned above are able to pass through the makernote tag with no problem and AP can not or does not.

[v_kyr]

5 hours ago, R C-R said:

...

That page also includes other info about the problems & pitfalls of writing metadata tags, even with tools based on an ExifTool backend.

It always depends on the way you write metadata tags and how well specific/individual tools deal with it and have implemented the overall specifications. Since Exif & DCF are specified standards ...
 

Quote

 

Exif was established by the JCIA (Japan Camera Industry Association), the central organization for deliberations and the predecessor of CIPA. JEITA (Japan Electronics and Information Technology Industries Association) manages the standards, while CIPA discusses technology and promotes the standards.

*CIPA and JEITA have jointly established and managed these standards since 2009.

 

... there are also concrete file format and implementation specifications available for it, see these references:

• Exif Exchangeable Image File Format, Version 2.2 and especially also the file format specification as PDF here !

• Description of Exif file format

So it's generally a matter of implementation and handling, as with other common specified file formats too!

Those Nikon related things you are referencing is a custom Nikon vendor makernote tag, where Nikon stores it's custom cam and lens settings etc. "According to the Exif 2.2 standard, the makernote is a tag for manufacturers of Exif writers to record any desired information. The contents are up to the manufacturer, but this tag should not be used for any other than its intended purpose". Most cam vendors/manufacturers do not officially publish their custom makernote specifications and do keep these pretty private (secret), mostly for market competitive reasons, or why they also have/sell own explicite software for their cams, which then knows better than the competition how to deal with their custom settings.-  So it's no surprise here, that Nikon does know it's own makernote specs (it's context) best and that some in hex encoded entries aren't decoded by others. BTW some other vendors do it in a similar fashion here via their makernotes.

However, makernote tags are usually not meant to be changed/edited by end users afterwards, since they contain important manufacturer related informations. Instead end users can write their own contents (copyright infos or whatever) into new tags or other already available Exif information tag sections. If a software deals correctly with Exif data according to the specs, then it's usually no problem to read and write certain tags, even if they don't know to decode the makernote hex contents meanings (they then have just to take these over 1:1). - Though it's always better for security reasons to keep/make a backup copy here then, as ExifTool does too.

5 hours ago, R C-R said:

...

So it seems pretty obvious that if we do dive into the big picture it becomes very clear that there is nothing simple about this, & that the issue is in fact a very difficult one to solve.

Why should that be an overall excuse? There are much more difficult to handle file formats and specs available than the Exif data stuff. And since other software can handle and solve this well, it should be a challenge to do it also the right way too here.

 

[R C-R]

1 hour ago, v_kyr said:

However, makernote tags are usually not meant to be changed/edited by end users afterwards, since they contain important manufacturer related informations. Instead end users can write their own contents (copyright infos or whatever) into new tags or other already available Exif information tag sections.

Of course. But the problem is if users write their own content into tags that precede the makernote tag the makernote may need to be moved to make room for the user content. To make sure that doesn't break the makernote tag, the app can't rely on Exif standards because according to those standards the contents (including any encryption methods, offset pointers, & so on) are left to the manufacturer to determine. That is why reverse engineering to decode at least part of it may be required, why 1:1 moves can be problematic, & why the tools that permit writing to the various tags require frequent updates & complex error checking routines to make sure nothing gets written to a location that would corrupt pointers, allow writing info to tags that would conflict with the info in other tags or violate Exif standards for length or formatting.

 

Obviously, this is doable but it is resource intensive, if for no other reason than the camera manufacturers are constantly adding tags & changing the info they store in them to support new features. It is considerably different from supporting the major image file formats, which are quite stable & rarely change.

[v_kyr]

Well as said before the Exif file format is specified and it's format parts are thus defined accordingly. - Usually you are NOT intended to write your own personal user stuff in front of the Makernote offset sections here, the stuff before the makernotes is fixed lenght size and freely user addable Exif comments do only follow the makernote tags and thus don't precede them. Also usually you can't write wildly before that.

It is the responsibility of each software which deals with changing/writing Exif data, to also allow only to add/change/overwrite content of tag sections, which are by the standard intended to be changed by end users and so to keep the whole in a well structured correct format, so to say ordered manner (...keep the order, offsets, indicate the correct used byte lengths for variable length sections, and so on). - Thus the specs and standard definitions!

 

 

8 hours ago, R C-R said:

...

Obviously, this is doable but it is resource intensive, if for no other reason than the camera manufacturers are constantly adding tags & changing the info they store in them to support new features. It is considerably different from supporting the major image file formats, which are quite stable & rarely change.

Every file format (image file formats here too) which allow to embedd custom (often vendor proprietary) data into specified file sections has to deal with this. - As far as cam manufactors keep the overall file format intact and follow the standard specs, things should be still (re)usable here. -On the other side cam manufacturers have to add new features and move on technology wise, so they have a urgent need to address new additions here.

 

[wps]

8 hours ago, R C-R said:

Obviously, this is doable but it is resource intensive, if for no other reason than the camera manufacturers are constantly adding tags & changing the info they store in them to support new features. It is considerably different from supporting the major image file formats, which are quite stable & rarely change.

You keep flying by the part that other editing software is passing the makernote tag through on export and AP is not.

[R C-R]

27 minutes ago, v_kyr said:

Every file format (image file formats here too) which allow to embedd custom (often vendor proprietary) data into specified file sections has to deal with this.

Which in turn means every app that can export to the various different image formats has to deal with this somehow as well, & in addition to any IPTC core or Plus data and/or XMP metadata the file might include.

 

As I said, all this is certainly doable, but there is nothing simple about it. It requires a lot of complex coding & frequent updates to keep up with the changes & additions of camera makers, who are not necessarily keen on the idea of making this easy to do. So my point is if this is really something we want Affinity to divert some of its limited development resources to implement, which (obviously, I hope) means other needed features won't get the attention they deserve any time soon?

 

Aside from that, adding any complex code can adversely affect performance, require still more changes to the native Affinity file format, create bugs that affect the existing code base, & have other undesirable side effects that end users rarely consider & could further delay the development of Publisher & the DAM, if & when that project ever gets any attention.

 

I am just trying to be realistic about this. It is all well & good to think about how great it would be if all the features supported by other apps were added to the Affinity ones, but considering the downside as well as the upside has to be a part of it. That is not an excuse, it is a reality that can't be wished away or ignored.

[v_kyr]

In short, since everything technical relevant has already be said!

Since we are talking about a software, which is advertized by their creators to be a "Professional photo editing software", "The outstanding choice for professionals" and so claims to offer these things ...

Quote

Special workspace for RAW editing
Develop RAW files from your camera in a special workspace that provides all the necessary adjustments and precise correction options in an infinite linear color space.
Supports EXIF & metadata
Lens profiles
Hundreds are supported

...and so on...

... well then I think it's also only legitime for customers to expect here, that these things then do indeed work in a pleasing, well behaving manner!

[wps]

If

3 hours ago, R C-R said:

I am just trying to be realistic about this. It is all well & good to think about how great it would be if all the features supported by other apps were added to the Affinity ones, but considering the downside as well as the upside has to be a part of it. That is not an excuse, it is a reality that can't be wished away or ignored.

If you are going to be realistic about it then you have to accept the reality that other programs (even free ones) are handling the tag in a way that make it still useable.  AP strips it out.  I don't understand why you keep ignoring this reality. This may not be a concern for many but for those that use multiple programs for editing, it can be and not what I would expect in a professional level program. Having said that, I like AP and will continue to use it but it will always be last in the chain of editing.

[R C-R]

@wps I am not ignoring anything. I am simply considering what happens when a lot of different features from different apps are added to one app. How well do you think your free apps would work if they included everything in the Affinity ones?

[wps]

1 hour ago, R C-R said:

@wps I am not ignoring anything. I am simply considering what happens when a lot of different features from different apps are added to one app. How well do you think your free apps would work if they included everything in the Affinity ones?

Sorry, I don't consider passing through exif tags a feature, I consider it a necessity.  BTW, I was just able to try Luminar 2018 (released today) , it also passes along the tag without issue.

[R C-R]

13 minutes ago, wps said:

Sorry, I don't consider passing through exif tags a feature, I consider it a necessity.

If you take a little time to browse through these forums, you will see that the list of features various users consider a necessity is enormous. Implementing even a small fraction of them will take considerable time, particularly if they are to avoid adversely affecting the performance, memory requirements, stability, or any other aspect of the apps.

 

The developers have mentioned many times that what seems simple to implement to end users may require extensive changes to the code & extended beta testing to eliminate serious bugs that would make the apps unsuitable for serious work. It is for this reason that while they say they consider all feature requests, they prioritize implementing them based on how much work is required, how each of them would affect the existing features, how many users would benefit from their inclusion, & to some extent even if there are other apps available at little or no cost that can provide these features now.

 

So, while posting to the Feature Requests, Suggestions and Feedback forum is encouraged & may result in something being given a higher priority, there is no guarantee that it will, nor is there much point in continuing to post about it or comparing the Affinity apps to others here in the Questions forum.

[wps]

It's really not a feature request, it is a bug that should be fixed. It strips out the makernote tag, other apps don't. The only reason I compared to other programs was because you kept posting about how difficult it would be to do,  based on all the other programs that handle it correctly, it must not be that difficult.  My interest is in improving an already good program. If you want to defend the status quo, have at it.  I'll leave it at that.

[R C-R]

Just now, wps said:

... based on all the other programs that handle it correctly, it must not be that difficult. 

From this, is it safe to assume you are not a programmer? 

2 minutes ago, wps said:

My interest is in improving an already good program.

Then post to the Feature Requests, Suggestions and Feedback forum. You are wasting your time by continuing to post about it here.

[wps]

1 minute ago, R C-R said:

From this, is it safe to assume you are not a programmer? 

Then post to the Feature Requests, Suggestions and Feedback forum. You are wasting your time by continuing to post about it here.

I started as a machine code and assembly language programmer on PDP 11s for Texas Instruments and  from there programmed machine automation in FORTH for a number of years and then on to C. It is pretty clear that passing through the tag is 'doable' and the fact that it is done in cheap and free programs seems to indicate it's not that difficult or the code to do so is readily available. You are free to ignore that all you want but it doesn't change it.

I have posted in the bug report forum.

I'm through with this discussion.  I came with a question and want to thank v_kyr  for his thoughtful and helpful  responses in getting to the bottom of what was happening.

[Asser82]

I have this problem also:

I import my Canon RAWs in Lightroom and develop the JPEGs in DxO PhotoLab. The lens information produced in the JPEGs is OK at that point, because Lightroom combines the CR2 and the JPEG under the same lens filter (Canon EF-S 18-135mm f/3.5-5.6 IS STM) after DxO export. Now I want to  remove red eyes inside the exported JPEG in Affinity Photo. What comes out is a JPEG with broken metadata. DxO can no longer find the lens that belongs to the Photo, Lightroom assigns the Photo under an own Lens Category which is different from the one of the RAW.

Parsing the Affinity modified JPEG with "ExifTool" reveals that many of the Canon specific metadata tags have been lost. This is a serious bug and makes AF photo unusable in environments, where tools must work together.

[wps]

30 minutes ago, Asser82 said:

I have this problem also:

I import my Canon RAWs in Lightroom and develop the JPEGs in DxO PhotoLab. The lens information produced in the JPEGs is OK at that point, because Lightroom combines the CR2 and the JPEG under the same lens filter (Canon EF-S 18-135mm f/3.5-5.6 IS STM) after DxO export. Now I want to  remove red eyes inside the exported JPEG in Affinity Photo. What comes out is a JPEG with broken metadata. DxO can no longer find the lens that belongs to the Photo, Lightroom assigns the Photo under an own Lens Category which is different from the one of the RAW.

Parsing the Affinity modified JPEG with "ExifTool" reveals that many of the Canon specific metadata tags have been lost. This is a serious bug and makes AF photo unusable in environments, where tools must work together.

It strips the Makersnote tag which is what DXO uses for lens correction. None of the other editing programs I have do this and, as far as I'm concerned, is a serious bug in Affinity.

[Asser82]

It might be the Makersnote, but the delta seemed to be much greater. I would not expect the metadata to be changed at all, except maybe the modification date and the creator tool. Lets discuss this in the bug forum, so we do not have two places to look at.

Goto here:

 

[wps]

1 hour ago, Asser82 said:

It might be the Makersnote, but the delta seemed to be much greater. I would not expect the metadata to be changed at all, except maybe the modification date and the creator tool. Lets discuss this in the bug forum, so we do not have two places to look at.

Goto here:

 

There may be other areas of metadata that are changed but Makernote is what DXO uses for lens correction according to their web site.  You can examine the metadata after export from Affinity and see that it is stripped out.  In my work flow now I always use Affinity last for that very reason. It really needs to be corrected.

[Asser82]

When I am at home I will try to use the exiftool to restore the metadata. Should be something like that:

exiftool -X -b -makernotes -all a.jpg > a.xml

Work with Affinity Photo

exiftool -tagsfromfile a.xml -all:all a.jpg

Perhaps -all arguments can be removed, if the problem lies in makernotes only. If this works, I will write a batch automation application while this is not addressed.