Jump to content
derei

Save Failed because ownership of the file could not be verified

Recommended Posts

 you loose all data since the last autosave due to this issue!!

 

Just got this nasty error when tried to save. Luckily I had a previous save and the file recovered automatically without too much work progress loss.
I'd like to know the cause and what possible steps to take in order to prevent it in the future.

Worth mentioning:

  • I am using pCloud, which creates a virtual drive on the computer and I was saving there. Could this cause any issue (the sync process could take ownership of the file temporary or, to lock it in some way)?
  • it is the first time when it happened, although i was saving the same for many times.

AF_Photo_Fail_crop.png

Share this post


Link to post
Share on other sites

Probably the saving to the virtual drive doesn't always work in a graceful manner here. Thus you better save locally and transfer the file later via the desktop file manager (Explorer) or the like.


☛ Affinity Designer 1.6.1 ◆ Affinity Photo 1.6.7 ◆ OSX El Capitan

Share this post


Link to post
Share on other sites
1 hour ago, v_kyr said:

Probably the saving to the virtual drive doesn't always work in a graceful manner here. Thus you better save locally and transfer the file later via the desktop file manager (Explorer) or the like.

So you think it might be the pCloud app causing the issue, not the way AF Photo is saving?

Share this post


Link to post
Share on other sites

Well others have lost their data when trying to save from APh/AD to some cloud based drives (similar setup as with virtual drives too as seen/mounted by the OS), so you are not alone here. I don't know how the Affinity products file I/O is concrete programmed here in terms for time out handling etc., but it seems they somehow do early loose the connection here, if that is not always stable available. - Thus I said (for your data and work security here) better save locally and often when possible.


☛ Affinity Designer 1.6.1 ◆ Affinity Photo 1.6.7 ◆ OSX El Capitan

Share this post


Link to post
Share on other sites

Hi derei!

We've seen this error before when using network/external drives and the connection to the drive is temporarily lost whilst working. Our developers are aware of this and are working to fix it, in the meantime we recommend saving locally and then backing up the file, as v_kyr has suggested :) 

Share this post


Link to post
Share on other sites
3 hours ago, Dan C said:

Hi derei!

We've seen this error before when using network/external drives and the connection to the drive is temporarily lost whilst working. Our developers are aware of this and are working to fix it, in the meantime we recommend saving locally and then backing up the file, as v_kyr has suggested :) 

Hi @Dan C and thanks for your response.
As a suggestion for fixing this: DO NOT CLOSE THE DOCUMENT IF THE SAVE FAILS!! Yeah, is quite basic... why "the document must now be closed"? I mean, keep it open and allow the user to save again...
I am sure that your developers are looking into more professional solutions, but until one has been found, NOT Closing the document could be a great workaround... at least we don't lose the work.
As for saving locally and then backing up, whilst this may be the ultimate solution, is not so convenient at times... one might forget to backup, or may end up with two different versions on two different places due human error, etc... the scribes in Ancient Egypt were manually duplicating papyrus documents, because they didn't know how to make CPU chips with all that silicon laying around. I'm hoping to not get there again.

Share this post


Link to post
Share on other sites
21 hours ago, derei said:

... why "the document must now be closed"?

It has been mentioned in some other topics that (as I understand it) the reason for this is the Affinity apps do not load everything in a saved file into the computer's physical memory (RAM) at once. Among other things, that significantly reduces the memory required to run the app & makes it possible for it to handle very large files. But unfortunately, it also means that if it can't access the file when it needs to get something from it not currently loaded into memory, it can't continue to work with it.

So while the file does not have to be closed, at that point there is no way to save a complete copy of it to a different location, or to do anything with it that it can't do using only whatever parts of it are currently in memory.

One way to avoid this is to allow the app to attempt to load the entire saved file into memory & rely on the memory management system built into the OS to page out whatever won't fit into virtual memory, and/or for the app to load it into temp file(s) on the startup or a designated 'scratch' drive. Many apps do this but it is not an ideal solution for several technical & practical reasons.

I am just guessing about this but an alternative that might work for the Affinity apps is to leverage the serial nature of the native Affinity file format & the crash recovery feature that (as I understand it) just saves changes to open files to an 'autosave' folder. So basically, treating the error in the same way as when the app crashes might work, assuming that they can come up with some straightforward way to make it possible to restore access to the saved file, & resolve whatever other issues that might involve that I have not considered.


Affinity Photo 1.6.7 & Affinity Designer 1.6.1; macOS High Sierra 10.13.6 iMac (27-inch, Late 2012); 2.9GHz i5 CPU; NVIDIA GeForce GTX 660M; 8GB RAM
Affinity Photo 1.6.11.85 & Affinity Designer 1.6..4.45 for iPad; 6th Generation iPad 32 GB; Apple Pencil; iOS 12.1.1

Share this post


Link to post
Share on other sites

@R C-R Thank you for the comprehensive clarification, this makes perfect sense.

Yet, if the situation is as you say it, then Serif could do a much better job in perfecting how their software works. For start, some of us have machines with A LOT of RAM... I've got only 16GB, but I am sure there are people with 64-128 here. So, those people did not pay for the ram to keep it unused and instead AF Photo to load bits of file from a slow media (hdd, usb, etc).
So, maybe the software should do a dynamic assessment of the available RAM and decide if it loads the file completely in RAM or loads from the disk piece by piece as needed... 
Also, in terms of disk, some people may have a very fast disk (i've got an m.2 pcie ssd and a sata ssd, beside my storage hdd). So, maybe the software should do a cached copy of the file being edited on a disk at user's choice (editable in settings), and to work from there... this way, even in an event of a crash, there would be a second copy to recover from. I don't mind at all using my ssd for caching, even if that "in theory" would shorten the life of the storage. In reality the drive becomes obsolete before it dies, so I would replace it in 4-5 years anyway. But instead I get the speed and the stability.

And when a saving command is being performed, the original file can be rewritten from the cached version, in this way, even if the media can't be accessed for some reason, affinity can try again, offer the user the option to save in a different place and for work it would always have the cached version, so no interruptions here either.

Maybe some of the development members would like to help us here with a bit of insight and/or get inspired from the above.


I'm not a developer, I'm a product designer with basic knowledge of programming, so I'm just seeing this matter from a very general point of view, thus I might misunderstand some aspects, omit some rather crucial details or just point the obvious without noticing. Apologies for any inconvenience.

Share this post


Link to post
Share on other sites
47 minutes ago, derei said:

Yet, if the situation is as you say it, then Serif could do a much better job in perfecting how their software works.

Assuming I have understood how it works correctly, maybe they can make this work better, but one of the problems with trying to load the entire file into RAM is it is also used by every other process running on the computer, so there is no way to know how much of it the memory management system of the OS will allow an Affinity (or any other) app or its open documents to remain resident in memory at any particular time. That means paging to & from VM can occur at any time, so at best the app would have to spend a lot of time keeping track of that highly dynamic process instead of processing user input, updating the display, & so on; & that in itself could increase the amount of paging. Obviously, it also would adversely affect performance.

Another potential problem is if the OS would even expose enough of this memory use info to the app for security reasons -- I do not pretend to understand the details but apparently if an app can get access to how memory is used by other processes than its own that increases the 'attack surface' malware can leverage to compromise data security ... or something like that.

All I know for sure is it is all very complicated & the details probably vary considerably depending on the OS & the hardware everything runs on, so it is unlikely there is any simple obvious way to make it all work better than it does now.


Affinity Photo 1.6.7 & Affinity Designer 1.6.1; macOS High Sierra 10.13.6 iMac (27-inch, Late 2012); 2.9GHz i5 CPU; NVIDIA GeForce GTX 660M; 8GB RAM
Affinity Photo 1.6.11.85 & Affinity Designer 1.6..4.45 for iPad; 6th Generation iPad 32 GB; Apple Pencil; iOS 12.1.1

Share this post


Link to post
Share on other sites
On 1/15/2019 at 2:41 PM, derei said:

As a suggestion for fixing this: DO NOT CLOSE THE DOCUMENT IF THE SAVE FAILS!! Yeah, is quite basic... why "the document must now be closed"? I mean, keep it open and allow the user to save again...

Well this depends highly on the mechanisms to detect this at all and then the way to deal with it, aka already opened file handles and all the possible occupied object memory here (...with objects meant here more in general terms of an OO programming language handling of these).

Meaning if you have an open file stream handle, but can't flush (sync) for what connectivity reasons ever all the buffered data, you will get a file write failure. In such cases you will have at least to delete (free) all the so far occupied buffered data memory and file streams for that specific write operation. Otherwise if not, you will sooner or later get memory leaks due to once occupied but never freed up objects in memory and the app crashes then.

So it might be that actually in Affinity a Document has to be closed under such circumstances, in order to free up all so far occupied memory and leave the app in a working/running state, if a network based file write operation fails for some reason.


☛ Affinity Designer 1.6.1 ◆ Affinity Photo 1.6.7 ◆ OSX El Capitan

Share this post


Link to post
Share on other sites

All things considered, maybe the best solution, workaround, or whatever you want to call it for this is for the Affinity apps to pop up a warning if a user tries to open a file on a network drive, saying that there is a risk of data loss unless the file is copied to a local drive, with an option to do that?


Affinity Photo 1.6.7 & Affinity Designer 1.6.1; macOS High Sierra 10.13.6 iMac (27-inch, Late 2012); 2.9GHz i5 CPU; NVIDIA GeForce GTX 660M; 8GB RAM
Affinity Photo 1.6.11.85 & Affinity Designer 1.6..4.45 for iPad; 6th Generation iPad 32 GB; Apple Pencil; iOS 12.1.1

Share this post


Link to post
Share on other sites
5 minutes ago, R C-R said:

All things considered, maybe the best solution, workaround, or whatever you want to call it for this is for the Affinity apps to pop up a warning if a user tries to open a file on a network drive, saying that there is a risk of data loss unless the file is copied to a local drive, with an option to do that?

Yes, I can agree on this: a warning would not solve the issue, but make the user aware of the risk before is too late.
But nonetheless, this should be investigated further. I found also another issue here on the forum with a guy saying he can't sync OneDrive after a file has been modified with AF Photo and he has to restart windows in order for the sync to function again. He has posted it recently. So, it might be a shared issue... we'll wait and see.

Share this post


Link to post
Share on other sites
19 minutes ago, R C-R said:

All things considered, maybe the best solution, workaround, or whatever you want to call it for this is for the Affinity apps to pop up a warning if a user tries to open a file on a network drive, saying that there is a risk of data loss unless the file is copied to a local drive, with an option to do that?

That would imply that they would first have to check/test all the possible file system locations/types for file read/write operations, aka is it local, shared and mounted over a network etc. However the software should detect and handle a failure here in some more graceful manner than actually.


☛ Affinity Designer 1.6.1 ◆ Affinity Photo 1.6.7 ◆ OSX El Capitan

Share this post


Link to post
Share on other sites
2 hours ago, v_kyr said:

That would imply that they would first have to check/test all the possible file system locations/types for file read/write operations, aka is it local, shared and mounted over a network etc.

That info should always be immediately available, provided by the OS in the same way it is to Finder or Windows Explorer, both of which are applications.

2 hours ago, v_kyr said:

However the software should detect and handle a failure here in some more graceful manner than actually.

Agreed, but the question is how to do that without causing undesirable side effects, like degrading responsiveness or causing lots of disk thrashing.


Affinity Photo 1.6.7 & Affinity Designer 1.6.1; macOS High Sierra 10.13.6 iMac (27-inch, Late 2012); 2.9GHz i5 CPU; NVIDIA GeForce GTX 660M; 8GB RAM
Affinity Photo 1.6.11.85 & Affinity Designer 1.6..4.45 for iPad; 6th Generation iPad 32 GB; Apple Pencil; iOS 12.1.1

Share this post


Link to post
Share on other sites

For Win there are functions which tell you (like GetDriveTypeA), for OSX/IOS don't know if there is something equivalent which can be used for both in the same manner here.


☛ Affinity Designer 1.6.1 ◆ Affinity Photo 1.6.7 ◆ OSX El Capitan

Share this post


Link to post
Share on other sites
4 hours ago, R C-R said:

the question is how to do that without causing undesirable side effects, like degrading responsiveness or causing lots of disk thrashing

Maybe in a similar fashion like common download/upload managers work, thread based and showing a progress bar/circle indicator etc.


☛ Affinity Designer 1.6.1 ◆ Affinity Photo 1.6.7 ◆ OSX El Capitan

Share this post


Link to post
Share on other sites
3 hours ago, v_kyr said:

Maybe in a similar fashion like common download/upload managers work, thread based and showing a progress bar/circle indicator etc.

How would that help? 


Affinity Photo 1.6.7 & Affinity Designer 1.6.1; macOS High Sierra 10.13.6 iMac (27-inch, Late 2012); 2.9GHz i5 CPU; NVIDIA GeForce GTX 660M; 8GB RAM
Affinity Photo 1.6.11.85 & Affinity Designer 1.6..4.45 for iPad; 6th Generation iPad 32 GB; Apple Pencil; iOS 12.1.1

Share this post


Link to post
Share on other sites

Those have typical stateChanged, error & resume tracking and also time out connectivity handling for data reading/writing. Some of those mechanisms are useful to have for data storage (save operations) over network drives and cloud service drives etc.


☛ Affinity Designer 1.6.1 ◆ Affinity Photo 1.6.7 ◆ OSX El Capitan

Share this post


Link to post
Share on other sites
15 minutes ago, v_kyr said:

Those have typical stateChanged, error & resume tracking and also time out connectivity handling for data reading/writing. Some of those mechanisms are useful to have for data storage (save operations) over network drives and cloud service drives etc.

OK, but once connectivity is lost, until it is regained nothing can be done with the file that requires anything not already in memory (assuming the entire file isn't copied to a local drive to begin with, which would take us back to dealing with potential VM, performance, and/or other issues).

Remember, there could be several large Affinity and/or other document files open at the same time, global source references in Affinity Photo documents (& soon linked files when Affinity Publisher is out of beta), & who knows what else that needs working memory. Drag & drop & copy/paste have to be supported, & their sources & targets could be anywhere, & that also needs working memory.

There is a limit to how much can be swapped with VM before performance starts to suffer, & particularly on systems with relatively small SSD's there is only so much space available for VM, caches, temp files, backup copies of open documents, & such. 

So unless I am missing something about what you are suggesting, I don't see how the app could do much until connectivity is restored, & there has to be some graceful way to deal with a temporary loss of connectivity that doesn't hang the app.


Affinity Photo 1.6.7 & Affinity Designer 1.6.1; macOS High Sierra 10.13.6 iMac (27-inch, Late 2012); 2.9GHz i5 CPU; NVIDIA GeForce GTX 660M; 8GB RAM
Affinity Photo 1.6.11.85 & Affinity Designer 1.6..4.45 for iPad; 6th Generation iPad 32 GB; Apple Pencil; iOS 12.1.1

Share this post


Link to post
Share on other sites

The apps have a history and crash restore mechanism at least until the crash point occurs. Further loading a file is nothing else than reading in already available data from a file via buffered streams (chunks) into the working mem. It can be (re)initiated when timeout errors occur, either on demand or automatically etc. You can also seek, remember and use positioning in streams in order to get/put (read/write) data etc. So things can be programmatic fine tuned here. - You can generally use some fitting design patterns in order to observ and get notified if certain read/write operations do fail, need to be reaccessed/restarted or recovered, in order to build up a better overall file load/save management.

3 hours ago, R C-R said:

Remember, there could be several large Affinity and/or other document files open at the same time, global source references in Affinity Photo documents (& soon linked files when Affinity Publisher is out of beta), & who knows what else that needs working memory. Drag & drop & copy/paste have to be supported, & their sources & targets could be anywhere, & that also needs working memory.

And? Those (memory handling/sharing etc.) are common things all software and the operating system itself always do deal with. For apps most of that is already encapsulated in reused abstraction layers/frameworks, you just have to use it and set it up in a more intelligent and foolprove manner.


☛ Affinity Designer 1.6.1 ◆ Affinity Photo 1.6.7 ◆ OSX El Capitan

Share this post


Link to post
Share on other sites
54 minutes ago, v_kyr said:

Further loading a file is nothing else than reading in already available data from a file via buffered streams (chunks) into the working mem.

But we are talking about a situation where the file is on a network drive or cloud service & the connection to it is lost. So how do you load anything from it, or do anything with it that requires loading anything not already in working memory until the connection is restored?

58 minutes ago, v_kyr said:

You can also seek, remember and use positioning in streams in order to get/put (read/write) data etc.

I am not sure what you mean by this. There is no guarantee that you will have continuous, direct access to a file on a remote server, particularly if the connection to it is lost, even briefly.

1 hour ago, v_kyr said:

And? Those (memory handling/sharing etc.) are common things all software and the operating system itself always do deal with.

And they can kill performance if they require too much paging to/from VM, even if your boot drive is a fast SSD. That is something we are trying to avoid, right?


Affinity Photo 1.6.7 & Affinity Designer 1.6.1; macOS High Sierra 10.13.6 iMac (27-inch, Late 2012); 2.9GHz i5 CPU; NVIDIA GeForce GTX 660M; 8GB RAM
Affinity Photo 1.6.11.85 & Affinity Designer 1.6..4.45 for iPad; 6th Generation iPad 32 GB; Apple Pencil; iOS 12.1.1

Share this post


Link to post
Share on other sites
1 hour ago, R C-R said:

But we are talking about a situation where the file is on a network drive or cloud service & the connection to it is lost. So how do you load anything from it, or do anything with it that requires loading anything not already in working memory until the connection is restored?

How does my web browsers download manager can re-establish a connection or resume a download from a partly disconnected web or cloud service etc., use the same technique or a similar method here. - If you open a file (local or over the net etc.) from a location you determine its size and start to read (transfer) its data, buffer the data and count the amount of bytes you've read so far, hold that in a var. If you loose the connection you will get an error and knowing that, but you also know how much data was read so far into the buffer. Now until some meaningful timeout period try to re-establish the data reading, if the connection is again there this continous data reading until all bytes are in buffer, load the buffer contents into the apps memory and free the buffer. If the connection doesn't re-establish and you get the final timeout period, inform the user that the file couldn't be read due to lost net connections.


☛ Affinity Designer 1.6.1 ◆ Affinity Photo 1.6.7 ◆ OSX El Capitan

Share this post


Link to post
Share on other sites
3 minutes ago, v_kyr said:

How does my web browsers download manager can re-establish a connection or resume a download from a partly disconnected web or cloud service etc.

What can you do with the content of a web page that has not yet been downloaded to your computer? That is equivalent to what we are talking about here.

Affinity document files are not streamed nor is all of one loaded into a local data buffer. As I understand it, the file itself effectively takes the place of a separate data buffer, so chunks of it can be read into memory as needed. Changes are temporarily stored locally in some unspecified way, but we know the files are serialized so whenever they are written back to the document file they are (presumably) just appended to its end & remain in that form until the redundancy reaches 33% (or whatever the number is), at which time they are 'de-serialized' to reduce file size & the whole file is rewritten, replacing the original.

Admittedly, there is a lot of guesswork in this because they are not willing to share the details of the native file format (because they consider that a proprietary trade secret) but we do know that one of the design goals was a high level of memory efficiency, so I don't think it is too far off the mark.


Affinity Photo 1.6.7 & Affinity Designer 1.6.1; macOS High Sierra 10.13.6 iMac (27-inch, Late 2012); 2.9GHz i5 CPU; NVIDIA GeForce GTX 660M; 8GB RAM
Affinity Photo 1.6.11.85 & Affinity Designer 1.6..4.45 for iPad; 6th Generation iPad 32 GB; Apple Pencil; iOS 12.1.1

Share this post


Link to post
Share on other sites
12 minutes ago, R C-R said:

Affinity document files are not streamed ...

File streams (as here in C++ for example) are a pretty basic still lower level way of opening a file for read/write operations. What you write there in and how that data is built up or structured depends on you.

Quote

...nor is all of one loaded into a local data buffer.

You don't have any clues of programming or? - I think you didn't understood my above suggestion at all or I didn't explained that well. I said just use a buffer Affinity's software uses a bunch of buffering everywhere in their programs. How do you think they perform read/write/copy and other operations?

26 minutes ago, R C-R said:

What can you do with the content of a web page that has not yet been downloaded to your computer? That is equivalent to what we are talking about here. 

Nothing, you can only collect the data until that page is downloaded on your computers temp store for show up. If the connection breaks away you retry to continue where you left off. That's exactly what I sketched above for a open file mechanism over a network, I gave a suggestions for a different mechanism to ensure that the data is transferred completely and buffered before loading into the workspace memory.


☛ Affinity Designer 1.6.1 ◆ Affinity Photo 1.6.7 ◆ OSX El Capitan

Share this post


Link to post
Share on other sites
23 minutes ago, v_kyr said:

I said just use a buffer Affinity's software uses a bunch of buffering everywhere in their programs. How do you think they perform read/write/copy and other operations?

Of course they use buffers. But they have told us they do not load all of the file into memory, nor transfer it all into a local file. That's why the loss of the connection to a network file makes continuing impossible.

They have a local file, but it's the auto-recovery file, which just saves changes to the original file to allow recovery after a restart if the program crashes.


-- Walt

Windows 10 Home, version 1809, 16GB memory, Intel Core i7-6700K @ 4.00Gz, GeForce GTX 970
Affinity Photo 1.6.5.123 and 1.7.0.331 Beta       / Affinity Designer 1.6..5.123 and 1.7.0.331 Beta    / Affinity Publisher 1.7.0.330 Beta

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×