Corrupted image previews

[MikeTO]

I think I've resolved this problem but it still makes me nervous so I thought I'd post about it.

Publisher stores a preview of each linked photo for use in the thumbnails as well as to immediately render a photo before loading the externally linked photo. I've been working on a big book in Publisher for a long time and today the image previews became corrupted and I don't know how to resolve it. The thumbnails appear correctly except for the photos - the four images shown in this Pages panel thumbnail are nothing like the images - it's just garbage data. Notably, my file is usually 7MB when compressed and I'm not impacted by the file bloat problem but it wouldn't go below 17MB with this issue. So there was an extra 10MB of garbage data.

When you go to a page, whether by double clicking it in the Pages panel or simply scrolling, Publisher shows the stored preview first and then replaces it with a higher resolution version. This became very noticeable to me because it first showed these garbage data previews in the main window and then replaced them with the actual photos. It's all very quick and it didn't impact printing or exporting but it's disconcerting and made me wonder whether I should abandon my current document and revert to a previous version.

I restarted Publisher and restarted my Mac to no avail. Looking at the photos in the Resource Manager showed the same problem. I tried embedding and then relinking all of the photos but that didn't solve the problem. If I deleted a photo and added it back it was fine but given that I have hundreds of photos I was loathe to do that.

I finally found a solution - I renamed the folder containing the images and allowed Publisher to relink them when I opened the file. With the photos linked, this resulted in a 2.5GB file due to the file bloat problem. But I embedded and relinked all the photos with the Resource Manager which got it back to 7MB.

Anyway, it's fixed now but I was a bit worried about the document. I have a backup from yesterday but I'm glad not to have lost all of today's work.

I'm using 1.10.5 with the latest macOS on an M1 Pro.

 

 

[thomaso]

Possibly the storage location of the images might indicate a culprit for your issue. Your screenshot mentions a file path that seems to include a "cloud" volume or folder:

It is a known, often reported issue of Affinity that it can't handle items (resources & Affinity documents) properly which are stored on external drives (e.g. NAS, Cloud, USB-Stick).

Can you confirm that these images are stored on a non-local volume? If yes, does it solve this problem if you move/copy their location to a local disk + then use the "Relink" button for the missing items in the Resource Manager?

[MikeTO]

2 hours ago, thomaso said:

Possibly the storage location of the images might indicate a culprit for your issue. Your screenshot mentions a file path that seems to include a "cloud" volume or folder:

It is a known, often reported issue of Affinity that it can't handle items (resources & Affinity documents) properly which are stored on external drives (e.g. NAS, Cloud, USB-Stick).

Can you confirm that these images are stored on a non-local volume? If yes, does it solve this problem if you move/copy their location to a local disk + then use the "Relink" button for the missing items in the Resource Manager?

I should have kept a copy of the file for testing but I can't re-test now.

Yes, the document and its images are saved to iCloud and I'm aware of that type of issue but I don't think that's what's going on. The cloud/NAS issue is due to the inherent latency of loading resources from an external location but in my case everything is also stored on the internal SSD and that's where Affinity is loading them from. Affinity can't load them from iCloud if they're stored locally.

There are a couple of scenarios were you could run into trouble with iCloud. First, combining a local network with cloud syncing can lead to issues. I've been burned so many times using Office 365 with Microsoft Azure on a corporate network with unreliable wifi. That combination makes it easy to wind up with a newer copy in Azure than you have locally. But another scenario even for a single user without a network is multiple devices. If the images are updated by device A and you open the document on device B, then the images in iCloud might be newer than those stored locally if they haven't synced down yet which could lead to latency in opening the document's images.

With iCloud, files are saved locally first and then asynchronously synced to the cloud. It's possible for the sync to iCloud to fail or stall but you'll always have a local copy, at least until you or the OS chooses to offload it. None of my files are offloaded so Affinity is opening the local copy, not the cloud copy.

[thomaso]

Vor 13 Minuten sagte MikeTO:

Bei iCloud werden Dateien zunächst lokal gespeichert und dann asynchron mit der Cloud synchronisiert. Es ist möglich, dass die Synchronisierung mit iCloud fehlschlägt oder ins Stocken gerät, aber Sie haben immer eine lokale Kopie, zumindest bis Sie oder das Betriebssystem sie auslagern. Keine meiner Dateien wird ausgelagert, sodass Affinity die lokale Kopie öffnet, nicht die Cloud-Kopie.

Thanks for the infos (I am not experienced with the iCloud).
Does this mean the Resource Manger does not report any of the images as missing even if you entirely unplug the computer from network connection before opening the document?

[Old Bruce]

2 hours ago, MikeTO said:

.... in my case everything is also stored on the internal SSD and that's where Affinity is loading them from....

... Affinity is opening the local copy, not the cloud copy.

Then why is the location

  ?

[MikeTO]

3 hours ago, thomaso said:

Thanks for the infos (I am not experienced with the iCloud).
Does this mean the Resource Manger does not report any of the images as missing even if you entirely unplug the computer from network connection before opening the document?

That's correct, if I disable my network connection Affinity will be none the wiser, I will still be able to open and save all of my iCloud files. The changes will be synced when my network connection is restored.

50 minutes ago, Old Bruce said:

Then why is the location

  ?

That's just the way iCloud works. Apps only know the iCloud path and when attempting to open an iCloud file macOS checks for a local copy in the user's Library folder and opens that if it exists. Only if there isn't a local copy does the app actually load the data from the cloud. Every one of my documents and images I have in iCloud is also on my local drive but this isn't true for a default setup. If you buy a new Mac and restore from iCloud your documents and images won't be downloaded to your Library folder until they're needed. I forced a full download to avoid issues with Affinity.

I also never modify my Affinity files on another device so they're never outdated. If you modify data on another device then it's possible that the iCloud version will be newer than what's on your Mac. MacOS should sync the new data automatically but I can imagine there could be latency issues if you tried to open a document that used an image that hadn't synced yet.

iCloud is far from perfect but this basic part of it works flawlessly. The biggest issue with iCloud is the bug with syncing getting stuck if you create too large of file which happened to me again while fixing the corrupted previews we're talking about. If you delete the big file immediately you'll be fine but if you don't notice it right away iCloud can get stuck and then you'll have to fix it manually. The approach Apple takes is to have you turn off iCloud syncing, delete all of your local files, and then restore from iCloud. This obviously makes me nervous so I backed up all of my local files to an external drive before doing it and after the restore was complete I went through every folder looking for lost files and I found several that hadn't been synced to iCloud after it got stuck. This took several hours so it's better to just delete huge files as quickly as possible.

[Old Bruce]

24 minutes ago, MikeTO said:

iCloud is far from perfect but this basic part of it works flawlessly. The biggest issue with iCloud is ... This took several hours so it's better to just delete huge files as quickly as possible.

If I used iCloud and had a problem which took several hours to fix then I would stop using iCloud. For the record I use it for nothing. And I continuously check every few weeks to make sure that it hasn't turned on with various OS updates. I don't trust any of the various cloud services.

[MikeTO]

19 hours ago, Old Bruce said:

If I used iCloud and had a problem which took several hours to fix then I would stop using iCloud. For the record I use it for nothing. And I continuously check every few weeks to make sure that it hasn't turned on with various OS updates. I don't trust any of the various cloud services.

I trust iCloud because I understand exactly what it's doing and I know it's limitations. You and I are both old timers and we could compare our data loss scars but I really think you should try iCloud even if just for fun. The worst that can happen is that iCloud will stop syncing - today that will happen only because of a super large file but perhaps there will be some new bug in the future. But even if it stops syncing you will still be opening and saving your files locally until you notice and fix the problem because that's the way iCloud works, it saves locally first and then syncs to the cloud. The only ways I can think of to lose your files with iCloud are:

• Have a sync problem and then you replace your Mac and do a restore from iCloud without realizing that iCloud wasn't fully up to date. But you and I wouldn't do this, we'd always have an external backup and would ensure iCloud was up to date before doing a restore.
• Have a sync problem and then edit an outdated iCloud file with another device, or directly from the cloud in a browser. I never edit files from a web browser and I rarely edit with a second device. But if I do I always check that the file I'm about to open is the right version.

It's different than Microsoft's cloud where the worst that can happen is really as bad as you can imagine - Office 365 can open the cloud version when the local version is newer or it can open the local version when the cloud version is newer, and then you'll lose the previously-saved changes. That won't happen with iCloud.

The problem with iCloud is that people rely on it as their backup. It's great that they have it as a backup if they lose their device because hardly anybody backs up their files, but it's not a complete replacement for backing up your data.

Using iCloud Drive is quite cool - it's great to be able to open a file on your iPhone when away from your Mac. And setting up a new Mac from iCloud is so much easier than the old fashioned way.

IMO the worst thing about iCloud is the other services that sync through it. With iCloud Drive syncing, you can see exactly what has synced and delete it directly from the cloud if you want, you're in control. But with app service synching, the data is in a hidden database. For example, with Safari syncing of open tabs and frequently visited sites, if the data gets corrupted (which it can) there's no way for you to purge it and start over.

[PaoloT]

On 7/21/2022 at 6:54 PM, MikeTO said:

when attempting to open an iCloud file macOS checks for a local copy in the user's Library folder

While the files reside in the internal drive, they are still under sync. If iCloud tries to write the files, while the Affinity app is using it, there will be two different versions of the file in you system: the one in memory, where the Affinity app is working, and the one in the local copy of the iCloud Drive. Save the file from the Affinity app, and you get a mix of both files (therefore, a damaged file).

Working on a local file should mean working on something residing in the Documents folder, not constantly synchronized.

Paolo

 

[thomaso]

3 hours ago, PaoloT said:

Save the file from the Affinity app, and you get a mix of both files (therefore, a damaged file).

What makes you think this is related to an external cloud drive? Why wouldn't a similar confusion or error happen if I edit a placed resource in an external, non-Affinity editor?

I have no external drive, all resources and Affinity files are stored on 1 local built-in SSD – but still I get the various known errors ("can't read", "access lost", "must close"up to "corrupt file" or app freeze while saving) from time to time with no apparent principle – except the fact that it only happens when I hadn't saved for several hours and apparently preferred when the computer was in sleep mode in between. <– This makes me assume these issues are caused by an Affinity internal time problem when waiting for feedback via macOS, or conflicting because of your mentioned incoming vs outgoing information. The oddity is I can't reproduce it – though it happens a lot more often since the last update 1.10.5.

The attached .afpub is saved with History, the attached JPG is linked. The .afpub was initially created by opening the attached JPG in APub and got the same JPG additionally placed as a linked resource. The yellow spot was added in another app while the .afpub was opened. No issues so far. Each Save action in APub was done as "Flattened" (the last Save as .afpub excepted) and thus replacing the placed JPG during the Save action. This caused the several pict-in-pict appearance. I assume this .afpub which worked like a charm would get one of the typical Affinity issues if I continue …

CumulusCloud Kopie.afpub.