File size changes

[Gear maker]

I have a question on file sizes.  Let's start with an observation; yesterday I worked on a fairly large drawing and did a "save as" so it would have a new revision, 428.5KB.  Today I copied the file with the name changed opened the file in MAS and just hit cmd S.  The file now is 726.5KB.

 

I exited AD then reopen the 726.5KB file, did a "save as" to another name, now it's 428.4KB.

 

I hit cmd S and it's back to 726.5KB.

 

Did another "save as" to another name and that file is 428.4KB.

 

So the question is what is up with file sizes?  Why does a "save as" make a file 41% smaller than a "save"?

[MattP]

You wouldn't believe how much work has gone into our file format - it's really a spectacular thing, but it sounds like there's a small issue there because I wouldn't expect a simple Save operation on an unmodified file to do anything at all to the file size - there's nothing to write... You'll notice some other weird things if you watch the file size carefully, after a while it'll get small on its own again, then grow, then reduce again... It's a carefully designed system that gives us performance and some level of being able to recover files. Save As stores a new version of this file, starting from scratch so it has no extra data in it, which is why it may appear smaller. I'm sure Ben will be along later to explain things better as he designed and implemented it, but I'll mention this issue to him and see what he says...

[Gear maker]

Thank you Matt.  This has taught me to use a save as for the final save of the project.  I assume you must save a pixelated version of the drawing along with the vector info because the size gets smaller when I hide layers.

 

I feel bad pointing out issues with the new Beta, it is so negative.  You all did a great job getting massive changes done quickly.  I think we all appreciate your team's work.  And the team is uniquely responsive.

[MEB]

You shouldn't feel bad about raising issues with the beta (or MAS version for that matter). That's exactly the point of it. We need to know about all possible issues with the software to correct them before submitting it for review on the Mac App Store.

The more problems we correct the more stable the app gets for general availability in the MAS.

[Ben]

Hi.

 

As Matt says - doing just a Save can result in the file growing most times, but then on occasion the file can end up smaller. This is because we save delta changes to the file to give to quickest save time possible.  Only the bits that change get saved.  We do this very much like how a database works by adding to the end of the file.  This method helps guarantee that your file doesn't get corrupted if you have a power out or you disconnect your drive.  It does also mean that the file starts accumulating obsolete data.  At a certain threshold we recompress the file during a regular Save.

 

Save As always writes a new file with no obsolete data - so it generally always comes out smaller.

 

We also, when possible, writes out thumbnail bitmaps for previews to the end of the file.  This is a optional last step, and can result in the file being slightly larger.

 

But, if you make no changes to your file and hit Save, nothing should be happening, so the thing you are describing seems quite odd.  Is it possible to upload a copy of that file for me to look at? I am assuming you still get the same behaviour?

 

Thanks.

[Gear maker]

Matt, always a pleasure.  What you say makes sense.  And during a crash yesterday, when I hadn't saved it after a bit of work, this featured saved a bit of re-doing for me.  My only comment against it is that my final version I want everything, but in as small a file as possible.  That's why I'm using vectors instead of pixels, well one of the reasons.

 

Okay the file I was using as the test, let me explain.  My granddaughter has always been crazy about Sponge Bob Squarepants, you know "Who lives in a pineapple under the sea..."  So I would kid her about his brother "Who lives in a coconut under the sea, Sponge Bob Roundpants".  So I have been using this as an idea to test out the features in AD.  I usually draw gears and mechanical such, which doesn't make use of most of AD.  So...

 

Anyway attached is a copy of an earlier version of the file, 428.5KB.  I just tried it in the new Beta again just to make sure it still does it and cmd S made it jump to 721.8KB.  I unhid the layer for Sponge Bob and saved it dropping the size to 504.7KB.  Unhid the ocean saved it again 876.1KB.  Unhid all layers 680.5KB.  The file sizes really don't make much sense to me.  All I'm really after is the smallest file size when I finish.  And a Lamborghini in my garage but that's another story and I digress.

 

I had figured that if all it were saving were the vectors then unhiding shouldn't change the size and that's why I was looking at the size closely.  I often keep intermediate sketches in hidden layers, so even my gears can get fairly large in size when I was drawing them in Photoshop.

[Ben]

I've tracked the problem.  Seems there was a problem with our saving code that checked for changes before saving, so it would have saved your document every time even if it hadn't changed.

 

Should be fixed in the next Beta.

 

You will, however still see the behaviour where the file grows and shrinks when doing regular saves.  We have a 50% redundancy threshold before the file gets compacted.  In real terms, for small files like this, that means the file will potentially get compacted every other save.  For small vector files, even though the file doubles in size, the real size is fairly small (in modern terms).  I'm going to tweak that down to 33% for the next Beta. In theory, this means that for pure vector file (without any pixel layers) your file will always get compacted when you save.  Since vector files should be fairly small, this shouldn't impact too much on saving time.

[MattP]

...and I'd just like to say at this point - your Sponge Bob design is very cute! :)

[Jim Monson]

This has been a very informative discussion, answering a question I have had for a long time. My experience parallels yours, Gear Maker, and I guessed that something like this was going on. I tried to address it in https://forum.affinity.serif.com/index.php?/topic/20885-deleting-previous-save-problem/but your post formulated the question much better.  With you I deeply appreciate the AD folk explaining this. Such answer would never appear on the Adobe Illustrator forum.

[Gear maker]

Jim, glad it helped.  You are most certainly correct about Adobe's response or rather lack of.

[Rakesh K]

On 12/12/2014 at 10:39 PM, Ben said:

As Matt says - doing just a Save can result in the file growing most times, but then on occasion the file can end up smaller. This is because we save delta changes to the file to give to quickest save time possible.  Only the bits that change get saved.  We do this very much like how a database works by adding to the end of the file.  This method helps guarantee that your file doesn't get corrupted if you have a power out or you disconnect your drive.  It does also mean that the file starts accumulating obsolete data.  At a certain threshold we recompress the file during a regular Save.

 

3

Ben, that sounds very thoughtful and as an excellent way to avoid losing data but when I have to send a file to a client and find what should have been in 2MB is 70MB it seems plain ridiculous; it makes AD unusable for me. One solution is to provide an option to delete all those obsolete data while saving.

[Ben]

There should never be a situation where a file would be that much larger than required.

 

If you save history, yes the file will be big.  If you don't save history, at most the file will be 25% oversized if it hasn't been streamlined.  To cut a file down to it's leanest, do a Save-As.  The new file will be streamlined to minimum required size.  If your file is still 70MB at that point, then that is the size of your file.

 

Usually, files in the MB rather than KB will have raster data. The raster data has already been compressed, but in lossless form.  So, your main Affinity document files are never going to compare well to JPEG or other lossy file sizes.

 

Affinity document files are intentionally lossless, and preserve editability. The comparable increase in size in Affinity files is the same would be true for PSD files form Photoshop, if you imported from a lossy source.

[Jim Monson]

Very clear and helpful new posts. I always keep the larger doc of a project in the same folder as my current doc and then Save As to reduce the file size, saving both docs as backups ... needless, I know, but just in case. I use the smaller one to send to my colleagues.