Placed PDFs disappear (Publisher on Windows)

[David Still]

I'm compiling a book of sheet music (outputted from music software Musescore as PDFs), and placing the PDFs in Publisher. The book is some 250 pages long. All the PDFs are linked.

When I open the book, many PDFs are invisible, probably more than a third of them. They appear to be in their containers, but are not displayed. They're present in the Layers panel, but their thumbnail is empty.

First I had a master page with image frames in which to place the PDFs (to make it quicker to place the pages centered on each page), but then I suspected that this was the source of the problem, so I deleted the master page and just placed each PDF as a standalone object on each page. This didn't help.

I can't update them from the Resource manager (since the source file hasn't been changed), and the only way to fix it is to delete the PDF and place it again. And when I open the book the next time, they may or may not have disappeared again.

The missing page seems to always be the first page (or only page is the song in question only has one page). If I select the placed (invisible) PDF I can choose other pages of the PDF from the context menu, and they are visible, but the first page is invisible no matter what.

There is always a large amount of pages missing this way, and it seems to be random.

The only workaround I can think of is to just keep working, and then replace each invisible PDFs when it's time to export. Since I will probably be exporting several versions with corrections, this will be very labor intensive!

I searched the forums and found some topics that were a bit similar, but no identical problems. Some of these topics were several years old, and said that their respective problems had been solved by the next software update. I have the latest version of Publisher, so clearly there's something else going on here. Please help!

[MikeTO]

I don't know why they're not visible, but you could use Resource Manager > Embed and Make Linked as a toggle. That should bring them back and would be easier than deleting and placing them again.

Some questions that might help Serif or somebody else figure this out:

1. Does the Resource Manager show them as missing?
2. What happens if you open the document - see some missing pdfs - close the document and re-open it, and close it and re-open it? Are the ones that are missing always the same or is it random?
3. What happens if you open the document - see some missing pdfs - save the document under a new name and re-open it, does the new document have the same missing pdfs?
4. Are the PDFs set to passthrough or interpret?
5. Are the PDFs single page or multi-page, and what were they created with?
6. Where are the PDFs stored? (Internal drive, external drive, network drive, cloud...)

Good luck.

[David Still]

Thanks for your reply!

First off, making the PDFs embedded and then linked again from the resource managare does not work, unfortunately.

To answer your questions in order:

1. No, the resource manager doesn't show anything odd, it even shows a thumbnail of the relevant PDF (the Layers panel does not)

2. It seems to be the same ones missing every time. I also tried fixing a number of pages (i.e. deleting the invisible PDF, and placing it again), saving and closing, and opening again. Some of the pages that were missing before were missing again, some had not disappeared again.

3. Saving as a new name doesn't seem to help - in fact a few of the pages that had kept working after my experiment above were now invisible again - back to where we were before I re-placed any PDFs.

4. They are set to interpret. Switching between modes doesn't change anything. (And at any rate I'd want them as interpret to ensure a crisp print, right?)

5. I could have sworn the single page songs also disappeared, but upon closer inspection, it's clearly always the first page of a multi-page PDF that disappears. In fact, I can select any of the other pages of the PDF (say page 2 on the other side of the spread from page 1), and from the context menu set it to show page 1, and it will also be invisible. I.e., once the first page of a placed multi-page PDF is invisible, that is true of all copies of that PDF currently placed.

The files were created with Musescore, the open source notation software.

6. They are stored on the same internal SSD as the Publisher file, locally on my computer.

thanks again!

[MikeTO]

I don't know the answer but I think the information you've provided will be helpful to Serif or somebody else in figuring this out.

Good luck!

[David Still]

Another thing I just noticed: When I quickly zoom in and out on a page with an invisible PDF, parts of the contents are visible in glitchy flashes. So it's like it's still there, but Publisher is unable to display it properly. I've tried exporting a few pages to PDF (which will also be the final output when the book goes to print), to see if this just happens within Publisher, but no dice. The exported PDF also shows a blank page.

[kenmcd]

Music PDFs can only be placed as passthrough. 

Interpret attempts to edit the PDF, and music PDFs are a jumble of text, some text objects with bad or missing encoding, and a bunch of shapes - all stacked on top of each other. As soon as you try to edit it APub it just makes a mess. And stuff disappears because it is embedded as a text object, but the connection to the font is lost due to bad or missing codes. And the vertical placement is handled by the music notation application. There is no normal "text flow." Etc., etc. Not going to work.

You could also use Ghostscript to convert it all to shapes (and remove all font info). 

[David Still]

Thanks for your reply,

When I first tried to place the music PDFs as interpret (long before I had the problem of this thread) it was indeed a mess of missing fonts, those square symbols indicating unknown glyphs, etc. But once I installed the correct fonts from the music software so that Publisher could use them, the notes all display correctly. I don't think this is the problem, why would it only affect the first page of songs? All other pages, and singe page songs all work fine. And as I said above, switching the PDFs to passthrough doesn't solve the problem, the first page of multi-page PDFs are still missing.

At any rate, the rasterized result of passthrough is unacceptable, if that were the only option I would rather go through the entire document and re-place each missing page before final export.

[MikeTO]

If it renders a bit when you zoom in and out, what are your performance settings? Have you tried turning off hardware acceleration?

[lacerto]

On 1/21/2024 at 11:05 PM, kenmcd said:

Music PDFs can only be placed as passthrough.

That depends on the app that produces the PDF. I have still the old version of Avid Sibelius (version 7.1.3) which exports PDF that renders practically perfectly when interpreted or opened in Affinity apps. As Sibelius produces RGB output, interpreting or opening is practically the only choice if the print PDF needs to be in K-only (unless a tool that converts the output to K-only is available) since Affinity apps will either keep RGB when passed through, or if forced to CMYK, will produce four-color gray.

As for passthrough, Sibelius produced PDFs are v 1.4 and they pass through just fine, without rasterization, but as said, the output is RGB.

I have tested many score creation apps in Publisher during the recent years and it is true that PDFs created by some apps do not work at all even if all fonts are present (I guess it depends on encoding). Sibelius comes with 17 Opus notation PostScript flavor OTF fonts (by Avid) and at least the scores I have tested them with (only a few) they appear to behave very well in Affinity apps, both rendered and in the Glyphs panel.

On 1/20/2024 at 11:11 AM, David Still said:

There is always a large amount of pages missing this way, and it seems to be random

As for disappearance of placed PDFs (for interpretation), I did experience something similar, but that appeared to be after first placing all pages to be passed through and only after that switching to interpret mode one by one, and all pages except the last one disappeared both on canvas and in the Layers panel. But I saved the document and then opened it, and then all pages showed. Rendering however was a bit sluggish and I can imagine that having over two hundred pages to be interpreted can be a challenge. But it would seem that this is a different issue from yours as you experienced missing pages also when using passthrough mode (which I have not; but I would imagine that switching from interpret mode to passthrough when page is invisible would not necessarily make it visible).Anyway, when later switching immediately to Interpret mode (Publisher remembering the last used setting and keeping the mode), I did not experience disappearance of pages. But I only tested this with a score consisting of 19 pages.

I did not try to export to PDF at the time the pages were invisible, but I am surprised to learn that the pages were not visible in the exported PDF, either. Can you post any of such PDF on the forum to be able to have a look on the file (whether the actual content is there but for some odd reason just invisible)?

UPDATE: In my case, I was placing different pages of the same PDF document, copying already placed pages and then defining the page to be displayed from within the context toolbar. It is probably different from your situation where (if I understood correctly) most placed pages come normally from different (linked) PDFs. My files in context of Affinity apps are always on an internal SSD.

[David Still]

18 hours ago, MikeTO said:

If it renders a bit when you zoom in and out, what are your performance settings? Have you tried turning off hardware acceleration?

Thanks for the suggestion, but this didn't help. I also tried changing the renderer to WARP, but that didn't have any effect. Weirdly, I now can't reproduce the glitchy effect at all, even when changing back the performance settings.

 

17 hours ago, lacerto said:

As Sibelius produces RGB output, interpreting or opening is practically the only choice if the print PDF needs to be in K-only (unless a tool that converts the output to K-only is available) since Affinity apps will either keep RGB when passed through, or if forced to CMYK, will produce four-color gray.

Thanks for your insight!

Musescore also outputs RGB, so what I've done is add a selective color adjustment over the placed PDFs, subtracting CMY, and adding K. At least the eyedropper tells me this gives a perfect K100 result. I did try to have a page without the selective color, to see if that's what causes the disappearing pages, but it doesn't seem to make a difference. A bit off topic maybe, but is this a sound method to get K-only blacks?

 

18 hours ago, lacerto said:

As for passthrough, Sibelius produced PDFs are v 1.4 and they pass through just fine, without rasterization, but as said, the output is RGB.

I thought passthrough always rasterized. At least when I set the pages from Musescore to passthrough, they quickly get pixely when zooming in. I tried placing a PDF as passhtrough, saving and reopening, and at least for this song this worked to keep the first page visible. But when I switched it back to interpret, it vanished. Now I have the curious situation where for invisible pages from a previous session it makes no difference to switch between passtrough and interpret; while for this one page, which I saved as passthrough, switching does have an effect.

 

18 hours ago, lacerto said:

UPDATE: In my case, I was placing different pages of the same PDF document, copying already placed pages and then defining the page to be displayed from within the context toolbar. It is probably different form your situation where (if I understood correctly) most placed pages come normally form different (linked) PDFs. My files in context of Affinity apps are always on an internal SSD.

When a song is several pages, I also generally place it once, then copy it and change which page to display. When the page count is more than 2, I often don't copy the PDF for page 3, but rather place it again. I only do this only because it's a fraction of a second quicker than dragging a copy from one spread to the next. At any rate it doesn't seem to make a difference, as it consistently is the first page that goes missing. I also tried placing just the first page, making no copies (since the one page PDFs consistently have no problems), but these first pages also disappeared.

Attached is one exported spread. When I had the export dialog open, the preview even showed the transparent background behind the missing page (attached screen shot). It seems that it isn't just the glyphs that disappear, but even the white background.

 

2024-0122-Invisible test-03.pdf

[lacerto]

21 minutes ago, David Still said:

Musescore also outputs RGB, so what I've done is add a selective color adjustment over the placed PDFs, subtracting CMY, and adding K.

Can you post a sample of the output? If I try this, my output is rasterized and four-color black, but it may be dependent on combination of document color mode, settings and adjustment that might work. Anyway, I have the document color mode as RGB and simply just export using Gray/8 with document profile (=D50, basically Gamma 2.2), not embedding profile, and the whole file will be DeviceGray (= K100).

Your adjustment, btw, might be cause to the disappearance or glitchyness of the PDF display!

Your example showed that there really is no object so if it is invisible, nothing seems to be output. Did you apply the adjustment on the sample score? The notation was in four-color black while separate text parts were in black (K100).

[lacerto]

42 minutes ago, David Still said:

I only do this only because it's a fraction of a second quicker than dragging a copy from one spread to the next.

I see. I normally just copy paste one spread (two placed pages) at a time. Ctrl+C (Cmd+C) on one spread and Ctl+V (Cmd+V) on the next, since Affinity apps paste in place by default. There might be point changing Passthrough to Interpret after the first placement as Affinity remembers the setting for the rest of the placement in that document. At least it should reduce one manual one-by-one step from the workflow. 

[David Still]

Here is the PDF output directly form Musescore.

I also tried exporting a spread that didn't have the adjustment, using Gray/8 like you said, but it was still four color black when I opened the exported PDF in Publisher. At any rate, I've tested not using the adjustment to see if that is the reason the pages go invisible, but it happens whether I have the adjustment or not.

I haven't applied the adjustments (I'm not sure how). I didn't notice it before, but you're right that the notes revert to four color black when exported from Publisher, even though they show as 100K inside Publisher! (The song title and author names at the top of the page are text elements in Publisher, and not present in the Musescore output, as I wanted to control them with Text Styles. This is why they are correctly exported as 100K)

Invisible song-Musescore output-01.pdf

[lacerto]

46 minutes ago, David Still said:

I also tried exporting a spread that didn't have the adjustment, using Gray/8 like you said, but it was still four color black when I opened the exported PDF in Publisher.

Please make sure that you have the document color mode as RGB and export color mode as Gray/8, ICC profile as "Use Document Profile", do not check "Embed ICC Profile", and make sure that you force conversion of image color spaces. This way you should get the following:

Invisible song-Musescore output-01_apubgray.pdf

Note: The PDF above has obviously notation errors since I do not have the fonts installed.

BTW: I applied channel adjustment to block CMY and managed to output an empty PDF page, so this might well explain your dilemma!

[David Still]

It appears what you see is really not what you get. Not only does the selective color adjustment not make it to the export, but I just realized that even though the PDFs appear rasterized and blurry inside Publisher when set to Passthrough, when I export it the result is perfectly clear vectors! That would solve the primary problem of this thread, since at least from an initial test of a few dozen pages, it seems like Passthrough PDFs do not disappear! So maybe kenmcd was correct after all.

Thank you for taking the time to test my file! But when I open it in Publisher the eyedropper tool gives the CMYK values 72, 68, 67, 88, exactly where they started. Color management really is black magic to me, so I don't know what to tell you here. I'll have to keep experimenting, and at any rate ask the printer's what they recommend.

Of course I also discovered a new, unrelated problem. I don't know if I should start a new thread about it. When I export a spread from the Publisher document (with visible notes), it looks all right, but when I try to print the exported PDF from Acrobat Reader (both to paper and print to PDF), a bunch of elements in the music (the thick brackets at the start of each system, the thick barline at the end of the song etc.) print as thin lines, instead of thick. These elements are all lines with a stroke, not fill, so it seems Acrobat Reader resets the stroke thickness to some default value when printing, even though Acrobat Reader displays them correctly. But when I open the same file in Publisher it prints fine. I suppose this is another thing I should check with the printer's.

Anyway, big thanks for all your help! I'll try to get the blacks working properly, but at least I hope the disappearing pages issue is resolved!

[lacerto]

16 minutes ago, David Still said:

Thank you for taking the time to test my file! But when I open it in Publisher the eyedropper tool gives the CMYK values 72, 68, 67, 88, exactly where they started.

No, that happens if you open it in CMYK color mode, or if you have the sliders locked and read color values in CMYK mode. If you let Publisher estimate the color mode, if will open it in Gray/8 color mode. If you then set the sliders of the Color panel to Grayscale mode, you can see that all elements including text are Gray 0. 

This can easily be shown in Adobe Acrobat Pro:

Note too, that as I mentioned, passthrough will keep the job as vectors (non-rasterized), but it will also keep the job RGB (or if you force to CMYK, it converts the RGB black to four-color black). Some printers do not have problems accepting RGB jobs, so I would ask if it is ok, since otherwise it is normally safer to place PDFs for pass through.

If you do not have prepress tools to convert RGB black to K100, and the printer requires K-only, then your choice is to either open or interpret the original score and export explicitly to gray, as I have demonstrated. As I do not have your notation fonts installed, I cannot check how well Publisher interprets the score but it does it basically 100% correct when using Sibelius scores.

The correct export settings are as follows (just remember to have your document color mode RGB):

 

[kenmcd]

Regarding the disappearing pages...
In the sample PDF you provided there is a shape which encompasses the entire page which is RGB white (255,255,255).

I am wondering if that is confusing APub and it thinks that area should just be white.
And it just drops everything else.
We have seen a similar issue before.

I removed those shapes from the PDF attached below.
Invisible song-Musescore output-01-no-outer-box.pdf

Please test placing that by replacing your missing page with this PDF.

 

[lacerto]

I fetched the fonts you had in the demo document so that I can demonstrate properly the method of producing purely DeviceGray print PDF keeping all in vectors and text as text [or basically all as "text" since notation is done using fonts]. The clip below also demonstrates what happens if you choose to place RGB based PDF for passthrough: you either get RGB output, or if you force to CMYK, rasterized four-color output. You can try to apply Channel or Colorize adjustments and force output to K100, but you cannot avoid rasterization.

As mentioned above, you can of course convert everything to outlines (before or after) if you do not mind losing text (and probably creating bigger file sizes). But if you pass through, you will have the same color issues as mentioned above.

As it seems that Publisher renders very well also the fonts used in MuseScore notation, I would try if interpreting (without using any adjustments) works. If you continue getting "invisible PDF" issues, then passing through and producing RGB would be the best option, and then making sure RGB output is not a problem to the printshop. [It is practically a question of pushing e.g. Adobe Acrobat Pro button to convert RGB PDF to grayscale so any printshop should be able to do it.]. 

Invisible song-Musescore output-01_gray.pdf

UPDATE: I forgot to mention that I have hw acceleration on, and while I have experienced the disappearance issue two times when using interpreted mode [the second time only for one page when using a CMY blocking adjustment], this has not happened consistently, and it has not prevented creating fully working print pdfs (from a relatively small, 20 page publication). 

[David Still]

Thank you both for your time!

23 hours ago, kenmcd said:

I removed those shapes from the PDF attached below.

Invisible song-Musescore output-01-no-outer-box.pdf

Please test placing that by replacing your missing page with this PDF.

 

Your file seems to work, although I can't say for sure, as the issue happens randomly across many pages. This one page might just happen to work, and others might not. But having to delete the white background of each page of over 140 songs might not be that practical!

 

22 hours ago, lacerto said:

As it seems that Publisher renders very well also the fonts used in MuseScore notation, I would try if interpreting (without using any adjustments) works. If you continue getting "invisible PDF" issues, then passing through and producing RGB would be the best option, and then making sure RGB output is not a problem to the printshop. [It is practically a question of pushing e.g. Adobe Acrobat Pro button to convert RGB PDF to grayscale so any printshop should be able to do it.]. 

 

I'm not sure I understand the difference between interpret and passthrough as regards to color modes and process black. If I pick passthrough, can I not export in grayscale as you explained above? In any case, it looks like interpret doesn't work even without the adjustment layer - pages are still disappearing. But since passthrough seems to be working, I think that's my best bet.

EDIT: If I do export it in grayscale as you suggested, the text elements (song title, page number etc.) which are 100K end up as gray 15 or so, whereas the RGB black of the music end up gray 0%. I suppose if I want to export to gray I should convert the document to RGB, as you say, and also change all the text elements to true black #000000?

But kenmcd's tip about removing the white background got me thinking - the separate PDFs for all songs come to a grand total of under 10MB, so maybe embedding them rather than linking would make editing them easier than opening each one separately and saving it sans white background. That's when I discovered that if I edit an embedded PDF that has a page missing (by double clicking it) the missing page in the edit view really is missing - it has no contents in the layers panel, and the size is just a tiny square. See attached screenshot. I don't know if that tells us anything.

 

 

[lacerto]

14 minutes ago, David Still said:

If I pick passthrough, can I not export in grayscale as you explained above?

No, Affinity "passthrough" means that the color mode is retained (contrary to many other similar apps that pass through PDF files). This is demonstrated on the video and can be easily tested. Only interpreted (and opened) PDFs can have their properties changed.

The invisibility issue is odd, and I cannot reproduce it consistently, but if you can, and as it is also an issue in the exported PDF, i do not think that you have other options than using passthrough. As mentioned, it is basically the foremost choice to work with PDFs though there are exceptions. I would really ask the printshop if RGB output is fine -- as mentioned, it is very easy for them to convert RGB to grayscale, but some printers have so automated workflows that they do not accept this, so it is best to make sure.

If you pass through, I do not see the point of editing the files (especially if you do not experience the invisibility issues when passing through).

[David Still]

I see. Well, at least I have a few options for workarounds for the invisibility, and I'll have to contact the printshop to make sure I can prepare a useful file for them. Thanks again for all your help!

[kenmcd]

1 hour ago, David Still said:

That's when I discovered that if I edit an embedded PDF that has a page missing (by double clicking it) the missing page in the edit view really is missing - it has no contents in the layers panel, and the size is just a tiny square. See attached screenshot. I don't know if that tells us anything.

It may. I remember a previous issue where APub confused the stacking of the objects in a PDF, and then just showed the blank area. IIRC the thinking was if all of this is hidden it does not need to be shown or even processed so APub appeared to just drop it (it is not visible why even deal with it).

You can test this by removing that shape from some more of the other missing-pages PDFs and see if they also now show correctly. If every one you remove the shape from now shows correctly that would be pretty good evidence that is the issue.

[lacerto]

8 hours ago, David Still said:

Your file seems to work, although I can't say for sure, as the issue happens randomly across many pages. This one page might just happen to work, and others might not. But having to delete the white background of each page of over 140 songs might not be that practical!

If you had experienced the invisibility issue (especially one that results in invisible exported PDF, as well) only or principally after having applied an adjustments trying to block CMY in an RGB based document, then having an RGB full page size white background rectangle (basically without a reason) might be a likely trigger for this error. But I understood that you have experienced this, similarly as I did, also without applying any adjustments, as long as there is an interpret mode PDF placement involved. But then this would appear to be more a rendering issue and something that I would expect would not happen in an exported PDF. This might somehow be also a system specific issue (even if you mentioned that disabling hardware acceleration in the app performance settings has not had any effect on the problem).

[David Still]

20 hours ago, kenmcd said:

You can test this by removing that shape from some more of the other missing-pages PDFs and see if they also now show correctly. If every one you remove the shape from now shows correctly that would be pretty good evidence that is the issue.

If a page has already disappeared, then I can't remove the background rectangle. After embedding the file and double klicking it to enter edit mode, the missing page is just a small square with no contents at all in the layers panel (see screen shot above). But if I remove the white background before the page disappears (i.e. directly after placing the document, but before saving and reopening), it seems to fix the issue (at least so far), so that still points to the background being at least part of the problem.

13 hours ago, lacerto said:

If you had experienced the invisibility issue (especially one that results in invisible exported PDF, as well) only or principally after having applied an adjustments trying to block CMY in an RGB based document, then having an RGB full page size white background rectangle (basically without a reason) might be a likely trigger for this error. But I understood that you have experienced this, similarly as I did, also without applying any adjustments, as long as there is an interpret mode PDF placement involved. But then this would appear to be more a rendering issue and something that I would expect would not happen in an exported PDF. This might somehow be also a system specific issue (even if you mentioned that disabling hardware acceleration in the app performance settings has not had any effect on the problem).

I realize now why the test file I posted above had the notes in four color black even when I had the adjustment on - I had the export settings set to 'rasterize nothing', which I guess makes it ignore the adjustment altogether. If I had set it to 'rasterize unsupported elements' I guess the adjustment would have worked, but the notes would have been rasterized? So I'm throwing away the adjustment in any case - turns out it is not a sound way to force K100!

But the evidence points to the adjustment layer NOT being the cause of the invisibility. But I now have two workarounds that have worked so far: either using passthrough, or embedding and manually deleting the white backgrounds. That will do for now, and I will consult with the printshop about how to deal with the RGB black!

Thanks again!

 

[lacerto]

49 minutes ago, David Still said:

embedding and manually deleting the white backgrounds

If you are sure that it is the white background rectangle that causes the invisibility issue (and your printshop cannot accept RGB files), you might want to consider purchasing a low-cost PDF editor (PDF/X-Change, the standard edition of which costs around EUR50, a perpetual license). A proper PDF editor (unlike apps that open a PDF and convert it to native objects for editing, like Affinity apps) can be used e.g. to simply just select and delete objects and save the document otherwise intact. That would make it reasonably fast to remove the background from the PDFs.