Publisher 2 and ICML files

[PaoloT]

Hi,

After the new version of Pandoc (v3.0) has been released, I have to reiterate a request for ICML import to Publisher. Pandoc v3 includes an improved ICML converter.

This converter would make going from (nearly) any markup file format to Publisher with no effort. If the source file contains text layout instructions (as you can do with Pandoc markdown, HTML, LaTeX, or Scrivener) Publisher could rebuild the layout faithfully.

ICML is a subset of the IDML file format that Publisher can already import. It is just one of the "Story" XML files inside the IDML package. Unfortunately, there is no way that I could find to convert an ICML file into an IDML one, so at the moment Publisher can't take advantage of this Pandoc converter.

HTML is universally used for the web, webhelp and ebooks. Pandoc and LaTeX are in strong use in the technical industry and in the academia. Such a converter would allow Publisher to be the final tool to generate PDF files from internal or end-user documentation, research materials, or web sites.

Paolo

 

[PaoloT]

I found a way to convert ICML files generated by Pandoc to IDML files that Publisher can read. This is tried on very basic files, so I don't know how it works on real, long and complex files.

First of all, I've exported from InDesign CS6 a very basic IDML template. It contains nothing but the default settings generated by InDesign on a new document.

IDML-Template.idml.zip

Then, from Scrivener I've compiled a simple Pandoc-flavored markdown file, containing a text that I want to load in Publisher.

Scrivener-source-in-Pandoc.md.zip

I then used Pandoc to convert it to an ICML file. I used the simplest of the commands, generating the headless output in the Mac Terminal:

pandoc -f markdown -t icml input_file.md

With BBEdit, I opened a copy of the IDML-Template.idml file, and replaced the text included inside the ParagraphStyleRange tags:

It is very important that the "Story Self=" tag matches the name of the current Story, as shown in the list of XML files inside the IDML package.

The resulting IDML file opens fine in Publisher (V1):

It would be great if Publisher could at least do the ICML Story replacement in an IDML template, and import it as an IDML file. Even better, it would be if it could even trigger Pandoc when importing a Markdown file, if installed on the system. It seems to be a very easy filter to program.

Paolo

 

[PaoloT]

I've done a little try with letting Pandoc read a web site, and convert it to ICML. I then transformed it into an IDML file, as explained above. Shame I don't know how to convert the CSS (also linked) into Publisher styles.

 

IDML-Template-with-Story-from-Web.idml.zip

 

 

[walt.farrell]

33 minutes ago, PaoloT said:

Shame I don't know how to convert the CSS (also linked) into Publisher styles.

Shouldn't the style info be in the ICML files created by Pandoc?

[PaoloT]

8 hours ago, walt.farrell said:

Shouldn't the style info be in the ICML files created by Pandoc?

As far as I can see, text style definitions are included in the ICML file, just at the beginning. But then they are not in the Publisher file. The same with InDesign CS6.

In the past I've done some other tests, and text styles were converted. So, I don't know what is going wrong here.

Paolo

 

 

[PaoloT]

Maybe I can guess why the text styles have not been converted. By reading the ICML filter code, it seems that text styles should have a predefined name. Custom names seem to be allowed, but they have to be declared.

https://github.com/jgm/pandoc/blob/main/src/Text/Pandoc/Writers/ICML.hs

This would explain why I was successful when trying with an original file (in Scrivener) adhering to these specs, and they don't work when grabbing a web site with too different style names.

Paolo

 

[PaoloT]

Something I noticed, while exploring Pandoc, is that the concept could be generalized. It is not just "from Markdown to Affinity Publisher", but "from Pandoc to Affinity Publisher". Pandoc is an incredibly capable tool, opening a world of conversions.

So, an "Import Pandoc" command could let one choose the starting file format, the source file, and end up into Publisher through an intermediate conversion to ICML/IDML. Or through a new Pandoc filter expressly conceived for Publisher.

Paolo

 

[PaoloT]

I discovered a way to avoid the command-line conversion from .md to .icml. There is a smart text editor called PanWriter, that acts as a graphical user interface for Pandoc. Conversions can be done there.

Paolo

 

[Print Monkey]

I'm interested in this as well. I don't need ICML specifically, but I want to be able to create a page layout in Publisher and import XML marked up text which has tagged stuff like "this is a chapter", "this is a section", "this is a paragraph in the section". That is, logical document layout markup imported into Publisher which then decides what styles and masters to apply.

[PaoloT]

33 minutes ago, Print Monkey said:

import XML marked up text which has tagged stuff like "this is a chapter", "this is a section", "this is a paragraph in the section". That is, logical document layout markup imported into Publisher which then decides what styles and masters to apply.

This is exactly what you could do with a markdown document, converted via Pandoc into a format that Publisher can understand. At the moment, the most immediate way of doing it for structured text is converting markdown to DOCX, and then import it into Publisher.

If you need more sophisticate layout, I guess the only way is to follow the ICML-to-IDML path described above.

Paolo

 

[JMaxSFU]

I've had a pandoc-to-InDesign workflow for years, using icml. It would be fabulous to be able to use Affinity this way, since I've parted ways with InDesign entirely now.

My workflow runs articles (from markdown or html) through pandoc to icml files, which are then 'placed' in custom indesign templates with master pages and paragraph/character styles already set up. That allows me to run the same content to the web (for CSS styling) or to print (with Indesign, or ideally, Affinity Publisher) doing the styling.  Best of both worlds. No compromises.

Currently playing with Typst for thmye print output, which is actually wonderful... but for anything visual, there's no substitute for a visual DTP program.

[PaoloT]

16 hours ago, JMaxSFU said:

It would be fabulous to be able to use Affinity this way

Absolutely. The Pandoc to ICML converter is good enough, if the source materials are correctly made and not too complicate from a page layout point of view.

Publisher could be the perfect PDF formatter for a workflow based on Markdown. Typst is great, but it is much less flexible than Publisher.

Publisher could already be the perfect solution for technical and academic documents to be released in different formats.

I'm sure it wouldn't be exceptionally difficult for Serif to extend the IDML importer to include ICML. All considered, it is the same format, only a headless one.

Paolo

 

[PaoloT]

As a side note: an ICML import filter would not be really needed, if Publisher could import and interpret Markdown…

Paolo

 

[adamrice]

I'll add my voice to this request. I've been using Pandoc and Pandoc-flavored Markdown for years. A direct Markdown > Publisher workflow would be ideal, but I can deal with ICML > Publisher. I could deal with DOCX > Publisher, but without having tried it, I imagine it introduces additional cruft.

[JMaxSFU]

"Small parts loosely joined," is a good old design principle. ICML is pretty pared-down: it's basically just paragraph- and character-style-ranges surrounding text, and some hooks to style definitions. It's pretty fit to purpose. And it is an open standard. Or at least it's transparent.

All Publisher would need to do is read it.

[PaoloT]

On 2/29/2024 at 11:14 PM, JMaxSFU said:

All Publisher would need to do is read it.

Well, they should already have the code. ICML being nothing else than the text container of an IDML file.

Paolo

 

[PaoloT]

Being in the middle of a series of tests for going from Markdown to InDesign, I have to say: yes, having an ICML importer in Publisher would make it a viable endpoint for a Markdown workflow.

This would make Publisher a favorite for single-sourcing, multi-output projects. That is, all the end-user documentation produced nowadays.

And i remain convinced that adding that Place option would be just a matter of reusing code they already have…

But – yes, maybe all the work could be devoted to a direct Markdown converter. All considered, the Pandoc ICML converter is not very flexible, is very barebones, and is under external control. Starting to implement Markdown from the most basic features would be a lot better in perspective.

Paolo