[Implemented] Data merge

[spidermurph]

I still use PagePlus for data merge and DrawPlus for trace. In fact there is a whole list of features that I call upon the Serif programs for because they are still, after all these years, missing from the Affinity suite. Nearly every job we start in Affinity has to be ultimately exported and finished in Serif.

So my advice would be try and get you hands on the legacy Serif programs. They will be super cheap now and will have the features we have been waiting for.

[undercovergypsy]

I do the same, however,  for some reason they both tend to crash on my computer at some point in a project. It is most likely my machine but I don't have the same problem with the Affinity software. I do agree, if a person can get Legacy software, they should give it a try. 

[Patrick Connor]

Readers of this thread may be interested to try the first beta implementation of Data Merge in the latest Affinity Publisher Beta

You can find Affinity Publisher Customer Beta 1.9.0.796 for macOS HERE and for Windows HERE

If you are coming to this post late the latest customer beta can be found in the last post in the relevant one of these announcement threads on macOS or Windows

[undercovergypsy]

Thank you and HOW EXCITING!!! I can't wait to figure out how to use it. I am assuming it is not up and running now - not the beta - the merge tool? Thanks for the heads up.

[PaulEC]

I'm really pleased that Data Merge has been added. Now all I have to do is fathom out how it works! 😉

[ajpearceUK]

Agreed! We need a howto.

[garrettm30]

16 minutes ago, ajpearceUK said:

We need a howto.

There will surely be some documentation and probably some videos once the feature is released. For the moment, as it is in its first beta appearance, we can expect things to be rough. Betas are not released so people can use the features early, but rather for those who are willing find the bugs and report them. And not just bugs, but also feedback about how something is implemented.

[dominik]

21 minutes ago, ajpearceUK said:

Agreed! We need a howto.

Perhaps you can give the developers a moment of breath to come up with some explanations 😉  But there is nothing wrong to gather together some insights on how this works by us forumsters.

Actually it is worth going over to the Mac and Windows beta forums, because there are already some things to read (and learn). Among them it turns out that merging image paths seems not to work on Windows, currently. But there is also a short video showing how that works on the Mac.

Basically it works like this:

1. There's a new menu entry 'Data Merge Manager...' where you first set up different (or just one) data sources. These can be .csv or .json files or Excel tables.
2. Then you go to 'Studio > Fields' where you find at the bottom of the list these data sources and their fields.
3. Double clicking on a field name (this is important) inserts a field placeholder into a text element (text frame or artistic text). Note: this also works for image frames if the source contains absolute paths to image files (but with a bug on Windows).
4. Finally, once your design is finished, you return to the 'Data Merge Manager...' and click on 'Generate'. This creates a new and unnamed document with the merged data.

There is also an additional new tool called 'Data Merge Node Tool'. With this you can create grid layouts that exactly distribute mulitple repeats of one base layout. It is very similar to a label printing setup in a word processor.

Cheers,
d.

[Frozen Death Knight]

50 minutes ago, garrettm30 said:

There will surely be some documentation and probably some videos once the feature is released. For the moment, as it is in its first beta appearance, we can expect things to be rough. Betas are not released so people can use the features early, but rather for those who are willing find the bugs and report them. And not just bugs, but also feedback about how something is implemented.

Indeed, but to be fair, it is going to be hard to test if people don't know how it works yet. Some pointers are always appreciated. ^-^

[garrettm30]

3 minutes ago, Frozen Death Knight said:

Indeed, but to be fair, it is going to be hard to test if people don't know how it works yet.

I don't disagree, but I also think the process of trying to figure it out is of value for UX and UI feedback.

[dominik]

29 minutes ago, garrettm30 said:

but I also think the process of trying to figure it out is of value for UX and UI feedback.

I completely agree. My first experience was that it seemed to be really a little hard to figure out even if it looked simple. This first implementation has rough edges but essentially it works (apart from the image path merge bug on Windows). It will be interesting to put it to some thorough testing with multiple data sources and more complex layouts like catalogs (which is something I personally have no experience).

d.

[PaulEC]

I think I've got it figured out now, thanks to dominik's guide above. Hopefully Serif will get the photo merge (or whatever they call it!) working on Windows soon. It will be interesting to see if it will be possible to combine data and photo merge in the same document.

[MikeW]

7 minutes ago, dominik said:

... It will be interesting to put it to some thorough testing with ... more complex layouts like catalogs (which is something I personally have no experience).

The method used will not work for anything resembling a "real" product catalog. Only simple "product listing" types of catalogs.

This is also true of native InDesign, QXP and PagePlus, et al, catalogs. Which is why true add-ons for catalog creation and maintenance exist for ID/QXP. Which cost from $750 and up.

Even the types of merges I do cannot be made in the current APub implementation--and I haven't even downloaded the beta.

[dominik]

3 minutes ago, MikeW said:

Even the types of merges I do cannot be made in the current APub implementation

I know (or at least assume from reading your posts) you have a lot of experience with complex publications like catalogs and other complex publications. I would be interested what 'types of merges' you are refering to. Really out of interest.

d.

[wonderings]

2 hours ago, dominik said:

Perhaps you can give the developers a moment of breath to come up with some explanations 😉  But there is nothing wrong to gather together some insights on how this works by us forumsters.

Actually it is worth going over to the Mac and Windows beta forums, because there are already some things to read (and learn). Among them it turns out that merging image paths seems not to work on Windows, currently. But there is also a short video showing how that works on the Mac.

Basically it works like this:

1. There's a new menu entry 'Data Merge Manager...' where you first set up different (or just one) data sources. These can be .csv or .json files or Excel tables.
2. Then you go to 'Studio > Fields' where you find at the bottom of the list these data sources and their fields.
3. Double clicking on a field name (this is important) inserts a field placeholder into a text element (text frame or artistic text). Note: this also works for image frames if the source contains absolute paths to image files (but with a bug on Windows).
4. Finally, once your design is finished, you return to the 'Data Merge Manager...' and click on 'Generate'. This creates a new and unnamed document with the merged data.

There is also an additional new tool called 'Data Merge Node Tool'. With this you can create grid layouts that exactly distribute mulitple repeats of one base layout. It is very similar to a label printing setup in a word processor.

Cheers,
d.

Is the only option to create a new document in Publisher? No direct to PDF?

I know in Indesign you can do both. I find making a new Indesign document from the merged file can take a lot longer then exporting a merged file to PDF. 

 

[undercovergypsy]

Thank you Dominik - I don't see the Excelf file that I chose in the Data Manager - can someone tell me what i am doing wrong?

 

[dominik]

1 minute ago, wonderings said:

Is the only option to create a new document in Publisher? No direct to PDF?

Currently 'Generate' generates a new APub file. Of course that one you can eyport to PDF 😉

The new 'Data Merge' feature really is in its early stages, though.

d.

[dominik]

4 minutes ago, MikeW said:

In this example:

1. The headings (here, Diffuser Supplies & Essential Oils) are exported from the database for each entry but only once appear at the beginning of their section.
2. The repeating text shading repeats. However, the darker text shading always begins anew after a new section starts.

Thanks, that is a very good example to understand what you mean. It's a good starting point to poke into APubs data merge and see if that can be achieved (I'm sceptical at the moment).

d.

[dominik]

12 minutes ago, undercovergypsy said:

Thank you Dominik - I don't see the Excelf file that I chose in the Data Manager - can someone tell me what i am doing wrong?

I just tried and a Excel file shows up just as a .csv did.

Make sure you save in .XLSX format.

d.

[dominik]

4 minutes ago, MikeW said:

Without scripting that can have logic applied to the incoming data

This is why I am sceptical at the moment 😉

I'll try and see if I can find something with sections and master page elements. But not today ... (yawn ... sorry).

d.

[undercovergypsy]

Thanks — I finally got it to show up and the fields. I jumped the “generate” gun and it was not showing up. That being said, — I have double-clicked each field and it does nothing. I tried clicking the text tool first and still nothing? I am an idiot...and so for now,  it is not idiot-proof. I will however keep, as someone else said, “poking away” at it. It is so great to have this opportunity! 

[Murfee]

21 minutes ago, undercovergypsy said:

I have double-clicked each field and it does nothing

Have you tried double clicking on the field name?

[dominik]

6 minutes ago, MikeW said:

Speaking of Master pages, in this example (which was adapted from a real job), I need to switch master pages during the merge.

Thank you for sharing.

I think it will take a long time (and a lot of expertise) until it will be - with APub - possible to achieve such a workflow.

Cheers,
d.

[wonderings]

**please delete somehow double posted**

 

[wonderings]

15 minutes ago, MikeW said:

Speaking of Master pages, in this example (which was adapted from a real job), I need to switch master pages during the merge.

I think I've used this example, too. Here I've included a screen shot of the simple code required. The logic sets the master page(s) and the first three words of the blurb to the company's color. The top background color and company logos are master page elements.

Would be nice to have an "if this then that" feature in the data merge but I think it is probably very early for that. You can do that in Indesign but it is a round about way as it is not directly supported. Think you are basically creating scripts in the csv file that then turns on the layer you need and the other layers off.

What are you currently using to accomplish this? If I need to do something like this I have SmartStream Designer. Fairly knew to it at this stage but it is pretty powerful and can do what you are asking really easily. Of course that comes with a price tag