Data merge that isn't just labels

[Loquos]

I absolutely love that Affinity Publisher now has data merge. It's definitely better than what Adobe was offering a million years ago. But after trying it I realized I was still going to be stuck with multiple text frames when what I really needed was for all of the merged data to flow from one columned text frame to another, page by page.

When you're dealing with hundreds of entries, there's nothing quite as mind-numbing as putting everything back together so that you can adjust for pagination, unused space, etc. 

Well, my husband (who writes code for scientific database uses) offered to make a little bit of Pearl code for me that I could then use to do my own data merge, which results in a simple .txt document being created with all of the info exactly as it needs to be all in one place that I can then copy and paste into Publisher and quickly do a bit of hand formatting (using a combination of find/replace for specific characters and and applying paragraph styles).

It has made my life a million times simpler - though it still means if my client comes back to me with a change, I either have to manually go in to make the update (which can sometimes require updated pagination), or redo the entire directory.

I realize Publisher's data merge functions are still in beta, but I would be ecstatic to know there's something in the works to have the resulting data merge exist in one continuous text frame.

Barring that, would there be a way my husband's code could mark up the data merge in such a way that when I copy and paste it into Affinity, Affinity will automatically apply certain formats? Or is that a bit too outside the box?

I'm attaching the little program here. For anyone who understands these sorts of things. It's custom made for me, so don't expect to be able to download as is and use it for your project, unless you know how to change the code.

convert_directory.pl

[wonderings]

Not sure I am understanding what you are expecting or wanting data merge to do. You would set your data fields on the page, then created your merged document, so each page has the next record from the csv file. 

Are you wanting to merge onto one page all records, ie record 1-10 - page 1? Not record 1 - page 1, record 2 - page 2, etc. Can you show an example of your desired result?

Also I don't think it is saying much to compare Publisher to Indesign years ago, not even sure why there would be comparison to anything but what is out now. 

 

[Old Bruce]

4 minutes ago, Loquos said:

Barring that, would there be a way my husband's code could mark up the data merge in such a way that when I copy and paste it into Affinity, Affinity will automatically apply certain formats? Or is that a bit too outside the box?

 

You could try setting up Paragraph Styles for each paragraph and each will call the next.

ParaSt 1

2

3

4

ParaSt 1

....

That would work for four paragraphs per record.

I do think that using the perl code may be the best method.

[Loquos]

1 hour ago, wonderings said:

Not sure I am understanding what you are expecting or wanting data merge to do. You would set your data fields on the page, then created your merged document, so each page has the next record from the csv file. 

Are you wanting to merge onto one page all records, ie record 1-10 - page 1? Not record 1 - page 1, record 2 - page 2, etc. Can you show an example of your desired result?

Also I don't think it is saying much to compare Publisher to Indesign years ago, not even sure why there would be comparison to anything but what is out now. 

 

Old comparison because that's when my knowledge of what Adobe InDesign could do. So no idea what they are doing now.

I have records of varying length. I want each entry to have one line break between them, and then flow from page to page. Some pages may end up with 10 records, some with 5, some with 13. Unless I'm missing something, the current beta merge means that each new record gets its own text frame - which means I have to manually move all the text frames, or move all the text into one text frame.

But if I'm missing something, please let me know! I'd love to know how to do a data merge that doesn't make massive amounts of text frames as a result.

[Loquos]

1 hour ago, Old Bruce said:

You could try setting up Paragraph Styles for each paragraph and each will call the next.

ParaSt 1

2

3

4

ParaSt 1

....

That would work for four paragraphs per record.

I do think that using the perl code may be the best method.

There are only certain lines that need to be formatted a certain way. It's still a problem because I can have anywhere between 5 and 15 records per page. So I guess Pearl is the way to go!

[MikeW]

5 hours ago, wonderings said:

Not sure I am understanding what you are expecting or wanting data merge to do. You would set your data fields on the page, then created your merged document, so each page has the next record from the csv file. 

Are you wanting to merge onto one page all records, ie record 1-10 - page 1? Not record 1 - page 1, record 2 - page 2, etc. Can you show an example of your desired result?

Also I don't think it is saying much to compare Publisher to Indesign years ago, not even sure why there would be comparison to anything but what is out now. 

Quick. I'm making a price list with records into the thousands...

I want all records to come into a flowing series of text frames from the master page(s) and be formatted. I don't want a thousand or more text frames across countless pages.

Pretty common scenario for me.

With ID, one can use a script to merge into linked text frames. I generally use QXP, but in both Q and ID I opt to use an XTension/Plug-in to deal with the merging as it handles expressions, etc.

Merging into 1 Frame per record is fine for mailing labels. But that's about it.

[wonderings]

11 hours ago, MikeW said:

Quick. I'm making a price list with records into the thousands...

I want all records to come into a flowing series of text frames from the master page(s) and be formatted. I don't want a thousand or more text frames across countless pages.

Pretty common scenario for me.

With ID, one can use a script to merge into linked text frames. I generally use QXP, but in both Q and ID I opt to use an XTension/Plug-in to deal with the merging as it handles expressions, etc.

Merging into 1 Frame per record is fine for mailing labels. But that's about it.

Never come across needing it that way, all the data merge I use is for mailings, name tags, large business card orders, etc. Could you not just import the excel file as a list in a text box?

[MikeW]

4 hours ago, wonderings said:

Never come across needing it that way, all the data merge I use is for mailings, name tags, large business card orders, etc. Could you not just import the excel file as a list in a text box?

Sure, the data could be massaged in Excel, copied and pasted into a text editor or a text frame. Then one could manually apply styles.

Or, one could just use something else that doesn't have such limitations. Then one could go from this...

To this...

In a mere couple seconds for 100+ pages.

We as layout people often have choices.

 

[wonderings]

8 minutes ago, MikeW said:

Sure, the data could be massaged in Excel, copied and pasted into a text editor or a text frame. Then one could manually apply styles.

Or, one could just use something else that doesn't have such limitations. Then one could go from this...

To this...

In a mere couple seconds for 100+ pages.

We as layout people often have choices.

 

The screen shot helps, I never thought about using it this way as I have not done a lot of catalogues like that. That makes sense to have that function for that purpose, would save a ton of time. Does Indesign have this function or is this something neglected there as well?

[MikeW]

1 minute ago, wonderings said:

... Does Indesign have this function or is this something neglected there as well?

The same Xtension I use in QXP is also available as an ID plug-in.

Native merge in ID is akin to APub's. But, because ID is built with plug-in architecture, there are plug-ins (like I use) that can extend functionality. Because ID supports scripting, there are free and pay-for scripts that can in-line merge.

As much as I use merging in Q, I also use tagged text as much. Sometimes merge & tagged text is combined. The above example could be done via tagged text for what is seen. However, the Excel sheet(s) are actually complicated (more involved) and also contain other fields that are used for in-store flyers, direct mailers, etc. So it is just easier/faster to pull just the data I require for each type of product versus manipulating the Excel sheets.

[witzend]

I'd just like to offer massive support for the idea of being able to merge into a single frame/textflow. Sure, it's great to be able to merge one record to one text frame BUT that's so limiting. A textflow can easily be split into multiple text frames, but it's a nightmare of copy/pasting to turn multiple text frames (and it can easily be hundreds) into a single textflow.

I suspect I use the same QXP/ID extension as MikeW: it's amazingly fast and powerful (though it's script based, so not exactly intuitive) and I keep a 15 year old version of QXP just to use it. It instantly converts databases/spreadsheets into really typographically complex lists (think menus, wine lists, catalogues etc) and is very flexible. But the new Publisher 1.9 data merge feature is most of the way there: so close, it's really frustrating. It just needs to offer the choice of single/multiple text frames to offer most of the functionality. (It would be nice to be able to store the 'Fields' settings as a template, too, to save re-inputting them each time).

Big vote for this, please! Like I say it's just a little change ... then I could finally let QXP7 and the ancient Mac mini/SnowLeopard setup go into dignified retirement.

[philipt18]

I'd also like this feature. As I see it I'd like to be able to import various fields, and have the text flow into a continuous text. Imagine a dictionary, for example. Each Word might have multiple components - the headword, the etymology, the pronunciation, the definitions, etc. Maybe some words don't have all of the components, so in those cases they would be left out. But after one word is finished, the next word begins.

[kjdavies]

I'd very much like to see the same. I've got data sets with some standard information (which is summarized in a block), followed by some number of paragraphs. It would be lovely to get them all in one flow so I can have as many per page as fit, with breaks happening in appropriate places as they flow to other pages.

I'm doing this now by having a script that punches out a big Word file, then copy and paste that into APub. It works, but it's one more step.

On the other hand, it allows me to have formatting and arbitrary content options not easily done via data merge, so this is a 'nice to have' or a 'really nice to have' for me at this point... but it would be a great thing to have available.