Jump to content
You must now use your email address to sign in [click for more info] ×

[Implemented] Data merge


Recommended Posts

5 hours ago, Mark Daniel said:

It's not currently possible to use more than one data source to merge a single output page, I'm not convinced that's desirable, it certainly would create a lot of complications regarding advancing the merge index and running out of records.

I can´t assume any complications. When the record is empty Publisher will ignore / write "" in it. So it is up to the User to fill the source File.

I think the best would be, if Publisher gets some sort of "Logic" like: If <mergefield1> = "" or NULL then "" else <mergefield1> and those Fields are formated. If there are empty, the field will not be skipped. For an count / specification-List it would be great. At the moment, if you have a record with 8 fields and another with 7, the 8. will only be the "dot" and nothing there.

This is the only thing we are missing.

Currently we have an excel sheet that gets the Source-files, format and arrange them with Formulas and link that file to Publisher. It is a Product Catalog. Perfect tool. You can even Point to an "afphoto"-File that will be linked in the Document and displayed there -> Picture frame. 

Link to comment
Share on other sites

6 hours ago, Mark Daniel said:

The multiple sources feature is designed for switching between alternatives (eg a small test and larger production data source) or for merging different page ranges in a document with different data sources (eg a catalog where each chapter has a different template page and a different data source file).

It's not currently possible to use more than one data source to merge a single output page, I'm not convinced that's desirable, it certainly would create a lot of complications regarding advancing the merge index and running out of records.

@jdo if you could explain your use-case more fully perhaps I can suggest a way to make it work.

Hi @Mark Daniel,

thanks for your excelent reply which makes things a lot more clear to me. I think I'm now able to understand the idea behind the whole function.

It is just a suggestion, but in my humble opinion, an error message (at any point) might be handy for a user while trying to insert data from a second data source due to the fact that actually nothing really happends when trying to insert data from a second source. In the described situation I asked myself if I have overseen something obvious, but perhaps it's just me. 🙃

 

It is true that I tried to use the data merge function for something it was probably never designed for: For creating automated reports. I really love LaTeX, a tool that was definetly not designed for creating fancy brochures or reports. My actual LaTeX dataflow looks like that:

  • A python script collects data from several sources and evaluates a lot of data / variables from them
  • The script also copies a default LaTeX document to a specified folder and changes
    • variables like company name, dates, etc.
    • pathes to images (or for being more precise, no absolute file names are used, so instead of adding images from C:\myfolder\default_template\graphs, images are just lying in the "graphs" folder of the actual folder where the python script adds some graphs to)
  • Due to the fact that I can split a document ober several files, it might just overwrite parts of the document with other files (that might be specified for a special customer)
  • It adds a dynamic number of tables at some parts of the text with a pre-defined number of rows and columns (think e.g. on evaluations for departments, where a table is created for each department)
  • Adds some text passages for different purposes

I really like Affinity Photo and Designer, so I just wanted to give the Publisher a fair chance and look how a usecase might look there. Why? Because for being honest, it is kind of a pain to create nice-looking documents with LaTeX. (I hope I didn't offend any member of the Science / LaTeX community with my last scentence. Of course LaTeX creates really great looking documents for many use-cases in science but not for highly customable design which was, and will probably never be, its purpose what's absolutely fine. It's possible, but it's hard work and not really intended. For that purpose we luckily have tools like Affinity Publisher.)

So I wanted to get the best of both worlds and miserably failed, because scripting is, as far as I know, actually not a big part of Affinity Publisher. That doesn't mean that I don't like the product, but I'm not sure if it's functions are sufficient this single purpose. Also some variables might be inserted by the data merge function, I'm not sure if it was ever designed for that many replacements where some take whole paragraphs in length. Please correct me if I'm wrong.

If we had christmas today and if I had the chance to get a present, I would wish for a simple concept to import data into publisher. This might look like that:

  1. Inside Publisher, I create a box with a unique name (like a textbox) and also templates for text and tables
  2. Inside a plain textfile I'm able to specify headings, paragraphs and tables (e.g. using the typical wiki style) and speficy in which textbox those shall be placed
  3. Being able to import the text / tables using the standard import function in Publisher

It's just a thought. Something that comes very close to that actually exists in Publisher, but as far as I know I would habe to import text for every textbox without any chance on automating this process. Please correct me here too if I'm wrong.

If something like that already exists in publisher, please let me know. Otherwise I will continue to apply fancy design elements in LaTeX documents, misusing some functions that were definetly not made for this purpose and hoping to get not stoned by the LaTeX community for writing code far away from each and every best-prace approach. 😀

Link to comment
Share on other sites

  • Staff

Thanks for your detailed reply. I think it's fair to say this is a long way from the use cases we were trying to cover with the initial version of this feature. I'm afraid Publisher can't do what you've asked for at this time. Amongst other things the most fundamental problem you're probably going to hit is that fields can't contain breaks and we don't have any way to link or merge tables. 

I'm glad to hear you've got a system that works using LaTeX.

I hope to visit more advanced data-driven documents at some point in the future but I'm afraid we have to focus on features that have the broadest applications first.

Link to comment
Share on other sites

  • 1 month later...

Hi all,

I really like the data merge feature but I wonder if it is planned to provide more automation for it. 

Currently we have the use case of creating CVs for our colleagues. Everyone can provide the content by writing a json file. We have a CI pipeline which merges all json CVs into a single json file, which then contains one dataset per colleague. This json file we use to generate the CVs and export them as PDF afterwards. But it would be super awesome if nobody would need to open APub to do that manually - instead I can imagine some sort of CLI / API or whatever which just takes a json file, does the data merge and gives some export options. 

I know this may not be in any focus right now - but are there plans?

Link to comment
Share on other sites

On 9/10/2021 at 11:04 AM, orschaef said:

I know this may not be in any focus right now - but are there plans?

Welcome to the Serif Affinity forums.

Serif generally does not disclose their plans to us, so you will probably not get an answer to that question.

-- Walt
Designer, Photo, and Publisher V1 and V2 at latest retail and beta releases
PC:
    Desktop:  Windows 11 Pro, version 23H2, 64GB memory, AMD Ryzen 9 5900 12-Core @ 3.00 GHz, NVIDIA GeForce RTX 3090 

    Laptop:  Windows 11 Pro, version 23H2, 32GB memory, Intel Core i7-10750H @ 2.60GHz, Intel UHD Graphics Comet Lake GT2 and NVIDIA GeForce RTX 3070 Laptop GPU.
iPad:  iPad Pro M1, 12.9": iPadOS 17.3, Apple Pencil 2, Magic Keyboard 
Mac:  2023 M2 MacBook Air 15", 16GB memory, macOS Sonoma 14.3.1

Link to comment
Share on other sites

  • 4 months later...
On 7/27/2021 at 11:04 AM, Mark Daniel said:

Thanks for your detailed reply. I think it's fair to say this is a long way from the use cases we were trying to cover with the initial version of this feature. I'm afraid Publisher can't do what you've asked for at this time. Amongst other things the most fundamental problem you're probably going to hit is that fields can't contain breaks and we don't have any way to link or merge tables. 

I'm glad to hear you've got a system that works using LaTeX.

I hope to visit more advanced data-driven documents at some point in the future but I'm afraid we have to focus on features that have the broadest applications first.

I haven't visited the forums lately and designer has been on a backshelf until data merge was implemented. To clarify, can we use a simple csv file imported to add both text and images, or no?

Link to comment
Share on other sites

17 minutes ago, debiant said:

I haven't visited the forums lately and designer has been on a backshelf until data merge was implemented. To clarify, can we use a simple csv file imported to add both text and images, or no?

With Publisher yes. Designer no.

Mac Pro (Late 2013) Mac OS 12.7.2 
Affinity Designer 2.3.1 | Affinity Photo 2.3.1 | Affinity Publisher 2.3.1 | Beta versions as they appear.

I have never mastered color management, period, so I cannot help with that.

Link to comment
Share on other sites

  • 1 year later...

Does the data merge function in AP 3 support line breaks in the merge data? I hope I haven't missed that in earlier posts.

I use Affinity Publisher for several different things. One use is to create envelopes.

The CSV file is a single column. Each row has a multiline address. Affinity Publisher 2 will merge the data as one line per record. The newline gets translated to two spaces.

If newlines in merge data became paragraph breaks that would be great.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...

Important Information

Terms of Use | Privacy Policy | Guidelines | We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.