Jump to content
You must now use your email address to sign in [click for more info] ×

Data Merge – can't link to 2nd source


Recommended Posts

Though the UI seems to offer the option to load more than 1 data source file I am unable to do so.
With a 2nd source entry selected in the Manager it jumps back to the 1st source and changes this URL instead of assigning the new URL to the 2nd source entry.

macOS 10.14.6 | MacBookPro Retina 15" | Eizo 27" | Affinity V1

Link to comment
Share on other sites

2 hours ago, thomaso said:

Though the UI seems to offer the option to load more than 1 data source file I am unable to do so.

This seems to be a bug on MacOSX only because, apparently, you can do that on windows.

2017 27” iMac 4.2 GHz Quad-Core Intel Core i7 • Radeon Pr 580 8GB • 64GB • Ventura 13.6.4.

iPad Pro (10.5-inch) • 256GB • Version 16.4

Link to comment
Share on other sites

@anon2,  could you please upload an .afpub with two data node entries in the manager? (I think the data files aren't necessary, I possibly can update them once I got their entries in an .afpub). Thank you!

macOS 10.14.6 | MacBookPro Retina 15" | Eizo 27" | Affinity V1

Link to comment
Share on other sites

In the .afpub of anon2 (thanks!) I see two sources and can click through record previews – but again I appear to be limited to the 1st source.
In this clip I try to insert a field name of source 2 – but always get an entry of source 1 instead:


By the way: Does the UI tell somewhere what source file formats are allowed? I would expect to get the info for instance when selecting a source file in the finder window (e.g. as top or bottom bar text like "Select a XLSX, CSV, TXT, XML, ..."). Though I currently can see grayed out files (e.g. .xls) I don't see what formats would work unless I would think of + create + try them.

macOS 10.14.6 | MacBookPro Retina 15" | Eizo 27" | Affinity V1

Link to comment
Share on other sites

9 hours ago, thomaso said:

By the way: Does the UI tell somewhere what source file formats are allowed?

On Windows I see this:

data-sources.thumb.jpg.b0c3f812c9eba5962b7c8d2588a0bcce.jpg

 

d.

Affinity Designer 1 & 2   |   Affinity Photo 1 & 2   |   Affinity Publisher 1 & 2
Affinity Designer 2 for iPad   |   Affinity Photo 2 for iPad   |   Affinity Publisher 2 for iPad

Windows 11 64-bit - Core i7 - 16GB - Intel HD Graphics 4600 & NVIDIA GeForce GTX 960M
iPad pro 9.7" + Apple Pencil

Link to comment
Share on other sites

11 hours ago, thomaso said:

In the .afpub of anon2 (thanks!) I see two sources and can click through record previews – but again I appear to be limited to the 1st source.

When multiple sources can output to a particular page of a merge template, you must disable preview (or disable output) of each source that precedes the source which you want to preview on that page.

 

11 hours ago, thomaso said:

In this clip I try to insert a field name of source 2 – but always get an entry of source 1 instead

The field names of all sources of a node are in a flat namespace. <symbol> in a cell of my node refers to the symbol field in both of my source1.csv and source2.csv.

In any case, the sources of a node are processed consecutively, not concurrently, therefore data from multiple sources would not appear simultaneously in a cell even if source fields with a common name could be uniquely referenced in the template.

 

Link to comment
Share on other sites

3 hours ago, dominik said:

On Windows I see this:

"Affinity Files (*.csv, *.tsv, *json, *.x...)"

Aha. This isn't shown in macOS. Here such info appearance depends on / varies with the app, some apps show a menu to choose either "all types" or a certain, suitable, file type and limit the list of offered files this way.

Is it in Windows a general info, shown the same way in all apps? (Interesting that it calls those data files "Affinity Files")

macOS 10.14.6 | MacBookPro Retina 15" | Eizo 27" | Affinity V1

Link to comment
Share on other sites

4 minutes ago, thomaso said:

Is it in Windows a general info, shown the same way in all apps? (Interesting that it calls those data files "Affinity Files")

I think it is a Windows standard. I'm so used to it that I hardly notice it anymore unless I need to change something. I checked in several applications and they all have that selection menu.

I do not know who 'injects' the wording 'Affinity Files', wether it's Serif or some Windows automatism.

d.

Affinity Designer 1 & 2   |   Affinity Photo 1 & 2   |   Affinity Publisher 1 & 2
Affinity Designer 2 for iPad   |   Affinity Photo 2 for iPad   |   Affinity Publisher 2 for iPad

Windows 11 64-bit - Core i7 - 16GB - Intel HD Graphics 4600 & NVIDIA GeForce GTX 960M
iPad pro 9.7" + Apple Pencil

Link to comment
Share on other sites

1 hour ago, anon2 said:

When multiple sources can output to a particular page of a merge template, you must disable preview (or disable output) of each source that precedes the source which you want to preview on that page.

Ah, yes, disabling in your .afpub the preview for the 1st source allows to display the preview for the 2nd.

a.) Both previews disabled:

1603486536_DM2srcs-preview1-none.jpg.a05bba58a9af5eec062231350a76a5d2.jpg


b.) Preview for 1st source disabled, for 2nd enabled –> does preview 2nd source:

2061106720_DM2srcs-preview2-source2.jpg.4e26ab0e3a536ff3a79e2e5cfdff082e.jpg


I wonder if this excluding behavior occurs only / in particular with your .afpub because both sources use identical field labels ("symbol", "code") which require a decision which of this 2 sets shall be used for the data merge (preview & output).

Also I wonder how you did create the node this way. It looks you inserted only 1 entry in each of the two text frames. I assume by clicking on 1 specific field label in the fields panel – which would reference to a certain, unique input source, not to both sources, as it currently does. So how comes that the fields of both sources are used (only 2 text frame entries but 4 connected source entries)?

1 hour ago, anon2 said:

In any case, the sources of a node are processed consecutively, not concurrently, therefore data from multiple sources would not appear simultaneously in a cell even if source fields with a common name could be uniquely referenced in the template.

What if you use the 2 sources in 2 different nodes (regardless of their position, maybe identical, maybe aside each other)?
I would expect then both sources would be able to preview their content simultaneously, and output accordingly both.

Unfortunately I still can't load a 2nd source to try it with unambiguous field labels of two sources / separated nodes.

macOS 10.14.6 | MacBookPro Retina 15" | Eizo 27" | Affinity V1

Link to comment
Share on other sites

35 minutes ago, thomaso said:

I wonder if this excluding behavior occurs only / in particular with your .afpub because both sources use identical field labels ("symbol", "code") which require a decision which of this 2 sets shall be used for the data merge (preview & output).

A template document's sources are processed consecutively.

The preview of a particular page of the template is populated by data from the first record of the first source that is:

  • set to output to that page
  • output enabled
  • preview enabled

In the example, both sources are set to output to page 1 of the template, therefore, no data from source2 can be previewed on page 1 unless output or preview of source1 is disabled. That is regardless of field names of source1 colliding with field names of source2.

 

1 hour ago, thomaso said:

Also I wonder how you did create the node this way. It looks you inserted only 1 entry in each of the two text frames. I assume by clicking on 1 specific field label in the fields panel – which would reference to a certain, unique input source, not to both sources, as it currently does. So how comes that the fields of both sources are used (only 2 text frame entries but 4 connected source entries)?

A field name in a merge template document does not refer to a field of a specific data source. A field name refers to a field with that name in any of the document's data sources.
Regardless of these name collisions, only one data source is used to populate a given page of the generated output, therefore unique source field references alone would not enable the combining of data from multiple sources in one cell, one node or an entire page of generated output.

 

2 hours ago, thomaso said:

What if you use the 2 sources in 2 different nodes (regardless of their position, maybe identical, maybe aside each other)?
I would expect then both sources would be able to preview their content simultaneously, and output accordingly both.

A page of the generated output document is populated by data from only one source file.
A preview of one page of the template document is populated by data from only one record, regardless of how many records will be used for a page of the real output. Hopefully we will get a true preview soon.

 

Link to comment
Share on other sites

3 hours ago, anon2 said:

1  -  Sources are processed consecutively. (...)

2  -  No data from source2 can be previewed on page 1 unless output or preview of source1 is disabled (...)

3  -  A field name in a merge template document does not refer to a field of a specific data source. (...)

4  -  A page of the generated output document is populated by data from only one source file. (...)

Thank you for this clear thoughts, they make my misunderstandings and possible misexpectations obvious. Your entire post reads like from a manual and is quite helpful this way. Each of your 4 thought groups appear currently to be true for APub although – to me – they all appear like issues. So I wonder whether you wrote just neutral observations of the current app state only, or whether you agree / disagree with the observations. Also I wonder if your true, clear observations also reflect the concept of the developers, as long this is not documented we only can guess whether what we see also is what we should or may see.

For multiple source files I still expect different capabilities / workflows from Affinity data merge:

The ability to mix source records within a.) single pages and b.) sequences of pages.
For instance: A catalog for content delivered by different departments of a firm, with source A (images)  +  source B (descriptive text)  +  source C (numbers, or 2nd language):

161093749_catalog4low.jpg.4aaf944ad821ef140bac43cb5c58686e.jpg1681974564_catalog2low.jpg.729b9482b31be673137c28222b7bec06.jpg

Or even with additional "empty" pages (chapter separators, designed / filled individually, without data merge) getting copied by data merge ("Repeat:"), but between sets of records, i.e. after several pages at the end of an entire record setnot after every page. While a record set can be defined by a certain section of 1 source (e.g. row 1 to 20, or sheet 1/2...).

1896156204_catalog1alow.jpg.7c0e0f6c8ec8607f5666683910bd1509.jpg

macOS 10.14.6 | MacBookPro Retina 15" | Eizo 27" | Affinity V1

Link to comment
Share on other sites

32 minutes ago, thomaso said:

I wonder whether you wrote just neutral observations of the current app state only, or whether you agree / disagree with the observations.

Neutral observations.

33 minutes ago, thomaso said:

Also I wonder if your true, clear observations also reflect the concept of the developers, as long this is not documented we only can guess whether what we see also is what we should or may see.

I, too, wonder about that, but I suspect the current behaviour is as designed and we should not expect great sophistication in the near future. The preview does need improvement, though, so that a page is filled by multiple records if it will be when the output is generated for real, and that should be relatively trivial for the developer.

 

 

Link to comment
Share on other sites

2 hours ago, thomaso said:

The ability to mix source records within a.) single pages and b.) sequences of pages.
For instance: A catalog for content delivered by different departments of a firm, with source A (images)  +  source B (descriptive text)  +  source C (numbers, or 2nd language):

Asking for trouble, Images are sorted in Source A by date of entry everything thing in Source B is sorted alphabetically (in its second column) and Source C is sorted by a third criteria. Nothing will line up as  the three separate items one_a in Sources A, B and C are all different  items. Sort a list of names by first name and then last name in a second copy of the database. Merge the first names from one with the last names from the other, is it an accurate list of names?

Mac Pro (Late 2013) Mac OS 12.7.4 
Affinity Designer 2.4.0 | Affinity Photo 2.4.0 | Affinity Publisher 2.4.0 | Beta versions as they appear.

I have never mastered color management, period, so I cannot help with that.

Link to comment
Share on other sites

On 10/25/2020 at 10:49 PM, Old Bruce said:

Asking for trouble, Images are sorted in Source A by date of entry everything thing in Source B is sorted alphabetically (in its second column) and Source C is sorted by a third criteria. Nothing will line up as  the three separate items one_a in Sources A, B and C are all different  items. Sort a list of names by first name and then last name in a second copy of the database. Merge the first names from one with the last names from the other, is it an accurate list of names?

Sorry Old Bruce, I can't follow your thoughts, probably of being not native English. If it was meant as a task / question to me could you try again with different grammar?
In my few tests the source items appeared in alphabetical order only, regardless of text or linked images (paths).

macOS 10.14.6 | MacBookPro Retina 15" | Eizo 27" | Affinity V1

Link to comment
Share on other sites

Also Beta v.1.9.0.822 doesn't allow me to apply a file path to a 2nd source.

I still do get the issue shown in this topic's initial post when trying to set the file path for a 2nd source (select the source in finder). As soon I select any item in the finder's window from this APub dialog then the currently selection of the 2nd source in the data merge manager always jumps back to select the 1st source and unwantedly changes its path, instead of the 2nd source.

macOS 10.14.6 | MacBookPro Retina 15" | Eizo 27" | Affinity V1

Link to comment
Share on other sites

1 hour ago, thomaso said:

Sorry Old Bruce, I can't follow your thoughts, probably of being not native English. If it was meant as a task / question to me could you try again with different grammar?
In my few tests the source items appeared in alphabetical order only, regardless of text or linked images (paths).

I was commenting on the possible problems with using more than one source data document. I have a list of names and a second data document with photos of people. The list of names is complete, 123 people, I have photos for 98 of them in the second data document. 

I will have no problem with a single source document with 123 names and a field in that document with photos for 98 of people.

The two source example will give me 123 records and the first 98 will have pictures, I doubt that the pictures will match with the names because they are from different source files. 

We would be best served by merging the various documents outside of  Publisher and then importing that one data file.

Mac Pro (Late 2013) Mac OS 12.7.4 
Affinity Designer 2.4.0 | Affinity Photo 2.4.0 | Affinity Publisher 2.4.0 | Beta versions as they appear.

I have never mastered color management, period, so I cannot help with that.

Link to comment
Share on other sites

  • Staff
Quote

I still do get the issue shown in this topic's initial post when trying to set the file path for a 2nd source (select the source in finder).

Neither me or Gabe can reproduce this, I was hoping the change in auto selecting a target when creating a new data merge source would help but it seems not.

Can I just clarify if this happens if you choose any data merge types, use different files etc?

Serif Europe Ltd. - www.serif.com

Link to comment
Share on other sites

58 minutes ago, Old Bruce said:

We would be best served by merging the various documents outside of  Publisher and then importing that one data file.

Either that or having an ID field that should be the same in both files. One-to-One relation basically.

2017 27” iMac 4.2 GHz Quad-Core Intel Core i7 • Radeon Pr 580 8GB • 64GB • Ventura 13.6.4.

iPad Pro (10.5-inch) • 256GB • Version 16.4

Link to comment
Share on other sites

26 minutes ago, Seneca said:

Either that or having an ID field that should be the same in both files. One-to-One relation basically.

Now we get to see how deep the rabbit hole goes 🙂

If there are two files with a subset of field names in common, should they be concatenated with blank values for the non-matching names or should the individual records joined on the shared field names?

If they are joined, what if there are duplicate "keys" within a file?  What if both files contain duplicate keys?  What about more than two files being joined?

Do the files need to be sorted in the same order in advance by the key they are joined on, or should the program index or scan the joined-to files for matches that may be out of sequence?

 

This is why there are relational databases to perform these tasks; they are outside the domain of the layout software.

What would be more helpful is to be able to select such a database as the data source without requiring the data to be exported first, but I suspect that is beyond the scope of 1.9 in any case.

Link to comment
Share on other sites

1 hour ago, Jon P said:

Can I just clarify if this happens if you choose any data merge types, use different files etc?

Well, it's new in v822 that I am able to load more than 1 item into the manager – but I can't change its initially linked source. Then the "Select" button opens a file window AND immediately – without having clicked any file (or folder to open) – the interface jumps to the 1st manager entry, regardless of the file type of the source I want to change.

Here I first try to alter the source of an .xlsx (2nd manager item), then of a .csv (3rd item) and I even haven't touched any file or folder in the list when the 1st manager item gets activated:

macOS 10.14.6 | MacBookPro Retina 15" | Eizo 27" | Affinity V1

Link to comment
Share on other sites

1 hour ago, fde101 said:

What would be more helpful is to be able to select such a database as the data source without requiring the data to be exported first, but I suspect that is beyond the scope of 1.9 in any case.

There is so much a publishing app can do but to be able to connect to a database (again rabbit hole: local?, cloud, etc) is something very desirable and I completely agree with you here.

2017 27” iMac 4.2 GHz Quad-Core Intel Core i7 • Radeon Pr 580 8GB • 64GB • Ventura 13.6.4.

iPad Pro (10.5-inch) • 256GB • Version 16.4

Link to comment
Share on other sites

Back in the day publishers used AppleScript to connect QuarkXPress with FileMaker databases and automagically produce TV listings mags, classified ad supplements, sales brochures and more. My feeling is that unless Serif build some sort of cross-application automation into their apps, the data would need to be manually exported

Affinity Photo 2.0.3,  Affinity Designer 2.0.3, Affinity Publisher 2.0.3, Mac OSX 13, 2018 MacBook Pro 15" Intel.

Link to comment
Share on other sites

×
×
  • Create New...

Important Information

Terms of Use | Privacy Policy | Guidelines | We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.