Data Merge – Page Numbers – may be invalid

[thomaso]

The pages fields currently work with invalid values & can cause an unexpected number of output pages:

• It is possible to choose an impossible page range. (e.g. higher value than total number of records)
• On switch from custom "Page Range" back to "All Pages" an invalid page number is maintained.
• The resulting output includes a different page range. (e.g. all pages if selected number is above actual total page numbers)

Invalid page range:

Invalid (> confusing) last of "All Pages":

EDIT / supplement:
Actually I would expect that the total page number gets auto-calculated + displayed in the DM manager window, e.g. in the "Source" section or preset in the "All Pages" > "To:" field.

[fde101]

1 hour ago, thomaso said:

higher value than total number of records

Haven't played much with data merge yet, but shouldn't it be possible for one record to be spread across multiple pages?

If not, that's quite an oversight...

[thomaso]

1 hour ago, fde101 said:

Haven't played much with data merge yet, but shouldn't it be possible for one record to be spread across multiple pages?

Maybe? – Here the point is the total number of pages: it is known to the app, can get output exactly according to the input/layout needs – therefore I'd expect the app would also know about valid page numbers which may get entered as custom range, or is displayed grayed-out correctly when "All Pages" is selected.

In the example above the total output spreads across 3 pages (4 + 4 + 2 records), so here the pages fields obviously may contain invalid values (12, 20, 21).

[fde101]

Hmm...

4 + 4 + 4 = 12, it looks like it is displaying the number of output records after expanding to include the blank ones at the end, instead of the actual number of pages.

[thomaso]

No, this 12 is coincidence. The 12 was typed manually with "Pages Range" selected, then switching back to "All Pages" maintained the 12 and grayed it out.
Note: above the data merge node contains 4 of 10 records and is placed on 1 page. So its obvious that 3 (2.5) pages will needed for output.

It would even be worse if the app would round the total record number (10 > 12) to virtually fill a layout (for what purpose?) and use that calculation for a rounded total page number in your assumed way. – Maybe the new data merge feature is too complex to argue theoretically without experimenting

[Jon P]

Quote

It is possible to choose an impossible page range. (e.g. higher value than total number of records)

I think you've misunderstood the page range functionality here, it is related to the pages in your document and has a min/max based on that.

If you have data merge nodes on pages 1,2,3 you can choose which ones get merged. The preview will show this (your other thread i think may be based of a similar confusion).

We should maybe be looking at making this a bit clearer?

[Joachim_L]

5 minutes ago, Jon P said:

I think you've misunderstood the page range functionality here, it is related to the pages in your document and has a min/max based on that.

This explain why it did not work for me on a single page document. When I saw the "Range" option I thought it would be possible to limit the data set. In the next days I am producing a serial letters with 1 document page and 2265 addresses (I know, Beta ). On the first test runs it was possible to produce the output document with 2265 document pages. It had some lags, but nevertheless behaving fine. Maybe you take this as a suggestion for upcoming versions?

[thomaso]

22 hours ago, Jon P said:

I think you've misunderstood the page range functionality here, it is related to the pages in your document and has a min/max based on that.

Correct: Indeed, I haven't noticed before: The pages field gets limited by the total page number of the layout document. (for the screenshots above I had used a 24 pg document with the node on pg 20. So that enabled me to type 12, 20, 21.)

Just saying "document" is ambiguous, because for data merge are 3 documents involved:
1 layout .afpub
1 external data document (or possibly more than 1 ... ?)
1 output .afpub

What indeed confuses is the use of page numbers within the Data Manager UI, tooltips included.
Various reasons why a user could rather focus on output page numbers:
1. the UI seems to be unclear what pages are meant (as just noted above), so the user might interprete it according to his interest:
2. it might not be interesting to preview pages which are not involved in the data merge process (for what purpose would I preview those, since I see them in my layout?)
3. it can be necessary/desired to output not the entire data set but parts only. For instance...
– a.) to check layout settings & their result with a sample page / a few sample pages only,
– b.) to limit for content reasons (e.g. with a task like "output only data records with property xy" / ~ "create for our company business cards of all employees ... with the function 'xyz' "/ or / "... with names from A – D".)
– c.) to limit for performance reason (e.g. a catalog with 1000s of images + texts might require to be limited (= split) to prevent APub from suffering, either when generating the output or when working on/with the output .afpub after data merge.)

Furthermore the UI confuses with page numbers because a custom set range gets reflected right below the data merge input file name (> top left corner of DM-Manager). That implies the page range someway is related to the output, in particular if the data input has 1 sheet (~page) only.

So, I think additionally to the page range which shall get repeated it would require a setting for the page range which shall get output. (or, alternatively, a setting to limit the range of records > which naturally would influence the total output page range).

 

22 hours ago, Jon P said:

If you have data merge nodes on pages 1,2,3 you can choose which ones get merged. The preview will show this (your other thread i think may be based of a similar confusion).

This hint made me try using 2 nodes, spread across two pages, feeded form 1 data source file.

The result appears odd in the output afpub. The total page number should be quite different: for the black text 3 pages (4+4+2 records) + for the orange text 4 pages (3+3+3+1 records) = total 7 pages. – However, only 4 pages are generated & data records are swallowed in between. (At least I would expect to get all records, even if currently in strange page order)

Layout input setup:

Output:

 

 

 

[fde101]

14 hours ago, thomaso said:

a custom set range gets reflected right below the data merge input file name (> top left corner of DM-Manager). That implies the page range someway is related to the output

Wouldn't putting it next to the name of the input file imply it was in some way related to the input file?

Agree it doesn't quite suggest it is related to the layout file, though...

Putting the controls in the "Output" section suggests they impact the output file, however...  there should probably be a separate "Layout" section for those.

 

I would expect control over the set of records being used to be referred to as "records" not "pages".

[thomaso]

Just now, fde101 said:

Wouldn't putting it next to the name of the input file imply it was in some way related to the input file?

That's what I first thought, too, but I knew my data input .xls didn't have that page range, opening the pulldown menu "Sheet:" confirmed my impression. Last but not least this page number (below the data input file name) always displays accordingly to the page range set in "Output" – not to any setting in "Input" (= "Source").

The latter aspect caused my thought above of an additional (or alternative) UI to limit the input range which gets used for output – e.g. either as number of output pages or number of input records, or via concrete cell names/numbers (e.g. "B 2 – G 500").

8 minutes ago, fde101 said:

Putting the controls in the "Output" section suggests they impact the output file, however...  there should probably be a separate "Layout" section for those.

An additional page range setting in an additional "Layout" section wouldn't avoid the possible misunderstanding of the current page range setting in Output > "Repeat:". In particular as long in a resulting .afpub the actual output pages appear as weird as in my 2-node experience above, which currently let me assume that more than 1 node doesn't work at all. (though the UI allows it and the manager window looks like expecting it by the large left column)

Maybe this "Repeat:" option should be used to inset (repeat) pages (within the data merge output pages) not related to data merge, maybe as separating pages with individual content (e.g. chapter start pages, "Our Product Group XY"), followed by the output data pages of a 2nd data input file (> 2nd node). – But then I would expect + need them to be repeated only after an entire set of records (= all its pages), not after each single page.

I haven't tried yet to load more than 1 data input file. Possible for those a not data related layout page between 2 nodes gets repeated differently on output than in my trial above (which used 1 input file for 2 nodes).

[fde101]

Looking more closely, the entire "Output" section should probably be renamed "Layout" as it represents the interpretation of the layout being used rather than directly reflecting the produced output.

[thomaso]

52 minutes ago, fde101 said:

Looking more closely, the entire "Output" section should probably be renamed "Layout" as it represents the interpretation of the layout being used rather than directly reflecting the produced output.

No, that wouldn't avoid misunderstandings. Note that 1.) the layout document does/should not reflect this "Repeat:" setting at all, and 2.) the current "Output" section indeed DOES influence the output "directly" (> output pages).

I think the entire manager UI requires improvement, not just a change of used terms. Currently this window's UI appears to lack in certain info/settings entirely, especially to define (limit) the INPUT which shall get used for "Layout" AND for "Output". Furthermore settings in the PREVIEW section currently do not cause a preview of the OUTPUT but rather display – in the node layout – just copies of single records, instead of displaying the output records as single but continuous 'flowing' instances. However, the preview topic has separate issues and I would prefer to discuss it in a different topic, e.g. here.

Finally we will need a window (UI layout + UX coding) which a.) uses all text and positions in an unambiguous way and b.) reflects all related aspects by offering according options. It wouldn't help to have either a.) or b.) but requires a.) AND b.). – Therefore a change of just single word(s) seems to be no useful adjustment because it may cause new, different questions or issues ... and confuse the discussion this way.

[fde101]

Maybe something like this?