[Publisher]: Many different data sources (csv) - easiest way to get into document?

[mikeLRBG]

I have over one hundred separate CSV files that I need to get into a single document. What's the best way to do this?
 

• On each page, I have 2 instances of one of these CSV files represented in a grid
• I began adding each separate file as a data source and I'm using the data merge layout tool
• When I click generate, the 2nd instance renders blank. I think it's because I need to point that second instance to the second data source. It is a copy and paste of the 1st grid, so it think it's data is coming from data source 1.
• Is there a simpler way to do this without having to remap all the individual cells of data? The headers are all the same label. It's just the data is different in each file. I'm hoping I can quickly re-assign each grid to the new data source through a few clicks, pointing at the new source.
  
  
  Or, can I scrap the data merge tool all together and just use a table? I tried copying and pasting my data directly from the CSV into a table - doesn't work out at all.
  
  An example of the header row and first 2 rows...
  
  C0,C1,C2,C3,C4,C5,C6,C7,C8
  7,,2,,6,,9,8,1
  8,,1,9,4,2,,3,7
  
  When I get this over to the actual grid on the page, I don't want the commas, nor the header row.
  
  Thanks.

[David in Яuislip]

1 hour ago, mikeLRBG said:

I have over one hundred separate CSV files that I need to get into a single document. What's the best way to do this?

If "The headers are all the same label. It's just the data is different in each file" then you can simply concatenate these in Windows, no idea about a mac

For all csv files in a clean folder, open a cmd window and do
copy *.csv new.csv
Open new.csv in your spreadsheet program and remove duplicates as the headers will be repeated

[mikeLRBG]

3 hours ago, David in Яuislip said:

If "The headers are all the same label. It's just the data is different in each file" then you can simply concatenate these in Windows, no idea about a mac

For all csv files in a clean folder, open a cmd window and do
copy *.csv new.csv
Open new.csv in your spreadsheet program and remove duplicates as the headers will be repeated

Thank you. I was able to look down the path of figuring out how to do this on Mac.

And this one gets rid of the header, I can just add it back in manually.
 

awk 'FNR > 1' *.csv > consolidated.csv

 

[mikeLRBG]

I need some tweaking, but now I know about the command 🙂