Revision 3611
Added by Aaron Marcuse-Kubitza over 12 years ago
README.TXT | ||
---|---|---|
48 | 48 |
make inputs/<name>/; make inputs/<name>/ |
49 | 49 |
Note: Must be run twice to properly bootstrap all maps. |
50 | 50 |
svn add inputs/<name>/maps/{*.csv,.*.last_cleanup} |
51 |
Install the staging tables: make inputs/<name>/install quiet=1 & |
|
52 |
To view the log: tail -f inputs/<name>/import/install-<table>.log.sql |
|
51 |
Install the staging tables: |
|
52 |
make inputs/<name>/reinstall quiet=1 & |
|
53 |
To view progress: tail -f inputs/<name>/import/install-<table>.log.sql |
|
54 |
View the logs: tail +1 inputs/<name>/import/install-*.log.sql |
|
55 |
tail provides a header line with the filename |
|
56 |
+1 starts at the first line, to show the whole file |
|
57 |
For every file with an error 'column "..." specified more than once': |
|
58 |
Add a header override file "+header.<table>.<ext>" in src/: |
|
59 |
Note: The leading "+" should sort it before the flat files. |
|
60 |
"_" unfortunately sorts *after* capital letters in ASCII. |
|
61 |
Create a text file containing the header line of the flat files |
|
62 |
Add an ! at the beginning of the line |
|
63 |
This signals cat_csv that this is a header override. |
|
64 |
For empty names, use their 0-based column # (by convention) |
|
65 |
For duplicate names, add a distinguishing suffix |
|
66 |
For long names that collided, rename them to <= 63 chars long |
|
67 |
Do NOT make readability changes in this step; that is what the |
|
68 |
map spreadsheets (below) are for. |
|
69 |
Save |
|
70 |
svn add inputs/<name>/src/<header_override> |
|
71 |
If you made any changes, re-run the install command above |
|
53 | 72 |
Map each table's columns: |
54 | 73 |
In the maps/ subdir, for each "via map" of the form "<via>.<table>.csv": |
55 | 74 |
Open the map in a spreadsheet editor |
Also available in: Unified diff
README.TXT: Datasource setup: Installing the staging tables: Added steps to deal with colliding column names in the flat file headers. Added command to view the logs.