Revision 4219
Added by Aaron Marcuse-Kubitza over 12 years ago
README.TXT | ||
---|---|---|
55 | 55 |
Download: make inputs/download-logs |
56 | 56 |
|
57 | 57 |
Datasource setup: |
58 |
Add a new datasource: make inputs/<name>/add
|
|
59 |
<name> may not contain spaces, and should be abbreviated.
|
|
60 |
If the datasource is a herbarium, <name> should be the herbarium code as
|
|
61 |
defined by the Index Herbariorum <http://sweetgum.nybg.org/ih/> |
|
58 |
Add a new datasource: make inputs/<datasrc>/add
|
|
59 |
<datasrc> may not contain spaces, and should be abbreviated.
|
|
60 |
If the datasource is a herbarium, <datasrc> should be the herbarium code
|
|
61 |
as defined by the Index Herbariorum <http://sweetgum.nybg.org/ih/>
|
|
62 | 62 |
Add input data for each table present in the datasource: |
63 | 63 |
Choose a table name from <https://projects.nceas.ucsb.edu/nceas/projects |
64 | 64 |
/bien/wiki/VegCSV#Suggested-table-names>, or use a custom name |
65 |
make inputs/<name>/<table>/add
|
|
66 |
Place the CSV for the table in inputs/<name>/<table>/
|
|
65 |
make inputs/<datasrc>/<table>/add
|
|
66 |
Place the CSV for the table in inputs/<datasrc>/<table>/
|
|
67 | 67 |
Important: When exporting relational databases to CSVs, you MUST ensure |
68 | 68 |
that embedded quotes are escaped by doubling them, *not* by |
69 | 69 |
preceding them with a "\" as is the default in phpMyAdmin |
... | ... | |
73 | 73 |
assumes the part files don't have individual headers and treats the |
74 | 74 |
subsequent headers as data rows.) |
75 | 75 |
Auto-create the map spreadsheets: |
76 |
make inputs/<name>/; make inputs/<name>/
|
|
76 |
make inputs/<datasrc>/; make inputs/<datasrc>/
|
|
77 | 77 |
Note: Must be run twice to properly bootstrap all maps. |
78 |
svn add inputs/<name>/*/{{src,map*,VegBIEN}.csv,.*.last_cleanup}
|
|
78 |
svn add inputs/<datasrc>/*/{{src,map*,VegBIEN}.csv,.*.last_cleanup}
|
|
79 | 79 |
Install the staging tables: |
80 |
make inputs/<name>/reinstall quiet=1 &
|
|
81 |
To view progress: tail -f inputs/<name>/<table>/logs/install.log.sql
|
|
82 |
View the logs: tail -n +1 inputs/<name>/*/logs/install.log.sql
|
|
80 |
make inputs/<datasrc>/reinstall quiet=1 &
|
|
81 |
To view progress: tail -f inputs/<datasrc>/<table>/logs/install.log.sql
|
|
82 |
View the logs: tail -n +1 inputs/<datasrc>/*/logs/install.log.sql
|
|
83 | 83 |
tail provides a header line with the filename |
84 | 84 |
+1 starts at the first line, to show the whole file |
85 | 85 |
For every file with an error 'column "..." specified more than once': |
... | ... | |
95 | 95 |
Do NOT make readability changes in this step; that is what the |
96 | 96 |
map spreadsheets (below) are for. |
97 | 97 |
Save |
98 |
svn add inputs/<name>/<table>/<header_override>
|
|
98 |
svn add inputs/<datasrc>/<table>/<header_override>
|
|
99 | 99 |
If you made any changes, re-run the install command above |
100 | 100 |
Map each table's columns: |
101 | 101 |
In each <table>/ subdir, for each "via map" map.csv: |
... | ... | |
104 | 104 |
In each row of the via map, set the right column to a value from the |
105 | 105 |
left column of the core map |
106 | 106 |
Save |
107 |
Regenerate the derived maps: make inputs/<name>/
|
|
107 |
Regenerate the derived maps: make inputs/<datasrc>/
|
|
108 | 108 |
Accept the test cases: |
109 |
make inputs/<name>/test
|
|
109 |
make inputs/<datasrc>/test
|
|
110 | 110 |
When prompted to "Accept new test output", enter y and press ENTER |
111 | 111 |
If you instead get errors, do one of the following for each one: |
112 | 112 |
- If the error was due to a bug, fix it |
... | ... | |
116 | 116 |
prevent the automatic mapper from auto-removing the mapping. |
117 | 117 |
When accepting tests, it's helpful to use WinMerge |
118 | 118 |
(see WinMerge setup below for configuration) |
119 |
svn add inputs/<name>/*/test.xml.ref
|
|
120 |
Commit: svn ci -m "Added inputs/<name>/" inputs/<name>/
|
|
119 |
svn add inputs/<datasrc>/*/test.xml.ref
|
|
120 |
Commit: svn ci -m "Added inputs/<datasrc>/" inputs/<datasrc>/
|
|
121 | 121 |
Update vegbiendev: |
122 | 122 |
On vegbiendev: svn up |
123 | 123 |
On local machine: make inputs/upload |
Also available in: Unified diff
README.TXT: Datasource setup: Changed "<name>" to "<datasrc>" to distinguish it more clearly from "<table>", which is also a name