Project

General

Profile

« Previous | Next » 

Revision 4125

Added by Aaron Marcuse-Kubitza over 12 years ago

README.TXT: Datasource setup: Updated for new subdirs layout

View differences:

README.TXT
61 61 
        defined by the Index Herbariorum <http://sweetgum.nybg.org/ih/>
62 62 
    Populate the src/ subdir with input data:
63 63 
        Obtain/create CSVs for the table(s) present in the datasource:
64 
            specimens, plots, organisms, stems
64 
            specimens, 0.plots, 1.organisms, 2.stems
65 65 
        If there are multiple part files for a table, and the header is repeated
66 66 
            in each part, make sure each header is EXACTLY the same.
67 67 
             (If the headers are not the same, the CSV concatenation script
68 68 
             assumes the part files don't have individual headers and treats the
69 69 
             subsequent headers as data rows.)
70 
        Rename each CSV so it ends in ".<table>.<ext>" (see tables above)
70 
        Place each CSV in a subdir named according to one of the tables above
71 71 
    Auto-create the map spreadsheets:
72 72 
        make inputs/<name>/; make inputs/<name>/
73 73 
            Note: Must be run twice to properly bootstrap all maps.
74 
        svn add inputs/<name>/maps/{*.csv,.*.last_cleanup}
74 
        svn add inputs/<name>/src/*/{{src,map*,VegBIEN}.csv,.*.last_cleanup}
75 75 
    Install the staging tables:
76 76 
        make inputs/<name>/reinstall quiet=1 &
77 
        To view progress: tail -f inputs/<name>/import/install-<table>.log.sql
78 
        View the logs: tail -n +1 inputs/<name>/import/install-*.log.sql
77 
        To view progress: tail -f inputs/<name>/src/<table>/logs/install.log.sql
78 
        View the logs: tail -n +1 inputs/<name>/src/*/logs/install.log.sql
79 79 
            tail provides a header line with the filename
80 80 
            +1 starts at the first line, to show the whole file
81 81 
        For every file with an error 'column "..." specified more than once':
82 
            Add a header override file "+header.<table>.<ext>" in src/:
82 
            Add a header override file "+header.<ext>" in src/<table>/:
83 83 
                Note: The leading "+" should sort it before the flat files.
84 84 
                    "_" unfortunately sorts *after* capital letters in ASCII.
85 85 
                Create a text file containing the header line of the flat files
......
91 91 
                Do NOT make readability changes in this step; that is what the
92 92 
                    map spreadsheets (below) are for.
93 93 
                Save
94 
                svn add inputs/<name>/src/<header_override>
94 
                svn add inputs/<name>/src/<table>/<header_override>
95 95 
        If you made any changes, re-run the install command above
96 96 
    Map each table's columns:
97 
        In the maps/ subdir, for each "via map" of the form "<via>.<table>.csv":
97 
        In each <table>/ subdir, for each "via map" map.csv:
98 98 
            Open the map in a spreadsheet editor
99 
            In /mappings, open the corresponding "core map" of the form
100 
                "<via>-VegBIEN.<table>.csv"
99 
            Open the "core map" /mappings/Veg+-VegBIEN.csv
101 100 
            In each row of the via map, set the right column to a value from the
102 101 
                left column of the core map
103 102 
            Save
104 103 
        Regenerate the derived maps: make inputs/<name>/
105 104 
    Accept the test cases:
106 
        make inputs/<name>/test/
105 
        make inputs/<name>/test
107 106 
            When prompted to "Accept new test output", enter y and press ENTER
108 107 
            If you instead get errors, do one of the following for each one:
109 108 
            -   If the error was due to a bug, fix it
......
113 112 
                prevent the automatic mapper from auto-removing the mapping.
114 113 
            When accepting tests, it's helpful to use WinMerge
115 114 
                (see WinMerge setup below for configuration)
116 
        svn add inputs/<name>/test/*.ref
115 
        svn add inputs/<name>/src/*/test.xml.ref
117 116 
    Commit: svn ci -m "Added inputs/<name>/" inputs/<name>/
118 117 
    Update vegbiendev:
119 118 
        On vegbiendev: svn up

Also available in: Unified diff