Installation:

    Install: make install

    Important: Whenever you install a system update that affects PostgreSQL or

        any of its dependencies, such as libc, you should restart the PostgreSQL

        server. Otherwise, you may get strange errors like "the database system

        is in recovery mode" which go away upon reimport.

        WARNING: This will delete the current public schema of your VegBIEN DB!

        Note: This will archive the last import.

        Rename the rotated schema using the date in the first datasource's log

            file name

        Check that every input's log ends in "Encountered 0 error(s)"

        If many do not, fix the bug and discard the current (partial) import

        Otherwise, continue

            make inputs/download-logs

        Back up: make backups/public.<date>.backup &

        Remove: make backups/public.<date>.backup/remove

        Test: make backups/vegbien.<date>.backup/test &

        Restore: make backups/vegbien.<date>.backup/restore &

        Download: make inputs/download-logs

        <name> may not contain spaces, and should be abbreviated.

        If the datasource is a herbarium, <name> should be the herbarium code as

        defined by the Index Herbariorum <http://sweetgum.nybg.org/ih/>

            in each part, make sure each header is EXACTLY the same.

             (If the headers are not the same, the CSV concatenation script

             assumes the part files don't have individual headers and treats the

             subsequent headers as data rows.)

        make inputs/<name>/; make inputs/<name>/

            Note: Must be run twice to properly bootstrap all maps.

        svn add inputs/<name>/maps/{*.csv,.*.last_cleanup}

        make inputs/<name>/reinstall quiet=1 &

        To view progress: tail -f inputs/<name>/import/install-<table>.log.sql

            +1 starts at the first line, to show the whole file

        For every file with an error 'column "..." specified more than once':

            Add a header override file "+header.<table>.<ext>" in src/:

                Note: The leading "+" should sort it before the flat files.

                    "_" unfortunately sorts *after* capital letters in ASCII.

                Create a text file containing the header line of the flat files

                Add an ! at the beginning of the line

                    This signals cat_csv that this is a header override.

                For empty names, use their 0-based column # (by convention)

                For duplicate names, add a distinguishing suffix

                For long names that collided, rename them to <= 63 chars long

                Do NOT make readability changes in this step; that is what the

                    map spreadsheets (below) are for.

                Save

                svn add inputs/<name>/src/<header_override>

        If you made any changes, re-run the install command above

        In the maps/ subdir, for each "via map" of the form "<via>.<table>.csv":

            Open the map in a spreadsheet editor

            In /mappings, open the corresponding "core map" of the form

                "<via>-VegBIEN.<table>.csv"

            In each row of the via map, set the right column to a value from the

                left column of the core map

            Save

        Regenerate the derived maps: make inputs/<name>/

        make inputs/<name>/test/

            When prompted to "Accept new test output", enter y and press ENTER

            -   If the error was due to a bug, fix it

            -   Add a SQL function that filters or transforms the invalid data

            -   Make an empty mapping for the columns that produced the error.

                Put something in the Comments column of the map spreadsheet to

                prevent the automatic mapper from auto-removing the mapping.

                (see WinMerge setup below for configuration)

        On vegbiendev: svn up

        On local machine: make inputs/upload

    Regenerate schema from installed DB: make schemas/remake

        Run make schemas/vegbien.my.sql

        Open schemas/vegbien.ERD.mwb in MySQLWorkbench

        Go to File > Export > Synchronize With SQL CREATE Script...

        For Input File, select schemas/vegbien.my.sql

        Click Continue

        Click in the changes list and press Ctrl+A or Apple+A to select all

        Click Update Model

        Click Continue

        Note: The generated SQL script will be empty because we are syncing in

            the opposite direction

        Click Execute

        Reposition any lines that have been reset

        Add any new tables by dragging them from the Catalog in the left sidebar

            to the diagram

        Remove any deleted tables by right-clicking the table's diagram element,

            selecting Delete '<table name>', and clicking Delete

        Save

    Update graphical ERD exports:

        In the lower left corner, click PDF > Save as PDF...

        Set the Title and Author to ""

    Mapping process: make test

    Map spreadsheet generation: make remake

    Install WinMerge from <http://winmerge.org/>

    Go to Edit > Options and click Compare in the left sidebar

        <http://manual.winmerge.org/Configuration.html#d0e5892>.

        <http://manual.winmerge.org/Configuration.html#d0e5758>.

    To generate a Redmine-formatted list of steps for column-based import:

        make inputs/QMOR/import/steps.by_col.sql

    To see a program's description, read its top-of-file comment

    To see a program's usage, run it without arguments

    To remake a file: make <file>-remake