Renamed inputs/NYBG to inputs/NY to match herbarium code

Renamed inputs/UNC-NCSC to inputs/NCU-NCSC to match herbarium code

Renamed inputs/UArizona to inputs/ARIZ to match herbarium code

Regenerated inputs/MO/maps/src.join.specimens.csv

Renamed inputs/MOBOT to inputs/MO to match herbarium code

Regenerated vegbien.ERD exports

vegbien.sql: taxonoccurrence: Added cultivatedbasis

vegbien.sql: Moved all accessioncode fields to the bottom of their tables. vegbien.ERD.mwb: Adjusted lines to remove overlaps.

vegbien.sql: taxonoccurrence: Added iscultivated, isnative. Moved accessioncode to bottom.

vegbien.sql: Changed taxonoccurrence.growthform type to more specific growthform

vegbien.sql: Added growthform and establishmentmeans_dwc enums using values from taxonclass. Documented that taxonclass is growthform + establishmentmeans_dwc + some other values.

VegBIEN: Moved aggregateoccurrence.growthform to taxonoccurrence

Added inputs/UNC-NCSC/maps/src.join.specimens.csv

VegBIEN: Merged aggregateoccurrence.verbatimcollectorname and specimenreplicate.verbatimcollectorname into taxonoccurrence

xml_func.py: parse_range(): Handle negative numbers by treating them as not a range

Added inputs/UNC-NCSC/test with initial accepted test outputs

Added inputs/UNC-NCSC/maps

xml_func.py: _replace: Fixed bug where value entry was not unpacked

Added inputs/UNC-NCSC

Added inputs/MOBOT/test with initial accepted test outputs

Added inputs/MOBOT/maps

Added inputs/MOBOT

VegX mappings: Updated plot place mappings to VegX 1.5.3 method of place type-tagged place names. This removes the userdef fields in plot.

VegX mappings: Changed userdef xPosition, yPosition to /relativePlotPosition/relativeX, /relativePlotPosition/relativeY

Regenerated mappings/DwC-VegBIEN.specimens.no_empty.csv

bin/map: map_table(): wrap_row(): Use util.list_as_length() to handle CSV rows of different lengths

util.py: Added list_as_length(). Documented that list_set_length() takes a list, not a tuple. Documented that ListDict must have len(list_) == len(keys).

util.py: Added list_set_length(). Changed list_set() to use list_set_length().

mappings/DwC2-VegBIEN.specimens.csv: Added empty *_id/taxonoccurrence attr to primary keys to ensure that a taxonoccurrence is always created for the specimenreplicate

xml_func.py: _label: Use ustr instead of str when checking types

csvs.py: Set dialect.doublequote to True because Sniffer doesn't turn this on by default

Merged inputs/NYBG-CSV into NYBG

Merged inputs/UArizona-CSV into UArizona

Added inputs/SpeciesLink/test

Added inputs/SpeciesLink/maps

xml_func.py: range-related funcs: Made inputs optional in case they get set to NULL by _nullIf

mappings/DwC1-DwC2.specimens.csv: Added common DwC1 fields that are not part of the official DwC1 schema

bin/map: Added support for getting columns with an optional prefix list for DB/CSV inputs

bin/map: Factored out code common to DB and CSV inputs into map_table()

bin/map: Parse any prefixes in map input column name. They will later be used to check for versions of columns with a prefix added when processing CSV/DB inputs.

strings.py: Added split(), remove_prefix(), remove_suffix(), and remove_prefixes(). Added section comments.

mappings/DwC2-VegBIEN.specimens.csv: minimumElevationInMeters: Handle embedded ranges using _rangeStart and _rangeEnd

xml_func.py: Added _rangeStart and _rangeEnd

xpath.py: parse(): Split paths: Raise a SyntaxException if can't attach a split path because there is no parent element to attach to

Parser.py: Renamed _syntax_err() to syntax_err() to make it a public method

mappings/DwC2-VegBIEN.specimens.csv: Mapped fieldNotes and taxonRemarks to description using _merge. inputs/UArizona*/maps/DwC.specimens.csv: Mapped Remarks to taxonRemarks, which now has a VegBIEN mapping.

Added inputs/GBIF/src with small files that can be under version control

input.Makefile: svn_props: Ignore everything in the src/ subdir that hasn't been explicitly checked in

Added inputs/GBIF/test with accepted test outputs

Added inputs/GBIF/maps

Regenerated inputs/UArizona*/maps VegBIEN maps

Regenerated mappings/DwC-VegBIEN.specimens.no_empty.csv

bin/map: Use new csvs.reader_and_header() to support CSVs/TSVs with other than the default Excel dialect

Added csvs.py for CSV I/O such as automatically detecting the dialect based on the header line

join: Don't append suffix to empty output mappings, so that they stay empty ("NULL")

input.Makefile: Added tsv to $(exts). Strip extra whitespace from $(inputs) so that it's the empty string if $(<in) (and $(<in).header) don't exist, and can be used in $(if ...).

input.Makefile: Fixed bug in inputFiles wildcard where extensions were manually listed instead of dynamically determined from the $(exts) config var

README.TXT: Tell user to `disown -h 1` after running `make import x%x` so that it won't be sent a SIGHUP if the user logs out

README.TXT: Tell user to `disown -h 1` after running `make import x%x` so that it won't be sent a SIGHUP if the user logs out

input.Makefile: Prepend separate CSV header when available

input.Makefile: Use with_cat in map to later support prepending separate CSV headers

Added with_cat to run a command, taking input from the concatenation of files

input.Makefile: Set mapEnv if $(dbEngine) is set, to eventually support pre-existing DB connections

input.Makefile: Changed $(dbFile) to $(dbExport) to make it unambiguous that it refers to a SQL export, not a pre-existing DB, which will be supported later

input.Makefile: Added .txt to list of input file extensions

Added inputs/SpeciesLink

root Makefile: python-Linux: Added pymetrics

bin/map: Consider \N to be None

util.py: none_if(): Allow multiple none_vals using varargs

Added inputs/GBIF

exc.py: Fixed bug in traceback-saving mechanism that didn't deal with nested Exceptions (such as Exceptions with causes in ExceptionWithCause). Renamed add_exc_info() to add_traceback() since we really only need to store the traceback.

dates.py: parse_date_range(): Fixed bug where the date parts were not joined back together into a string for each date range element. Use strings.single_space() after the date has been split into range parts so that whitespace around the range separator is removed instead of being replaced with a single space.

xml_func.py: process(): Also catch XML func internal errors to assist in debugging. Use new exc.add_exc_info() to save traceback in case later code throws exception, overwriting exc_info().

exc.py: str_(): Add the traceback at the end of the exception string. Added add_exc_info() and get_exc_info() for providing traceback info for str_().

mappings/DwC2-VegBIEN.specimens.csv: eventDate, dateIdentified: Use _dateRangeStart and _dateRangeEnd

xml_func.py: Added _dateRangeStart and _dateRangeEnd

dates.py: Added parse_date_range() and helper funcs could_be_year() and could_be_day()

strings.py: Added single_space()

inputs/UArizona*: Map the ScientificNameAuthor to the binomial instead since it contains the binomial in addition to the authority

Added inputs/UArizona-CSV/test

input.Makefile: Use .PRECIOUS to save outputs of failed tests so they can be accepted (needed now that .DELETE_ON_ERROR is turned on globally)

bin/map: Moved string-cleanup code from get_value() to cleanup(), called by process_row(). process_row() now cleans up the string before checking if it's None, because cleanup() uses none_if() to map "" to None.

util.py: Added do_ignore_none()

Added inputs/UArizona-CSV/verify

Added inputs/UArizona-CSV/maps

mappings/DwC2-VegBIEN.specimens.csv: Mapped coordinateUncertaintyInMeters to the same place as coordinatePrecision (input sources generally use only one of these columns, which is most likely the accuracy regardless of what it's named)

join: In error message when map column names don't match, include the actual column names

Makefiles: Added .DELETE_ON_ERROR to delete target if recipe fails

VegBIEN mappings: plantnames: Nest taxons hierarchically using plantname.parent_id. Mappings using _forEach: Append a "," to the `in` list so that mappings will sort from shortest to longest `in` list ("]" comes after "," in ASCII, causing this not to happen without the trailing ",").

xpath.py: parse(): _paths(): Remove trailing ","

xpath_func.py: _forEach: Made syntax more natural-looking by using values instead of names for string args and attrs instead of branches for array args

xpath.py: parse() Fixed bug in _paths() where empty lists would be parsed as a list containing a single empty path, instead of as an empty list

VegBIEN mappings: Place names: Use _forEach to simplify XPaths for recursively nested places

bin/map: In debug mode, print output XPaths

xpath_func.py: _forEach: Fixed to support _val replacements anywhere, by doing a string-based search-and-replace on a quoted XPath instead of a list-based search-and-replace on an already-parsed XPath

xpath_func.py: Renamed _for to _forEach. Finished implementing _forEach.

xpath.py: Import xpath_func after defining XpathElem because xpath_func depends on XpathElem and it hasn't yet been factored into a separate file

util.py: Added list_replace()

xpath_func.py: Changed XPath function signature to take arguments (args, path), and process() to parse out the args. Implemented basic for that repeats its do arg as many times as there are in elements.

xpath.py: parse(): Run xpath_func.process() on the parsed XPath