VegBIEN: Renamed plot to location to reflect that a location can also be used for specimens
vegbien.sql: Added individualplant.stemcount
vegbien.sql: Added accessioncode fields to existing, modified tables
vegbien.sql: Added accessioncode fields to new tables
vegbien.sql: Removed no longer needed individualplant.taxonarea
VegBIEN: Moved taxondetermination.sourceaccessionnumber and museum_id to specimen
vegbien.sql: Split taxondetermination UNIQUE CONSTRAINT into one constraint for each main key (taxonoccurrence_id, specimen_id, individualplant_id) to deal with Postgres treating a row with NULL in any of those columns as distinct. Added CHECK constraint to require one of taxonoccurrence_id, specimen_id, individualplant_id to be NOT NULL.
VegBIEN: Renamed museumaccessionnumber to sourceaccessionnumber
VegBIEN: Renamed authorcode to sourceid
vegbien.sql: Updated sizeclass to use ranges for parameters and remove taxonarea
input.Makefile: Fixed verify by making %.out: % rule use %.make instead
input.Makefile: Moved verifications into one set of verify.* files
input.Makefile: All tests are run in the same transaction, with output to test.out
bin/map: Added support for processing multiple map_paths at once in the same transaction
SALVIAS organisms mapping: Removed redundant PlotCode mapping because the association to plotevent is done with PlotID
bin/map: Started adding support for multiple map_paths, all run in the same db transaction
Filled in SALVIAS stems mappings. make test2's stems test fails because we don't wrap the import transaction around all the tests so that organisms are already in the database when stems are imported.
mappings/Makefile: Corrected stemobservation to stem
Use baseDistance/complexUserDefined/method to distinguish between different dbhs in VegX
VegX mappings: Use baseDistance/value instead of baseDistance so we can later use complexUserDefined to distinguish between different types of dbh
VegX-VegBIEN.organisms.csv: Added initial stems mappings
mappings/Makefile: Updated VegX-VegBIEN.stems.csv input/output roots
ch_root: Deal with roots followed by something other than \b
input.Makefile: Remove *.out in new verify and test subdirs
Moved inputs files into subdirs for maps, verify, and test operations
vegbien.sql: Removed taxondetermination.collector_id because that's now in specimen
VegBIEN: Map to specimen.collector_id instead of taxondetermination
vegbien.sql: Added specimen.collector_id column
vegbien.sql: Changed taxondetermination.stem_id to individualplant_id
vegbien.sql: Removed taxondetermination columns that have been moved to specimen
VegBIEN: Merged specimen.collectionnumber and catalognumber_dwc fields. Removed NOT NULL constraints on several specimen columns because not all specimens have all identifiers. Mapped to new specimen fields.
VegX-VegBIEN mapping: Map collectiondate to specimen table
vegbien.sql: Added specimen.collectionnumber and collectiondate fields
Updated README.TXT to reflect new make targets
vegbien.sql: Added sizeclass columns
VegBIEN: Removed stem* prefix from stem measurements because it's redundant and because these same field names were reused in non-stem tables
VegBIEN: Renamed stemobservation to stem
vegbien.sql: Added comments to renamed tables specifying VegBank equivalent
vegbien.sql: Added comments to new tables
vegbien.sql: Removed r# prefix from foreign key constraints
vegbien.sql: Added plot.namedplace_id for theplots which are identified by a namedplace rather than lat/long or name
vegbien.sql: Made plot.authorcode optional as not all plots have names
vegbien.sql: Added _dwc to specimen columns that are from DwC
vegbien.sql: Added UNIQUE constraint for specimen table. Made catalognumber required so that all specimens can be uniquely identified.
vegbien.sql: Made specimen.individualplant_id optional because specimens are not required to have a corresponding plot event (which all individualplants must have)
VegBIEN-VegBank.csv: Updated to reflect renames
vegbien.sql: Renamed specimen.code to collectioncode to match DwC term
vegbien.sql: Added sizeclass table
vegbien.sql: Renamed collection to specimen to reflect that it's for a single collection specimen
VegBIEN: Renamed collectiveobservation to aggregateoccurrence to reflect VegX usage of aggregate and the new taxonoccurrence table
VegBIEN: Renamed stemobservation.stemcode to authorcode
vegbien.sql: Renamed individualplant.code to authorcode
sql.py: Reordered get() args to match put()
db_xml.py: Use new sql.put() which only eliminates duplicates based on database UNIQUE constraints
sql.py: Added new function put() and changed get() to use it
sql.py: Fixed raising of NullValueException to pass array of columns rather than single column
vegbien.sql: Added UNIQUE constraints to tables used in the mapping so that duplicates are eliminated by the database
sql.py: ExceptionWithColumn now stores multiple columns so that they can all be used in DuplicateKeyExceptions
util.py: Fixed dict_subset() to work correctly when a key does not exist
util.py: Added dict_subset()
sql.py: Use constraint_cols() to determine DuplicateKeyException column in try_insert()
sql.py: Added constraint_cols() to get columns of a constraint for use in determining the columns for a DuplicateKeyException
db_xml.py: Map empty fields to None instead of ignoring them, to ensure that an existing row with a value in that field is not used in the duplicate elimination
bin/map: Use db_xml.put()'s new on_error handler
db_xml.py: put(): Added on_error handler. Reordered arguments according to how often they are likely to be used.
bin/map: Print # input rows processed
VegX-VegBIEN mapping: Merge different sources of plot elevation
vegbien.sql: Added individualplant.code field
xml_func.py: Expand XML funcs in XML func args
xml_dom.py: In NodeTextEntryIter, return non-text children as-is instead of ignoring them so that XML functions can use them
xml_dom.py: Improved readability by separating into sections
db_xml.py: Ignore only fields that are actually empty (rather than just not having element children), as a field containing anything other than a comment should be flagged as invalid input
xml_dom.py: Changed is_empty() to consider nodes with only comments to be empty
bin/map: Add input row to exceptions
db_xml.py: Ignore empty fields. Fixes bug where a field that's not text but has no Element children will be treated as a table, and an error is triggered because the corresponding table does not exist in the database.
xml_dom.py: Added has_elems() to check if node has Element children
bin/map: Use xml_func.process()'s new on_error error handler. Add row to database errors and xml_func.SyntaxExceptions.
xml_func.py: Changed formatting of comment that node with syntax errors is replaced with so that it starts on its own line
xml_func.py: Use on_error exception handler to allow caller to print exceptions instead of raising them
db_xml.py: Add node to any database exceptions generated in put()
exc.py: Added raise_() wrapper for raise statement
Ignore .~* (lock files generated by OpenOffice.org) in inputs
inputs/SALVIAS: Added stems map
bin/map: Handle empty rows for db output
mappings: Use stemParent instead of stemOrganism to identify stems so that it's clear that the stemParent is an individualOrganismObservation, not an individualOrganism
input.Makefile: Include VegX maps not in tablesSort
mappings/Makefile: Generate initial VegX-VegBIEN.stems.csv
mappings/Makefile: Cleaned up in preparation for adding rule to generate VegX-VegBIEN.stems.csv
bin/map: Continue with database insertion even if a row had XML func syntax errors, because these errors have been fixed by removing the offending function node
xml_func.py: Remove function nodes with syntax errors to prevent them from causing DatabaseErrors later
xml_dom.py: Added remove() function to remove a node
bin/map: Track Exceptions with ExTracker
exc.py: Added ExTracker to track printed (suppressed) Exceptions and exit with nonzero status if any were encountered
bin/map: Parse each output XPath at the beginning so XPath syntax errors are detected right away
vegbien.sql: Reran renames to capture cases where a name is in the middle of an identifier
vegbien.sql: Use character varying(255) instead of text or character varying in new tables and columns
input.Makefile: Print Postgres NULL as NULL instead of empty string to match MySQL output
VegX-VegBIEN mapping: Fixed syntax error in mapping
vegbien.sql: Added trait table
vegbien.sql: Added columns to stemobservation