xml_dom.py: Changed is_empty() to consider nodes with only comments to be empty
bin/map: Add input row to exceptions
db_xml.py: Ignore empty fields. Fixes bug where a field that's not text but has no Element children will be treated as a table, and an error is triggered because the corresponding table does not exist in the database.
xml_dom.py: Added has_elems() to check if node has Element children
bin/map: Use xml_func.process()'s new on_error error handler. Add row to database errors and xml_func.SyntaxExceptions.
xml_func.py: Changed formatting of comment that node with syntax errors is replaced with so that it starts on its own line
xml_func.py: Use on_error exception handler to allow caller to print exceptions instead of raising them
db_xml.py: Add node to any database exceptions generated in put()
exc.py: Added raise_() wrapper for raise statement
Ignore .~* (lock files generated by OpenOffice.org) in inputs
inputs/SALVIAS: Added stems map
bin/map: Handle empty rows for db output
mappings: Use stemParent instead of stemOrganism to identify stems so that it's clear that the stemParent is an individualOrganismObservation, not an individualOrganism
input.Makefile: Include VegX maps not in tablesSort
mappings/Makefile: Generate initial VegX-VegBIEN.stems.csv
mappings/Makefile: Cleaned up in preparation for adding rule to generate VegX-VegBIEN.stems.csv
bin/map: Continue with database insertion even if a row had XML func syntax errors, because these errors have been fixed by removing the offending function node
xml_func.py: Remove function nodes with syntax errors to prevent them from causing DatabaseErrors later
xml_dom.py: Added remove() function to remove a node
bin/map: Track Exceptions with ExTracker
exc.py: Added ExTracker to track printed (suppressed) Exceptions and exit with nonzero status if any were encountered
bin/map: Parse each output XPath at the beginning so XPath syntax errors are detected right away
vegbien.sql: Reran renames to capture cases where a name is in the middle of an identifier
vegbien.sql: Use character varying(255) instead of text or character varying in new tables and columns
input.Makefile: Print Postgres NULL as NULL instead of empty string to match MySQL output
VegX-VegBIEN mapping: Fixed syntax error in mapping
vegbien.sql: Added trait table
vegbien.sql: Added columns to stemobservation
vegbien.sql: Renamed collectiveobservation.stratum_id to taxonbin_id
vegbien.sql: Added some DarwinCore attributes to collection table
Makefiles: Added remake target to run clean and all
vegbien.sql: Added taxondetermination.collection_id column
VegBIEN: Made taxondetermination.plantname_id required
vegbien.sql: Made taxondetermination.plantconcept_id optional
VegBIEN: Renamed author*code to authorcode
Makefiles: Added empty rules for targets without a rule so that these rules wouldn't trigger the recursive subdir invocation for the default target
VegX-VegBIEN mapping: Convereted all database names to lowercase to correspond to their case in the database
Makefiles: Added empty Makefile target so that Makefiles are not included in the targets recursively run on subdirs
input.Makefile: Dependencies in other directories are included as prerequisistes. Use tablesSort variable instead of sort order prefix to order tables.
Makefile: Added default target that recursed into subdirs with Makefiles. Used default target for all and clean.
Regenerated inputs/SALVIAS/map.VegBIEN.0.plots.csv
Removed scripts symlink because it was causing Makefile recursion issues
Makefile: Take advantage of new inputs/Makefile
Added inputs/Makefile to run input targets recursively on each input
Makefile: Run inputs-* targets for all and clean
input.Makefile: all doesn't run import. import runs verify.
vegbien.sql: Set all FOREIGN KEY constraints to ON UPDATE CASCADE ON DELETE CASCADE
SALVIAS mappings: use PlotID as authorObsCode to link plot observations and organisms correctly for organisms without a PlotCode
input.Makefile: Fixed bug in verify's diff rule
Added CTFS input
Makefile: Generate error when trying to run subMake on a directory with no Makefile
input.Makefile: Tests use bin/map's new new test mode
bin/map: Added test mode that empties the database before doing anything
sql.py: Added functions to truncate tables, list all tables, and empty a database
Accepted new SALVIAS test outputs
Makefile: Print Finished message after running test2 (inputs tests)
Accepted test outputs for SALVIAS
input.Makefile: Added ability to accept a test output to use as the new reference output
vegbien db: Renamed all occurrences of interpretation to determination
Makefile: Tell make that subMake rules run make recursively
vegbien.sql: Updated collection table for recent refactorings
Made more of Bob Peet's changes to the vegbien db and updated VegX-VegBIEN mapping
Makefile: Fixed pattern for recursive invocation of subdirectory make
Moved mappings/schemas to schemas
Makefile: Changed all to make schemas and mappings instead of running install
Makefile: Added inputs import target
input.Makefile: Fixed syntax error with use of import_cmd variable
vegbien.sql: Added collection table
Makefiles: Changed _not_file to _always because _not_file is sometimes used to force a file to always be remade
Decoupled VegBIEN from VegBank and removed mappings to VegBank
Regenerate vegbien.sql by dumping the vegbien db
psql_vegbien: Changed to use postgres_vegbien
Added postgres_vegbien to run PostgreSQL command on the vegbien db
SALVIAS verification: Added plot methods
SALVIAS verification: Added # plots
Makefile: Added targets to run input targets on all inputs
SALVIAS verification: Added projects list to examine why project counts differ
SALVIAS verification: Made statement-label SELECT statements use the same syntax for MySQL and PostgreSQL so they are easy to copy and paste between the input and output queries
input.Makefile: Don't remove logs in clean
input.Makefile: Added support for multi-column verification queries. inputs/SALVIAS: Added a multi-column verification query.
input.Makefile: Remove all *.out in clean
inputs: Don't keep *.out under version control
input.Makefile: Added rules to verify inserted data and run tests
Regenerated inputs/SALVIAS/map.VegBIEN.1.organisms.csv
exc.py: Print ! before error messages so they can be found and counted more easily in log files
input.Makefile: Only output to log if importing all rows
Moved vegbien CREATE DATABASE command to Makefile and run vegbien.sql as bien user
Renamed vegbien tables according to Bob Peet's new VegBank-VegBIEN mapping
input.Makefile: Don't print commands that use set -x to be echoed
input.Makefile: Use system's time utility instead of shell's time command
sql.py: Include sql.DbException in DatabaseErrors list
Generate VegX-VegBIEN.organisms.csv from VegBank-VegBIEN.map.csv, which is generated from VegBank-VegBIEN.map_only.csv
repl: Process whole file as one string so that regexps can match EOF for appending
mappings/Makefile: Clean -VegBIEN..csv maps
Generate VegBank-VegBIEN.schema.csv from VegBank-VegBIEN.schema_only.csv and VegBank-VegBIEN.csv
Added cat_csv to concatenate two spreadsheets
Removed no longer needed inputs/SALVIAS/db.sh
