sql.py: Use esc_name() to escape fields in SELECT statements
sql.py: Added esc_name() to escape identifiers like column names
vegbien.sql: Added comments to specimen.collectioncode_dwc and collectionnumber to differentiate them
vegbien.sql: Renamed authorspecimencode to collectionnumber to match its name in source data
input.Makefile: Use pipefail to cause a test to fail even when the output is filtered by grep. Print message for failing tests with command to run to accept the new test output.
VegX-VegBIEN mapping: Map additional taxondetermination.determinationdate input formats straight through
test/input/SALVIAS_db.sh: Updated DB name
Regenerated vegbien.ERD exports
input.Makefile: Added documentation for why import errors for one input do not abort the import process for all inputs
input.Makefile: Determine DB name from input directory name, rather than DB file name
input.Makefile: Added documentation for accepting a test output
mappings/Makefile: Don't delete DwC-VegBIEN.specimens.csv in clean
VegBIEN: Renamed taxondetermination.*determination to is*
inputs/SALVIAS/test: Ignore test outputs
input.Makefile: Added test that generates VegBIEN.2-step.xml by mapping via a VegX.xml
input.Makefile: Added test that generates VegX.xml
input.Makefile: Added test that generates VegBIEN.xml
input.Makefile: Factored test/import.out out of DB section
input.Makefile: Renamed test/import.ref to import.out.ref. Changed syntax for accepting a test output to work with all types of test outputs.
Makefiles: Recurse into outermost subdir rather than bypassing it and going directly to innermost subdir
input.Makefile: Deal with inputs without a DB file, tests, verifications, etc.
inputs/Makefile: Don't use subdir makefiles because they are no longer needed
input.Makefile: Detect DB engine automatically from SQL file available in src subdir
input.Makefile: Factored as much as possible out of section for each DB engine
input.Makefile: Moved tests into test subdir
Added initial DwC-VegBIEN mappings spreadsheet with DwC terms
inputs/SALVIAS/verify: Updated to use new names for renamed tables
vegbien.sql: Removed no longer needed specimen.collector_id
VegX-VegBIEN mapping: Map collector name to new verbatimcollectorname field
vegbien.sql: Removed specimen.collectornumber_dwc and replaced it with verbatimcollectorname to reflect that the collectornumber_dwc is actually an ID of the specimen, and the collector's name is what we want to store
mappings/Makefile: Run simplify_xpath on VegX-VegBIEN.organisms.csv
simplify_xpath: Be case sensitive to handle VegX correctly
VegX-VegBIEN mapping: Avoid using a dummy taxondetermination with role=collector
VegX-VegBIEN mapping: Map stem count to new stemcount field
VegX-VegBIEN mapping: Take advantage of aggregateoccurrence.count being optional
vegbien.sql: Made aggregateoccurrence.count to handle individuals data (for which count should be dynamically determined from # individual plants inside the aggregateoccurrence)
NYBG-VegBIEN mapping: Don't map dummy values to locationcode, etc. (e.g. in specimens data) because these tables are no longer required
vegbien.sql: Made several pointers to parent elements optional to deal with specimens data that might not have a location, etc.
vegbien.sql: Added taxondetermination UNIQUE constraint
VegX-VegBIEN mapping: Took advantage of location.confidentialitystatus being optional
VegX-VegBIEN mapping: Took advantage of userdefined.userdefinedtype being optional
vegbien.sql: Gave userdefined.userdefinedtype a default value
VegX-VegBIEN mappings: Took advantage of plantconcept.reference_id becoming optional
vegbien.sql: Made plantconcept.reference_id optional. Merge plantconcepts with no reference_id when eliminating duplicates.
PostgreSQL-MySQL.csv: Deal with all non-NOT NULL timestamp fields
vegbien.sql: Removed confusing plantconcept.plantname field since we are using plantname.plantname instead
VegBIEN: Renamed aux_role to role
VegX-VegBIEN mappings: Took advantage of several fields becoming optional
vegbien.sql: taxonbinmethod points to stratumtype instead of stratum because stratumtype is a method table, but stratum is a measurements table. stratum does not point directly to stratummethod because it points to it via stratumtype.
vegbien.sql: Made taxondetermination.determinationdate optional because some determinations might not have a date
vegbien.sql: Added specimen.authorspecimencode
Adjusted vegbien.ERD.mwb
VegBIEN: Renamed sourceaccessionnumber to sourceaccessioncode to show that they are the data source's analog of accessioncode. Added sourceaccessioncode to all applicable tables because this is the database pkey, which is distinct from any author*code applied by the collector.
vegbien.sql: Changed taxonbinmethod_keys to UNIQUE INDEX to take advantage of COALESCE for dealing with NULL values
vegbien.sql: Renamed taxonbin to taxonbinmethod to reflect that it does not contain actual organisms (those go in aggregateoccurrence), but rather defined a method of aggregating organisms
vegbien.sql: Removed taxonbin.count because that belongs in aggregateoccurrence and taxonbin is more similar to a sampling method. Added taxonbin UNIQUE constraint.
vegbien.sql: Do location duplicate elimination independently on code or lat/long, allowing duplicate entries with NULLs to exist when a location is incompletely specified
vegbien.sql: Require location to have either an authorlocationcode or a lat/long. Distinguish between regular and subplots in UNIQUE constraint.
vegbien.sql: Renamed location.latitude and longitude to publiclatitude, publiclongitude to reflect that they are not the actual lat/long. Switched to requiring reallatitude/reallongitude.
Added inputs/TurboVeg
vegbien.ERD.mwb: Deal with MySQL assuming that a timestamp field is NOT NULL
PostgreSQL-MySQL.csv: Deal with MySQL assuming that a timestamp field is NOT NULL
vegbien.sql: Made specimen.taxonoccurrence_id required
vegbien.sql: Made several fields optional, adding defaults where needed
PostgreSQL-MySQL.csv: Deal with PostgreSQL-style :: casts
NYBG mappings: Add mapping for CollectorNumber to specimen.collectornumber_dwc
vegbien.sql: Added specimen.collectornumber_dwc
VegBIEN: Renamed sourceid to author*code
mappings: Map ScientificNameAuthor to plantconcept with rank author
vegbien.sql: Removed sizeclass.*precision fields
vegbien.sql: Added count field to taxonBin
VegBIEN: Renamed place to locationplace
vegbien.sql.make: Exclude 'CREATE OR REPLACE PROCEDURAL LANGUAGE plpgsql' to avoid restore errors
repl: When matching words, treat _ as word boundary and ignore case.
VegBIEN: Renamed individualplant.height to overallheight
PostgreSQL-MySQL.csv: Convert double quotes to backticks (`)
vegbien.for_ERD.sql: Fixed syntax error in stratum table
vegbien.sql: Renamed sizeclass.*accuracy to *precision to reflect intended purpose
vegbien.sql: Added stem.diameteraccuracy
vegbien.sql: Removed taxonoccurrence.currentdetermination_id and originaldetermination_id because it's not possible to create a two-way pointer when the child-to-parent pointer is required (chicken-and-egg problem)
README.TXT: Added make empty_db
vegbien.sql: Added taxonoccurrence.currentdetermination_id and originaldetermination_id
vegbien.ERD.mwb: Added stratum and place tables
Added vegbien.ERD.png
Added vegbien.ERD.svg
Added vegbien.ERD.pdf
Added vegbien ERD
Ignore *.bak in schemas
VegBIEN: specimen links directly to taxonoccurrence rather than indirectly via individualplant
vegbien.sql: Made voucher fields NOT NULL
