Makefile: Added documentation labels to each section
input.Makefile: Added documentation labels to each section
xml_dom.py: Fixed conversion of minidom.Node to a string to use toprettyxml() again
Added inputs/SALVIAS-CSV/maps/VegBIEN.* maps
input.Makefile: If a test doesn't yet have accepted output, print its current output
mappings/Makefile: Only clean up a core map (intermediate format->VegBIEN) when it's modified. Note that the .last_cleanup files need to be under version control to prevent cleanups upon running any map command in a new checkout.
xml_func.py: Support dates with year/month/day == 0
input.Makefile: Print informative message if test failure is ignored
input.Makefile: Print path to test output so you can tell which directory a test is running in
input.Makefile: Fixed 2-step tests, which were still using $(root)/map when all bin/map calls were supposed to be routed through $(map)
xml_dom.py: Print simple XML elements on one line
Added inputs/SALVIAS-CSV/maps
input.Makefile: Added import support for CSV and XML inputs (no test support yet)
inputs/SALVIAS-CSV: Added src files
inputs: Removed no longer needed svn:ignore
input.Makefile: Coded svn:ignore directly in Makefile
inputs/Makefile: Added %-add target to add new inputs
input.Makefile: Added svn_props rule for updating svn:ignore of dir and subdirs
vegbien.sql: Made aggregateoccurrence:taxonoccurrence relationship 1:1
Added inputs/CTFS/src, with subset of files that will be in version control
mappings/for_review/DwC-VegBIEN.specimens.csv: Updated
inputs/NYBG: Map via DwC instead of VegX
mappings/DwC-VegBIEN.specimens.csv: Provide schema URLs for namespaces (in the column header)
input.Makefile: Deal with core maps (the main *-VegBIEN map) whose roots are other than organisms
bin/map: In the xml_func.process() for XML outputs, pass an on_error that outputs to ex_tracker.track()
xpath.py: set_id(): Deal with XPaths with an
xml_dom.py: replace_with_text(): Accept Node inputs for cases where e.g. an XML func with an error has been replaced with a comment
xml_func.py: _date: Year defaults to 1900
DwC-VegBIEN.specimens.csv: Output DwC as XML instead of CSV
xpath.py: Allow XPath names to be quoted literals, just like values. Handle : embedded in a name (even without quotes), often indicating a namespace.
Removed no longer needed lib/ERD-wiki.csv
Parser.py: In syntax errors, only print <END> at end of string if error was at end of string
bin/map: Print error message rather than assertion failure for CSV output, which isn't supported yet
input.Makefile: Don't abort tester if via test fails, because CSV output is not supported yet
input.Makefile: Deal properly with more than one format that's being mapped via (e.g. DwC and VegX)
Renamed mappings/DwC-VegBIEN.organisms.csv back to mappings/DwC-VegBIEN.specimens.csv because it now uses /specimenreplicate as the output root
mappings/DwC-VegBIEN.organisms.csv: Fixed syntax error in a mapping
mappings/DwC-VegBIEN.organisms.csv: Sorted
mappings/Makefile: Sort DwC-VegBIEN.organisms.csv
inputs/NYBG/maps/VegBIEN.organisms.csv: Regenerated
mappings/DwC-VegBIEN.organisms.csv: Sort on output column
Renamed mappings/DwC-VegBIEN.specimens.csv to DwC-VegBIEN.organisms.csv because it (currently) uses /taxonoccurrence as the output root
Makefile: Support recursing into subdir when no target is specified (e.g. make mappings/)
union: Merge rows (and column labels) using maps.merge_mappings()
join: Merge the column labels as well
maps.py: Eliminate duplicates when merging values in the same column
join: Moved mappings-specific merge functionality into maps.merge_mappings()
join: Use merge_rows() from new maps.py
Added new library maps.py for map spreadsheet manipulation
join: Merge comments of input map and join map
join: Report which input mappings are missing a mapping in the join map
inputs/NYBG/maps/VegX.organisms.csv: Added note that the primary key has NULL values in some rows
input.Makefile: Preserve as many intermediate files as possible (make likes to delete intermediates if they aren't marked as .PRECIOUS)
sort: Sort empty strings last so that inputs with no mapping go at the end of the map spreadsheet
VegBIEN-VegBank.csv: Updated for recent table renames
join: For input mappings with no match in the join map, include them in the output map with an empty mapping
input.Makefile: Generalized to handle mapping via any format, not just VegX
input.Makefile: Don't print message to accept output for failing 2-step tests, because they use another test's accepted output
input.Makefile: Don't abort tester if only 2-step test fails, as it's often finicky
xml_func.py: Raise xml_func.SyntaxException for ValueErrors generated by date.strftime() (e.g. year out of range due to poor Y2K support in some OS implementations of strftime)
xml_func.py: Raise xml_func.SyntaxException for ValueErrors generated by datetime.date() (e.g. month out of range)
vegbien.sql: Added project.reference_id to namespace project names by datasource
input.Makefile: Import all tables at once by default
bin/map: Print "Inserted ... new rows into database" message to stdout rather than stderr so it can be stored in the test case output as a validation check
Accepted initial test output for NYBG/test/import.out.ref
bin/map: Clean up datasource input values
strings.py: Added std_newl() to convert line endings and cleanup() to process strings with extra or nonstandard whitespace
PostgreSQL-MySQL.csv: Deal with custom types
vegbien.sql: Added aggregateoccurrence.occurrencestatus_dwc field
Regenerated vegbien.ERD exports
vegbien.ERD.mwb: Added commclass table to ERD
vegbien.sql: Removed direct pointer from location to namedplace because locationplace already has this relationship and we don't want to have an extra pointer just for duplicate elimination
vegbien.ERD.mwb: Added stratummethod to ERD
vegbien.sql: Removed locationevent.stratummethod_id because the stratummethod is a per-stratum (or technically, per-stratumtype) field
PostgreSQL-MySQL.csv: Remove CHECK constraints
PostgreSQL-MySQL.csv: Remove functions and triggers
vegbien.sql: Ensure that aggregateoccurrence.count == 1 when the aggregateoccurrence has a plantobservation. Use a trigger to do this automatically.
README.TXT: Added command for reimporting data
README.TXT: Added instructions to sync ERD with vegbien.sql schema. Organized commands into categories.
Added BIEN_logo.png
vegbien.ERD.mwb: Added color group legend
vegbien.ERD.mwb: Fixed lines
vegbien.ERD.mwb: Fixed lines and moved plant to its own color category
vegbien.ERD.mwb: Added colors to ERD
vegbien.ERD.mwb: Simplified diagram by removing column types
schemas/Makefile: Don't generate for_ERD DDLs because the ERD is now synced with the full schema
vegbien.ERD.mwb: Synced with whole schema
vegbien.sql: Reordered fields in tables truncated in the ERD so that all removed fields are at the end of the table
schemas/Makefile: Generate MySQL version of vegbien.sql as well as vegbien.for_ERD.sql for eventual use in syncing the ERD with the whole schema
PostgreSQL-MySQL.csv: Added translations for syntaxes used by pg_dump
repl: All regexps are by default in multiline and ignore case mode
