map: Added support for XML input

Accepted new test output for sorted SALVIAS_db-VegBank mapping

mappings to VegBank: Sorted by output column to help VegX-VegBank conversion put elements in the same order as source-VegBank

join_all_vegbank: Updated to sort output maps

Added script to sort a spreadsheet

xpath.py: Allowed empty names in XPaths

xpath.py: Added automatic conversion of strings to paths where needed.

xpath.py: Added caching of parsed XPaths. Added automatic conversion of strings to paths where needed.

Added str() method to XML nodes

Fixed VegX-VegBank mapping syntax error

Added faded beginning of string in Parser syntax errors

Updated mappings Makefile

Added Makefiles for scripts and test

Added mappings Makefile

Added human-readable SALVIAS_db mappings

db_xml.get(): Pass limit through to SQL query

Regenerated human-readable mappings

Fixed documentation for xml_funcs

Refactored xml_dom.set_value() to avoid needing a doc parameter for the XML document

xpath.py: Refactored xml_func.py to avoid needing a doc parameter for the XML document

xpath.py: Refactored to avoid needing a doc parameter for the XML document

Fixed DB input to ignore NULL values

xml_dom.py: Changed all uses of name_of(node) to node.tagName

Made XML node names case-sensitive

mappings to VegBank: Fixed incorrect mappings found after disabling heuristic search for missing fields

test/map: Ignore diff exit status

map: Implemented DB input support for querying a single table

Added SALVIAS_db test accepted output

map: Continued to add DB input support

test/map: Echo command used to import db config

Added support for multiple database engines. Changed SALVIAS_db input to use user-entered password.

map: Allow db config vars to be optional. SALVIAS_db test: Changed to use salvias_plots and XPath mapping syntax.

Renamed SALVIAS_db test input to use organisms table

Re-committed accepted_outputs

Renamed test/map output to remove CSV/DB indicator because that is now specified in the datasource name

map: Started adding database get by XPath functionality

format_for_review: Fixed bug where Comments column would be reformatted in addition to mappings columns

Regenerated human-readable mappings

Added human-readable versions of mappings and scripts to generate them

VegX-VegBank mapping: Removed a duplicated mapping

NYBG-VegX mapping: Added conference call feedback

Added Comments column with Brad's and Aaron's comments to mapping spreadsheets

Added stub for SALVIAS database test

test/map: Added support for database input

Preparing map to input from DB

Started preparing map to input from DB

xml_func.py: Added optimization to first check if function name starts with _ before looking it up in the table

Added _alt functions for mappings to VegBank authorPlotCode

xml_func.py: Added _alt function to choose between alternative values and used it for the collector plantName mapping

VegX-VegBank mapping: Added mapping from taxonName/Simple (NYBG ScientificName) to collector plantName so that collector plantName will always have a value

xml_func.py: Added support for decimal years (with day as the fraction)

test/map: Added echoing of commands run

Added psql_vegbank to easily access vegbank db from the command line

Ignore OpenOffice lock files in mappings

Added SALVIAS data CSVs and accepted test output

test/map: Expanded to include all input CSVs in test/input

Removed unneeded joins dir

Moved VegBank mapping joins to main mappings dir so they would have similar paths for the upcoming all-sources tester

Moved test scripts and files from util to test

xml_func.py: Added _namepart function for extracting parts of names

Finished NYBG mapping to VegBank\!

test_map: Added debug option to print VegBank XML instead of importing it into the database

xpath.py: Created is_positive() function

Further refinements to mappings to support database constraints

xpath.py: Added support for negative attribute assertions with !

Changed mappings to use keys vs. attrs

xpath.py: Fixed creation of attrs so it happens even when node already exists

xpath.py: Added concept of keys vs attrs in XPath elem

Started filling in required values for VegBank fields in mappings. Will need to refactor to move these to metadata for the datasources.

Now allow empty rows. Added support for select statement limit.

Added support for quoted values in XPaths

Fixed name XML function. Fixed accept_test_output.

Added support for name XML function. Added error handling for empty rows.

Made it easier to accept test output

Added NYBG stemCount metadata

Added xml_func.py to process mappings whose output needs postprocessing

Changed VegBank mappings to use XML functions (not implemented yet) to calculate averages and ranges

Added support for mapping datasource metadata

Changed for loops to use enumerate() where the index is also needed

Moved XPath prep code (setting ID, value) to xpath.py

xpath.py: Added deepcopy() before setting value of other branches to traverse

NYSpecimenDataAmericas.test.xml: Updated for new NYBG-VegX.organisms.csv

NYBG-VegX.organisms.csv: Changed voucher (primary key) column to be UniqueNYInternalRecordNumber because CatalogNumber contained an empty value

xpath.py: Added basic support for split paths

Merged xml_xpath.py into xpath.py in preparation for changing the XPath parse tree to be the XML DOM tree itself

Refactored xpath.parse() to use a nested function instead of a class extending Parser

map: Fixed mislocated import for Parser.SyntaxException

Removed SALVIAS voucher_string mapping per conference call discussion

map: Fixed bugs to enable mapping straight from CSV to a database. Still need a way to set plot.authorPlotCode for specimens data.

Fixed ch_map_root to support subpaths which follow the root by -> rather than /. Changed spreadsheet syntax to have : between label and root.

Updated extract_plot_map to use new name for VegX-VegBank mapping and re-ran it and join_all_vegbank

Finished VegX-VegBank mapping and created VegBank joins of mappings to VegX

Finished ch_map_root (renamed from submap)

Added submap and extract_plot_map to extract plot subpaths from VegX-VegBank.csv

Moved env usage string creation to opts.py. Changed db config var names to use in/out instead of from/to.

Keep *.test.xml out of version control

Moved options-processing code to opts.py: Added opts.py

Moved options-processing code to opts.py

test_map: Compares generated XML to correct version

Fixed xml_xpath.get() last_only optimization to handle attrs correctly. Turned off stack traces for errors intended for the user to see.