mappings to VegBank: Fixed incorrect mappings found after disabling heuristic search for missing fields

test/map: Ignore diff exit status

map: Implemented DB input support for querying a single table

Added SALVIAS_db test accepted output

map: Continued to add DB input support

test/map: Echo command used to import db config

Added support for multiple database engines. Changed SALVIAS_db input to use user-entered password.

map: Allow db config vars to be optional. SALVIAS_db test: Changed to use salvias_plots and XPath mapping syntax.

Renamed SALVIAS_db test input to use organisms table

Re-committed accepted_outputs

Renamed test/map output to remove CSV/DB indicator because that is now specified in the datasource name

map: Started adding database get by XPath functionality

format_for_review: Fixed bug where Comments column would be reformatted in addition to mappings columns

Regenerated human-readable mappings

Added human-readable versions of mappings and scripts to generate them

VegX-VegBank mapping: Removed a duplicated mapping

NYBG-VegX mapping: Added conference call feedback

Added Comments column with Brad's and Aaron's comments to mapping spreadsheets

Added stub for SALVIAS database test

test/map: Added support for database input

Preparing map to input from DB

Started preparing map to input from DB

xml_func.py: Added optimization to first check if function name starts with _ before looking it up in the table

Added _alt functions for mappings to VegBank authorPlotCode

xml_func.py: Added _alt function to choose between alternative values and used it for the collector plantName mapping

VegX-VegBank mapping: Added mapping from taxonName/Simple (NYBG ScientificName) to collector plantName so that collector plantName will always have a value

xml_func.py: Added support for decimal years (with day as the fraction)

test/map: Added echoing of commands run

Added psql_vegbank to easily access vegbank db from the command line

Ignore OpenOffice lock files in mappings

Added SALVIAS data CSVs and accepted test output

test/map: Expanded to include all input CSVs in test/input

Removed unneeded joins dir

Moved VegBank mapping joins to main mappings dir so they would have similar paths for the upcoming all-sources tester

Moved test scripts and files from util to test

xml_func.py: Added _namepart function for extracting parts of names

Finished NYBG mapping to VegBank\!

test_map: Added debug option to print VegBank XML instead of importing it into the database

xpath.py: Created is_positive() function

Further refinements to mappings to support database constraints

xpath.py: Added support for negative attribute assertions with !

Changed mappings to use keys vs. attrs

xpath.py: Fixed creation of attrs so it happens even when node already exists

xpath.py: Added concept of keys vs attrs in XPath elem

Started filling in required values for VegBank fields in mappings. Will need to refactor to move these to metadata for the datasources.

Now allow empty rows. Added support for select statement limit.

Added support for quoted values in XPaths

Fixed name XML function. Fixed accept_test_output.

Added support for name XML function. Added error handling for empty rows.

Made it easier to accept test output

Added NYBG stemCount metadata

Added xml_func.py to process mappings whose output needs postprocessing

Changed VegBank mappings to use XML functions (not implemented yet) to calculate averages and ranges

Added support for mapping datasource metadata

Changed for loops to use enumerate() where the index is also needed

Moved XPath prep code (setting ID, value) to xpath.py

xpath.py: Added deepcopy() before setting value of other branches to traverse

NYSpecimenDataAmericas.test.xml: Updated for new NYBG-VegX.organisms.csv

NYBG-VegX.organisms.csv: Changed voucher (primary key) column to be UniqueNYInternalRecordNumber because CatalogNumber contained an empty value

xpath.py: Added basic support for split paths

Merged xml_xpath.py into xpath.py in preparation for changing the XPath parse tree to be the XML DOM tree itself

Refactored xpath.parse() to use a nested function instead of a class extending Parser

map: Fixed mislocated import for Parser.SyntaxException

Removed SALVIAS voucher_string mapping per conference call discussion

map: Fixed bugs to enable mapping straight from CSV to a database. Still need a way to set plot.authorPlotCode for specimens data.

Fixed ch_map_root to support subpaths which follow the root by -> rather than /. Changed spreadsheet syntax to have : between label and root.

Updated extract_plot_map to use new name for VegX-VegBank mapping and re-ran it and join_all_vegbank

Finished VegX-VegBank mapping and created VegBank joins of mappings to VegX

Finished ch_map_root (renamed from submap)

Added submap and extract_plot_map to extract plot subpaths from VegX-VegBank.csv

Moved env usage string creation to opts.py. Changed db config var names to use in/out instead of from/to.

Keep *.test.xml out of version control

Moved options-processing code to opts.py: Added opts.py

Moved options-processing code to opts.py

test_map: Compares generated XML to correct version

Fixed xml_xpath.get() last_only optimization to handle attrs correctly. Turned off stack traces for errors intended for the user to see.

Changed mappings to place prefix common to all XPaths in the column header

simplify_xpath: Made it case-insensitive

map: Added support for custom fkeys to parent in db XML trees. Removed extraneous csv reader/writer config because Excel format is default. Improved documentation.

map: Added stub for database input

map: Added more stubs for XML-XML mapping

Started adding XML-XML mapping support to map

Split off xpath.py XML functionality into xml_xpath.py

map: Using SystemExit for usage errors to avoid stack trace

Merged data2xml and xml2db into map

Removed trailing whitespace from VegX-VegBank.csv map

Created join_maps to join two 2-column map spreadsheets

Renamed mappings to be compatible with Redmine allowed characters in attachment filenames

Added refactored mappings and changed data2xml to use the new 2-column format

Refactored db_xml.py's db insertion function to avoid extra nested functions

Added README.TXT

Renamed modules to remove _util

Added svn:ignore for *.pyc

Renamed xml2db_ and data2xml_ to remove _

Moved scripts to main directory and associated files to util

Moved Python modules to shared lib folder

xml2db: Started refactoring xml2db() to support getting as well as inserting data

xml2db: Started refactoring xml2db() to support getting as well as inserting data

xml2db: Changed to return ID (pkey) of inserted record and use this returned value as parent_id instead of getting the parent_id from the parent XML node

data2xml: Added syntax for split paths, which map to multiple leaves