union: Added full flag to turn off merging mappings that are in both maps, in order to review inputs which appear in both maps but map to different places
mappings/Makefile: Merged .VegX-VegCSV.stems.csv.last_cleanup into .%.last_cleanup, since VegX-VegCSV.stems.csv now uses the same cleanup operations as the other non-derived maps. Note that this automatically creates a file in for_review for VegX-VegCSV.stems.csv, which is currently identical to it.
mappings/Makefile: .%.last_cleanup: Removed simplify_xpath because non-derived maps will now have VegX XPaths in their Source column URLs, which should not be modified
mappings/Makefile: VegX-VegCSV.stems.csv: Removed autogeneration command because once file has been generated, regeneration is no longer needed
mappings/Makefile: Fixed bug where VegX-VegCSV.stems.csv needed to be removed from $(vegcsvMaps) so it wouldn't be deleted on `make clean`
mappings/VegX-VegCSV.stems.csv: Source: Put URLs in the order their terms appear in the VegCSV term name
mappings/VegX-VegCSV.stems.csv: Comments: Changed "Table name" to "Table" to be concise
mappings/VegX-VegCSV.stems.csv: Mapped VegX community fields
mappings/VegX-VegCSV.stems.csv: Mapped VegX cover-related fields
mappings/VegX-VegCSV.stems.csv: Changed authorPlantCode to the associated DwC term fieldNumber
mappings/VegX-VegCSV.stems.csv: Changed locationNarrative to the associated DwC term locality
mappings/VegX-VegCSV.stems.csv: Changed collectedDate to the associated DwC term eventDate
mappings/VegX-VegCSV.stems.csv: Added plot prefix to eventStartDate/eventEndDate to distinguish it from the DwC eventDate, which is the date the specimen was collected
mappings/VegX-VegCSV.stems.csv: Order within table: Updated order #s for salvias_plots terms that got changed to SALVIAS data dictionary terms
mappings/VegX-VegCSV.stems.csv: Changed collector name parts to the associated DwC term recordedBy
mappings/VegX-VegCSV.stems.csv: Mapped SALVIAS voucher type
mappings/VegX-VegCSV.stems.csv: Mapped collector name parts
mappings/VegX-VegCSV.stems.csv: Table names ("." prefixes) merged into name where possible, for consistency. computer taxonomic elements have not been merged because the field part should exactly match the corresponding DwC term.
mappings/VegX-VegCSV.stems.csv: Order within table: If Source has multiple URLs, ensure each source has its own order
mappings/VegX-VegCSV.stems.csv: Order within table: Separate orders of multiple elements with "," instead of ";", for consistency with the Source column
mappings/VegX-VegCSV.stems.csv: Changed authorPlotCode terms to a variation of VegX's plotName, for standardization with VegX
mappings/VegX-VegCSV.stems.csv: Changed uniqueIDs with table names to the table name + "ID", for standardization
mappings/VegX-VegCSV.stems.csv: Changed terms with table names to DwC terms where possible
mappings/VegX-VegCSV.stems.csv: Removed comments about alternate names, as these will be included in a separate "VegCSV-alt" mapping to "VegCSV-core" terms
mappings/VegX-VegCSV.stems.csv: Clarified comments about the inclusion of the table name
mappings/VegX-VegCSV.stems.csv: Mapped plotObservation user-defined terms
mappings/VegX-VegCSV.stems.csv: Mapped VegX plotObservation fields
mappings/VegX-VegCSV.stems.csv: Corrected sources of DwC terms to point to the actual DwC term, where needed. eventDate parts: Added source for VegBank field used as named suffix.
mappings/VegX-VegCSV.stems.csv: Corrected sources of VegX names to point to the actual VegX field name, where needed
mappings/VegX-VegCSV.stems.csv: Mapped SALVIAS stem tags
mappings/VegX-VegCSV.stems.csv: Corrected parent plot-only mappings by prefixing "parentPlot."
mappings/VegX-VegCSV.stems.csv: Mapped VegX //plot/plotName
mappings/VegX-VegCSV.stems.csv: Mapped VegX //plot/plotUniqueIdentifier
mappings/VegX-VegCSV.stems.csv: Source SALVIAS terms from the SALVIAS data dictionary when possible, to provide an automatic link to the description of the term. Having these direct links will also assist in creating a data dictionary for VegCSV and eventually VegBIEN (using mappings/VegCSV-VegBIEN.specimens.csv). Note that many SALVIAS terms exist only in the live database, as they are not part of the export format documented in the data dictionary.
mappings/VegX-VegCSV.stems.csv: Source VegBank terms directly from the appropriate VegBank data dictionary page, to provide an automatic link to the description of the term. Having these direct links will also assist in creating a data dictionary for VegCSV and eventually VegBIEN (using mappings/VegCSV-VegBIEN.specimens.csv).
mappings/VegX-VegCSV.stems.csv: Mapped VegX relativePlotPosition terms
maps with Order column: Renamed Order column to Order within table for clarity
maps with Source column: Added original column name to source URLs, so that source name is completely specified. For official DwC terms, this also allows linking directly to the term. Fixed nimoy phpMyAdmin links so that going to the link in a browser would take you straight there after login.
mappings/VegX-VegCSV.stems.csv: Corrected SALVIAS stem diameter terms to place original name (before expansion for clarity) in the Comments column instead of appending it to the source URL, because the source URL should point just to the table the term is in. The actual term is identified directly by its order # and indirectly by the name of the VegCSV term, which should be similar (if not, the original term should be listed in the comments).
mappings/VegX-VegCSV.stems.csv: Mapped SALVIAS stem diameter terms
mappings/VegX-VegCSV.stems.csv: Mapped VegX project terms
mappings/VegX-VegCSV.stems.csv: VegX plot terms: Added order
mappings/VegX-VegCSV.stems.csv: Mapped non-user-defined height XPath
mappings/VegX-VegCSV.stems.csv: Changed source of height to VegX, because there is a VegX height field
mappings/VegX-VegCSV.stems.csv: Mapped VegX plot terms except unique keys
mappings/VegX-VegCSV.stems.csv: Mapped remaining sourceAccessionCode user-defined terms to <VegX-table>.uniqueID
mappings/VegX-VegCSV.stems.csv: Corrected sources of VegX names to point to the appropriate element in veg.xsd, rather than the appropriate type, because the names we used actually came from veg.xsd's top-level elements rather than from the type names
mappings/VegX-VegCSV.stems.csv: Changed plantObservation.sourceAccessionCode to individualOrganismObservation.uniqueID, to be consistent with VegX names. (*source*AccessionCode only applies to an aggregate DB that preserves info from its inputs. accessionCode made less sense, because this field is for the datasource's primary key, which it may or may not consider an accession code.)
mappings/VegX-VegCSV.stems.csv: Mapped aggregateOrganismObservation terms
mappings/VegX-VegCSV.stems.csv: Changed base back to baseSaturation to distinguish this pH-related concept from other meanings of base, and to match VegBank
mappings/DwC2-VegBIEN.specimens.csv: Removed no longer applicable comments, which were from the very first NY/SALVIAS->VegX/VegBank mapping and had been preserved by the map spreadsheet transformation scripts. Note that many comments have been left, because they either provide explanatory information or because we never reached a decision on the questions posed (such as many of Brad's "OMIT" comments).
mappings/VegX-VegCSV.stems.csv: Removed no longer applicable comments, which were from the very first NY/SALVIAS->VegX/VegBank mapping and had been preserved by the map spreadsheet transformation scripts
mappings/VegX-VegCSV.stems.csv: Mapped individualOrganismObservation user-defined terms
Regenerated vegbien.ERD exports
schemas/vegbien.ERD.mwb: Added link to VegBIEN schema wiki page
inputs/import.stats.xls: Updated with stats from latest import
README.TXT: After a new import: Added steps to check inputs' error counts and only continue with deleting previous imports, etc. if there were little to no errors. Added step to record the import times.
mappings/VegX-VegCSV.stems.csv: Mapped VegBank and SALVIAS abioticObservation terms
mappings/VegX-VegCSV.stems.csv: Resolved ambiguous terms that appeared twice on the output side
mappings/VegX-VegCSV.stems.csv: Mapped VegX abioticObservation terms
mappings/VegX-VegCSV.stems.csv: Mapped standard DwC terms
mappings/DwC2-VegBIEN.specimens.csv, DwC1-DwC2.specimens.csv: Sources: Replaced DwC with http://rs.tdwg.org/dwc/terms/, because DwC terms can come from many places but the DwC source referred specifically to this web page
mappings/DwC1-DwC2.specimens.csv: Corrected mapping for previousCatalogNumber
mappings/DwC1-DwC2.specimens.csv: Added source of datasources' custom terms
mappings/DwC1-DwC2.specimens.csv: Added source of DwC 1.2 (http://digir.net/schema/conceptual/darwin/2003/1.0/darwin2.xsd), aka DwC Classic, terms
mappings/DwC1-DwC2.specimens.csv: Added source of custom NY staging table terms in nimoy.bien2_staging.nybg_raw
mappings/DwC1-DwC2.specimens.csv: Added source of DwC 1.21 (http://digir.net/schema/conceptual/darwin/manis/1.21/darwin2.xsd) terms
mappings/DwC1-DwC2.specimens.csv: Added source of remappings of DwC terms with /_alt added
mappings/DwC1-DwC2.specimens.csv: Added source of DwC terms with namespace removed
mappings/VegX-VegCSV.stems.csv: Added "computer." before taxonomic terms whose VegX mapping used the "computer" role. (This is useful for datasources that supply separate determinations in the same row, such as SALVIAS.)
mappings/DwC2-VegBIEN.specimens.csv: Added Source column containing "DwC" for every field with a an entry in the Order column, so that the source of the term can be tracked once we start combining DwC and VegCSV
inputs/SALVIAS*/maps/VegX.organisms.csv: Fixed missing join mappings for stemobservation-related fields
mappings/DwC2-VegBIEN.specimens.csv: Repopulated Order values for the few rows that had lost it in the process of copying and pasting mappings
mappings/Makefile: VegX-VegCSV.stems.csv: Clean up when edited using sort_map
Added mappings/VegCSV-VegBIEN.specimens.csv, which is generated from VegX-VegCSV.stems.csv
mappings/for_review: svn:ignore OpenOffice.org lock files
Added mappings/VegX-VegCSV.stems.csv. The initial version is autogenerated by joining the simplified VegBIEN XPaths of related maps.
join: Support discarding multiple outputs if they should be considered ambiguous
input.Makefile: Maps validation: $(missingMappingsCmd): Support non-DwC mappings by matching entire line containing mapping, not just word characters. Remove any XML function so that merging of non-empty join mappings still works properly.
mappings/Makefile: Use new invert
Added invert
mappings/Makefile: for_review/VegBIEN-DwC2.specimens.csv: Include all comments column(s), not just the first
cols: Removed special handling of '+' because list_subset() now handles this col_num value itself, by appending the rest of the columns. Support intermixing int and '+' columns, by using new format.str2int_passthru().
util.py: list_subset(): Made an index of '+' append the rest of the list
format.py: Added str2int_passthru()
cols: Changed value for all columns to '+' so that it wouldn't need to be shell-escaped as '*' was
review: Remove keys except last. This should increase the number of matches between human-readable VegBIEN XPaths of VegX and DwC2.
mappings/DwC2-VegBIEN.specimens.csv: Use :[] instead of [] for all XML functions, so that the XML function args will get removed by review
review: Remove XML functions. This should increase the number of matches between human-readable VegBIEN XPaths of VegX and DwC2.
mappings/Makefile: human-readable maps in for_review: Simplify just the output column so that the input column can be programmatically linked back to the original input names/XPaths
mappings/Makefile: Removed no longer used $(chRoot), $(cpReview)
Removed the human-readable mappings mappings/for_review/VegX-VegBIEN.plots.csv, VegX-VegBIEN.organisms.csv because these are now duplicates of VegX-VegBIEN.stems.csv
review: Support limiting the XPath simplifying to custom columns, rather than always the first two
review: Usage message: Fixed typo
Added mappings/for_review/VegBIEN-DwC2.specimens.csv, generated by inverting for_review/DwC2-VegBIEN.specimens.csv. This will be used to help translate VegX->VegCSV.
mappings: Made VegX-VegBIEN.organisms.csv, VegX-VegBIEN.plots.csv symlinks to VegX-VegBIEN.stems.csv instead of building them in the Makefile by copying VegX-VegBIEN.stems.csv, since these files are now always the same
mappings/VegX-VegBIEN.stems.csv: _if that maps to specimenreplicate via plantobservation or voucher: Refactored to map right-hand side of _eq in the left-hand side mapping, rather than in all then/else mappings. Distinguish this _if statement from others using new name param.
