VegCSV-DwC merging

Check for inconsistent mappings

  1. Add DwC mappings to VegCSV-VegBIEN.specimens.csv:
    bin/in_place mappings/VegCSV-VegBIEN.specimens.csv env all=1 bin/union mappings/DwC2-VegBIEN.specimens.csv;\
    bin/in_place mappings/VegCSV-VegBIEN.specimens.csv bin/sort 0 1;\
    bin/in_place mappings/VegCSV-VegBIEN.specimens.csv uniq
  2. Check VegCSV-VegBIEN.specimens.csv for VegCSV input terms that appear multiple times in a row
    If some of the output mappings are incompatible:
    1. svn revert mappings/VegCSV-VegBIEN.specimens.csv
    2. Edit the applicable mappings in mappings/VegCSV-VegBIEN.specimens.csv and mappings/DwC2-VegBIEN.specimens.csv
    3. make test
    4. Commit
    5. Continue with step 1, checking that the inconsistent multiple mappings have been merged into one mapping

Check for colliding mappings

  1. Check VegCSV-VegBIEN.specimens.csv for VegBIEN output XPaths that appear multiple times in a row
    Add an /_alt/# suffix to each output XPath to specify which mapping should take priority when both input terms are present

Add VegCSV mappings to mappings/DwC2-VegBIEN.specimens.csv

  1. cp -p mappings/VegCSV-VegBIEN.specimens.csv mappings/DwC2-VegBIEN.specimens.csv
  2. Open mappings/DwC2-VegBIEN.specimens.csv and change VegCSV[DwC] to DwC
  3. make test
  4. Check that tests pass
  5. If not:
    1. Add VegCSV mappings to DwC2
  6. make test
  7. Accept tests
  8. Commit
  9. Continue with step 1

Replace occurrences of DwC and VegCSV with Veg+ (or sometimes VegCore)

  1. Rename the maps using svn mv: (order matters)
    # DwC2-VegBIEN.specimens.csv
    svn mv mappings/DwC2-VegBIEN.specimens.csv mappings/VegCore-VegBIEN.specimens.csv
    svn mv mappings/.DwC2-VegBIEN.specimens.csv.last_cleanup mappings/.VegCore-VegBIEN.specimens.csv.last_cleanup
    svn mv mappings/for_review/DwC2-VegBIEN.specimens.csv mappings/for_review/VegCore-VegBIEN.specimens.csv
    # DwC1-DwC2.specimens.csv
    svn mv mappings/DwC1-DwC2.specimens.csv mappings/Veg+-VegCore.specimens.csv
    svn mv mappings/.DwC1-DwC2.specimens.csv.last_cleanup mappings/.Veg+-VegCore.specimens.csv.last_cleanup
    svn mv mappings/for_review/DwC1-DwC2.specimens.csv mappings/for_review/Veg+-VegCore.specimens.csv
    # DwC-VegBIEN.specimens.csv
    svn mv mappings/DwC.cs-VegBIEN.specimens.csv mappings/Veg+.cs-VegBIEN.specimens.csv
    svn mv mappings/DwC-VegBIEN.specimens.csv mappings/Veg+-VegBIEN.specimens.csv
    # VegX-VegCSV.stems.csv
    svn mv mappings/VegX-VegCSV.stems.csv mappings/VegX-VegCore.stems.csv
    svn mv mappings/.VegX-VegCSV.stems.csv.last_cleanup mappings/.VegX-VegCore.stems.csv.last_cleanup
    svn mv mappings/for_review/VegX-VegCSV.stems.csv mappings/for_review/VegX-VegCore.stems.csv
    # VegCSV-VegBIEN.specimens.csv
    svn rm mappings/VegCSV-VegBIEN.specimens.csv
    svn rm mappings/.VegCSV-VegBIEN.specimens.csv.last_cleanup
    svn rm mappings/for_review/VegCSV-VegBIEN.specimens.csv
    # VegCSV-VegBIEN.*.csv
    for map in mappings/VegCSV-VegBIEN.*.csv; do rm "$map"; ln -s Veg+-VegBIEN.specimens.csv "$map"; done
    for map in mappings/VegCSV-VegBIEN.{plots,organisms,stems}.csv; do svn mv "$map" "${map/VegCSV/Veg+}"; done
    # *.self.*.csv
    svn mv mappings/DwC.self.specimens.csv mappings/Veg+.self.specimens.csv
    for map in mappings/VegCSV.self.{plots,organisms,stems}.csv; do svn rm "$map"; svn cp mappings/Veg+.self.specimens.csv "${map/VegCSV/Veg+}"; done
    # inputs/*/maps/{DwC,VegCSV}.*.csv
    for map in inputs/*/maps/{,.}DwC.*.csv{,.last_cleanup}; do svn mv "$map" "${map/DwC/Veg+}"; done
    for map in inputs/*/maps/{,.}VegCSV.*.csv{,.last_cleanup}; do svn mv "$map" "${map/VegCSV/Veg+}"; done
  2. Remove .last_cleanup files: find . -name "*.last_cleanup" -delete
  3. Change all map roots to Veg+, including in mappings/roots
  4. Change mappings/Makefile with the new map names, and add + to $(filenameVia)
  5. make mappings/
  6. svn add mappings/VegCore.self.specimens.csv
    • There was never a corresponding mappings/DwC2.self.specimens.csv because it was manually prevented from being created by a regexp filter.
      There is no harm in having one, and the filter should eventually be removed.