inputs/SALVIAS/*/map.csv: Remapped all versions of stem and tree tags to tag, with the second tag superceding the first, to avoid the complex VegCore-VegBIEN mapping logic that attempts to place both tags in VegBIEN in the correct order but does not work for column-based import. inputs/SALVIAS-CSV/Organism/map.csv: stem and tree tags: Made the stem tag supercede the tree tag instead of vice versa, to have as specific of a tag as possible.
inputs/SALVIAS/stems/map.csv: Copied Brad's comments on plotObservations.tag1, tag2 to stem_tag1, stem_tag2
mappings/VegCore-VegBIEN.csv: Removed _rangeStart and _rangeEnd filters from fields which should contain decimal values. These filters should be added on a per-datasource basis instead.
inputs/ARIZ/Specimen/map.csv: Documented that MinimumElevationInMeters, MinimumElevationInMeters contain some verbatim values, including ranges and units
mappings/VegCore-VegBIEN.csv: Removed /_units:[default=m,to=m,to=]/value filter from fields. It should be added on a per-datasource basis instead.
mappings/VegCore-VegBIEN.csv: Removed /_replace:["\bca\.?"=]/value filter from fields. It should be added on a per-datasource basis instead.
mappings/VegCore-VegBIEN.csv: verbatimElevation->elevation_m mapping: Translate units automatically (currently only works in row-based mode). Don't remove any "ca." prefix because this is a datasource-specific filter that does not apply to current datasources with verbatimElevation. Also map verbatimElevation to location.verbatimelevation.
inputs/NCU-NCSC/Specimen/map.csv: Elevation: Removed comment that it includes units, because this is now part of the definition of verbatimElevation
mappings/Veg+.terms.csv: Documented that verbatimElevation must include units
inputs/ARIZ/Specimen/map.csv: Remapped VerbatimElevation to UNUSED
inputs/*/*/map.csv: Remapped all unused terms to special value UNUSED. Remapped all private terms to special value PRIVATE. Remapped all deliberately unmapped terms to special value OMIT.
mappings/Veg+-VegCore.csv: Remapped realLatitude, realLongitude to new special value PRIVATE, which is more specific than OMIT
mappings/Veg+.terms.csv: Added special value PRIVATE
mappings/Veg+.terms.csv: Added special values OMIT, UNUSED
inputs/VegBank/plot_/map.csv: Remapped elevation from verbatimElevation to elevationInMeters, since the values are all decimals. The units come from the data dictionary.
inputs/SALVIAS/plotMetadata/map.csv, inputs/SALVIAS-CSV/Plot/map.csv: Remapped elev_m from verbatimElevation to elevationInMeters, since the values are all decimals. Note that the units of SALVIAS Elev were provided by a comment from Brad (and can also be assumed to be the same as SALVIAS-CSV elev_m).
inputs/NCU-NCSC/Specimen/map.csv: Documented that Elevation includes units
inputs/Madidi/Plot/map.csv: Remapped Minimum altitude from minimumElevationInMeters to verbatimElevation_m, since it is a range, not a minimum. Note that the units are assumed based on the range of values present and the region the data is from (Madidi National Park).
mappings/VegCore-VegBIEN.csv: Also mapped verbatimElevation_m to verbatimelevation
mappings/VegCore-VegBIEN.csv: Also mapped verbatimElevation_m to elevationrange_m
mappings/VegCore-VegBIEN.csv: Mapped verbatimElevation_m
mappings/Veg+.terms.csv: Added verbatimElevation_m
mappings/Veg+-VegCore.csv: Mapped realLatitude, realLongitude to OMIT because private data should not be placed in a public database
mappings/Veg+.terms.csv: Added realLatitude, realLongitude
inputs/VegBank/plot_/map.csv: Documented that elevationrange is unused
inputs/Madidi/Plot/map.csv: Fixed comments on Direction and Orientación/exposicion so each comment refers to the other field that is equivalent
inputs/Madidi/Plot/map.csv: Remapped Altitude from verbatimElevation to elevationInMeters, since the values are all decimals. Note that the units are assumed based on the range of values present and the region the data is from (Madidi National Park).
inputs/CTFS/Plot/map.csv: Remapped Elevation from verbatimElevation to elevationInMeters, since it is a float in the original bci.sql database. Note that the units are assumed based on the range of values present and the country the data is from (Panama).
mappings/VegCore-VegBIEN.csv: Mapped elevationInMeters
mappings/Veg+.terms.csv: Added elevationInMeters
schemas/vegbien.sql: location: Added verbatimelevation
README.TXT: Data import: Added note that `make schemas/reinstall` must be done after running make_analytical_db on a previous import
schemas/vegbien.sql: Added indexes for additional analytical_db_view joins, as described at <https://projects.nceas.ucsb.edu/nceas/issues/494>
schemas/vegbien.sql: Added indexes for the analytical_db_view joins, as described at <https://projects.nceas.ucsb.edu/nceas/issues/494>
README.TXT: Data import: Added note that `make schemas/rotate` must be done after running make_analytical_db
schemas/functions.sql: Renamed _pct_to_frac() to _percent_to_fraction() and _frac_to_pct() to _fraction_to_percent(), for clarity and for consistency with _percent (which is spelled out), as used by SALVIAS (http://salvias.net/Documents/salvias_data_dictionary.html) and elsewhere
review: Don't remove XML functions that are unit conversions
schemas/vegbien.sql: Changed _frac units suffix to _fraction for clarity and for consistency with _percent (which is spelled out), as used by SALVIAS (http://salvias.net/Documents/salvias_data_dictionary.html) and elsewhere
inputs/*/*/map.csv: Remapped intercept_cm to new intercept_cm so that units match
mappings/VegCore-VegBIEN.csv: Mapped intercept_cm
schemas/functions.sql: Added _cm_to_m()
mappings/Veg+.terms.csv: Added intercept_cm
mappings/VegCore-VegBIEN.csv: Changed volumeCanopy to the more accurate intercept_m. volumeCanopy was the closest equivalent VegX term, but did not really fit line-intercept information, nor did it include units.
mappings/Veg+.terms.csv: Added intercept_m
schemas/vegbien.sql: taxonscope: Added comment that it stores the scope of a morphospecies name
inputs/import.stats.xls: Updated import times
README.TXT: Data import: Commit: Shortened import message to fit on one line in the README, to avoid issues when copying and pasting
schemas/functions.sql: Added _ha_to_m2(text), _pct_to_frac(text)
schemas/vegbien.sql: analytical_db_view: Use _m2_to_ha() on location.area_m2 to get plotAreaHa
schemas/functions.sql: Added _m2_to_ha()
mappings/VegCore-VegBIEN.csv, Veg+.terms.csv: Removed imprecise and no longer used plotArea and area. Use plotArea_<units> instead.
inputs/*/*/map.csv: Remapped applicable plotArea fields to plotArea_m2
mappings/VegCore-VegBIEN.csv: Mapped plotArea_m2
mappings/Veg+.terms.csv: Added plotArea_m2
mappings/VegCore-VegBIEN.csv: Renamed plotAreaHa to plotArea_ha for consistency with VegBIEN units suffixing convention, which includes an "_"
inputs/*/*/map.csv: Remapped applicable plotArea fields to plotAreaHa
mappings/Veg+-VegCore.csv: Removed inaccurate SizeOfSite->plotArea mapping, which does not match units
mappings/VegCore-VegBIEN.csv: Mapped plotAreaHa
schemas/functions.sql: Added _ha_to_m2()
mappings/Veg+.terms.csv: Added plotAreaHa
mappings/Veg+.terms.csv: Standardize area using VegX /plots/plot/area instead of Madidi Inventory+description.Area
schemas/vegbien.sql: analytical_db_view: Use _frac_to_pct() on aggregateoccurrence.cover_frac to get pctCover
schemas/functions.sql: Added _pct_to_frac()
mappings/VegCore-VegBIEN.csv: coverPercent: Convert to fraction using _pct_to_frac()
xml_dom.py: replace_with_text(): Support ints and floats
xml_func.py: simplify(): Run xml_dom.prune_empty() on function nodes that don't have an explicit simplifying function. This allows single-arg functions with no arg to be pruned rather than called with no args (causing errors if the single param does not have a default value).
Regenerated vegbien.ERD exports
schemas/vegbien.sql: Added units suffix to additional VegBIEN fields that have units
schemas/vegbien.sql: Added units suffix to all core VegBIEN fields that have units. It is the responsibility of the mappings to ensure that all units are properly translated.
root Makefile: PostgreSQL: postgres-Linux: Added postgresql-postgis apt-get
backups/Makefile: Backups: Full DB: Specify the date suffix of the backup when it's created rather than adding it afterwards. This allows the user to specify a suffix that matches the corresponding public-schema backup.
inputs/*/*/map.csv: Mapped variants of subspecies directly to new subspecies term
mappings/VegCore-VegBIEN.csv: subspecies, infraspecificEpithet: Added _alts for datasources that specify both
input.Makefile: Mapping: $(map2db): Inline $(map) because this is the only place it's used
input.Makefile: Mapping: $(map): Don't require flat files because they don't need to be used directly anymore (staging tables are used instead)
input.Makefile: Mapping: $(map2db): Always use staging tables, because the flat files don't need to be used directly anymore
mappings/Veg+-VegCore.csv: Remapped subspecies, subSpeciesName to new subspecies term
mappings/VegCore-VegBIEN.csv: Mapped subspecies, variety, forma, cultivar
mappings/Veg+.terms.csv: Added subspecies, variety, forma, cultivar
schemas/vegbien.sql: taxon.authority_id: Added descriptive comment that this is the authority which defines the taxon name (as opposed to the author of the taxon name)
schemas/vegbien.sql: taxon: Added author_id for the author of the taxon name. This is distinct from authority_id, which is the authority used to determine which taxon name to apply.
schemas/vegbien.sql: analytical_db_view: Use new denormalized placepath table instead of place, which significantly reduces the number of joins
schemas/vegbien.sql: location: Removed stateprovince, country because these are now in placepath (as well as in place.rank)
schemas/vegbien.sql: analytical_db_view: LEFT JOIN locationcoords and locationplace so that locations will be included even if they don't have one of these two determinations
schemas/vegbien.sql: analytical_db_view: Fixed bug where method was being joined instead of left-joined, causing only rows with a method to be included
schemas/vegbien.sql: locationplace: Added identifier_id, so that different identifiers (e.g. the data provider and GNRS) can provide separate locationplaces even if the standardized name happens to be the same as the original name
mappings/VegBank-VegBIEN.csv: Added place->locationplace renaming
mappings/VegBIEN-VegBank.csv: Reversed the order of the columns so it's a more natural forward renaming, and renamed the file to VegBank-VegBIEN.csv to reflect the new column order
mappings/VegBIEN-VegBank.csv: Fixed order of plantconcept->taxon renaming because the VegBIEN column is on the right
schemas/vegbien.sql: Renamed namedplace to place for simplicity and consistency with placepath and locationplace
schemas/vegbien.sql: taxon: Made authority an fkey to reference instead of a text field
schemas/vegbien.sql: Moved steps to include a taxon name at a rank with no explicit column from taxon's comment to taxonpath's comment, because that is the table the steps apply to
schemas/vegbien.sql: Added placepath (analogous to taxonpath), and point locationplace to it instead of directly to namedplace
schemas/vegbien.sql: Split locationdetermination into locationcoords and locationplace, so that coordinate determinations can be made separately from place determinations
schemas/vegbien.sql: location: Removed authore, authorn because this information is now in locationdetermination as verbatimlongitude, verbatimlatitude
