mappings/VegCore.csv, Veg+-VegCore.csv: Sources: Added sort order comments to URLs so they sort in the order indicated at <https://projects.nceas.ucsb.edu/nceas/projects/bien/wiki/VegCore#Sources>. URL comments are enclosed in (), and the sort order element of a comment is a number right after the ( .
mappings/Makefile: .Veg+-VegCore.csv.last_cleanup: Sort by the source URL instead of the VegCore term
mappings/Makefile: Split .Veg+-VegCore.csv.last_cleanup and .VegX-VegCore.csv.last_cleanup into separate targets so their recipes can be different
mappings/VegCore-VegBIEN.csv: Mapped dcterms:rights
backups/Makefile: Synchronization: Also sync *.md5
import_all: Fixed bug where need to wait for all asynchronous commands started before the main import, not just the first
import_all: Import all Source tables before the herbaria list, so that any custom metadata will override the info in the herbaria list
input.Makefile: Tables discovery: $(dontImport): Don't import the Source table when $import_source env var is set to ""
input.Makefile: SVN: add: Add a Source table to store datasource metadata. This adds a Source table to all herbaria which are listed in .herbaria, and therefore didn't previously need a Source table to indicate their referenceType and sampleType.
Added inputs/VASCAN/Source/
csvs.py: stream_info(): Use the Excel dialect and an empty header if the CSV file is empty
pg_dump_limit: Also remove CREATE DATABASE statements
Added inputs/JBM/Source/
mappings/Veg+-VegCore.csv: Removed type->dcterms:type automapping because this term can have many different meanings
Added inputs/NVS/Source/
Added inputs/IUCN/European_Red_List_Plants/header.csv
Added inputs/CVS/_src/
input.Makefile: SVN: $(svnFilesGlob): Include test.xml.ref instead of all test*.xml* to avoid including test outputs
inputs/*/verify/: Updated svn:ignore
mappings/VegCore-VegBIEN.csv: Mapped verbatimCoordinates
Updated inputs/HIBG/Specimen/new_terms.csv
Added inputs/HIBG/Source/
inputs/HIBG/verify/: Updated svn:ignore
Added inputs/NCU-NCSC/Source/
inputs/NCU-NCSC/verify/: Updated svn:ignore
backups/Makefile: Checksums: %.md5/test: Made it an _always target
backups/Makefile: Checksums: Added %.md5/test to test generated checksums
backups/Makefile: Moved md5-related targets to separate Checksums section
backups/Makefile: %.md5: Removed not applicable comment which had been copied from %.sql
inputs/VegBank/plot_/map.csv: Mapped confidentialitystatus to coordinateUncertaintyInMeters, overriding locationaccuracy when the confidentialitystatus indicates fuzzing
inputs/NVS/*/map.csv: Mapped Taxon Growth Form values to growthform enum
inputs/NVS/import_order.txt: Specified import order
inputs/input.Makefile: Staging tables installation: $(allInstalls): Exclude the Source table, which contains only (metadata) mappings, not data
Regenerated vegbien.ERD exports
Added inputs/NVS/TaxonOccurrence.Understory/
Added inputs/NVS/Coordinates/
inputs/NVS/Plot/map.csv: Mapped Plot
inputs/NVS/: Renamed Organism to AggregateOccurrence because this actually contains aggregated samplings
inputs/NVS/StemObservation/map.csv: Mapped Item ID, Item Obs ID
Added inputs/NVS/StemObservation/
Added inputs/NVS/TaxonOccurrence/
Added inputs/NVS/map.csv (global mappings)
inputs/NVS/Project/map.csv: Remapped Project Abbreviation to projectName instead of Name, because Project Abbreviation is what's used throughout the tables to link to the project
inputs/NVS/Plot/map.csv: Mapped Physiography
inputs/NVS/Plot/map.csv: Mapped Area
inputs/NVS/Plot/map.csv: Altitude: Provided rationale for units determination
Added inputs/NVS/Organism/
Added inputs/NVS/Project/
mappings/VegCore-VegBIEN.csv: Mapped projectStartDate, projectEndDate
mappings/VegCore.csv: Added projectStartDate, projectEndDate
mappings/VegCore.csv: Renamed plotName to locationName because this term also applies to the location of a specimen. This replaces CTFS's definition of locationName as locality.
root Makefile: apt-get: Use --yes to allow unattended installations
schemas/vegbien.sql: analytical_*: Renamed plotName to locationName to match the new VegCore term name
mappings/VegCore.csv: Added subInstitutionCode
schemas/vegbien.sql: analytical_stem_view: locationevent info: Fixed bug where need to use project.sourceaccessioncode instead of locationevent.project_id for the projectID
schemas/vegbien.sql: analytical_stem_view: locationevent info: Fixed bug where need to use the parent locationevent's obsstartdate instead when the subevent does not provide it
schemas/vegbien.sql: analytical_stem_view: locationevent info: Fixed bug where need to use the parent locationevent's project and method instead when the subevent does not provide them, because they are often attached to it instead
schemas/vegbien.sql: analytical_stem_view: geolocation info: Fixed bug where need to use the parent location instead when provided, because lat/long and placenames are attached to it instead of the subplot's location
backups/Makefile: %.md5: Fixed bug where md5sum does not have a -q option like md5
backups/Makefile: %.md5: Fixed bug where need to use md5sum instead of md5 on Linux
schemas/vegbien.sql: analytical_stem_view: Filter out non-current taxondeterminations (occurrences with no taxondetermination are preserved)
schemas/vegbien.sql: Removed no longer needed darwin_core table. Use analytical_stem instead, which is now identical.
schemas/vegbien.sql: sync_analytical_*_to_view(): Creating analytical_* table: Fixed bug where need LIMIT 0 so that it can be used on a full DB, which will have data in the tables used by analytical_stem_view
schemas/vegbien.sql: Merged darwin_core into analytical_stem
schemas/vegbien.sql: darwin_core_view, analytical_stem_view: Updated now that newWorldCountries.isoCode is a text field
README.TXT: Data import: backups: Step to copy backups to jupiter: Added full path to aaronmk/ (/data/dev/aaronmk)
inputs/newWorld/geoscrub.schema.~.changes.sql: Reversed order of adding unique constraints and changing types
inputs/newWorld/geoscrub.schema.~.changes.sql: Changed isoCode type to text. Added unique constraint on isoCode.
backups/Makefile: Added md5s target to generate .md5 files for all backups
inputs/import.stats.xls: Updated import times
backups/Makefile: %.md5: Run with `nice -n +5` to avoid slowing down the UI
backups/: svn:ignore: Added *.md5. Removed no longer applicable *.log.
backups/Makefile: Changed paths to be relative to the Makefile rather than the current directory, so this Makefile can be used in other directories as well (such as jupiter:/aaronmk/VegBIEN.backups/)
backups/Makefile: %.backup: Also create MD5 of backup
backups/Makefile: Added %.md5 target to create checksums of each backup
README.TXT: Data import: backups: Added step to copy backups to jupiter in /aaronmk/VegBIEN.backups/ . The jupiter folder, which has several TB of space available, will replace local backup drives as the location for archived backups.
README.TXT: Data import: Removed additional backup of just the public schema, which is not needed because the public schema is included in the full DB backup. The additional public schema backup increased the total backup size by 60-70%, so this will help conserve limited disk space on vegbiendev as well as on local archives of the backups.
README.TXT: Backups: Full DB: Updated steps to match Data import steps, which add the date to the backup filename when it's created rather than afterwards
README.TXT: Backups: Archived imports: Back up: Added instructions for archiving the last import before backing it up
schemas/vegbien.sql: analytical_*: Removed NOT NULL constraint on dateCollected
schemas/vegbien.sql: source: Added sampletype field to indicate a plot or specimen datasource
schemas/vegbien.sql: sync_analytical_*_to_view(): Added NOT NULL constraints
make_analytical_db: Added step to create darwin_core materialized view
inputs/*/Source/map.csv for non-herbaria: Mapped sampleType
inputs/.herbaria/herbaria/map.csv: Set sampleType to "specimen"
mappings/VegCore-VegBIEN.csv: Mapped sampleType
mappings/VegCore.csv: Added sampleType
schemas/vegbien.sql: Added sampletype enum
root Makefile: $(postgresReload-*): Confirm the operation before continuing, since it involves changing PostgreSQL config files in nontrivial ways. Added instructions for setting kernel.shmmax to at least 4GB minus 1 byte on Linux, to work with the shared_buffers setting in postgresql.conf.
schemas/postgresql.conf: shared_buffers: Documented that it must be less than ~95% of SHMMAX
schemas/vegbien.sql: analytical_stem_view: identifiedBy: Fixed bug where need to use party.fullname instead of name components because the name is now mapped to fullname
schemas/vegbien.sql: analytical_stem_view, darwin_core_view: dateCollected: Use the parent plot event's obsstartdate when the subplot event does not have its own obsstartdate
