Added inputs/SALVIAS/_archive/salvias_bien_19nov2009.tar.gz.url, .md5
Added inputs/REMIB/_archive/remib_raw.0.header.specimens.txt
Added inputs/CTFS/_src/ctfs_extract.tab.url, .md5
Added inputs/NVS/_src/NVS_VegX.XSLT, _src/VegX/
Added inputs/NVS/_src/_README.TXT
Added inputs/TurboVeg/_src/DBASEDIC_rkp2011*.xlsx
Added inputs/TurboVeg/_src/_README.TXT
inputs/import.stats.xls: Updated import times
README.TXT: Full database import: Don't run backups/fix_perms with sudo because owners are not rsynced to jupiter, so fixing them first is unnecessary. (It's not necessary to change the file owner to bien at any other time, either, because commands are not run as the bien user. It's in fact better to leave the file owner as yourself, to allow rsync to set the modification times.)
README.TXT: Full database import: Added steps to set $version in all vegbiendev terminals
lib/common.Makefile: rsync: $(rsync): Also exclude .lk* lock files
README.TXT: Full database import: Updated log file sync steps for syncing via jupiter
inputs/BRIT/specimen_flat/test.xml.ref: Updated inserted row count
schemas/py_functions.sql: Added _date(year integer, month integer, day integer)
inputs/BRIT/specimen_flat/postprocess.sql: Added primary key
schemas/functions.sql: Added _dms_to_dd() that takes text arguments and casts them to the appropriate type
inputs/HVAA/Specimen/map.csv: Remapped fieldNotes to UNUSED. This also fixes a bug where fieldNotes was colliding with other occurrenceRemarks inputs because it was missing a filter.
schemas/vegbien.sql: analytical_specimen/analytical_plot: Added occurrenceRemarks
inputs/REMIB/Specimen/map.csv: Mapping NULL-equivalent lat/long to NULL: Fixed bug where need special *=* mapping to pass through values not in the map
inputs/REMIB/Specimen/map.csv: Map NULL-equivalent text fields (country, state, etc.) to NULL
inputs/REMIB/Specimen/map.csv: Map NULL-equivalent lat/long to NULL
inputs/SpeciesLink/Specimen/map.csv: dwc_dwcore_Remarks: Remapped to occurrenceRemarks (it had been incorrectly automapped to taxonRemarks). Combined occurrenceRemarks inputs using /_alt.
mappings/VegCore.htm: Regenerated from wiki. Remapped organismNotes to be a synonym of occurrenceRemarks, since notes on an organism are more generally notes on an occurrence.
inputs/HVAA/Specimen/map.csv: Remapped dynamicProperties to occurrenceRemarks because it contains occurrenceRemarks data
schemas/vegbien.sql: analytical_stem_view: Added occurrenceRemarks
mappings/VegCore-VegBIEN.csv: Mapped occurrenceRemarks
mappings/VegCore-VegBIEN.csv: Only use one of collection and collectionURL instead of concatenating them when both are provided (as requested by Brad <https://projects.nceas.ucsb.edu/nceas/projects/bien/wiki/Spot-checking#ACAD>)
mappings/VegCore-VegBIEN.csv: _avg(): Use numeric param names to work with SQL functions
inputs/FIA/_src/_README.TXT: Added e-mail from Brad Boyle on 2013-2-8
inputs/FIA/_src/: Added attachments from Brad's e-mail on 2013-2-8
xml_func.py: Removed no longer used _range(). Use SQL functions._range() instead.
schemas/functions.sql: Added _range()
xml_func.py: Removed no longer used _avg(). Use SQL functions._avg() instead.
schemas/functions.sql: Added _avg()
schemas/functions.sql: _dms_to_dd(text): Corrected concatenated form to interpret last 5 digits as MMmmm = MM.mmm instead of MMSSS, per Tom Wendt's explanation that this form is "degrees/minutes/decimalminutes"
inputs/MO/Specimen/postprocess.sql: Use :table instead of hardcoding "MO"."Specimen"
inputs/MO/Specimen/postprocess.sql: Also remove frameshifted rows where InstitutionCode is NULL
inputs/UNCC/Specimen/map.csv: herbarium->specimenIndexer: Replace UNCCD (UNCC database?) with UNCC
schemas/functions.sql: functions._dms_to_dd(): Added support for DMS values without separators
README.TXT: Datasource setup: Update vegbiendev: Removed command to run the tests on vegbiendev because vegbiendev no longer has an empty public schema for running tests on (the public schema instead contains the latest published import)
Mapped inputs/BRIT/
my2pg: Remove (0) after char
my2pg: Remove any (#) after bytea
my2pg: Translate binary to bytea
README.TXT: Datasource setup: MySQL inputs: Editing _MySQL/*.make: Set $server to vegbiendev instead so the export command will work from any machine (though it will be faster from vegbiendev itself)
inputs/BRIT/_MySQL/MySQL.*.sql.make: Fixed bug where needs to be set to just localhost and --user to bien instead
README.TXT: Datasource setup: MySQL inputs: Editing _MySQL/*.make: Fixed bug where $server needs to be set to just localhost and --user to bien instead.
lib/MySQL.*.sql.make: Added --user option to demonstrate how to set the DB user vs. the ssh user
inputs/BRIT/_MySQL/MySQL.*.sql.make: Fixed bug where $server needs to be set to bien@localhost
README.TXT: Datasource setup: MySQL inputs: Editing _MySQL/*.make: Specified that $server needs to be set to bien@localhost. Install the staging tables: Specified that for a MySQL .sql export, this must be done on vegbiendev.
inputs/BRIT/: Added _MySQL/MySQL.*.sql.make
README.TXT: Datasource setup: MySQL inputs: .sql exports: Run mysql_bien in the background
README.TXT: Datasource setup: MySQL inputs: .sql exports: Added steps to sync the export to vegbiendev. Specified that the database for the MySQL export should be created on vegbiendev.
README.TXT: Datasource setup: Update vegbiendev: Updated steps for syncing via jupiter
Added inputs/BRIT/ (not yet translated to PostgreSQL)
mappings/VegCore-VegBIEN.csv: Mapped latitude/longitude_DMS to coordinates.latitude_deg using new _dms_to_dd(text)
schemas/functions.sql: Added _dms_to_dd(text)
schemas/functions.sql: _dms_to_dd(): Fixed bug where need to use 1 as the multiplier when dir is NULL
schemas/functions.sql: _dms_to_dd(): Fixed bug where need to apply the sign multiplier after the components are added, because it applies to the DMS as a whole rather than just to the degree component
README.TXT: Full database import: Before running `make inputs/upload`, added step to do ./fix_perms on local machine so that files are not uploaded to jupiter with incorrect permissions
README.TXT: Full database import: Before running `make inputs/upload`, added step to do svn up on jupiter so that unversioned files are not uploaded before their versioned equivalents are checked out from svn (requiring each file to be manually svn-added before the svn up can proceed)
schemas/functions.sql: _dms_to_dd(): Added dir param to specify compass dir suffix, in order to calculate the sign correctly
Moved compass_dir from schemas/vegbien.sql to schemas/functions.sql so it can be used by _dms_to_dd()
Moved _dms_to_dd() from schemas/vegbien.sql to schemas/functions.sql so it can use functions schema functions
schemas/vegbien.sql: Added compass_dir enum
inputs/TEX/Specimen/map.csv: Remapped LAT/LONG_DEGREES,MINUTES,SECONDS to new latitude/longitude_deg,min,sec, allowing the DMS coordinates to be translated
inputs/REMIB/Specimen/map.csv: Remapped lat/long_deg,min,sec to new latitude/longitude_deg,min,sec, allowing the DMS coordinates to be translated
mappings/VegCore-VegBIEN.csv: latitude/longitude_deg,min,sec: Also mapped to the geoscrub coordinates entry
mappings/VegCore-VegBIEN.csv: latitude/longitude_sec: Fixed name, which had been incorrectly automapped to verbatim*
mappings/VegCore.htm: Regenerated from wiki. latitude/longitude: Moved _deg,min,sec components after unit alternatives so that unit alternatives following them wouldn't incorrectly be interpreted as alternatives for the *components instead of for the overall, unitless term.
mappings/VegCore-VegBIEN.csv: Mapped latitude/longitude_deg,min,sec
mappings/VegCore.htm: Regenerated from wiki. Merged coordinateUncertainty and coordinatePrecision, since datasources often use these interchangeably (the only difference is the DwC-recommended units).
mappings/VegCore.htm: Regenerated from wiki. Source: Added authors. Renamed creator to firstPublisher. Moved dataOwner to Source as list. Coordinates: latitude/longitude, coordinatePrecision: Added units disambiguation. latitude/longitude_DMS: Added components. Global IDs: Changed "globally unique" tag to "global ID" and hyperlinked it to <https://projects.nceas.ucsb.edu/nceas/projects/bien/wiki/Global_IDs> (which now includes detailed examples of the various types of global IDs).
inputs/SpeciesLink/Specimen/map.csv: conceptual_darwin_2003_1_0_CoordinatePrecision: Remapped to coordinateUncertaintyInMeters, assuming units based on the range and precision of values
inputs/HVAA/Specimen/map.csv: Remapped coordinatePrecision to UNUSED
inputs/ARIZ/omoccurrences/map.csv: Remapped coordinatePrecision to UNUSED
schemas/vegbien.sql: Added _dms_to_dd()
inputs/bien2_traits/TraitObservation/map.csv: Remapped Source to sourceType
Added inputs/TEX/
units.py: convert(): Support ranges
Moved parse_range() from xml_func.py to units.py
units.py: conversions: Added ft->m conversion
src_map: Reopen stdin (/dev/stdin) in universal newlines mode to support files with \r as the line ending
csv2db: Open input stream in universal newlines mode to support files with \r as the line ending
mappings/VegCore.htm: Regenerated from wiki. Moved Forming IDs section to separate Global IDs wiki page (https://projects.nceas.ucsb.edu/nceas/projects/bien/wiki/Global_IDs).
input.Makefile: $(exts): Added .tab
mappings/VegCore.htm: Regenerated from wiki. Moved preferred ID order to Forming IDs section as examples list. Made examples globally unique by prefixing the outer ID that scopes the value.
mappings/VegCore.htm: Regenerated from wiki. Forming IDs: Added instructions for making a local ID global. Added note that these globally unique IDs are not the same as UUIDs. Forming preferred IDs: Reworded as a set of instructions rather than a definition. Clarified that this becomes the database primary key.
mappings/VegCore.htm: Regenerated from wiki. Added section on forming preferred IDs, which are the ID which is most directly associated with the identified object.
mappings/VegCore.htm: Regenerated from wiki. Documentation has been added on how to choose term names (https://projects.nceas.ucsb.edu/nceas/projects/bien/wiki/VegCore#Naming) and how to form globally unique ID values (https://projects.nceas.ucsb.edu/nceas/projects/bien/wiki/VegCore#Forming-IDs). Source and Specimen terms have been renamed to be self-explanatory and unambiguous (the DwC equivalents remain as synonyms). Short definitions of Source terms have been added to explain the differences between them. Source, Specimen, and Collection terms have been shortened according to the new instructions for choosing preferred term names (https://projects.nceas.ucsb.edu/nceas/projects/bien/wiki/VegCore#Naming).
inputs/NY/Ecatalog_all/test.xml.ref: Updated inserted row count
schemas/vegbien.sql: analytical_stem_view: scientificName_verbatim: Use the taxonname (which omits the family and author) instead when available, at Brad's request
schemas/vegbien.sql: Disambiguated taxonlabel.taxonomicname, taxonverbatim.taxonomicname, and taxonverbatim.taxonname
schemas/vegbien.sql: taxon_trait_view: Use taxonverbatim.taxonname (the binomial) in preference to taxonverbatim.taxonomicname (the binomial+author) or taxonlabel.taxonomicname (the family+binomial+author), at Mark's request
to_do/timeline.2013.xls: Obtain and load any additional new data: Extended beyond April because it's ongoing
README.TXT: Full database import: On nimoy: Moved analytical_aggregate table creation right after analytical_aggregate..csv download because it can happen while the file is downloaded
to_do/timeline.2013.xls: Reload existing data in need of refresh: Added additional cell because this is still in progress
