schemas/Makefile: `%/uninstall: vegbien.sql`: added support for *_validation schema

schemas/Makefile: %/uninstall: split into 2 rules like for %/install, so that different actions can easily be added for the public schema

schemas/Makefile: %/publish: support auxiliary *_validation schemas

bugfix: schemas/Makefile: `%/install: vegbien.sql`: need -e before each s/// expression

schemas/Makefile: `%/install: vegbien.sql`: create a custom _validation schema for *each public schema, rather than one for the most recently-created public schema. this allows validations to continue to be run against a previous version of the DB while a new version is being imported.

validation/aggregating/traits/bien3_validations_traits_bien3.sql: include only records from the bien2_traits datasource, to ensure that the scope of all queries is limited to the datasource

validation/aggregating/*/*.sql (the output queries): added `SET search_path TO validation`

validation/aggregating/specimens/qualitative_validations_specimens.sql: added CREATE VIEW prefixes using the steps at wiki.vegpath.org/Aggregating_validations_refactoring#prepend-CREATE-VIEW

validation/aggregating/traits/bien3_validations_traits_bien3.sql: added CREATE VIEW prefixes using the steps at wiki.vegpath.org/Aggregating_validations_refactoring#prepend-CREATE-VIEW

validation/aggregating/plots/bien3_validations_salvias_vegbien.sql: added CREATE VIEW prefixes using the steps at wiki.vegpath.org/Aggregating_validations_refactoring#prepend-CREATE-VIEW

validation/aggregating/plots/bien3_validations_salvias_vegbien.sql: added CREATE VIEW prefixes using the steps at wiki.vegpath.org/Aggregating_validations_refactoring#prepend-CREATE-VIEW

added validation/aggregating/specimens/qualitative_validations_specimens.sql template, with query descriptions from NY/qualitative_validations_source_db_NYBG.sql

validation/**: removed old versions of by=-tagged files. these were not renamed in the last commit because there were newer versions that they collided with. note that because , sorts before . , the newer version (with more ",name@inst" tags) was correctly renamed instead of the older version.

validation/**: removed by=... indicators so that we don't have to add an author name every time a file is changed. the author indicator is not needed now that both Brad's and my edits are open-source.

inputs/bien2_traits/TraitObservation/postprocess.sql: remove rows with no taxon name, which are invalid, and which helps simplify the aggregating validations queries

added validation/aggregating/pipeline/aggregating_validations_pipeline.odg with .png export

added validation/aggregating/plots/FIA/bien3_validations_fia_input.sql.eml.txt. note that a .url file is not needed because this information is in the .eml.txt file.

fix: validation/aggregating/traits/_archive/bien3_validations_traits_bien3.by=Brad@iPlant.sql: renamed to indicate that this is actually not Brad's original file, as the filename would seem to indicate

added validation/aggregating/plots/FIA/

validation/aggregating/*.by=Brad@iPlant*: placed under version control, since these are now open-source

validation/**: removed svn:ignore exclusion of iPlant-owned files, since these are now open-source

fix: inputs/VegBIEN/Redmine/svn/.htaccess: updated repository URL to point to trunk/

added wiki/LICENSE.TXT,_license/, with license terms that apply specifically to the wiki (which is BSD, not GPL). note that the wiki attachments are NOT open-source, only the page text.

schemas/vegbien.ERD.mwb: regenerated exports

_license/VegBank/: renamed to ESA/ following the convention that each _license/ subfolder is a copyright owner

/LICENSE.TXT: added collective license conditions (GPL2+) for the svn repository as a whole, except the non-open-source portions

added _license/VegBank/

added LICENSE.TXT and related files (in _license/). the open-source portions of the svn repository are licensed as described in _license/. note that some portions of the svn repository are NOT open-source. these are copyright their respective owners.

schemas/vegbien.ERD.mwb: synced to schemas/vegbien.my.sql, which adds the validation schema

schemas/util.sql: added diff_views()

schemas/util.sql: added diff_cols(), for use with the aggregating validations

bugfix: schemas/Makefile: public/uninstall: need to uninstall the validation schema in addition to the public schema (deleting the public schema would only cascade to the contents of the validation schema)

schemas/vegbien.sql: added validation schema, which contains queries used in the aggregating validations. schemas/Makefile: include schemas that depend on `public` (eg. `validation`) so they are restored along with it.

bugfix: bin/pg_dump_vegbien: fixed arg-count check to allow passing command-line options to pg_dump via args

bugfix: inputs/SALVIAS/verify/plots.out.sql: fixed ' quoting syntax to use '' instead of \' to escape '

inputs/input.Makefile: verify/%.out: use a *.sql file in the verify/ directory itself to generate *.out, so that each datasource can have its own set of output queries. for datasources that should share the same set of queries, they can instead be symlinked to the same file.

fix: inputs/CVS/project/: added _no_import since this should not also be imported separately from taxon_observation.**

fix: validation/aggregating/traits/_archive/bien3_validations_traits_bien3.by=Brad@iPlant,Aaron@UCSB,Brad@iPlant.sql: added missing .url extension

added validation/aggregating/traits/_archive/bien3_validations_traits_bien3.by=Brad@iPlant,Aaron@UCSB,Brad@iPlant.sql

validation/aggregating/traits/BIEN2_traits/_archive/bien3_validations_traits_original_mysql.by=Brad@iPlant,Aaron@UCSB,Brad@iPlant.sql.url: renamed to indicate that it's just by Brad (it was the other file, the output queries, that were created by both of us)

added validation/aggregating/traits/BIEN2_traits/_archive/bien3_validations_traits_original_mysql.by=Brad@iPlant,Aaron@UCSB,Brad@iPlant.sql.url

validation/aggregating/plots/_archive/bien3_validations_salvias_vegbien.by=Brad@iPlant.sql.url: removed trailing @

deleted validation/aggregating/traits/_archive/bien3_validations_traits_bien3.by=Brad@iPlant.sql.url@, since there is already a version without the trailing @

validation/aggregating/traits/BIEN2_traits/_archive/bien3_validations_traits_original.by=Brad@iPlant.sql.url: removed trailing (apparently, the escape for paths containing @ should only be used on the input path)

added inputs/XAL/Specimen/_no_import, since this is a demo-only datasource and there isn't a staging table for it

inputs/.geoscrub/county_centroids/test.xml.ref, inputs/.NCBI/{names.src,nodes.src}/test.xml.ref: accepted test outputs (generated now that these tables are in import_order.txt)

inputs/FIA/taxon_observation.**/header.csv: updated for new REF_RESEARCH_STATION.country metadata value col

/README.TXT: Full database import: `make test by_col=1`: documented runtime (20 min)

/README.TXT: Full database import: `. bin/import_all`: documented how to view progress

fix: schemas/Makefile: %/uninstall: don't prompt to delete auxiliary schemas, as these do not contain data

schemas/vegbien.sql: added top_plot view

/README.TXT: to back up the version history: added steps to sync the git repository to jupiter and the local machine

/README.TXT: added steps to back up the version history

validation/aggregating/plots/bien3_validations_salvias_vegbien.by=Brad@iPlant.sql.url: moved to _archive/ so only the most recent version would be in the main plots/ directory

planning/resources/dev_machine.specs/sys_info.spx+/system_power_settings.txt: updated: enabled "Automatic Restart on Power Loss" and "Wake on LAN" for UPS (this requires using pmset, as described at http://vegpath.org/links/#pmset%281%29%20Mac%20OS%20X%20Manual%20Page )

web/links/index.htm: updated to Firefox bookmarks: wake for network access: added link to pmset man page, which contains commands for setting "wake for network access" for all power sources. note that "automatic restart on power loss" also needs to be set for all power sources, because the computer will always be in UPS mode at the time of a power failure.

inputs/input.Makefile: add!: verify/: also svn:ignore *.tsv, *.txt

validation/aggregating/traits/bien3_validations_traits_bien3.by=Brad@iPlant.sql*: moved to _archive/

validation/aggregating/traits/BIEN2_traits/bien3_validations_traits_original.by=Brad@iPlant.sql*: moved to _archive/

schemas/vegbien.sql: location_set_top_plot(): only do the lookup of the parent's top_plot when there is a parent_id

schemas/vegbien.sql: location: added autopopulated top_plot

schemas/vegbien.sql: locationevent: added locationevent_pull_forward_from_parent() trigger which populates project_id from the parent locationevent

web/links/index.htm: updated to Firefox bookmarks: NCEAS: added link to phone list, which contains the Balboa Building maintenance numbers

/README.TXT: Notes on running programs: added warning that you should always start with a clean shell to avoid spurious bugs

schemas/vegbien.sql: locationevent__pull_forward_from_parent(): renamed to locationevent_pull_forward_from_parent_for_stratum() since it actually has a filter that causes it to apply only to stratum events

added validation/aggregating/, containing folders for Brad's scripts (the scripts themselves cannot be in the repository because they are not UCSB-owned)

web/links/index.htm: updated to Firefox bookmarks: svn: added steps to add a file containing (append to the path)

web/links/index.htm: updated to Firefox bookmarks: `mysql`: added commands to get interactive formatting (including echoed queries) when inputting from a file, and to get raw formatting when inputting from the console. git: added equivalent to `svn update`.

bugfix: schemas/vegbien.sql: commconcept: added missing unique constraint on commdescription, which is also an identifying field in addition to the ID (which is in commname.commname)

inputs/publishable datasources.xlsx: updated

inputs/publishable datasources.xlsx: updated

fix: inputs/SALVIAS/projects/postprocess.sql: remove private data that should not be publicly visible: remove projects that do not have "There are no specific use conditions attached to this dataset"

fix: inputs/SALVIAS/salvias_plots.~.clean_up.sql: Remove private data that should not be publicly visible: also need to remove metadata-only plots

bugfix: inputs/SALVIAS/plotMetadata_/map.csv: things mapped to project_participant: remapped to event__participant because these actually relate to the event, not the project, even though they seem like project-related fields

mappings/VegCore-VegBIEN.csv: event__participant: allow multiple event__participant columns

fix: inputs/SALVIAS/plotMetadata_/map.csv, inputs/Madidi/LocationObservation/map.csv: things mapped to communityID: remapped to communityName, which is what's used in analytical_stem (communityID is for numeric IDs)

bugfix: schemas/vegbien.sql: locationevent__communities(): inner-join to commdetermination instead of left-joining, so that this does not produce community entries (which occur because there is one commclass per locationevent, but only some commclasses will then have a commdetermination)

inputs/SALVIAS/plotMetadata_/create.sql, map.csv: expanded plot_administrator:party_code_party_ and mapped plot_administrator_name to a 2nd project_participant

mappings/VegCore-VegBIEN.csv: project_participant: use [!...] negative lookahead assertion so that multiple project_participant columns will properly map to separate projectcontributor rows

fix: lib/Parser.py: syntax_err(): error message: lightened the shade of gray so the boundary between the strings before and after the error would be clearer

inputs/SALVIAS/plotMetadata_/map.csv: mapped PrimOwnerID_name->project_participant

inputs/SALVIAS/plotMetadata_/create.sql: added join to PrimOwnerID:party_code_party_

bugfix: inputs/SALVIAS/import_order.txt: added party_code_party_

bugfix: inputs/SALVIAS/party_code_party_/create.sql: need to remove duplicate entries in party_code_party

schemas/vegbien.sql: projectcontributor: added missing unique constraint

web/links/index.htm: updated to Firefox bookmarks: OS X 10.9 Mavericks upgrade: added links for whether to upgrade (conclusion: don't, it breaks existing programs)

inputs/SALVIAS/party_code_party_/map.csv: mapped fullname->event_participant_name for use by other tables

mapped inputs/SALVIAS/party_code_party_/

planning/resources/dev_machine.specs/sys_info.spx+/system_power_settings.txt: updated: enabled PrioritizeNetworkReachabilityOverSleep

/README.TXT: Testing: added pointer to development machine specs

added planning/resources/dev_machine.specs/

inputs/SALVIAS/_MySQL/salvias_plots.*.sql: refreshed. this adds the party and party_code_party tables Brad provided for mapping the plot contributors.

fix: inputs/SALVIAS/salvias_plots.~.clean_up.sql: Delete rows that do not satisfy foreign key constraints: also need to do this for plotObservations, since the refreshed data contains dangling rows for that as well

inputs/SALVIAS/run_: documented *.sql install runtime (3 min), as separate from the full `datasrc_make reinstall` runtime (3.5 min)

inputs/SALVIAS/run_: refresh(): `datasrc_make reinstall`: updated runtime. documented that runtimes are from starscream.

added inputs/SALVIAS/run_, which includes a refresh() target

lib/runscripts/in_datasrc_dir.run: datasrc_make(): use `if remaking ...` instead of accessing $_remake manually, for clarity

bugfix: lib/runscripts/in_datasrc_dir.run: datasrc_make(): use set_make_vars and $_remake as required in lib/sh/make.sh `remaking`

lib/sh/make.sh: remaking alias: documented that you MUST use set_make_vars at the beginning of any function that uses this, so that $_remake is properly set to $remake and not left at its previous value

web/links/index.htm: updated to Firefox bookmarks. git: added instructions to migrate from svn.