lib/sh/util.sh: set `shopt -s lastpipe`, to allow setting vars in the last command of a pipeline

bugfix: lib/sh/util.sh: ignore_err_msg(): use `try` properly with prep_try and `"try"`

lib/sh/util.sh: added prep_try and use it in try alias. try: documented how to run it with a wrapper command.

lib/sh/db.sh: pg_dump(): ignore "No matching tables were found" error, using new ignore_err_msg()

lib/sh/util.sh: added ignore_err_msg()

lib/sh/db.sh: pg_schema_exists(): documented that `try` is used to suppress the error exit status

lib/sh/util.sh: stderr_matches(): usage: surrounded command in if statement to indicate what context it would usually be used in

lib/runscripts/file.pg.sql.run, schema.pg.sql.run: support custom options to pg_dump in $@

validation/aggregating/traits/BIEN2_traits/bien3_validations_traits_original_mysql.VegCore.sql: added view names using the steps at wiki.vegpath.org/Aggregating_validations_refactoring#prepend-CREATE-VIEW

validation/aggregating/traits/BIEN2_traits/bien3_validations_traits_original_mysql.VegCore.sql: removed trailing whitespace

schemas/vegbien.sql: public_validations_*: merged into single public_validations schema, with type-specific prefixes for each query, so that datasources can use validations queries from multiple type categories, and so that each datasource doesn't have to indicate which validations output schema it's using

schemas/util.sql: diff_cols(): documented how to run EXPLAIN on the FULL JOIN query

schemas/util.sql: diff_cols(): embed left_query/right_query directly in the main SQL statement, so that the query planner can take the sort order of the queries into account in planning the FULL JOIN (and where possible, use a fast merge join)

schemas/vegbien.sql: regenerated from DB, which adds an autogenerated column alias

schemas/vegbien.sql, validation/aggregating/*/*.sql: renamed public_validation_* to plural public_validations_* since we refer to them as aggregating validation*s*

schemas/util.sql: added eval2col_pair()

web/.phpPgAdmin/.htaccess: support linking to a function

web/.phpPgAdmin/.htaccess: support linking to the functions list

web/.phpPgAdmin/.htaccess: support linking to a view

web/.phpPgAdmin/.htaccess: allow linking directly to the views list of a schema

added backups/vegbien.r12025.backup.md5

web/.phpPgAdmin/.htaccess: refactored to build the URL incrementally, extracting path components 1st->last, instead of handling each # of path components as a separate case (which was more rigid and created unnecessary duplication)

backups/TNRS.backup.md5: updated

bugfix: web/.phpPgAdmin/.htaccess: path RewriteRules: need to go directly to the applicable tab, rather than going via redirect.php, because redirect.php will use the last selected tab, rather than the first tab for that section, leading to unexpected results when a non-default tab was previously selected

schemas/vegbien.sql, validation/aggregating/*/*.sql: renamed public_validation_* to plural public_validations_* since we refer to them as aggregating validation*s*

validation/aggregating/pipeline/aggregating_validations_pipeline.odg: made "diff tables" the same size as the other major labels

validation/aggregating/pipeline/aggregating_validations_pipeline.odg: added CSVs export arrows and import process arrows, as decided in conference call

schemas/vegbien.sql: public_validation_plots: added Brad's notes as comments

schemas/vegbien.sql: public_validation_plots: populated queries from validation/aggregating/plots/bien3_validations_salvias_vegbien.sql

validation/aggregating/plots/bien3_validations_salvias_vegbien.sql: _12_count_of_verbatim_taxa_per_plot_in_each_project: removed typo (in the original queries, not added by the refactoring)

validation/aggregating/plots/bien3_validations_salvias_vegbien.sql: shortened view names to fit in the 63-char limit

schemas/vegbien.sql: public_validation_traits: added Brad's notes as comments

schemas/vegbien.sql: public_validation_traits: populated queries from validation/aggregating/traits/bien3_validations_traits_bien3.sql

validation/aggregating/*/*.sql: use "" to preserve case in column aliases, so that they match the input queries

validation/aggregating/*/*.sql: use current_schema instead of a hardcoded datasource name or psql variable

validation/aggregating/*/*.sql: search_path: added public

validation/aggregating/*/*.sql: use the new type-specific validation schema names

validation/aggregating/*/*.sql: removed type-specific prefix from the view name, which is now part of the validation schema

inputs/import.stats.xls: updated import times

inputs/import.stats.xls: updated import times

schemas/vegbien.sql: analytical_stem.datasource: removed fkey to source.shortname because this prevents individually-reloaded datasources from being published

schemas/vegbien.sql: publish(), rm(): documented runtime (1 min)

schemas/vegbien.sql: added datasource_unpublish()

lib/sql_io.py: automatic handling of input/output column type mismatches: also do this for identifying columns, which first cause an error in a join in sql.distinct_table() rather than in the main insert (and thus were not handled by the existing error handling). previously, the user would have had to manually cast the input column in postprocess.sql. this involves getting handle_MissingCastException() to update join_cols as well as mapping.

lib/sql.py: distinct_table(): don't cache the table creation, because this prevents different distinct_tables from being created for the same input table

lib/sql.py: create_table(), copy_table_struct(): support custom query options, such as cacheable

lib/sql_io.py: put_table(): main loop MissingCastException handler: factored out into nested function so that it can also be used elsewhere

lib/sql.py: parse_exception(): parse "operator does not exist" errors as MissingCastExceptions (these appear when a staging table column is of the wrong type)

bugfix: /README.TXT: Full database import: Check that source contains [# datasources] rows up through XAL: added alternative verification method when this is not the case (some datasources may be near the end depending on import order)

schemas/vegbien.sql: public_validation: split into separate subschemas for plots, specimens, traits

schemas/Makefile: vegbien.sql: support auxiliary schemas besides *_validation

fix: schemas/Makefile: %/publish: removed no longer applicable instructions about running `make schemas/rotate`. you should not publish a schema until you are satisfied that it can replace the previous public schema, so that the previous public schema doesn't need to be saved.

fix: schemas/vegbien.sql: public_validation comment: undid incorrect replacement of "validation" with "public_validation"

bugfix: schemas/Makefile: "public": dont

schemas/Makefile: %/publish: don't ignore errors if schema doesn't exist, because it should always exist if you are publishing it

schemas/Makefile: `%/uninstall: vegbien.sql`: use util.schema_bundle_rm() instead of public.rm(), because when the public schema is incompletely imported, public.rm() will not yet exist

schemas/Makefile: `%/install: vegbien.sql`: support auxiliary schemas besides *_validation

schemas/Makefile: %/uninstall, %/publish: use new public.rm(), publish(), which encapsulate the process of renaming multiple related schemas and also allow additional auxiliary schemas beyond just *_validation

schemas/vegbien.sql: added rm(), which deletes this schema

schemas/vegbien.sql: added publish(), which publishes this schema

schemas/vegbien.sql: added public_schema_publish()

schemas/util.sql: added functions for maintaining schema bundles, groups of schemas with a common prefix

schemas/util.sql: added schema_bundle_get_schemas()

schemas/util.sql: added schema_replace()

schemas/util.sql: added schema_rm(), schema_rename()

schemas/vegbien.sql: datasource_publish(): removed nested transaction, which is not needed because Postgres runs each top-level command in a transaction (including in autocommit mode)

bugfix: schemas/Makefile,vegbien.sql: renamed validation to public_validation to match the suffixed name applied by `make schemas/public/install`

schemas/Makefile: %/install, %/publish for public schema: added usage

/README.TXT: to back up the version history: added back `git svn fetch` so we keep the git export up-to-date, too

/README.TXT: to back up the version history: added runtimes (1.5 h for the initial svnsync)

/README.TXT: to back up the version history: added trailing /s to dirs

bugfix: /README.TXT: to back up the version history: fixed svn_repo/ path

**/LICENSE.TXT: corrected "it recommended" to "it is recommended"

_license/non-open-source/applies_to.txt: deleted items: tip revision needs to refer to the revision at which the problem was discovered, not the revision before that

_license/non-open-source/applies_to.txt: deleted items: clarified that some things deleted after r11705 (when we started copyright scrubbing) might also be non-open-source, because they might have been deleted for reasons other than copyright scrubbing and thus would be missing the "copyright scrub" tag

_license/non-open-source/applies_to.txt: deleted items: clarified that some things deleted after r11705 (when we started copyright scrubbing) might also be non-open-source, because they might have been deleted for reasons other than copyright scrubbing and thus would be missing the "copyright scrub" tag

/LICENSE.TXT: removed separate non-open-source disclaimer about parts of the version history before r11705, because this is now listed in _license/non-open-source/applies_to.txt

_license/non-open-source/applies_to.txt: included deleted items that are non-open-source, so that the list correctly applies to the entire repository (which includes the version history)

**/LICENSE.TXT: disclaimer about things that are not open-source: removed "these are copyright their respective owners", because this is understood

_license/README.TXT: changed "parts" to "components" for clarity

_license/non-open-source/applies_to.txt: UArizona?: added BIEN2/traits_observation_counts.xls

/README.TXT: to back up the version history: use svnsync instead of `git svn fetch`, so that the backup is in a format that can be directly reimported into an svn repo

/LICENSE.TXT: parts of the repository that are not open-source: added ,s to clarify the meaning

/LICENSE.TXT: documented that this license refers specifically to the repository hosted at https://code.nceas.ucsb.edu/code/projects/bien/trunk/, since we also have a publicly-accessible checkout at http://fs.vegpath.org/, which contains numerous non-open-source files (more than are listed in _license/non-open-source/applies_to.txt)

fix: inputs/VegBIEN/Redmine/wiki/.htaccess: redirect to new main page when accessed without trailing /

web/BIEN3: redirected to .wiki since the wiki homepage now contains the list of all BIEN3 components

web/links/index.htm: updated to Firefox bookmarks. Quick links: use bien3.vegpath.org to link to the main BIEN3 page

added web/BIEN3 symlink, with .bien3 case-insensitive alias

_license/*/applies_to.txt: added exclusion for "UCSB modifications"

added _license/GPL2.txt.url

/LICENSE.TXT, _license/VegBank_schema/LICENSE.TXT: moved text of the GPL2 license into separate file _license/GPL2.txt to avoid cluttering up LICENSE.TXT

_license/*/LICENSE.TXT: documented that the first section of the file is from license.eml.txt in each folder

/LICENSE.TXT, _license/VegBank_schema/LICENSE.TXT: moved text of the GPL2 license into separate file _license/GPL2.txt to avoid cluttering up LICENSE.TXT

/LICENSE.TXT: GPL text: include the full range of copyright years

/LICENSE.TXT: added disclaimer that parts of the version history before r11705 (2013-11-21) are NOT open-source. this is the revision of the first copyright scrub.

added issues/ with LICENSE.TXT for the issue tracker (BSD)

wiki/_license/*/: symlink the contents instead of the folders, because the root dir's and wiki's applies_to.txt's will be different

wiki/_license/*/: symlink the contents instead of the folders, because the root dir's and wiki's applies_to.txt's will be different

wiki/LICENSE.TXT: documented what this license refers to (the wiki hosted at https://projects.nceas.ucsb.edu/nceas/projects/bien/wiki/)

wiki/LICENSE.TXT: added message about including _license/ when redistributing, as for /LICENSE.TXT