schemas/util.sql: added relation_exists()

schemas/util.sql: save_drop_views()/restore_views(): store the view path in addition to the definition so it can be used in restoring

bugfix: schemas/util.sql: in_reverse(): can't use `SELECT *` for composite types because this expands them

bugfix: schemas/util.sql: array_reverse(): can't use `SELECT *` for composite types because this expands them

schemas/util.sql: save_drop_views()/restore_views(): use opaque return type so it can be changed without changing the local var type in functions that use this

schemas/util.sql: force_recreate(): recreate_users_cmds: renamed to restore_views_info since this is now whatever is needed by util.restore_views()

schemas/util.sql: added restore_views() and use it in force_recreate()

bugfix: schemas/util.sql: save_drop_views(): views must be dropped in reverse dependency order, but returned in forward dependency order

bugfix: schemas/util.sql: force_recreate(): dependent views need to be re-created in forward dependency order. this is done by reversing the dependencies only for save_drop_views().

bugfix: schemas/util.sql: mk_set_comment(): need to use VIEW instead of TABLE for views

bugfix: schemas/util.sql: show_create_view(): also need to include view comment

fix: schemas/util.sql: mk_set_comment(): added ;

fix: schemas/util.sql: show_set_comment(): don't display a COMMENT ON statement if no comment, because this will be appended to table defs, etc. and would create clutter

bugfix: schemas/util.sql: mk_set_comment(): need to handle NULL comment properly

schemas/util.sql: added show_set_comment(regclass)

schemas/util.sql: set_comment(): use util.mk_set_comment()

schemas/Makefile: added vegbien/install, vegbien/uninstall which also handle TNRS

fix: schemas/Makefile: vegbien.sql: also remake inputs/.TNRS/schema.sql since its contents change along with vegbien.sql

schemas/util.sql: added mk_set_comment()

bugfix: lib/runscripts/util.run: $is_runscript: unexport so don't pass it to invoked scripts

backups/pg_snapshot: documented runtime for initial backup (4 h)

bugfix: schemas/util.sql: force_recreate(): recreate_users_cmds: use create_if_not_exists() rather than eval(), because cmd might manually re-create a deleted dependent view, causing it to already exist

bugfix: schemas/util.sql: save_drop_view(): util.show_create_view(): don't include `OR REPLACE` because that causes nonuniform errors (eg. invalid_table_definition), instead of the standard duplicate_table exception caught by util.create_if_not_exists()

fix: schemas/util.sql: show_create_view(): removed extra trailing ; because pg_get_viewdef() now includes one

schemas/util.sql: show_create_view(): made inclusion of `OR REPLACE` configurable

schemas/util.sql: save_drop_view(): use util.drop_view()

/README.TXT: Maintenance: added steps to back up the vegbiendev databases

bugfix: lib/runscripts/util.run: run_args_cmd(): don't prepend main to args if no args, because for a non-runscript, all args will be passed to main(), leading `main` to be doubled

lib/tnrs.py: use the TNRS dev server (with private URL in tnrs.url) instead of the live server, since that contains datasources that we need

lib/streams.py: added file_get_contents()

lib/tnrs.py: configure the server separately from the base URL

lib/: svn:ignore tnrs.url so the TNRS dev server URL does not become public

bugfix: web/.phpPgAdmin/.htaccess: only append subject if non-empty, to avoid confusing redirect.php

bugfix: web/.phpPgAdmin/.htaccess: custom pages: need to handle in query string because .php filename would be treated as path element

web/.phpPgAdmin/.htaccess: support custom .php pages

planning/meetings/BIEN conference call availability.xlsx: updated

schemas/util.sql: save_drop_views(): return text[] instead of concatenated text so that the queries will be run individually and be easier to debug

schemas/util.sql: added eval(queries text[])

bugfix: schemas/util.sql: force_recreate(): need to drop users in reverse to avoid dependencies that block the drop

schemas/util.sql: added array_reverse()

schemas/util.sql: added in_reverse()

bugfix: schemas/util.sql: force_recreate(): need to use :lower:, not [:lower:]

schemas/util.sql: force_recreate(): debug-print PG_EXCEPTION_DETAIL, users to assist in debugging

fix: schemas/util.sql: debug_print_value(): added trailing newline to visually distinguish the printed value

schemas/util.sql: added debug_print_var()

schemas/util.sql: debug_print_return_value(): use new util.debug_print_value()

schemas/util.sql: added debug_print_value()

fix: schemas/util.sql: util.force_recreate(): also support dropping things besides tables

inputs/.TNRS/schema.sql: MatchedTaxon.Accepted_name_species: mapped to accepted_species_binomial

fix: inputs/.TNRS/schema.sql: COMMENTs: always include newline before and after

bugfix: schemas/public_.sql: plots_20_tnrs_names: tnrs_accepted*: use new accepted_*, not scrubbed_*, as scrubbed_* does not contain only the accepted name (as implied by the tnrs_accepted_* column name)

bugfix: inputs/.TNRS/schema.sql: taxon_scrub, etc.: undid rename of accepted name columns to scrubbed_* (r13435), because these are actually not the same (scrubbed_* is the combination of accepted and no-opinion names). the accepted name columns will now be named accepted_*, following the standard naming scheme.

bugfix: schemas/public_.sql: _plots_20_tnrs_names: morphospeciesSuffix must still be called morphospecies_suffix, not taxon_morphospecies, because that column has not been updated to the taxon_morphospecies yet (fixes r13435)

fix: inputs/.TNRS/schema.sql: taxon_scrub, etc.: scrubbed_*: use columns from MatchedTaxon whenever possible, to as much as possible avoid the need to join to taxon_scrub.scrubbed_unique_taxon_name.*

schemas/Makefile: vegbien.sql: exclude the same set of Source tables excluded by inputs/.TNRS/schema.sql, so that reinstalling TNRS doesn't change the contents of this file

bugfix: inputs/.TNRS/grants.sql: added GRANT statements from schema.sql because these aren't run by `make inputs/.TNRS/reinstall`

lib/tnrs.py: retrieval_request_template: taxonomic_constraint, source_sorting: documented their meaning and why they need to be on/off

schemas/public_.sql: _plots_20_tnrs_names: changed to set of columns requested by Martha

validation/aggregating/plots/bien3_validations_salvias_vegbien.sql: updated to DB

schemas/public_.sql: _plots_20_tnrs_names: added morphospecies suffix in order to also validate it

schemas/public_.sql: _plots_20_tnrs_names: added verbatim name in order to also validate the formation of the concatenated name

schemas/public_.sql: merged _plots_20_tnrs_input_names, _plots_21_tnrs_output_names into _plots_20_tnrs_names so the input and output names can be compared side-by-side

schemas/vegbien.sql: synced to DB, which adds Source.errors table indexes added in the import

/README.TXT: Full database import: for test import, turn off DB backup: added () after after_import to show that it's a function

/README.TXT: Full database import: refer to the shell instead of screen because it's also possible to use a nested shell for this

bugfix: /README.TXT: Full database import: to import just a subset of the datasources: need to set @inputs separately from declaring it, because declare does not support () on Mac

fix: /README.TXT: Full database import: on local machine: need to allow logging output to go to log files

/README.TXT: Full database import: for test import, added step to turn off DB backup

fix: /README.TXT: Full database import: setting $version: moved to inside screen to avoid affecting the outer shell

/README.TXT: Full database import: screen: added alternative for a small import (just use nested shell)

fix: /README.TXT: Full database import: clear any limit set in .profile: moved to inside screen because it must happen within screen to avoid affecting the outer shell

fix: /README.TXT: Full database import: added step to clear any limit set in .profile (applicable to local machine)

bugfix: schemas/Makefile: `%/install: public_.sql`: also need to replace `public` when preceded by ( , as it is in the validations queries

bugfix: /Makefile: install: use schemas/public/*re*install because schemas/public/install would errexit if the public schema already exists

bugfix: inputs/input.Makefile: $(datasrc_schema_exists): need to use $(datasrc), not $(schema), as $schema is only what this var is called in the runscripts

bugfix: inputs/analytical_db/: need dummy table.run file to cause a schema to be created for this datasource

fix: inputs/input.Makefile: $(sortFile): don't print the "add any missing tables to $(sortFile)" message every time the Makefile is run

bugfix: inputs/input.Makefile: install: only run this for datasource dirs

inputs/input.Makefile: install: use ./run's install target for clarity

lib/runscripts/import.run: added install() target

bugfix: inputs/input.Makefile: install: made it idempotent (using new $(datasrc_schema_exists)) so that it could be run by `make install` on an existing system

bugfix: inputs/input.Makefile: $(datasrc_schema_exists): need to use $(shell ...)

inputs/input.Makefile: added $(datasrc_schema_exists)

lib/runscripts/in_datasrc_dir.run: use new local.run

added lib/runscripts/local.run

web/links/index.htm: updated to Firefox bookmarks: iPlant links: fixed broken links

web/links/index.htm: updated to Firefox bookmarks: starscream links: removed favicons so they wouldn't wake the Mac up on network access whenever someone opens the links page

added validation/aggregating/bien3_validation_sequence.xlsx from Brad

validation/aggregating/plots/bien3_validations_salvias_vegbien.sql: _plots_21_tnrs_output_names: removed no longer needed disabling of enable_hashjoin (the lack of an additional join now makes `SET enable_seqscan = off;` sufficient to prevent a slow hash join)

bugfix: schemas/util.sql: explain2notice_msg_if_can(): don't use util.is_explainable() because the list provided by Postgres (http://www.postgresql.org/docs/9.3/static/sql-explain.html#AEN77691) excludes some query types that are in fact EXPLAIN-able

added inputs/VegBank/verify/outputBien.log.url

inputs/input.Makefile: add: verify/: also svn:ignore *.log

added validation/aggregating/plots/VegBank/bien3_validations_salvias_db_original_adjToVegBank.txt from Mike Lee

validation/aggregating/plots/bien3_validations_salvias_vegbien.sql: updated to DB

schemas/public_.sql: _plots_20_tnrs_input_names, _plots_21_tnrs_output_names: added Brad's comments

bugfix: schemas/util.sql: materialize_query(): contents generated from: need to include search_path so query is runnable

planning/meetings/BIEN conference call availability.xlsx: updated

validation/aggregating/plots/bien3_validations_salvias_vegbien.sql: _plots_21_tnrs_output_names: removed no longer needed setting of join_collapse_limit (the column now used does not involve an additional join)

schemas/vegbien.sql: _plots_21_tnrs_output_names: use scrubbed_unique_taxon_name (concatenated output name) because this is more similar to the concatenated name used in _plots_20_tnrs_input_names

bugfix: validation/aggregating/plots/bien3_validations_salvias_vegbien.sql: _plots_21_tnrs_output_names: needs to be run with `SET join_collapse_limit = DEFAULT; SET enable_hashjoin = off;` to produce the right query plan