lib/sql.py: distinct_table(): don't cache the table creation, because this prevents different distinct_tables from being created for the same input table
lib/sql.py: create_table(), copy_table_struct(): support custom query options, such as cacheable
lib/sql_io.py: put_table(): main loop MissingCastException handler: factored out into nested function so that it can also be used elsewhere
lib/sql.py: parse_exception(): parse "operator does not exist" errors as MissingCastExceptions (these appear when a staging table column is of the wrong type)
bugfix: /README.TXT: Full database import: Check that source contains [# datasources] rows up through XAL: added alternative verification method when this is not the case (some datasources may be near the end depending on import order)
schemas/vegbien.sql: public_validation: split into separate subschemas for plots, specimens, traits
schemas/Makefile: vegbien.sql: support auxiliary schemas besides *_validation
fix: schemas/Makefile: %/publish: removed no longer applicable instructions about running `make schemas/rotate`. you should not publish a schema until you are satisfied that it can replace the previous public schema, so that the previous public schema doesn't need to be saved.
fix: schemas/vegbien.sql: public_validation comment: undid incorrect replacement of "validation" with "public_validation"
bugfix: schemas/Makefile: "public": dont
schemas/Makefile: %/publish: don't ignore errors if schema doesn't exist, because it should always exist if you are publishing it
schemas/Makefile: `%/uninstall: vegbien.sql`: use util.schema_bundle_rm() instead of public.rm(), because when the public schema is incompletely imported, public.rm() will not yet exist
schemas/Makefile: `%/install: vegbien.sql`: support auxiliary schemas besides *_validation
schemas/Makefile: %/uninstall, %/publish: use new public.rm(), publish(), which encapsulate the process of renaming multiple related schemas and also allow additional auxiliary schemas beyond just *_validation
schemas/vegbien.sql: added rm(), which deletes this schema
schemas/vegbien.sql: added publish(), which publishes this schema
schemas/vegbien.sql: added public_schema_publish()
schemas/util.sql: added functions for maintaining schema bundles, groups of schemas with a common prefix
schemas/util.sql: added schema_bundle_get_schemas()
schemas/util.sql: added schema_replace()
schemas/util.sql: added schema_rm(), schema_rename()
schemas/vegbien.sql: datasource_publish(): removed nested transaction, which is not needed because Postgres runs each top-level command in a transaction (including in autocommit mode)
bugfix: schemas/Makefile,vegbien.sql: renamed validation to public_validation to match the suffixed name applied by `make schemas/public/install`
schemas/Makefile: %/install, %/publish for public schema: added usage
/README.TXT: to back up the version history: added back `git svn fetch` so we keep the git export up-to-date, too
/README.TXT: to back up the version history: added runtimes (1.5 h for the initial svnsync)
/README.TXT: to back up the version history: added trailing /s to dirs
bugfix: /README.TXT: to back up the version history: fixed svn_repo/ path
**/LICENSE.TXT: corrected "it recommended" to "it is recommended"
_license/non-open-source/applies_to.txt: deleted items: tip revision needs to refer to the revision at which the problem was discovered, not the revision before that
_license/non-open-source/applies_to.txt: deleted items: clarified that some things deleted after r11705 (when we started copyright scrubbing) might also be non-open-source, because they might have been deleted for reasons other than copyright scrubbing and thus would be missing the "copyright scrub" tag
/LICENSE.TXT: removed separate non-open-source disclaimer about parts of the version history before r11705, because this is now listed in _license/non-open-source/applies_to.txt
_license/non-open-source/applies_to.txt: included deleted items that are non-open-source, so that the list correctly applies to the entire repository (which includes the version history)
**/LICENSE.TXT: disclaimer about things that are not open-source: removed "these are copyright their respective owners", because this is understood
_license/README.TXT: changed "parts" to "components" for clarity
_license/non-open-source/applies_to.txt: UArizona?: added BIEN2/traits_observation_counts.xls
/README.TXT: to back up the version history: use svnsync instead of `git svn fetch`, so that the backup is in a format that can be directly reimported into an svn repo
/LICENSE.TXT: parts of the repository that are not open-source: added ,s to clarify the meaning
/LICENSE.TXT: documented that this license refers specifically to the repository hosted at https://code.nceas.ucsb.edu/code/projects/bien/trunk/, since we also have a publicly-accessible checkout at http://fs.vegpath.org/, which contains numerous non-open-source files (more than are listed in _license/non-open-source/applies_to.txt)
fix: inputs/VegBIEN/Redmine/wiki/.htaccess: redirect to new main page when accessed without trailing /
web/BIEN3: redirected to .wiki since the wiki homepage now contains the list of all BIEN3 components
web/links/index.htm: updated to Firefox bookmarks. Quick links: use bien3.vegpath.org to link to the main BIEN3 page
added web/BIEN3 symlink, with .bien3 case-insensitive alias
_license/*/applies_to.txt: added exclusion for "UCSB modifications"
added _license/GPL2.txt.url
/LICENSE.TXT, _license/VegBank_schema/LICENSE.TXT: moved text of the GPL2 license into separate file _license/GPL2.txt to avoid cluttering up LICENSE.TXT
_license/*/LICENSE.TXT: documented that the first section of the file is from license.eml.txt in each folder
/LICENSE.TXT: GPL text: include the full range of copyright years
/LICENSE.TXT: added disclaimer that parts of the version history before r11705 (2013-11-21) are NOT open-source. this is the revision of the first copyright scrub.
added issues/ with LICENSE.TXT for the issue tracker (BSD)
wiki/_license/*/: symlink the contents instead of the folders, because the root dir's and wiki's applies_to.txt's will be different
wiki/LICENSE.TXT: documented what this license refers to (the wiki hosted at https://projects.nceas.ucsb.edu/nceas/projects/bien/wiki/)
wiki/LICENSE.TXT: added message about including _license/ when redistributing, as for /LICENSE.TXT
wiki/_license/LICENSE.TXT: replaced with README.TXT as in /_license
wiki/LICENSE.TXT: changed "the page text of the wiki" to "the wiki's components" since some of the attachments will eventually be marked as open-source
/LICENSE.TXT: moved emphasized sections to top
/LICENSE.TXT: changed "the open-source parts of the repository" to "the repository's components" so that this sentence makes sense once the non-open-source exception is removed
/LICENSE.TXT: documented that the non-open-source files are now listed in _license/non-open-source/applies_to.txt
added _license/non-open-source/applies_to.txt
removed schemas/VegX/run.py, which is a duplicate of _archive/bienDjango/trunk/bien/v/models.py (this may have ended up here because it was a wiki page attachment, but the initial revision did not indicate the source)
added _license/VegBank_schema/applies_to.txt
schemas/VegCore/vegbien_identifier_examples.xlsx: changed to symlink to Brad_Boyle/vegbien_identifier_examples.xlsx, to avoid duplicate copies
added _license/TWiki/ (for inputs/VegBIEN/TWiki/)
added _license/PostgreSQL_config (for schemas/*.conf)
*/LICENSE.TXT: GPL: removed divider between the copyright header and license text, since these are part of the same license
_license/ESA/: renamed to VegBank_schema/ because this license applies only to the VegBank schema
_license/README.TXT: merged ¶2 into ¶1
_license/README.TXT: replaced "authority information proving the open-source status" with the simpler "open-source proof". clarified that each LICENSE.TXT is meant as a proof.
/LICENSE.TXT: added note that when redistributing, it recommended to include _license/, so that the open-source proof is included along with the license text (this information should be here as well as in _license/README.TXT because redistributors might not think to look in that file)
/LICENSE.TXT: removed "svn" before "repository" because this would apply equally to a git export of the repository
/LICENSE.TXT: replaced "portions" with "parts" for consistency with the _license/README.TXT wording
_license/README.TXT: clarified that this folder contains authority information proving the open-source status of parts of the codebase. added recommendation to include this folder when redistributing, so that the open-source proof is included along with the license text.
_license/LICENSE.TXT: renamed to README.TXT because this actually contains a description of the folder, not license text
schemas/Makefile: removed no longer used rename/%, rotate. use %/install, %/publish instead to manage public schema versions.
schemas/Makefile: removed no longer needed public/uninstall, which is handled more generally by `%/uninstall: vegbien.sql` instead
schemas/Makefile: `%/uninstall: vegbien.sql`: added support for *_validation schema
schemas/Makefile: %/uninstall: split into 2 rules like for %/install, so that different actions can easily be added for the public schema
schemas/Makefile: %/publish: support auxiliary *_validation schemas
bugfix: schemas/Makefile: `%/install: vegbien.sql`: need -e before each s/// expression
schemas/Makefile: `%/install: vegbien.sql`: create a custom _validation schema for *each public schema, rather than one for the most recently-created public schema. this allows validations to continue to be run against a previous version of the DB while a new version is being imported.
validation/aggregating/traits/bien3_validations_traits_bien3.sql: include only records from the bien2_traits datasource, to ensure that the scope of all queries is limited to the datasource
validation/aggregating/*/*.sql (the output queries): added `SET search_path TO validation`
validation/aggregating/specimens/qualitative_validations_specimens.sql: added CREATE VIEW prefixes using the steps at wiki.vegpath.org/Aggregating_validations_refactoring#prepend-CREATE-VIEW
validation/aggregating/traits/bien3_validations_traits_bien3.sql: added CREATE VIEW prefixes using the steps at wiki.vegpath.org/Aggregating_validations_refactoring#prepend-CREATE-VIEW
validation/aggregating/plots/bien3_validations_salvias_vegbien.sql: added CREATE VIEW prefixes using the steps at wiki.vegpath.org/Aggregating_validations_refactoring#prepend-CREATE-VIEW
added validation/aggregating/specimens/qualitative_validations_specimens.sql template, with query descriptions from NY/qualitative_validations_source_db_NYBG.sql
validation/**: removed old versions of by=-tagged files. these were not renamed in the last commit because there were newer versions that they collided with. note that because , sorts before . , the newer version (with more ",name@inst" tags) was correctly renamed instead of the older version.
validation/**: removed by=... indicators so that we don't have to add an author name every time a file is changed. the author indicator is not needed now that both Brad's and my edits are open-source.
inputs/bien2_traits/TraitObservation/postprocess.sql: remove rows with no taxon name, which are invalid, and which helps simplify the aggregating validations queries
added validation/aggregating/pipeline/aggregating_validations_pipeline.odg with .png export
added validation/aggregating/plots/FIA/bien3_validations_fia_input.sql.eml.txt. note that a .url file is not needed because this information is in the .eml.txt file.
fix: validation/aggregating/traits/_archive/bien3_validations_traits_bien3.by=Brad@iPlant.sql: renamed to indicate that this is actually not Brad's original file, as the filename would seem to indicate
added validation/aggregating/plots/FIA/
validation/aggregating/*.by=Brad@iPlant*: placed under version control, since these are now open-source
validation/**: removed svn:ignore exclusion of iPlant-owned files, since these are now open-source
fix: inputs/VegBIEN/Redmine/svn/.htaccess: updated repository URL to point to trunk/
