Project

General

Profile

Statistics
| Revision:
  • svn:ignore: extern

# Date Author Comment
13442 05/13/2014 03:50 AM Aaron Marcuse-Kubitza

bugfix: schemas/public_.sql: plots_20_tnrs_names: tnrs_accepted*: use new accepted_*, not scrubbed_*, as scrubbed_* does not contain only the accepted name (as implied by the tnrs_accepted_* column name)

13441 05/13/2014 03:46 AM Aaron Marcuse-Kubitza

bugfix: inputs/.TNRS/schema.sql: taxon_scrub, etc.: undid rename of accepted name columns to scrubbed_* (r13435), because these are actually not the same (scrubbed_* is the combination of accepted and no-opinion names). the accepted name columns will now be named accepted_*, following the standard naming scheme.

13440 05/13/2014 03:28 AM Aaron Marcuse-Kubitza

bugfix: schemas/public_.sql: _plots_20_tnrs_names: morphospeciesSuffix must still be called morphospecies_suffix, not taxon_morphospecies, because that column has not been updated to the taxon_morphospecies yet (fixes r13435)

13439 05/13/2014 03:13 AM Aaron Marcuse-Kubitza

fix: inputs/.TNRS/schema.sql: taxon_scrub, etc.: scrubbed_*: use columns from MatchedTaxon whenever possible, to as much as possible avoid the need to join to taxon_scrub.scrubbed_unique_taxon_name.*

13438 05/13/2014 02:53 AM Aaron Marcuse-Kubitza

schemas/Makefile: vegbien.sql: exclude the same set of Source tables excluded by inputs/.TNRS/schema.sql, so that reinstalling TNRS doesn't change the contents of this file

13437 05/13/2014 02:29 AM Aaron Marcuse-Kubitza

bugfix: inputs/.TNRS/grants.sql: added GRANT statements from schema.sql because these aren't run by `make inputs/.TNRS/reinstall`

13436 05/12/2014 07:06 PM Aaron Marcuse-Kubitza

lib/tnrs.py: retrieval_request_template: taxonomic_constraint, source_sorting: documented their meaning and why they need to be on/off

13435 05/12/2014 04:28 PM Aaron Marcuse-Kubitza

schemas/public_.sql: _plots_20_tnrs_names: changed to set of columns requested by Martha

13434 05/10/2014 09:32 PM Aaron Marcuse-Kubitza

validation/aggregating/plots/bien3_validations_salvias_vegbien.sql: updated to DB

13433 05/10/2014 09:17 PM Aaron Marcuse-Kubitza

schemas/public_.sql: _plots_20_tnrs_names: added morphospecies suffix in order to also validate it

13432 05/10/2014 09:13 PM Aaron Marcuse-Kubitza

schemas/public_.sql: _plots_20_tnrs_names: added verbatim name in order to also validate the formation of the concatenated name

13431 05/10/2014 08:49 PM Aaron Marcuse-Kubitza

schemas/public_.sql: merged _plots_20_tnrs_input_names, _plots_21_tnrs_output_names into _plots_20_tnrs_names so the input and output names can be compared side-by-side

13430 05/10/2014 08:10 PM Aaron Marcuse-Kubitza

schemas/vegbien.sql: synced to DB, which adds Source.errors table indexes added in the import

13429 05/09/2014 07:27 PM Aaron Marcuse-Kubitza

/README.TXT: Full database import: for test import, turn off DB backup: added () after after_import to show that it's a function

13428 05/09/2014 07:24 PM Aaron Marcuse-Kubitza

/README.TXT: Full database import: refer to the shell instead of screen because it's also possible to use a nested shell for this

13427 05/09/2014 02:51 AM Aaron Marcuse-Kubitza

bugfix: /README.TXT: Full database import: to import just a subset of the datasources: need to set @inputs separately from declaring it, because declare does not support () on Mac

13426 05/09/2014 02:16 AM Aaron Marcuse-Kubitza

fix: /README.TXT: Full database import: on local machine: need to allow logging output to go to log files

13425 05/09/2014 02:05 AM Aaron Marcuse-Kubitza

/README.TXT: Full database import: for test import, added step to turn off DB backup

13424 05/09/2014 01:48 AM Aaron Marcuse-Kubitza

fix: /README.TXT: Full database import: setting $version: moved to inside screen to avoid affecting the outer shell

13423 05/09/2014 01:46 AM Aaron Marcuse-Kubitza

/README.TXT: Full database import: screen: added alternative for a small import (just use nested shell)

13422 05/09/2014 01:42 AM Aaron Marcuse-Kubitza

fix: /README.TXT: Full database import: clear any limit set in .profile: moved to inside screen because it must happen within screen to avoid affecting the outer shell

13421 05/09/2014 01:40 AM Aaron Marcuse-Kubitza

fix: /README.TXT: Full database import: added step to clear any limit set in .profile (applicable to local machine)

13420 05/09/2014 01:35 AM Aaron Marcuse-Kubitza

bugfix: schemas/Makefile: `%/install: public_.sql`: also need to replace `public` when preceded by ( , as it is in the validations queries

13419 05/07/2014 08:14 PM Aaron Marcuse-Kubitza

bugfix: /Makefile: install: use schemas/public/*re*install because schemas/public/install would errexit if the public schema already exists

13418 05/07/2014 07:17 PM Aaron Marcuse-Kubitza

bugfix: inputs/input.Makefile: $(datasrc_schema_exists): need to use $(datasrc), not $(schema), as $schema is only what this var is called in the runscripts

13417 05/07/2014 06:48 PM Aaron Marcuse-Kubitza

bugfix: inputs/analytical_db/: need dummy table.run file to cause a schema to be created for this datasource

13416 05/07/2014 06:44 PM Aaron Marcuse-Kubitza

fix: inputs/input.Makefile: $(sortFile): don't print the "add any missing tables to $(sortFile)" message every time the Makefile is run

13415 05/07/2014 06:44 PM Aaron Marcuse-Kubitza

bugfix: inputs/input.Makefile: install: only run this for datasource dirs

13414 05/07/2014 05:18 PM Aaron Marcuse-Kubitza

inputs/input.Makefile: install: use ./run's install target for clarity

13413 05/07/2014 05:18 PM Aaron Marcuse-Kubitza

lib/runscripts/import.run: added install() target

13412 05/07/2014 04:56 PM Aaron Marcuse-Kubitza

bugfix: inputs/input.Makefile: install: made it idempotent (using new $(datasrc_schema_exists)) so that it could be run by `make install` on an existing system

13411 05/07/2014 04:02 PM Aaron Marcuse-Kubitza

bugfix: inputs/input.Makefile: $(datasrc_schema_exists): need to use $(shell ...)

13410 05/07/2014 03:31 PM Aaron Marcuse-Kubitza

inputs/input.Makefile: added $(datasrc_schema_exists)

13409 05/07/2014 03:29 PM Aaron Marcuse-Kubitza

lib/runscripts/in_datasrc_dir.run: use new local.run

13408 05/07/2014 03:25 PM Aaron Marcuse-Kubitza

added lib/runscripts/local.run

13407 05/06/2014 03:39 PM Aaron Marcuse-Kubitza

web/links/index.htm: updated to Firefox bookmarks: iPlant links: fixed broken links

13406 05/06/2014 03:34 PM Aaron Marcuse-Kubitza

web/links/index.htm: updated to Firefox bookmarks: starscream links: removed favicons so they wouldn't wake the Mac up on network access whenever someone opens the links page

13405 05/03/2014 04:15 PM Aaron Marcuse-Kubitza

added validation/aggregating/bien3_validation_sequence.xlsx from Brad

13404 05/03/2014 03:17 PM Aaron Marcuse-Kubitza

validation/aggregating/plots/bien3_validations_salvias_vegbien.sql: _plots_21_tnrs_output_names: removed no longer needed disabling of enable_hashjoin (the lack of an additional join now makes `SET enable_seqscan = off;` sufficient to prevent a slow hash join)

13403 05/03/2014 03:10 PM Aaron Marcuse-Kubitza

bugfix: schemas/util.sql: explain2notice_msg_if_can(): don't use util.is_explainable() because the list provided by Postgres (http://www.postgresql.org/docs/9.3/static/sql-explain.html#AEN77691) excludes some query types that are in fact EXPLAIN-able

13402 05/03/2014 02:03 PM Aaron Marcuse-Kubitza

added inputs/VegBank/verify/outputBien.log.url

13401 05/03/2014 02:03 PM Aaron Marcuse-Kubitza

inputs/input.Makefile: add: verify/: also svn:ignore *.log

13400 05/03/2014 12:44 PM Aaron Marcuse-Kubitza

added validation/aggregating/plots/VegBank/bien3_validations_salvias_db_original_adjToVegBank.txt from Mike Lee

13399 05/03/2014 01:39 AM Aaron Marcuse-Kubitza

validation/aggregating/plots/bien3_validations_salvias_vegbien.sql: updated to DB

13398 05/03/2014 01:37 AM Aaron Marcuse-Kubitza

schemas/public_.sql: _plots_20_tnrs_input_names, _plots_21_tnrs_output_names: added Brad's comments

13397 05/03/2014 01:28 AM Aaron Marcuse-Kubitza

bugfix: schemas/util.sql: materialize_query(): contents generated from: need to include search_path so query is runnable

13396 05/03/2014 01:05 AM Aaron Marcuse-Kubitza

planning/meetings/BIEN conference call availability.xlsx: updated

13395 05/03/2014 12:47 AM Aaron Marcuse-Kubitza

validation/aggregating/plots/bien3_validations_salvias_vegbien.sql: _plots_21_tnrs_output_names: removed no longer needed setting of join_collapse_limit (the column now used does not involve an additional join)

13394 05/02/2014 11:34 PM Aaron Marcuse-Kubitza

schemas/vegbien.sql: _plots_21_tnrs_output_names: use scrubbed_unique_taxon_name (concatenated output name) because this is more similar to the concatenated name used in _plots_20_tnrs_input_names

13393 05/02/2014 11:22 PM Aaron Marcuse-Kubitza

bugfix: validation/aggregating/plots/bien3_validations_salvias_vegbien.sql: _plots_21_tnrs_output_names: needs to be run with `SET join_collapse_limit = DEFAULT; SET enable_hashjoin = off;` to produce the right query plan

13392 05/02/2014 10:57 PM Aaron Marcuse-Kubitza

fix: lib/util.py: dict_subset(): raise an error if collections.OrderedDict isn't available, because some callers may depend on this. note that using dict instead of OrderedDict may be the cause of the joining on the wrong columns bug (issue #902).

13391 05/02/2014 10:42 PM Aaron Marcuse-Kubitza

validation/aggregating/plots/bien3_validations_salvias_vegbien.sql: updated to DB

13390 05/02/2014 10:01 PM Aaron Marcuse-Kubitza

bugfix: schemas/public_.sql: _plots_21_tnrs_output_names: need to use TNRS output rather than input name

13389 05/02/2014 09:54 PM Aaron Marcuse-Kubitza

schemas/public_.sql: added _plots_20_tnrs_input_names, _plots_21_tnrs_output_names

13388 05/02/2014 06:17 PM Aaron Marcuse-Kubitza

schemas/public_.sql: public_validations: schema comment: documented how to run just the output queries on a particular datasource

13387 05/02/2014 04:53 PM Aaron Marcuse-Kubitza

fix: schemas/public_.sql: rm_all_queries(): also remove output-only result tables

13386 05/02/2014 04:50 PM Aaron Marcuse-Kubitza

schemas/public_.sql: query_view_regexp(): also match auxiliary tables

13385 05/02/2014 04:14 PM Aaron Marcuse-Kubitza

schemas/public_.sql: added remake_output_tables(datasource_schema, prefix)

13384 05/02/2014 03:53 PM Aaron Marcuse-Kubitza

schemas/util.sql: added loop_ignore_errors()

13383 05/01/2014 10:38 PM Aaron Marcuse-Kubitza

schemas/public_.sql: added output_queries(prefix)

13382 05/01/2014 10:27 PM Aaron Marcuse-Kubitza

schemas/public_.sql: validation_views(): added prefix param, which can be used for additional subsetting

13381 05/01/2014 10:13 PM Aaron Marcuse-Kubitza

schemas/public_.sql: query_relations(): removed no longer used max_prefix_len default param

13380 05/01/2014 10:05 PM Aaron Marcuse-Kubitza

schemas/public_.sql: query_relations(): use new public_validations.truncated_prefixed_name_regexp(), which does not require passing in a max_prefix_len

13379 05/01/2014 10:03 PM Aaron Marcuse-Kubitza

schemas/vegbien.sql: added public_validations.truncated_prefixed_name_regexp(), which does not require passing in a max_prefix_len

13378 05/01/2014 08:55 PM Aaron Marcuse-Kubitza

schemas/public_.sql: rm_output_queries(), validation_views(): use public_validations.query_view_regexp()

13377 05/01/2014 08:42 PM Aaron Marcuse-Kubitza

schemas/public_.sql: added public_validations.query_view_regexp()

13376 05/01/2014 08:06 PM Aaron Marcuse-Kubitza

bugfix: schemas/public_.sql: public_validations.rm_output_queries(): default regexp needs to be anchored at beginning of string, and should match the regexp in public_validations.validation_views()

13375 05/01/2014 01:58 PM Aaron Marcuse-Kubitza

bugfix: inputs/input.Makefile: %/postprocess: invoke runscript if it exists

13374 05/01/2014 01:37 PM Aaron Marcuse-Kubitza

lib/runscripts/validations.pg.sql.run: export_(): make the export idempotent for easier re-runnability

13373 05/01/2014 01:37 PM Aaron Marcuse-Kubitza

bugfix: lib/runscripts/validations.pg.sql.run: updated to reflect that validations.sql is now located inside a subdir, not the datasrc dir

13372 05/01/2014 01:29 PM Aaron Marcuse-Kubitza

fix: lib/runscripts/file.pg.sql.run: removed include of in_datasrc_dir.run, because this location does not apply to all .sql export scripts

13371 05/01/2014 01:15 PM Aaron Marcuse-Kubitza

bugfix: inputs/input.Makefile: validations.sql must be in a subdir so it won't get run by sql/install

13370 05/01/2014 01:11 PM Aaron Marcuse-Kubitza

bugfix: inputs/input.Makefile: validations.sql must be in a subdir so it won't get run by sql/install

13369 05/01/2014 05:20 AM Aaron Marcuse-Kubitza

inputs/input.Makefile: install: also run validate/install

13368 05/01/2014 04:44 AM Aaron Marcuse-Kubitza

inputs/input.Makefile: added validate/install

13367 05/01/2014 04:09 AM Aaron Marcuse-Kubitza

lib/runscripts/validations.pg.sql.run: export_(): make the export idempotent for easier re-runnability

13366 05/01/2014 03:22 AM Aaron Marcuse-Kubitza

bugfix: inputs/SALVIAS/validations.sql: need to cast character varying to text so that the types of each side of if() match

13365 05/01/2014 03:14 AM Aaron Marcuse-Kubitza

bugfix: lib/sh/db.sh: pg_dump(): need use_pg to import $pg_database before checking for existence of $database

13364 05/01/2014 03:11 AM Aaron Marcuse-Kubitza

lib/sh/util.sh: import_vars: documented that it's idempotent

13363 04/30/2014 07:10 PM Aaron Marcuse-Kubitza

/Makefile: python-Linux: removed ordereddict, since OrderedDict is now obtained from `collections`

13362 04/30/2014 07:06 PM Aaron Marcuse-Kubitza

web/links/index.htm: updated to Firefox bookmarks: fixed broken links

13361 04/30/2014 06:58 PM Aaron Marcuse-Kubitza

bugfix: lib/util.py: use OrderedDict from collections rather than ordereddict to work with Mac OS X 10.8 Mountain Lion (http://vegpath.org/links/#OrderedDict)

13360 04/30/2014 06:42 PM Aaron Marcuse-Kubitza

/Makefile: apache-Linux: updated for Ubuntu 14.04, which includes Apache 2.4 in the distribution

13359 04/30/2014 06:23 PM Aaron Marcuse-Kubitza

/Makefile: install: documented that schemas/public/install also tests that a clean public schema will be installable by full-database import

13358 04/30/2014 06:12 PM Aaron Marcuse-Kubitza

/Makefile: install: skip the test step at the end because this is not needed for the installation (it can be run separately if desired)

13357 04/30/2014 05:46 PM Aaron Marcuse-Kubitza

bugfix: **/postprocess.sql: don't use the public schema, because this creates an unsatisfied dependency while the database is being installed, and breaks `make install`

13356 04/30/2014 05:36 PM Aaron Marcuse-Kubitza

schemas/public_.sql: _km_to_m(): made it a wrapper around util._km_to_m() so this function is only in one place

13355 04/30/2014 05:32 PM Aaron Marcuse-Kubitza

schemas/util.sql: added _km_to_m() for use when public._km_to_m() is not defined yet

13354 04/29/2014 11:36 PM Aaron Marcuse-Kubitza

bugfix: benign_does_not_exist_error(): removed ignore_e=3, because this exit status is also used for other errors

13353 04/29/2014 11:35 PM Aaron Marcuse-Kubitza

fix: lib/sh/db.sh: benign_does_not_exist_error(): use benign_error=1, which is now supported properly by stderr_matches()

13352 04/29/2014 11:34 PM Aaron Marcuse-Kubitza

bugfix: lib/sh/util.sh: stderr_matches(): support $benign_error properly, by handling exit status logging in this func instead

13351 04/29/2014 11:03 PM Aaron Marcuse-Kubitza

bugfix: lib/sh/db.sh: pg_schema_exists(): also need to benignify "does not exist" error if returns false

13350 04/29/2014 10:42 PM Aaron Marcuse-Kubitza

bugfix: lib/sh/util.sh: stderr_matches(): need to separately display errors that were incorrectly suppressed due to $benign_error

13349 04/29/2014 10:36 PM Aaron Marcuse-Kubitza

bugfix: lib/sh/util.sh: is_err(): rethrow must be inverted (rethrow->false if error)

13348 04/29/2014 10:32 PM Aaron Marcuse-Kubitza

lib/sh/util.sh: added is_err()

13347 04/29/2014 09:53 PM Aaron Marcuse-Kubitza

lib/sh/local.sh: public_schema_exists(): moved to lib/sh/db.sh since this no longer depends on BIEN-specific configurations

13346 04/29/2014 09:42 PM Aaron Marcuse-Kubitza

bugfix: lib/sh/db.sh: public_schema_exists(): don't hide the function call tree so it's clear which function is running the psql commands

13345 04/29/2014 09:40 PM Aaron Marcuse-Kubitza

bugfix: lib/sh/db.sh: public_schema_exists(): don't hide the function call tree so it's clear which function is running the psql commands

13344 04/29/2014 09:38 PM Aaron Marcuse-Kubitza

fix: *{.sh,run}: stderr_matches() callers: added benign_error=1 where needed

13343 04/29/2014 09:36 PM Aaron Marcuse-Kubitza

fix: *{.sh,run}: stderr_matches() callers: usage: documented that they may require benign_error=1