Activity
From 05/02/2014 to 05/31/2014
05/31/2014
- 10:31 PM Revision 13580: inputs/.TNRS/schema.sql: tnrs.match_num: made it NOT NULL now that it's populated
- 10:28 PM Revision 13579: inputs/.TNRS/schema.sql: tnrs: populate match_num
- 10:25 PM Revision 13578: inputs/.TNRS/schema.sql: tnrs: populate match_num
- 09:50 PM Revision 13577: inputs/.TNRS/schema.sql: tnrs: documented how to add and remove columns
- 08:58 PM Revision 13576: inputs/.TNRS/schema.sql: made COMMENTs start on their own line, using the steps at wiki.vegpath.org/Postgres_queries#make-COMMENTs-start-on-their-own-line
- 08:58 PM Revision 13575: inputs/.TNRS/schema.sql: made COMMENTs start on their own line, using the steps at wiki.vegpath.org/Postgres_queries#make-COMMENTs-start-on-their-own-line
- 08:11 PM Revision 13574: inputs/test_taxonomic_names/_scrub/*: updated to TNRS schema
- 08:10 PM Revision 13573: inputs/.TNRS/schema.sql: tnrs: added match_num
- 08:06 PM Revision 13572: inputs/.TNRS/data.sql.run: refresh(): documented runtime (1 min)
- 08:03 PM Revision 13571: schemas/Makefile: added back rename/%, which *is* used by `inputs/.TNRS/data.sql.run refresh`. updated it to use schema bundles.
- 06:44 PM Revision 13570: inputs/.TNRS/schema.sql: added tnrs__match_num__next()
- 06:29 PM Revision 13569: inputs/.TNRS/schema.sql: added tnrs__batch_begin() trigger to populate the match_num (match sort order)
- 05:06 AM Revision 13568: exports/2014-3-11.Jeff_Ott.climatic_range_determinants.csv.run: documented export_() runtime (11 min) and rowcount (11 million matching the filter criteria)
05/30/2014
- 06:34 PM Revision 13567: inputs/.TNRS/schema.sql: added tnrs__batch_begin() trigger to populate the match_num (match sort order)
- 06:22 PM Revision 13566: schemas/util.sql: added seq__reset()
- 06:15 PM Revision 13565: schemas/util.sql: added seq__create()
- 06:11 PM Revision 13564: fix: schemas/util.sql: try_cast(), is_castable(): also catch invalid_schema_name, thrown by `'pg_temp.__'::regclass`
- 02:49 PM Revision 13563: backups/TNRS.backup.md5: updated
- 07:50 AM Revision 13562: lib/tnrs.py: max_names: increased to 100000 because the dev server can handle more names (no simultaneous users), as decided in the conference call (wiki.vegpath.org/2014-05-29_conference_call#TNRS)
- 07:42 AM Revision 13561: schemas/vegbien.ERD.mwb: regenerated exports
- 07:39 AM Revision 13560: schemas/vegbien.ERD.mwb: re-updated to schemas/vegbien.my.sql, which now recognizes the broken tables. fixed sync issues. vegbien.ERD.mwb is now fully in sync with vegbien.my.sql.
- 07:37 AM Revision 13559: fix: lib/PostgreSQL-MySQL.csv: need to replace "double precision" with "double" to work with MySQL Workbench 5.2.47
- 06:58 AM Revision 13558: schemas/vegbien.ERD.mwb: updated to schemas/vegbien.my.sql. some tables weren't recognized (likely due to bugs in MySQL Workbench 5.2.47), and have been left as-is (unsynced). note that downgrading to 5.2.35 is not an option, because that is fatally broken by a system upgrade.
- 05:58 AM Revision 13557: fix: schemas/vegbien.ERD.mwb: use schemas/vegbien.my.sql instead of schemas/vegbien.my.sql.changes.sql as the sync source
- 05:27 AM Revision 13556: schemas/vegbien.ERD.mwb: switched back to MySQL Workbench 6.1.6 version, which also works with MySQL Workbench 5.2.47
- 05:17 AM Revision 13555: schemas/vegbien.ERD.mwb: restored version for MySQL Workbench 5.2.35 (undid r13549), as 6.1.6 has bugs in the DDL file sync
- 05:05 AM Revision 13554: schemas/vegbien.ERD.mwb: renamed to vegbien.ERD.MySQL_Workbench_6.1.6.mwb to differentiate the versions for different versions of MySQL Workbench
- 04:49 AM Revision 13553: removed no longer used config/VirtualBox_VMs/VegCore ERD/. use the Ubuntu 14.04 VM instead, which now has the VegCore ERD setup.
- 04:48 AM Revision 13552: config/VirtualBox_VMs/Ubuntu 14.04/Ubuntu 14.04.vbox: added VegCore ERD setup
- 03:30 AM Revision 13551: schemas/: svn:ignore *.changes.sql, needed for MySQL Workbench 6.1.6
- 03:30 AM Revision 13550: schemas/vegbien.ERD.mwb: updated
- 03:16 AM Revision 13549: schemas/vegbien.ERD.mwb: updated layout for MySQL Workbench 6.1.6, which uses a different line spacing
05/29/2014
- 11:53 AM Revision 13548: lib/tnrs.py: commented out the value of max_names that is not active, for clarity
05/28/2014
- 03:54 PM Revision 13547: config/VirtualBox_VMs/vegbiendev/vegbiendev.vbox: updated: merged post-bootloader installation steps into one snapshot now that eth1 addition is successful
- 03:49 PM Revision 13546: config/VirtualBox_VMs/vegbiendev/vegbiendev.vbox: updated, which adds eth1
- 03:47 PM Revision 13545: config/VirtualBox_VMs/Ubuntu 12.04/Ubuntu 12.04.vbox: updated
05/27/2014
- 11:12 PM Revision 13544: lib/tnrs.py: sources: updated to list/sort order in issue #917
- 10:39 PM Revision 13543: added exports/2014-3-11.Jeff_Ott.climatic_range_determinants.csv.run
- 10:38 PM Revision 13542: schemas/public_.sql: added 2014-3-11.Jeff_Ott.climatic_range_determinants
- 10:17 PM Revision 13541: schemas/public_.sql: analytical_stem_view: added scrubbed_taxon_name_with_author, needed by Jeff Ott's analysis (wiki.vegpath.org/Data_requests)
- 10:13 PM Revision 13540: inputs/.TNRS/schema.sql: taxon_scrub.scrubbed_unique_taxon_name.*: added scrubbed_taxon_name_with_author, needed by Jeff Ott's analysis (wiki.vegpath.org/Data_requests)
- 10:09 PM Revision 13539: schemas/public_.sql: added scrubbed_specific_epithet, scrubbed_species_binomial, which are needed by Jeff Ott's analysis (wiki.vegpath.org/Data_requests)
- 10:08 PM Revision 13538: fix: schemas/public_.sql: sync_analytical_stem_to_view(): removed fkey to source.shortname because this prevents reloading individual datasources
- 09:24 PM Task #927 (Resolved): move morphospecies name formation to normalized DB
- see [[TNRS derived columns#scrubbed_morphospecies_binomial]] > @scrubbed_morphospecies_binomial@
- 09:19 PM Revision 13537: added downloads/
- 03:27 AM Revision 13536: schemas/public_.sql: added scrubbed_specific_epithet, scrubbed_species_binomial, which are needed by Jeff Ott's analysis (wiki.vegpath.org/Data_requests)
- 03:21 AM Revision 13535: fix: schemas/util.sql: mk_drop_from_create(): also support CREATE queries that include the SELECT statement on the same line as the CREATE
- 12:33 AM Revision 13534: schemas/public_.sql: analytical_stem_view: scrubbed_morphospecies_binomial: use new taxon_scrub.scrubbed_morphospecies_binomial
- 12:28 AM Revision 13533: inputs/.TNRS/schema.sql: taxon_scrub: added scrubbed_morphospecies_binomial, analogous to accepted_morphospecies_binomial for scrubbed_*
- 12:13 AM Revision 13532: inputs/.TNRS/schema.sql: taxon_scrub: added scrubbed_morphospecies_binomial, analogous to accepted_morphospecies_binomial for scrubbed_*
05/26/2014
- 11:54 PM Revision 13531: inputs/.TNRS/schema.sql: taxon_scrub: documented how to modify it
- 11:51 PM Revision 13530: inputs/.TNRS/schema.sql: added taxon_scrub_modify()
- 11:41 PM Revision 13529: schemas/util.sql: create_if_not_exists(): print message if already exists, so the function doesn't inexplicably appear not to have run at all
- 11:20 PM Revision 13528: inputs/.TNRS/schema.sql: added taxon_scrub_modify()
05/23/2014
- 06:17 PM Revision 13527: inputs/.TNRS/schema.sql: MatchedTaxon_modify(): use simpler util.recreate_view()
- 06:15 PM Revision 13526: inputs/.TNRS/schema.sql: MatchedTaxon_modify(): documented usage
- 06:09 PM Revision 13525: schemas/util.sql: added recreate_view(), a special case of util.recreate()
- 04:24 PM Revision 13524: fix: schemas/util.sql: recreate(): usage: use `schema` instead of `schemas`
- 04:20 PM Revision 13523: config/VirtualBox_VMs/vegbiendev/vegbiendev.vbox: updated
- 03:57 PM Revision 13522: bugfix: schemas/public_.sql: _plots_20_tnrs_names: verbatim_name_with_author: use taxonverbatim.taxonomicname rather than taxonlabel.taxonomicname
- 02:16 AM Revision 13521: config/VirtualBox_VMs/Ubuntu */*.vbox: updated
05/22/2014
- 04:43 PM Task #932 (New): Discovering BIEN data through iPlant
iPlant will assign DOIs to datasets made public through its infrastructure. For the BIEN db data, the plan is:
* t...- 04:32 PM Task #931 (New): BIEN db at iPlant
- Parent issue for tasks related to moving BIEN3 relational database and associated analytical views to iPlant.
- 04:20 PM Revision 13520: bugfix: config/VirtualBox_VMs/: switched from symlinks to hard links, because svn does not follow symlinks
- 02:07 PM Revision 13519: added config/VirtualBox_VMs/, containing the .vbox settings and password.txt (a non-empty password is needed for some system commands)
05/21/2014
- 07:30 PM Revision 13518: inputs/.TNRS/schema.sql: MatchedTaxon_modify(): removed no longer needed DROP VIEW statement
- 07:30 PM Revision 13517: schemas/util.sql: recreate(): perform the correct DROP VIEW in the function itself so that the caller does not have to worry about forming it properly
- 07:19 PM Revision 13516: bugfix: schemas/util.sql: mk_drop_from_create(): added `DROP`
- 07:18 PM Revision 13515: schemas/util.sql: added mk_drop_from_create()
- 06:52 PM Revision 13514: schemas/util.sql: added regexp_match()
- 05:58 PM Revision 13513: planning/meetings/BIEN conference call availability.xlsx: updated
- 05:50 PM Revision 13512: fix: schemas/util.sql: force_recreate(): renamed to just recreate(), because "force" normally implies that things will be deleted, which this function does not do
- 05:49 PM Revision 13511: fix: schemas/public_.sql: _plots_20_tnrs_names: fixed order of tnrs_taxonomic_status, tnrs_accepted_name_family
- 04:45 PM Revision 13510: schemas/public_.sql: _plots_20_tnrs_names: added tnrs_taxonomic_status, as requested in issue #915
- 04:34 PM Revision 13509: schemas/public_.sql: analytical_stem_view: taxonomic_status: don't filter with map_taxonomic_status() anymore since this is now done by "TNRS"."MatchedTaxon"
- 04:25 PM Revision 13508: fix: inputs/.TNRS/schema.sql: MatchedTaxon.taxonomicStatus: filter using map_taxonomic_status() so that the corrected value is available in the normalized DB, not just analytical_stem
- 04:05 PM Revision 13507: inputs/.TNRS/schema.sql: MatchedTaxon: to modify: use new MatchedTaxon_modify(), which eliminates the work of putting together the dependent views
- 03:53 PM Revision 13506: inputs/.TNRS/schema.sql: added MatchedTaxon_modify()
- 03:44 PM Revision 13505: schemas/util.sql: show_create_view(): use new mk_set_relation_metadata()
- 03:41 PM Revision 13504: schemas/util.sql: added mk_set_relation_metadata()
- 04:13 AM Revision 13503: bugfix: inputs/.TNRS/schema.sql: map_taxonomic_status(): need to use accepted name instead of scrubbed name (which also includes no-opinion names), as described at http://wiki.vegpath.org/2013-11-14_conference_call#taxonomic-fields. this used to be the accepted name, but got switched when the concatenated name was also used to store the matched name for no-opinion names.
- 01:50 AM Revision 13502: schemas/public_.sql: _plots_20_tnrs_names: use accepted_morphospecies_binomial, which is now available directly in the TNRS results, instead of just the morphospecies_suffix, as requested in issue #915
- 01:27 AM Revision 13501: inputs/.TNRS/schema.sql: MatchedTaxon: documented how to modify it (using util.force_recreate())
- 01:23 AM Revision 13500: fix: lib/PostgreSQL-MySQL.csv: also remove left-behind lines such as `$$);`
- 12:59 AM Revision 13499: schemas/util.sql: force_recreate(): documented usage, which is somewhat complex and has several important parts
05/20/2014
- 05:46 PM Revision 13498: inputs/.TNRS/schema.sql: MatchedTaxon, etc.: added accepted_morphospecies_binomial derived field
- 05:40 PM Revision 13497: bugfix: schemas/util.sql: restore_views(): need to specify view name for a manual existence check, in case the view definition becomes invalid, which would produce a nonstandard (uncatchable) exception
- 05:33 PM Revision 13496: bugfix: schemas/util.sql: explain(): don't display any log messages printed by IMMUTABLE functions in the query, which EXPLAIN apparently runs
- 05:21 PM Revision 13495: schemas/util.sql: create_if_not_exists(): added relation param, which can be used to ensure that a standard exception is always generated if the relation exists, even if the table definition would be invalid (which generates a variety of exceptions)
- 04:26 PM Task #929 (New): change TNRS client to store metadata
- * see details in [[2013-06-13 conference call#include-TNRS-version-and-settings-in-TNRS-cache]]
* after implementi... - 04:17 PM Task #928 (New): switch to new TNRS setup
- # #927 (part of #915)
# #917
# #916 (using rescrubbed names)
# #915
# -#929 (metadata)- _postponed_ - 03:26 PM Task #927 (Resolved): move morphospecies name formation to normalized DB
- * currently, it's only in the analytical_stem view
- 02:39 PM Revision 13494: /README.TXT: to back up the vegbiendev databases: added step to review diff before performing backup, in case the backup scripts break and try to synchronize things incorrectly
05/19/2014
- 09:08 PM Revision 13493: bugfix: schemas/util.sql: try_cast(), is_castable(): also need to handle syntax_error_or_access_rule_violation, which is thrown by ::regclass
- 08:52 PM Revision 13492: schemas/util.sql: added relation_exists()
- 08:27 PM Revision 13491: schemas/util.sql: save_drop_views()/restore_views(): store the view path in addition to the definition so it can be used in restoring
- 08:24 PM Revision 13490: bugfix: schemas/util.sql: in_reverse(): can't use `SELECT *` for composite types because this expands them
- 08:21 PM Revision 13489: bugfix: schemas/util.sql: array_reverse(): can't use `SELECT *` for composite types because this expands them
- 07:50 PM Revision 13488: schemas/util.sql: save_drop_views()/restore_views(): use opaque return type so it can be changed without changing the local var type in functions that use this
- 07:22 PM Revision 13487: schemas/util.sql: force_recreate(): recreate_users_cmds: renamed to restore_views_info since this is now whatever is needed by util.restore_views()
- 07:16 PM Revision 13486: schemas/util.sql: added restore_views() and use it in force_recreate()
- 02:07 AM Revision 13485: bugfix: schemas/util.sql: save_drop_views(): views must be *dropped* in reverse dependency order, but *returned* in forward dependency order
- 01:36 AM Revision 13484: bugfix: schemas/util.sql: force_recreate(): dependent views need to be re-created in *forward* dependency order. this is done by reversing the dependencies only for save_drop_views().
- 12:48 AM Revision 13483: bugfix: schemas/util.sql: mk_set_comment(): need to use VIEW instead of TABLE for views
- 12:29 AM Revision 13482: bugfix: schemas/util.sql: show_create_view(): also need to include view comment
- 12:24 AM Revision 13481: fix: schemas/util.sql: mk_set_comment(): added ;
- 12:20 AM Revision 13480: fix: schemas/util.sql: show_set_comment(): don't display a COMMENT ON statement if no comment, because this will be appended to table defs, etc. and would create clutter
- 12:16 AM Revision 13479: bugfix: schemas/util.sql: mk_set_comment(): need to handle NULL comment properly
- 12:06 AM Revision 13478: schemas/util.sql: added show_set_comment(regclass)
05/18/2014
- 11:57 PM Revision 13477: schemas/util.sql: set_comment(): use util.mk_set_comment()
- 11:48 PM Revision 13476: schemas/Makefile: added vegbien/install, vegbien/uninstall which also handle TNRS
05/17/2014
- 06:26 PM Revision 13475: fix: schemas/Makefile: vegbien.sql: also remake inputs/.TNRS/schema.sql since its contents change along with vegbien.sql
- 06:22 PM Revision 13474: schemas/util.sql: added mk_set_comment()
- 06:22 PM Revision 13473: bugfix: lib/runscripts/util.run: $is_runscript: unexport so don't pass it to invoked scripts
- 06:01 PM Revision 13472: backups/pg_snapshot: documented runtime for initial backup (4 h)
- 05:36 PM Revision 13471: bugfix: schemas/util.sql: force_recreate(): recreate_users_cmds: use create_if_not_exists() rather than eval(), because cmd might manually re-create a deleted dependent view, causing it to already exist
- 05:27 PM Revision 13470: bugfix: schemas/util.sql: save_drop_view(): util.show_create_view(): don't include `OR REPLACE` because that causes nonuniform errors (eg. invalid_table_definition), instead of the standard duplicate_table exception caught by util.create_if_not_exists()
- 05:25 PM Revision 13469: fix: schemas/util.sql: show_create_view(): removed extra trailing ; because pg_get_viewdef() now includes one
- 05:24 PM Revision 13468: schemas/util.sql: show_create_view(): made inclusion of `OR REPLACE` configurable
- 05:10 PM Revision 13467: schemas/util.sql: save_drop_view(): use util.drop_view()
- 03:40 PM Revision 13466: /README.TXT: Maintenance: added steps to back up the vegbiendev databases
- 02:15 PM Revision 13465: bugfix: lib/runscripts/util.run: run_args_cmd(): don't prepend main to args if no args, because for a non-runscript, all args will be passed to main(), leading `main` to be doubled
- 01:30 PM Revision 13464: lib/tnrs.py: use the TNRS dev server (with private URL in tnrs.url) instead of the live server, since that contains datasources that we need
- 01:29 PM Revision 13463: lib/streams.py: added file_get_contents()
- 01:14 PM Revision 13462: lib/tnrs.py: configure the server separately from the base URL
- 01:12 PM Revision 13461: lib/: svn:ignore tnrs.url so the TNRS dev server URL does not become public
05/16/2014
05/15/2014
- 11:57 PM Revision 13459: bugfix: web/.phpPgAdmin/.htaccess: custom pages: need to handle in query string because .php filename would be treated as path element
- 11:32 PM Revision 13458: web/.phpPgAdmin/.htaccess: support custom .php pages
- 10:30 AM Revision 13457: planning/meetings/BIEN conference call availability.xlsx: updated
- 09:43 AM Task #917: TNRS: Instructions for new version with TPL
- Aaron, for now (next couple of months, this must be done on the development TNRS web app since TPL is only on the dev...
05/13/2014
- 02:32 PM Task #917 (New): TNRS: Instructions for new version with TPL
- From Brad:
I’ve given some thought to the TPL matter. The algorithm isn’t hard, but Aaron will have to do the sort... - 11:19 AM Revision 13456: schemas/util.sql: save_drop_views(): return text[] instead of concatenated text so that the queries will be run individually and be easier to debug
- 11:00 AM Revision 13455: schemas/util.sql: added eval(queries text[])
- 09:43 AM Revision 13454: bugfix: schemas/util.sql: force_recreate(): need to drop users in reverse to avoid dependencies that block the drop
- 09:25 AM Revision 13453: schemas/util.sql: added array_reverse()
- 09:25 AM Revision 13452: schemas/util.sql: added in_reverse()
- 08:39 AM Revision 13451: bugfix: schemas/util.sql: force_recreate(): need to use [[:lower:]], not [:lower:]
- 08:34 AM Revision 13450: schemas/util.sql: force_recreate(): debug-print PG_EXCEPTION_DETAIL, users to assist in debugging
- 08:33 AM Revision 13449: fix: schemas/util.sql: debug_print_value(): added trailing newline to visually distinguish the printed value
- 08:27 AM Revision 13448: schemas/util.sql: added debug_print_var()
- 07:06 AM Revision 13447: schemas/util.sql: debug_print_return_value(): use new util.debug_print_value()
- 06:59 AM Revision 13446: schemas/util.sql: added debug_print_value()
- 05:45 AM Revision 13445: fix: schemas/util.sql: util.force_recreate(): also support dropping things besides tables
- 04:50 AM Revision 13444: inputs/.TNRS/schema.sql: MatchedTaxon.Accepted_name_species: mapped to accepted_species_binomial
- 04:09 AM Revision 13443: fix: inputs/.TNRS/schema.sql: COMMENTs: always include newline before and after
- 03:50 AM Revision 13442: bugfix: schemas/public_.sql: _plots_20_tnrs_names: tnrs_accepted_*: use new accepted_*, *not* scrubbed_*, as scrubbed_* does *not* contain only the accepted name (as implied by the tnrs_accepted_* column name)
- 03:46 AM Revision 13441: bugfix: inputs/.TNRS/schema.sql: taxon_scrub, etc.: undid rename of accepted name columns to scrubbed_* (r13435), because these are actually *not* the same (scrubbed_* is the combination of accepted and no-opinion names). the accepted name columns will now be named accepted_*, following the standard naming scheme.
- 03:28 AM Revision 13440: bugfix: schemas/public_.sql: _plots_20_tnrs_names: morphospeciesSuffix must still be called morphospecies_suffix, not taxon_morphospecies, because that column has not been updated to the taxon_morphospecies yet (fixes r13435)
- 03:13 AM Revision 13439: fix: inputs/.TNRS/schema.sql: taxon_scrub, etc.: scrubbed_*: use columns from MatchedTaxon whenever possible, to as much as possible avoid the need to join to taxon_scrub.scrubbed_unique_taxon_name.*
- 02:53 AM Revision 13438: schemas/Makefile: vegbien.sql: exclude the same set of Source tables excluded by inputs/.TNRS/schema.sql, so that reinstalling TNRS doesn't change the contents of this file
- 02:29 AM Revision 13437: bugfix: inputs/.TNRS/grants.sql: added GRANT statements from schema.sql because these aren't run by `make inputs/.TNRS/reinstall`
05/12/2014
- 07:06 PM Revision 13436: lib/tnrs.py: retrieval_request_template: taxonomic_constraint, source_sorting: documented their meaning and why they need to be on/off
- 05:03 PM Task #916: Taxon name validation: VegBank
- regarding:
Line 617: TNRS gives a synonym for Aronia prunifolia in a different genus, but this is missed here
L... - 04:35 PM Task #916 (New): Taxon name validation: VegBank
- Hi Aaron,
Bob only had time to get part way through the VegBank taxon validation file you sent, but there are some... - 04:28 PM Revision 13435: schemas/public_.sql: _plots_20_tnrs_names: changed to set of columns requested by Martha
- 03:13 PM Task #915 (New): Taxon Name Validation
- 1. Please correct the following in taxonMorphospecies:
1.1. -Family should be included in taxonMorphospecies only ...
05/10/2014
- 09:32 PM Revision 13434: validation/aggregating/plots/bien3_validations_salvias_vegbien.sql: updated to DB
- 09:17 PM Revision 13433: schemas/public_.sql: _plots_20_tnrs_names: added morphospecies suffix in order to also validate it
- 09:13 PM Revision 13432: schemas/public_.sql: _plots_20_tnrs_names: added verbatim name in order to also validate the formation of the concatenated name
- 08:49 PM Revision 13431: schemas/public_.sql: merged _plots_20_tnrs_input_names, _plots_21_tnrs_output_names into _plots_20_tnrs_names so the input and output names can be compared side-by-side
- 08:10 PM Revision 13430: schemas/vegbien.sql: synced to DB, which adds Source.errors table indexes added in the import
05/09/2014
- 07:27 PM Revision 13429: /README.TXT: Full database import: for test import, turn off DB backup: added () after after_import to show that it's a function
- 07:24 PM Revision 13428: /README.TXT: Full database import: refer to the shell instead of screen because it's also possible to use a nested shell for this
- 02:51 AM Revision 13427: bugfix: /README.TXT: Full database import: to import just a subset of the datasources: need to set @inputs separately from declaring it, because declare does not support () on Mac
- 02:16 AM Revision 13426: fix: /README.TXT: Full database import: on local machine: need to allow logging output to go to log files
- 02:05 AM Revision 13425: /README.TXT: Full database import: for test import, added step to turn off DB backup
- 01:48 AM Revision 13424: fix: /README.TXT: Full database import: setting $version: moved to inside screen to avoid affecting the outer shell
- 01:46 AM Revision 13423: /README.TXT: Full database import: screen: added alternative for a small import (just use nested shell)
- 01:42 AM Revision 13422: fix: /README.TXT: Full database import: clear any limit set in .profile: moved to inside screen because it must happen within screen to avoid affecting the outer shell
- 01:40 AM Revision 13421: fix: /README.TXT: Full database import: added step to clear any limit set in .profile (applicable to local machine)
- 01:35 AM Revision 13420: bugfix: schemas/Makefile: `%/install: public_.sql`: also need to replace `public` when preceded by ( , as it is in the validations queries
05/07/2014
- 08:14 PM Revision 13419: bugfix: /Makefile: install: use schemas/public/*re*install because schemas/public/install would errexit if the public schema already exists
- 07:17 PM Revision 13418: bugfix: inputs/input.Makefile: $(datasrc_schema_exists): need to use $(datasrc), not $(schema), as $schema is only what this var is called in the runscripts
- 06:48 PM Revision 13417: bugfix: inputs/analytical_db/: need dummy table.run file to cause a schema to be created for this datasource
- 06:44 PM Revision 13416: fix: inputs/input.Makefile: $(sortFile): don't print the "add any missing tables to $(sortFile)" message every time the Makefile is run
- 06:44 PM Revision 13415: bugfix: inputs/input.Makefile: install: only run this for datasource dirs
- 05:18 PM Revision 13414: inputs/input.Makefile: install: use ./run's install target for clarity
- 05:18 PM Revision 13413: lib/runscripts/import.run: added install() target
- 04:56 PM Revision 13412: bugfix: inputs/input.Makefile: install: made it idempotent (using new $(datasrc_schema_exists)) so that it could be run by `make install` on an existing system
- 04:02 PM Revision 13411: bugfix: inputs/input.Makefile: $(datasrc_schema_exists): need to use $(shell ...)
- 03:31 PM Revision 13410: inputs/input.Makefile: added $(datasrc_schema_exists)
- 03:29 PM Revision 13409: lib/runscripts/in_datasrc_dir.run: use new local.run
- 03:25 PM Revision 13408: added lib/runscripts/local.run
05/06/2014
- 03:39 PM Revision 13407: web/links/index.htm: updated to Firefox bookmarks: iPlant links: fixed broken links
- 03:34 PM Revision 13406: web/links/index.htm: updated to Firefox bookmarks: starscream links: removed favicons so they wouldn't wake the Mac up on network access whenever someone opens the links page
05/03/2014
- 05:30 PM Task #909 (New): in aggregating validations pipeline, support using input result sets from CSVs instead of only by querying the staging tables
- * allows using input query results run on the raw data rather than the preprocessed staging tables
* enables providi... - 04:15 PM Revision 13405: added validation/aggregating/bien3_validation_sequence.xlsx from Brad
- 03:17 PM Revision 13404: validation/aggregating/plots/bien3_validations_salvias_vegbien.sql: _plots_21_tnrs_output_names: removed no longer needed disabling of enable_hashjoin (the lack of an additional join now makes `SET enable_seqscan = off;` sufficient to prevent a slow hash join)
- 03:10 PM Revision 13403: bugfix: schemas/util.sql: explain2notice_msg_if_can(): don't use util.is_explainable() because the list provided by Postgres (http://www.postgresql.org/docs/9.3/static/sql-explain.html#AEN77691) excludes some query types that are in fact EXPLAIN-able
- 02:03 PM Revision 13402: added inputs/VegBank/verify/outputBien.log.url
- 02:03 PM Revision 13401: inputs/input.Makefile: add: verify/: also svn:ignore *.log
- 12:44 PM Revision 13400: added validation/aggregating/plots/VegBank/bien3_validations_salvias_db_original_adjToVegBank.txt from Mike Lee
- 01:39 AM Revision 13399: validation/aggregating/plots/bien3_validations_salvias_vegbien.sql: updated to DB
- 01:37 AM Revision 13398: schemas/public_.sql: _plots_20_tnrs_input_names, _plots_21_tnrs_output_names: added Brad's comments
- 01:28 AM Revision 13397: bugfix: schemas/util.sql: materialize_query(): contents generated from: need to include search_path so query is runnable
- 01:05 AM Revision 13396: planning/meetings/BIEN conference call availability.xlsx: updated
- 12:47 AM Revision 13395: validation/aggregating/plots/bien3_validations_salvias_vegbien.sql: _plots_21_tnrs_output_names: removed no longer needed setting of join_collapse_limit (the column now used does not involve an additional join)
05/02/2014
- 11:34 PM Revision 13394: schemas/vegbien.sql: _plots_21_tnrs_output_names: use scrubbed_unique_taxon_name (concatenated output name) because this is more similar to the concatenated name used in _plots_20_tnrs_input_names
- 11:22 PM Revision 13393: bugfix: validation/aggregating/plots/bien3_validations_salvias_vegbien.sql: _plots_21_tnrs_output_names: needs to be run with `SET join_collapse_limit = DEFAULT; SET enable_hashjoin = off;` to produce the right query plan
- 10:57 PM Revision 13392: fix: lib/util.py: dict_subset(): raise an error if collections.OrderedDict isn't available, because some callers may depend on this. note that using dict instead of OrderedDict may be the cause of the joining on the wrong columns bug (issue #902).
- 10:42 PM Revision 13391: validation/aggregating/plots/bien3_validations_salvias_vegbien.sql: updated to DB
- 10:01 PM Revision 13390: bugfix: schemas/public_.sql: _plots_21_tnrs_output_names: need to use TNRS output rather than input name
- 09:54 PM Revision 13389: schemas/public_.sql: added _plots_20_tnrs_input_names, _plots_21_tnrs_output_names
- 06:17 PM Revision 13388: schemas/public_.sql: public_validations: schema comment: documented how to run just the output queries on a particular datasource
- 04:53 PM Revision 13387: fix: schemas/public_.sql: rm_all_queries(): also remove output-only result tables
- 04:50 PM Revision 13386: schemas/public_.sql: query_view_regexp(): also match auxiliary tables
- 04:14 PM Revision 13385: schemas/public_.sql: added remake_output_tables(datasource_schema, prefix)
- 03:53 PM Revision 13384: schemas/util.sql: added loop_ignore_errors()
Also available in: Atom