Activity
From 07/21/2014 to 08/19/2014
08/19/2014
- 05:13 PM Revision 14528: fix: schemas/util.sql: contained_within_approx(point geocoord, region postgis.geography): use util.geography() instead of implicit cast to suppress "Coordinate values were coerced into range [-180 -90, 180 90] for GEOGRAPHY" NOTICEs
- 05:10 PM Revision 14527: schemas/util.sql: added geography(util.geocoord), which suppresses "Coordinate values were coerced into range [-180 -90, 180 90] for GEOGRAPHY" NOTICEs
- 04:48 PM Revision 14526: exports/native_status_resolver.csv.run: updated export_() runtime (5 min, now that we're using the narrower New World criterion)
- 04:39 PM Revision 14525: fix: schemas/public_.sql: native_status_resolver: don't include rows with New World coordinates that don't also have New World country names, since the NSR only uses the country name
- 04:26 PM Revision 14524: schemas/public_.sql: native_status_resolver: removed rows with is_geovalid NULL, at Brad's request. note that this removes valid rows with standardized country names.
- 04:01 PM Revision 14523: exports/native_status_resolver.csv.run: updated export_() runtime (30 min)
- 03:28 PM Revision 14522: fix: schemas/public_.sql: native_status_resolver: added country IS NOT NULL filter requested by Brad
- 02:35 PM Revision 14521: fix: schemas/public_.sql: native_status_resolver: remove the id because this prevents SELECT DISTINCT from having the desired effect. instead, the results will be joined back using the other columns.
- 02:25 PM Revision 14520: exports/native_status_resolver.csv.run: upload_(): documented runtime (2.5 min)
- 02:18 PM Revision 14519: bugfix: exports/native_status_resolver.csv.run: upload_(): $live must be exported
- 02:15 PM Revision 14518: exports/native_status_resolver.csv.run: upload_(): use `live=1` instead for consistency with other invocations of put
- 02:14 PM Revision 14517: fix: exports/native_status_resolver.csv.run: upload_(): need `l=1`
- 02:12 PM Revision 14516: exports/native_status_resolver.csv.run: documented export_() runtime (45 min)
- 01:48 PM Revision 14515: exports/native_status_resolver.csv.run: added upload_() to get the file onto nimoy
- 01:25 PM Revision 14514: added exports/native_status_resolver.csv.run
- 01:23 PM Revision 14513: schemas/public_.sql: added native_status_resolver view, requested by Brad (wiki.vegpath.org/Data_requests)
- 11:47 AM Revision 14512: inputs/publishable datasources.xlsx: udpated
- 11:20 AM Task #946 (New): update the column names in the VegBIEN example queries to reflect Brian M's renamings
- * the VegBIEN example queries are at *[[VegBIEN FAQ#asked-by-Brian-E-on-2014-8-8]] > asked by Brian E on 2014-8-8*
*... - 08:37 AM Revision 14511: lib/tnrs.py: documentation about output of the retrieve step: added that this is also unusable because the array does not contain all the columns and contains no column names
- 07:54 AM Revision 14510: removed no longer used web/BIEN3/Redmine/main/. use Redmine/!__ instead.
- 07:52 AM Revision 14509: web/BIEN3/Redmine/issues/.htaccess: perform .. redirects using new ! prefix
- 07:51 AM Revision 14508: web/BIEN3/Redmine/.htaccess: enable redirects that avoid using a subdir's .htaccess
- 07:46 AM Revision 14507: web/BIEN3/Redmine/wiki/.htaccess: removed no longer needed ignore_fs, since the .htaccess does not have RewriteRules that would need this in a RewriteCond
- 07:42 AM Revision 14506: web/BIEN3/Redmine/issues/.htaccess: main issues page: added default filter conditions
- 07:41 AM Revision 14505: bugfix: web/BIEN3/Redmine/issues/.htaccess: need to redirect to separate URL for individual issues, because they are not located under the issues/ subdir in Redmine
- 06:35 AM Revision 14504: added web/.issues symlink and dest dir (needed because Apache does not support dangling symlinks)
- 06:35 AM Revision 14503: web/BIEN3/Redmine/wiki/.htaccess: documented that this dir is needed because Apache does not support dangling symlinks
- 06:32 AM Revision 14502: bugfix: web/.htaccess: need to expand top-level symlinks to avoid RewriteBase issues
- 06:29 AM Revision 14501: web/main.conf: added RewriteMap for readlink
- 06:28 AM Revision 14500: added web/readlink
- 05:52 AM Revision 14499: web/links/index.htm: updated to Firefox bookmarks: updated favicons
- 05:47 AM Revision 14498: web/links/index.htm: updated to Firefox bookmarks: updated favicons
08/18/2014
- 04:52 PM Revision 14497: web/BIEN3/Redmine/wiki/.htaccess: just use this dir as symlink dest, since the dir name is the same as the URL path within Redmine
- 04:50 PM Revision 14496: web/.htaccess: don't rewrite existing files/dirs: allow forcing rewrite of existing things with %{ENV:ignore_fs}
- 03:41 PM Revision 14495: web/BIEN3/Redmine/svn-web/.htaccess: use Redmine/ instead of main/ subdir
- 03:37 PM Revision 14494: web/BIEN3/Redmine/.htaccess: point this to the Redmine root instead of to the wiki, to avoid the need to append /main
- 03:21 PM Revision 14493: backups/vegbien.r14089.backup.md5: updated
- 01:58 PM Revision 14492: inputs/.TNRS/schema.sql: taxon_match: added taxon_scrub_best_match_jerry_lu index to facilitate finding names affected by the match-picking bug (#943)
- 10:01 AM Bug #945 (New): incorrect synonym being selected in TNRS
- _this is "TNRS-195":https://pods.iplantcollaborative.org/jira/browse/TNRS-195 on the TNRS JIRA bug tracker_
TNRS's...
08/17/2014
- 12:04 PM Revision 14491: backups/TNRS.backup.md5: updated
- 07:49 AM Revision 14490: fix: inputs/HVAA/Specimen/postprocess.sql, map.csv: monthCollected/dayCollected: fix indefinite dates (which aren't supported by Postgres), as decided by Bob (https://docs.google.com/spreadsheets/d/1PI8n0CRttN7ttsXs5qfh5OFFzSoAfJj0gSbylgX6vj4/edit#gid=0)
- 07:23 AM Revision 14489: schemas/util.sql: added date_part_fix()
08/16/2014
- 05:55 PM Revision 14488: schemas/public_.sql: viewFullOccurrence_*: renamed to view_full_occurrence_* at Brian M's and Martha's request (e-mails from Martha on 2014-8-12 at 17:37PT, and from Brian M on 2014-8-13 at 16:21PT). note that this change has already been made on vegbiendev.
- 05:51 PM Revision 14487: schemas/public_.sql: added view_full_occurrence_individual_*GBIF, analytical_stem_*GBIF requested by Mark. note that this change has already been made on vegbiendev.
- 05:37 PM Revision 14486: schemas/VegBIEN/data_dictionary/VegBIEN data dictionary.xlsx: updated
- 05:15 PM Revision 14485: schemas/public_.sql: viewFullOccurrence_*: renamed to view_full_occurrence_* at Brian M's request (e-mail from Martha on 2014-8-12 at 17:37PT). note that this change has already been made on vegbiendev.
- 05:11 PM Revision 14484: schemas/public_.sql: view_full_occurrence_individual, analytical_stem: renamed columns to shortened names at Brian M's request (e-mail from Martha on 2014-8-12 at 17:37PT). note that this change has already been made on vegbiendev.
- 02:58 PM Revision 14483: schemas/public_.sql: analytical_stem_view: renamed to analytical_stem since this replaces the materialized analytical_stem table. note that this change has already been made on vegbiendev.
- 02:35 PM Revision 14482: schemas/public_.sql: viewFullOccurrence_*: renamed to view_full_occurrence_* at Brian M's request (e-mail from Martha on 2014-8-12 at 17:37PT). note that this change has already been made on vegbiendev.
- 01:20 PM Revision 14481: schemas/public_.sql: viewFullOccurrence_individual: re-ran viewFullOccurrence_individual_view_modify() to udpate this
- 01:15 PM Revision 14480: bugfix: schemas/util.sql: copy_struct(): also need to util.copy_comment() because CREATE TABLE __ INCLUDING ALL is missing this
- 01:11 PM Revision 14479: schemas/util.sql: added copy_comment(regclass, regclass)
- 12:29 PM Revision 14478: schemas/public_.sql: viewFullOccurrence_individual_view: CREATE INDEX runtime: documented 2-column runtime (~2 h)
- 12:06 PM Revision 14477: exports/analytical_stem_*GBIF.csv.run: documented runtime (40 min-1.2 h)
- 09:36 AM Revision 14476: exports/view_full_occurrence_individual_*GBIF.csv.run: documented runtime (20-30 min)
- 08:37 AM Revision 14475: added exports/analytical_stem_GBIF.csv.run, analytical_stem_non_GBIF.csv.run
08/15/2014
- 07:15 PM Revision 14474: added exports/view_full_occurrence_individual_GBIF.csv.run, view_full_occurrence_individual_non_GBIF.csv.run
- 04:13 PM Revision 14473: schemas/public_.sql: viewFullOccurrence_individual_view: added "after updating this" instructions
- 04:01 PM Revision 14472: schemas/public_.sql: viewFullOccurrence_individual_view: documented CREATE INDEX runtime (10 min - 1.5 h depending on the datatype and % populated)
- 03:32 PM Revision 14471: web/links/index.htm: updated to Firefox bookmarks: updated to use frenzy instead of starscream
08/14/2014
- 03:25 PM Revision 14470: fix: lib/tnrs.py: retrieval_request_template: source_sorting (Constrain by Source): corrected explanation to reflect that the behavior is actually the same in both modes, since only one match is ever marked as selected, and that match should always come first
- 03:03 PM Bug #943: in TNRS, the Constrain by Source option is applied incorrectly to the results download
- there is now a much faster workaround, described in the last paragraph of the issue description
- 10:50 AM Revision 14469: planning/meetings/BIEN conference call availability.xlsx: updated
08/11/2014
- 08:53 AM Bug #944 (New): fix phpPgAdmin bug that crashes the DB when browsing large tables
- use "@provider_count@":http://vegbiendev.nceas.ucsb.edu/VegBIEN/public/provider_count/:display to test this
- 07:48 AM Revision 14468: inputs/.TNRS/schema.sql: taxon_match: added taxon_scrub_by_name index
- 07:46 AM Revision 14467: inputs/.TNRS/schema.sql: taxon_match: added taxon_scrub_by_family index
- 07:10 AM Revision 14466: inputs/.TNRS/schema.sql: taxon_match: added taxon_scrub_by_species_binomial index
- 05:28 AM Revision 14465: bugfix: schemas/public_.sql: *_view_modify(): need to set the search_path so that tables are created in the same schema as the function, rather than whichever schema happens to be at the beginning of the search_path
- 05:19 AM Revision 14464: bugfix: schemas/public_.sql: analytical_stem_view_modify(): updated to support being used directly instead of via a materialized table
- 05:13 AM Revision 14463: schemas/public_.sql: removed no longer used analytical_stem. use analytical_stem_view or viewFullOccurrence_individual instead.
- 05:08 AM Revision 14462: fix: schemas/public_.sql: viewfulloccurrence_individual_view_modify(): fixed capitalization in function name (needs "" to preserve case)
- 05:03 AM Revision 14461: schemas/public_.sql: other derived views: use viewFullOccurrence_individual instead of analytical_stem because analytical_stem is no longer materialized (viewFullOccurrence_individual_view is materialized instead)
- 05:01 AM Revision 14460: schemas/public_.sql: validation views: use analytical_stem_view instead of analytical_stem because analytical_stem is no longer materialized (viewFullOccurrence_individual_view is materialized instead)
- 04:57 AM Revision 14459: schemas/public_.sql: derived views: use analytical_stem_view instead of analytical_stem because analytical_stem is no longer materialized (viewFullOccurrence_individual_view is materialized instead)
- 04:46 AM Revision 14458: bugfix: schemas/util.sql: expansion_min_cols(): increased to properly handle all public-schema views (the largest is currently 2014-6-12.Jeff_Ott.climatic_range_determinants)
- 04:35 AM Revision 14457: schemas/util.sql: view_def_to_orig(): use new expansion_min_cols() for easier configuration
- 04:33 AM Revision 14456: schemas/util.sql: added expansion_min_cols() (the minimum # of cols from the same table to be treated as a * expression)
- 04:05 AM Revision 14455: fix: schemas/public_.sql: analytical_stem_view: regenerated columns list for viewFullOccurrence_individual (instead of viewFullOccurrence_individual_view)
- 03:40 AM Revision 14454: bugfix: mappings/VegCore-VegBIEN.csv: prefixed taxonomic ranks: use _concat_nullify() so that the prefix is only added if the epithet is non-NULL
- 03:29 AM Revision 14453: fix: schemas/util.sql: col_re(): support column names with " in them
- 02:59 AM Revision 14452: schemas/util.sql: added _concat_nullify(), which uses || instead of concat()
- 02:16 AM Revision 14451: bugfix: inputs/FIA/REF_RESEARCH_STATION/map.csv: mapped country, which is not provided in the FIA data
- 01:46 AM Revision 14450: schemas/public_.sql: removed no longer used threatened_taxonlabel_view. use iucn_red_list_view instead.
- 01:35 AM Revision 14449: schemas/public_.sql: viewFullOccurrence_individual_view: documented materialize time (22 h)
08/10/2014
- 05:53 AM Revision 14448: inputs/.TNRS/schema.sql: taxon_match: removed no longer used scrubbed_unique_taxon_name. the scrubbed name ranks are now generated from the other TNRS columns instead.
- 05:28 AM Revision 14447: /README.TXT: Full database import: added steps to re-run geoscrubbing
- 05:22 AM Revision 14446: exports/geoscrub_input.csv.run: export_(): updated runtime (30 s)
- 03:57 AM Revision 14445: fix: exports/viewFullOccurrence_*.csv.run: need to limit # rows to 5000 as requested by Bob
- 03:42 AM Revision 14444: schemas/public_.sql: viewFullOccurrence_*: documented materialize time
08/09/2014
- 10:23 PM Revision 14443: bin/make_analytical_db: materialize viewFullOccurrence_individual_view instead of analytical_stem_view because analytical_stem_view is now generatable via a simple join onto viewFullOccurrence_individual_view. this avoids running into potential disk space constraints when materializing and backing up both tables (~50 GB/table * 2 tables * 2 copies (incl. the backup) = 200 GB, which is very close to the available disk space).
- 10:17 PM Revision 14442: schemas/public_.sql: analytical_stem_view: use new materialized viewFullOccurrence_individual
- 10:08 PM Revision 14441: schemas/public_.sql: added viewFullOccurrence_individual by running viewFullOccurrence_individual_view_modify()
- 10:05 PM Revision 14440: schemas/public_.sql: added viewFullOccurrence_individual_view_modify(), analogous to analytical_stem_view_modify()
- 09:52 PM Revision 14439: bin/make_analytical_db: removed extra () around psql_verbose_vegbien
- 09:51 PM Revision 14438: bin/make_analytical_db: removed no longer used mk_table()
- 09:49 PM Revision 14437: bin/make_analytical_db: use more up-to-date *_view_modify() functions instead of mk_table()
- 08:46 PM Revision 14436: inputs/.TNRS/schema.sql: removed no longer used view ValidMatchedTaxon. use taxon_scrub instead.
- 08:44 PM Revision 14435: schemas/public_.sql: iucn_red_list_view: use taxon_scrub instead of ValidMatchedTaxon since they are equivalent
- 08:42 PM Revision 14434: inputs/.TNRS/schema.sql: taxon_scrub: use taxon_best_match directly, to avoid the need for a separate ValidMatchedTaxon view
- 08:25 PM Revision 14433: fix: inputs/.TNRS/schema.sql: taxon_scrub: merged synonymous columns
- 08:11 PM Revision 14432: schemas/vegbien.sql: taxon_scrub: documented steps to merge synonymous columns
- 07:45 PM Revision 14431: inputs/.TNRS/schema.sql: removed no longer used view MatchedTaxon. use taxon_best_match instead.
- 07:43 PM Revision 14430: inputs/.TNRS/schema.sql: ValidMatchedTaxon: use taxon_best_match now that it's equivalent to MatchedTaxon
- 07:38 PM Revision 14429: bugfix: schemas/public_.sql: tnrs_input_name: use taxon_match instead of taxon_best_match because there is no index on taxon_match that includes just the filters used by taxon_best_match
- 07:37 PM Revision 14428: schemas/public_.sql: tnrs_input_name: use taxon_best_match now that it's equivalent to MatchedTaxon
- 07:20 PM Revision 14427: backups/TNRS.backup.md5: updated
- 07:19 PM Revision 14426: fix: inputs/.TNRS/schema.sql: MatchedTaxon: merged synonymous columns
- 07:02 PM Revision 14425: fix: schemas/util.sql: view_is_subset_or_renaming(): views with CASE statements are also not subsets/renamings
- 06:52 PM Revision 14424: inputs/.TNRS/schema.sql: removed no longer used taxon_scrub.scrubbed_unique_taxon_name.* . use taxon_scrub instead.
- 06:50 PM Revision 14423: inputs/.TNRS/schema.sql: taxon_scrub: use taxon_match derived columns instead of the incorrect values in taxon_scrub.scrubbed_unique_taxon_name.* (which does not work with the multi-match strategy)
- 05:54 AM Revision 14422: inputs/.TNRS/schema.sql: MatchedTaxon: use derived columns from taxon_match. this also incorporates the fixes in the new derived columns.
- 05:27 AM Revision 14421: inputs/.TNRS/schema.sql: taxon_scrub: use derived columns from taxon_match. this also incorporates the fixes in the new derived columns.
08/08/2014
- 07:14 PM Revision 14420: added exports/viewFullOccurrence_{CVS,VegBank,NCU}.csv.run
- 07:13 PM Revision 14419: schemas/public_.sql: added viewFullOccurrence_{CVS,VegBank,NCU} for Bob
- 02:32 PM Revision 14418: bugfix: schemas/util.sql: added GRANT USAGE for bien_read, public_ so that util schema functions can be called by other users
08/05/2014
08/04/2014
- 06:25 AM Revision 14416: bugfix: inputs/.TNRS/schema.sql: removed no longer used derived column __accepted_infraspecific_label, which had a buggy formula that broke derived_cols_populate()
- 06:18 AM Revision 14415: bugfix: schemas/util.sql: drop_derived_col(): also need to run util.derived_cols_trigger_update()
- 06:17 AM Revision 14414: bugfix: inputs/.TNRS/schema.sql: removed no longer used derived column __accepted_infraspecific_label, which had a buggy formula that broke derived_cols_populate()
- 06:10 AM Revision 14413: fix: inputs/.TNRS/schema.sql: taxon_match: to remove a column: updated instructions
- 05:09 AM Revision 14412: bugfix: lib/sh/util.sh: str2varname(): need to lowercase str because on case-insensitive filesystems, paths sometimes canonicalize to a different capitalization than the original
- 05:00 AM Revision 14411: lib/sh/util.sh: added lowercase()
08/03/2014
- 09:54 PM Revision 14410: bugfix: lib/sh/util.sh: die(): need stub since this is invoked before it's defined
- 09:12 PM Revision 14409: bugfix: lib/sh/util.sh: setup_log_fd(): don't change $log_fd to stdlog until stdlog is set up, to avoid "Bad file descriptor" errors
08/02/2014
- 07:58 PM Revision 14408: **: updated to use the local machine's new hostname, frenzy
- 07:13 PM Revision 14407: lib/sh/util.sh: func_override(), copy_func(): added echo_func to facilitate debugging
- 07:12 PM Revision 14406: bugfix: lib/sh/util.sh: stubs: log++ alias also needs to be moved to stub section
08/01/2014
- 06:31 PM Revision 14405: bugfix: lib/Firefox_bookmarks.reformat.csv: URLs: match only the uppercase tags used by Firefox, not any lowercase tags added by the user
- 05:42 PM Revision 14404: fix: lib/Firefox_bookmarks.reformat.csv: page's self-description: updated comment to match regexp
- 05:39 PM Revision 14403: bugfix: lib/Firefox_bookmarks.reformat.csv: page's self-description: updated "page's self-description: " prefix to remove
- 04:50 PM Revision 14402: web/links/index.htm: updated to Firefox bookmarks: added links for SSD vs. HDD reliability
- 04:22 PM Revision 14401: /README.TXT: added steps to restore from Time Machine
- 04:15 PM Revision 14400: /README.TXT: added steps to back up the local machine's hard drive
- 04:13 PM Revision 14399: /README.TXT: to synchronize a Mac's settings with my testing machine's: renamed to "to back up the local machine's settings"
- 04:11 PM Revision 14398: /README.TXT: to synchronize a Mac's settings with my testing machine's: removed download section because we don't use this
- 06:01 AM Revision 14397: added backups/vegbien.r14089.backup.md5
- 06:01 AM Revision 14396: backups/TNRS.backup.md5: updated
- 05:47 AM Revision 14395: bin/after_import: use new bin/make_backups
- 05:47 AM Revision 14394: bugfix: bin/make_backups: need to `popd` when done
- 05:46 AM Revision 14393: bugfix: bin/make_backups: need to `set +x` when done
- 05:44 AM Revision 14392: bin/make_backups: run with initial "." so background processes will be owned by the invoking shell
- 04:40 AM Revision 14391: added bin/make_backups
- 01:59 AM Revision 14390: web/links/index.htm: updated to Firefox bookmarks: VirtualBox: performance: added that context-switching also imposes an overhead, in addition to sandboxing.
- 01:16 AM Revision 14389: web/links/index.htm: updated to Firefox bookmarks: BIEN: added list of machines this has been tested on. VirtualBox: added links to performance considerations of running in a VM.
07/28/2014
- 11:58 PM Revision 14388: inputs/.TNRS/schema.sql: added new derived columns to derived views
- 11:58 PM Revision 14387: fix: schemas/util.sql: view_def_to_orig(): need to support aliased columns (produced when a column is renamed)
- 05:59 PM Revision 14386: schemas/util.sql: added aliased_col_re()
- 05:57 PM Revision 14385: schemas/util.sql: added alias_re()
- 05:50 PM Revision 14384: fix: schemas/util.sql: view_is_subset(): renamed to view_is_subset_or_renaming() because this also supports views that just rename columns, which should not be .*-ed by view_def_to_orig()
- 05:33 PM Revision 14383: fix: schemas/util.sql: view_def_to_orig(): can't use FROM to alias util.col_re() because that prevents inlining the function
- 05:31 PM Revision 14382: fix: schemas/util.sql: view_def_to_orig(): can't use FROM to alias util.col_re() because that prevents inlining the function
- 05:24 PM Revision 14381: schemas/util.sql: view_def_to_orig(): use util.col_re() for clarity
- 05:20 PM Revision 14380: schemas/util.sql: added col_re()
- 04:45 PM Revision 14379: fix: schemas/util.sql: derived_col_update(): also need steps to drop column, because DROP __ CASCADE doesn't work when there are dependent views
- 04:33 PM Revision 14378: inputs/.TNRS/schema.sql: __accepted_infraspecific_{rank,epithet}: use array slice of new __accepted_{genus,specific_epithet,infra_{rank,epithet}}, which is simpler than using remove_prefix() in __accepted_infraspecific_label
- 04:21 PM Revision 14377: inputs/.TNRS/schema.sql: "[accepted_]genus__@DwC__@vegpath.org": don't need to use *Accepted_name anymore because __accepted_{genus,specific_epithet,infra_{rank,epithet}} is now generated from *Accepted_name
- 04:02 PM Revision 14376: inputs/.TNRS/schema.sql: taxon_match."__accepted_{genus,specific_epithet}": renamed to "__accepted_{genus,specific_epithet,infra_{rank,epithet}}" since this now includes these other ranks as well
- 03:48 PM Revision 14375: bugfix: inputs/.TNRS/schema.sql: taxon_match."__accepted_{genus,specific_epithet}": use "*Accepted_name" instead of "Accepted_species[_binomial]__@TNRS__@vegpath.org" (from "*Accepted_name_species") because Accepted_name_species apparently sometimes does not match the Accepted_name and uses malformed Unicode characters
- 03:20 PM Revision 14374: backups/README.TXT: added backup retention policy
07/27/2014
- 03:19 PM Revision 14373: bugfix: schemas/util.sql: derived_cols_export(): also need to first run util.derived_cols_remove() if changing the derived column order
- 03:15 PM Revision 14372: schemas/util.sql: added derived_cols_remove(), which allows derived columns to be re-created in a different order
- 03:01 PM Revision 14371: schemas/util.sql: added drop_derived_col(), which ensures that drop_column() only cascades to views
- 02:54 PM Revision 14370: fix: schemas/util.sql: drop_column(): also need to recreate dependent views
- 02:26 PM Revision 14369: schemas/util.sql: added drop_constraint()
07/26/2014
- 10:16 PM Revision 14368: inputs/.TNRS/schema.sql: taxon_match: `inputs/.TNRS/data.sql.run refresh`: documented runtime (1 min)
- 10:15 PM Revision 14367: bugfix: inputs/.TNRS/schema.sql: taxon_match: use "Accepted_species[_binomial]__@TNRS__@vegpath.org" instead of "*Accepted_name_species". this fixes a bug in __accepted_infraspecific_label where Accepted_name_species with trailing whitespace could not be prefix-removed from names that contained just a species binomial.
- 09:57 PM Revision 14366: fix: inputs/.TNRS/schema.sql: taxon_match: added derived column "Accepted_species[_binomial]__@TNRS__@vegpath.org", which removes trailing whitespace
- 08:58 PM Revision 14365: inputs/.TNRS/schema.sql: added steps to remove a column and to add a non-derived column
- 08:48 PM Revision 14364: inputs/.TNRS/schema.sql: taxon_match: to remove columns or add columns at the end: merged into "to add a new derived column"
- 08:44 PM Revision 14363: inputs/.TNRS/schema.sql: to add columns in the middle: renamed to "to move a column to the middle" for clarity
- 08:38 PM Revision 14362: inputs/.TNRS/schema.sql: to populate a new column: updated to use util.derived_col_update()
- 08:24 PM Revision 14361: fix: inputs/.TNRS/schema.sql: taxon_match: to remove columns or add columns: also need to run util.recreate_view()
- 08:19 PM Revision 14360: inputs/.TNRS/schema.sql: taxon_match: to remove columns or add columns at the end: don't need to run `rm=1 inputs/.TNRS/data.sql.run` because this is now run by `make schemas/remake`
- 08:13 PM Revision 14359: inputs/.TNRS/schema.sql: added new derived columns to derived views
- 07:52 PM Revision 14358: schemas/util.sql: remove_prefix(), remove_suffix(): support case-insensitive matching
- 07:43 PM Revision 14357: bugfix: inputs/.TNRS/schema.sql: taxon_match.__accepted_infraspecific_label: need to use case-insensitive matching of the removed prefix because TNRS lowercases part of the Accepted_name
- 07:40 PM Revision 14356: schemas/util.sql: remove_prefix(), remove_suffix(): support case-insensitive matching
- 06:54 PM Revision 14355: schemas/util.sql: remove_prefix(), remove_suffix(): support case-insensitive matching
- 06:52 PM Revision 14354: schemas/util.sql: added case-sensitive-optional versions of starts_with(), ends_with()
- 06:50 PM Revision 14353: schemas/util.sql: added canon_case(str text, case_sensitive boolean)
- 06:36 PM Revision 14352: schemas/util.sql: check_constraint_disable(): documented that we can't use ADD CONSTRAINT NOT VALID for this because it does not disable the constraint completely, but instead causes new rows to be validated against it. the ADD CONSTRAINT NOT VALID behavior does not work for the derived columns use case of this function.
- 06:17 PM Revision 14351: schemas/public_.sql: *_view_modify(): added repopulate param, which is passed to rematerialize_view()
- 06:14 PM Revision 14350: schemas/util.sql: rematerialize_view(): support updating just the table's schema, without populating it
- 05:37 PM Revision 14349: schemas/util.sql: rematerialize_view(): don't need extra eval() around table_mod_sql because recreate() performs eval()
- 05:28 PM Revision 14348: lib/sh/sync.sh: db_snapshot(): before backing up, trim bloated temp files (eg. from rolled back changes)
- 05:16 PM Revision 14347: bugfix: inputs/.TNRS/schema.sql: taxon_match: use wrapper for util.remove_prefix() so CHECK constraints that use it don't get dropped when the util schema is reinstalled
- 04:32 PM Revision 14346: schemas/public_.sql: *_view_modify(): use util.rematerialize_view(), which now supports table mods
- 03:47 PM Revision 14345: schemas/util.sql: rematerialize_view(): added support for table mods to run after creating table
- 04:53 AM Revision 14344: bugfix: schemas/util.sql: rematerialize_view(): don't populate unless recreate() can successfully recreate the dependent views, to avoid populating the entire table (which for some views can take hours) only to have the changes rolled back by a failing dependent view
- 04:42 AM Revision 14343: schemas/util.sql: removed no longer used materialize_view(). use util.copy() instead.
- 04:41 AM Revision 14342: schemas/util.sql: rematerialize_view(): call util.copy() directly instead of via util.materialize_view()
- 04:35 AM Revision 14341: schemas/util.sql: removed no longer used copy_types_and_data(). use util.materialize_view() instead.
- 04:20 AM Revision 14340: schemas/util.sql: to_freq(): use util.copy() instead of util.copy_types_and_data() to avoid a dependency on util.copy_types_and_data()
07/25/2014
- 09:57 PM Revision 14339: fix: schemas/util.sql: copy_data(): also need to ANALYZE table after loading data into it
- 09:13 PM Revision 14338: schemas/public_.sql: analytical_stem_view: use viewFullOccurrence_individual_view to avoid duplicated code between the views. this moves the stem-related columns to the end of the view.
- 06:26 PM Revision 14337: schemas/public_.sql: added viewFullOccurrence_individual_view
- 06:20 PM Revision 14336: web/username_prefix.php: substitutions: also support "" because Postgres doubles " to escape them
- 06:17 PM Revision 14335: web/username_prefix.php: added support for substitutions ( x."_y@url -> x.x_y@url ), which can be used when a table name is repeated in a column name
- 06:15 PM Revision 14334: backups/TNRS.backup.md5: updated
- 04:09 PM Bug #943 (New): in TNRS, the Constrain by Source option is applied incorrectly to the results download
- h3. issue
from "TNRS issue 188":https://pods.iplantcollaborative.org/jira/browse/TNRS-188 (in iPlant JIRA):
> t... - 02:31 PM Revision 14333: schemas/util.sql: derived_col_constraint_initially_enabled(): clarified that this is always based on whether the existing constraint is enabled
- 02:21 PM Revision 14332: inputs/.TNRS/schema.sql: taxon_match: COMMENT: added steps to port derived column changes to vegbiendev
- 02:12 PM Revision 14331: schemas/util.sql: added derived_cols_export(), which creates SQL to port the derived cols to another DB
- 12:41 PM Revision 14330: bugfix: inputs/.TNRS/schema.sql: taxon_match: derived columns: use new "matched~Name[_no_author]___@TNRS__@vegpath.org" instead of "*Name_matched" so that "No suitable matches found." is removed before concatenating with other fields
- 12:14 PM Revision 14329: inputs/.TNRS/schema.sql: taxon_match: added derived column "matched~Name[_no_author]___@TNRS__@vegpath.org", which removes the "No suitable matches found." string
07/22/2014
- 06:50 AM Revision 14328: inputs/.TNRS/schema.sql: reordered derived columns in dependency order
- 06:46 AM Revision 14327: bugfix: inputs/.TNRS/schema.sql: "[accepted_]morphospecies[_binomial]__@Brad__.TNRS@vegpath.org": need to use "[accepted_]genus__@DwC__@vegpath.org" rather than "*Accepted_name" for this for rank = genus
- 06:44 AM Revision 14326: inputs/.TNRS/schema.sql: taxon_match: added derived column "[scrubbed_]morphospecies[_binomial]__@Brad__.TNRS@vegpath.org"
- 06:41 AM Revision 14325: bugfix: inputs/.TNRS/schema.sql: "[accepted_]genus__@DwC__@vegpath.org": need to populate this for rank = genus
- 05:47 AM Revision 14324: inputs/.TNRS/schema.sql: taxon_match: added derived column "[scrubbed_]taxonomicStatus__@DwC__@vegpath.org"
- 05:30 AM Revision 14323: bugfix: inputs/.TNRS/schema.sql: derived columns: use "Accepted_family__@TNRS__@vegpath.org" instead of "*Accepted_name_family" because "*Accepted_name_family" is sometimes missing
- 05:24 AM Revision 14322: bugfix: derived_col_constraints_enable(): also need to update col comments, in addition to table__fill_derived()
- 04:54 AM Revision 14321: schemas/util.sql: check_constraint_enabled(): made it IMMUTABLE
- 04:48 AM Revision 14320: bugfix: schemas/util.sql: derived_col_constraints_enable(): need to update table__fill_derived() w/ formatting changes from enabling constraints
- 04:40 AM Revision 14319: bugfix: schemas/util.sql: derived_col_constraints_enable(): need to update table__fill_derived() w/ formatting changes from enabling constraints
- 04:12 AM Revision 14318: fix: inputs/.TNRS/schema.sql: taxon_match: added derived column "Accepted_family__@TNRS__@vegpath.org", which is needed because "*Accepted_name_family" isn't always populated
- 04:01 AM Revision 14317: bugfix: inputs/.TNRS/schema.sql: taxon_match: to add columns in the middle: also need to run util.derived_cols_repopulate() since the dependency order has changed
- 03:57 AM Revision 14316: fix: schemas/util.sql: derived_cols_populate(): documented that this function will sometimes not repopulate columns when needed, such as after changing the column order. in these cases, it is necessary to run util.derived_cols_repopulate() manually.
- 03:53 AM Revision 14315: schemas/util.sql: derived_cols_populate(): use new derived_cols_repopulate()
- 03:51 AM Revision 14314: schemas/util.sql: added derived_cols_repopulate()
- 03:25 AM Revision 14313: fix: inputs/.TNRS/schema.sql: taxon_match: COMMENT: to add columns in the middle: also need to run util.derived_cols_update()
- 03:23 AM Revision 14312: fix: inputs/.TNRS/schema.sql: taxon_match: COMMENT: updated util.derived_cols_sync() to util.derived_cols_update()
- 03:20 AM Revision 14311: inputs/.TNRS/schema.sql: reordered derived columns in dependency order
- 03:12 AM Revision 14310: inputs/.TNRS/schema.sql: reordered derived columns in dependency order
- 02:44 AM Revision 14309: bugfix: schemas/util.sql: derived_cols_trigger_update(): need to clear derived cols first so old values won't be used in calculations
- 02:36 AM Revision 14308: inputs/.TNRS/schema.sql: derived_cols_trigger_update(): put expr on same line as var, to save space
- 02:33 AM Revision 14307: inputs/.TNRS/schema.sql: derived_cols_trigger_update(): removed lines between statements, which add too much space
- 02:27 AM Revision 14306: fix: schemas/util.sql: derived_cols_trigger_update(): don't indent expr because may have multiple lines
- 02:25 AM Revision 14305: bugfix: schemas/util.sql: derived_col_update(): set_comment(): vars can't have same name as params, which will be substituted
- 01:48 AM Revision 14304: inputs/.TNRS/schema.sql: taxon_match: added derived column "[accepted_]morphospecies[_binomial]__@Brad__.TNRS@vegpath.org"
- 01:37 AM Revision 14303: inputs/.TNRS/schema.sql: taxon_match: added derived column "[parsed_]morphospecies[_suffix]__@Brad__.morphosp@vegpath.org"
- 01:27 AM Revision 14302: added web/people/Brad_Boyle/morphosp symlink to morphospecies
- 12:37 AM Revision 14301: added derived/TNRS/Morphospecies and scope.eml* from Brad
- 12:36 AM Revision 14300: added web/people/Brad_Boyle/morphospecies redirect to Morphospecies and scope.eml.txt
- 12:35 AM Revision 14299: added web/people/Brad_Boyle/Morphospecies and scope.eml.txt symlink
- 12:34 AM Revision 14298: added derived/TNRS/Morphospecies and scope.eml* from Brad
- 12:27 AM Revision 14297: added web/people/Brad_Boyle/TNRS redirect to Modifications to procedure to scrubbing names using TNRS.docx
- 12:27 AM Revision 14296: added web/people/Brad_Boyle/Modifications to procedure to scrubbing names using TNRS.docx symlink
- 12:18 AM Revision 14295: added derived/TNRS/Modifications to procedure to scrubbing names using TNRS.docx from Brad
07/21/2014
- 11:59 PM Revision 14294: schemas/public_.sql: analytical_stem: synced to analytical_stem_view
- 11:56 PM Revision 14293: bugfix: schemas/util.sql: view_body(): need to match view defs without trailing \n
- 11:53 PM Revision 14292: bugfix: schemas/util.sql: view_body(): need to support both CREATE statements and view bodies
- 11:23 PM Revision 14291: bugfix: schemas/util.sql: raise(): use eval_raw() so that functions called by eval() (such as view_def_to_orig()) can use raise() for debugging
- 09:30 PM Revision 14290: schemas/util.sql: eval(): use new eval_raw()
- 09:29 PM Revision 14289: schemas/util.sql: added eval_raw()
- 07:52 PM Revision 14288: bugfix: schemas/util.sql: view_is_automatically_updatable(), view_is_subset(): use util.view_body() so not matching COMMENT statements, etc
- 07:35 PM Revision 14287: schemas/util.sql: added view_body()
- 07:03 PM Revision 14286: bugfix: schemas/util.sql: view_is_subset(): need to support "(-x-)" in col name
- 06:34 PM Revision 14285: schemas/util.sql: added copy_struct(from_ regtype, to_ text) and copy_data(from_ regtype, to_ text), and use these in a general-purpose copy()
- 05:44 PM Revision 14284: inputs/.TNRS/schema.sql: added new derived columns to derived views
- 05:30 PM Revision 14283: fix: schemas/util.sql: derived_col_update(): column comment: formula: put on own line to support multiline exprs
- 05:26 PM Revision 14282: bugfix: schemas/util.sql: derived_col_update(): column comment: need to use expr from DB, not as provided by user, to avoid this changing when derived_col_update() is rerun
- 05:13 PM Revision 14281: inputs/.TNRS/schema.sql: taxon_match: added derived columns scrubbed_*
- 05:03 PM Revision 14280: bugfix: schemas/util.sql: derived_col_expr_from_check_constraint(): need to handle strs with \n after IS DISTINCT FROM
- 04:34 PM Revision 14279: inputs/.TNRS/schema.sql: taxon_match: put matched-taxon derived columns before accepted-taxon derived columns
- 04:25 PM Revision 14278: inputs/.TNRS/schema.sql: taxon_match: added derived column "[matched_]species[_binomial]~(-Accepted_-)__@TNRS__@vegpath.org"
- 04:08 PM Revision 14277: inputs/.TNRS/schema.sql: _matched_has_accepted: renamed to matched_has_accepted because the leading _ did not create vertical alignment anyway, due to different quoting syntaxes
- 04:05 PM Revision 14276: schemas/util.sql: added derived_col_name_from_check_constraint()
- 03:48 PM Revision 14275: schemas/util.sql: derived_col_update(): add steps to rename column
- 03:42 PM Revision 14274: web/username_prefix.php: transposes: removed recommendation for vertical alignment because different quoting syntaxes, etc. make it difficult to have a consistent alignment anyway
- 03:34 PM Revision 14273: bugfix: schemas/util.sql: derived_col_update(): don't set_comment() until CHECK constraint exists so that we can use its canon-ed formula
- 03:24 PM Revision 14272: bugfix: schemas/util.sql: check_constraint_expr_enable(): don't remove suffix without also removing prefix. this is done by using extract_by_regexp() instead. also changed check_constraint_enabled() to use regexps to match this.
- 03:08 PM Revision 14271: bugfix: schemas/util.sql: extract_by_regexp(): needs surrounding COALESCE() in case no match
- 03:05 PM Revision 14270: schemas/util.sql: added extract_by_regexp()
- 09:02 AM Revision 14269: inputs/.TNRS/schema.sql: taxon_match: added derived column _matched_has_accepted
- 08:47 AM Revision 14268: inputs/.TNRS/schema.sql: added new derived columns to derived views
- 08:44 AM Revision 14267: inputs/.TNRS/schema.sql: taxon_match: added derived columns "[matched_]scientificName[_with_author]__@DwC__@vegpath.org", "[accepted_]scientificName[_with_author]__@DwC__@vegpath.org"
- 08:29 AM Revision 14266: inputs/.TNRS/schema.sql: taxon_match: added derived columns "[accepted_]Infraspecific_rank[_abbr]__@TNRS__@vegpath.org", "[accepted_]infraspecificEpithet__@DwC__@vegpath.org"
- 08:26 AM Revision 14265: inputs/.TNRS/schema.sql: taxon_match: added derived column "__accepted_infraspecific_{rank,epithet}"
- 08:19 AM Revision 14264: inputs/.TNRS/schema.sql: taxon_match: added derived column __accepted_infraspecific_label
- 08:12 AM Revision 14263: bugfix: schemas/util.sql: canon_sql(): need to use the right search_path
- 08:12 AM Revision 14262: bugfix: schemas/util.sql: check_constraint view: def: must use our wrapper so that the right search_path is used
- 08:08 AM Revision 14261: schemas/util.sql: added pg_get_expr() wrapper, which sets search_path to pg_temp
- 07:41 AM Revision 14260: bugfix: schemas/util.sql: check_constraint_expr_*(): updated strings for new Mac Postgres update, which was installed on reboot
- 07:40 AM Revision 14259: bugfix: schemas/util.sql: check_constraint_expr_*(): updated strings for new Mac Postgres update, which was installed on reboot
- 06:55 AM Revision 14258: bugfix: schemas/util.sql: derived_col_update(): steps to modify: also need to run util.derived_cols_populate()
- 06:54 AM Revision 14257: inputs/.TNRS/schema.sql: taxon_match: added derived columns "[accepted_]genus__@DwC__@vegpath.org", "[accepted_]specificEpithet__@DwC__@vegpath.org"
- 06:50 AM Revision 14256: bugfix: schemas/util.sql: check_constraint view: can't use consrc because this doesn't get auto-updated with column renames
- 06:37 AM Revision 14255: bugfix: schemas/util.sql: derived_col_constraints_enable(): need final `SELECT NULL::void` so don't fold away functions called in previous query
- 06:26 AM Revision 14254: inputs/.TNRS/schema.sql: taxon_match: ran derived_cols_update()
- 06:19 AM Revision 14253: bugfix: derived_col_constraint_initially_enabled(): must be both already enabled *and* unchanged to be initially enabled (r14240 was only partially correct)
- 06:08 AM Revision 14252: schemas/util.sql: derived_cols_trigger_update(): to regenerate instructions: use util.derived_cols_update(), which also performs other updates
- 06:07 AM Revision 14251: schemas/util.sql: derived_cols_sync(): renamed to derived_cols_trigger_update() for clarity
- 06:05 AM Revision 14250: schemas/util.sql: added derived_cols_update()
- 05:58 AM Revision 14249: schemas/util.sql: derived_cols_sync(): renamed to derived_cols_trigger_update() for clarity
- 05:54 AM Revision 14248: schemas/util.sql: derived_col_update(): add comment describing how to modify derived column
- 05:41 AM Revision 14247: schemas/util.sql: added quote_typed(derived_col_def)
- 05:37 AM Revision 14246: schemas/util.sql: added quote_typed(col)
- 05:14 AM Revision 14245: schemas/util.sql: derived_cols_sync(): table__fill_derived(): document in the function comment how to regenerate it
- 05:08 AM Revision 14244: schemas/util.sql: added mk_set_comment(regprocedure, text)
- 05:07 AM Revision 14243: schemas/util.sql: set_comment(): accept anything with a mk_set_comment()
- 03:39 AM Revision 14242: fix: schemas/util.sql: check_constraint_expr_disable(): need to use CASE instead of `true OR ` because OR always evaluates both sides, slowing down constraint evaluation
- 03:23 AM Revision 14241: fix: schemas/util.sql: check_constraint view: conrelid: don't cast to regclass so can use index scan
- 03:21 AM Revision 14240: bugfix: derived_col_constraint_initially_enabled(): use whether existing constraint is already enabled, not whether expr is unchanged (which does not affect the *enabled* status)
- 02:52 AM Revision 14239: schemas/util.sql: check_constraint_canon_sql(): use util.debug_print_return_value() to assist debugging
- 02:49 AM Revision 14238: bugfix: schemas/util.sql: check_constraint_replace(): need to use util.check_constraint_canon_sql() instead of util.canon_sql()
- 02:40 AM Revision 14237: schemas/util.sql: added check_constraint_canon_sql(). this must be used instead of plain util.canon_sql() because unlike queries, CHECK constraints are not simplified, just standardized.
- 02:06 AM Revision 14236: schemas/util.sql: check_constraint_replace(): support not canon-ing def.expr
- 12:23 AM Revision 14235: bugfix: schemas/util.sql: CHECK constraint exprs: need to remove the enclosing () to be consistent everywhere
Also available in: Atom