Project

General

Profile

Activity

From 07/16/2014 to 08/14/2014

08/14/2014

03:25 PM Revision 14470: fix: lib/tnrs.py: retrieval_request_template: source_sorting (Constrain by Source): corrected explanation to reflect that the behavior is actually the same in both modes, since only one match is ever marked as selected, and that match should always come first
Aaron Marcuse-Kubitza
03:03 PM Bug #943: in TNRS, the Constrain by Source option is applied incorrectly to the results download
there is now a much faster workaround, described in the last paragraph of the issue description Aaron Marcuse-Kubitza
10:50 AM Revision 14469: planning/meetings/BIEN conference call availability.xlsx: updated
Aaron Marcuse-Kubitza

08/11/2014

08:53 AM Bug #944 (New): fix phpPgAdmin bug that crashes the DB when browsing large tables
use "@provider_count@":http://vegbiendev.nceas.ucsb.edu/VegBIEN/public/provider_count/:display to test this Aaron Marcuse-Kubitza
07:48 AM Revision 14468: inputs/.TNRS/schema.sql: taxon_match: added taxon_scrub_by_name index
Aaron Marcuse-Kubitza
07:46 AM Revision 14467: inputs/.TNRS/schema.sql: taxon_match: added taxon_scrub_by_family index
Aaron Marcuse-Kubitza
07:10 AM Revision 14466: inputs/.TNRS/schema.sql: taxon_match: added taxon_scrub_by_species_binomial index
Aaron Marcuse-Kubitza
05:28 AM Revision 14465: bugfix: schemas/public_.sql: *_view_modify(): need to set the search_path so that tables are created in the same schema as the function, rather than whichever schema happens to be at the beginning of the search_path
Aaron Marcuse-Kubitza
05:19 AM Revision 14464: bugfix: schemas/public_.sql: analytical_stem_view_modify(): updated to support being used directly instead of via a materialized table
Aaron Marcuse-Kubitza
05:13 AM Revision 14463: schemas/public_.sql: removed no longer used analytical_stem. use analytical_stem_view or viewFullOccurrence_individual instead.
Aaron Marcuse-Kubitza
05:08 AM Revision 14462: fix: schemas/public_.sql: viewfulloccurrence_individual_view_modify(): fixed capitalization in function name (needs "" to preserve case)
Aaron Marcuse-Kubitza
05:03 AM Revision 14461: schemas/public_.sql: other derived views: use viewFullOccurrence_individual instead of analytical_stem because analytical_stem is no longer materialized (viewFullOccurrence_individual_view is materialized instead)
Aaron Marcuse-Kubitza
05:01 AM Revision 14460: schemas/public_.sql: validation views: use analytical_stem_view instead of analytical_stem because analytical_stem is no longer materialized (viewFullOccurrence_individual_view is materialized instead)
Aaron Marcuse-Kubitza
04:57 AM Revision 14459: schemas/public_.sql: derived views: use analytical_stem_view instead of analytical_stem because analytical_stem is no longer materialized (viewFullOccurrence_individual_view is materialized instead)
Aaron Marcuse-Kubitza
04:46 AM Revision 14458: bugfix: schemas/util.sql: expansion_min_cols(): increased to properly handle all public-schema views (the largest is currently 2014-6-12.Jeff_Ott.climatic_range_determinants)
Aaron Marcuse-Kubitza
04:35 AM Revision 14457: schemas/util.sql: view_def_to_orig(): use new expansion_min_cols() for easier configuration
Aaron Marcuse-Kubitza
04:33 AM Revision 14456: schemas/util.sql: added expansion_min_cols() (the minimum # of cols from the same table to be treated as a * expression)
Aaron Marcuse-Kubitza
04:05 AM Revision 14455: fix: schemas/public_.sql: analytical_stem_view: regenerated columns list for viewFullOccurrence_individual (instead of viewFullOccurrence_individual_view)
Aaron Marcuse-Kubitza
03:40 AM Revision 14454: bugfix: mappings/VegCore-VegBIEN.csv: prefixed taxonomic ranks: use _concat_nullify() so that the prefix is only added if the epithet is non-NULL
Aaron Marcuse-Kubitza
03:29 AM Revision 14453: fix: schemas/util.sql: col_re(): support column names with " in them
Aaron Marcuse-Kubitza
02:59 AM Revision 14452: schemas/util.sql: added _concat_nullify(), which uses || instead of concat()
Aaron Marcuse-Kubitza
02:16 AM Revision 14451: bugfix: inputs/FIA/REF_RESEARCH_STATION/map.csv: mapped country, which is not provided in the FIA data
Aaron Marcuse-Kubitza
01:46 AM Revision 14450: schemas/public_.sql: removed no longer used threatened_taxonlabel_view. use iucn_red_list_view instead.
Aaron Marcuse-Kubitza
01:35 AM Revision 14449: schemas/public_.sql: viewFullOccurrence_individual_view: documented materialize time (22 h)
Aaron Marcuse-Kubitza

08/10/2014

05:53 AM Revision 14448: inputs/.TNRS/schema.sql: taxon_match: removed no longer used scrubbed_unique_taxon_name. the scrubbed name ranks are now generated from the other TNRS columns instead.
Aaron Marcuse-Kubitza
05:28 AM Revision 14447: /README.TXT: Full database import: added steps to re-run geoscrubbing
Aaron Marcuse-Kubitza
05:22 AM Revision 14446: exports/geoscrub_input.csv.run: export_(): updated runtime (30 s)
Aaron Marcuse-Kubitza
03:57 AM Revision 14445: fix: exports/viewFullOccurrence_*.csv.run: need to limit # rows to 5000 as requested by Bob
Aaron Marcuse-Kubitza
03:42 AM Revision 14444: schemas/public_.sql: viewFullOccurrence_*: documented materialize time
Aaron Marcuse-Kubitza

08/09/2014

10:23 PM Revision 14443: bin/make_analytical_db: materialize viewFullOccurrence_individual_view instead of analytical_stem_view because analytical_stem_view is now generatable via a simple join onto viewFullOccurrence_individual_view. this avoids running into potential disk space constraints when materializing and backing up both tables (~50 GB/table * 2 tables * 2 copies (incl. the backup) = 200 GB, which is very close to the available disk space).
Aaron Marcuse-Kubitza
10:17 PM Revision 14442: schemas/public_.sql: analytical_stem_view: use new materialized viewFullOccurrence_individual
Aaron Marcuse-Kubitza
10:08 PM Revision 14441: schemas/public_.sql: added viewFullOccurrence_individual by running viewFullOccurrence_individual_view_modify()
Aaron Marcuse-Kubitza
10:05 PM Revision 14440: schemas/public_.sql: added viewFullOccurrence_individual_view_modify(), analogous to analytical_stem_view_modify()
Aaron Marcuse-Kubitza
09:52 PM Revision 14439: bin/make_analytical_db: removed extra () around psql_verbose_vegbien
Aaron Marcuse-Kubitza
09:51 PM Revision 14438: bin/make_analytical_db: removed no longer used mk_table()
Aaron Marcuse-Kubitza
09:49 PM Revision 14437: bin/make_analytical_db: use more up-to-date *_view_modify() functions instead of mk_table()
Aaron Marcuse-Kubitza
08:46 PM Revision 14436: inputs/.TNRS/schema.sql: removed no longer used view ValidMatchedTaxon. use taxon_scrub instead.
Aaron Marcuse-Kubitza
08:44 PM Revision 14435: schemas/public_.sql: iucn_red_list_view: use taxon_scrub instead of ValidMatchedTaxon since they are equivalent
Aaron Marcuse-Kubitza
08:42 PM Revision 14434: inputs/.TNRS/schema.sql: taxon_scrub: use taxon_best_match directly, to avoid the need for a separate ValidMatchedTaxon view
Aaron Marcuse-Kubitza
08:25 PM Revision 14433: fix: inputs/.TNRS/schema.sql: taxon_scrub: merged synonymous columns
Aaron Marcuse-Kubitza
08:11 PM Revision 14432: schemas/vegbien.sql: taxon_scrub: documented steps to merge synonymous columns
Aaron Marcuse-Kubitza
07:45 PM Revision 14431: inputs/.TNRS/schema.sql: removed no longer used view MatchedTaxon. use taxon_best_match instead.
Aaron Marcuse-Kubitza
07:43 PM Revision 14430: inputs/.TNRS/schema.sql: ValidMatchedTaxon: use taxon_best_match now that it's equivalent to MatchedTaxon
Aaron Marcuse-Kubitza
07:38 PM Revision 14429: bugfix: schemas/public_.sql: tnrs_input_name: use taxon_match instead of taxon_best_match because there is no index on taxon_match that includes just the filters used by taxon_best_match
Aaron Marcuse-Kubitza
07:37 PM Revision 14428: schemas/public_.sql: tnrs_input_name: use taxon_best_match now that it's equivalent to MatchedTaxon
Aaron Marcuse-Kubitza
07:20 PM Revision 14427: backups/TNRS.backup.md5: updated
Aaron Marcuse-Kubitza
07:19 PM Revision 14426: fix: inputs/.TNRS/schema.sql: MatchedTaxon: merged synonymous columns
Aaron Marcuse-Kubitza
07:02 PM Revision 14425: fix: schemas/util.sql: view_is_subset_or_renaming(): views with CASE statements are also not subsets/renamings
Aaron Marcuse-Kubitza
06:52 PM Revision 14424: inputs/.TNRS/schema.sql: removed no longer used taxon_scrub.scrubbed_unique_taxon_name.* . use taxon_scrub instead.
Aaron Marcuse-Kubitza
06:50 PM Revision 14423: inputs/.TNRS/schema.sql: taxon_scrub: use taxon_match derived columns instead of the incorrect values in taxon_scrub.scrubbed_unique_taxon_name.* (which does not work with the multi-match strategy)
Aaron Marcuse-Kubitza
05:54 AM Revision 14422: inputs/.TNRS/schema.sql: MatchedTaxon: use derived columns from taxon_match. this also incorporates the fixes in the new derived columns.
Aaron Marcuse-Kubitza
05:27 AM Revision 14421: inputs/.TNRS/schema.sql: taxon_scrub: use derived columns from taxon_match. this also incorporates the fixes in the new derived columns.
Aaron Marcuse-Kubitza

08/08/2014

07:14 PM Revision 14420: added exports/viewFullOccurrence_{CVS,VegBank,NCU}.csv.run
Aaron Marcuse-Kubitza
07:13 PM Revision 14419: schemas/public_.sql: added viewFullOccurrence_{CVS,VegBank,NCU} for Bob
Aaron Marcuse-Kubitza
02:32 PM Revision 14418: bugfix: schemas/util.sql: added GRANT USAGE for bien_read, public_ so that util schema functions can be called by other users
Aaron Marcuse-Kubitza

08/05/2014

05:31 PM Revision 14417: inputs/.TNRS/schema.sql: taxon_match: to port derived column changes to vegbiendev: derived_cols_export() code: documented runtime (6 h)
Aaron Marcuse-Kubitza

08/04/2014

06:25 AM Revision 14416: bugfix: inputs/.TNRS/schema.sql: removed no longer used derived column __accepted_infraspecific_label, which had a buggy formula that broke derived_cols_populate()
Aaron Marcuse-Kubitza
06:18 AM Revision 14415: bugfix: schemas/util.sql: drop_derived_col(): also need to run util.derived_cols_trigger_update()
Aaron Marcuse-Kubitza
06:17 AM Revision 14414: bugfix: inputs/.TNRS/schema.sql: removed no longer used derived column __accepted_infraspecific_label, which had a buggy formula that broke derived_cols_populate()
Aaron Marcuse-Kubitza
06:10 AM Revision 14413: fix: inputs/.TNRS/schema.sql: taxon_match: to remove a column: updated instructions
Aaron Marcuse-Kubitza
05:09 AM Revision 14412: bugfix: lib/sh/util.sh: str2varname(): need to lowercase str because on case-insensitive filesystems, paths sometimes canonicalize to a different capitalization than the original
Aaron Marcuse-Kubitza
05:00 AM Revision 14411: lib/sh/util.sh: added lowercase()
Aaron Marcuse-Kubitza

08/03/2014

09:54 PM Revision 14410: bugfix: lib/sh/util.sh: die(): need stub since this is invoked before it's defined
Aaron Marcuse-Kubitza
09:12 PM Revision 14409: bugfix: lib/sh/util.sh: setup_log_fd(): don't change $log_fd to stdlog until stdlog is set up, to avoid "Bad file descriptor" errors
Aaron Marcuse-Kubitza

08/02/2014

07:58 PM Revision 14408: **: updated to use the local machine's new hostname, frenzy
Aaron Marcuse-Kubitza
07:13 PM Revision 14407: lib/sh/util.sh: func_override(), copy_func(): added echo_func to facilitate debugging
Aaron Marcuse-Kubitza
07:12 PM Revision 14406: bugfix: lib/sh/util.sh: stubs: log++ alias also needs to be moved to stub section
Aaron Marcuse-Kubitza

08/01/2014

06:31 PM Revision 14405: bugfix: lib/Firefox_bookmarks.reformat.csv: URLs: match only the uppercase tags used by Firefox, not any lowercase tags added by the user
Aaron Marcuse-Kubitza
05:42 PM Revision 14404: fix: lib/Firefox_bookmarks.reformat.csv: page's self-description: updated comment to match regexp
Aaron Marcuse-Kubitza
05:39 PM Revision 14403: bugfix: lib/Firefox_bookmarks.reformat.csv: page's self-description: updated "page's self-description: " prefix to remove
Aaron Marcuse-Kubitza
04:50 PM Revision 14402: web/links/index.htm: updated to Firefox bookmarks: added links for SSD vs. HDD reliability
Aaron Marcuse-Kubitza
04:22 PM Revision 14401: /README.TXT: added steps to restore from Time Machine
Aaron Marcuse-Kubitza
04:15 PM Revision 14400: /README.TXT: added steps to back up the local machine's hard drive
Aaron Marcuse-Kubitza
04:13 PM Revision 14399: /README.TXT: to synchronize a Mac's settings with my testing machine's: renamed to "to back up the local machine's settings"
Aaron Marcuse-Kubitza
04:11 PM Revision 14398: /README.TXT: to synchronize a Mac's settings with my testing machine's: removed download section because we don't use this
Aaron Marcuse-Kubitza
06:01 AM Revision 14397: added backups/vegbien.r14089.backup.md5
Aaron Marcuse-Kubitza
06:01 AM Revision 14396: backups/TNRS.backup.md5: updated
Aaron Marcuse-Kubitza
05:47 AM Revision 14395: bin/after_import: use new bin/make_backups
Aaron Marcuse-Kubitza
05:47 AM Revision 14394: bugfix: bin/make_backups: need to `popd` when done
Aaron Marcuse-Kubitza
05:46 AM Revision 14393: bugfix: bin/make_backups: need to `set +x` when done
Aaron Marcuse-Kubitza
05:44 AM Revision 14392: bin/make_backups: run with initial "." so background processes will be owned by the invoking shell
Aaron Marcuse-Kubitza
04:40 AM Revision 14391: added bin/make_backups
Aaron Marcuse-Kubitza
01:59 AM Revision 14390: web/links/index.htm: updated to Firefox bookmarks: VirtualBox: performance: added that context-switching also imposes an overhead, in addition to sandboxing.
Aaron Marcuse-Kubitza
01:16 AM Revision 14389: web/links/index.htm: updated to Firefox bookmarks: BIEN: added list of machines this has been tested on. VirtualBox: added links to performance considerations of running in a VM.
Aaron Marcuse-Kubitza

07/28/2014

11:58 PM Revision 14388: inputs/.TNRS/schema.sql: added new derived columns to derived views
Aaron Marcuse-Kubitza
11:58 PM Revision 14387: fix: schemas/util.sql: view_def_to_orig(): need to support aliased columns (produced when a column is renamed)
Aaron Marcuse-Kubitza
05:59 PM Revision 14386: schemas/util.sql: added aliased_col_re()
Aaron Marcuse-Kubitza
05:57 PM Revision 14385: schemas/util.sql: added alias_re()
Aaron Marcuse-Kubitza
05:50 PM Revision 14384: fix: schemas/util.sql: view_is_subset(): renamed to view_is_subset_or_renaming() because this also supports views that just rename columns, which should not be .*-ed by view_def_to_orig()
Aaron Marcuse-Kubitza
05:33 PM Revision 14383: fix: schemas/util.sql: view_def_to_orig(): can't use FROM to alias util.col_re() because that prevents inlining the function
Aaron Marcuse-Kubitza
05:31 PM Revision 14382: fix: schemas/util.sql: view_def_to_orig(): can't use FROM to alias util.col_re() because that prevents inlining the function
Aaron Marcuse-Kubitza
05:24 PM Revision 14381: schemas/util.sql: view_def_to_orig(): use util.col_re() for clarity
Aaron Marcuse-Kubitza
05:20 PM Revision 14380: schemas/util.sql: added col_re()
Aaron Marcuse-Kubitza
04:45 PM Revision 14379: fix: schemas/util.sql: derived_col_update(): also need steps to drop column, because DROP __ CASCADE doesn't work when there are dependent views
Aaron Marcuse-Kubitza
04:33 PM Revision 14378: inputs/.TNRS/schema.sql: __accepted_infraspecific_{rank,epithet}: use array slice of new __accepted_{genus,specific_epithet,infra_{rank,epithet}}, which is simpler than using remove_prefix() in __accepted_infraspecific_label
Aaron Marcuse-Kubitza
04:21 PM Revision 14377: inputs/.TNRS/schema.sql: "[accepted_]genus__@DwC__@vegpath.org": don't need to use *Accepted_name anymore because __accepted_{genus,specific_epithet,infra_{rank,epithet}} is now generated from *Accepted_name
Aaron Marcuse-Kubitza
04:02 PM Revision 14376: inputs/.TNRS/schema.sql: taxon_match."__accepted_{genus,specific_epithet}": renamed to "__accepted_{genus,specific_epithet,infra_{rank,epithet}}" since this now includes these other ranks as well
Aaron Marcuse-Kubitza
03:48 PM Revision 14375: bugfix: inputs/.TNRS/schema.sql: taxon_match."__accepted_{genus,specific_epithet}": use "*Accepted_name" instead of "Accepted_species[_binomial]__@TNRS__@vegpath.org" (from "*Accepted_name_species") because Accepted_name_species apparently sometimes does not match the Accepted_name and uses malformed Unicode characters
Aaron Marcuse-Kubitza
03:20 PM Revision 14374: backups/README.TXT: added backup retention policy
Aaron Marcuse-Kubitza

07/27/2014

03:19 PM Revision 14373: bugfix: schemas/util.sql: derived_cols_export(): also need to first run util.derived_cols_remove() if changing the derived column order
Aaron Marcuse-Kubitza
03:15 PM Revision 14372: schemas/util.sql: added derived_cols_remove(), which allows derived columns to be re-created in a different order
Aaron Marcuse-Kubitza
03:01 PM Revision 14371: schemas/util.sql: added drop_derived_col(), which ensures that drop_column() only cascades to views
Aaron Marcuse-Kubitza
02:54 PM Revision 14370: fix: schemas/util.sql: drop_column(): also need to recreate dependent views
Aaron Marcuse-Kubitza
02:26 PM Revision 14369: schemas/util.sql: added drop_constraint()
Aaron Marcuse-Kubitza

07/26/2014

10:16 PM Revision 14368: inputs/.TNRS/schema.sql: taxon_match: `inputs/.TNRS/data.sql.run refresh`: documented runtime (1 min)
Aaron Marcuse-Kubitza
10:15 PM Revision 14367: bugfix: inputs/.TNRS/schema.sql: taxon_match: use "Accepted_species[_binomial]__@TNRS__@vegpath.org" instead of "*Accepted_name_species". this fixes a bug in __accepted_infraspecific_label where Accepted_name_species with trailing whitespace could not be prefix-removed from names that contained just a species binomial.
Aaron Marcuse-Kubitza
09:57 PM Revision 14366: fix: inputs/.TNRS/schema.sql: taxon_match: added derived column "Accepted_species[_binomial]__@TNRS__@vegpath.org", which removes trailing whitespace
Aaron Marcuse-Kubitza
08:58 PM Revision 14365: inputs/.TNRS/schema.sql: added steps to remove a column and to add a non-derived column
Aaron Marcuse-Kubitza
08:48 PM Revision 14364: inputs/.TNRS/schema.sql: taxon_match: to remove columns or add columns at the end: merged into "to add a new derived column"
Aaron Marcuse-Kubitza
08:44 PM Revision 14363: inputs/.TNRS/schema.sql: to add columns in the middle: renamed to "to move a column to the middle" for clarity
Aaron Marcuse-Kubitza
08:38 PM Revision 14362: inputs/.TNRS/schema.sql: to populate a new column: updated to use util.derived_col_update()
Aaron Marcuse-Kubitza
08:24 PM Revision 14361: fix: inputs/.TNRS/schema.sql: taxon_match: to remove columns or add columns: also need to run util.recreate_view()
Aaron Marcuse-Kubitza
08:19 PM Revision 14360: inputs/.TNRS/schema.sql: taxon_match: to remove columns or add columns at the end: don't need to run `rm=1 inputs/.TNRS/data.sql.run` because this is now run by `make schemas/remake`
Aaron Marcuse-Kubitza
08:13 PM Revision 14359: inputs/.TNRS/schema.sql: added new derived columns to derived views
Aaron Marcuse-Kubitza
07:52 PM Revision 14358: schemas/util.sql: remove_prefix(), remove_suffix(): support case-insensitive matching
Aaron Marcuse-Kubitza
07:43 PM Revision 14357: bugfix: inputs/.TNRS/schema.sql: taxon_match.__accepted_infraspecific_label: need to use case-insensitive matching of the removed prefix because TNRS lowercases part of the Accepted_name
Aaron Marcuse-Kubitza
07:40 PM Revision 14356: schemas/util.sql: remove_prefix(), remove_suffix(): support case-insensitive matching
Aaron Marcuse-Kubitza
06:54 PM Revision 14355: schemas/util.sql: remove_prefix(), remove_suffix(): support case-insensitive matching
Aaron Marcuse-Kubitza
06:52 PM Revision 14354: schemas/util.sql: added case-sensitive-optional versions of starts_with(), ends_with()
Aaron Marcuse-Kubitza
06:50 PM Revision 14353: schemas/util.sql: added canon_case(str text, case_sensitive boolean)
Aaron Marcuse-Kubitza
06:36 PM Revision 14352: schemas/util.sql: check_constraint_disable(): documented that we can't use ADD CONSTRAINT NOT VALID for this because it does not disable the constraint completely, but instead causes new rows to be validated against it. the ADD CONSTRAINT NOT VALID behavior does not work for the derived columns use case of this function.
Aaron Marcuse-Kubitza
06:17 PM Revision 14351: schemas/public_.sql: *_view_modify(): added repopulate param, which is passed to rematerialize_view()
Aaron Marcuse-Kubitza
06:14 PM Revision 14350: schemas/util.sql: rematerialize_view(): support updating just the table's schema, without populating it
Aaron Marcuse-Kubitza
05:37 PM Revision 14349: schemas/util.sql: rematerialize_view(): don't need extra eval() around table_mod_sql because recreate() performs eval()
Aaron Marcuse-Kubitza
05:28 PM Revision 14348: lib/sh/sync.sh: db_snapshot(): before backing up, trim bloated temp files (eg. from rolled back changes)
Aaron Marcuse-Kubitza
05:16 PM Revision 14347: bugfix: inputs/.TNRS/schema.sql: taxon_match: use wrapper for util.remove_prefix() so CHECK constraints that use it don't get dropped when the util schema is reinstalled
Aaron Marcuse-Kubitza
04:32 PM Revision 14346: schemas/public_.sql: *_view_modify(): use util.rematerialize_view(), which now supports table mods
Aaron Marcuse-Kubitza
03:47 PM Revision 14345: schemas/util.sql: rematerialize_view(): added support for table mods to run after creating table
Aaron Marcuse-Kubitza
04:53 AM Revision 14344: bugfix: schemas/util.sql: rematerialize_view(): don't populate unless recreate() can successfully recreate the dependent views, to avoid populating the entire table (which for some views can take hours) only to have the changes rolled back by a failing dependent view
Aaron Marcuse-Kubitza
04:42 AM Revision 14343: schemas/util.sql: removed no longer used materialize_view(). use util.copy() instead.
Aaron Marcuse-Kubitza
04:41 AM Revision 14342: schemas/util.sql: rematerialize_view(): call util.copy() directly instead of via util.materialize_view()
Aaron Marcuse-Kubitza
04:35 AM Revision 14341: schemas/util.sql: removed no longer used copy_types_and_data(). use util.materialize_view() instead.
Aaron Marcuse-Kubitza
04:20 AM Revision 14340: schemas/util.sql: to_freq(): use util.copy() instead of util.copy_types_and_data() to avoid a dependency on util.copy_types_and_data()
Aaron Marcuse-Kubitza

07/25/2014

09:57 PM Revision 14339: fix: schemas/util.sql: copy_data(): also need to ANALYZE table after loading data into it
Aaron Marcuse-Kubitza
09:13 PM Revision 14338: schemas/public_.sql: analytical_stem_view: use viewFullOccurrence_individual_view to avoid duplicated code between the views. this moves the stem-related columns to the end of the view.
Aaron Marcuse-Kubitza
06:26 PM Revision 14337: schemas/public_.sql: added viewFullOccurrence_individual_view
Aaron Marcuse-Kubitza
06:20 PM Revision 14336: web/username_prefix.php: substitutions: also support "" because Postgres doubles " to escape them
Aaron Marcuse-Kubitza
06:17 PM Revision 14335: web/username_prefix.php: added support for substitutions ( x."_y@url -> x.x_y@url ), which can be used when a table name is repeated in a column name
Aaron Marcuse-Kubitza
06:15 PM Revision 14334: backups/TNRS.backup.md5: updated
Aaron Marcuse-Kubitza
04:09 PM Bug #943 (New): in TNRS, the Constrain by Source option is applied incorrectly to the results download
h3. issue
from "TNRS issue 188":https://pods.iplantcollaborative.org/jira/browse/TNRS-188 (in iPlant JIRA):
> t...
Aaron Marcuse-Kubitza
02:31 PM Revision 14333: schemas/util.sql: derived_col_constraint_initially_enabled(): clarified that this is always based on whether the existing constraint is enabled
Aaron Marcuse-Kubitza
02:21 PM Revision 14332: inputs/.TNRS/schema.sql: taxon_match: COMMENT: added steps to port derived column changes to vegbiendev
Aaron Marcuse-Kubitza
02:12 PM Revision 14331: schemas/util.sql: added derived_cols_export(), which creates SQL to port the derived cols to another DB
Aaron Marcuse-Kubitza
12:41 PM Revision 14330: bugfix: inputs/.TNRS/schema.sql: taxon_match: derived columns: use new "matched~Name[_no_author]___@TNRS__@vegpath.org" instead of "*Name_matched" so that "No suitable matches found." is removed before concatenating with other fields
Aaron Marcuse-Kubitza
12:14 PM Revision 14329: inputs/.TNRS/schema.sql: taxon_match: added derived column "matched~Name[_no_author]___@TNRS__@vegpath.org", which removes the "No suitable matches found." string
Aaron Marcuse-Kubitza

07/22/2014

06:50 AM Revision 14328: inputs/.TNRS/schema.sql: reordered derived columns in dependency order
Aaron Marcuse-Kubitza
06:46 AM Revision 14327: bugfix: inputs/.TNRS/schema.sql: "[accepted_]morphospecies[_binomial]__@Brad__.TNRS@vegpath.org": need to use "[accepted_]genus__@DwC__@vegpath.org" rather than "*Accepted_name" for this for rank = genus
Aaron Marcuse-Kubitza
06:44 AM Revision 14326: inputs/.TNRS/schema.sql: taxon_match: added derived column "[scrubbed_]morphospecies[_binomial]__@Brad__.TNRS@vegpath.org"
Aaron Marcuse-Kubitza
06:41 AM Revision 14325: bugfix: inputs/.TNRS/schema.sql: "[accepted_]genus__@DwC__@vegpath.org": need to populate this for rank = genus
Aaron Marcuse-Kubitza
05:47 AM Revision 14324: inputs/.TNRS/schema.sql: taxon_match: added derived column "[scrubbed_]taxonomicStatus__@DwC__@vegpath.org"
Aaron Marcuse-Kubitza
05:30 AM Revision 14323: bugfix: inputs/.TNRS/schema.sql: derived columns: use "Accepted_family__@TNRS__@vegpath.org" instead of "*Accepted_name_family" because "*Accepted_name_family" is sometimes missing
Aaron Marcuse-Kubitza
05:24 AM Revision 14322: bugfix: derived_col_constraints_enable(): also need to update col comments, in addition to table__fill_derived()
Aaron Marcuse-Kubitza
04:54 AM Revision 14321: schemas/util.sql: check_constraint_enabled(): made it IMMUTABLE
Aaron Marcuse-Kubitza
04:48 AM Revision 14320: bugfix: schemas/util.sql: derived_col_constraints_enable(): need to update table__fill_derived() w/ formatting changes from enabling constraints
Aaron Marcuse-Kubitza
04:40 AM Revision 14319: bugfix: schemas/util.sql: derived_col_constraints_enable(): need to update table__fill_derived() w/ formatting changes from enabling constraints
Aaron Marcuse-Kubitza
04:12 AM Revision 14318: fix: inputs/.TNRS/schema.sql: taxon_match: added derived column "Accepted_family__@TNRS__@vegpath.org", which is needed because "*Accepted_name_family" isn't always populated
Aaron Marcuse-Kubitza
04:01 AM Revision 14317: bugfix: inputs/.TNRS/schema.sql: taxon_match: to add columns in the middle: also need to run util.derived_cols_repopulate() since the dependency order has changed
Aaron Marcuse-Kubitza
03:57 AM Revision 14316: fix: schemas/util.sql: derived_cols_populate(): documented that this function will sometimes not repopulate columns when needed, such as after changing the column order. in these cases, it is necessary to run util.derived_cols_repopulate() manually.
Aaron Marcuse-Kubitza
03:53 AM Revision 14315: schemas/util.sql: derived_cols_populate(): use new derived_cols_repopulate()
Aaron Marcuse-Kubitza
03:51 AM Revision 14314: schemas/util.sql: added derived_cols_repopulate()
Aaron Marcuse-Kubitza
03:25 AM Revision 14313: fix: inputs/.TNRS/schema.sql: taxon_match: COMMENT: to add columns in the middle: also need to run util.derived_cols_update()
Aaron Marcuse-Kubitza
03:23 AM Revision 14312: fix: inputs/.TNRS/schema.sql: taxon_match: COMMENT: updated util.derived_cols_sync() to util.derived_cols_update()
Aaron Marcuse-Kubitza
03:20 AM Revision 14311: inputs/.TNRS/schema.sql: reordered derived columns in dependency order
Aaron Marcuse-Kubitza
03:12 AM Revision 14310: inputs/.TNRS/schema.sql: reordered derived columns in dependency order
Aaron Marcuse-Kubitza
02:44 AM Revision 14309: bugfix: schemas/util.sql: derived_cols_trigger_update(): need to clear derived cols first so old values won't be used in calculations
Aaron Marcuse-Kubitza
02:36 AM Revision 14308: inputs/.TNRS/schema.sql: derived_cols_trigger_update(): put expr on same line as var, to save space
Aaron Marcuse-Kubitza
02:33 AM Revision 14307: inputs/.TNRS/schema.sql: derived_cols_trigger_update(): removed lines between statements, which add too much space
Aaron Marcuse-Kubitza
02:27 AM Revision 14306: fix: schemas/util.sql: derived_cols_trigger_update(): don't indent expr because may have multiple lines
Aaron Marcuse-Kubitza
02:25 AM Revision 14305: bugfix: schemas/util.sql: derived_col_update(): set_comment(): vars can't have same name as params, which will be substituted
Aaron Marcuse-Kubitza
01:48 AM Revision 14304: inputs/.TNRS/schema.sql: taxon_match: added derived column "[accepted_]morphospecies[_binomial]__@Brad__.TNRS@vegpath.org"
Aaron Marcuse-Kubitza
01:37 AM Revision 14303: inputs/.TNRS/schema.sql: taxon_match: added derived column "[parsed_]morphospecies[_suffix]__@Brad__.morphosp@vegpath.org"
Aaron Marcuse-Kubitza
01:27 AM Revision 14302: added web/people/Brad_Boyle/morphosp symlink to morphospecies
Aaron Marcuse-Kubitza
12:37 AM Revision 14301: added derived/TNRS/Morphospecies and scope.eml* from Brad
Aaron Marcuse-Kubitza
12:36 AM Revision 14300: added web/people/Brad_Boyle/morphospecies redirect to Morphospecies and scope.eml.txt
Aaron Marcuse-Kubitza
12:35 AM Revision 14299: added web/people/Brad_Boyle/Morphospecies and scope.eml.txt symlink
Aaron Marcuse-Kubitza
12:34 AM Revision 14298: added derived/TNRS/Morphospecies and scope.eml* from Brad
Aaron Marcuse-Kubitza
12:27 AM Revision 14297: added web/people/Brad_Boyle/TNRS redirect to Modifications to procedure to scrubbing names using TNRS.docx
Aaron Marcuse-Kubitza
12:27 AM Revision 14296: added web/people/Brad_Boyle/Modifications to procedure to scrubbing names using TNRS.docx symlink
Aaron Marcuse-Kubitza
12:18 AM Revision 14295: added derived/TNRS/Modifications to procedure to scrubbing names using TNRS.docx from Brad
Aaron Marcuse-Kubitza

07/21/2014

11:59 PM Revision 14294: schemas/public_.sql: analytical_stem: synced to analytical_stem_view
Aaron Marcuse-Kubitza
11:56 PM Revision 14293: bugfix: schemas/util.sql: view_body(): need to match view defs without trailing \n
Aaron Marcuse-Kubitza
11:53 PM Revision 14292: bugfix: schemas/util.sql: view_body(): need to support both CREATE statements and view bodies
Aaron Marcuse-Kubitza
11:23 PM Revision 14291: bugfix: schemas/util.sql: raise(): use eval_raw() so that functions called by eval() (such as view_def_to_orig()) can use raise() for debugging
Aaron Marcuse-Kubitza
09:30 PM Revision 14290: schemas/util.sql: eval(): use new eval_raw()
Aaron Marcuse-Kubitza
09:29 PM Revision 14289: schemas/util.sql: added eval_raw()
Aaron Marcuse-Kubitza
07:52 PM Revision 14288: bugfix: schemas/util.sql: view_is_automatically_updatable(), view_is_subset(): use util.view_body() so not matching COMMENT statements, etc
Aaron Marcuse-Kubitza
07:35 PM Revision 14287: schemas/util.sql: added view_body()
Aaron Marcuse-Kubitza
07:03 PM Revision 14286: bugfix: schemas/util.sql: view_is_subset(): need to support "(-x-)" in col name
Aaron Marcuse-Kubitza
06:34 PM Revision 14285: schemas/util.sql: added copy_struct(from_ regtype, to_ text) and copy_data(from_ regtype, to_ text), and use these in a general-purpose copy()
Aaron Marcuse-Kubitza
05:44 PM Revision 14284: inputs/.TNRS/schema.sql: added new derived columns to derived views
Aaron Marcuse-Kubitza
05:30 PM Revision 14283: fix: schemas/util.sql: derived_col_update(): column comment: formula: put on own line to support multiline exprs
Aaron Marcuse-Kubitza
05:26 PM Revision 14282: bugfix: schemas/util.sql: derived_col_update(): column comment: need to use expr from DB, not as provided by user, to avoid this changing when derived_col_update() is rerun
Aaron Marcuse-Kubitza
05:13 PM Revision 14281: inputs/.TNRS/schema.sql: taxon_match: added derived columns scrubbed_*
Aaron Marcuse-Kubitza
05:03 PM Revision 14280: bugfix: schemas/util.sql: derived_col_expr_from_check_constraint(): need to handle strs with \n after IS DISTINCT FROM
Aaron Marcuse-Kubitza
04:34 PM Revision 14279: inputs/.TNRS/schema.sql: taxon_match: put matched-taxon derived columns before accepted-taxon derived columns
Aaron Marcuse-Kubitza
04:25 PM Revision 14278: inputs/.TNRS/schema.sql: taxon_match: added derived column "[matched_]species[_binomial]~(-Accepted_-)__@TNRS__@vegpath.org"
Aaron Marcuse-Kubitza
04:08 PM Revision 14277: inputs/.TNRS/schema.sql: _matched_has_accepted: renamed to matched_has_accepted because the leading _ did not create vertical alignment anyway, due to different quoting syntaxes
Aaron Marcuse-Kubitza
04:05 PM Revision 14276: schemas/util.sql: added derived_col_name_from_check_constraint()
Aaron Marcuse-Kubitza
03:48 PM Revision 14275: schemas/util.sql: derived_col_update(): add steps to rename column
Aaron Marcuse-Kubitza
03:42 PM Revision 14274: web/username_prefix.php: transposes: removed recommendation for vertical alignment because different quoting syntaxes, etc. make it difficult to have a consistent alignment anyway
Aaron Marcuse-Kubitza
03:34 PM Revision 14273: bugfix: schemas/util.sql: derived_col_update(): don't set_comment() until CHECK constraint exists so that we can use its canon-ed formula
Aaron Marcuse-Kubitza
03:24 PM Revision 14272: bugfix: schemas/util.sql: check_constraint_expr_enable(): don't remove suffix without also removing prefix. this is done by using extract_by_regexp() instead. also changed check_constraint_enabled() to use regexps to match this.
Aaron Marcuse-Kubitza
03:08 PM Revision 14271: bugfix: schemas/util.sql: extract_by_regexp(): needs surrounding COALESCE() in case no match
Aaron Marcuse-Kubitza
03:05 PM Revision 14270: schemas/util.sql: added extract_by_regexp()
Aaron Marcuse-Kubitza
09:02 AM Revision 14269: inputs/.TNRS/schema.sql: taxon_match: added derived column _matched_has_accepted
Aaron Marcuse-Kubitza
08:47 AM Revision 14268: inputs/.TNRS/schema.sql: added new derived columns to derived views
Aaron Marcuse-Kubitza
08:44 AM Revision 14267: inputs/.TNRS/schema.sql: taxon_match: added derived columns "[matched_]scientificName[_with_author]__@DwC__@vegpath.org", "[accepted_]scientificName[_with_author]__@DwC__@vegpath.org"
Aaron Marcuse-Kubitza
08:29 AM Revision 14266: inputs/.TNRS/schema.sql: taxon_match: added derived columns "[accepted_]Infraspecific_rank[_abbr]__@TNRS__@vegpath.org", "[accepted_]infraspecificEpithet__@DwC__@vegpath.org"
Aaron Marcuse-Kubitza
08:26 AM Revision 14265: inputs/.TNRS/schema.sql: taxon_match: added derived column "__accepted_infraspecific_{rank,epithet}"
Aaron Marcuse-Kubitza
08:19 AM Revision 14264: inputs/.TNRS/schema.sql: taxon_match: added derived column __accepted_infraspecific_label
Aaron Marcuse-Kubitza
08:12 AM Revision 14263: bugfix: schemas/util.sql: canon_sql(): need to use the right search_path
Aaron Marcuse-Kubitza
08:12 AM Revision 14262: bugfix: schemas/util.sql: check_constraint view: def: must use our wrapper so that the right search_path is used
Aaron Marcuse-Kubitza
08:08 AM Revision 14261: schemas/util.sql: added pg_get_expr() wrapper, which sets search_path to pg_temp
Aaron Marcuse-Kubitza
07:41 AM Revision 14260: bugfix: schemas/util.sql: check_constraint_expr_*(): updated strings for new Mac Postgres update, which was installed on reboot
Aaron Marcuse-Kubitza
07:40 AM Revision 14259: bugfix: schemas/util.sql: check_constraint_expr_*(): updated strings for new Mac Postgres update, which was installed on reboot
Aaron Marcuse-Kubitza
06:55 AM Revision 14258: bugfix: schemas/util.sql: derived_col_update(): steps to modify: also need to run util.derived_cols_populate()
Aaron Marcuse-Kubitza
06:54 AM Revision 14257: inputs/.TNRS/schema.sql: taxon_match: added derived columns "[accepted_]genus__@DwC__@vegpath.org", "[accepted_]specificEpithet__@DwC__@vegpath.org"
Aaron Marcuse-Kubitza
06:50 AM Revision 14256: bugfix: schemas/util.sql: check_constraint view: can't use consrc because this doesn't get auto-updated with column renames
Aaron Marcuse-Kubitza
06:37 AM Revision 14255: bugfix: schemas/util.sql: derived_col_constraints_enable(): need final `SELECT NULL::void` so don't fold away functions called in previous query
Aaron Marcuse-Kubitza
06:26 AM Revision 14254: inputs/.TNRS/schema.sql: taxon_match: ran derived_cols_update()
Aaron Marcuse-Kubitza
06:19 AM Revision 14253: bugfix: derived_col_constraint_initially_enabled(): must be both already enabled *and* unchanged to be initially enabled (r14240 was only partially correct)
Aaron Marcuse-Kubitza
06:08 AM Revision 14252: schemas/util.sql: derived_cols_trigger_update(): to regenerate instructions: use util.derived_cols_update(), which also performs other updates
Aaron Marcuse-Kubitza
06:07 AM Revision 14251: schemas/util.sql: derived_cols_sync(): renamed to derived_cols_trigger_update() for clarity
Aaron Marcuse-Kubitza
06:05 AM Revision 14250: schemas/util.sql: added derived_cols_update()
Aaron Marcuse-Kubitza
05:58 AM Revision 14249: schemas/util.sql: derived_cols_sync(): renamed to derived_cols_trigger_update() for clarity
Aaron Marcuse-Kubitza
05:54 AM Revision 14248: schemas/util.sql: derived_col_update(): add comment describing how to modify derived column
Aaron Marcuse-Kubitza
05:41 AM Revision 14247: schemas/util.sql: added quote_typed(derived_col_def)
Aaron Marcuse-Kubitza
05:37 AM Revision 14246: schemas/util.sql: added quote_typed(col)
Aaron Marcuse-Kubitza
05:14 AM Revision 14245: schemas/util.sql: derived_cols_sync(): table__fill_derived(): document in the function comment how to regenerate it
Aaron Marcuse-Kubitza
05:08 AM Revision 14244: schemas/util.sql: added mk_set_comment(regprocedure, text)
Aaron Marcuse-Kubitza
05:07 AM Revision 14243: schemas/util.sql: set_comment(): accept anything with a mk_set_comment()
Aaron Marcuse-Kubitza
03:39 AM Revision 14242: fix: schemas/util.sql: check_constraint_expr_disable(): need to use CASE instead of `true OR ` because OR always evaluates both sides, slowing down constraint evaluation
Aaron Marcuse-Kubitza
03:23 AM Revision 14241: fix: schemas/util.sql: check_constraint view: conrelid: don't cast to regclass so can use index scan
Aaron Marcuse-Kubitza
03:21 AM Revision 14240: bugfix: derived_col_constraint_initially_enabled(): use whether existing constraint is already enabled, not whether expr is unchanged (which does not affect the *enabled* status)
Aaron Marcuse-Kubitza
02:52 AM Revision 14239: schemas/util.sql: check_constraint_canon_sql(): use util.debug_print_return_value() to assist debugging
Aaron Marcuse-Kubitza
02:49 AM Revision 14238: bugfix: schemas/util.sql: check_constraint_replace(): need to use util.check_constraint_canon_sql() instead of util.canon_sql()
Aaron Marcuse-Kubitza
02:40 AM Revision 14237: schemas/util.sql: added check_constraint_canon_sql(). this must be used instead of plain util.canon_sql() because unlike queries, CHECK constraints are not simplified, just standardized.
Aaron Marcuse-Kubitza
02:06 AM Revision 14236: schemas/util.sql: check_constraint_replace(): support not canon-ing def.expr
Aaron Marcuse-Kubitza
12:23 AM Revision 14235: bugfix: schemas/util.sql: CHECK constraint exprs: need to remove the enclosing () to be consistent everywhere
Aaron Marcuse-Kubitza

07/20/2014

11:59 PM Revision 14234: bugfix: schemas/util.sql: canon_sql(): need to include the enclosing () because the DB includes them
Aaron Marcuse-Kubitza
11:42 PM Revision 14233: fix: schemas/util.sql: canon_sql(): use explain()'s verbose mode to facilitate debugging
Aaron Marcuse-Kubitza
11:38 PM Revision 14232: bugfix: schemas/util.sql: explain(): need to restore NOTICEs in verbose mode
Aaron Marcuse-Kubitza
11:22 PM Revision 14231: schemas/util.sql: explain(): support running in verbose mode
Aaron Marcuse-Kubitza
11:19 PM Revision 14230: bugfix: schemas/util.sql: check_constraint_replace(): need to canon the expr, using new util.canon_sql(), so it can be compared with the expr in the DB
Aaron Marcuse-Kubitza
11:18 PM Revision 14229: schemas/util.sql: added canon_sql(expr text, table_ regclass)
Aaron Marcuse-Kubitza
10:45 PM Revision 14228: fix: schemas/util.sql: check_constraint view: *don't* use pretty-print mode, because this format does not match util.canon_sql()
Aaron Marcuse-Kubitza
10:43 PM Revision 14227: fix: schemas/util.sql: check_constraint_expr_enable(), derived_col_expr_from_check_constraint(): trim parens in order to support non-pretty-print mode, which is needed with canon_sql()
Aaron Marcuse-Kubitza
09:27 PM Revision 14226: schemas/util.sql: added canon_sql(expr text)
Aaron Marcuse-Kubitza
09:21 PM Revision 14225: schemas/util.sql: added trim_parens()
Aaron Marcuse-Kubitza
08:19 PM Revision 14224: schemas/util.sql: added copy_struct(from_ regtype, to_ text) and copy_data(from_ regtype, to_ text), and use these in a general-purpose copy()
Aaron Marcuse-Kubitza
08:02 PM Revision 14223: bugfix: schemas/util.sql: copy(from_ regtype, to_ text): must be declared STRICT to handle NULL properly
Aaron Marcuse-Kubitza
07:58 PM Revision 14222: schemas/util.sql: added copy(from_ regtype, to_ text)
Aaron Marcuse-Kubitza
07:39 PM Revision 14221: fix: schemas/util.sql: is_raise_condition(): should be called is_log_level()
Aaron Marcuse-Kubitza
07:36 PM Revision 14220: schemas/util.sql: raise(): support SQLSTATE values
Aaron Marcuse-Kubitza
07:33 PM Revision 14219: bugfix: schemas/util.sql: raise_expr(raise_type): only treat as raise condition if not SQLSTATE, to allow custom SQLSTATEs
Aaron Marcuse-Kubitza
07:30 PM Revision 14218: schemas/util.sql: raise(): support SQLSTATE values
Aaron Marcuse-Kubitza
07:28 PM Revision 14217: schemas/util.sql: added raise_expr(raise_type)
Aaron Marcuse-Kubitza
07:28 PM Revision 14216: schemas/util.sql: added is_raise_condition()
Aaron Marcuse-Kubitza
07:14 PM Revision 14215: schemas/util.sql: added is_sqlstate()
Aaron Marcuse-Kubitza
06:44 PM Revision 14214: schemas/util.sql: typeof(expr text, table_ regtype): use simpler util.eval2val() instead of EXECUTE
Aaron Marcuse-Kubitza
06:13 PM Revision 14213: fix: schemas/util.sql: derived_col_constraint_initially_enabled(): also should be initially enabled if expr unchanged, to avoid re-running constraint on all rows
Aaron Marcuse-Kubitza
06:09 PM Revision 14212: schemas/util.sql: derived_col_update(): use new util.derived_col_constraint_initially_enabled()
Aaron Marcuse-Kubitza
06:09 PM Revision 14211: schemas/util.sql: added derived_col_constraint_initially_enabled()
Aaron Marcuse-Kubitza
06:05 PM Revision 14210: schemas/util.sql: added derived_col_expr(col)
Aaron Marcuse-Kubitza
05:56 PM Revision 14209: derived_col_constraint_enable(): documented that this now avoids re-running constraint on all rows if already enabled
Aaron Marcuse-Kubitza
05:43 PM Revision 14208: fix: check_constraint_replace(): avoid re-running the constraint on all rows if expr hasn't changed
Aaron Marcuse-Kubitza
05:40 PM Revision 14207: schemas/util.sql: check_constraint_replace(): shortened param to `def` for readability
Aaron Marcuse-Kubitza
05:39 PM Revision 14206: schemas/util.sql: added check_constraint_expr(constraint_ table_item)
Aaron Marcuse-Kubitza
04:34 PM Revision 14205: schemas/util.sql: added derived_col_update()
Aaron Marcuse-Kubitza
04:31 PM Revision 14204: schemas/util.sql: check_constraint__disabled(): made disabled configurable (renaming func to check_constraint())
Aaron Marcuse-Kubitza
04:20 PM Revision 14203: schemas/util.sql: added derived_col_constraint_expr()
Aaron Marcuse-Kubitza
04:07 PM Revision 14202: schemas/util.sql: derived_col_constraints_enable(): use new derived_col_constraint_enable()
Aaron Marcuse-Kubitza
04:06 PM Revision 14201: schemas/util.sql: added derived_col_constraint_enable()
Aaron Marcuse-Kubitza
03:52 PM Revision 14200: schemas/util.sql: added table_has_data()
Aaron Marcuse-Kubitza
03:47 PM Revision 14199: schemas/util.sql: added col_add(derived_col_def)
Aaron Marcuse-Kubitza
03:44 PM Revision 14198: schemas/util.sql: added typeof(text, regclass)
Aaron Marcuse-Kubitza
03:40 PM Revision 14197: bugfix: /README.TXT: Full database import: added warnings that you should not run backups/pg_snapshot while the import is running, nor until the previous import has been replaced
Aaron Marcuse-Kubitza
03:22 PM Revision 14196: schemas/util.sql: added regtype(regclass)
Aaron Marcuse-Kubitza
02:52 PM Revision 14195: schemas/util.sql: added col_add()
Aaron Marcuse-Kubitza
02:32 PM Revision 14194: schemas/util.sql: added col_def type
Aaron Marcuse-Kubitza
07:22 AM Revision 14193: schemas/util.sql: derived_cols_populate(): only run if needed
Aaron Marcuse-Kubitza
07:09 AM Revision 14192: schemas/util.sql: added derived_cols_populated()
Aaron Marcuse-Kubitza
07:09 AM Revision 14191: schemas/util.sql: added derived_cols_populated()
Aaron Marcuse-Kubitza
07:08 AM Revision 14190: schemas/util.sql: added derived_col_populated()
Aaron Marcuse-Kubitza
07:07 AM Revision 14189: schemas/util.sql: added check_constraint_enabled()
Aaron Marcuse-Kubitza
06:46 AM Revision 14188: schemas/util.sql: remove_prefix(), remove_suffix(): use new starts_with(), ends_with()
Aaron Marcuse-Kubitza
06:45 AM Revision 14187: schemas/util.sql: added starts_with(), ends_with()
Aaron Marcuse-Kubitza
05:57 AM Revision 14186: schemas/util.sql: derived_cols_populate(): also run util.derived_col_constraints_enable() once populated
Aaron Marcuse-Kubitza
05:55 AM Revision 14185: schemas/util.sql: added derived_col_constraints_enable()
Aaron Marcuse-Kubitza
05:28 AM Revision 14184: schemas/util.sql: added check_constraint__disabled(), which creates a CHECK constraint which is initially disabled
Aaron Marcuse-Kubitza
05:27 AM Revision 14183: schemas/util.sql: added check_constraint__disabled(), which creates a CHECK constraint which is initially disabled
Aaron Marcuse-Kubitza
05:07 AM Revision 14182: fix: schemas/util.sql: run_triggers(): need to re-sort rows if table needs this
Aaron Marcuse-Kubitza
05:05 AM Revision 14181: schemas/util.sql: added try_cluster()
Aaron Marcuse-Kubitza
04:05 AM Revision 14180: schemas/util.sql: added derived_cols_populate()
Aaron Marcuse-Kubitza
04:04 AM Revision 14179: schemas/util.sql: added run_triggers(regclass)
Aaron Marcuse-Kubitza
04:00 AM Revision 14178: schemas/util.sql: added first_col(regclass)
Aaron Marcuse-Kubitza
02:42 AM Revision 14177: fix: schemas/util.sql: read-only functions should be STABLE rather than VOLATILE
Aaron Marcuse-Kubitza
02:42 AM Revision 14176: bugfix: schemas/util.sql: functions that access the DB must be STABLE rather than IMMUTABLE
Aaron Marcuse-Kubitza
02:23 AM Revision 14175: bugfix: schemas/util.sql: derived_col_defs(): derived columns *must* be returned in table order (which should = dependency order), so that they are populated in dependency order
Aaron Marcuse-Kubitza
02:17 AM Revision 14174: bugfix: schemas/util.sql: remove_prefix(), remove_suffix(): must be declared STRICT to handle NULL properly
Aaron Marcuse-Kubitza
02:16 AM Revision 14173: bugfix: schemas/util.sql: remove_suffix(): need to use quote_nullable(str) so a NULL str doesn't NULL out the entire message
Aaron Marcuse-Kubitza
02:10 AM Revision 14172: bugfix: schemas/util.sql: remove_prefix(): need to use quote_nullable(str) so a NULL str doesn't NULL out the entire message
Aaron Marcuse-Kubitza
02:09 AM Revision 14171: bugfix: schemas/util.sql: raise(): use new quote_as_str() to avoid "RAISE statement option cannot be null" errors
Aaron Marcuse-Kubitza
02:08 AM Revision 14170: schemas/util.sql: added quote_as_str()
Aaron Marcuse-Kubitza

07/19/2014

11:23 PM Revision 14169: schemas/util.sql: added CAST (util.col AS util.derived_col_def)
Aaron Marcuse-Kubitza
11:05 PM Revision 14168: schemas/util.sql: derived_col_def(col): use existing util.check_constraint_def()
Aaron Marcuse-Kubitza
11:01 PM Revision 14167: schemas/util.sql: added CAST (util.col AS util.table_item)
Aaron Marcuse-Kubitza
10:50 PM Revision 14166: schemas/util.sql: added derived_col_def(col)
Aaron Marcuse-Kubitza
10:30 PM Revision 14165: schemas/util.sql: added check_constraint_disable()
Aaron Marcuse-Kubitza
10:23 PM Revision 14164: bugfix: schemas/util.sql: check_constraint_drop(): need IF EXISTS
Aaron Marcuse-Kubitza
10:09 PM Revision 14163: schemas/util.sql: added check_constraint_expr_disable()
Aaron Marcuse-Kubitza
09:50 PM Revision 14162: schemas/util.sql: check_constraint_enable(): documented that this is idempotent
Aaron Marcuse-Kubitza
09:21 PM Revision 14161: inputs/.TNRS/schema.sql: taxon_match: documented that whenever columns are renamed, util.derived_cols_sync() should be run
Aaron Marcuse-Kubitza
09:14 PM Revision 14160: schemas/util.sql: derived_cols_sync(): also create trigger, since this is necessary for a complete sync
Aaron Marcuse-Kubitza
09:13 PM Revision 14159: schemas/util.sql: added name(regprocedure)
Aaron Marcuse-Kubitza
09:02 PM Revision 14158: schemas/util.sql: derived_cols_sync(): set the trigger name in a var
Aaron Marcuse-Kubitza
08:52 PM Revision 14157: schemas/util.sql: added check_constraint_enable()
Aaron Marcuse-Kubitza
08:46 PM Revision 14156: schemas/util.sql: added check_constraint_replace()
Aaron Marcuse-Kubitza
08:43 PM Revision 14155: schemas/util.sql: added CAST (util.table_item AS util.check_constraint_def)
Aaron Marcuse-Kubitza
08:38 PM Revision 14154: schemas/util.sql: added check_constraint_drop()
Aaron Marcuse-Kubitza
08:02 PM Revision 14153: schemas/util.sql: derived_col_expr_from_check_constraint(): enable initially-disabled exprs
Aaron Marcuse-Kubitza
08:00 PM Revision 14152: schemas/util.sql: added check_constraint_expr_enable()
Aaron Marcuse-Kubitza
07:42 PM Revision 14151: fix: schemas/util.sql: check_constraint view: def: use pg_get_expr() with pretty-print, to avoid extra () that complicate parsing
Aaron Marcuse-Kubitza
06:54 PM Revision 14150: inputs/.TNRS/schema.sql: taxon_match__fill_derived(): updated using new util.derived_cols_sync()
Aaron Marcuse-Kubitza
06:50 PM Revision 14149: schemas/util.sql: added derived_cols_sync()
Aaron Marcuse-Kubitza
06:45 PM Revision 14148: bugfix: CAST (util.check_constraint_def AS util.derived_col_def): need to extract the derived column expr *from* the CHECK constraint expr
Aaron Marcuse-Kubitza
06:44 PM Revision 14147: schemas/util.sql: added derived_col_expr_from_check_constraint()
Aaron Marcuse-Kubitza
06:39 PM Revision 14146: schemas/util.sql: added extract_str()
Aaron Marcuse-Kubitza
06:34 PM Revision 14145: schemas/util.sql: added remove_suffix()
Aaron Marcuse-Kubitza
06:21 PM Revision 14144: schemas/util.sql: added remove_prefix()
Aaron Marcuse-Kubitza
05:47 PM Revision 14143: schemas/util.sql: added CAST (util.table_item AS util.col)
Aaron Marcuse-Kubitza
05:37 PM Revision 14142: schemas/util.sql: derived_col_defs(): return derived_col_def instead of check_constraint_def
Aaron Marcuse-Kubitza
05:36 PM Revision 14141: schemas/util.sql: added derived_col_def type with cast from util.check_constraint_def
Aaron Marcuse-Kubitza
04:51 PM Revision 14140: schemas/util.sql: added derived_col_defs()
Aaron Marcuse-Kubitza
04:35 PM Revision 14139: schemas/util.sql: check_constraint_defs(): use new util.check_constraint view
Aaron Marcuse-Kubitza
04:30 PM Revision 14138: schemas/util.sql: added check_constraint view
Aaron Marcuse-Kubitza
03:53 PM Revision 14137: schemas/util.sql: added check_constraint_defs()
Aaron Marcuse-Kubitza
03:53 PM Revision 14136: schemas/util.sql: added check_constraint_def type
Aaron Marcuse-Kubitza
03:51 PM Revision 14135: schemas/util.sql: added table_item type
Aaron Marcuse-Kubitza
01:59 PM Revision 14134: schemas/util.sql: col_ref: renamed to just col
Aaron Marcuse-Kubitza
01:38 PM Revision 14133: web/username_prefix.php: remove padding used to visually separate elements: documented that this leaves embedded __ (eg. x__y) untouched
Aaron Marcuse-Kubitza
02:52 AM Revision 14132: fix: inputs/.TNRS/schema.sql: taxon_match: to populate a new column: also need to recluster table so rows are exported in sorted order
Aaron Marcuse-Kubitza
02:47 AM Revision 14131: schemas/util.sql: cluster(): support omitting index to recluster a table
Aaron Marcuse-Kubitza
02:26 AM Revision 14130: schemas/util.sql: cluster_once(): use util.cluster()
Aaron Marcuse-Kubitza
02:24 AM Revision 14129: schemas/util.sql: cluster_once(): documented that this runs cluster only *once* if called repeatedly
Aaron Marcuse-Kubitza
02:24 AM Revision 14128: schemas/util.sql: added cluster(), which reclusters each time
Aaron Marcuse-Kubitza
02:10 AM Revision 14127: schemas/util.sql: cluster_once(): use simpler `LANGUAGE sql` now that we're using util.eval() instead of EXECUTE
Aaron Marcuse-Kubitza
02:03 AM Revision 14126: schemas/util.sql: cluster_once(): use util.eval() instead of EXECUTE to print the CLUSTER statement (if any) it's using
Aaron Marcuse-Kubitza

07/18/2014

11:40 PM Revision 14125: fix: inputs/.TNRS/schema.sql: taxon_match: cluster the table so the rows are always exported in the same order, even when an UPDATE statement is used to run triggers
Aaron Marcuse-Kubitza
11:28 PM Revision 14124: bugfix: schemas/util.sql: cluster_once(): need to use just the name of the index, without the schema prefix
Aaron Marcuse-Kubitza
05:00 PM Revision 14123: schemas/util.sql: cluster_once(): use simpler IS DISTINCT FROM instead of array !=
Aaron Marcuse-Kubitza
03:18 PM Revision 14122: inputs/.TNRS/schema.sql: added __accepted_{genus,specific_epithet} derived column, using the steps at wiki.vegpath.org/Postgres_queries#maintaining-derived-column-formulas-across-column-renames
Aaron Marcuse-Kubitza
02:01 AM Revision 14121: web/username_prefix.php: remove padding used to visually separate elements: documented that this also removes a leading __ that indicates an internal field
Aaron Marcuse-Kubitza

07/17/2014

07:18 PM Revision 14120: web/links/index.htm: updated to Firefox bookmarks: Quick links: added VegBIEN data dictionary spreadsheet, TNRS data dictionary.
Aaron Marcuse-Kubitza
06:15 PM Revision 14119: fix: web/username_prefix.php: transposes: changed recommended syntax to _y~x@url so that _y~ lines up vertically with [y_] paths
Aaron Marcuse-Kubitza
06:11 PM Revision 14118: web/username_prefix.php: removed special category comments syntax because this is now handled by transposes and insertion comments
Aaron Marcuse-Kubitza
06:10 PM Revision 14117: web/username_prefix.php: added support for transposes ( y~x_@url -> x_y@url , Y~x@url -> xY@url ), which can be used to put the category in a column name first without duplicating the category
Aaron Marcuse-Kubitza
05:35 PM Revision 14116: web/username_prefix.php: removed linewraps code because Google spreadsheets no longer supports wrapping on a - , and because a shorter syntax for category comments is now available
Aaron Marcuse-Kubitza
05:32 PM Revision 14115: web/username_prefix.php: category comments: moved _-_ outside [] in comparison example since it will be deleted separately
Aaron Marcuse-Kubitza
05:27 PM Revision 14114: bugfix: web/username_prefix.php: category comments: need /__/ around regexp
Aaron Marcuse-Kubitza
05:24 PM Revision 14113: web/username_prefix.php: category comments: added support for a shorter syntax than [x_-_]y@url : x+y@url -> y@url
Aaron Marcuse-Kubitza
04:14 PM Revision 14112: schemas/vegbien.ERD.mwb: updated
Aaron Marcuse-Kubitza
04:13 PM Revision 14111: fix: inputs/.TNRS/schema.sql: taxon_match: added back * prefixes on TNRS-namespace column names
Aaron Marcuse-Kubitza
03:34 PM Revision 14110: inputs/.TNRS/schema.sql: MatchedTaxon: taxon_best_match: don't alias to `s` since this is no longer a nested select
Aaron Marcuse-Kubitza
03:32 PM Revision 14109: inputs/.TNRS/schema.sql: MatchedTaxon: nested select: use just taxon_best_match since this no longer performs renamings
Aaron Marcuse-Kubitza
03:27 PM Revision 14108: inputs/.TNRS/schema.sql: MatchedTaxon: renamed output columns to match input columns, so that the nested select would not be performing any renamings
Aaron Marcuse-Kubitza
03:05 PM Revision 14107: fix: inputs/.TNRS/schema.sql: MatchedTaxon: added all columns from taxon_match (some of them apparently hadn't been included in MatchedTaxon)
Aaron Marcuse-Kubitza
03:01 PM Revision 14106: fix: inputs/.TNRS/schema.sql: reordered columns to match taxon_match
Aaron Marcuse-Kubitza
02:38 PM Revision 14105: fix: inputs/.TNRS/schema.sql: MatchedTaxon: added all columns from taxon_match (some of them apparently hadn't been included in MatchedTaxon)
Aaron Marcuse-Kubitza
01:02 PM Revision 14104: fix: inputs/.TNRS/schema.sql: added back Family_matched, which had gotten merged with Name_matched_accepted_family when TNRS temporarily stopped populating Name_matched_accepted_family
Aaron Marcuse-Kubitza
12:32 PM Revision 14103: inputs/.TNRS/schema.sql: MatchedTaxon: taxonomicStatus: moved outside the inner SELECT so that the inner SELECT would consist solely of renamings
Aaron Marcuse-Kubitza
12:12 PM Revision 14102: inputs/.TNRS/schema.sql: MatchedTaxon: removed unneeded "Name_matched." prefix on source-specific names (only the * is needed because there is only one table)
Aaron Marcuse-Kubitza
12:10 PM Revision 14101: bugfix: schemas/util.sql: view_is_subset(): auto-updatable views that are *not* plain subsets: include ones with WHERE clauses
Aaron Marcuse-Kubitza
12:03 PM Revision 14100: bugfix: schemas/util.sql: view_is_subset(): also need to handle auto-updatable views that are *not* plain subsets (there are some of these)
Aaron Marcuse-Kubitza
09:05 AM Revision 14099: bin/import_all: hidden_srcs(): removed `by_col=1` because these should be done in the same mode as the main datasources
Aaron Marcuse-Kubitza

07/16/2014

07:55 PM Revision 14098: schemas/public_.sql: removed no longer used threatened_taxonlabel. use iucn_red_list instead.
Aaron Marcuse-Kubitza
07:39 PM Revision 14097: fix: schemas/public_.sql: analytical_stem_view: don't use threatened_taxonlabel, which was never populated correctly. instead, this can eventually be rewritten to use new iucn_red_list.
Aaron Marcuse-Kubitza
07:13 PM Revision 14096: bin/make_analytical_db: removed threatened_taxonlabel because this is now handled by iucn_red_list
Aaron Marcuse-Kubitza
07:12 PM Revision 14095: bin/make_analytical_db: added iucn_red_list_view_modify()
Aaron Marcuse-Kubitza
06:48 PM Revision 14094: bin/make_analytical_db: removed unused code to create views in the analytical_db schema
Aaron Marcuse-Kubitza
06:35 PM Revision 14093: bin/make_analytical_db: merged mk_table and mk_analytical_table since they now do the same thing
Aaron Marcuse-Kubitza
06:34 PM Revision 14092: bin/make_analytical_db: removed unused code to create views in the analytical_db schema
Aaron Marcuse-Kubitza
05:05 PM Revision 14091: /README.TXT: Notes on system stability: added warning that when shutting down the VM, one should always first stop Postgres, to prevent the OS from SIGKILLing it
Aaron Marcuse-Kubitza
05:04 PM Revision 14090: inputs/SALVIAS/Source/header.csv: updated
Aaron Marcuse-Kubitza
03:50 PM Revision 14089: bugfix: bin/with_all: isset(): need to use `&>/dev/null` instead of `>&-`, etc because closing an fd causes declare to return false
Aaron Marcuse-Kubitza
03:31 PM Revision 14088: bugfix: bin/with_all, import_all: don't disown processes because they should be auto-killed if the shell is (disown was only needed before we used screen)
Aaron Marcuse-Kubitza
03:24 PM Revision 14087: bugfix: Full database import: need nested shell to prevent errexit from closing the window
Aaron Marcuse-Kubitza
01:24 PM Revision 14086: bugfix: schemas/public_.sql: analytical_stem_view_modify(): added columns: need to include COMMENT statements
Aaron Marcuse-Kubitza
01:21 PM Revision 14085: bugfix: schemas/util.sql: comment(element oid): need `objsubid = 0` filter so this doesn't use comments for any other objsubids
Aaron Marcuse-Kubitza
12:53 PM Revision 14084: schemas/public_.sql: added iucn_red_list_view_modify() and use it in iucn_red_list_view's "after updating this" instructions
Aaron Marcuse-Kubitza
12:47 PM Revision 14083: bugfix: schemas/util.sql: rematerialize_view(): also need to handle the target table's dependent views
Aaron Marcuse-Kubitza
12:38 PM Revision 14082: bugfix: schemas/util.sql: mk_drop_from_create(): need to use `IF EXISTS` so that the resulting statement also works if the target object does not yet exist
Aaron Marcuse-Kubitza
12:24 PM Revision 14081: schemas/util.sql: mk_drop_from_create(): also support `SELECT util.drop_*()` statements, in addition to DROP
Aaron Marcuse-Kubitza
11:37 AM Revision 14080: inputs/.TNRS/Source/test.xml.ref: updated
Aaron Marcuse-Kubitza
 

Also available in: Atom