Project

General

Profile

Statistics
| Revision:
  • svn:ignore: extern

# Date Author Comment
14450 08/11/2014 01:46 AM Aaron Marcuse-Kubitza

schemas/public_.sql: removed no longer used threatened_taxonlabel_view. use iucn_red_list_view instead.

14449 08/11/2014 01:35 AM Aaron Marcuse-Kubitza

schemas/public_.sql: viewFullOccurrence_individual_view: documented materialize time (22 h)

14448 08/10/2014 05:53 AM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: taxon_match: removed no longer used scrubbed_unique_taxon_name. the scrubbed name ranks are now generated from the other TNRS columns instead.

14447 08/10/2014 05:28 AM Aaron Marcuse-Kubitza

/README.TXT: Full database import: added steps to re-run geoscrubbing

14446 08/10/2014 05:22 AM Aaron Marcuse-Kubitza

exports/geoscrub_input.csv.run: export_(): updated runtime (30 s)

14445 08/10/2014 03:57 AM Aaron Marcuse-Kubitza

fix: exports/viewFullOccurrence_*.csv.run: need to limit # rows to 5000 as requested by Bob

14444 08/10/2014 03:42 AM Aaron Marcuse-Kubitza

schemas/public_.sql: viewFullOccurrence_*: documented materialize time

14443 08/09/2014 10:23 PM Aaron Marcuse-Kubitza

bin/make_analytical_db: materialize viewFullOccurrence_individual_view instead of analytical_stem_view because analytical_stem_view is now generatable via a simple join onto viewFullOccurrence_individual_view. this avoids running into potential disk space constraints when materializing and backing up both tables (~50 GB/table * 2 tables * 2 copies (incl. the backup) = 200 GB, which is very close to the available disk space).

14442 08/09/2014 10:17 PM Aaron Marcuse-Kubitza

schemas/public_.sql: analytical_stem_view: use new materialized viewFullOccurrence_individual

14441 08/09/2014 10:08 PM Aaron Marcuse-Kubitza

schemas/public_.sql: added viewFullOccurrence_individual by running viewFullOccurrence_individual_view_modify()

14440 08/09/2014 10:05 PM Aaron Marcuse-Kubitza

schemas/public_.sql: added viewFullOccurrence_individual_view_modify(), analogous to analytical_stem_view_modify()

14439 08/09/2014 09:52 PM Aaron Marcuse-Kubitza

bin/make_analytical_db: removed extra () around psql_verbose_vegbien

14438 08/09/2014 09:51 PM Aaron Marcuse-Kubitza

bin/make_analytical_db: removed no longer used mk_table()

14437 08/09/2014 09:49 PM Aaron Marcuse-Kubitza

bin/make_analytical_db: use more up-to-date *_view_modify() functions instead of mk_table()

14436 08/09/2014 08:46 PM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: removed no longer used view ValidMatchedTaxon. use taxon_scrub instead.

14435 08/09/2014 08:44 PM Aaron Marcuse-Kubitza

schemas/public_.sql: iucn_red_list_view: use taxon_scrub instead of ValidMatchedTaxon since they are equivalent

14434 08/09/2014 08:42 PM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: taxon_scrub: use taxon_best_match directly, to avoid the need for a separate ValidMatchedTaxon view

14433 08/09/2014 08:25 PM Aaron Marcuse-Kubitza

fix: inputs/.TNRS/schema.sql: taxon_scrub: merged synonymous columns

14432 08/09/2014 08:11 PM Aaron Marcuse-Kubitza

schemas/vegbien.sql: taxon_scrub: documented steps to merge synonymous columns

14431 08/09/2014 07:45 PM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: removed no longer used view MatchedTaxon. use taxon_best_match instead.

14430 08/09/2014 07:43 PM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: ValidMatchedTaxon: use taxon_best_match now that it's equivalent to MatchedTaxon

14429 08/09/2014 07:38 PM Aaron Marcuse-Kubitza

bugfix: schemas/public_.sql: tnrs_input_name: use taxon_match instead of taxon_best_match because there is no index on taxon_match that includes just the filters used by taxon_best_match

14428 08/09/2014 07:37 PM Aaron Marcuse-Kubitza

schemas/public_.sql: tnrs_input_name: use taxon_best_match now that it's equivalent to MatchedTaxon

14427 08/09/2014 07:20 PM Aaron Marcuse-Kubitza

backups/TNRS.backup.md5: updated

14426 08/09/2014 07:19 PM Aaron Marcuse-Kubitza

fix: inputs/.TNRS/schema.sql: MatchedTaxon: merged synonymous columns

14425 08/09/2014 07:02 PM Aaron Marcuse-Kubitza

fix: schemas/util.sql: view_is_subset_or_renaming(): views with CASE statements are also not subsets/renamings

14424 08/09/2014 06:52 PM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: removed no longer used taxon_scrub.scrubbed_unique_taxon_name.* . use taxon_scrub instead.

14423 08/09/2014 06:50 PM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: taxon_scrub: use taxon_match derived columns instead of the incorrect values in taxon_scrub.scrubbed_unique_taxon_name.* (which does not work with the multi-match strategy)

14422 08/09/2014 05:54 AM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: MatchedTaxon: use derived columns from taxon_match. this also incorporates the fixes in the new derived columns.

14421 08/09/2014 05:27 AM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: taxon_scrub: use derived columns from taxon_match. this also incorporates the fixes in the new derived columns.

14420 08/08/2014 07:14 PM Aaron Marcuse-Kubitza

added exports/viewFullOccurrence_{CVS,VegBank,NCU}.csv.run

14419 08/08/2014 07:13 PM Aaron Marcuse-Kubitza

schemas/public_.sql: added viewFullOccurrence_{CVS,VegBank,NCU} for Bob

14418 08/08/2014 02:32 PM Aaron Marcuse-Kubitza

bugfix: schemas/util.sql: added GRANT USAGE for bien_read, public_ so that util schema functions can be called by other users

14417 08/05/2014 05:31 PM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: taxon_match: to port derived column changes to vegbiendev: derived_cols_export() code: documented runtime (6 h)

14416 08/04/2014 06:25 AM Aaron Marcuse-Kubitza

bugfix: inputs/.TNRS/schema.sql: removed no longer used derived column __accepted_infraspecific_label, which had a buggy formula that broke derived_cols_populate()

14415 08/04/2014 06:18 AM Aaron Marcuse-Kubitza

bugfix: schemas/util.sql: drop_derived_col(): also need to run util.derived_cols_trigger_update()

14414 08/04/2014 06:17 AM Aaron Marcuse-Kubitza

bugfix: inputs/.TNRS/schema.sql: removed no longer used derived column __accepted_infraspecific_label, which had a buggy formula that broke derived_cols_populate()

14413 08/04/2014 06:10 AM Aaron Marcuse-Kubitza

fix: inputs/.TNRS/schema.sql: taxon_match: to remove a column: updated instructions

14412 08/04/2014 05:09 AM Aaron Marcuse-Kubitza

bugfix: lib/sh/util.sh: str2varname(): need to lowercase str because on case-insensitive filesystems, paths sometimes canonicalize to a different capitalization than the original

14411 08/04/2014 05:00 AM Aaron Marcuse-Kubitza

lib/sh/util.sh: added lowercase()

14410 08/03/2014 09:54 PM Aaron Marcuse-Kubitza

bugfix: lib/sh/util.sh: die(): need stub since this is invoked before it's defined

14409 08/03/2014 09:12 PM Aaron Marcuse-Kubitza

bugfix: lib/sh/util.sh: setup_log_fd(): don't change $log_fd to stdlog until stdlog is set up, to avoid "Bad file descriptor" errors

14408 08/02/2014 07:58 PM Aaron Marcuse-Kubitza

**: updated to use the local machine's new hostname, frenzy

14407 08/02/2014 07:13 PM Aaron Marcuse-Kubitza

lib/sh/util.sh: func_override(), copy_func(): added echo_func to facilitate debugging

14406 08/02/2014 07:12 PM Aaron Marcuse-Kubitza

bugfix: lib/sh/util.sh: stubs: log++ alias also needs to be moved to stub section

14405 08/01/2014 06:31 PM Aaron Marcuse-Kubitza

bugfix: lib/Firefox_bookmarks.reformat.csv: URLs: match only the uppercase tags used by Firefox, not any lowercase tags added by the user

14404 08/01/2014 05:42 PM Aaron Marcuse-Kubitza

fix: lib/Firefox_bookmarks.reformat.csv: page's self-description: updated comment to match regexp

14403 08/01/2014 05:39 PM Aaron Marcuse-Kubitza

bugfix: lib/Firefox_bookmarks.reformat.csv: page's self-description: updated "page's self-description: " prefix to remove

14402 08/01/2014 04:50 PM Aaron Marcuse-Kubitza

web/links/index.htm: updated to Firefox bookmarks: added links for SSD vs. HDD reliability

14401 08/01/2014 04:22 PM Aaron Marcuse-Kubitza

/README.TXT: added steps to restore from Time Machine

14400 08/01/2014 04:15 PM Aaron Marcuse-Kubitza

/README.TXT: added steps to back up the local machine's hard drive

14399 08/01/2014 04:13 PM Aaron Marcuse-Kubitza

/README.TXT: to synchronize a Mac's settings with my testing machine's: renamed to "to back up the local machine's settings"

14398 08/01/2014 04:11 PM Aaron Marcuse-Kubitza

/README.TXT: to synchronize a Mac's settings with my testing machine's: removed download section because we don't use this

14397 08/01/2014 06:01 AM Aaron Marcuse-Kubitza

added backups/vegbien.r14089.backup.md5

14396 08/01/2014 06:01 AM Aaron Marcuse-Kubitza

backups/TNRS.backup.md5: updated

14395 08/01/2014 05:47 AM Aaron Marcuse-Kubitza

bin/after_import: use new bin/make_backups

14394 08/01/2014 05:47 AM Aaron Marcuse-Kubitza

bugfix: bin/make_backups: need to `popd` when done

14393 08/01/2014 05:46 AM Aaron Marcuse-Kubitza

bugfix: bin/make_backups: need to `set +x` when done

14392 08/01/2014 05:44 AM Aaron Marcuse-Kubitza

bin/make_backups: run with initial "." so background processes will be owned by the invoking shell

14391 08/01/2014 04:40 AM Aaron Marcuse-Kubitza

added bin/make_backups

14390 08/01/2014 01:59 AM Aaron Marcuse-Kubitza

web/links/index.htm: updated to Firefox bookmarks: VirtualBox: performance: added that context-switching also imposes an overhead, in addition to sandboxing.

14389 08/01/2014 01:16 AM Aaron Marcuse-Kubitza

web/links/index.htm: updated to Firefox bookmarks: BIEN: added list of machines this has been tested on. VirtualBox: added links to performance considerations of running in a VM.

14388 07/28/2014 11:58 PM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: added new derived columns to derived views

14387 07/28/2014 11:58 PM Aaron Marcuse-Kubitza

fix: schemas/util.sql: view_def_to_orig(): need to support aliased columns (produced when a column is renamed)

14386 07/28/2014 05:59 PM Aaron Marcuse-Kubitza

schemas/util.sql: added aliased_col_re()

14385 07/28/2014 05:57 PM Aaron Marcuse-Kubitza

schemas/util.sql: added alias_re()

14384 07/28/2014 05:50 PM Aaron Marcuse-Kubitza

fix: schemas/util.sql: view_is_subset(): renamed to view_is_subset_or_renaming() because this also supports views that just rename columns, which should not be .*-ed by view_def_to_orig()

14383 07/28/2014 05:33 PM Aaron Marcuse-Kubitza

fix: schemas/util.sql: view_def_to_orig(): can't use FROM to alias util.col_re() because that prevents inlining the function

14382 07/28/2014 05:31 PM Aaron Marcuse-Kubitza

fix: schemas/util.sql: view_def_to_orig(): can't use FROM to alias util.col_re() because that prevents inlining the function

14381 07/28/2014 05:24 PM Aaron Marcuse-Kubitza

schemas/util.sql: view_def_to_orig(): use util.col_re() for clarity

14380 07/28/2014 05:20 PM Aaron Marcuse-Kubitza

schemas/util.sql: added col_re()

14379 07/28/2014 04:45 PM Aaron Marcuse-Kubitza

fix: schemas/util.sql: derived_col_update(): also need steps to drop column, because DROP __ CASCADE doesn't work when there are dependent views

14378 07/28/2014 04:33 PM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: _accepted_infraspecific{rank,epithet}: use array slice of new _accepted{genus,specific_epithet,infra_{rank,epithet}}, which is simpler than using remove_prefix() in __accepted_infraspecific_label

14377 07/28/2014 04:21 PM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: "[accepted_]genus__@DwC__@vegpath.org": don't need to use *Accepted_name anymore because _accepted{genus,specific_epithet,infra_{rank,epithet}} is now generated from *Accepted_name

14376 07/28/2014 04:02 PM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: taxon_match."__accepted_{genus,specific_epithet}": renamed to "__accepted_{genus,specific_epithet,infra_{rank,epithet}}" since this now includes these other ranks as well

14375 07/28/2014 03:48 PM Aaron Marcuse-Kubitza

bugfix: inputs/.TNRS/schema.sql: taxon_match."__accepted_{genus,specific_epithet}": use "*Accepted_name" instead of "Accepted_species[_binomial]__@TNRS__@vegpath.org" (from "*Accepted_name_species") because Accepted_name_species apparently sometimes does not match the Accepted_name and uses malformed Unicode characters

14374 07/28/2014 03:20 PM Aaron Marcuse-Kubitza

backups/README.TXT: added backup retention policy

14373 07/27/2014 03:19 PM Aaron Marcuse-Kubitza

bugfix: schemas/util.sql: derived_cols_export(): also need to first run util.derived_cols_remove() if changing the derived column order

14372 07/27/2014 03:15 PM Aaron Marcuse-Kubitza

schemas/util.sql: added derived_cols_remove(), which allows derived columns to be re-created in a different order

14371 07/27/2014 03:01 PM Aaron Marcuse-Kubitza

schemas/util.sql: added drop_derived_col(), which ensures that drop_column() only cascades to views

14370 07/27/2014 02:54 PM Aaron Marcuse-Kubitza

fix: schemas/util.sql: drop_column(): also need to recreate dependent views

14369 07/27/2014 02:26 PM Aaron Marcuse-Kubitza

schemas/util.sql: added drop_constraint()

14368 07/26/2014 10:16 PM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: taxon_match: `inputs/.TNRS/data.sql.run refresh`: documented runtime (1 min)

14367 07/26/2014 10:15 PM Aaron Marcuse-Kubitza

bugfix: inputs/.TNRS/schema.sql: taxon_match: use "Accepted_species[_binomial]__@TNRS__@vegpath.org" instead of "*Accepted_name_species". this fixes a bug in __accepted_infraspecific_label where Accepted_name_species with trailing whitespace could not be prefix-removed from names that contained just a species binomial.

14366 07/26/2014 09:57 PM Aaron Marcuse-Kubitza

fix: inputs/.TNRS/schema.sql: taxon_match: added derived column "Accepted_species[_binomial]__@TNRS__@vegpath.org", which removes trailing whitespace

14365 07/26/2014 08:58 PM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: added steps to remove a column and to add a non-derived column

14364 07/26/2014 08:48 PM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: taxon_match: to remove columns or add columns at the end: merged into "to add a new derived column"

14363 07/26/2014 08:44 PM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: to add columns in the middle: renamed to "to move a column to the middle" for clarity

14362 07/26/2014 08:38 PM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: to populate a new column: updated to use util.derived_col_update()

14361 07/26/2014 08:24 PM Aaron Marcuse-Kubitza

fix: inputs/.TNRS/schema.sql: taxon_match: to remove columns or add columns: also need to run util.recreate_view()

14360 07/26/2014 08:19 PM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: taxon_match: to remove columns or add columns at the end: don't need to run `rm=1 inputs/.TNRS/data.sql.run` because this is now run by `make schemas/remake`

14359 07/26/2014 08:13 PM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: added new derived columns to derived views

14358 07/26/2014 07:52 PM Aaron Marcuse-Kubitza

schemas/util.sql: remove_prefix(), remove_suffix(): support case-insensitive matching

14357 07/26/2014 07:43 PM Aaron Marcuse-Kubitza

bugfix: inputs/.TNRS/schema.sql: taxon_match.__accepted_infraspecific_label: need to use case-insensitive matching of the removed prefix because TNRS lowercases part of the Accepted_name

14356 07/26/2014 07:40 PM Aaron Marcuse-Kubitza

schemas/util.sql: remove_prefix(), remove_suffix(): support case-insensitive matching

14355 07/26/2014 06:54 PM Aaron Marcuse-Kubitza

schemas/util.sql: remove_prefix(), remove_suffix(): support case-insensitive matching

14354 07/26/2014 06:52 PM Aaron Marcuse-Kubitza

schemas/util.sql: added case-sensitive-optional versions of starts_with(), ends_with()

14353 07/26/2014 06:50 PM Aaron Marcuse-Kubitza

schemas/util.sql: added canon_case(str text, case_sensitive boolean)

14352 07/26/2014 06:36 PM Aaron Marcuse-Kubitza

schemas/util.sql: check_constraint_disable(): documented that we can't use ADD CONSTRAINT NOT VALID for this because it does not disable the constraint completely, but instead causes new rows to be validated against it. the ADD CONSTRAINT NOT VALID behavior does not work for the derived columns use case of this function.

14351 07/26/2014 06:17 PM Aaron Marcuse-Kubitza

schemas/public_.sql: *_view_modify(): added repopulate param, which is passed to rematerialize_view()