Project

General

Profile

Statistics
| Revision:
  • svn:ignore: extern

# Date Author Comment
14462 08/11/2014 05:08 AM Aaron Marcuse-Kubitza

fix: schemas/public_.sql: viewfulloccurrence_individual_view_modify(): fixed capitalization in function name (needs "" to preserve case)

14461 08/11/2014 05:03 AM Aaron Marcuse-Kubitza

schemas/public_.sql: other derived views: use viewFullOccurrence_individual instead of analytical_stem because analytical_stem is no longer materialized (viewFullOccurrence_individual_view is materialized instead)

14460 08/11/2014 05:01 AM Aaron Marcuse-Kubitza

schemas/public_.sql: validation views: use analytical_stem_view instead of analytical_stem because analytical_stem is no longer materialized (viewFullOccurrence_individual_view is materialized instead)

14459 08/11/2014 04:57 AM Aaron Marcuse-Kubitza

schemas/public_.sql: derived views: use analytical_stem_view instead of analytical_stem because analytical_stem is no longer materialized (viewFullOccurrence_individual_view is materialized instead)

14458 08/11/2014 04:46 AM Aaron Marcuse-Kubitza

bugfix: schemas/util.sql: expansion_min_cols(): increased to properly handle all public-schema views (the largest is currently 2014-6-12.Jeff_Ott.climatic_range_determinants)

14457 08/11/2014 04:35 AM Aaron Marcuse-Kubitza

schemas/util.sql: view_def_to_orig(): use new expansion_min_cols() for easier configuration

14456 08/11/2014 04:33 AM Aaron Marcuse-Kubitza

schemas/util.sql: added expansion_min_cols() (the minimum # of cols from the same table to be treated as a * expression)

14455 08/11/2014 04:05 AM Aaron Marcuse-Kubitza

fix: schemas/public_.sql: analytical_stem_view: regenerated columns list for viewFullOccurrence_individual (instead of viewFullOccurrence_individual_view)

14454 08/11/2014 03:40 AM Aaron Marcuse-Kubitza

bugfix: mappings/VegCore-VegBIEN.csv: prefixed taxonomic ranks: use _concat_nullify() so that the prefix is only added if the epithet is non-NULL

14453 08/11/2014 03:29 AM Aaron Marcuse-Kubitza

fix: schemas/util.sql: col_re(): support column names with " in them

14452 08/11/2014 02:59 AM Aaron Marcuse-Kubitza

schemas/util.sql: added _concat_nullify(), which uses || instead of concat()

14451 08/11/2014 02:16 AM Aaron Marcuse-Kubitza

bugfix: inputs/FIA/REF_RESEARCH_STATION/map.csv: mapped country, which is not provided in the FIA data

14450 08/11/2014 01:46 AM Aaron Marcuse-Kubitza

schemas/public_.sql: removed no longer used threatened_taxonlabel_view. use iucn_red_list_view instead.

14449 08/11/2014 01:35 AM Aaron Marcuse-Kubitza

schemas/public_.sql: viewFullOccurrence_individual_view: documented materialize time (22 h)

14448 08/10/2014 05:53 AM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: taxon_match: removed no longer used scrubbed_unique_taxon_name. the scrubbed name ranks are now generated from the other TNRS columns instead.

14447 08/10/2014 05:28 AM Aaron Marcuse-Kubitza

/README.TXT: Full database import: added steps to re-run geoscrubbing

14446 08/10/2014 05:22 AM Aaron Marcuse-Kubitza

exports/geoscrub_input.csv.run: export_(): updated runtime (30 s)

14445 08/10/2014 03:57 AM Aaron Marcuse-Kubitza

fix: exports/viewFullOccurrence_*.csv.run: need to limit # rows to 5000 as requested by Bob

14444 08/10/2014 03:42 AM Aaron Marcuse-Kubitza

schemas/public_.sql: viewFullOccurrence_*: documented materialize time

14443 08/09/2014 10:23 PM Aaron Marcuse-Kubitza

bin/make_analytical_db: materialize viewFullOccurrence_individual_view instead of analytical_stem_view because analytical_stem_view is now generatable via a simple join onto viewFullOccurrence_individual_view. this avoids running into potential disk space constraints when materializing and backing up both tables (~50 GB/table * 2 tables * 2 copies (incl. the backup) = 200 GB, which is very close to the available disk space).

14442 08/09/2014 10:17 PM Aaron Marcuse-Kubitza

schemas/public_.sql: analytical_stem_view: use new materialized viewFullOccurrence_individual

14441 08/09/2014 10:08 PM Aaron Marcuse-Kubitza

schemas/public_.sql: added viewFullOccurrence_individual by running viewFullOccurrence_individual_view_modify()

14440 08/09/2014 10:05 PM Aaron Marcuse-Kubitza

schemas/public_.sql: added viewFullOccurrence_individual_view_modify(), analogous to analytical_stem_view_modify()

14439 08/09/2014 09:52 PM Aaron Marcuse-Kubitza

bin/make_analytical_db: removed extra () around psql_verbose_vegbien

14438 08/09/2014 09:51 PM Aaron Marcuse-Kubitza

bin/make_analytical_db: removed no longer used mk_table()

14437 08/09/2014 09:49 PM Aaron Marcuse-Kubitza

bin/make_analytical_db: use more up-to-date *_view_modify() functions instead of mk_table()

14436 08/09/2014 08:46 PM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: removed no longer used view ValidMatchedTaxon. use taxon_scrub instead.

14435 08/09/2014 08:44 PM Aaron Marcuse-Kubitza

schemas/public_.sql: iucn_red_list_view: use taxon_scrub instead of ValidMatchedTaxon since they are equivalent

14434 08/09/2014 08:42 PM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: taxon_scrub: use taxon_best_match directly, to avoid the need for a separate ValidMatchedTaxon view

14433 08/09/2014 08:25 PM Aaron Marcuse-Kubitza

fix: inputs/.TNRS/schema.sql: taxon_scrub: merged synonymous columns

14432 08/09/2014 08:11 PM Aaron Marcuse-Kubitza

schemas/vegbien.sql: taxon_scrub: documented steps to merge synonymous columns

14431 08/09/2014 07:45 PM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: removed no longer used view MatchedTaxon. use taxon_best_match instead.

14430 08/09/2014 07:43 PM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: ValidMatchedTaxon: use taxon_best_match now that it's equivalent to MatchedTaxon

14429 08/09/2014 07:38 PM Aaron Marcuse-Kubitza

bugfix: schemas/public_.sql: tnrs_input_name: use taxon_match instead of taxon_best_match because there is no index on taxon_match that includes just the filters used by taxon_best_match

14428 08/09/2014 07:37 PM Aaron Marcuse-Kubitza

schemas/public_.sql: tnrs_input_name: use taxon_best_match now that it's equivalent to MatchedTaxon

14427 08/09/2014 07:20 PM Aaron Marcuse-Kubitza

backups/TNRS.backup.md5: updated

14426 08/09/2014 07:19 PM Aaron Marcuse-Kubitza

fix: inputs/.TNRS/schema.sql: MatchedTaxon: merged synonymous columns

14425 08/09/2014 07:02 PM Aaron Marcuse-Kubitza

fix: schemas/util.sql: view_is_subset_or_renaming(): views with CASE statements are also not subsets/renamings

14424 08/09/2014 06:52 PM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: removed no longer used taxon_scrub.scrubbed_unique_taxon_name.* . use taxon_scrub instead.

14423 08/09/2014 06:50 PM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: taxon_scrub: use taxon_match derived columns instead of the incorrect values in taxon_scrub.scrubbed_unique_taxon_name.* (which does not work with the multi-match strategy)

14422 08/09/2014 05:54 AM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: MatchedTaxon: use derived columns from taxon_match. this also incorporates the fixes in the new derived columns.

14421 08/09/2014 05:27 AM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: taxon_scrub: use derived columns from taxon_match. this also incorporates the fixes in the new derived columns.

14420 08/08/2014 07:14 PM Aaron Marcuse-Kubitza

added exports/viewFullOccurrence_{CVS,VegBank,NCU}.csv.run

14419 08/08/2014 07:13 PM Aaron Marcuse-Kubitza

schemas/public_.sql: added viewFullOccurrence_{CVS,VegBank,NCU} for Bob

14418 08/08/2014 02:32 PM Aaron Marcuse-Kubitza

bugfix: schemas/util.sql: added GRANT USAGE for bien_read, public_ so that util schema functions can be called by other users

14417 08/05/2014 05:31 PM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: taxon_match: to port derived column changes to vegbiendev: derived_cols_export() code: documented runtime (6 h)

14416 08/04/2014 06:25 AM Aaron Marcuse-Kubitza

bugfix: inputs/.TNRS/schema.sql: removed no longer used derived column __accepted_infraspecific_label, which had a buggy formula that broke derived_cols_populate()

14415 08/04/2014 06:18 AM Aaron Marcuse-Kubitza

bugfix: schemas/util.sql: drop_derived_col(): also need to run util.derived_cols_trigger_update()

14414 08/04/2014 06:17 AM Aaron Marcuse-Kubitza

bugfix: inputs/.TNRS/schema.sql: removed no longer used derived column __accepted_infraspecific_label, which had a buggy formula that broke derived_cols_populate()

14413 08/04/2014 06:10 AM Aaron Marcuse-Kubitza

fix: inputs/.TNRS/schema.sql: taxon_match: to remove a column: updated instructions

14412 08/04/2014 05:09 AM Aaron Marcuse-Kubitza

bugfix: lib/sh/util.sh: str2varname(): need to lowercase str because on case-insensitive filesystems, paths sometimes canonicalize to a different capitalization than the original

14411 08/04/2014 05:00 AM Aaron Marcuse-Kubitza

lib/sh/util.sh: added lowercase()

14410 08/03/2014 09:54 PM Aaron Marcuse-Kubitza

bugfix: lib/sh/util.sh: die(): need stub since this is invoked before it's defined

14409 08/03/2014 09:12 PM Aaron Marcuse-Kubitza

bugfix: lib/sh/util.sh: setup_log_fd(): don't change $log_fd to stdlog until stdlog is set up, to avoid "Bad file descriptor" errors

14408 08/02/2014 07:58 PM Aaron Marcuse-Kubitza

**: updated to use the local machine's new hostname, frenzy

14407 08/02/2014 07:13 PM Aaron Marcuse-Kubitza

lib/sh/util.sh: func_override(), copy_func(): added echo_func to facilitate debugging

14406 08/02/2014 07:12 PM Aaron Marcuse-Kubitza

bugfix: lib/sh/util.sh: stubs: log++ alias also needs to be moved to stub section

14405 08/01/2014 06:31 PM Aaron Marcuse-Kubitza

bugfix: lib/Firefox_bookmarks.reformat.csv: URLs: match only the uppercase tags used by Firefox, not any lowercase tags added by the user

14404 08/01/2014 05:42 PM Aaron Marcuse-Kubitza

fix: lib/Firefox_bookmarks.reformat.csv: page's self-description: updated comment to match regexp

14403 08/01/2014 05:39 PM Aaron Marcuse-Kubitza

bugfix: lib/Firefox_bookmarks.reformat.csv: page's self-description: updated "page's self-description: " prefix to remove

14402 08/01/2014 04:50 PM Aaron Marcuse-Kubitza

web/links/index.htm: updated to Firefox bookmarks: added links for SSD vs. HDD reliability

14401 08/01/2014 04:22 PM Aaron Marcuse-Kubitza

/README.TXT: added steps to restore from Time Machine

14400 08/01/2014 04:15 PM Aaron Marcuse-Kubitza

/README.TXT: added steps to back up the local machine's hard drive

14399 08/01/2014 04:13 PM Aaron Marcuse-Kubitza

/README.TXT: to synchronize a Mac's settings with my testing machine's: renamed to "to back up the local machine's settings"

14398 08/01/2014 04:11 PM Aaron Marcuse-Kubitza

/README.TXT: to synchronize a Mac's settings with my testing machine's: removed download section because we don't use this

14397 08/01/2014 06:01 AM Aaron Marcuse-Kubitza

added backups/vegbien.r14089.backup.md5

14396 08/01/2014 06:01 AM Aaron Marcuse-Kubitza

backups/TNRS.backup.md5: updated

14395 08/01/2014 05:47 AM Aaron Marcuse-Kubitza

bin/after_import: use new bin/make_backups

14394 08/01/2014 05:47 AM Aaron Marcuse-Kubitza

bugfix: bin/make_backups: need to `popd` when done

14393 08/01/2014 05:46 AM Aaron Marcuse-Kubitza

bugfix: bin/make_backups: need to `set +x` when done

14392 08/01/2014 05:44 AM Aaron Marcuse-Kubitza

bin/make_backups: run with initial "." so background processes will be owned by the invoking shell

14391 08/01/2014 04:40 AM Aaron Marcuse-Kubitza

added bin/make_backups

14390 08/01/2014 01:59 AM Aaron Marcuse-Kubitza

web/links/index.htm: updated to Firefox bookmarks: VirtualBox: performance: added that context-switching also imposes an overhead, in addition to sandboxing.

14389 08/01/2014 01:16 AM Aaron Marcuse-Kubitza

web/links/index.htm: updated to Firefox bookmarks: BIEN: added list of machines this has been tested on. VirtualBox: added links to performance considerations of running in a VM.

14388 07/28/2014 11:58 PM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: added new derived columns to derived views

14387 07/28/2014 11:58 PM Aaron Marcuse-Kubitza

fix: schemas/util.sql: view_def_to_orig(): need to support aliased columns (produced when a column is renamed)

14386 07/28/2014 05:59 PM Aaron Marcuse-Kubitza

schemas/util.sql: added aliased_col_re()

14385 07/28/2014 05:57 PM Aaron Marcuse-Kubitza

schemas/util.sql: added alias_re()

14384 07/28/2014 05:50 PM Aaron Marcuse-Kubitza

fix: schemas/util.sql: view_is_subset(): renamed to view_is_subset_or_renaming() because this also supports views that just rename columns, which should not be .*-ed by view_def_to_orig()

14383 07/28/2014 05:33 PM Aaron Marcuse-Kubitza

fix: schemas/util.sql: view_def_to_orig(): can't use FROM to alias util.col_re() because that prevents inlining the function

14382 07/28/2014 05:31 PM Aaron Marcuse-Kubitza

fix: schemas/util.sql: view_def_to_orig(): can't use FROM to alias util.col_re() because that prevents inlining the function

14381 07/28/2014 05:24 PM Aaron Marcuse-Kubitza

schemas/util.sql: view_def_to_orig(): use util.col_re() for clarity

14380 07/28/2014 05:20 PM Aaron Marcuse-Kubitza

schemas/util.sql: added col_re()

14379 07/28/2014 04:45 PM Aaron Marcuse-Kubitza

fix: schemas/util.sql: derived_col_update(): also need steps to drop column, because DROP __ CASCADE doesn't work when there are dependent views

14378 07/28/2014 04:33 PM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: _accepted_infraspecific{rank,epithet}: use array slice of new _accepted{genus,specific_epithet,infra_{rank,epithet}}, which is simpler than using remove_prefix() in __accepted_infraspecific_label

14377 07/28/2014 04:21 PM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: "[accepted_]genus__@DwC__@vegpath.org": don't need to use *Accepted_name anymore because _accepted{genus,specific_epithet,infra_{rank,epithet}} is now generated from *Accepted_name

14376 07/28/2014 04:02 PM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: taxon_match."__accepted_{genus,specific_epithet}": renamed to "__accepted_{genus,specific_epithet,infra_{rank,epithet}}" since this now includes these other ranks as well

14375 07/28/2014 03:48 PM Aaron Marcuse-Kubitza

bugfix: inputs/.TNRS/schema.sql: taxon_match."__accepted_{genus,specific_epithet}": use "*Accepted_name" instead of "Accepted_species[_binomial]__@TNRS__@vegpath.org" (from "*Accepted_name_species") because Accepted_name_species apparently sometimes does not match the Accepted_name and uses malformed Unicode characters

14374 07/28/2014 03:20 PM Aaron Marcuse-Kubitza

backups/README.TXT: added backup retention policy

14373 07/27/2014 03:19 PM Aaron Marcuse-Kubitza

bugfix: schemas/util.sql: derived_cols_export(): also need to first run util.derived_cols_remove() if changing the derived column order

14372 07/27/2014 03:15 PM Aaron Marcuse-Kubitza

schemas/util.sql: added derived_cols_remove(), which allows derived columns to be re-created in a different order

14371 07/27/2014 03:01 PM Aaron Marcuse-Kubitza

schemas/util.sql: added drop_derived_col(), which ensures that drop_column() only cascades to views

14370 07/27/2014 02:54 PM Aaron Marcuse-Kubitza

fix: schemas/util.sql: drop_column(): also need to recreate dependent views

14369 07/27/2014 02:26 PM Aaron Marcuse-Kubitza

schemas/util.sql: added drop_constraint()

14368 07/26/2014 10:16 PM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: taxon_match: `inputs/.TNRS/data.sql.run refresh`: documented runtime (1 min)

14367 07/26/2014 10:15 PM Aaron Marcuse-Kubitza

bugfix: inputs/.TNRS/schema.sql: taxon_match: use "Accepted_species[_binomial]__@TNRS__@vegpath.org" instead of "*Accepted_name_species". this fixes a bug in __accepted_infraspecific_label where Accepted_name_species with trailing whitespace could not be prefix-removed from names that contained just a species binomial.

14366 07/26/2014 09:57 PM Aaron Marcuse-Kubitza

fix: inputs/.TNRS/schema.sql: taxon_match: added derived column "Accepted_species[_binomial]__@TNRS__@vegpath.org", which removes trailing whitespace

14365 07/26/2014 08:58 PM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: added steps to remove a column and to add a non-derived column

14364 07/26/2014 08:48 PM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: taxon_match: to remove columns or add columns at the end: merged into "to add a new derived column"

14363 07/26/2014 08:44 PM Aaron Marcuse-Kubitza

inputs/.TNRS/schema.sql: to add columns in the middle: renamed to "to move a column to the middle" for clarity