Activity
From 08/04/2014 to 09/02/2014
08/29/2014
- 03:55 PM Revision 14633: added inputs/CVS/verify/Review of CVS data in BIEN3.docx
- 12:40 AM Revision 14632: backups/*retention_policy*: added explanations
- 12:39 AM Revision 14631: backups/*retention_policy*: on jupiter: backups further back: removed "if disk space permits" because this is already labeled "optionally"
- 12:38 AM Revision 14630: backups/*retention_policy*: changed to require retaining *.backup of the last 2 successful imports on all machines
- 12:25 AM Revision 14629: backups/*retention_policy*: allow keeping *.backup of the last 2 successful imports on all machines, not just jupiter
- 12:17 AM Revision 14628: **: renamed 2TB drive's BIEN3 partition to BIEN3.**SAVE** since one might not see the **SAVE** file in it
- 12:13 AM Revision 14627: **: renamed 2TB drive's BIEN3 partition to BIEN3.**SAVE** since one might not see the **SAVE** file in it
- 12:09 AM Revision 14626: **/"**DO_NOT_DELETE**": renamed to shorter **SAVE**
- 12:04 AM Revision 14625: added backups/*retention_policies*/ with retention policy files for each partition
08/28/2014
- 11:58 PM Revision 14624: backups/README.TXT: renamed to *retention_policy* to match the naming convention of the retention policy files in the various partitions
- 11:42 PM Revision 14623: /README.TXT: to back up the local machine's hard drive: also exclude *-files indicating the (differing) retention statuses of the partitions involved
- 08:13 PM Revision 14622: lib/tnrs.py single_tnrs_request(), bin/tnrs_client: use_tnrs_export: default to False because this mode uses incorrect selected matches (vegpath.org/issues/943), and the JSON mode that fixes this is now available
- 08:05 PM Revision 14621: bin/tnrs_db: tnrs.tnrs_request() call: explicitly set use_tnrs_export=True so that this continues to work if the default value is changed
- 07:57 PM Revision 14620: bugfix: lib/csvs.py: JsonReader: need to pass col_order to row_dict_to_list_reader
- 07:43 PM Revision 14619: config/VirtualBox_VMs/vegbiendev/README.TXT: ~/Documents/BIEN/vegbiendev.2014-2-2_1-07-32PT.+VirtualBox_changes/: renamed to vegbiendev.2014-2-2_1-07-32PT.VirtualBox/ to make clear that this is the VirtualBox version of vegbiendev
- 07:12 PM Revision 14618: bugfix: lib/tnrs.py: JSON output: need to stringify arrays so they match what is output in TSV-export mode
- 07:10 PM Revision 14617: lib/csvs.py: JsonReader: added support for values that are arrays
- 07:05 PM Revision 14616: lib/csvs.py: MultiFilter: inherit from WrapReader instead of Filter to avoid needing to define a no-op filter_() function
- 06:49 PM Revision 14615: bugfix: lib/csvs.py: row_dict_to_list_reader: need to override next() directly instead of just using Filter, because Filter doesn't support returning multiple rows for one input row (in this case, prepending a header row). this caused the 1st data row to be missing.
- 06:47 PM Revision 14614: lib/csvs.py: Filter: inherit from WrapReader, which separates out the CSV-reader API code
- 06:43 PM Revision 14613: lib/csvs.py: added WrapReader
- 06:43 PM Revision 14612: lib/csvs.py: added Reader
- 06:00 PM Revision 14611: schemas/public_.sql: views that use view_full_occurrence_individual_view: use the view_full_occurrence_individual table instead, now that this is materialized.
- 05:58 PM Revision 14610: planning/meetings/BIEN conference call availability.xlsx: updated
- 08:57 AM Revision 14609: /README.TXT: to back up the local machine's hard drive: renamed backup partition to BIEN3 to make clear what the backup drive contains
- 08:54 AM Revision 14608: fix: /README.TXT: to back up the local machine's hard drive: updated location of `screen` for added commands
- 08:53 AM Revision 14607: /README.TXT: added trailing / on dirs to make clear that they're dirs
- 08:40 AM Revision 14606: config/VirtualBox_VMs/vegbiendev/README.TXT: added instructions to configure the VM to support VirtualBox
- 08:22 AM Revision 14605: config/VirtualBox_VMs/vegbiendev/README.TXT: added instructions to retrieve the contents of the VM, with the VirtualBox changes added
- 07:47 AM Revision 14604: config/VirtualBox_VMs/vegbiendev/README.TXT: to retrieve the original contents of the backup from the VM: added steps to restore the correct VM snapshot
- 07:40 AM Revision 14603: config/VirtualBox_VMs/vegbiendev/README.TXT: also generate list of all the files whose permissions were changed since the backup, but which are extracted with their changed permissions instead of their original ones in the backup
- 07:05 AM Revision 14602: config/VirtualBox_VMs/vegbiendev/README.TXT: added instructions to retrieve the original contents of the backup from the VM
- 05:47 AM Revision 14601: fix: /README.TXT: to back up vegbiendev: also back up /home/aaronmk/bien/ (instead of just symlinking to the local copy), since this can be done space-efficiently with hardlinks. this ensures that the vegbiendev backup will not be modified when the local copy of bien/ is.
- 03:10 AM Revision 14600: lib/csvs.py: JsonReader: factored out row-dict-to-list into new row_dict_to_list_reader so that JSON-specific preprocessing is kept separate from the row format translation
08/27/2014
- 03:17 PM Revision 14599: lib/csvs.py: added MultiFilter, which enables applying multiple filters by nesting
08/26/2014
- 07:57 PM Revision 14598: lib/tnrs.py: single_tnrs_request(): JSON mode: implemented output of JSON data
- 07:53 PM Revision 14597: lib/tnrs.py: single_tnrs_request(): factored out wrapping in TnrsOutputStream, since this is done for both modes
- 07:47 PM Revision 14596: fix: lib/tnrs.py: JSON mode: TSV export columns: need to translate these to JSON column names before they can be used with the JSON data
- 07:44 PM Revision 14595: lib/csvs.py: added JsonReader, which reads parsed JSON data as row tuples
- 07:43 PM Revision 14594: lib/csvs.py: added row_dict_to_list(), which translates a CSV dict-based row to a list-based one
- 07:43 PM Revision 14593: lib/csvs.py: RowNumFilter: added support for filtering the header row as well
- 07:42 PM Revision 14592: lib/csvs.py: ColInsertFilter: added support for filtering the header row as well
- 05:12 PM Revision 14591: lib/csvs.py: InputRewriter: documented that this is also a stream (in addition to inheriting from StreamFilter)
- 05:11 PM Revision 14590: bugfix: lib/csvs.py: InputRewriter: accept a reader, as would be expected, instead of a custom stream whose lines are tuples
- 05:08 PM Revision 14589: fix: lib/sql_io.py: append_csv(): use new csvs.ProgressInputFilter instead of streams.ProgressInputStream(csvs.StreamFilter(__)), so that the input to csvs.InputRewriter is a reader, not a stream. this avoids the need for csvs.InputRewriter to accept a stream whose lines are tuples, instead of the expected reader.
- 05:02 PM Revision 14588: bugfix: inputs/input.Makefile: %/install: $(exportHeader) must come before postprocess because postprocess renames columns
- 04:50 PM Revision 14587: exports/: svn:ignore: added *.gz
- 04:49 PM Revision 14586: lib/csvs.py: added ProgressInputFilter, analogous to streams.ProgressInputStream
- 04:46 PM Revision 14585: lib/sql_io.py: added commented-out debug statement used to troubleshoot copy_expert() errors
- 04:45 PM Revision 14584: lib/dicts.py: added pair_keys(), pair_values()
- 04:15 PM Revision 14583: bugfix: lib/streams.py: CaptureStream: end_idx must also be > start_idx
- 04:07 PM Revision 14582: bugfix: inputs/input.Makefile: $(import_install_): need `set -o pipefail` to enable errexit
- 03:47 AM Revision 14581: /README.TXT: to backup files not in Time Machine: don't need to review diff because command is unidirectional
- 02:59 AM Revision 14580: fix: /README.TXT: to back up the local machine's hard drive: "repeat until only minimal changes" should refer to the first sync command
- 02:52 AM Revision 14579: inputs/.geoscrub/geoscrub_output/run: documented postprocess() rm=1 runtime (6 min)
08/25/2014
- 10:17 PM Revision 14578: lib/tnrs.py: single_tnrs_request(): use_tnrs_export=False: need to obtain export columns
- 10:16 PM Revision 14577: lib/csvs.py: added header(stream)
- 10:16 PM Revision 14576: fix: lib/tnrs.py: single_tnrs_request(): need to `assert name_ct >= 1`, because with no names, TNRS hangs indefinitely
- 09:13 PM Revision 14575: bin/tnrs_client: added env var to configure use_tnrs_export
- 08:18 PM Revision 14574: /README.TXT: to back up vegbiendev: use inplace=1 to speed stopping and resuming transfer
- 07:54 PM Revision 14573: fix: /README.TXT: to back up the local machine's hard drive: removed --extended-attributes (after initial sync) because rsync apparently has to visit every file for this
- 07:35 PM Revision 14572: fix: /README.TXT: to back up the local machine's hard drive: also need --extended-attributes
- 07:34 PM Revision 14571: /README.TXT: to back up the local machine's hard drive: removed --delete-before now that that partition has been expanded
- 07:16 PM Revision 14570: fix: /README.TXT: to back up vegbiendev: exclude /var/lib/mysql.bak,postgresql.bak because the local machine doesn't need 2 copies of this information
- 07:05 PM Revision 14569: /README.TXT: to back up vegbiendev: removed no longer needed exclude of Dropbox subdir backup
- 06:58 PM Revision 14568: fix: /README.TXT: to back up vegbiendev: also need to do steps under Maintenance > "to synchronize vegbiendev, jupiter, and your local machine" because /home/aaronmk/bien is not synced here
- 06:52 PM Revision 14567: bugfix: /README.TXT: to back up vegbiendev: need `overwrite=1`
- 06:47 PM Revision 14566: /README.TXT: to back up vegbiendev: removed no longer needed exclude of Dropbox subdir backup
- 06:46 PM Revision 14565: /README.TXT: to back up the version history: don't also need this on vegbiendev because it's already on jupiter and the local machine
- 06:43 PM Revision 14564: bugfix: /README.TXT: to back up vegbiendev: need to include Postgres config files
- 06:24 PM Revision 14563: /README.TXT: to back up the local machine's hard drive: don't back up temp files: added /.fseventsd/
- 05:54 PM Revision 14562: fix: /README.TXT: to back up the local machine's hard drive: initial runtime: use range instead because some of the later runtime might have been from the same files
- 05:52 PM Revision 14561: /README.TXT: to back up the local machine's hard drive: updated initial runtime to include additional transferred files (17 h)
- 05:36 PM Revision 14560: fix: /README.TXT: to back up the local machine's hard drive: need to use --delete-before because the backup partition is near capacity
- 05:34 PM Revision 14559: /README.TXT: to back up the local machine's hard drive: don't back up temp files such as /private/var/vm/*
- 05:30 PM Revision 14558: fix: /README.TXT: to back up the local machine's hard drive: back up most Dropbox/Postgres files before stopping processes, to minimize downtime
08/21/2014
- 07:35 PM Revision 14557: bugfix: /README.TXT: to back up the local machine's hard drive: can't use ~ with --exclude
- 07:31 PM Revision 14556: fix: inputs/.geoscrub/geoscrub_output/postprocess.sql: map_geovalidity(): unscrubbable names should actually be geo*in*valid, not geovalid=NULL, according to Brad
- 07:24 PM Revision 14555: /README.TXT: to back up the local machine's hard drive: back up the non-Dropbox, non-Postgres files separately to minimize the Dropbox and Postgres downtime
- 06:03 PM Revision 14554: /README.TXT: to back up the vegbiendev databases: don't need to review diff for these as it's always unidirectional
- 05:55 PM Revision 14553: /README.TXT: added instructions to back up vegbiendev
- 05:12 PM Revision 14552: fix: /README.TXT: to back up the local machine's hard drive: also need to repeat backup command until only minimal changes
- 05:11 PM Revision 14551: /README.TXT: to back up the local machine's hard drive: added step to stop Postgres
- 05:10 PM Revision 14550: bugfix: /README.TXT: to back up the local machine's hard drive: also need to stop Dropbox
- 05:06 PM Revision 14549: /README.TXT: to back up the local machine's settings: added step to remove .DS_Store
- 04:47 PM Revision 14548: fix: /README.TXT: to back up the local machine's settings: Dropbox: shoudl not run with `del=`, because the backup should be an exact replica
- 04:25 PM Revision 14547: backups/TNRS.*: removed no longer needed old TNRS backups, which are part of the respective full-database backups in any case
- 02:57 PM Revision 14546: added config/phpMyAdmin/ symlink to schemas/VegCore/phpMyAdmin/
- 12:40 PM Revision 14545: bugfix: lib/sh/archives.sh: compress(): don't include dir prefix in zip archive
- 12:40 PM Revision 14544: lib/sh/util.sh: cd(): use echo_run instead of a manual echo_cmd call
- 12:35 PM Revision 14543: fix: lib/sh/util.sh: cd(): indent after running cd rather than before
- 12:32 PM Revision 14542: lib/sh/util.sh: cd(): support rebasing path vars for the new dir
- 11:51 AM Revision 14541: bugfix: lib/sh/archives.sh: compress(): need to use zip's path syntax to avoid the file in the archive being named "-"
- 08:56 AM Revision 14540: lib/tnrs.py: added option to avoid using TNRS's TSV export feature, which currently returns incorrect selected matches (vegpath.org/issues/943). this has been implemented up through the GWT/JSON decoding.
- 08:50 AM Revision 14539: lib/tnrs.py: added gwt_decode()
- 08:49 AM Revision 14538: lib/strings.py: added unesc_quotes() and helper functions
- 08:49 AM Revision 14537: lib/strings.py: added json_decode()
- 08:38 AM Revision 14536: /README.TXT: To re-run geoscrubbing: updated runtimes
- 08:25 AM Revision 14535: exports/*_GBIF.csv.run: documented compress_() runtime (20 min-1 h)
08/20/2014
- 11:12 PM Revision 14534: lib/runscripts/extract.run: export_(): also compress created file
- 11:11 PM Revision 14533: lib/sh/archives.sh: added compress(), expand(), which handle compression of individual files
- 04:26 PM Revision 14532: bugfix: inputs/input.Makefile: sql/install: ";" for commands inside $(if) blocks need to be inside the $(if) block, too, because otherwise there will be dangling ";" without a statement (bash does not support empty statements containing just ";")
- 11:29 AM Revision 14531: /README.TXT: Full database import: converted database commands to command-line commands to make them easier to run
- 11:28 AM Revision 14530: /README.TXT: Full database import: converted database commands to command-line commands to make them easier to run
- 10:55 AM Revision 14529: web/links/index.htm: updated to Firefox bookmarks: added instructions for how to enable automatic restart on power loss for the UPS (which isn't accessible in the GUI)
08/19/2014
- 05:13 PM Revision 14528: fix: schemas/util.sql: contained_within_approx(point geocoord, region postgis.geography): use util.geography() instead of implicit cast to suppress "Coordinate values were coerced into range [-180 -90, 180 90] for GEOGRAPHY" NOTICEs
- 05:10 PM Revision 14527: schemas/util.sql: added geography(util.geocoord), which suppresses "Coordinate values were coerced into range [-180 -90, 180 90] for GEOGRAPHY" NOTICEs
- 04:48 PM Revision 14526: exports/native_status_resolver.csv.run: updated export_() runtime (5 min, now that we're using the narrower New World criterion)
- 04:39 PM Revision 14525: fix: schemas/public_.sql: native_status_resolver: don't include rows with New World coordinates that don't also have New World country names, since the NSR only uses the country name
- 04:26 PM Revision 14524: schemas/public_.sql: native_status_resolver: removed rows with is_geovalid NULL, at Brad's request. note that this removes valid rows with standardized country names.
- 04:01 PM Revision 14523: exports/native_status_resolver.csv.run: updated export_() runtime (30 min)
- 03:28 PM Revision 14522: fix: schemas/public_.sql: native_status_resolver: added country IS NOT NULL filter requested by Brad
- 02:35 PM Revision 14521: fix: schemas/public_.sql: native_status_resolver: remove the id because this prevents SELECT DISTINCT from having the desired effect. instead, the results will be joined back using the other columns.
- 02:25 PM Revision 14520: exports/native_status_resolver.csv.run: upload_(): documented runtime (2.5 min)
- 02:18 PM Revision 14519: bugfix: exports/native_status_resolver.csv.run: upload_(): $live must be exported
- 02:15 PM Revision 14518: exports/native_status_resolver.csv.run: upload_(): use `live=1` instead for consistency with other invocations of put
- 02:14 PM Revision 14517: fix: exports/native_status_resolver.csv.run: upload_(): need `l=1`
- 02:12 PM Revision 14516: exports/native_status_resolver.csv.run: documented export_() runtime (45 min)
- 01:48 PM Revision 14515: exports/native_status_resolver.csv.run: added upload_() to get the file onto nimoy
- 01:25 PM Revision 14514: added exports/native_status_resolver.csv.run
- 01:23 PM Revision 14513: schemas/public_.sql: added native_status_resolver view, requested by Brad (wiki.vegpath.org/Data_requests)
- 11:47 AM Revision 14512: inputs/publishable datasources.xlsx: udpated
- 08:37 AM Revision 14511: lib/tnrs.py: documentation about output of the retrieve step: added that this is also unusable because the array does not contain all the columns and contains no column names
- 07:54 AM Revision 14510: removed no longer used web/BIEN3/Redmine/main/. use Redmine/!__ instead.
- 07:52 AM Revision 14509: web/BIEN3/Redmine/issues/.htaccess: perform .. redirects using new ! prefix
- 07:51 AM Revision 14508: web/BIEN3/Redmine/.htaccess: enable redirects that avoid using a subdir's .htaccess
- 07:46 AM Revision 14507: web/BIEN3/Redmine/wiki/.htaccess: removed no longer needed ignore_fs, since the .htaccess does not have RewriteRules that would need this in a RewriteCond
- 07:42 AM Revision 14506: web/BIEN3/Redmine/issues/.htaccess: main issues page: added default filter conditions
- 07:41 AM Revision 14505: bugfix: web/BIEN3/Redmine/issues/.htaccess: need to redirect to separate URL for individual issues, because they are not located under the issues/ subdir in Redmine
- 06:35 AM Revision 14504: added web/.issues symlink and dest dir (needed because Apache does not support dangling symlinks)
- 06:35 AM Revision 14503: web/BIEN3/Redmine/wiki/.htaccess: documented that this dir is needed because Apache does not support dangling symlinks
- 06:32 AM Revision 14502: bugfix: web/.htaccess: need to expand top-level symlinks to avoid RewriteBase issues
- 06:29 AM Revision 14501: web/main.conf: added RewriteMap for readlink
- 06:28 AM Revision 14500: added web/readlink
- 05:52 AM Revision 14499: web/links/index.htm: updated to Firefox bookmarks: updated favicons
- 05:47 AM Revision 14498: web/links/index.htm: updated to Firefox bookmarks: updated favicons
08/18/2014
- 04:52 PM Revision 14497: web/BIEN3/Redmine/wiki/.htaccess: just use this dir as symlink dest, since the dir name is the same as the URL path within Redmine
- 04:50 PM Revision 14496: web/.htaccess: don't rewrite existing files/dirs: allow forcing rewrite of existing things with %{ENV:ignore_fs}
- 03:41 PM Revision 14495: web/BIEN3/Redmine/svn-web/.htaccess: use Redmine/ instead of main/ subdir
- 03:37 PM Revision 14494: web/BIEN3/Redmine/.htaccess: point this to the Redmine root instead of to the wiki, to avoid the need to append /main
- 03:21 PM Revision 14493: backups/vegbien.r14089.backup.md5: updated
- 01:58 PM Revision 14492: inputs/.TNRS/schema.sql: taxon_match: added taxon_scrub_best_match_jerry_lu index to facilitate finding names affected by the match-picking bug (#943)
08/17/2014
- 12:04 PM Revision 14491: backups/TNRS.backup.md5: updated
- 07:49 AM Revision 14490: fix: inputs/HVAA/Specimen/postprocess.sql, map.csv: monthCollected/dayCollected: fix indefinite dates (which aren't supported by Postgres), as decided by Bob (https://docs.google.com/spreadsheets/d/1PI8n0CRttN7ttsXs5qfh5OFFzSoAfJj0gSbylgX6vj4/edit#gid=0)
- 07:23 AM Revision 14489: schemas/util.sql: added date_part_fix()
08/16/2014
- 05:55 PM Revision 14488: schemas/public_.sql: viewFullOccurrence_*: renamed to view_full_occurrence_* at Brian M's and Martha's request (e-mails from Martha on 2014-8-12 at 17:37PT, and from Brian M on 2014-8-13 at 16:21PT). note that this change has already been made on vegbiendev.
- 05:51 PM Revision 14487: schemas/public_.sql: added view_full_occurrence_individual_*GBIF, analytical_stem_*GBIF requested by Mark. note that this change has already been made on vegbiendev.
- 05:37 PM Revision 14486: schemas/VegBIEN/data_dictionary/VegBIEN data dictionary.xlsx: updated
- 05:15 PM Revision 14485: schemas/public_.sql: viewFullOccurrence_*: renamed to view_full_occurrence_* at Brian M's request (e-mail from Martha on 2014-8-12 at 17:37PT). note that this change has already been made on vegbiendev.
- 05:11 PM Revision 14484: schemas/public_.sql: view_full_occurrence_individual, analytical_stem: renamed columns to shortened names at Brian M's request (e-mail from Martha on 2014-8-12 at 17:37PT). note that this change has already been made on vegbiendev.
- 02:58 PM Revision 14483: schemas/public_.sql: analytical_stem_view: renamed to analytical_stem since this replaces the materialized analytical_stem table. note that this change has already been made on vegbiendev.
- 02:35 PM Revision 14482: schemas/public_.sql: viewFullOccurrence_*: renamed to view_full_occurrence_* at Brian M's request (e-mail from Martha on 2014-8-12 at 17:37PT). note that this change has already been made on vegbiendev.
- 01:20 PM Revision 14481: schemas/public_.sql: viewFullOccurrence_individual: re-ran viewFullOccurrence_individual_view_modify() to udpate this
- 01:15 PM Revision 14480: bugfix: schemas/util.sql: copy_struct(): also need to util.copy_comment() because CREATE TABLE __ INCLUDING ALL is missing this
- 01:11 PM Revision 14479: schemas/util.sql: added copy_comment(regclass, regclass)
- 12:29 PM Revision 14478: schemas/public_.sql: viewFullOccurrence_individual_view: CREATE INDEX runtime: documented 2-column runtime (~2 h)
- 12:06 PM Revision 14477: exports/analytical_stem_*GBIF.csv.run: documented runtime (40 min-1.2 h)
- 09:36 AM Revision 14476: exports/view_full_occurrence_individual_*GBIF.csv.run: documented runtime (20-30 min)
- 08:37 AM Revision 14475: added exports/analytical_stem_GBIF.csv.run, analytical_stem_non_GBIF.csv.run
08/15/2014
- 07:15 PM Revision 14474: added exports/view_full_occurrence_individual_GBIF.csv.run, view_full_occurrence_individual_non_GBIF.csv.run
- 04:13 PM Revision 14473: schemas/public_.sql: viewFullOccurrence_individual_view: added "after updating this" instructions
- 04:01 PM Revision 14472: schemas/public_.sql: viewFullOccurrence_individual_view: documented CREATE INDEX runtime (10 min - 1.5 h depending on the datatype and % populated)
- 03:32 PM Revision 14471: web/links/index.htm: updated to Firefox bookmarks: updated to use frenzy instead of starscream
08/14/2014
- 03:25 PM Revision 14470: fix: lib/tnrs.py: retrieval_request_template: source_sorting (Constrain by Source): corrected explanation to reflect that the behavior is actually the same in both modes, since only one match is ever marked as selected, and that match should always come first
- 10:50 AM Revision 14469: planning/meetings/BIEN conference call availability.xlsx: updated
08/11/2014
- 07:48 AM Revision 14468: inputs/.TNRS/schema.sql: taxon_match: added taxon_scrub_by_name index
- 07:46 AM Revision 14467: inputs/.TNRS/schema.sql: taxon_match: added taxon_scrub_by_family index
- 07:10 AM Revision 14466: inputs/.TNRS/schema.sql: taxon_match: added taxon_scrub_by_species_binomial index
- 05:28 AM Revision 14465: bugfix: schemas/public_.sql: *_view_modify(): need to set the search_path so that tables are created in the same schema as the function, rather than whichever schema happens to be at the beginning of the search_path
- 05:19 AM Revision 14464: bugfix: schemas/public_.sql: analytical_stem_view_modify(): updated to support being used directly instead of via a materialized table
- 05:13 AM Revision 14463: schemas/public_.sql: removed no longer used analytical_stem. use analytical_stem_view or viewFullOccurrence_individual instead.
- 05:08 AM Revision 14462: fix: schemas/public_.sql: viewfulloccurrence_individual_view_modify(): fixed capitalization in function name (needs "" to preserve case)
- 05:03 AM Revision 14461: schemas/public_.sql: other derived views: use viewFullOccurrence_individual instead of analytical_stem because analytical_stem is no longer materialized (viewFullOccurrence_individual_view is materialized instead)
- 05:01 AM Revision 14460: schemas/public_.sql: validation views: use analytical_stem_view instead of analytical_stem because analytical_stem is no longer materialized (viewFullOccurrence_individual_view is materialized instead)
- 04:57 AM Revision 14459: schemas/public_.sql: derived views: use analytical_stem_view instead of analytical_stem because analytical_stem is no longer materialized (viewFullOccurrence_individual_view is materialized instead)
- 04:46 AM Revision 14458: bugfix: schemas/util.sql: expansion_min_cols(): increased to properly handle all public-schema views (the largest is currently 2014-6-12.Jeff_Ott.climatic_range_determinants)
- 04:35 AM Revision 14457: schemas/util.sql: view_def_to_orig(): use new expansion_min_cols() for easier configuration
- 04:33 AM Revision 14456: schemas/util.sql: added expansion_min_cols() (the minimum # of cols from the same table to be treated as a * expression)
- 04:05 AM Revision 14455: fix: schemas/public_.sql: analytical_stem_view: regenerated columns list for viewFullOccurrence_individual (instead of viewFullOccurrence_individual_view)
- 03:40 AM Revision 14454: bugfix: mappings/VegCore-VegBIEN.csv: prefixed taxonomic ranks: use _concat_nullify() so that the prefix is only added if the epithet is non-NULL
- 03:29 AM Revision 14453: fix: schemas/util.sql: col_re(): support column names with " in them
- 02:59 AM Revision 14452: schemas/util.sql: added _concat_nullify(), which uses || instead of concat()
- 02:16 AM Revision 14451: bugfix: inputs/FIA/REF_RESEARCH_STATION/map.csv: mapped country, which is not provided in the FIA data
- 01:46 AM Revision 14450: schemas/public_.sql: removed no longer used threatened_taxonlabel_view. use iucn_red_list_view instead.
- 01:35 AM Revision 14449: schemas/public_.sql: viewFullOccurrence_individual_view: documented materialize time (22 h)
08/10/2014
- 05:53 AM Revision 14448: inputs/.TNRS/schema.sql: taxon_match: removed no longer used scrubbed_unique_taxon_name. the scrubbed name ranks are now generated from the other TNRS columns instead.
- 05:28 AM Revision 14447: /README.TXT: Full database import: added steps to re-run geoscrubbing
- 05:22 AM Revision 14446: exports/geoscrub_input.csv.run: export_(): updated runtime (30 s)
- 03:57 AM Revision 14445: fix: exports/viewFullOccurrence_*.csv.run: need to limit # rows to 5000 as requested by Bob
- 03:42 AM Revision 14444: schemas/public_.sql: viewFullOccurrence_*: documented materialize time
08/09/2014
- 10:23 PM Revision 14443: bin/make_analytical_db: materialize viewFullOccurrence_individual_view instead of analytical_stem_view because analytical_stem_view is now generatable via a simple join onto viewFullOccurrence_individual_view. this avoids running into potential disk space constraints when materializing and backing up both tables (~50 GB/table * 2 tables * 2 copies (incl. the backup) = 200 GB, which is very close to the available disk space).
- 10:17 PM Revision 14442: schemas/public_.sql: analytical_stem_view: use new materialized viewFullOccurrence_individual
- 10:08 PM Revision 14441: schemas/public_.sql: added viewFullOccurrence_individual by running viewFullOccurrence_individual_view_modify()
- 10:05 PM Revision 14440: schemas/public_.sql: added viewFullOccurrence_individual_view_modify(), analogous to analytical_stem_view_modify()
- 09:52 PM Revision 14439: bin/make_analytical_db: removed extra () around psql_verbose_vegbien
- 09:51 PM Revision 14438: bin/make_analytical_db: removed no longer used mk_table()
- 09:49 PM Revision 14437: bin/make_analytical_db: use more up-to-date *_view_modify() functions instead of mk_table()
- 08:46 PM Revision 14436: inputs/.TNRS/schema.sql: removed no longer used view ValidMatchedTaxon. use taxon_scrub instead.
- 08:44 PM Revision 14435: schemas/public_.sql: iucn_red_list_view: use taxon_scrub instead of ValidMatchedTaxon since they are equivalent
- 08:42 PM Revision 14434: inputs/.TNRS/schema.sql: taxon_scrub: use taxon_best_match directly, to avoid the need for a separate ValidMatchedTaxon view
- 08:25 PM Revision 14433: fix: inputs/.TNRS/schema.sql: taxon_scrub: merged synonymous columns
- 08:11 PM Revision 14432: schemas/vegbien.sql: taxon_scrub: documented steps to merge synonymous columns
- 07:45 PM Revision 14431: inputs/.TNRS/schema.sql: removed no longer used view MatchedTaxon. use taxon_best_match instead.
- 07:43 PM Revision 14430: inputs/.TNRS/schema.sql: ValidMatchedTaxon: use taxon_best_match now that it's equivalent to MatchedTaxon
- 07:38 PM Revision 14429: bugfix: schemas/public_.sql: tnrs_input_name: use taxon_match instead of taxon_best_match because there is no index on taxon_match that includes just the filters used by taxon_best_match
- 07:37 PM Revision 14428: schemas/public_.sql: tnrs_input_name: use taxon_best_match now that it's equivalent to MatchedTaxon
- 07:20 PM Revision 14427: backups/TNRS.backup.md5: updated
- 07:19 PM Revision 14426: fix: inputs/.TNRS/schema.sql: MatchedTaxon: merged synonymous columns
- 07:02 PM Revision 14425: fix: schemas/util.sql: view_is_subset_or_renaming(): views with CASE statements are also not subsets/renamings
- 06:52 PM Revision 14424: inputs/.TNRS/schema.sql: removed no longer used taxon_scrub.scrubbed_unique_taxon_name.* . use taxon_scrub instead.
- 06:50 PM Revision 14423: inputs/.TNRS/schema.sql: taxon_scrub: use taxon_match derived columns instead of the incorrect values in taxon_scrub.scrubbed_unique_taxon_name.* (which does not work with the multi-match strategy)
- 05:54 AM Revision 14422: inputs/.TNRS/schema.sql: MatchedTaxon: use derived columns from taxon_match. this also incorporates the fixes in the new derived columns.
- 05:27 AM Revision 14421: inputs/.TNRS/schema.sql: taxon_scrub: use derived columns from taxon_match. this also incorporates the fixes in the new derived columns.
08/08/2014
- 07:14 PM Revision 14420: added exports/viewFullOccurrence_{CVS,VegBank,NCU}.csv.run
- 07:13 PM Revision 14419: schemas/public_.sql: added viewFullOccurrence_{CVS,VegBank,NCU} for Bob
- 02:32 PM Revision 14418: bugfix: schemas/util.sql: added GRANT USAGE for bien_read, public_ so that util schema functions can be called by other users
08/05/2014
08/04/2014
- 06:25 AM Revision 14416: bugfix: inputs/.TNRS/schema.sql: removed no longer used derived column __accepted_infraspecific_label, which had a buggy formula that broke derived_cols_populate()
- 06:18 AM Revision 14415: bugfix: schemas/util.sql: drop_derived_col(): also need to run util.derived_cols_trigger_update()
- 06:17 AM Revision 14414: bugfix: inputs/.TNRS/schema.sql: removed no longer used derived column __accepted_infraspecific_label, which had a buggy formula that broke derived_cols_populate()
- 06:10 AM Revision 14413: fix: inputs/.TNRS/schema.sql: taxon_match: to remove a column: updated instructions
- 05:09 AM Revision 14412: bugfix: lib/sh/util.sh: str2varname(): need to lowercase str because on case-insensitive filesystems, paths sometimes canonicalize to a different capitalization than the original
- 05:00 AM Revision 14411: lib/sh/util.sh: added lowercase()
Also available in: Atom