bugfix: /README.TXT: to back up the local machine's hard drive: can't use ~ with --exclude

fix: inputs/.geoscrub/geoscrub_output/postprocess.sql: map_geovalidity(): unscrubbable names should actually be geo*in*valid, not geovalid=NULL, according to Brad

/README.TXT: to back up the local machine's hard drive: back up the non-Dropbox, non-Postgres files separately to minimize the Dropbox and Postgres downtime

/README.TXT: to back up the vegbiendev databases: don't need to review diff for these as it's always unidirectional

/README.TXT: added instructions to back up vegbiendev

fix: /README.TXT: to back up the local machine's hard drive: also need to repeat backup command until only minimal changes

/README.TXT: to back up the local machine's hard drive: added step to stop Postgres

bugfix: /README.TXT: to back up the local machine's hard drive: also need to stop Dropbox

/README.TXT: to back up the local machine's settings: added step to remove .DS_Store

fix: /README.TXT: to back up the local machine's settings: Dropbox: shoudl not run with `del=`, because the backup should be an exact replica

backups/TNRS.*: removed no longer needed old TNRS backups, which are part of the respective full-database backups in any case

added config/phpMyAdmin/ symlink to schemas/VegCore/phpMyAdmin/

bugfix: lib/sh/archives.sh: compress(): don't include dir prefix in zip archive

lib/sh/util.sh: cd(): use echo_run instead of a manual echo_cmd call

fix: lib/sh/util.sh: cd(): indent after running cd rather than before

lib/sh/util.sh: cd(): support rebasing path vars for the new dir

bugfix: lib/sh/archives.sh: compress(): need to use zip's path syntax to avoid the file in the archive being named "-"

lib/tnrs.py: added option to avoid using TNRS's TSV export feature, which currently returns incorrect selected matches (vegpath.org/issues/943). this has been implemented up through the GWT/JSON decoding.

lib/tnrs.py: added gwt_decode()

lib/strings.py: added unesc_quotes() and helper functions

lib/strings.py: added json_decode()

/README.TXT: To re-run geoscrubbing: updated runtimes

exports/*_GBIF.csv.run: documented compress_() runtime (20 min-1 h)

lib/runscripts/extract.run: export_(): also compress created file

lib/sh/archives.sh: added compress(), expand(), which handle compression of individual files

bugfix: inputs/input.Makefile: sql/install: ";" for commands inside $(if) blocks need to be inside the $(if) block, too, because otherwise there will be dangling ";" without a statement (bash does not support empty statements containing just ";")

/README.TXT: Full database import: converted database commands to command-line commands to make them easier to run

/README.TXT: Full database import: converted database commands to command-line commands to make them easier to run

web/links/index.htm: updated to Firefox bookmarks: added instructions for how to enable automatic restart on power loss for the UPS (which isn't accessible in the GUI)

fix: schemas/util.sql: contained_within_approx(point geocoord, region postgis.geography): use util.geography() instead of implicit cast to suppress "Coordinate values were coerced into range [-180 -90, 180 90] for GEOGRAPHY" NOTICEs

schemas/util.sql: added geography(util.geocoord), which suppresses "Coordinate values were coerced into range [-180 -90, 180 90] for GEOGRAPHY" NOTICEs

exports/native_status_resolver.csv.run: updated export_() runtime (5 min, now that we're using the narrower New World criterion)

fix: schemas/public_.sql: native_status_resolver: don't include rows with New World coordinates that don't also have New World country names, since the NSR only uses the country name

schemas/public_.sql: native_status_resolver: removed rows with is_geovalid NULL, at Brad's request. note that this removes valid rows with standardized country names.

exports/native_status_resolver.csv.run: updated export_() runtime (30 min)

fix: schemas/public_.sql: native_status_resolver: added country IS NOT NULL filter requested by Brad

fix: schemas/public_.sql: native_status_resolver: remove the id because this prevents SELECT DISTINCT from having the desired effect. instead, the results will be joined back using the other columns.

exports/native_status_resolver.csv.run: upload_(): documented runtime (2.5 min)

bugfix: exports/native_status_resolver.csv.run: upload_(): $live must be exported

exports/native_status_resolver.csv.run: upload_(): use `live=1` instead for consistency with other invocations of put

fix: exports/native_status_resolver.csv.run: upload_(): need `l=1`

exports/native_status_resolver.csv.run: documented export_() runtime (45 min)

exports/native_status_resolver.csv.run: added upload_() to get the file onto nimoy

added exports/native_status_resolver.csv.run

schemas/public_.sql: added native_status_resolver view, requested by Brad (wiki.vegpath.org/Data_requests)

inputs/publishable datasources.xlsx: udpated

lib/tnrs.py: documentation about output of the retrieve step: added that this is also unusable because the array does not contain all the columns and contains no column names

removed no longer used web/BIEN3/Redmine/main/. use Redmine/!__ instead.

web/BIEN3/Redmine/issues/.htaccess: perform .. redirects using new ! prefix

web/BIEN3/Redmine/.htaccess: enable redirects that avoid using a subdir's .htaccess

web/BIEN3/Redmine/wiki/.htaccess: removed no longer needed ignore_fs, since the .htaccess does not have RewriteRules that would need this in a RewriteCond

web/BIEN3/Redmine/issues/.htaccess: main issues page: added default filter conditions

bugfix: web/BIEN3/Redmine/issues/.htaccess: need to redirect to separate URL for individual issues, because they are not located under the issues/ subdir in Redmine

added web/.issues symlink and dest dir (needed because Apache does not support dangling symlinks)

web/BIEN3/Redmine/wiki/.htaccess: documented that this dir is needed because Apache does not support dangling symlinks

bugfix: web/.htaccess: need to expand top-level symlinks to avoid RewriteBase issues

web/main.conf: added RewriteMap for readlink

added web/readlink

web/links/index.htm: updated to Firefox bookmarks: updated favicons

web/links/index.htm: updated to Firefox bookmarks: updated favicons

web/BIEN3/Redmine/wiki/.htaccess: just use this dir as symlink dest, since the dir name is the same as the URL path within Redmine

web/.htaccess: don't rewrite existing files/dirs: allow forcing rewrite of existing things with %{ENV:ignore_fs}

web/BIEN3/Redmine/svn-web/.htaccess: use Redmine/ instead of main/ subdir

web/BIEN3/Redmine/.htaccess: point this to the Redmine root instead of to the wiki, to avoid the need to append /main

backups/vegbien.r14089.backup.md5: updated

inputs/.TNRS/schema.sql: taxon_match: added taxon_scrub_best_match_jerry_lu index to facilitate finding names affected by the match-picking bug (#943)

backups/TNRS.backup.md5: updated

fix: inputs/HVAA/Specimen/postprocess.sql, map.csv: monthCollected/dayCollected: fix indefinite dates (which aren't supported by Postgres), as decided by Bob (https://docs.google.com/spreadsheets/d/1PI8n0CRttN7ttsXs5qfh5OFFzSoAfJj0gSbylgX6vj4/edit#gid=0)

schemas/util.sql: added date_part_fix()

schemas/public_.sql: viewFullOccurrence_*: renamed to view_full_occurrence_* at Brian M's and Martha's request (e-mails from Martha on 2014-8-12 at 17:37PT, and from Brian M on 2014-8-13 at 16:21PT). note that this change has already been made on vegbiendev.

schemas/public_.sql: added view_full_occurrence_individual_*GBIF, analytical_stem_*GBIF requested by Mark. note that this change has already been made on vegbiendev.

schemas/VegBIEN/data_dictionary/VegBIEN data dictionary.xlsx: updated

schemas/public_.sql: viewFullOccurrence_*: renamed to view_full_occurrence_* at Brian M's request (e-mail from Martha on 2014-8-12 at 17:37PT). note that this change has already been made on vegbiendev.

schemas/public_.sql: view_full_occurrence_individual, analytical_stem: renamed columns to shortened names at Brian M's request (e-mail from Martha on 2014-8-12 at 17:37PT). note that this change has already been made on vegbiendev.

schemas/public_.sql: analytical_stem_view: renamed to analytical_stem since this replaces the materialized analytical_stem table. note that this change has already been made on vegbiendev.

schemas/public_.sql: viewFullOccurrence_*: renamed to view_full_occurrence_* at Brian M's request (e-mail from Martha on 2014-8-12 at 17:37PT). note that this change has already been made on vegbiendev.

schemas/public_.sql: viewFullOccurrence_individual: re-ran viewFullOccurrence_individual_view_modify() to udpate this

bugfix: schemas/util.sql: copy_struct(): also need to util.copy_comment() because CREATE TABLE __ INCLUDING ALL is missing this

schemas/util.sql: added copy_comment(regclass, regclass)

schemas/public_.sql: viewFullOccurrence_individual_view: CREATE INDEX runtime: documented 2-column runtime (~2 h)

exports/analytical_stem_*GBIF.csv.run: documented runtime (40 min-1.2 h)

exports/view_full_occurrence_individual_*GBIF.csv.run: documented runtime (20-30 min)

added exports/analytical_stem_GBIF.csv.run, analytical_stem_non_GBIF.csv.run

added exports/view_full_occurrence_individual_GBIF.csv.run, view_full_occurrence_individual_non_GBIF.csv.run

schemas/public_.sql: viewFullOccurrence_individual_view: added "after updating this" instructions

schemas/public_.sql: viewFullOccurrence_individual_view: documented CREATE INDEX runtime (10 min - 1.5 h depending on the datatype and % populated)

web/links/index.htm: updated to Firefox bookmarks: updated to use frenzy instead of starscream

fix: lib/tnrs.py: retrieval_request_template: source_sorting (Constrain by Source): corrected explanation to reflect that the behavior is actually the same in both modes, since only one match is ever marked as selected, and that match should always come first

planning/meetings/BIEN conference call availability.xlsx: updated

inputs/.TNRS/schema.sql: taxon_match: added taxon_scrub_by_name index

inputs/.TNRS/schema.sql: taxon_match: added taxon_scrub_by_family index

inputs/.TNRS/schema.sql: taxon_match: added taxon_scrub_by_species_binomial index

bugfix: schemas/public_.sql: *_view_modify(): need to set the search_path so that tables are created in the same schema as the function, rather than whichever schema happens to be at the beginning of the search_path

bugfix: schemas/public_.sql: analytical_stem_view_modify(): updated to support being used directly instead of via a materialized table

schemas/public_.sql: removed no longer used analytical_stem. use analytical_stem_view or viewFullOccurrence_individual instead.

fix: schemas/public_.sql: viewfulloccurrence_individual_view_modify(): fixed capitalization in function name (needs "" to preserve case)

schemas/public_.sql: other derived views: use viewFullOccurrence_individual instead of analytical_stem because analytical_stem is no longer materialized (viewFullOccurrence_individual_view is materialized instead)

schemas/public_.sql: validation views: use analytical_stem_view instead of analytical_stem because analytical_stem is no longer materialized (viewFullOccurrence_individual_view is materialized instead)

schemas/public_.sql: derived views: use analytical_stem_view instead of analytical_stem because analytical_stem is no longer materialized (viewFullOccurrence_individual_view is materialized instead)

bugfix: schemas/util.sql: expansion_min_cols(): increased to properly handle all public-schema views (the largest is currently 2014-6-12.Jeff_Ott.climatic_range_determinants)