lib/sh/util.sh: import_vars: documented that it's idempotent

/Makefile: python-Linux: removed ordereddict, since OrderedDict is now obtained from `collections`

web/links/index.htm: updated to Firefox bookmarks: fixed broken links

bugfix: lib/util.py: use OrderedDict from collections rather than ordereddict to work with Mac OS X 10.8 Mountain Lion (http://vegpath.org/links/#OrderedDict)

/Makefile: apache-Linux: updated for Ubuntu 14.04, which includes Apache 2.4 in the distribution

/Makefile: install: documented that schemas/public/install also tests that a clean public schema will be installable by full-database import

/Makefile: install: skip the test step at the end because this is not needed for the installation (it can be run separately if desired)

bugfix: **/postprocess.sql: don't use the public schema, because this creates an unsatisfied dependency while the database is being installed, and breaks `make install`

schemas/public_.sql: _km_to_m(): made it a wrapper around util._km_to_m() so this function is only in one place

schemas/util.sql: added _km_to_m() for use when public._km_to_m() is not defined yet

bugfix: benign_does_not_exist_error(): removed ignore_e=3, because this exit status is also used for other errors

fix: lib/sh/db.sh: benign_does_not_exist_error(): use benign_error=1, which is now supported properly by stderr_matches()

bugfix: lib/sh/util.sh: stderr_matches(): support $benign_error properly, by handling exit status logging in this func instead

bugfix: lib/sh/db.sh: pg_schema_exists(): also need to benignify "does not exist" error if returns false

bugfix: lib/sh/util.sh: stderr_matches(): need to separately display errors that were incorrectly suppressed due to $benign_error

bugfix: lib/sh/util.sh: is_err(): rethrow must be inverted (rethrow->false if error)

lib/sh/util.sh: added is_err()

lib/sh/local.sh: public_schema_exists(): moved to lib/sh/db.sh since this no longer depends on BIEN-specific configurations

bugfix: lib/sh/db.sh: public_schema_exists(): don't hide the function call tree so it's clear which function is running the psql commands

bugfix: lib/sh/db.sh: public_schema_exists(): don't hide the function call tree so it's clear which function is running the psql commands

fix: *{.sh,run}: stderr_matches() callers: added benign_error=1 where needed

fix: *{.sh,run}: stderr_matches() callers: usage: documented that they may require benign_error=1

fix: lib/sh/util.sh: stderr_matches(): usage: documented that this may require benign_error=1

fix: /README.TXT: Mac backup: exclude ~/VirtualBox VMs/Ubuntu/Ubuntu.vdi, to avoid it being re-uploaded twice each time, due to an rsync verification error (https://projects.nceas.ucsb.edu/nceas/issues/907)

bugfix: lib/sh/db.sh: pg_cmd(): updated for new echo_vars log_level

fix: lib/sh/db.sh: pg_schema_exists(): display the function name so it's clear which function is running the psql commands

fix: lib/sh/db.sh: pg_schema_exists(): don't use log++ because it hides the command that produces the benign error

/README.TXT: changed "then rerun with l=1 ..." to "then review diff, and rerun with `l=1` prepended" to ensure that user reviews diff before syncing

/README.TXT: to synchronize a Mac's settings with my testing machine's: removed separate step to upload just the VirtualBox VMs, because that is now part of the main upload

fix: /README.TXT: to synchronize a Mac's settings with my testing machine's: need to sync VirtualBox VMs with inplace=1 because they are very large files

web/links/index.htm: updated to Firefox bookmarks: Ubuntu 14.04 upgrade: rsync: documented that this breaks use of ~ to reference the remote user's home dir

/README.TXT: to back up the version history: back up first on the local machine, because often only the svnsync command gets run, and that way it will get backed up immediately to Dropbox (and hourly to Time Machine), while vegbiendev only gets backed up daily to tape

bugfix: /README.TXT: to back up the version history: use absolute path for vegbiendev commands because the Ubuntu 14.04 version of rsync doesn't expand ~ properly

/README.TXT: to back up the version history: use $HOME to make paths platform-independent

bugfix: exports/.rsync_filter.upload: *.csv: need to use `protect` rather than `hide` to prevent overwriting

fix: exports/.rsync_filter.upload: *.csv: don't allow test runs' exports to overwrite the backed up ones

fix: lib/sh/db.sh: psql(): removed debugging changes

bugfix: lib/sh/util.sh: highlight_log_msg(): when not can_highlight_log_msg, need to remove any surrounding formatting

fix: lib/sh/util.sh: die_error_hidden(): always log local vars at same log_level as echo_func

fix: *{.sh,run}: always log kw_params at same log_level as echo_func

lib/sh/util.sh: split_lines(): usage: matched up and synced different syntaxes

bugfix: lib/sh/util.sh: log_msg!(): split_lines does not support being invoked by wrapper; need to use `declare lines; wrapper "split_lines" str` instead

fix: lib/sh/util.sh: split_lines(): usage: documented different syntax for when using wrapper

bugfix: lib/sh/util.sh: die_error_hidden(): echo_func to assist debugging

bugfix: lib/sh/util.sh: split(): need to limit the effects of IFS to just the splitting, so it doesn't cause strange errors in other functions

bugfix: stderr2stdout(): fd 2 must be redirected back to fd 2, not log-filtered, in case there are other errors in addition to the benign error. this fixes a bug in pg_schema_exists(), where errors about the DB being down were not displayed because they were log-filtered out.

lib/sh/make.sh: set_make_vars: don't display make vars at verbosity 2 to avoid clutter

bugfix: lib/sh/make.sh: begin_target: don't echo_func twice

inputs/GBIF/_MySQL/.rsync_ignore: added GBIFPortalDB-*.data.sql.gz, because these are intermediate files

bugfix: /Makefile: $(pg_ctl-Darwin): need to call the command rather than echoing it, as is needed for the Linux version

bugfix: /Makefile: $(pg_ctl-Darwin): need to `cd /` because due to pg_ctl bug, current directory must be accessible by it

bugfix: lib/runscripts/util.run: a non-runscript should have all args passed to main(). this fixes a bug in backups/*_snapshot where "main" would need to be prepended to any args for the script to run correctly.

bugfix: lib/runscripts/util.run: $wrap_fn: invoked script must always run as runscript so that wrapped command is run

lib/runscripts/util.run: added $is_runscript, for use by $wrap_fn

bugfix: lib/runscripts/util.run: $wrap_fn: $top_script doesn't need to be world-executable for most uses of sudo (only if sudoing to non-root)

bin/in_place: diff: use --brief to avoid scanning the entire file for large files

bin/in_place: added $preserve_mtime flag

web/links/index.htm: updated to Firefox bookmarks: Ubuntu 14.04 upgrade: Apache: documented that MultiViews is actually only broken for redirects with the filename "index"

web/.htaccess: for dirs, redirect to index.*: document it is actually just the filename "index" that MultiViews is broken for, other filenames work fine

bugfix: web/index.php: full directory index: only display if invoked as "vegpath.org/", not "vegpath.org/index.php"

bugfix: web/.htaccess: for dirs, redirect to index.*: added workaround for Ubuntu 14.04, which breaks MultiViews

/Makefile: postgres-Linux: updated to use the official version that comes with Ubuntu 14.04

web/links/index.htm: updated to Firefox bookmarks: Ubuntu 14.04 upgrade: Apache: documented that this breaks MultiViews, so you need to rewrite .htaccess files to avoid using MultiViews

web/links/index.htm: updated to Firefox bookmarks: Ubuntu 14.04 upgrade: added Postgres upgrading instructions

_license/non-open-source/applies_to.txt: Brad: added "anything he created while not working for iPlant, from 2013-7-1..10-31"

web/links/index.htm: updated to Firefox bookmarks: Ubuntu 14.04 upgrade

web/links/index.htm: updated to Firefox bookmarks: Ubuntu 14.04 upgrade: added phpMyAdmin fixing instructions

web/links/index.htm: updated to Firefox bookmarks: Ubuntu 14.04 upgrade: added phpMyAdmin fixing instructions

web/links/index.htm: updated to Firefox bookmarks: Ubuntu: Ubuntu 14.04 upgrade: added things broken by it. PostgreSQL: fixed links.

/Makefile: postgres-Linux: added warning that the install commands were designed to run on Ubuntu 12.04, which is no longer the version used by vegbiendev (it is now 14.04)

backups/mysql_snapshot: documented initial vegbiendev->jupiter upload time for GBIF/raw_occurrence_record.MYD (7 h for 91 GB = 3.7 MB/s)

fix: web/links/index.htm: updated to Firefox bookmarks: PostgreSQL: backups: wal_keep_segments method: clarified how to determine the value of wal_keep_segments. filesystem-level backups: documented the advantages of filesystem-level backups over traditional database-level backups with pg_dump.

fix: web/links/index.htm: updated to Firefox bookmarks: PostgreSQL: backups: wal_keep_segments: restored annotations

web/links/index.htm: updated to Firefox bookmarks: PostgreSQL: backups: documented how to set up online and offline backups (with two possible approaches for online backups)

web/links/index.htm: updated to Firefox bookmarks: PostgreSQL: backups: documented how to set up online and offline backups (with two possible approaches for online backups)

web/links/index.htm: updated to Firefox bookmarks: PostgreSQL: backups: documented how to set up online and offline backups (with two possible approaches for online backups)

lib/sh/db.sh: pg_snapshot(): perform online backup if possible, to avoid needing to restart the server

lib/sh/db.sh: added pg_start_backup(), pg_stop_backup()

lib/sh/db.sh: psql(): only set redirs if can redirect

bugfix: psql(): when as_root is on, need to avoid redirections as these are not passed through by sudo

/README.TXT: use `sudo -u ... -i` instead of `sudo su - ...` to avoid using two commands to accomplish the login

bugfix: lib/sh/db.sh: psql(): don't use `--file /dev/fd/40` when can't redirect

fix: lib/sh/db.sh: psql(): when using sudo with custom $stdin, raise error that this is not supported

lib/sh/db.sh: psql(): $data_bypasses_filter: renamed to $data2stdout for clarity

lib/sh/db.sh: psql(): $bypass_ok: renamed to $can_redir for clarity

lib/sh/db.sh: psql(): usage: $stdin: documented that can also use process substitution for this

bugfix: lib/sh/util.sh: `type` calls: need -- before cmd in case it starts with -

lib/sh/util.sh: cmd2rel_path: use is_extern()

lib/sh/sync.sh: db_snapshot(): use `end_try` as specified in `try` usage

lib/sh/util.sh: try: usage: added location of finally block

schemas/postgresql.conf: wal_level: set to hot_standby to enable online backup with pg_start_backup()

lib/sh/sync.sh: upload(): always print the function and kw_params

added backups/mysql_snapshot, pg_snapshot

bugfix: lib/sh/util.sh: type(): need to handle options before command name

lib/sh/util.sh: added 1st_non_opt()

lib/sh/util.sh: unalias(): use self_builtin, which is now defined before it

lib/runscripts/util.run: sudo(): avoid slow $wrap_fn when using `command` (ie. always executable)

lib/sh/util.sh: unalias(): use self_builtin, which is now defined before it

bugfix: lib/sh/util.sh: commands: `type` calls: need to account for the fact that any alias is already expanded

lib/sh/util.sh: functions: moved before commands since commands are more complex