bugfix: lib/util.py: use OrderedDict from collections rather than ordereddict to work with Mac OS X 10.8 Mountain Lion (http://vegpath.org/links/#OrderedDict)
/Makefile: apache-Linux: updated for Ubuntu 14.04, which includes Apache 2.4 in the distribution
/Makefile: install: documented that schemas/public/install also tests that a clean public schema will be installable by full-database import
/Makefile: install: skip the test step at the end because this is not needed for the installation (it can be run separately if desired)
bugfix: **/postprocess.sql: don't use the public schema, because this creates an unsatisfied dependency while the database is being installed, and breaks `make install`
schemas/public_.sql: _km_to_m(): made it a wrapper around util._km_to_m() so this function is only in one place
schemas/util.sql: added _km_to_m() for use when public._km_to_m() is not defined yet
bugfix: benign_does_not_exist_error(): removed ignore_e=3, because this exit status is also used for other errors
fix: lib/sh/db.sh: benign_does_not_exist_error(): use benign_error=1, which is now supported properly by stderr_matches()
bugfix: lib/sh/util.sh: stderr_matches(): support $benign_error properly, by handling exit status logging in this func instead
bugfix: lib/sh/db.sh: pg_schema_exists(): also need to benignify "does not exist" error if returns false
bugfix: lib/sh/util.sh: stderr_matches(): need to separately display errors that were incorrectly suppressed due to $benign_error
bugfix: lib/sh/util.sh: is_err(): rethrow must be inverted (rethrow->false if error)
lib/sh/util.sh: added is_err()
lib/sh/local.sh: public_schema_exists(): moved to lib/sh/db.sh since this no longer depends on BIEN-specific configurations
bugfix: lib/sh/db.sh: public_schema_exists(): don't hide the function call tree so it's clear which function is running the psql commands
fix: *{.sh,run}: stderr_matches() callers: added benign_error=1 where needed
fix: *{.sh,run}: stderr_matches() callers: usage: documented that they may require benign_error=1
fix: lib/sh/util.sh: stderr_matches(): usage: documented that this may require benign_error=1
fix: /README.TXT: Mac backup: exclude ~/VirtualBox VMs/Ubuntu/Ubuntu.vdi, to avoid it being re-uploaded twice each time, due to an rsync verification error (https://projects.nceas.ucsb.edu/nceas/issues/907)
bugfix: lib/sh/db.sh: pg_cmd(): updated for new echo_vars log_level
fix: lib/sh/db.sh: pg_schema_exists(): display the function name so it's clear which function is running the psql commands
fix: lib/sh/db.sh: pg_schema_exists(): don't use log++ because it hides the command that produces the benign error
/README.TXT: changed "then rerun with l=1 ..." to "then review diff, and rerun with `l=1` prepended" to ensure that user reviews diff before syncing
/README.TXT: to synchronize a Mac's settings with my testing machine's: removed separate step to upload just the VirtualBox VMs, because that is now part of the main upload
fix: /README.TXT: to synchronize a Mac's settings with my testing machine's: need to sync VirtualBox VMs with inplace=1 because they are very large files
web/links/index.htm: updated to Firefox bookmarks: Ubuntu 14.04 upgrade: rsync: documented that this breaks use of ~ to reference the remote user's home dir
/README.TXT: to back up the version history: back up first on the local machine, because often only the svnsync command gets run, and that way it will get backed up immediately to Dropbox (and hourly to Time Machine), while vegbiendev only gets backed up daily to tape
bugfix: /README.TXT: to back up the version history: use absolute path for vegbiendev commands because the Ubuntu 14.04 version of rsync doesn't expand ~ properly
/README.TXT: to back up the version history: use $HOME to make paths platform-independent
bugfix: exports/.rsync_filter.upload: *.csv: need to use `protect` rather than `hide` to prevent overwriting
fix: exports/.rsync_filter.upload: *.csv: don't allow test runs' exports to overwrite the backed up ones
fix: lib/sh/db.sh: psql(): removed debugging changes
bugfix: lib/sh/util.sh: highlight_log_msg(): when not can_highlight_log_msg, need to remove any surrounding formatting
fix: lib/sh/util.sh: die_error_hidden(): always log local vars at same log_level as echo_func
fix: *{.sh,run}: always log kw_params at same log_level as echo_func
lib/sh/util.sh: split_lines(): usage: matched up and synced different syntaxes
bugfix: lib/sh/util.sh: log_msg!(): split_lines does not support being invoked by wrapper; need to use `declare lines; wrapper "split_lines" str` instead
fix: lib/sh/util.sh: split_lines(): usage: documented different syntax for when using wrapper
bugfix: lib/sh/util.sh: die_error_hidden(): echo_func to assist debugging
bugfix: lib/sh/util.sh: split(): need to limit the effects of IFS to just the splitting, so it doesn't cause strange errors in other functions
bugfix: stderr2stdout(): fd 2 must be redirected back to fd 2, not log-filtered, in case there are other errors in addition to the benign error. this fixes a bug in pg_schema_exists(), where errors about the DB being down were not displayed because they were log-filtered out.
lib/sh/make.sh: set_make_vars: don't display make vars at verbosity 2 to avoid clutter
bugfix: lib/sh/make.sh: begin_target: don't echo_func twice
inputs/GBIF/_MySQL/.rsync_ignore: added GBIFPortalDB-*.data.sql.gz, because these are intermediate files
bugfix: /Makefile: $(pg_ctl-Darwin): need to call the command rather than echoing it, as is needed for the Linux version
bugfix: /Makefile: $(pg_ctl-Darwin): need to `cd /` because due to pg_ctl bug, current directory must be accessible by it
bugfix: lib/runscripts/util.run: a non-runscript should have all args passed to main(). this fixes a bug in backups/*_snapshot where "main" would need to be prepended to any args for the script to run correctly.
bugfix: lib/runscripts/util.run: $wrap_fn: invoked script must always run as runscript so that wrapped command is run
lib/runscripts/util.run: added $is_runscript, for use by $wrap_fn
bugfix: lib/runscripts/util.run: $wrap_fn: $top_script doesn't need to be world-executable for most uses of sudo (only if sudoing to non-root)
bin/in_place: diff: use --brief to avoid scanning the entire file for large files
bin/in_place: added $preserve_mtime flag
web/links/index.htm: updated to Firefox bookmarks: Ubuntu 14.04 upgrade: Apache: documented that MultiViews is actually only broken for redirects with the filename "index"
web/.htaccess: for dirs, redirect to index.*: document it is actually just the filename "index" that MultiViews is broken for, other filenames work fine
bugfix: web/index.php: full directory index: only display if invoked as "vegpath.org/", not "vegpath.org/index.php"
bugfix: web/.htaccess: for dirs, redirect to index.*: added workaround for Ubuntu 14.04, which breaks MultiViews
/Makefile: postgres-Linux: updated to use the official version that comes with Ubuntu 14.04
web/links/index.htm: updated to Firefox bookmarks: Ubuntu 14.04 upgrade: Apache: documented that this breaks MultiViews, so you need to rewrite .htaccess files to avoid using MultiViews
web/links/index.htm: updated to Firefox bookmarks: Ubuntu 14.04 upgrade: added Postgres upgrading instructions
_license/non-open-source/applies_to.txt: Brad: added "anything he created while not working for iPlant, from 2013-7-1..10-31"
web/links/index.htm: updated to Firefox bookmarks: Ubuntu 14.04 upgrade
web/links/index.htm: updated to Firefox bookmarks: Ubuntu 14.04 upgrade: added phpMyAdmin fixing instructions
web/links/index.htm: updated to Firefox bookmarks: Ubuntu: Ubuntu 14.04 upgrade: added things broken by it. PostgreSQL: fixed links.
/Makefile: postgres-Linux: added warning that the install commands were designed to run on Ubuntu 12.04, which is no longer the version used by vegbiendev (it is now 14.04)
backups/mysql_snapshot: documented initial vegbiendev->jupiter upload time for GBIF/raw_occurrence_record.MYD (7 h for 91 GB = 3.7 MB/s)
fix: web/links/index.htm: updated to Firefox bookmarks: PostgreSQL: backups: wal_keep_segments method: clarified how to determine the value of wal_keep_segments. filesystem-level backups: documented the advantages of filesystem-level backups over traditional database-level backups with pg_dump.
fix: web/links/index.htm: updated to Firefox bookmarks: PostgreSQL: backups: wal_keep_segments: restored annotations
web/links/index.htm: updated to Firefox bookmarks: PostgreSQL: backups: documented how to set up online and offline backups (with two possible approaches for online backups)
lib/sh/db.sh: pg_snapshot(): perform online backup if possible, to avoid needing to restart the server
lib/sh/db.sh: added pg_start_backup(), pg_stop_backup()
lib/sh/db.sh: psql(): only set redirs if can redirect
bugfix: psql(): when as_root is on, need to avoid redirections as these are not passed through by sudo
/README.TXT: use `sudo -u ... -i` instead of `sudo su - ...` to avoid using two commands to accomplish the login
bugfix: lib/sh/db.sh: psql(): don't use `--file /dev/fd/40` when can't redirect
fix: lib/sh/db.sh: psql(): when using sudo with custom $stdin, raise error that this is not supported
lib/sh/db.sh: psql(): $data_bypasses_filter: renamed to $data2stdout for clarity
lib/sh/db.sh: psql(): $bypass_ok: renamed to $can_redir for clarity
lib/sh/db.sh: psql(): usage: $stdin: documented that can also use process substitution for this
bugfix: lib/sh/util.sh: `type` calls: need -- before cmd in case it starts with -
lib/sh/util.sh: cmd2rel_path: use is_extern()
lib/sh/sync.sh: db_snapshot(): use `end_try` as specified in `try` usage
lib/sh/util.sh: try: usage: added location of finally block
schemas/postgresql.conf: wal_level: set to hot_standby to enable online backup with pg_start_backup()
lib/sh/sync.sh: upload(): always print the function and kw_params
added backups/mysql_snapshot, pg_snapshot
bugfix: lib/sh/util.sh: type(): need to handle options before command name
lib/sh/util.sh: added 1st_non_opt()
lib/sh/util.sh: unalias(): use self_builtin, which is now defined before it
lib/runscripts/util.run: sudo(): avoid slow $wrap_fn when using `command` (ie. always executable)
bugfix: lib/sh/util.sh: commands: `type` calls: need to account for the fact that any alias is already expanded
lib/sh/util.sh: functions: moved before commands since commands are more complex
lib/sh/sync.sh: db_copy() and callers: pass args as rsync options
fix: lib/sh/sync.sh: db_copy(): need to exclude files which prevent tape backup
lib/sh/db.sh: added pg_ctl(), pg_snapshot()