bugfix: lib/sh/db.sh: public_schema_exists(): don't hide the function call tree so it's clear which function is running the psql commands
fix: *{.sh,run}: stderr_matches() callers: added benign_error=1 where needed
fix: *{.sh,run}: stderr_matches() callers: usage: documented that they may require benign_error=1
fix: lib/sh/util.sh: stderr_matches(): usage: documented that this may require benign_error=1
fix: /README.TXT: Mac backup: exclude ~/VirtualBox VMs/Ubuntu/Ubuntu.vdi, to avoid it being re-uploaded twice each time, due to an rsync verification error (https://projects.nceas.ucsb.edu/nceas/issues/907)
bugfix: lib/sh/db.sh: pg_cmd(): updated for new echo_vars log_level
fix: lib/sh/db.sh: pg_schema_exists(): display the function name so it's clear which function is running the psql commands
fix: lib/sh/db.sh: pg_schema_exists(): don't use log++ because it hides the command that produces the benign error
/README.TXT: changed "then rerun with l=1 ..." to "then review diff, and rerun with `l=1` prepended" to ensure that user reviews diff before syncing
/README.TXT: to synchronize a Mac's settings with my testing machine's: removed separate step to upload just the VirtualBox VMs, because that is now part of the main upload
fix: /README.TXT: to synchronize a Mac's settings with my testing machine's: need to sync VirtualBox VMs with inplace=1 because they are very large files
web/links/index.htm: updated to Firefox bookmarks: Ubuntu 14.04 upgrade: rsync: documented that this breaks use of ~ to reference the remote user's home dir
/README.TXT: to back up the version history: back up first on the local machine, because often only the svnsync command gets run, and that way it will get backed up immediately to Dropbox (and hourly to Time Machine), while vegbiendev only gets backed up daily to tape
bugfix: /README.TXT: to back up the version history: use absolute path for vegbiendev commands because the Ubuntu 14.04 version of rsync doesn't expand ~ properly
/README.TXT: to back up the version history: use $HOME to make paths platform-independent
bugfix: exports/.rsync_filter.upload: *.csv: need to use `protect` rather than `hide` to prevent overwriting
fix: exports/.rsync_filter.upload: *.csv: don't allow test runs' exports to overwrite the backed up ones
fix: lib/sh/db.sh: psql(): removed debugging changes
bugfix: lib/sh/util.sh: highlight_log_msg(): when not can_highlight_log_msg, need to remove any surrounding formatting
fix: lib/sh/util.sh: die_error_hidden(): always log local vars at same log_level as echo_func
fix: *{.sh,run}: always log kw_params at same log_level as echo_func
lib/sh/util.sh: split_lines(): usage: matched up and synced different syntaxes
bugfix: lib/sh/util.sh: log_msg!(): split_lines does not support being invoked by wrapper; need to use `declare lines; wrapper "split_lines" str` instead
fix: lib/sh/util.sh: split_lines(): usage: documented different syntax for when using wrapper
bugfix: lib/sh/util.sh: die_error_hidden(): echo_func to assist debugging
bugfix: lib/sh/util.sh: split(): need to limit the effects of IFS to just the splitting, so it doesn't cause strange errors in other functions
bugfix: stderr2stdout(): fd 2 must be redirected back to fd 2, not log-filtered, in case there are other errors in addition to the benign error. this fixes a bug in pg_schema_exists(), where errors about the DB being down were not displayed because they were log-filtered out.
lib/sh/make.sh: set_make_vars: don't display make vars at verbosity 2 to avoid clutter
bugfix: lib/sh/make.sh: begin_target: don't echo_func twice
inputs/GBIF/_MySQL/.rsync_ignore: added GBIFPortalDB-*.data.sql.gz, because these are intermediate files
bugfix: /Makefile: $(pg_ctl-Darwin): need to call the command rather than echoing it, as is needed for the Linux version
bugfix: /Makefile: $(pg_ctl-Darwin): need to `cd /` because due to pg_ctl bug, current directory must be accessible by it
bugfix: lib/runscripts/util.run: a non-runscript should have all args passed to main(). this fixes a bug in backups/*_snapshot where "main" would need to be prepended to any args for the script to run correctly.
bugfix: lib/runscripts/util.run: $wrap_fn: invoked script must always run as runscript so that wrapped command is run
lib/runscripts/util.run: added $is_runscript, for use by $wrap_fn
bugfix: lib/runscripts/util.run: $wrap_fn: $top_script doesn't need to be world-executable for most uses of sudo (only if sudoing to non-root)
bin/in_place: diff: use --brief to avoid scanning the entire file for large files
bin/in_place: added $preserve_mtime flag
web/links/index.htm: updated to Firefox bookmarks: Ubuntu 14.04 upgrade: Apache: documented that MultiViews is actually only broken for redirects with the filename "index"
web/.htaccess: for dirs, redirect to index.*: document it is actually just the filename "index" that MultiViews is broken for, other filenames work fine
bugfix: web/index.php: full directory index: only display if invoked as "vegpath.org/", not "vegpath.org/index.php"
bugfix: web/.htaccess: for dirs, redirect to index.*: added workaround for Ubuntu 14.04, which breaks MultiViews
/Makefile: postgres-Linux: updated to use the official version that comes with Ubuntu 14.04
web/links/index.htm: updated to Firefox bookmarks: Ubuntu 14.04 upgrade: Apache: documented that this breaks MultiViews, so you need to rewrite .htaccess files to avoid using MultiViews
web/links/index.htm: updated to Firefox bookmarks: Ubuntu 14.04 upgrade: added Postgres upgrading instructions
_license/non-open-source/applies_to.txt: Brad: added "anything he created while not working for iPlant, from 2013-7-1..10-31"
web/links/index.htm: updated to Firefox bookmarks: Ubuntu 14.04 upgrade
web/links/index.htm: updated to Firefox bookmarks: Ubuntu 14.04 upgrade: added phpMyAdmin fixing instructions
web/links/index.htm: updated to Firefox bookmarks: Ubuntu: Ubuntu 14.04 upgrade: added things broken by it. PostgreSQL: fixed links.
/Makefile: postgres-Linux: added warning that the install commands were designed to run on Ubuntu 12.04, which is no longer the version used by vegbiendev (it is now 14.04)
backups/mysql_snapshot: documented initial vegbiendev->jupiter upload time for GBIF/raw_occurrence_record.MYD (7 h for 91 GB = 3.7 MB/s)
fix: web/links/index.htm: updated to Firefox bookmarks: PostgreSQL: backups: wal_keep_segments method: clarified how to determine the value of wal_keep_segments. filesystem-level backups: documented the advantages of filesystem-level backups over traditional database-level backups with pg_dump.
fix: web/links/index.htm: updated to Firefox bookmarks: PostgreSQL: backups: wal_keep_segments: restored annotations
web/links/index.htm: updated to Firefox bookmarks: PostgreSQL: backups: documented how to set up online and offline backups (with two possible approaches for online backups)
lib/sh/db.sh: pg_snapshot(): perform online backup if possible, to avoid needing to restart the server
lib/sh/db.sh: added pg_start_backup(), pg_stop_backup()
lib/sh/db.sh: psql(): only set redirs if can redirect
bugfix: psql(): when as_root is on, need to avoid redirections as these are not passed through by sudo
/README.TXT: use `sudo -u ... -i` instead of `sudo su - ...` to avoid using two commands to accomplish the login
bugfix: lib/sh/db.sh: psql(): don't use `--file /dev/fd/40` when can't redirect
fix: lib/sh/db.sh: psql(): when using sudo with custom $stdin, raise error that this is not supported
lib/sh/db.sh: psql(): $data_bypasses_filter: renamed to $data2stdout for clarity
lib/sh/db.sh: psql(): $bypass_ok: renamed to $can_redir for clarity
lib/sh/db.sh: psql(): usage: $stdin: documented that can also use process substitution for this
bugfix: lib/sh/util.sh: `type` calls: need -- before cmd in case it starts with -
lib/sh/util.sh: cmd2rel_path: use is_extern()
lib/sh/sync.sh: db_snapshot(): use `end_try` as specified in `try` usage
lib/sh/util.sh: try: usage: added location of finally block
schemas/postgresql.conf: wal_level: set to hot_standby to enable online backup with pg_start_backup()
lib/sh/sync.sh: upload(): always print the function and kw_params
added backups/mysql_snapshot, pg_snapshot
bugfix: lib/sh/util.sh: type(): need to handle options before command name
lib/sh/util.sh: added 1st_non_opt()
lib/sh/util.sh: unalias(): use self_builtin, which is now defined before it
lib/runscripts/util.run: sudo(): avoid slow $wrap_fn when using `command` (ie. always executable)
bugfix: lib/sh/util.sh: commands: `type` calls: need to account for the fact that any alias is already expanded
lib/sh/util.sh: functions: moved before commands since commands are more complex
lib/sh/sync.sh: db_copy() and callers: pass args as rsync options
fix: lib/sh/sync.sh: db_copy(): need to exclude files which prevent tape backup
lib/sh/db.sh: added pg_ctl(), pg_snapshot()
lib/sh/sync.sh: db_snapshot(): copy changes before stopping DB to minimize the time that it's shut down
lib/sh/sync.sh: db_snapshot(): factored copy operation out into separate db_copy() function
lib/sh/db.sh: mysql_snapshot(): use new db_snapshot()
lib/sh/sync.sh: added db_snapshot()
lib/Firefox_bookmarks.reformat.csv: changed "page's own description" to "page's self-description" for clarity
web/links/index.htm: updated to Firefox bookmarks: removed dead links
web/links/index.htm: updated to Firefox bookmarks: updated favicons
web/links/index.htm: updated to Firefox bookmarks: modifying a running shell script: updated to document that `svn up` actually does use two-stage save automatically
lib/sh/db.sh: mysql_snapshot(): for large files, don't re-copy entire file
lib/sh/db.sh: mysql_snapshot(): use live mode as the default
fix: lib/sh/db.sh: mysql_snapshot(): need to create dest dir if doesn't exist
bugfix: lib/sh/db.sh: mysql_snapshot(): try: need to use split syntax with prep_try instead, to work with prefix vars
fix: lib/sh/util.sh: try usage: documented that the split syntax with prep_try is meant to be used with vars before the cmd
fix: lib/sh/util.sh: echo_vars(): also need to print unset vars (including unset kw_params)
lib/sh/util.sh: echo_vars(): put loop var on same line as `for`
