bugfix: lib/sh/sync.sh, db.sh: *_snapshot() commands that run multiple sudo commands: use become_sudo so that the program won't hang at later sudo prompts. for pg_snapshot() with pg_start_backup(), this also prevents the backup from staying open at the end waiting on a sudo prompt, causing excess WAL logs to accrue.

fix: lib/sh/db.sh: pg_snapshot(): first need to run pg_stop_backup() in case a backup was not stopped cleanly

fix: lib/sh/db.sh: pg_stop_backup(): ignore benign "a backup is not in progress" error

fix: lib/sh/util.sh: already_exists_msg(): changed calling convention to avoid it seeming like `return 0` is run if already_exists_msg() throws an error, when in fact already_exists_msg() is just a command that should be run before returning/errexiting

lib/sh/db.sh: pg_cmd(): log vars on same line to avoid clutter

bugfix: lib/sh/db.sh: pg_dump(): need use_pg to import $pg_database before checking for existence of $database

lib/sh/util.sh: import_vars: documented that it's idempotent

bugfix: benign_does_not_exist_error(): removed ignore_e=3, because this exit status is also used for other errors

fix: lib/sh/db.sh: benign_does_not_exist_error(): use benign_error=1, which is now supported properly by stderr_matches()

bugfix: lib/sh/db.sh: pg_schema_exists(): also need to benignify "does not exist" error if returns false

lib/sh/local.sh: public_schema_exists(): moved to lib/sh/db.sh since this no longer depends on BIEN-specific configurations

fix: *{.sh,run}: stderr_matches() callers: usage: documented that they may require benign_error=1

bugfix: lib/sh/db.sh: pg_cmd(): updated for new echo_vars log_level

fix: lib/sh/db.sh: pg_schema_exists(): display the function name so it's clear which function is running the psql commands

fix: lib/sh/db.sh: pg_schema_exists(): don't use log++ because it hides the command that produces the benign error

fix: lib/sh/db.sh: psql(): removed debugging changes

fix: *{.sh,run}: always log kw_params at same log_level as echo_func

lib/sh/db.sh: pg_snapshot(): perform online backup if possible, to avoid needing to restart the server

lib/sh/db.sh: added pg_start_backup(), pg_stop_backup()

lib/sh/db.sh: psql(): only set redirs if can redirect

bugfix: psql(): when as_root is on, need to avoid redirections as these are not passed through by sudo

bugfix: lib/sh/db.sh: psql(): don't use `--file /dev/fd/40` when can't redirect

fix: lib/sh/db.sh: psql(): when using sudo with custom $stdin, raise error that this is not supported

lib/sh/db.sh: psql(): $data_bypasses_filter: renamed to $data2stdout for clarity

lib/sh/db.sh: psql(): $bypass_ok: renamed to $can_redir for clarity

lib/sh/db.sh: psql(): usage: $stdin: documented that can also use process substitution for this

lib/sh/sync.sh: db_copy() and callers: pass args as rsync options

lib/sh/db.sh: added pg_ctl(), pg_snapshot()

lib/sh/db.sh: mysql_snapshot(): use new db_snapshot()

lib/sh/db.sh: mysql_snapshot(): for large files, don't re-copy entire file

lib/sh/db.sh: mysql_snapshot(): use live mode as the default

fix: lib/sh/db.sh: mysql_snapshot(): need to create dest dir if doesn't exist

bugfix: lib/sh/db.sh: mysql_snapshot(): try: need to use split syntax with prep_try instead, to work with prefix vars

bugfix: lib/sh/db.sh: mysql_snapshot(): try: need to use split syntax with prep_try instead, to work with prefix vars

bugfix: lib/sh/db.sh: mysql_ctl(): need to ignore errors if not running

bugfix: lib/sh/db.sh: mysql_snapshot(): need to run `mysql_ctl start` even if there is an error

lib/sh/db.sh: mysql_snapshot(): $to: default to $from.bak

lib/sh/db.sh: added mysql_snapshot()

lib/sh/db.sh: added mysql_ctl()

lib/sh/db.sh: pg_cmd(): updated to use new sudo()

lib/sh/db.sh: pg_cmd(): $as_root: use $sudo

lib/sh/db.sh: removed no longer used pg_as_root(), which was buggy anyway. use `as_root=1 ...` instead.

lib/sh/db.sh: mysql_ANSI: fixed comment

lib/sh/db.sh: pg_dump(): support dumping entire cluster, and cluster users

lib/sh/db.sh: pg_cmd(): added $as_root switch

lib/sh/db.sh pg_dump(), bin/pg_dump_vegbien: --format=plain: removed comment that this is the plain format, because this is now self-documenting

lib/sh/db.sh pg_dump(), bin/pg_dump_vegbien: --format: use the long form of the formats to make the code self-documenting

bugfix: lib/sh/db.sh: pg_table_exists(): need ! to negate boolean result

fix: lib/sh/db.sh: pg_cmd(): hide PGPASSWORD at the normal verbosity so that the value of it doesn't appear in any log files

lib/sh/util.sh: clog*: renamed to log* for clarity (possible now that log* is no longer used for function-local log_level setting)

: local setting of log_level: use log_local instead of relying on the log aliases, so that these aliases can instead be used for wrapping commands (the more common use case)

lib/sh/db.sh: pg_*_exists(): log the DB statements to check this at a higher log_level so that they don't clutter up the log output

*{.sh,run}: stderr_matches() wrapper calls: removed no longer needed prep_try/rethrow

fix: lib/sh/db.sh: pg_table_exists(): use stderr_matches() rather than just the exit status. this also avoids highlighting the benign error.

stderr_matches(): wrapper caller usage: added alternative usage when using `||`

fix: *{.sh,run}: stderr_matches() wrappers: usage: added `rethrow`

bugfix: lib/sh/db.sh: pg_schema_exists(): need to ignore benign error exit status from the "cannot create temporary relation in non-temporary schema" error

lib/sh/db.sh: psql(): $verbose_ok: renamed to $bypass_ok for clarity, because this applies only to the `--output /dev/fd/41` bypass (which when not possible, requires turning off verbose output

fix: lib/sh/db.sh: psql(): added $output_data switch analogous to what mysql() has. this causes query results of eg. void-returning functions to be correctly filtered by the logging mechanism, rather than output to stdout.

fix: lib/sh/db.sh: psql(): verbosity=0 (errors only) mode: use `SET client_min_messages = WARNING;` instead of NOTICE to hide verbose messages within psql as well

lib/sh/db.sh: psql(): replaced `test "$verbose_ok" && can_log` with bool var $verbose_

fix: lib/sh/db.sh: psql(): $verbose_: renamed to $verbose_ok for clarity

bugfix: lib/sh/util.sh: die_error_hidden(): min verbosity to display error should not be hardcoded

lib/sh/db.sh: psql(): "to see error details" msg: use new die_error_hidden()

lib/sh/db.sh: psql(): "to see error details" msg: use new log_hint()

bugfix: lib/sh/db.sh: psql(): "to see error details" msg: also don't print it for benign errors ($benign_error)

lib/sh/db.sh: psql(): on error, display message describing how to see error details (prepend `vb=2` to the command)

lib/sh/util.sh: stderr_matches(): inline the stderr_matches alias to avoid needing to quote stderr_matches as "stderr_matches" in the most common use case (with pattern as a prefix env var)

bugfix: lib/sh/util.sh: stderr_matches(): when passing `pattern=...` as a prefix env var, must be invoked as `"stderr_matches"` to avoid the env var applying to the prep_try portion of the stderr_matches alias

*{.sh,run}: stderr_matches calls: don't need to wrap the command in `"try"` because stderr_matches now does this

lib/sh/db.sh: pg_dump(): ignore "No matching tables were found" error, using new ignore_err_msg()

lib/sh/db.sh: pg_schema_exists(): documented that `try` is used to suppress the error exit status

moved everything into /trunk/ to create the standard svn layout, for use with tools that require this (eg. git-svn). IMPORTANT: do NOT do an `svn up`. instead, re-use your working copy's existing files with `svn switch` (http://svnbook.red-bean.com/en/1.6/svn.ref.svn.c.switch.html).

lib/sh/db.sh: added pg_require_schema()

lib/sh/db.sh: added pg_schema_exists()

fix: lib/sh/db.sh: pg_table_exists(): usage: documented that $table is actually required for this function

lib/sh/util.sh: import_vars: don't overwrite vars that are already defined, to allow the caller to specify their own values for the vars to create. this requires callers that rely on the overwriting functionality to reverse the order in which they run use_* commands, so that the higher-precedence use_* is applied first and the other one as the default values for the first.

lib/sh/db.sh: pg_table_exists(): use `SELECT NULL` instead of `SELECT *` to avoid a long column list cluttering up the log output

lib/sh/db.sh: added pg_table_exists()

lib/sh/db.sh: mk_select(): usage: documented that this also takes a $limit/$n param

lib/sh/db.sh: limit(): also support using $n as the limit param, since this var name is used by other parts of the import process

lib/sh/db.sh: limit(): usage: documented that this also need a $limit param

bugfix: lib/sh/db.sh: avoid outputting to /dev/fd/# when running as sudo on Linux, because this causes a "Permission denied" error (due to the /dev/fd/# file being owned by a different user). this is not a problem with normal redirects (>&#), because they do not use /dev/fd/# files which can have access permissions.

lib/sh/db.sh: pg_as_root(): run sudo with echo_run to help debug

bugfix: lib/sh/db.sh: pg_cmd(): only set PG* connection/login env vars when the corresponding var is non-empty. there are some situations in which these must be unset (in order to use the default value), and other situations when the var must be set to something (i.e. "") to avoid it being defaulted to a value in local.sh > connection vars.

bugfix: lib/sh/local.sh: pg_as_root(): need to use -E (preserve environment) option to sudo, so that $schema, $table get passed through

lib/sh/db.sh: added pg_as_root()

lib/sh/db.sh: mysql_cmd(): added caller usage with connection/login opts

lib/sh/db.sh: mysql(), mysql_export(): usage: added database=...

lib/sh/db.sh: mk_select(): added support for ORDER BY

lib/sh/db.sh: added pg_export_table_to_dir(), analogous to pg_export_table_to_dir_no_header()

bugfix: lib/sh/db.sh: pg_dump(): don't default $struct flag to on, because both structure and data should be printed by default

lib/sh/db.sh: pg_dump(): added create_schema= flag to remove CREATE SCHEMA statements (useful if the schema already exists)

lib/sh/db.sh: added pg_dump(), using the code in bin/pg_dump_vegbien with clarity improvements

lib/sh/db.sh: added pg_cmd() (analogous to mysql_cmd() for PostgreSQL), and use it in psql(), so that other PostgreSQL operations can use this to set the PG* connection/login vars

lib/sh/db.sh: pg_export(): added usage

lib/sh/db.sh: mysqldump(): added create_db=1 flag to print the CREATE DATABASE statement

lib/sh/db.sh: mysqldump(): don't use --compatible=postgresql when the table structure is being exported, because this removes the table options (which include the COMMENT attribute). --compatible=postgresql remains on in data-only mode because embedded ` in data cannot easily be distinguished from ` around column names, so ANSI_QUOTES is needed to do the translation to " (and data sections do not contain table options). note that all --compatible modes that offer ANSI_QUOTES unfortunately exclude the table options, and there is no way to run a SQL query to set the SQL mode before beginning the dump, so ANSI_QUOTES translation must be handled by my2pg instead.

lib/sh/db.sh: mk_select(): support passing $cols as array instead of SQL string, which is easier to enter in a shell script (less quotes, \ , etc.)

lib/sh/db.sh: added cols2list()