lib/sh/util.sh: highlight_msg(): support custom format

lib/sh/util.sh: log(): highlight log_level 1 messages to stand out against other output, for easier debugging

bugfix: catch(): also need to support $1='' because this is a now a use case of ignore_e()

bugfix: lib/sh/util.sh: ignore_err_msg(): also need to ignore false exit status on no match

lib/sh/util.sh: stderr_matches(): moved prep_try/rethrow into the function itself so that callers don't have to wrap this function in a complex sequence of prep_try/rethrow statements

lib/sh/util.sh: stderr_matches(): moved prep_try/rethrow into the function itself so that callers don't have to wrap this function in a complex sequence of prep_try/rethrow statements

lib/sh/util.sh: added rethrow_exit alias

bugfix: lib/sh/util.sh: rethrow*: only `return` if $e is actually nonzero, because rethrow is now being used as a catch-all in situations where there might not be an error

lib/sh/util.sh: prep_try: initialize $e to 0 to simplify error-handling coding

stderr_matches(): wrapper caller usage: added alternative usage when using `||`

lib/sh/util.sh: stderr_matches(): wrapper caller usage: documented usage for a negated condition (ie. prefixed w/ !)

lib/sh/util.sh: stderr_matches(): usage: split into wrapper usage and wrapper caller usage for clarity

fix: lib/sh/util.sh: stderr_matches(): usage: `rethrow` must be called right after stderr_matches(), to avoid calling running other commands if there is an error

fix: lib/sh/util.sh: stderr_matches(): when using $ignore_e, also set benign_error=1 to suppress the highlighting of the error

lib/sh/util.sh: stderr_matches(): supporting ignoring any benign error exit status associated with the error message being tested for

lib/sh/util.sh: stderr_matches(): usage: documented where any ignore_e statement would go

bugfix: lib/sh/util.sh: stderr_matches(): can't use `try` because this clears the exit status, which is needed for @PIPESTATUS to work. to support this, also need to avoid errexiting since @PIPESTATUS will be used instead.

lib/sh/util.sh: added dp(), which debug-prints a message

fix: lib/sh/util.sh: stdout_contains(): add another pipe_delay because the `grep` statement was sometimes getting printed before its filtered output

bugfix: lib/sh/util.sh: die_error_hidden(): min verbosity to display error should not be hardcoded

lib/sh/util.sh: added die_error_hidden()

lib/sh/util.sh: added log_hint(), whose msg is only displayed if not a benign error

bugfix: lib/sh/util.sh: log_err(): don't override verbosity manually, as this will not set log_level or PS4. instead, use new log! , which sets these correctly.

lib/sh/util.sh: added log! , which force-displays next log message

lib/sh/util.sh: save_e: made it idempotent so that it also works if save_e was already called

lib/sh/util.sh: save_e: made it idempotent so that it also works if save_e was already called

lib/sh/util.sh: rethrow: documented why can't use `(exit "$e")` (bash bug that prevents errexit)

lib/sh/util.sh: added enter_top_dir and use it in in_top_dir

fix: lib/sh/util.sh: commands run inside $(...): need to run with log++ so that these aren't normally debug-printed

lib/sh/util.sh: added pv(), which debug-prints var(s)

lib/sh/util.sh: added wildcard.()

lib/sh/util.sh: added wildcard/()

lib/sh/util.sh: added esc_args()

lib/sh/util.sh: added is_callable()

lib/sh/util.sh: stderr_matches(): inline the stderr_matches alias to avoid needing to quote stderr_matches as "stderr_matches" in the most common use case (with pattern as a prefix env var)

bugfix: lib/sh/util.sh: stderr_matches(): when passing `pattern=...` as a prefix env var, must be invoked as `"stderr_matches"` to avoid the env var applying to the prep_try portion of the stderr_matches alias

lib/sh/util.sh: stderr_matches(): documented that the purpose of not redirecting fd 2 back to fd 2 is to allow log-filtering out an otherwise-confusing benign error

lib/sh/util.sh: ignore_err_msg(): documented that unlike `|| true`, this suppresses only errors caused by a particular error message, rather than all error exit statuses

lib/sh/util.sh: stdout_contains(): usage: documented that this requires a `{ ... } 41>&1` wrapper

lib/sh/util.sh: stderr2stdout(): clarified that fd 2 is not redirected back to fd 2

bugfix: lib/sh/util.sh: stderr_matches(): need to avoid redirecting stderr and stdout to the same place, because this prevents redirecting stdout back to the original stdout after stderr has been filtered using |

lib/sh/util.sh: local_export_array: renamed to just export_array because this was a replacement for export, not local_export

lib/sh/util.sh: local_*array: don't need -a because that it's an array is autodetected by the ()

lib/sh/util.sh: stderr_matches(): actually don't need to declare PIPESTATUS_ in a separate command, because local does support arrays

lib/sh/util.sh: added local_export_array

fix: lib/sh/util.sh: local_array: clarified that this is only needed for older versions of bash (the lack of support for arrays has apparently been fixed)

lib/sh/util.sh: added local_array

fix: lib/sh/util.sh: stderr_matches(): usage: documented that now need to manually rethrow any command error, if applicable

*{.sh,run}: stderr_matches calls: don't need to wrap the command in `"try"` because stderr_matches now does this

fix: lib/sh/util.sh: `shopt -s lastpipe`: suppress error message if not supported

lib/sh/util.sh: ignore_err_msg(): use new stderr_matches alias, which includes prep_try (requires loading new aliases)

bugfix: lib/sh/util.sh: stderr_matches(): need to save PIPESTATUS and then use the saved var because it's reset after each cmd

lib/sh/util.sh: stderr_matches(): place exit status of cmd in $e for use with exception handling

lib/sh/util.sh: set `shopt -s lastpipe`, to allow setting vars in the last command of a pipeline

bugfix: lib/sh/util.sh: ignore_err_msg(): use `try` properly with prep_try and `"try"`

lib/sh/util.sh: added prep_try and use it in try alias. try: documented how to run it with a wrapper command.

lib/sh/util.sh: added ignore_err_msg()

lib/sh/util.sh: stderr_matches(): usage: surrounded command in if statement to indicate what context it would usually be used in

moved everything into /trunk/ to create the standard svn layout, for use with tools that require this (eg. git-svn). IMPORTANT: do NOT do an `svn up`. instead, re-use your working copy's existing files with `svn switch` (http://svnbook.red-bean.com/en/1.6/svn.ref.svn.c.switch.html).

fix: lib/sh/util.sh: verbosity_min(): usage: clarified that '' is a special value that causes $verbosity to be overwritten to ''

lib/sh/util.sh: stderr2stdout(): documented that this redirects fd 2->1 and log_fd (but not back to 2)

bugfix: lib/sh/util.sh: stderr2stdout() use `command` before tee, which re-filters log_fd so that stderr itself is also filtered. this allows log-filtering out an otherwise-confusing benign error when using e.g. stderr_matches().

lib/sh/util.sh: added not(), for use in prefixing wrapped commands

lib/sh/util.sh: added stderr_matches()

lib/sh/util.sh: documented that fds 2x/3x should not be used because we use these, as opposed to 1x which is used by the shell internally

lib/sh/util.sh: added stdout_contains()

lib/sh/util.sh: added stderr2stdout()

lib/sh/util.sh: import_vars: don't overwrite vars that are already defined, to allow the caller to specify their own values for the vars to create. this requires callers that rely on the overwriting functionality to reverse the order in which they run use_* commands, so that the higher-precedence use_* is applied first and the other one as the default values for the first.

lib/sh/util.sh: already_exists_msg(): added instructions on how to force-remake when the file already exists (prepend `rm=1` to the command)

lib/sh/util.sh: added require_dot_script()

bugfix: lib/sh/util.sh: $top_script: use @BASH_SOURCE instead of $0, because this is also defined for .-scripts

bugfix: lib/sh/util.sh: is_dot_script(): need to subtract 1 from ${#BASH_LINENO[@]}, because this is the array length rather than the index of the last element as in Perl

lib/sh/util.sh: added is_dot_script()

bugfix: lib/sh/util.sh: alias_append(): need to enclose $(alias) call in "" because its result may contain separator chars (i.e. whitespace) that will be parsed incorrectly. this appears to only be a bug when runscripts are run as shell-includes, with a leading ".".

lib/sh/util.sh: added instructions for making an export only visible locally

bugfix: lib/sh/util.sh: set_fds(): don't add surrounding quotes to empty redirect dest

bugfix: lib/sh/util.sh: set_fds(): need to check if redirect is empty before escaping it with `printf %q`, which may add surrounding quotes to an empty string

bugfix: lib/sh/util.sh: set_fds(): need to escape redirect destinations which are files, because they may contain special shell characters

lib/sh/util.sh: added rm_prefix()

lib/sh/util.sh: is_array(): handle unset vars (=false). this fixes a bug in pg_export_table_no_header, which produced the error "lib/sh/util.sh: line 290: declare: cols: not found".

fix: lib/sh/util.sh: join(): documented that delim must be a single char

bugfix: lib/sh/util.sh: set_fds(): remove empty redirects resulting from using `redirs= cmd...` to clear the redirs and then using $redirs as an array

fix: lib/sh/util.sh: set_fds(): documented that it does not currently support redirecting an fd to itself (due to bash bugs that require the dest fd to be closed before it can be reopened)

bugfix: lib/sh/util.sh: stdout2fd(): don't add >&$fd redirect if the fd is 1, because redir does not currently support redirecting an fd to itself (due to bash bugs that require the dest fd to be closed before it can be reopened)

lib/sh/util.sh: filter_fd(): factored out >() subshell command into stdout2fd() for clarity

bugfix: lib/sh/util.sh: redir(): unset redirs so that you don't redirect again in the invoked command

fix: lib/sh/util.sh: filter_fd(): documented that ${redirs[@]} must not be set to an outer value

bugfix: lib/sh/util.sh: pf(): echo func decl to stderr instead of stdout

lib/sh/util.sh: echo_vars(): documented that it only prints vars that are defined

lib/sh/util.sh: local_inv: also echo_vars the new var

lib/sh/util.sh: $sed_cmd: added usage

bugfix: lib/sh/util.sh: $sed_cmd: make output unbuffered, so that running e.g. bin/my2pg at the command line produces output as each line is read

lib/sh/util.sh: added is_array()

lib/sh/util.sh: moved runscript-related commands to lib/runscripts/util.run because these only apply to runscripts

bugfix: lib/sh/util.sh: sed: must use alias instead of function because function causes segfault in redir() subshell when used with make.sh make() filter (may be bug in bash?). this involves translating `unset LANG` to `env LANG=` (`env -u` to unset a var isn't supported on Mac, but fortunately sed treats LANG="" the same as unset LANG).

bugfix: lib/sh/util.sh: sed: unset LANG to avoid "illegal byte sequence" errors on invalid UTF-8 for LANG=*.UTF-8. these occur e.g. with MySQL data that is in Latin-1.

lib/sh/util.sh: sed: use function instead of alias so that env can be set up before calling sed

*{.sh,run}: use new log-() instead of log+() with a negative #

lib/sh/util.sh: added log-() because it's non-obvious that you would otherwise have to invoke log+() with a negative #

lib/sh/util.sh: added canon_dir_rel_path(), which canonicalizes just the parent dir if the path is a symlink, to leave the symlink itself untouched