lib/sh/util.sh: log+(): use easier-to-understand log_local instead of prefix-assignments to limit assignments to the invoked command

lib/sh/util.sh: log+(): use easier-to-understand log_local instead of prefix-assignments to limit assignments to the invoked command

: use clog instead of "log*"

bugfix: lib/sh/util.sh: log+(): removed spurious ; between setting of PS4 and log_level, which was causing erratic mismatches between PS4 and log_level. (the ; caused $PS4 to be set in the caller when invoked via one of the clog* aliases, rather than being passed as a command-specific env var.)

lib/sh/util.sh: $verbosity: stay constant at what the user set it to instead of changing in tandem with $log_level, to facilitate debugging verbosity/log_level-related issues

lib/sh/util.sh: log+(): usage: use aliases instead of ""-ed function names

lib/sh/util.sh: log_err(): use red background for better visibility of errors, in the same way that lib/exc.py print_ex() does for column-based import

bugfix: lib/sh/util.sh: removed echo_func in functions used by log++, to avoid spurious highlighted output

lib/sh/util.sh: added missing clog+ alias

bugfix: lib/sh/util.sh: log_hint(): use the standard log_fd and log_info() format, not err_fd and log_err() format, for hint messages

fix: lib/sh/util.sh: log_msg!(): indent each line, not just the first

lib/sh/util.sh: added split_lines()

lib/sh/util.sh: log(): factored out helper function log_msg!()

fix: lib/sh/util.sh: highlight_msg(): bold instead of underlining because the underlining interferes with the readability of the commands

lib/sh/util.sh: highlight_msg(): allow turning off formatting w/ empty $format

fix: lib/sh/util.sh: log_err() calls: removed manual highlighting

lib/sh/util.sh: log_err(): highlight all error messages using highlight_msg()'s new $format

lib/sh/util.sh: highlight_msg(): support custom format

lib/sh/db.sh: pg_*_exists(): log the DB statements to check this at a higher log_level so that they don't clutter up the log output

lib/sh/util.sh: log(): highlight log_level 1 messages to stand out against other output, for easier debugging

*{.sh,run}: stderr_matches() wrapper calls: removed no longer needed prep_try/rethrow

bugfix: catch(): also need to support $1='' because this is a now a use case of ignore_e()

bugfix: lib/sh/util.sh: ignore_err_msg(): also need to ignore false exit status on no match

lib/sh/util.sh: stderr_matches(): moved prep_try/rethrow into the function itself so that callers don't have to wrap this function in a complex sequence of prep_try/rethrow statements

*{.sh,run}: stderr_matches() wrapper calls: removed no longer needed prep_try/rethrow

lib/sh/util.sh: stderr_matches(): moved prep_try/rethrow into the function itself so that callers don't have to wrap this function in a complex sequence of prep_try/rethrow statements

lib/sh/util.sh: added rethrow_exit alias

fix: lib/sh/db.sh: pg_table_exists(): use stderr_matches() rather than just the exit status. this also avoids highlighting the benign error.

fix: lib/sh/db.sh: pg_table_exists(): use stderr_matches() rather than just the exit status. this also avoids highlighting the benign error.

lib/sh/make.sh: begin_target: echo all targets to facilitate debugging without needing the verbose stack trace mode

bugfix: lib/sh/make.sh: echo_target: don't include filename/line #, since this is not for the stack trace mode

lib/sh/make.sh: added echo_target

*{.sh,run}: use new begin_target instead of `echo_func; set_make_vars`

lib/runscripts/util.run: runscript template: added sample make target, using new make target template

lib/sh/make.sh: added make target template

lib/sh/make.sh: added begin_target alias

lib/runscripts/datasrc_dir.run: documented how to reinstall staging tables (`rm=1 .../run import`)

bugfix: *{.sh,run}: stderr_matches() wrapper callers: use the required wrapper caller usage, which now includes rethrow and prep_try

bugfix: lib/sh/util.sh: rethrow*: only `return` if $e is actually nonzero, because rethrow is now being used as a catch-all in situations where there might not be an error

lib/sh/util.sh: prep_try: initialize $e to 0 to simplify error-handling coding

stderr_matches(): wrapper caller usage: added alternative usage when using `||`

lib/sh/util.sh: stderr_matches(): wrapper caller usage: documented usage for a negated condition (ie. prefixed w/ !)

lib/sh/util.sh: stderr_matches(): usage: split into wrapper usage and wrapper caller usage for clarity

fix: *{.sh,run}: stderr_matches() wrappers: usage: added `rethrow`

fix: *{.sh,run}: stderr_matches() wrappers: usage: added `rethrow`

fix: lib/sh/util.sh: stderr_matches(): usage: `rethrow` must be called right after stderr_matches(), to avoid calling running other commands if there is an error

fix: lib/sh/util.sh: stderr_matches(): when using $ignore_e, also set benign_error=1 to suppress the highlighting of the error

bugfix: lib/sh/db.sh: pg_schema_exists(): need to ignore benign error exit status from the "cannot create temporary relation in non-temporary schema" error

lib/sh/util.sh: stderr_matches(): supporting ignoring any benign error exit status associated with the error message being tested for

lib/sh/util.sh: stderr_matches(): usage: documented where any ignore_e statement would go

bugfix: lib/sh/util.sh: stderr_matches(): can't use `try` because this clears the exit status, which is needed for @PIPESTATUS to work. to support this, also need to avoid errexiting since @PIPESTATUS will be used instead.

lib/sh/util.sh: added dp(), which debug-prints a message

bugfix: lib/runscripts/table.run: custom_postprocess(): need to apply renames to SQL statements in postprocess.sql before it can be run

lib/sh/db.sh: psql(): $verbose_ok: renamed to $bypass_ok for clarity, because this applies only to the `--output /dev/fd/41` bypass (which when not possible, requires turning off verbose output

fix: lib/sh/db.sh: psql(): added $output_data switch analogous to what mysql() has. this causes query results of eg. void-returning functions to be correctly filtered by the logging mechanism, rather than output to stdout.

fix: lib/sh/db.sh: psql(): verbosity=0 (errors only) mode: use `SET client_min_messages = WARNING;` instead of NOTICE to hide verbose messages within psql as well

lib/sh/db.sh: psql(): replaced `test "$verbose_ok" && can_log` with bool var $verbose_

fix: lib/sh/db.sh: psql(): $verbose_: renamed to $verbose_ok for clarity

fix: lib/sh/util.sh: stdout_contains(): add another pipe_delay because the `grep` statement was sometimes getting printed before its filtered output

bugfix: lib/sh/util.sh: die_error_hidden(): min verbosity to display error should not be hardcoded

lib/sh/db.sh: psql(): "to see error details" msg: use new die_error_hidden()

lib/sh/util.sh: added die_error_hidden()

lib/sh/db.sh: psql(): "to see error details" msg: use new log_hint()

lib/sh/util.sh: added log_hint(), whose msg is only displayed if not a benign error

bugfix: lib/sh/db.sh: psql(): "to see error details" msg: also don't print it for benign errors ($benign_error)

lib/sh/db.sh: psql(): on error, display message describing how to see error details (prepend `vb=2` to the command)

bugfix: lib/sh/util.sh: log_err(): don't override verbosity manually, as this will not set log_level or PS4. instead, use new log! , which sets these correctly.

lib/sh/util.sh: added log! , which force-displays next log message

lib/sh/util.sh: save_e: made it idempotent so that it also works if save_e was already called

lib/sh/util.sh: save_e: made it idempotent so that it also works if save_e was already called

lib/sh/util.sh: rethrow: documented why can't use `(exit "$e")` (bash bug that prevents errexit)

bugfix: lib/runscripts/util.run: $auto_ignore: need to unexport it so don't pass this to invoked scripts except through fwd()

bugfix: lib/runscripts/util.run: auto_fwd's fallback() must be set after auto_ignore's fallback() to overwrite it (auto_ignore should only apply if an error would otherwise have been generated by the fallback)

lib/runscripts/util.run: fwd(): support subdirs that don't contain a runscript, so that the default value of @subdirs will work in most cases

lib/runscripts/util.run: fwd(): set default @subdirs (`{.,}*/`)

lib/sh/util.sh: added enter_top_dir and use it in in_top_dir

fix: lib/sh/util.sh: commands run inside $(...): need to run with log++ so that these aren't normally debug-printed

lib/sh/util.sh: added pv(), which debug-prints var(s)

lib/sh/util.sh: added wildcard.()

lib/sh/util.sh: added wildcard/()

lib/sh/util.sh: added esc_args()

lib/runscripts/datasrc_dir.run: removed postprocess(), which now does the same thing its auto-forwarded equivalent would

lib/runscripts/datasrc_dir.run: removed separate @table_subdirs, because the table-only targets can now safely be invoked on all subdirs, being auto-ignored in subdirs that don't support them

lib/runscripts/util.run: fwd(): enable $auto_ignore so that each subdir doesn't have to have a definition for the forwarded target

lib/runscripts/util.run: added $auto_ignore switch, which causes fallback() not to generate an error that a non-existant target doesn't exist

lib/runscripts/datasrc_dir.run: use new fwd_self alias

lib/runscripts/util.run: added fwd_self alias

lib/runscripts/datasrc_dir.run: enable $auto_fwd, to create the functionality of lib/forwarding.Makefile's `%` target

lib/runscripts/util.run: added $auto_fwd switch

bugfix: lib/runscripts/util.run: gateway(): need to use is_callable() rather than func_exists() to check whether the target exists, because external commands (eg. echo) are supported as targets, too

lib/sh/util.sh: added is_callable()

lib/runscripts/util.run: support custom handlers for all targets (gateway()) as well as targets w/o function (fallback())

lib/runscripts/table.run: remake_VegBIEN_mappings(): renamed to just mappings() since action make targets should be short names

lib/sh/util.sh: stderr_matches(): inline the stderr_matches alias to avoid needing to quote stderr_matches as "stderr_matches" in the most common use case (with pattern as a prefix env var)

bugfix: lib/sh/util.sh: stderr_matches(): when passing `pattern=...` as a prefix env var, must be invoked as `"stderr_matches"` to avoid the env var applying to the prep_try portion of the stderr_matches alias

fix: lib/phpPgAdmin.login.php.diff: "For user public_, leave password blank" instruction: moved this to right under the Password field and increased the font size so people would be more likely to see it

lib/Firefox_bookmarks.reformat.csv: "page's description": changed to "page's own description" to clarify that this is a description provided by the page itself

bugfix: lib/common.Makefile: `svn add`: need to run with --force because the new version of svn in the latest Mac upgrade errors if the file is already under version control

validation/aggregating/*/*.sql, schemas/vegbien.sql, lib/runscripts/validations.pg.sql.run, inputs/bien2_traits/validations.sql: added _ to beginning of each view name so the validation views would sort at the top in the datasource's tables list. this will also make the validation result sets easily distinguishable from the data tables.

lib/sh/util.sh: stderr_matches(): documented that the purpose of not redirecting fd 2 back to fd 2 is to allow log-filtering out an otherwise-confusing benign error