lib/sh/util.sh: clog*: renamed to log* for clarity (possible now that log* is no longer used for function-local log_level setting)
*{.sh,run}: stderr_matches() wrapper calls: removed no longer needed prep_try/rethrow
stderr_matches(): wrapper caller usage: added alternative usage when using `||`
fix: *{.sh,run}: stderr_matches() wrappers: usage: added `rethrow`
moved everything into /trunk/ to create the standard svn layout, for use with tools that require this (eg. git-svn). IMPORTANT: do NOT do an `svn up`. instead, re-use your working copy's existing files with `svn switch` (http://svnbook.red-bean.com/en/1.6/svn.ref.svn.c.switch.html).
lib/sh/local.sh: public_schema_exists(): use a higher log_level for pg_schema_exists, to avoid all the verbose output involved in running the query
bugfix: lib/sh/local.sh: public_schema_exists(): can no longer use psql_script_vegbien for this, because using `SET search_path` (called by psql_script_vegbien) with a schema that does not exist no longer produces an error. instead, use new pg_schema_exists(), which uses a different command that does produce an error if the schema does not exist.
lib/sh/local.sh: psql(): also accept $public as the $schema param, since this is used by a lot of import scripts
bugfix: lib/sh/local.sh: psql(): $is_root: use `` around case statement instead of $(), because it contains an embedded unbalanced )
bugfix: lib/sh/local.sh: psql(): don't default the connection vars using use_local if running as the postgres user. in that case, connection must happen via a socket, with server="", and as the user running the command (postgres), with user="".
bugfix: lib/sh/local.sh: psql(): only \set schema, table if $schema, $table are non-empty, because otherwise, you will get a "zero-length delimited identifier" error
lib/sh/local.sh: added require_remote()
lib/sh/local.sh: added pg_dump_local()
bugfix: lib/sh/local.sh: prevent automated tests when the public schema contains the live DB, so the user doesn't have to explicitly specify can_test= when running the import on vegbiendev
lib/sh/local.sh: added PATH_add= flag to allow turning off the addition of $bin_dir_abs to the PATH. this is useful for wrapper scripts that should not alter the PATH passed to their invoked command.
lib/sh/local.sh: $sync_remote_url: allow user to override just the sync subdir (not the whole URL) in $sync_remote_subdir. this is useful e.g. for backing up the Mac's files to jupiter.
bugfix: lib/sh/local.sh: sync_upload(): need to use --no-group to prevent the group from being reset to aaronmk upon download from jupiter (which uses group aaronmk instead of bien). use ./fix_perms to set the group of all files to bien. also use --no-owner in case running as root.
lib/sh/sync.sh: removed sync_download(). use swap=1 sync_upload() instead.
bugfix: lib/sh/local.sh: removed make() override, which is no longer needed now that its operations are performed by verbosity_compat(), and which caused errors by setting $verbosity to the invalid value ""
bugfix: lib/sh/local.sh: make(): when invoking overridden func, need make__make_sh
lib/sh/local.sh: added make() override, which uses the default verbosity (i.e. verbosity="") when verbosity == 1. scripts that use lib/sql.py (which uses $verbosity) have different default verbosities, and this default should not be overriden by an env var, unless a higher verbosity has been set.
lib/sh/local.sh: added missing include of make.sh, used by root_make()
*{.sh,run}: use simpler .rel() instead of `. "$(dirname "${BASH_SOURCE0}")"/...` for relative includes
lib/sh/local.sh: psql(): don't put util in the search_path because psql scripts now add it themselves if they need it, using `SELECT util.search_path_append(util);`
lib/sh/local.sh: added sync_upload(), sync_download() with $sync_local_dir, $sync_remote_url config vars
bugfix: lib/sh/local.sh: root_rel_path(): added echo_func
bugfix: lib/sh/local.sh: root_rel_path(): use canon_rel_path instead of rel_path because $1 may be absolute rather than relative to the current dir, so $root_dir needs to be absoluted (which requires $1 to be absoluted as well)
lib/sh/local.sh: added root_rel_path()
lib/sh/local.sh: psql(): use new psql() from db.sh instead of psql_script_vegbien/psql_verbose_vegbien. this requires setting local_pg_database=vegbien to replace vegbien_dest used by psql_*_vegbien.
bugfix: lib/sh/local.sh: psql(): $file can't both be passed as a --file param and be prefixed with the necessary \set schema, etc. commands, so instead include $file when cat-ing stdin
lib/sh/local.sh: database connection vars: connect to vegbiendev via ssh and run commands locally, to allow running commands as root (which can only connect to the database locally). this effectively requires an ssh account on vegbiendev, but any ssh account (including an anonymous one, if we set one up) will do. this causes schemas/VegCore/VegCore.my.sql, VegCore.pg.sql to change, because they are now created by mysqldump running on vegbiendev (Linux) instead of on a Mac.
lib/sh/local.sh: added mysql_local()
lib/sh/local.sh: added mysql_root()
lib/sh/local.sh: added $root_user, $root_password
lib/sh/local.sh: psql(): documented that --output is actually for query results, not echoed statements (and thus must be redirected back to fd 1 while fd 1 with the statements gets sent to the logging port)
lib/sh/local.sh: psql(): documented why can't use fd 11
lib/sh/local.sh: use @redirs instead of manual redirection to set up --output fd, so that the redirection will be echoed along with the command. for some reason, this requires switching to fd 13 instead of 11, because fd 11 gives a "/dev/fd/11: Bad file descriptor" error when 11 is set with exec right before the command instead of on the subshell the command is executed in. (13 was chosen rather than 12 because 2 is for errors, while *3 (or 3) is for logging.)
lib/sh/local.sh: psql(): use $schema_esc, $table_esc instead of just putting $schema, $table in ""
bugfix: lib/sh/db.sh: log_sql(): use can_log() instead because the verbosity now gets decremented as the log_level increases, so the threshold to compare to is 0 instead of 2
bugfix: lib/sh/local.sh: add a manual errexit for $() exprs by embedding them in just a var assignment (without local or declare), whose exit status will then equal the of the $(). a `|| return` also needs to be added because errexit does not work on assignment statements. this commit adds them for func_loc(), echo_func(), canon_rel_path(), set_paths(), save_cache, cached realpath(), local.sh global vars
lib/sh/local.sh: $root_dir: run canon_rel_path on it
lib/sh/util.sh: renamed params() to kw_params() to clarify the type of params it handles
: use new `params` in all functions that have keyword params, in order to remove their vars from the environment. note that functions that use $? (such as die()) must save it *before running params, because params will overwrite $?.
lib/sh/util.sh: command(): use just one control var $cmd_log_fd instead of three flags (limit_log_fd, limit_stdout, stderr_is_errors) that indicated various common fd configurations. this is much clearer (you state which fd the common uses as its logging fd), more configurable (the logging fd can be any fd, not just 1 or 2), and more automatic (redirecting fd 2 to err_fd happens automatically if it isn't used for logging).
bugfix: lib/sh/local.sh: psql(): need to use limit_stdout=1 instead of stderr_is_errors=1 because logging info is output to stdout, not stderr, in spite of the >&2 redirection, since the >&2 is applied after command()'s logging redirects. note that the >&2 can now be removed, because it will be performed by command().
lib/sh/local.sh: added mysql_export_local()
lib/sh/local.sh: psql(): use new stderr_is_errors=1 since psql's logging output comes on stdout, so stderr contains only errors. (the caveat of this is that the query output is normally mixed with logging info on stdout, and needs to be separated out to a different file descriptor using --output.)
lib/sh/local.sh: psql(): use new convention for command-specific alternate stdout
*{.sh,run}: removed extra space between function name and (), which is apparently not needed even though `help function` includes it. this greatly improves readability, including when function names are pasted into commit messages!
*{.sh,run}: use shorter `test` instead of `test -n` and `test !` instead of `test -z` (http://www.gnu.org/software/bash/manual/bash.html#Bourne-Shell-Builtins > test)
lib/sh/local.sh: added public_schema_exists (), ported from lib/import.sh
lib/sh/util.sh: split databases utils out into separate db.sh
moved lib/*.sh to sh/ subdir so it's easier to find the .sh files among all the other lib/ files
lib/local.sh: removed mysql () since its functionality is now provided by mysql () in util.sh
*{.sh,run}: put functions on one line where possible (and where they are not expected to expand)
lib/local.sh: root_make (): use make () rather than extern make now that make () does not set its own current dir. using make () also automatically enables \$stdout2stderr, which is needed to redirect the echoed make commands.
lib/local.sh: added root_make ()
lib/local.sh: psql (): use fd 4 for the logging output instead of 3 because 3 is generally used for passing inline input commands (it is used this way by inline_make)
bugfix: lib/local.sh: $PATH: use $bin_dir_abs so the path will remain valid if the current directory is changed
lib/local.sh: added \$bin_dir_abs
bugfix: lib/local.sh: updated $root_dir
renamed lib/runscripts/local.run to lib/local.sh since the things it defines are not just for runscripts
lib/runscripts/local.run: added mysqldump_local, which uses vegbiendev
lib/runscripts/local.run: use postgres_compat mode because all our MySQL schemas should be usable as inputs to my2pg
lib/runscripts/local.run: added connection vars for local (bien) user
*{.sh,run}: changed echo_func to an alias that includes the "$", so that callers don't need to specify the "$" manually. although the original echo_func function is still there, callers need to switch over to the alias at the same time as it's defined because otherwise the "$@" would be doubled, since echo_func refers to the alias instead.
", so that callers don't need to specify the "$
*{.sh,run}: use just env instead of echo_run env now that env is an auto-echoing alias
bugfix: *{.sh,run}: use `local` when setting default values for params to prevent these values from leaking into the calling context, where it may have unexpected effects (e.g. pg_copy_to would permanently set $source, which would then be used on the next invocation of pg_copy_to instead of regenerating $source from the next $table value). this involves using ${var-default} or ${var:-default} to return the value to the local statement instead of setting it directly with ${var=default} or ${var:=default}.
*{.sh,run}: commands that take named arguments: usage messages: changed to use simpler bash syntax for specifying command-specific env vars, which does not require a subshell ("The environment for any simple command or function may be augmented temporarily by prefixing it with parameter assignments" <http://www.gnu.org/software/bash/manual/bash.html#Environment>)
lib/runscripts/local.run: moved log_sql (), pg_* () to lib/util.sh because these do not depend on local external scripts in bin/
lib/runscripts/local.run: added pg_copy_to (), pg_header (), pg_export_table_no_header (), pg_export_table_to_dir_no_header ()
lib/runscripts/local.run: added mk_schema_esc, mk_table_esc aliases as shortcuts for `local *_esc; mk_esc_name *`
lib/runscripts/local.run: psql (): output only the query results to stdout (using --output), and redirect everything else to stderr
lib/runscripts/local.run: added log_sql (). mysql (): don't echo SQL commands when not in verbose mode. this is needed to support commands that use the query result in a $() expression, and should not have echoed commands cluttering up stdout. (unfortunately, the SQL commands are echoed to stdout rather than stderr.)
bugfix: lib/util.sh: moved mysql (), psql () to local.run because they use local commands that are only available after local.run sets the PATH. retain a mysql () function in util.sh that adds the --verbose option to mysql.
moved lib/*.run into runscripts/ subdir so runscripts are grouped together and easier to find rather than being scattered throughout lib/
lib/local.run: add $bin_dir to the PATH
*run: added include guards
added lib/local.run, which links to locally-available resources such as $bin_dir
