Project

General

Profile

  • svn:executable: *

# Date Author Comment
13347 04/29/2014 09:53 PM Aaron Marcuse-Kubitza

lib/sh/local.sh: public_schema_exists(): moved to lib/sh/db.sh since this no longer depends on BIEN-specific configurations

13346 04/29/2014 09:42 PM Aaron Marcuse-Kubitza

bugfix: lib/sh/db.sh: public_schema_exists(): don't hide the function call tree so it's clear which function is running the psql commands

13345 04/29/2014 09:40 PM Aaron Marcuse-Kubitza

bugfix: lib/sh/db.sh: public_schema_exists(): don't hide the function call tree so it's clear which function is running the psql commands

12829 03/21/2014 11:48 PM Aaron Marcuse-Kubitza

lib/sh/util.sh: clog*: renamed to log* for clarity (possible now that log* is no longer used for function-local log_level setting)

12800 03/21/2014 03:42 PM Aaron Marcuse-Kubitza

*{.sh,run}: stderr_matches() wrapper calls: removed no longer needed prep_try/rethrow

12770 03/20/2014 06:56 PM Aaron Marcuse-Kubitza

stderr_matches(): wrapper caller usage: added alternative usage when using `||`

12767 03/20/2014 06:45 PM Aaron Marcuse-Kubitza

fix: *{.sh,run}: stderr_matches() wrappers: usage: added `rethrow`

11970 01/20/2014 11:33 AM Aaron Marcuse-Kubitza

moved everything into /trunk/ to create the standard svn layout, for use with tools that require this (eg. git-svn). IMPORTANT: do NOT do an `svn up`. instead, re-use your working copy's existing files with `svn switch` (http://svnbook.red-bean.com/en/1.6/svn.ref.svn.c.switch.html).

11861 12/06/2013 05:52 AM Aaron Marcuse-Kubitza

lib/sh/local.sh: public_schema_exists(): use a higher log_level for pg_schema_exists, to avoid all the verbose output involved in running the query

11860 12/06/2013 05:44 AM Aaron Marcuse-Kubitza

bugfix: lib/sh/local.sh: public_schema_exists(): can no longer use psql_script_vegbien for this, because using `SET search_path` (called by psql_script_vegbien) with a schema that does not exist no longer produces an error. instead, use new pg_schema_exists(), which uses a different command that does produce an error if the schema does not exist.

11433 10/24/2013 05:06 PM Aaron Marcuse-Kubitza

lib/sh/local.sh: psql(): also accept $public as the $schema param, since this is used by a lot of import scripts

11365 10/19/2013 12:52 AM Aaron Marcuse-Kubitza

bugfix: lib/sh/local.sh: psql(): $is_root: use `` around case statement instead of $(), because it contains an embedded unbalanced )

11362 10/18/2013 10:32 PM Aaron Marcuse-Kubitza

bugfix: lib/sh/local.sh: psql(): don't default the connection vars using use_local if running as the postgres user. in that case, connection must happen via a socket, with server="", and as the user running the command (postgres), with user="".

11352 10/18/2013 06:05 PM Aaron Marcuse-Kubitza

bugfix: lib/sh/local.sh: psql(): only \set schema, table if $schema, $table are non-empty, because otherwise, you will get a "zero-length delimited identifier" error

11350 10/18/2013 05:29 PM Aaron Marcuse-Kubitza

lib/sh/local.sh: added require_remote()

10765 08/27/2013 07:10 PM Aaron Marcuse-Kubitza

lib/sh/local.sh: added pg_dump_local()

10247 07/11/2013 02:20 AM Aaron Marcuse-Kubitza

bugfix: lib/sh/local.sh: prevent automated tests when the public schema contains the live DB, so the user doesn't have to explicitly specify can_test= when running the import on vegbiendev

10048 06/26/2013 12:30 AM Aaron Marcuse-Kubitza

lib/sh/local.sh: added PATH_add= flag to allow turning off the addition of $bin_dir_abs to the PATH. this is useful for wrapper scripts that should not alter the PATH passed to their invoked command.

10033 06/25/2013 02:57 PM Aaron Marcuse-Kubitza

lib/sh/local.sh: $sync_remote_url: allow user to override just the sync subdir (not the whole URL) in $sync_remote_subdir. this is useful e.g. for backing up the Mac's files to jupiter.

9896 06/12/2013 02:45 PM Aaron Marcuse-Kubitza

bugfix: lib/sh/local.sh: sync_upload(): need to use --no-group to prevent the group from being reset to aaronmk upon download from jupiter (which uses group aaronmk instead of bien). use ./fix_perms to set the group of all files to bien. also use --no-owner in case running as root.

9895 06/12/2013 02:12 PM Aaron Marcuse-Kubitza

lib/sh/sync.sh: removed sync_download(). use swap=1 sync_upload() instead.

9879 06/12/2013 10:30 AM Aaron Marcuse-Kubitza

bugfix: lib/sh/local.sh: removed make() override, which is no longer needed now that its operations are performed by verbosity_compat(), and which caused errors by setting $verbosity to the invalid value ""

9866 06/12/2013 06:47 AM Aaron Marcuse-Kubitza

bugfix: lib/sh/local.sh: make(): when invoking overridden func, need make__make_sh

9861 06/12/2013 06:26 AM Aaron Marcuse-Kubitza

lib/sh/local.sh: added make() override, which uses the default verbosity (i.e. verbosity="") when verbosity == 1. scripts that use lib/sql.py (which uses $verbosity) have different default verbosities, and this default should not be overriden by an env var, unless a higher verbosity has been set.

9860 06/12/2013 06:24 AM Aaron Marcuse-Kubitza

lib/sh/local.sh: added missing include of make.sh, used by root_make()

9854 06/12/2013 03:25 AM Aaron Marcuse-Kubitza

*{.sh,run}: use simpler .rel() instead of `. "$(dirname "${BASH_SOURCE0}")"/...` for relative includes

9827 06/11/2013 03:27 PM Aaron Marcuse-Kubitza

lib/sh/local.sh: psql(): don't put util in the search_path because psql scripts now add it themselves if they need it, using `SELECT util.search_path_append(util);`

9801 06/09/2013 06:27 PM Aaron Marcuse-Kubitza

lib/sh/local.sh: added sync_upload(), sync_download() with $sync_local_dir, $sync_remote_url config vars

9793 06/09/2013 04:01 PM Aaron Marcuse-Kubitza

bugfix: lib/sh/local.sh: root_rel_path(): added echo_func

9792 06/09/2013 04:01 PM Aaron Marcuse-Kubitza

bugfix: lib/sh/local.sh: root_rel_path(): use canon_rel_path instead of rel_path because $1 may be absolute rather than relative to the current dir, so $root_dir needs to be absoluted (which requires $1 to be absoluted as well)

9787 06/09/2013 01:00 PM Aaron Marcuse-Kubitza

lib/sh/local.sh: added root_rel_path()

9650 05/30/2013 02:43 PM Aaron Marcuse-Kubitza

lib/sh/local.sh: psql(): use new psql() from db.sh instead of psql_script_vegbien/psql_verbose_vegbien. this requires setting local_pg_database=vegbien to replace vegbien_dest used by psql_*_vegbien.

9645 05/30/2013 01:05 PM Aaron Marcuse-Kubitza

bugfix: lib/sh/local.sh: psql(): $file can't both be passed as a --file param and be prefixed with the necessary \set schema, etc. commands, so instead include $file when cat-ing stdin

9630 05/30/2013 06:16 AM Aaron Marcuse-Kubitza

lib/sh/local.sh: database connection vars: connect to vegbiendev via ssh and run commands locally, to allow running commands as root (which can only connect to the database locally). this effectively requires an ssh account on vegbiendev, but any ssh account (including an anonymous one, if we set one up) will do. this causes schemas/VegCore/VegCore.my.sql, VegCore.pg.sql to change, because they are now created by mysqldump running on vegbiendev (Linux) instead of on a Mac.

9577 05/24/2013 11:35 AM Aaron Marcuse-Kubitza

lib/sh/local.sh: added mysql_local()

9576 05/24/2013 11:24 AM Aaron Marcuse-Kubitza

lib/sh/local.sh: added mysql_root()

9575 05/24/2013 11:24 AM Aaron Marcuse-Kubitza

lib/sh/local.sh: added $root_user, $root_password

9479 05/21/2013 12:16 PM Aaron Marcuse-Kubitza

lib/sh/local.sh: psql(): documented that --output is actually for query results, not echoed statements (and thus must be redirected back to fd 1 while fd 1 with the statements gets sent to the logging port)

9478 05/21/2013 12:14 PM Aaron Marcuse-Kubitza

lib/sh/local.sh: psql(): documented why can't use fd 11

9477 05/21/2013 12:09 PM Aaron Marcuse-Kubitza

lib/sh/local.sh: use @redirs instead of manual redirection to set up --output fd, so that the redirection will be echoed along with the command. for some reason, this requires switching to fd 13 instead of 11, because fd 11 gives a "/dev/fd/11: Bad file descriptor" error when 11 is set with exec right before the command instead of on the subshell the command is executed in. (13 was chosen rather than 12 because 2 is for errors, while *3 (or 3) is for logging.)

9427 05/16/2013 06:55 PM Aaron Marcuse-Kubitza

lib/sh/local.sh: psql(): use $schema_esc, $table_esc instead of just putting $schema, $table in ""

9421 05/16/2013 05:48 PM Aaron Marcuse-Kubitza

bugfix: lib/sh/db.sh: log_sql(): use can_log() instead because the verbosity now gets decremented as the log_level increases, so the threshold to compare to is 0 instead of 2

9331 05/13/2013 06:30 AM Aaron Marcuse-Kubitza

bugfix: lib/sh/local.sh: add a manual errexit for $() exprs by embedding them in just a var assignment (without local or declare), whose exit status will then equal the of the $(). a `|| return` also needs to be added because errexit does not work on assignment statements. this commit adds them for func_loc(), echo_func(), canon_rel_path(), set_paths(), save_cache, cached realpath(), local.sh global vars

9243 05/12/2013 07:47 AM Aaron Marcuse-Kubitza

lib/sh/local.sh: $root_dir: run canon_rel_path on it

9235 05/12/2013 05:36 AM Aaron Marcuse-Kubitza

lib/sh/util.sh: renamed params() to kw_params() to clarify the type of params it handles

9231 05/12/2013 05:09 AM Aaron Marcuse-Kubitza

: use new `params` in all functions that have keyword params, in order to remove their vars from the environment. note that functions that use $? (such as die()) must save it *before running params, because params will overwrite $?.

9214 05/11/2013 09:07 PM Aaron Marcuse-Kubitza

lib/sh/util.sh: command(): use just one control var $cmd_log_fd instead of three flags (limit_log_fd, limit_stdout, stderr_is_errors) that indicated various common fd configurations. this is much clearer (you state which fd the common uses as its logging fd), more configurable (the logging fd can be any fd, not just 1 or 2), and more automatic (redirecting fd 2 to err_fd happens automatically if it isn't used for logging).

9213 05/11/2013 08:58 PM Aaron Marcuse-Kubitza

bugfix: lib/sh/local.sh: psql(): need to use limit_stdout=1 instead of stderr_is_errors=1 because logging info is output to stdout, not stderr, in spite of the >&2 redirection, since the >&2 is applied after command()'s logging redirects. note that the >&2 can now be removed, because it will be performed by command().

9153 05/11/2013 12:05 PM Aaron Marcuse-Kubitza

lib/sh/local.sh: added mysql_export_local()

9136 05/11/2013 06:24 AM Aaron Marcuse-Kubitza

lib/sh/local.sh: psql(): use new stderr_is_errors=1 since psql's logging output comes on stdout, so stderr contains only errors. (the caveat of this is that the query output is normally mixed with logging info on stdout, and needs to be separated out to a different file descriptor using --output.)

9131 05/11/2013 05:29 AM Aaron Marcuse-Kubitza

lib/sh/local.sh: psql(): use new convention for command-specific alternate stdout

9074 05/10/2013 12:08 PM Aaron Marcuse-Kubitza

*{.sh,run}: removed extra space between function name and (), which is apparently not needed even though `help function` includes it. this greatly improves readability, including when function names are pasted into commit messages!

9050 05/10/2013 09:06 AM Aaron Marcuse-Kubitza

*{.sh,run}: use shorter `test` instead of `test -n` and `test !` instead of `test -z` (http://www.gnu.org/software/bash/manual/bash.html#Bourne-Shell-Builtins > test)

9046 05/10/2013 08:40 AM Aaron Marcuse-Kubitza

lib/sh/local.sh: added public_schema_exists (), ported from lib/import.sh

9014 05/10/2013 04:24 AM Aaron Marcuse-Kubitza

lib/sh/util.sh: split databases utils out into separate db.sh

9013 05/10/2013 04:05 AM Aaron Marcuse-Kubitza

moved lib/*.sh to sh/ subdir so it's easier to find the .sh files among all the other lib/ files

8969 05/09/2013 06:11 PM Aaron Marcuse-Kubitza

lib/local.sh: removed mysql () since its functionality is now provided by mysql () in util.sh

8959 05/09/2013 04:38 PM Aaron Marcuse-Kubitza

*{.sh,run}: put functions on one line where possible (and where they are not expected to expand)

8958 05/09/2013 04:32 PM Aaron Marcuse-Kubitza

lib/local.sh: root_make (): use make () rather than extern make now that make () does not set its own current dir. using make () also automatically enables \$stdout2stderr, which is needed to redirect the echoed make commands.

8939 05/09/2013 08:19 AM Aaron Marcuse-Kubitza

lib/local.sh: added root_make ()

8937 05/09/2013 08:12 AM Aaron Marcuse-Kubitza

lib/local.sh: psql (): use fd 4 for the logging output instead of 3 because 3 is generally used for passing inline input commands (it is used this way by inline_make)

8925 05/09/2013 07:03 AM Aaron Marcuse-Kubitza

bugfix: lib/local.sh: $PATH: use $bin_dir_abs so the path will remain valid if the current directory is changed

8924 05/09/2013 07:02 AM Aaron Marcuse-Kubitza

lib/local.sh: added \$bin_dir_abs

8894 05/09/2013 02:32 AM Aaron Marcuse-Kubitza

bugfix: lib/local.sh: updated $root_dir

8893 05/09/2013 02:31 AM Aaron Marcuse-Kubitza

renamed lib/runscripts/local.run to lib/local.sh since the things it defines are not just for runscripts

8892 05/09/2013 02:22 AM Aaron Marcuse-Kubitza

lib/runscripts/local.run: added mysqldump_local, which uses vegbiendev

8891 05/09/2013 02:21 AM Aaron Marcuse-Kubitza

lib/runscripts/local.run: use postgres_compat mode because all our MySQL schemas should be usable as inputs to my2pg

8890 05/09/2013 02:20 AM Aaron Marcuse-Kubitza

lib/runscripts/local.run: added connection vars for local (bien) user

8881 05/08/2013 08:02 PM Aaron Marcuse-Kubitza

*{.sh,run}: changed echo_func to an alias that includes the "$", so that callers don't need to specify the "$" manually. although the original echo_func function is still there, callers need to switch over to the alias at the same time as it's defined because otherwise the "$@" would be doubled, since echo_func refers to the alias instead.

8873 05/08/2013 06:02 PM Aaron Marcuse-Kubitza

*{.sh,run}: use just env instead of echo_run env now that env is an auto-echoing alias

8834 05/06/2013 04:12 AM Aaron Marcuse-Kubitza

bugfix: *{.sh,run}: use `local` when setting default values for params to prevent these values from leaking into the calling context, where it may have unexpected effects (e.g. pg_copy_to would permanently set $source, which would then be used on the next invocation of pg_copy_to instead of regenerating $source from the next $table value). this involves using ${var-default} or ${var:-default} to return the value to the local statement instead of setting it directly with ${var=default} or ${var:=default}.

8833 05/06/2013 04:00 AM Aaron Marcuse-Kubitza

*{.sh,run}: commands that take named arguments: usage messages: changed to use simpler bash syntax for specifying command-specific env vars, which does not require a subshell ("The environment for any simple command or function may be augmented temporarily by prefixing it with parameter assignments" <http://www.gnu.org/software/bash/manual/bash.html#Environment&gt;)

8796 05/02/2013 08:35 PM Aaron Marcuse-Kubitza

lib/runscripts/local.run: moved log_sql (), pg_* () to lib/util.sh because these do not depend on local external scripts in bin/

8795 05/02/2013 08:27 PM Aaron Marcuse-Kubitza

lib/runscripts/local.run: added pg_copy_to (), pg_header (), pg_export_table_no_header (), pg_export_table_to_dir_no_header ()

8794 05/02/2013 08:26 PM Aaron Marcuse-Kubitza

lib/runscripts/local.run: added mk_schema_esc, mk_table_esc aliases as shortcuts for `local *_esc; mk_esc_name *`

8792 05/02/2013 02:49 PM Aaron Marcuse-Kubitza

lib/runscripts/local.run: psql (): output only the query results to stdout (using --output), and redirect everything else to stderr

8790 05/02/2013 02:24 PM Aaron Marcuse-Kubitza

lib/runscripts/local.run: added log_sql (). mysql (): don't echo SQL commands when not in verbose mode. this is needed to support commands that use the query result in a $() expression, and should not have echoed commands cluttering up stdout. (unfortunately, the SQL commands are echoed to stdout rather than stderr.)

8777 05/02/2013 06:14 AM Aaron Marcuse-Kubitza

bugfix: lib/util.sh: moved mysql (), psql () to local.run because they use local commands that are only available after local.run sets the PATH. retain a mysql () function in util.sh that adds the --verbose option to mysql.

8774 05/02/2013 04:47 AM Aaron Marcuse-Kubitza

moved lib/*.run into runscripts/ subdir so runscripts are grouped together and easier to find rather than being scattered throughout lib/

8708 04/25/2013 05:39 PM Aaron Marcuse-Kubitza

lib/local.run: add $bin_dir to the PATH

8705 04/25/2013 04:49 PM Aaron Marcuse-Kubitza

*run: added include guards

8699 04/25/2013 03:04 PM Aaron Marcuse-Kubitza

added lib/local.run, which links to locally-available resources such as $bin_dir