lib/util.sh: exceptions: end_try* aliases: use new rethrow*
lib/util.sh: exceptions: added rethrow, rethrow_subshell aliases
lib/util.sh: always use '' rather than "" around alias definitions, to ensure that variables are evaluated at expand time rather than compile time
lib/util.sh: zip_newer/unzip_newer: evaluate $no_force at alias expansion time rather than at alias compile time, so that the $no_force can be overridden in the context the alias is used in
lib/util.sh: use `|| return` instead of `|| exit` because `|| exit` doesn't seem to work inside functions (it does not have the errexit effect). also, `|| return` has the advantage of not exiting the program if the caller used || after the command (i.e. as an error handler) to temporarily disable errexit.
lib/runscripts/util.run: tell users to override run_args_cmd rather than this function to perform other commands, so that on_exit () can contain other exit-related processing that should not be overriden by the user
lib/util.sh: removed "" around $?, $# because they are guaranteed to always be non-empty and contain no special chars
lib/util.sh: say "last space" instead of "trailing space" so the comment will be more likely to fit at the end of the line
lib/local.sh: removed mysql () since its functionality is now provided by mysql () in util.sh
lib/util.sh: mysql_cmd (): $server: like $ssh_server, clear the value when it's equal to this machine's hostname
lib/util.sh: mysql_cmd (): $ssh_server: use new localize_url to clear the value when it's equal to this machine's hostname
lib/util.sh: added localize_url ()
inputs/GBIF/table.run: table.tsv/make (): use extern so the MySQL_export command is echoed
inputs/GBIF/table.run: table.tsv/make (): don't make the target if it already exists
inputs/GBIF/table.run: table.tsv/make (): create the file directly rather than using inline_make, so that the command to create the file can use shell functions such as mysql
bugfix: inputs/GBIF/table.run: table.tsv/make (): need to use $(target) (target's path relative to curdir) instead of just the filename, now that make does not change the directory
inputs/GBIF/table.run: moved table.tsv creation to separate table.tsv/make () function
lib/util.sh: echo_stdin (): display the echoed input after leading output of the echoed-to command, by using sleep with a small fractional # of seconds
*{.sh,run}: put functions on one line where possible (and where they are not expected to expand)
lib/local.sh: root_make (): use make () rather than extern make now that make () does not set its own current dir. using make () also automatically enables \$stdout2stderr, which is needed to redirect the echoed make commands.
lib/util.sh: extern (): use exec to ensure that the executed command is in fact external, and not a shell builtin or other type of non-function command. this also simplifies the code because any shell function for the command does not need to be unset.
lib/util.sh: make (): don't change the directory to $top_dir, because paths used by the caller will instead be relative to the current dir (note that in runscripts, paths are generally not absolute because of canon_rel_path ()). this requires manually specifying $top_dir when running make on a physical (rather than inlined) makefile.
lib/util.sh: log_stderr_cmd (): replaced the `|| exit` after the () with `|| return`, for the same reason that this was a problem for extern () (see r8954)
lib/util.sh: extern (): replaced the `|| exit` after the () with `|| return`, because `|| exit` strangely does not use $? as its exit status (even though `|| exit $?` works fine). in a shell function, it is actually sufficient, for errexit purposes, to use `|| return`, because the exit status is checked upon return from the function. it is in fact preferable to use return rather than exit, because then the caller can catch the exception using `|| true`.
lib/util.sh: extern (): removed the `|| exit` inside the (), because this strangely prevents the exit status from being propagated to the exit status of the ()
lib/util.sh: added explicit `set -o errexit` in case caller did not have -e in #! line
inputs/GBIF/table.run: renamed MySQL_export () to export_mysql () to match schemas/VegCore/VegCore.ERD.mwb.run
inputs/FIA/_archive/2011-10-17/: set svn:ignore to *
web/links/index.htm: updated to Firefox bookmarks. added Mac upgrade and recovery-related bookmarks.
bugfix: lib/import.sh: remake_VegBIEN_mappings (): don't run if there is no public schema to perform the test on (e.g. if the staging tables have just been reinstalled)
lib/import.sh: added public_schema_exists (), which just calls psql_script_vegbien (i.e. with the default search_path temp, public, etc.) and sees if returns an error
/README.TXT: Maintenance: to synchronize vegbiendev, jupiter, and your local machine: when changes are made on vegbiendev: added step to download TWiki separately to your machine, to ensure that it gets backed up even if you don't sync the rest of the files. also, TWiki needs to be run with delete enabled to remove old session files, while the rest of the files should be run without delete.
schemas/VegCore/VegCore.ERD.mwb.run: use my2pg's new sql_ascii mode
bin/my2pg: support $sql_ascii mode, where varbinary is translated to text instead of bytea
bin/my2pg: remove any () after text
schemas/VegCore/VegCore.ERD.mwb.run: VegCore.my.sql/make (): leave out the DROP TABLE statements for now using --skip-add-drop-table
added schemas/VegCore/VegCore.pg.sql.run with install () command (also part of all ())
lib/local.sh: added root_make ()
bin/my2pg: unpack custom types set('...')
lib/local.sh: psql (): use fd 4 for the logging output instead of 3 because 3 is generally used for passing inline input commands (it is used this way by inline_make)
schemas/VegCore/VegCore.ERD.mwb.run: sync (): don't pass "$@" to export_/import because they do not accept args, and the same set of args would not be meaningful for both
schemas/VegCore/VegCore.ERD.mwb.run: use new $top_filename instead of $archive_filename
lib/util.sh: added $top_script (alias for $0), $top_file, $top_filename. use new $top_file in $top_dir.
schemas/VegCore/VegCore.ERD.mwb.run: put zip_newer/unzip_newer on the same line as the `cd "$top_dir"`
bin/my2pg: translate varbinary to bytea
schemas/VegCore/VegCore.ERD.mwb.run: added export_postgres (), which calls VegCore.pg.sql/make ()
lib/util.sh: wrapper functions of external commands: use new self alias
lib/util.sh: added self alias, for use with wrapper functions of external commands
schemas/VegCore/VegCore.my.sql: regenerated using `VegCore.ERD.mwb.run export_mysql` (also part of the all target). note that this uses mysqldump, so the format and table order is different than it was for MySQL Workbench's custom export script.
schemas/VegCore/VegCore.ERD.mwb.run: added export_mysql (), which runs VegCore.my.sql/make ()
bugfix: lib/local.sh: $PATH: use $bin_dir_abs so the path will remain valid if the current directory is changed
lib/local.sh: added \$bin_dir_abs
bugfix: lib/util.sh: extern (): added `|| exit` after "$@" in addition to after the (), because this additional `|| exit` is apparently needed in order to propagate the exit status of the command to the exit status of the () and cause an error to abort the script properly
lib/util.sh: echo_vars (): output vars with log_level 3 so they can be excluded separately from function calls to reduce clutter in the logging output. increase the default util.run verbosity so that vars are still printed by default.
bugfix: lib/util.sh: echo_vars (): output each var separately so that each is prefixed by $PS4
lib/util.sh: inc/dec_log_level: indent commands at higher log levels using $log_indent
bugfix: lib/util.sh: make (): use stdout2stderr=1 so that commands make runs are properly output to stderr
bugfix: lib/util.sh: include_guard_var (): use extern () when invoking sed so that this internal use of it isn't verbosely logged along with other external commands. note the "" around extern to refer to the function, not the alias.
bugfix: lib/util.sh: zip/unzip (): use new $stdout2stderr instead of >&2 so that the redirect happens in the right order
lib/util.sh: log_stderr_cmd (): support redirecting stdout to stderr *after*setting up the stderr redirect, so that stdout is also subject to that redirect
lib/util.sh: added stdout2stderr ()
lib/util.sh: changed echo_run_extern () to an alias
lib/util.sh: echo_run_extern (): use new log_stderr_cmd
lib/util.sh: added log_stderr_cmd ()
lib/util.sh: use extern instead of env to run external commands, because it's simpler and auto-captures the logging output
lib/util.sh: echo_func (): use the much simpler function keyword to avoid expanding the echo_func alias when defining the function, rather than going through the complex process of unaliasing and realiasing the echo_func alias
lib/util.sh: mark env as always being an external command. it is no longer necessary to automatically echo it using echo_run because the extern alias does this now.
lib/util.sh: automatically echo external commands. this requires using the function keyword when declaring extern () so that the new extern alias is not expanded in the function name.
lib/util.sh: added echo_run_extern (), which automatically runs log_stderr to capture the external command's stderr
lib/util.sh: added extern ()
lib/util.sh: added log_stderr ()
lib/util.sh: added end_try_subshell alias
lib/util.sh: zip/unzip: redirect logging output to stderr
lib/util.sh: use inc_log_level where logging info should only be output for verbosity >= 2: echo_func, echo_vars, echo_stdin
lib/util.sh: verbose output: added inc_log_level, dec_log_level aliases which manipulate $verbosity. note that changes in $log_level are now instead indicated by moving $verbosity in the opposite direction.
lib/util.sh: added let () wrapper that prevents it from triggering error-exit
lib/util.sh: verbose output: use just one var ($verbosity) instead of two ($verbosity and $log_level), to track log level, for simplicity. $verbosity would now be subtracted from to alter the log level instead of $log_level being added to.
lib/util.sh: use new can_log wherever logging info is output to stderr
renamed lib/runscripts/local.run to lib/local.sh since the things it defines are not just for runscripts
lib/util.sh: verbose output: added $log_level (must always be > 0 so verbosity=0 turns off all logging) and can_log (), which compares $log_level to $verbosity
bugfix: lib/local.sh: updated $root_dir
lib/runscripts/local.run: added mysqldump_local, which uses vegbiendev
lib/runscripts/local.run: use postgres_compat mode because all our MySQL schemas should be usable as inputs to my2pg
lib/runscripts/local.run: added connection vars for local (bien) user
lib/util.sh: moved unalias inside include guard since it is not needed by anything before the include guard (even though it is used by commands outside the include guard, i.e. when the include guard is temporarily closed and reopened to load/unload aliases)
lib/util.sh: use declare instead of local in aliases so that the aliases can also be used outside a function (declare will create a local var when used inside a function, and a global var otherwise)
lib/util.sh: zip: ignore "zip has nothing to do" exit status as this is not an error
lib/util.sh: added exception-handling (i.e. error-suppressing) functions
bugfix: lib/util.sh: echo_func alias: moved it outside the include guard because the unalias statement is needs to be outside the include guard, and therefore so does the alias statement in order to always restore the alias
lib/util.sh: renamed echo_func_ () back to echo_func () and instead add an unalias statement before the if statement containing the function definition. this is necessary because if statements are "compound commands", which have aliases expanded when their definition is read (i.e. at the beginning of the if statement, for the entire if statement), not when they are executed, and any aliases for function names in them need to be unaliased before the if statement containing the function definition. this is necessary even if the if statement is not executed, because the expansion of such aliases will cause syntax errors in the parsing of the if statement itself....
lib/util.sh: added unalias () override, which doesn't produce an error if the alias is undefined
bugfix: lib/util.sh: renamed echo_func () to echo_func_ (), because you apparently can't have a function with the same name as an alias, even if the function is defined before the alias and an include guard prevents the function from being defined again after the alias is set. there may be some kind of alias-expanding preprocessing pass that bash only does in script mode, or maybe there is some other problem related to our use of aliases inside if statements.
*{.sh,run}: changed echo_func to an alias that includes the "$", so that callers don't need to specify the "$" manually. although the original echo_func function is still there, callers need to switch over to the alias at the same time as it's defined because otherwise the "$@" would be doubled, since echo_func refers to the alias instead.
", so that callers don't need to specify the "$
lib/util.sh: added set_make_vars alias, for use with make-target-style shell functions
lib/util.sh: mysql_cmd (): interpret an empty string for $user, $database as if they were unset
bugfix: lib/util.sh: mysql_cmd (): add --tables after specifying --databases so that future unnamed arguments are correctly interpreted as tables
