inputs/TEX/Specimen2/test.xml.ref: updated

inputs/GBIF/raw_occurrence_record_plants/test.xml.ref: updated

bugfix: lib/sql_io.py: put_table(): handle_MissingCastException(): when updating join_cols, don't add new entry for join_cols[out_col], only update existing one. this fixes #902 (import bug), and with #902 fixed, #887 (disk space leak) should no longer occur.

bin/import_all: delete_logs(): documented that `trap EXIT` doesn't run until shell exit

bin/import_all: delete_logs(): print when this happens, so it can be verified that it's happening properly

bugfix: bin/import_all: need to run delete_logs manually because `trap EXIT` doesn't run until bg cmds done

bin/import_all: delete_logs: moved testing of whether to delete logs to delete_logs() so that delete_logs() can be run regardless of the $delete_logs setting

bugfix: bin/import_all: delete_logs(): also need to match log filenames when n=""

bugfix: /Makefile: postgres_restart-*: argument to echo always needs quotes

bugfix: /Makefile: $(pg_ctl-*): tell user that if it doesn't work, need to run `rm $(macPostgresDir)/postmaster.pid` and retry

/Makefile: $(pg_ctl-Darwin): documented the pg_ctl version of this command in case `brew services` doesn't work

bugfix: bin/with_all: isset(): need to use `>&- 2>&-` because &> does not work with - as the dest

fix: config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: determine if the revision works: use the in# of the query instead of waiting an arbitrary time, because this is more accurate in determining whether the import is successful

config/VirtualBox_VMs/vegbiendev/README.TXT: regressions that occurred during the revision range: named these "concurrent regressions"

fix: config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: need to update system time so wait times can be calculated correctly

config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: documented that the patches are specifically for regressions that occurred during the revision range, not all bugs that were fixed in that range

bugfix: config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: waiting for query: increased time to min. 10 min because some revisions take longer

fix: config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: bin/import_all: added step to type y and press ENTER

config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: added steps to determine if the revision works, and then stop the import

bugfix: config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: bugfixes: added r12994

config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: bugfixes: use for loop to avoid needing to repeat the command

bugfix: config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: bugfixes: added r12760

bugfix: config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: bugfixes: --fuzz apparently needs to be 3 even though we have only 2 lines of context

bugfix: config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: bugfixes: use --fuzz to allow patching even when surrounding lines have changed

config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: bugfixes: sorted by revision #

bugfix: config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: bugfixes: added r12865

bugfix: config/VirtualBox_VMs/vegbiendev/README.TXT: patch: need to use --force to avoid prompting user

bugfix: config/VirtualBox_VMs/vegbiendev/README.TXT: `svn revert`: need to use `--depth infinity` when reverting more than a file

bugfix: config/VirtualBox_VMs/vegbiendev/README.TXT: use `patch` instead of `svn merge` because `svn merge` sometimes inexplicably doens't apply patches

config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: bugfixes: `svn revert` all changes instead of specific files, to support bugfixes to other files

config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: lib/sh/util.sh bugfix: use `svn merge` instead of `sed --in-place` because this can be used for more complex changes

config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: lib/sh/util.sh bugfix: documented revision

bugfix: config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: need to apply r12872 bugfix to lib/sh/util.sh because it occurred after r12025

fix: config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: `make` commands: use `yes|` to avoid needing to confirm prompts

fix: config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: `make mk_db ...`: use `yes|` to avoid needing to confirm prompts

config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: to test a different revision: added step that if any of the commands after `svn up` are broken, try a different revision

bugfix: config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: `make db`: need to run `schemas/reinstall` manually because older versions of the codebase just used `schemas/install`

config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: documented the range from within the revision should be chosen

fix: inputs/.IRMNG/*/: added _no_import because this datasource is only used as a lookup table and does not need to be imported/scrubbed

config/VirtualBox_VMs/vegbiendev/README.TXT: `svn up`: documented runtime (1 min)

bugfix: added nodes/_no_import, because the NCBI import doesn't work due to a Postgres 9.3 bug (#859). _no_import used to not be needed because we didn't import the hidden sources.

fix: bin/with_all: removed debug statements

bugfix: bin/with_all: testing if @inputs is set: `"${inputs+isset}"` syntax doesn't work for empty arrays, so need to use `declare -p` instead

bugfix: to set up the VM: to use a database backup: also need to `make rm_initial_public` because initdb creates a non-empty DB

config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: added steps to test a different revision

/Makefile: db: use reinstall instead of install to allow running this on an existing DB

config/VirtualBox_VMs/vegbiendev/README.TXT: `sudo apt-get dist-upgrade`: documented runtime (12 min)

config/VirtualBox_VMs/vegbiendev/README.TXT: updated VM snapshot names

fix: config/VirtualBox_VMs/vegbiendev/README.TXT: to install system updates: added step to restart, because some system updates require this

fix: config/VirtualBox_VMs/vegbiendev/README.TXT: to install system updates: added step to restart, because some system updates require this

config/VirtualBox_VMs/vegbiendev/README.TXT: added instructions to install system updates

config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: bin/import_all: documented runtime (50 min)

config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: added steps to run the import

config/VirtualBox_VMs/vegbiendev/README.TXT: to set up the VM: added steps to move vegbiendev.vdi to the internal hard drive

config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: added steps to start the VM

config/VirtualBox_VMs/vegbiendev/README.TXT: moved "set up the VM" steps to separate section

config/VirtualBox_VMs/vegbiendev/README.TXT: pg_restore: added steps to create a booted snapshot

config/VirtualBox_VMs/vegbiendev/README.TXT: moved creation of pg_restore snapshot to be part of just the database backup section

config/VirtualBox_VMs/vegbiendev/README.TXT: pg_restore: documented runtime (15 min)

config/VirtualBox_VMs/vegbiendev/README.TXT: pg_restore: time the operation

config/VirtualBox_VMs/vegbiendev/README.TXT: pg_restore: added name for VM snapshot after this step

bugfix: config/VirtualBox_VMs/vegbiendev/README.TXT: pg_restore: need to run as DB superuser to restore py_util

config/VirtualBox_VMs/vegbiendev/README.TXT: rsync: updated runtime (25 min)

config/VirtualBox_VMs/vegbiendev/README.TXT: added steps to create snapshot before running pg_restore

config/VirtualBox_VMs/vegbiendev/README.TXT: added steps to configure the VM before booting

config/VirtualBox_VMs/vegbiendev/README.TXT: added instructions for selecting the initial snapshot and booting the VM

config/VirtualBox_VMs/vegbiendev/README.TXT: perform long-running operations inside `screen`

bugfix: config/VirtualBox_VMs/vegbiendev/README.TXT: to use a database backup: only run mk_db, not db, to avoid restoring schemas that are part of the backup

bugfix: config/VirtualBox_VMs/vegbiendev/README.TXT: pg_restore: schemas must be in dependency order

bugfix: config/VirtualBox_VMs/vegbiendev/README.TXT: pg_restore: don't restore analytical_db schema because we don't use it and it depends on public, which we're not restoring

web/links/index.htm: updated to Firefox bookmarks: WebEx: added steps to join a test meeting. pg_restore: documented that you need to create the schema before you restore into it, or else you will get a cryptic error message that it's using pg_catalog instead. added Homebrew links, including instructions for setting psycopg2 up with it. added PostGIS links. username URLs: documented how to enable these in Internet Explorer. TWiki: documented what to do when moving the TWiki install dir.

bugfix: lib/Firefox_bookmarks.reformat.csv: updated for new Firefox bookmarks format, which indents the <DD> tag

bugfix: config/VirtualBox_VMs/vegbiendev/README.TXT: pg_restore: need to create the schema before you restore into it, to avoid cryptic error messages

bugfix: config/VirtualBox_VMs/vegbiendev/README.TXT: schemas/postgresql.conf: need to decrease shared_buffers to match VM's allocated memory (1 GB)

bugfix: config/VirtualBox_VMs/vegbiendev/README.TXT: schemas/postgresql.conf: need to decrease shared_buffers to match VM's allocated memory (1 GB)

bugfix: config/VirtualBox_VMs/vegbiendev/README.TXT: need to run `make db` to restore users, which are not part of the backup

config/VirtualBox_VMs/vegbiendev/README.TXT: rsync: updated runtime (35 min)

config/VirtualBox_VMs/vegbiendev/README.TXT: turning off of non-ARIZ inputs: removed metadata datasources because these have been renamed to start with ".", and so aren't included in */

bugfix: config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: database backup approach: pg_restore: need to use for loop because pg_restore doesn't support multiple --schema options (it just uses the last one)

config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: database backup approach: added step to restore the necessary schemas from the DB

config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: database backup approach: rsync: documented runtime (13 min)

config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: database backup approach: added step to fetch the database backup via rsync

config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: database backup approach: moved steps to enable the user to access VirtualBox shared folders to separate section because VirtualBox shared folders currently don't work, likely because of a bug in VirtualBox Guest Additions when installed on something other than a clean Linux install

config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: database backup approach: added steps to enable the user to access VirtualBox shared folders

config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: added section for database backup approach, as alternative to clean database approach

fix: config/VirtualBox_VMs/vegbiendev/README.TXT: use ".." instead of - to indicate a range, to avoid this being confused for a minus sign

bugfix: /Makefile: postgres_restart-Darwin: need to have the user run the command outside of make because it doesn't appear to do anything when run inside make (this is the same problem as on Linux)

/Makefile: $(pg_ctl-Darwin): use simpler `brew services` instead of `launchctl __/homebrew.mxcl.postgresql.plist`

bugfix: bin/stop_imports: also need to include `bin/after_import`

inputs/.TNRS/Source/map.csv: datasetURL: updated to indicate that this uses the TNRS dev server (the actual URL is private)

fix: inputs/VASCAN/Source/map.csv: observationGranularity should not be specimen because this datasource contains only taxa

bugfix: bin/import_all: now that always using log files to fix output clutter, need to delete created logs if logging is turned off

bugfix: bin/import_all: don't errexit if a background process is Ctrl-C'd

bugfix: bin/import_all: was run without initial "." test: don't exit nonzero because this will close the subshell

bugfix: bin/import_all: ensure that this is run in a subshell, which is needed so errexits don't close the terminal window

bin/import_all: documented that this must be run in a subshell (obtained by running `$0`)

bugfix: bin/import_all: need to always use log files for background processes

fix: bin/import_all: Source/import: don't use by_col=1 for this because it's slower for small #s of rows. by_col mode is no longer needed for metadata-only tables because these tables now have a single empty row so that they also work in row-based mode.

fix: bin/import_all: hidden srcs: use with_all for this to avoid needing to list every source, and to display the backgrounded command with the variables substituted

bin/import_all: TNRS, geoscrub: integrated into the list of metadata sources