exports/2014-10-18.Juergen_Dengler.sPlot.csv.run: export_only(): documented runtime (45 min)

bugfix: schemas/public_.sql: analytical_stem_CVS: need to reset columns list to *

bugfix: exports/analytical_stem_CVS.run needs .csv extension

added exports/analytical_stem_CVS.run

schemas/public_.sql: added analytical_stem_CVS view

added exports/2014-10-18.Juergen_Dengler.sPlot.csv.run

schemas/public_.sql: added 2014-10-18.Juergen_Dengler.sPlot view

schemas/public_.sql: added public_datasources()

removed no longer used inputs/datasource_release_status.xlsx. instead, use inputs/public datasources.xlsx.

inputs/publishable datasources.xlsx: updated: changed "publishable" to shorter "public" (and also in filename)

added backups/vegbien.r14827.backup.md5

backups/TNRS.backup.md5: updated

bugfix: /README.TXT: to back up the local machine's settings: backing up / to vegbiendev: also need standard excludes from "to back up the local machine's hard drive"

/README.TXT: to back up the local machine's settings: also back up / to vegbiendev

/README.TXT: `sudo find / -name .DS_Store -print -delete`: documented runtime (23 min)

web/links/index.htm: updated to Firefox bookmarks: Ubuntu: added VNC server links

bugfix: /README.TXT: to back up the local machine's hard drive: need to turn off Time Machine to prevent Time Machine from running when the backup is booted, which messes up the Time Machine backup because the partition UUID is different

fix: /README.TXT: to back up the local machine's hard drive: don't need to pause Dropbox, since it will be reinitialized for the new partition UUID anyway

planning/meetings/BIEN conference call availability.xlsx: updated

bugfix: /README.TXT: to back up the local machine's hard drive: first need to resize the BIEN3.**SAVE** partition so there is enough space

bugfix: /README.TXT: to backup files not in Time Machine: need to use `brew services _ postgresql` instead of `launchctl _ ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist`

bugfix: /README.TXT: to back up the local machine's hard drive: need to use `brew services _ postgresql` instead of `launchctl _ ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist`

bugfix: config/VirtualBox_VMs/vegbiendev/README.TXT: to retrieve the contents of the VM: can't use inplace=1 optimization because this messes up the shared hardlinks used by ~/Documents/BIEN/vegbiendev/

fix: config/VirtualBox_VMs/vegbiendev/README.TXT: to retrieve the contents of the VM: don't need to boot the VM because those snapshots are already booted

bugfix: /README.TXT: to back up vegbiendev: can't use inplace=1 optimization because this messes up the shared hardlinks used by ~/Documents/BIEN/vegbiendev.2014-2-2_1-07-32PT*/

bugfix: /README.TXT: to back up vegbiendev: also need to lock hardlinked files at end

lib/sh/util.sh: added errexit_sig()

lib/sh/util.sh: sig_e(): use new sig_num()

lib/sh/util.sh: added sig_num()

bin/import_all: `make -s`: removed no longer needed output_data=1

bugfix: lib/sh/make.sh: make() wrapper: need to set output_data mode if -s is set, to match the behavior of the underlying make

bugfix: bin/import_all: don't disable errexit because this prevents the program from being Ctrl-C'd. this functionality is no longer needed now that the README.TXT instructs to run bin/import_all in a subshell.

bin/import_all: removed functionality now provided by util.run

bin/import_all: converted to a runscript so it can use runscript functionality

/README.TXT: Installation: `make install`: added alternative command to install a complete DB with all the datasources (instead of a blank DB)

inputs/bien_web/observation/map.csv: updated

bugfix: inputs/Makefile: install: need to use $(wildcard+suffix) to support Linux

lib/common.Makefile: added $(wildcard+suffix), needed because on Linux, builtin $(wildcard) doesn't handle suffixes that don't exist

inputs/Makefile: install: support $(all) mode like for download

inputs/Makefile: download: don't download log files, to speed up installation on a new machine

inputs/Makefile: download: don't download all the inputs by default, to speed up installation on a new machine. instead, enable this with the $(all) flag.

bugfix: /README.TXT: Full database import: need to unlock hardlinked files

/README.TXT: back up Postgres: backups/pg_snapshot: documented runtime when queries have been run (1 h)

fix: /README.TXT: Full database import: also need to run `unset TMOUT` in the outer `screen` shell

fix: /README.TXT: back up Postgres: needs to be run in `screen`

/README.TXT: to synchronize vegbiendev, jupiter, and your local machine: update mtimes in Dropbox: preserve uncommitted changes

fix: /README.TXT: to synchronize vegbiendev, jupiter, and your local machine: update mtimes in Dropbox: don't update perms because this would incorrectly transfer the hardlink lock status

fix: /README.TXT: to synchronize vegbiendev, jupiter, and your local machine: TWiki: need to fix_perms after running this to reset perms on hardlinks

/README.TXT: syncing: use del instead of overwrite where possible, to avoid overwriting newer files

/README.TXT: Full database import: should also snapshot Postgres before starting import

bugfix: inputs/import.stats.xls: need to exclude postprocessing from the ms/row and Change formulas, also for the "<2014-2-2" tab

inputs/import.stats.xls: updated import times

inputs/import.stats.xls: updated import times

/README.TXT: record the import times: documented how to update the Postprocessing times

bugfix: /README.TXT: globs that list all the datasources: this was incorrectly excluding U, so fixed to match bin/with_all $hidden_srcs

fix: inputs/VegBank/project/: added _no_import because this is imported as part of taxon_observation.**

bugfix: inputs/import.stats.xls: need to exclude postprocessing from the ms/row and Change formulas. removed deleted rows that don't apply to the most recent imports. updated runtime formulas to match bin/import_all.

bugfix: inputs/import.stats.xls: restored missing formatting for multi-day times. prepped tabs to have new import stats data added.

fix: /README.TXT: record the import times: need to first click the "current" tab

/README.TXT: To re-run geoscrubbing: updated runtimes and revision #s

bugfix: /README.TXT: To re-run geoscrubbing: don't close subshell because it closes automatically (this would instead exit `screen`)

/README.TXT: To re-run geoscrubbing: updated runtimes

_license/VegBank_schema/applies_to.txt: added exports/sPlot/BIEN-sPLot-Skype-finalNotes.docx

added exports/sPlot/BIEN-sPLot-Skype-finalNotes.docx from Bob

/README.TXT: To re-run geoscrubbing: updated runtimes

bugfix: /README.TXT: To re-run geoscrubbing: "press Enter" and `unset TMOUT` steps are reversed

bugfix: schemas/public_.sql: geoscrub_input_new: need to join on the ARRAY[] of each column, to compare NULLs properly

bugfix: lib/sh/util.sh: become_sudo don't continue running function after self_sudo finishes

/README.TXT: datasource removal: linked to runtimes on the wiki

bugfix: inputs/input.Makefile: %/map.csv: need to save it if errors occur in unmapped_terms.csv, new_terms.csv

fix: inputs/FIA/PLOT/map.csv: added plotArea_ha, requested by Brian McGill

fix: inputs/FIA/PLOT/map.csv: added plotArea_ha, requested by Brian McGill

added backups/old_backups.txt

added backups/TNRS.2014-6-2.backup.md5

_license/non-open-source/applies_to.txt: clarified that the things in this list are not embargoed, the issue is just that they are not formally open-source

schemas/public_.sql: view_full_occurrence_individual_view: join to taxonoccurrence: can use a USING join for this because plot.** only exports one locationevent_id column

inputs/Cyrille_traits/trait_observation/postprocess.sql: accessRights index: added vegbiendev runtime

fix: inputs/Cyrille_traits/trait_observation/postprocess.sql: need to remove private data, as clarified by Cyrille

inputs/publishable datasources.xlsx: updated: Cyrille traits: updated with Cyrille's clarifications on publishability

bugfix: lib/sh/sync.sh, db.sh: *_snapshot() commands that run multiple sudo commands: use become_sudo so that the program won't hang at later sudo prompts. for pg_snapshot() with pg_start_backup(), this also prevents the backup from staying open at the end waiting on a sudo prompt, causing excess WAL logs to accrue.

lib/sh/util.sh: become_sudo: documented that this is for funcs that run multiple sudo commands

lib/sh/util.sh: $sudo_cmd_wrapper: use esc() so this does not need to be embedded with [@] array notation

bugfix: lib/runscripts/util.run: $wrap_fn: this always needs to contain `is_runscript=1` because $is_runscript is not (and cannot be) exported

bugfix: lib/sh/util.sh: sudo(): when don't need sudo: do need `command` to have effects of invoking external command

fix: lib/sh/db.sh: pg_snapshot(): first need to run pg_stop_backup() in case a backup was not stopped cleanly

fix: lib/sh/db.sh: pg_stop_backup(): ignore benign "a backup is not in progress" error

lib/sh/util.sh: sudo(): only run sudo if needed

lib/sh/util.sh: sudo(): factored PATH-preserving code out into separate wrapper var that can also be used separately, eg. with `sudo -u`

lib/sh/util.sh: sudo(): moved addition of -E (which should always happen) to separate, overridden sudo() function

lib/sh/util.sh: moved -E to is_callable section because it just applies to that

fix: inputs/input.Makefile: $(svnFilesGlob): only svn:ignore *.log in the top-level dir

added inputs/XAL/Specimen/data.csv (empty template)

fix: inputs/input.Makefile: add!: verify/: also svn:ignore .zip

fix: inputs/input.Makefile: add!: verify/: also svn:ignore .zip

lib/sh/util.sh: added become_sudo alias

lib/sh/util.sh: added $is_root

inputs/publishable datasources.xlsx: updated: replaced BIEN2 traits with Cyrille traits

exports/view_full_occurrence_CVS.csv.run: documented runtime (2 min for 5000 rows)

fix: inputs/publishable datasources.xlsx: updated: conditions of use: Canadensys sources: these should actually be marked as no restrictions, in contrast to Brian E's earlier annotations, because they are public domain: the citation is requested, but not legally required

inputs/publishable datasources.xlsx: updated