web/username_prefix.php: remove padding used to visually separate elements: documented that this also removes a leading __ that indicates an internal field
web/links/index.htm: updated to Firefox bookmarks: Quick links: added VegBIEN data dictionary spreadsheet, TNRS data dictionary.
fix: web/username_prefix.php: transposes: changed recommended syntax to y~x@url so that _y~ lines up vertically with [y] paths
web/username_prefix.php: removed special category comments syntax because this is now handled by transposes and insertion comments
web/username_prefix.php: added support for transposes ( y~x_@url -> x_y@url , Y~x@url -> xY@url ), which can be used to put the category in a column name first without duplicating the category
web/username_prefix.php: removed linewraps code because Google spreadsheets no longer supports wrapping on a - , and because a shorter syntax for category comments is now available
web/username_prefix.php: category comments: moved _-_ outside [] in comparison example since it will be deleted separately
bugfix: web/username_prefix.php: category comments: need /__/ around regexp
web/username_prefix.php: category comments: added support for a shorter syntax than [x_-_]y@url : x+y@url -> y@url
schemas/vegbien.ERD.mwb: updated
fix: inputs/.TNRS/schema.sql: taxon_match: added back * prefixes on TNRS-namespace column names
inputs/.TNRS/schema.sql: MatchedTaxon: taxon_best_match: don't alias to `s` since this is no longer a nested select
inputs/.TNRS/schema.sql: MatchedTaxon: nested select: use just taxon_best_match since this no longer performs renamings
inputs/.TNRS/schema.sql: MatchedTaxon: renamed output columns to match input columns, so that the nested select would not be performing any renamings
fix: inputs/.TNRS/schema.sql: MatchedTaxon: added all columns from taxon_match (some of them apparently hadn't been included in MatchedTaxon)
fix: inputs/.TNRS/schema.sql: reordered columns to match taxon_match
fix: inputs/.TNRS/schema.sql: added back Family_matched, which had gotten merged with Name_matched_accepted_family when TNRS temporarily stopped populating Name_matched_accepted_family
inputs/.TNRS/schema.sql: MatchedTaxon: taxonomicStatus: moved outside the inner SELECT so that the inner SELECT would consist solely of renamings
inputs/.TNRS/schema.sql: MatchedTaxon: removed unneeded "Name_matched." prefix on source-specific names (only the * is needed because there is only one table)
bugfix: schemas/util.sql: view_is_subset(): auto-updatable views that are not plain subsets: include ones with WHERE clauses
bugfix: schemas/util.sql: view_is_subset(): also need to handle auto-updatable views that are not plain subsets (there are some of these)
bin/import_all: hidden_srcs(): removed `by_col=1` because these should be done in the same mode as the main datasources
schemas/public_.sql: removed no longer used threatened_taxonlabel. use iucn_red_list instead.
fix: schemas/public_.sql: analytical_stem_view: don't use threatened_taxonlabel, which was never populated correctly. instead, this can eventually be rewritten to use new iucn_red_list.
bin/make_analytical_db: removed threatened_taxonlabel because this is now handled by iucn_red_list
bin/make_analytical_db: added iucn_red_list_view_modify()
bin/make_analytical_db: removed unused code to create views in the analytical_db schema
bin/make_analytical_db: merged mk_table and mk_analytical_table since they now do the same thing
/README.TXT: Notes on system stability: added warning that when shutting down the VM, one should always first stop Postgres, to prevent the OS from SIGKILLing it
inputs/SALVIAS/Source/header.csv: updated
bugfix: bin/with_all: isset(): need to use `&>/dev/null` instead of `>&-`, etc because closing an fd causes declare to return false
bugfix: bin/with_all, import_all: don't disown processes because they should be auto-killed if the shell is (disown was only needed before we used screen)
bugfix: Full database import: need nested shell to prevent errexit from closing the window
bugfix: schemas/public_.sql: analytical_stem_view_modify(): added columns: need to include COMMENT statements
bugfix: schemas/util.sql: comment(element oid): need `objsubid = 0` filter so this doesn't use comments for any other objsubids
schemas/public_.sql: added iucn_red_list_view_modify() and use it in iucn_red_list_view's "after updating this" instructions
bugfix: schemas/util.sql: rematerialize_view(): also need to handle the target table's dependent views
bugfix: schemas/util.sql: mk_drop_from_create(): need to use `IF EXISTS` so that the resulting statement also works if the target object does not yet exist
schemas/util.sql: mk_drop_from_create(): also support `SELECT util.drop_*()` statements, in addition to DROP
inputs/.TNRS/Source/test.xml.ref: updated
bugfix: /README.TXT: logs glob: don't unintentionally match . or ..
inputs/VegBank/stemlocation_/test.xml.ref: updated
/README.TXT: Full database import: `make test by_col=1`: updated runtime (1 h)
inputs/TEX/Specimen2/test.xml.ref: updated
inputs/GBIF/raw_occurrence_record_plants/test.xml.ref: updated
bugfix: lib/sql_io.py: put_table(): handle_MissingCastException(): when updating join_cols, don't add new entry for join_cols[out_col], only update existing one. this fixes #902 (import bug), and with #902 fixed, #887 (disk space leak) should no longer occur.
bin/import_all: delete_logs(): documented that `trap EXIT` doesn't run until shell exit
bin/import_all: delete_logs(): print when this happens, so it can be verified that it's happening properly
bugfix: bin/import_all: need to run delete_logs manually because `trap EXIT` doesn't run until bg cmds done
bin/import_all: delete_logs: moved testing of whether to delete logs to delete_logs() so that delete_logs() can be run regardless of the $delete_logs setting
bugfix: bin/import_all: delete_logs(): also need to match log filenames when n=""
bugfix: /Makefile: postgres_restart-*: argument to echo always needs quotes
bugfix: /Makefile: $(pg_ctl-*): tell user that if it doesn't work, need to run `rm $(macPostgresDir)/postmaster.pid` and retry
/Makefile: $(pg_ctl-Darwin): documented the pg_ctl version of this command in case `brew services` doesn't work
bugfix: bin/with_all: isset(): need to use `>&- 2>&-` because &> does not work with - as the dest
fix: config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: determine if the revision works: use the in# of the query instead of waiting an arbitrary time, because this is more accurate in determining whether the import is successful
config/VirtualBox_VMs/vegbiendev/README.TXT: regressions that occurred during the revision range: named these "concurrent regressions"
fix: config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: need to update system time so wait times can be calculated correctly
config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: documented that the patches are specifically for regressions that occurred during the revision range, not all bugs that were fixed in that range
bugfix: config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: waiting for query: increased time to min. 10 min because some revisions take longer
fix: config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: bin/import_all: added step to type y and press ENTER
config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: added steps to determine if the revision works, and then stop the import
bugfix: config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: bugfixes: added r12994
config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: bugfixes: use for loop to avoid needing to repeat the command
bugfix: config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: bugfixes: added r12760
bugfix: config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: bugfixes: --fuzz apparently needs to be 3 even though we have only 2 lines of context
bugfix: config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: bugfixes: use --fuzz to allow patching even when surrounding lines have changed
config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: bugfixes: sorted by revision #
bugfix: config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: bugfixes: added r12865
bugfix: config/VirtualBox_VMs/vegbiendev/README.TXT: patch: need to use --force to avoid prompting user
bugfix: config/VirtualBox_VMs/vegbiendev/README.TXT: `svn revert`: need to use `--depth infinity` when reverting more than a file
bugfix: config/VirtualBox_VMs/vegbiendev/README.TXT: use `patch` instead of `svn merge` because `svn merge` sometimes inexplicably doens't apply patches
config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: bugfixes: `svn revert` all changes instead of specific files, to support bugfixes to other files
config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: lib/sh/util.sh bugfix: use `svn merge` instead of `sed --in-place` because this can be used for more complex changes
config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: lib/sh/util.sh bugfix: documented revision
bugfix: config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: need to apply r12872 bugfix to lib/sh/util.sh because it occurred after r12025
fix: config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: `make` commands: use `yes|` to avoid needing to confirm prompts
fix: config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: `make mk_db ...`: use `yes|` to avoid needing to confirm prompts
config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: to test a different revision: added step that if any of the commands after `svn up` are broken, try a different revision
bugfix: config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: `make db`: need to run `schemas/reinstall` manually because older versions of the codebase just used `schemas/install`
config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: documented the range from within the revision should be chosen
fix: inputs/.IRMNG/*/: added _no_import because this datasource is only used as a lookup table and does not need to be imported/scrubbed
config/VirtualBox_VMs/vegbiendev/README.TXT: `svn up`: documented runtime (1 min)
bugfix: added nodes/_no_import, because the NCBI import doesn't work due to a Postgres 9.3 bug (#859). _no_import used to not be needed because we didn't import the hidden sources.
fix: bin/with_all: removed debug statements
bugfix: bin/with_all: testing if @inputs is set: `"${inputs+isset}"` syntax doesn't work for empty arrays, so need to use `declare -p` instead
bugfix: to set up the VM: to use a database backup: also need to `make rm_initial_public` because initdb creates a non-empty DB
config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: added steps to test a different revision
/Makefile: db: use reinstall instead of install to allow running this on an existing DB
config/VirtualBox_VMs/vegbiendev/README.TXT: `sudo apt-get dist-upgrade`: documented runtime (12 min)
config/VirtualBox_VMs/vegbiendev/README.TXT: updated VM snapshot names
fix: config/VirtualBox_VMs/vegbiendev/README.TXT: to install system updates: added step to restart, because some system updates require this
config/VirtualBox_VMs/vegbiendev/README.TXT: added instructions to install system updates
config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: bin/import_all: documented runtime (50 min)
config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: added steps to run the import
config/VirtualBox_VMs/vegbiendev/README.TXT: to set up the VM: added steps to move vegbiendev.vdi to the internal hard drive
config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: added steps to start the VM