fix: inputs/input.Makefile: add!: verify/: also svn:ignore .zip
lib/sh/util.sh: added become_sudo alias
lib/sh/util.sh: added $is_root
inputs/publishable datasources.xlsx: updated: replaced BIEN2 traits with Cyrille traits
exports/view_full_occurrence_CVS.csv.run: documented runtime (2 min for 5000 rows)
fix: inputs/publishable datasources.xlsx: updated: conditions of use: Canadensys sources: these should actually be marked as no restrictions, in contrast to Brian E's earlier annotations, because they are public domain: the citation is requested, but not legally required
inputs/publishable datasources.xlsx: updated
inputs/publishable datasources.xlsx: updated: conditions of use: indicated which datasources have no restrictions
web/BIEN3/Redmine/issues/.htaccess: dest: decoded chars that don't need to be escaped, for clarity
bugfix: web/BIEN3/Redmine/issues/.htaccess: need to store dest in separate var because % is treated specially in the replacement str
inputs/CVS/^taxon_observation.**.sample/test.xml.ref: updated
fix: inputs/CVS/plantConcept_/header.csv: regenerated after fixing the postprocess/cleanup ordering bug (r14827), which had caused header.csv to be incorrectly generated after renames in postprocess.sql were applied
web/BIEN3/Redmine/issues/.htaccess: main page: sort by priority
planning/meetings/BIEN conference call availability.xlsx: updated
planning/meetings/BIEN conference call availability.xlsx: updated: added and populated fall 2014 tab
bugfix: inputs/input.Makefile: postprocess must be run after cleanup rather than before because it depends on the cleanup having been performed.
this bug was not previously detected because this is only a problem when refreshing a datasource to data in the same format: this would attempt to run an existing postprocess.sql, out of order, instead of starting with no postprocess.sql as we usually do....
lib/sql_io.py: cleanup_table(): trim(): documented that this also converts character varying fields to text
bugfix: inputs/input.Makefile: $(dbExports): also need to put data.sql before clean_up.sql, etc. previously, this ordering had to be done by naming clean_up.sql, etc so they would sort after data.sql alphabetically, but it can be confusing to have to remember to do this. this fixes a bug in the CVS refresh where cvs.~.clean_up.sql was being run before data.sql, causing some private columns to have been deleted before the data was imported into the tables, creating a column mismatch error.
/README.TXT: To re-run geoscrubbing: re-create public-schema views that were cascadingly deleted: listed which views these currently are
/README.TXT: To re-run geoscrubbing: updated runtimes
bugfix: lib/sql_io.py: null_strs_str_default: removed "NA" because this is the abbr for a Spanish province (Navarra). this fixes the 2nd bug of #955, geovalidation duplicated rows.
lib/sql_io.py: cleanup_table(): debug-print null_strs
lib/sql_io.py: null_strs: made it customizable from an env var, since the same list of null_strs doesn't work for all datasources (see #957)
inputs/Cyrille_traits/Makefile: set custom $(null_strs) which handles "NA"
inputs/input.Makefile: pass make var $(null_strs) to invoked commands so it can be used by lib/sql_io.py
fix: *Makefile: changed line endings to \n so that `patch` can work with pasted input. use `svn di --extensions --ignore-eol-style` to verify no diff.
bugfix: web/links/index.htm: updated to Firefox bookmarks: Mac: automatic restart on power loss: also need to disable the UPSShutdownAtLevel/UPSShutdownAtMinutesLeft functions because an orderly shutdown prevents the computer from auto-restarting on power loss. Bookmarks Toolbar: reordered links to put those most relevant to VegBIEN users first.
bugfix: config/dev_machine/com.apple.PowerManagement.plist: disable the UPSShutdownAtLevel/UPSShutdownAtMinutesLeft functions because an orderly shutdown prevents the computer from auto-restarting on power loss
bugfix: /README.TXT: Datasource setup: For MS Access databases: also need to comment out `CREATE DATABASE` because pg_dump_limit is only applied to data.sql (due to requirements of the TNRS datasource)
added inputs/CVS/_src/cvs-eep-archive-2014-10-07-correctedCVSData.{data,schema}.sql.ini
bugfix: inputs/CVS/_src/{data,schema}.sql.ini: sourcefilename: this needs to be on the VM's own HD to avoid crashing MSAccess to PostgreSQL. destinationdatabase: added this back since it is fine to leave this blank.
added inputs/CVS/_src/cvs-eep-archive-2013-10-22-VegBIEN.{data,schema}.sql.ini
added inputs/CVS/_src/{data,schema}.sql.ini
added backups/vegbien.r14673.backup.md5
inputs/publishable datasources.xlsx: updated: use white text on dark backgrounds for better visibility, and to create more visual contrast for the unredistributable indicators
inputs/publishable datasources.xlsx: updated: consolidated Brian E's new columns into a single "conditions?" column. fix: "what is needed to publish it": renamed to "conditions of use/remaining tasks". "conditions of use": changed color scheme to match "publishable?" columns.
added inputs/CVS/verify/Review of CVS data in BIEN3-RKP2014Sept7-Revised.docx from Bob
bugfix: lib/tnrs.py: encode_map: also need to encode + because TNRS removes it from the morphospecies (vegpath.org/wiki/CVS_validation#Bobs-revised-document > issue #4)
/README.TXT: To re-run geoscrubbing: also need to `unset TMOUT` because TMOUT causes shell to exit even with background processes
config/dev_machine/com.apple.PowerManagement.plist: UPSShutdownAtLevel: decreased to 5% to allow working longer on UPS power
config/dev_machine/com.apple.PowerManagement.plist: UPSShutdownAtMinutesLeft: decreased to 3 min to allow working longer on UPS power
bugfix: inputs/CVS/plot_/postprocess.sql: locality: site_name should come before directions_to_place because it is at a higher level of granularity
/README.TXT: To re-run geoscrubbing: `rm=1 inputs/.geoscrub/run`: should just need to reinstall the geoscrub_output table
schemas/public_.sql: geoscrub_input_view_modify(): add unique constraint on the same columns as geoscrub.geoscrub_output, as a sanity check to assist future debugging
bugfix: schemas/postgresql.conf: need to set extra_float_digits = 3 (the maximum) to enable verbatim export. this should fix issue #955.
added inputs/CVS/verify/Review of CVS data in BIEN3-RKP2014Sept7.docx from Bob
added config/dev_machine/com.apple.PowerManagement.plist from /Library/Preferences/SystemConfiguration/com.apple.PowerManagement.plist , so that this file can be easily restored when the system spuriously resets settings in it
/README.TXT: To re-run geoscrubbing: updated runtimes. note that these are faster on the new hardware.
fix: /README.TXT: To re-run geoscrubbing: `rm=1 inputs/.geoscrub/geoscrub_output/geoscrub.csv.run`: use a subshell to avoid closing screen on errexit
fix: /README.TXT: Single datasource refresh: also back up DB after this
/README.TXT: datasource removal: documented bien2_traits runtime
added inputs/bien2_traits/_no_import since bien2_traits has been replaced by Cyrille_traits
/README.TXT: added datasource removal instructions
fix: /README.TXT: Single datasource refresh: place updated extract: generalized datasrc name
added inputs/Cyrille_traits/
lib/sql_io.py: null_strs: added N/A and NA (this will remove a common abbr for North America, but we don't use the continent, so this is OK)
mappings/VegCore.htm: regenerated from wiki. made verbatimLocality a synonym of locality since they are used to store the same data.
bugfix: schemas/util.sql: col_rename(): also need to run util.name_truncate() on the names used in the "don't try to rename to itself" comparison
fix: schemas/util.sql: col_rename(): don't try to rename a column to itself (this will cause an error)
schemas/util.sql: set_col_names(): use util.col_rename() so that all renames uses this function
schemas/util.sql: added col_rename()
schemas/util.sql: added quote_ident() wrapper
schemas/util.sql: added name_truncate()
fix: inputs/input.Makefile: $(nonHeaderSrcs): updated to exclude new header.txt
lib/runscripts/table.run: added check_headers()
bugfix: lib/runscripts/table.run: srcs: missing []
srcs: missing [
lib/runscripts/table.run: added header() and use it in header.txt()
lib/runscripts/table.run: 1st_src(): use a variable for this instead, to avoid needing to run this function each time it's used, and to make @srcs available
lib/sh/util.sh: wildcard.(): clarified that it only removes . .. when at then beginning of the list
fix: schemas/public_.sql: view_full_occurrence_individual_view: materialize time: updated revision
schemas/public_.sql: view_full_occurrence_individual_view: materialize time: updated (2 days)
web/links/index.htm: updated to Firefox bookmarks: added bash shellshock links. LibreOffice: added steps to convert files from the command line.
lib/runscripts/table.run: added header.txt()
lib/runscripts/table.run: added 1st_src()
inputs/input.Makefile: added %/list_srcs
lib/runscripts/subdir.run: subdir_make(): use new $datasrc_dir
lib/runscripts/subdir.run: added $datasrc_dir
lib/sh/make.sh: make(): added support for $output_data mode which uses --silent
fix: lib/sh/util.sh: already_exists_msg(): changed calling convention to avoid it seeming like `return 0` is run if already_exists_msg() throws an error, when in fact already_exists_msg() is just a command that should be run before returning/errexiting
fix: inputs/input.Makefile: need to escape $ in commands, including inside comments
bugfix: lib/common.Makefile: $(wildcard/): need final pass with $(wildcard) to support inputs without wildcard chars
bugfix: inputs/input.Makefile: `$(call add*,$(svnFiles))` must be invoked externally to clear the $(wildcard) cache before expanding $(svnFiles)
/README.TXT: Installation: added instructions for inputs/download/live prompts
/README.TXT: Installation: added instructions for MySQL installation prompts
fix: /README.TXT: Installation: "Are you sure you want to continue connecting" prompt comes before "aaronmk@jupiter's password" prompt
/Makefile: phppgadmin-Linux: removed no longer needed $(editApacheConfForPhppgadmin), as phpPgAdmin no longer uses this file (/etc/phppgadmin/apache.conf)
/Makefile: phppgadmin-Linux: removed no longer needed $(editPhppgadminApacheConf)
/Makefile: misc-Linux: added htop
/Makefile: misc-Linux: added jedit
/Makefile: postgres-Linux: added pgadmin3
bugfix: /Makefile: $(postgresReload-*) calls: do not prepend $(MAKE) or - because these are multi-command functions
bugfix: /Makefile: $(bien*Password): need to use = instead of := because these files are not available until config/download/live has run
fix: /README.TXT: Installation: added instructions for "aaronmk@jupiter's password" prompt
/README.TXT: Installation: added step to open a terminal window
fix: /README.TXT: Installation: added instructions for "Are you sure you want to continue connecting" prompt
inputs/VegBank/run*.log: updated. this adds the function call context in addition to the function location.