Project

General

Profile

Statistics
| Revision:
Name Size Revision Age Author Comment
  _archive 1598 over 12 years Aaron Marcuse-Kubitza Moved _archive/tapir2flatClient/trunk/client/ t...
  analysis 3076 over 12 years Aaron Marcuse-Kubitza Added top-level analysis dir for range modeling
  backups 9496 over 11 years Aaron Marcuse-Kubitza added backups/*.md5
  bin 9785 over 11 years Aaron Marcuse-Kubitza bin/repl: added unescape_html() filter function...
  config 7801 over 11 years Aaron Marcuse-Kubitza root Makefile: VegBIEN DB: mk_db: Added command...
  exports 8798 over 11 years Aaron Marcuse-Kubitza exports/: svn:ignore *.csv
  inputs 9822 over 11 years Aaron Marcuse-Kubitza inputs/GBIF/raw_occurrence_record_plants/run: t...
  lib 9820 over 11 years Aaron Marcuse-Kubitza added lib/runscripts/mysql.table.run (general t...
  mappings 9459 over 11 years Aaron Marcuse-Kubitza bugfix: mappings/VegCore-VegBIEN.csv: place.geo...
  planning 9671 over 11 years Aaron Marcuse-Kubitza *.url: mailto URLs: use the standard e-mail dot...
  schemas 9761 over 11 years Aaron Marcuse-Kubitza bugfix: schemas/vegbien.sql: tnrs_input_name: m...
  web 9786 over 11 years Aaron Marcuse-Kubitza lib/Firefox_bookmarks.reformat.csv: unescape HT...
.htaccess 326 Bytes 8771 over 11 years Aaron Marcuse-Kubitza /.htaccess: use canonical URL without symlinks
Makefile 12.4 KB 9742 over 11 years Aaron Marcuse-Kubitza /Makefile: new config target (part of install):...
README.TXT 22.8 KB 9532 over 11 years Aaron Marcuse-Kubitza bugfix: README.TXT: Full database import: scree...
fix_perms 97 Bytes 7560 over 11 years Aaron Marcuse-Kubitza Added root fix_perms
map 1001 Bytes 6949 almost 12 years Aaron Marcuse-Kubitza vegbien_dest: Changed default $prefix to "", so...
new_terms.csv 38.1 KB 7222 almost 12 years Aaron Marcuse-Kubitza new_terms.csv: Regenerated
run 450 Bytes 9074 over 11 years Aaron Marcuse-Kubitza *{.sh,run}: removed extra space between functio...
unmapped_terms.csv 13.1 KB 7201 almost 12 years Aaron Marcuse-Kubitza **/new_terms.csv, **/unmapped_terms.csv: Regene...

Latest revisions

# Date Author Comment
9822 06/10/2013 11:58 PM Aaron Marcuse-Kubitza

inputs/GBIF/raw_occurrence_record_plants/run: table.tsv.gz/make() runtime: noted that this excludes the upload time

9821 06/10/2013 11:58 PM Aaron Marcuse-Kubitza

inputs/GBIF/raw_occurrence_record_plants/run: added table.tsv.gz/upload() runtime (15 min)

9820 06/10/2013 11:48 PM Aaron Marcuse-Kubitza

added lib/runscripts/mysql.table.run (general to all MySQL datasources) and use it in inputs/GBIF/table.run

9819 06/10/2013 11:13 PM Aaron Marcuse-Kubitza

inputs/GBIF/raw_occurrence_record_plants/run: table.tsv/make(): to view runtime when using `screen`: keys used to scroll: added Ctrl-B/Ctrl-F for page-at-a-time scrolling (there are a lot of pages of output for the import() target!)

9818 06/09/2013 09:21 PM Aaron Marcuse-Kubitza

bugfix: inputs/GBIF/table.run: table.tsv.gz/make(): don't run table.tsv.gz/upload in test mode, to avoid clobbering the backup of a full table.tsv with a partial, testing table.tsv

9817 06/09/2013 09:18 PM Aaron Marcuse-Kubitza

lib/sh/db.sh: set test mode when using limited # rows

9816 06/09/2013 09:08 PM Aaron Marcuse-Kubitza

bugfix: inputs/GBIF/table.run: table.tsv.gz/upload(): don't use inplace mode because it leaves a newer mtime when aborted, causing rsync to think that the partial upload is actually newer than the source. note that rsync's --partial-dir mode is just as capable of resuming an aborted upload (it will just use a file in .rsync-tmp instead). inplace mode is primarily designed for fixed-offset files which don't change much between edits, but this is not true for exports (or the gzips of them), which will change the file offsets of most data if even one row or column is added or removed.

9815 06/09/2013 09:01 PM Aaron Marcuse-Kubitza

bugfix: inputs/GBIF/table.run: table.tsv.gz/make(): run table.tsv.gz/upload here instead of in table.tsv/make() because it should not run until table.tsv.gz is finished being made, which is not the case in table.tsv/make() because table.tsv.gz/make is run in the background

9814 06/09/2013 08:59 PM Aaron Marcuse-Kubitza

inputs/GBIF/table.run: table.tsv.gz/upload(): moved before table.tsv.gz/make() so it can be used by it

9813 06/09/2013 08:39 PM Aaron Marcuse-Kubitza

bugfix: inputs/GBIF/table.run: table.tsv.gz/upload(): need overwrite=1 because the mtime of an aborted inplace upload is newer

View all revisions | View revisions

Also available in: Atom