Project

General

Profile

Statistics
| Revision:
Name Size Revision Age Author Comment
  _archive 1598 over 12 years Aaron Marcuse-Kubitza Moved _archive/tapir2flatClient/trunk/client/ t...
  analysis 3076 over 12 years Aaron Marcuse-Kubitza Added top-level analysis dir for range modeling
  backups 10752 about 11 years Aaron Marcuse-Kubitza updated backups/TNRS.backup.md5
  bin 10742 about 11 years Aaron Marcuse-Kubitza bin/tnrs_db: add entry to new batch table
  config 7801 over 11 years Aaron Marcuse-Kubitza root Makefile: VegBIEN DB: mk_db: Added command...
  derived 10707 about 11 years Aaron Marcuse-Kubitza added derived/biengeo/ from https://projects.nc...
  exports 9928 over 11 years Aaron Marcuse-Kubitza added exports/_archive/
  inputs 10798 about 11 years Aaron Marcuse-Kubitza inputs/input.Makefile: rm: use new datasource_r...
  lib 10781 about 11 years Aaron Marcuse-Kubitza added lib/runscripts/data.pg.sql.run (analogous...
  mappings 10722 about 11 years Aaron Marcuse-Kubitza mappings/VegCore-VegBIEN.csv: mapped datasetURL
  planning 10805 about 11 years Aaron Marcuse-Kubitza planning/timeline/timeline.2013.xls: added sepa...
  schemas 10802 about 11 years Aaron Marcuse-Kubitza schemas/vegbien.sql: datasource_rm(): runtime: ...
  web 10705 about 11 years Aaron Marcuse-Kubitza web/links/index.htm: updated to Firefox bookmar...
.htaccess 326 Bytes 8771 over 11 years Aaron Marcuse-Kubitza /.htaccess: use canonical URL without symlinks
.rsync_filter.upload 33 Bytes 10042 over 11 years Aaron Marcuse-Kubitza /.rsync_ignore: temp files: hide them on upload...
.rsync_ignore 12 Bytes 10042 over 11 years Aaron Marcuse-Kubitza /.rsync_ignore: temp files: hide them on upload...
Makefile 12.6 KB 10539 over 11 years Aaron Marcuse-Kubitza bugfix: /Makefile: postgres-Linux: phppgadmin.c...
README.TXT 23.5 KB 10804 about 11 years Aaron Marcuse-Kubitza /README.TXT: Single datasource import: run comm...
fix_perms 97 Bytes 7560 over 11 years Aaron Marcuse-Kubitza Added root fix_perms
map 1001 Bytes 6949 almost 12 years Aaron Marcuse-Kubitza vegbien_dest: Changed default $prefix to "", so...
new_terms.csv 38.1 KB 7222 almost 12 years Aaron Marcuse-Kubitza new_terms.csv: Regenerated
run 433 Bytes 9916 over 11 years Aaron Marcuse-Kubitza /run: geoscrub_input/make(): documented runtime...
unmapped_terms.csv 13.1 KB 7201 almost 12 years Aaron Marcuse-Kubitza **/new_terms.csv, **/unmapped_terms.csv: Regene...

Latest revisions

# Date Author Comment
10805 08/30/2013 01:39 AM Aaron Marcuse-Kubitza

planning/timeline/timeline.2013.xls: added separate task for Individual datasource refresh (separate from Individual datasource removal), because we also need to optimize the reload of datasources. the reload is most likely slow because rows are being added to very large tables.

10804 08/30/2013 01:21 AM Aaron Marcuse-Kubitza

/README.TXT: Single datasource import: run commands in the background, since these are long-running commands

10803 08/30/2013 12:57 AM Aaron Marcuse-Kubitza

planning/timeline/timeline.2013.xls: moved Attribution and conditions of use before Flatten the datasources as suggested in meeting with Mark

10802 08/30/2013 12:42 AM Aaron Marcuse-Kubitza

schemas/vegbien.sql: datasource_rm(): runtime: added runtime of MO (55 min, 0.85 ms/row), which has a much larger # of rows than ACAD (4 million instead of 45,000). updated GBIF runtime estimate (~13 h) with more accurate ms/row from MO.

10801 08/29/2013 11:19 PM Aaron Marcuse-Kubitza

schemas/vegbien.sql: datasource_rm(): estimated runtime for GBIF (~10 h). note that this is still significantly shorter than the import time (3.4 days).

10800 08/29/2013 11:11 PM Aaron Marcuse-Kubitza

schemas/vegbien.sql: datasource_rm(): documented how to calculate runtime

10799 08/29/2013 11:04 PM Aaron Marcuse-Kubitza

schemas/vegbien.sql: datasource_rm(): documented runtime for ACAD: 30 s; 0.61 ms/row

10798 08/29/2013 05:09 PM Aaron Marcuse-Kubitza

inputs/input.Makefile: rm: use new datasource_rm(), which encapsulates the schema-specific aspects of removing a datasource

10797 08/29/2013 04:59 PM Aaron Marcuse-Kubitza

bugfix: schemas/vegbien.sql: datasource_rm(): set_config(): don't name the is_local param because it is not a named parameter

10796 08/29/2013 04:50 PM Aaron Marcuse-Kubitza

schemas/vegbien.sql: added datasource_rm(). this uses an internal schema-scoping parameter to ensure that the function always operates on tables in the schema it was defined in, rather than tables in the search_path. this ensures that when the public schema is renamed (e.g. from an imported version), the function will continue to operate on its own schema rather than whichever schema happens to be called public. this avoids any surprises if you are trying to remove a datasource in one schema, and don't want it to unintentionally be removed in another schema instead.

View all revisions | View revisions

Also available in: Atom