Project

General

Profile

Statistics
| Revision:
Name Size Revision Age Author Comment
  BIEN2 10896 about 11 years Aaron Marcuse-Kubitza added BIEN2/traits_observation_counts.xls
  _archive 1598 over 12 years Aaron Marcuse-Kubitza Moved _archive/tapir2flatClient/trunk/client/ t...
  backups 10873 about 11 years Aaron Marcuse-Kubitza added backups/vegbien.r10848.backup.md5
  bin 10871 about 11 years Aaron Marcuse-Kubitza bugfix: bin/import_all: use reimport_scrub inst...
  config 7801 over 11 years Aaron Marcuse-Kubitza root Makefile: VegBIEN DB: mk_db: Added command...
  derived 10897 about 11 years Aaron Marcuse-Kubitza added derived/biengeo/Geovalidation_and_geoscru...
  exports 10853 about 11 years Aaron Marcuse-Kubitza exports/: svn:ignore *.zip
  inputs 11007 about 11 years Aaron Marcuse-Kubitza inputs/VegBank/stemcount_/postprocess.sql: move...
  lib 11000 about 11 years Aaron Marcuse-Kubitza bugfix: lib/runscripts/*: calls to rm: use `rm ...
  mappings 10848 about 11 years Aaron Marcuse-Kubitza bugfix: mappings/VegCore-VegBIEN.csv: don't map...
  planning 10982 about 11 years Aaron Marcuse-Kubitza planning/timeline/timeline.2013.xls: updated fo...
  schemas 11005 about 11 years Aaron Marcuse-Kubitza schemas/util.sql: added ||% operator to append ...
  web 10894 about 11 years Aaron Marcuse-Kubitza web/links/index.htm: updated to Firefox bookmar...
.htaccess 326 Bytes 8771 over 11 years Aaron Marcuse-Kubitza /.htaccess: use canonical URL without symlinks
.rsync_filter.upload 33 Bytes 10042 over 11 years Aaron Marcuse-Kubitza /.rsync_ignore: temp files: hide them on upload...
.rsync_ignore 12 Bytes 10042 over 11 years Aaron Marcuse-Kubitza /.rsync_ignore: temp files: hide them on upload...
Makefile 12.6 KB 10539 over 11 years Aaron Marcuse-Kubitza bugfix: /Makefile: postgres-Linux: phppgadmin.c...
README.TXT 23.6 KB 10981 about 11 years Aaron Marcuse-Kubitza bugfix: /README.TXT: to backup files not in Tim...
fix_perms 97 Bytes 7560 almost 12 years Aaron Marcuse-Kubitza Added root fix_perms
map 1001 Bytes 6949 almost 12 years Aaron Marcuse-Kubitza vegbien_dest: Changed default $prefix to "", so...
new_terms.csv 38.1 KB 7222 almost 12 years Aaron Marcuse-Kubitza new_terms.csv: Regenerated
run 661 Bytes 10881 about 11 years Aaron Marcuse-Kubitza /run: geoscrub_input/make(): updated runtime (2...
unmapped_terms.csv 13.1 KB 7201 almost 12 years Aaron Marcuse-Kubitza **/new_terms.csv, **/unmapped_terms.csv: Regene...

Latest revisions

# Date Author Comment
11007 09/18/2013 10:28 PM Aaron Marcuse-Kubitza

inputs/VegBank/stemcount_/postprocess.sql: moved stemcount___parent index before the derived columns section because it does not depend on them

11006 09/18/2013 10:26 PM Aaron Marcuse-Kubitza

bugfix: inputs/VegBank/stemcount_/postprocess.sql: added missing index on taxonOccurrenceID, needed for the 1:many portion of the taxon_observation.** left-join

11005 09/18/2013 10:14 PM Aaron Marcuse-Kubitza

schemas/util.sql: added ||% operator to append to escaped strings (the % indicates an identifier, as in Perl hashes and one of the x86 assembler syntaxes for registers

11004 09/18/2013 03:50 PM Aaron Marcuse-Kubitza

bugfix: inputs/VegBank/taxon_observation.**/postprocess.sql: added sort_col (=identificationID) at beginning because column-based import will always sort a view by the first column, which may lead to slow query plans if the first column is not a joined table's pkey

11003 09/18/2013 02:04 PM Aaron Marcuse-Kubitza

inputs/VegBank/taxon_observation.**/postprocess.sql: documented that there is no row_num because left-join to stemcount_, stemlocation_ adds rows to each taxonobservation_

11002 09/18/2013 02:03 PM Aaron Marcuse-Kubitza

bugfix: inputs/VegBank/taxon_observation.**/postprocess.sql: removed row_num (=identificationID), because there is actually more than one row per VegBank taxonobservation_, so this does not properly enumerate the view rows. this is because there is a 1:many left-join to stemcount_, stemlocation_ which adds rows to each taxonobservation_. since the row_num is gone, any row-subsetting of the view using OFFSET will always need to materialize the entire view up to the OFFSET value. this works for smaller datasources like VegBank that fit almost entirely into one column-based import chunk (1 million rows), but not for larger datasources like FIA where it would be much slower to materialize all preceding 16 million rows on the last chunk (which is what OFFSET normally does with left-joins).

11001 09/18/2013 01:51 PM Aaron Marcuse-Kubitza

bugfix: inputs/VegBank/taxon_observation.**/: generated header.csv and related files, which were previously not generated because the error in `rm header.csv` aborted the runscript

11000 09/17/2013 10:05 PM Aaron Marcuse-Kubitza

bugfix: lib/runscripts/*: calls to rm: use `rm -f` instead to avoid an error (which aborts the program) if the file does not yet exist

10999 09/16/2013 07:51 AM Aaron Marcuse-Kubitza

inputs/VegBank/: added taxon_observation.** left-join of the tables, using the steps at http://wiki.vegpath.org/Left-joining_a_datasource

10998 09/16/2013 07:48 AM Aaron Marcuse-Kubitza

inputs/VegBank/taxonobservation_/create.sql: join starting with taxoninterpretation so that we can use the taxoninterpretation_id as the row_num (text strings, formed from concatenated #s cannot be used as a row_num). there is only 1 taxonobservation without a taxoninterpretation, so we can just include one row for each taxoninterpretation.

View all revisions | View revisions

Also available in: Atom