Activity
From 12/08/2013 to 01/06/2014
12/20/2013
- 10:45 PM Revision 11939: fix: planning/timeline/timeline.xls: realigned legend
- 10:44 PM Revision 11938: planning/timeline/timeline.xls: hid previous weeks
- 10:43 PM Revision 11937: planning/timeline/timeline.xls: crossed out and hid completed tasks
- 10:41 PM Revision 11936: planning/timeline/timeline.xls: updated for progress
- 05:41 PM Revision 11935: web/links/index.htm: updated to Firefox bookmarks. open-sourcing BIEN: added links for VegBank's license (GPLv2), and how these terms apply to us (a diff of our changes is not GPL-ed under GPLv2, although it is claimed to be GPL-ed under GPLv3)
- 04:41 PM Revision 11934: inputs/VegBank/^taxon_observation.**.sample/create.sql, map.csv: added new project columns
- 04:31 PM Revision 11933: inputs/VegBank/taxon_observation.**/postprocess.sql: added the project table
- 04:25 PM Revision 11932: mapped inputs/VegBank/project/, which includes the projectName for attribution
- 02:56 PM Revision 11931: inputs/CVS/^taxon_observation.**.sample/create.sql, map.csv: added new project columns
- 02:44 PM Revision 11930: inputs/CVS/taxon_observation.**/postprocess.sql: added the project table
- 02:42 PM Revision 11929: inputs/CVS/project/map.csv: mapped stopDate->projectEndDate
- 02:35 PM Revision 11928: mapped inputs/CVS/project/, which includes the projectName for attribution
- 01:25 AM Revision 11927: inputs/VegBIEN/Redmine/svn/.htaccess: updated to use *much* faster direct repository URL rather than Redmine web interface, now that the repository itself is publicly accessible in addition to the Redmine view of it
- 01:18 AM Revision 11926: planning/timeline/timeline.xls: updated for progress
- 01:13 AM Revision 11925: planning/timeline/timeline.2013.xls: renamed to timeline.xls so that this can continue to be used for 2014 (leaving a symlink from the old filename to preserve permalinks)
- 12:28 AM Revision 11924: fix: inputs/TEX/Specimen*/map.csv, postprocess.sql: habitat: also placed in occurrenceRemarks so that this field gets parsed for growth form information, as requested by Brad (wiki.vegpath.org/TEX_validation#2013-2-26)
12/19/2013
- 11:49 PM Revision 11923: fix: inputs/TEX/Specimen*/map.csv: mapped constant values for specimenHolderInstitutions, country. these have to be added with `rm=1 ./inputs/TEX/Specimen.../run postprocess`.
- 11:42 PM Revision 11922: bugfix: inputs/TEX/Specimen2/map.csv: mapped BARCODE to accessionNumber so that we have a unique ID for each row
- 11:11 PM Revision 11921: schemas/vegbien.sql: analytical_stem_view: scientificName_verbatim: don't use taxonverbatim.taxonname+author as the scientificName_verbatim if only the author is provided. (this lead to weird scientificName_verbatims that contain just the author.)
12/17/2013
- 08:06 AM Revision 11920: inputs/datasource_release_status.xlsx: updated
- 07:28 AM Revision 11919: web/links/index.htm: updated to Firefox bookmarks. added links for fixing the "App Store is temporarily unavailable" error (turn on Spotlight) and modifying a running shell script (unlink it first).
- 05:47 AM Revision 11918: bugfix: bin/map: in_is_db: don't ignore errors when the table does not exist, because these prevent an errexit and allow an import to continue when a staging table is missing. suppressing this error had previously been necessary because metadata-only tables (Source/) used to not have installed staging tables, and the program had to react accordingly.
12/16/2013
- 07:05 PM Revision 11917: inputs/CVS/^taxon_observation.**.sample/create.sql: added Mike Lee's additional plots used to validate confidentiality-related fields (wiki.vegpath.org/CVS_validation#plots-to-include)
- 06:00 PM Revision 11916: bugfix: inputs/CVS/^taxon_observation.**.sample/create.sql: include taxonName in the subset of columns that's imported for the validation, because it is _alt-ed with scientificName for forming the TNRS input name. this is unique to CVS, which is why it was not part of the validation subset copied from the VegBank subset.
- 05:46 PM Revision 11915: /README.TXT: Full database import: documented that you should always start with a clean shell, which does not have changes to the env vars. (there have been inexplicable bugs that went away after closing and reopening the terminal window.) note that running `exec bash` is not sufficient to *reset* the env vars.
- 04:58 PM Revision 11914: fix: lib/sh/util.sh: verbosity_min(): usage: clarified that '' is a special value that causes $verbosity to be overwritten to ''
- 04:45 PM Revision 11913: lib/runscripts/table.run: added test_() target and use it in remake_VegBIEN_mappings() (it would not be clear that remake_VegBIEN_mappings() runs the tests)
- 01:43 PM Revision 11912: bugfix: inputs/.TNRS/schema.sql: granted bien_read SELECT access to derived views as well as the core tnrs table
12/15/2013
- 05:30 PM Revision 11911: updated inputs/datasource_release_status.xlsx
- 05:27 PM Revision 11910: added inputs/datasource_release_status.xlsx, export of Google spreadsheet at https://docs.google.com/spreadsheet/ccc?key=0ArZXrTAXd-TYdDRRb2RxYi11TWZrQVh5bVdKOURCeFE
12/12/2013
- 08:57 AM Revision 11909: planning/timeline/timeline.2013.xls: updated for progress
- 08:35 AM Revision 11908: bugfix: schemas/vegbien.sql: location: use the place_id from the parent location when no place_id is specified. this fixes a bug in analytical_stem_view where the parent location's place_id was used because it was sometimes missing from the sublocation, but the parent place_id *itself* was sometimes missing instead if sublocations each had their own place information. this way, it is always available directly in the sublocation, populated from the parent location if needed.
- 08:27 AM Revision 11907: bugfix: schemas/vegbien.sql: location: added place_id which is autopopulated from the current locationplace. join on this in plot.**, to avoid a 1:many join when a location has multiple locationplaces.
12/11/2013
- 11:10 PM Revision 11906: bugfix: schemas/vegbien.sql: locationevent_unique_within_parent_by_location unique index: need COALESCE() around location_id since it's nullable
- 10:54 PM Revision 11905: fix: inputs/CVS/^taxon_observation.**.sample/: added _no_import because this table duplicates part of what's imported from taxon_observation.**
- 10:42 PM Revision 11904: bugfix: inputs/VegBank/plot/: added _no_import because this table is left-joined and should not be imported separately
- 10:40 PM Revision 11903: bugfix: inputs/{.NCBI,CTFS}/*.src/: added _no_import because these tables are left-joined and should not be imported separately
- 09:56 PM Revision 11902: inputs/import.stats.xls: removed table names from datasources where only one table is imported
- 09:52 PM Revision 11901: fix: inputs/import.stats.xls: removed deleted tables from current import
- 09:51 PM Revision 11900: inputs/import.stats.xls: updated import times
- 07:56 PM Revision 11899: updated backups/TNRS.backup.md5
- 07:56 PM Revision 11898: added backups/vegbien.r11786.backup.md5
- 07:53 PM Revision 11897: /README.TXT: Full database import: backups: added step to download backup to local machine
- 07:45 PM Revision 11896: bugfix: /Makefile: install: need to run inputs/download in live mode so that the flat files are actually downloaded
- 07:43 PM Revision 11895: lib/common.Makefile: added %/live, for use with `make inputs/download`
12/10/2013
- 07:44 AM Revision 11894: planning/timeline/timeline.2013.xls: rescheduled tasks
- 07:40 AM Revision 11893: planning/timeline/timeline.2013.xls: updated for progress
- 07:36 AM Revision 11892: /README.TXT: Full database import: In PostgreSQL: documented that the tables to check are located in the *r# schema*, not public
- 07:32 AM Revision 11891: planning/timeline/timeline.2013.xls: updated for progress
- 07:32 AM Revision 11890: planning/timeline/timeline.2013.xls: datasource validations: reordered datasources according to Brian Enquist's new validation order (wiki.vegpath.org/Spot-checking_validation_order)
- 07:10 AM Revision 11889: fix: schemas/vegbien.sql: analytical_specimen: added specimens-related columns that are in analytical_plot
- 06:35 AM Revision 11888: inputs/GBIF/raw_occurrence_record_plants/map.csv: row_num: remapped to plain *row_num, like the other datasources that have this field
- 06:31 AM Revision 11887: inputs/GBIF/raw_occurrence_record_plants/postprocess.sql: Remove institutions that we have direct data for: rerun time: noted that this is only fast *after* manual vacuuming of the table (to remove the deleted rows from the index). autovacuum apparently does not run, although it should.
- 05:18 AM Revision 11886: planning/timeline/timeline.2013.xls: hid previous weeks
- 05:18 AM Revision 11885: planning/timeline/timeline.2013.xls: added timespan dots ◦ for supertasks
- 05:15 AM Revision 11884: planning/timeline/timeline.2013.xls: legend: changed to movable text box to avoid needing to erase and repopulate the header columns with the legend cells
- 05:03 AM Revision 11883: planning/timeline/timeline.2013.xls: crossed out and hid completed tasks
- 04:58 AM Revision 11882: planning/timeline/timeline.2013.xls: updated for progress
12/09/2013
- 07:24 PM Revision 11881: inputs/GBIF/raw_occurrence_record_plants/test.xml.ref: reran test, which added yearCollected/monthCollected/dayCollected
- 07:23 PM Revision 11880: inputs/CVS/plantConcept_/create.sql: documented runtime (3 min)
- 06:59 PM Revision 11879: inputs/CTFS/*.src/: added test.xml.ref
- 06:58 PM Revision 11878: inputs/CTFS/*.src/: added VegBIEN.csv
- 06:56 PM Revision 11877: bugfix: inputs/CTFS/TaxonOccurrence*/map.csv: things mapped to taxonObservationID: remapped to taxonOccurrenceID since taxonObservationID is not mapped to anything in VegBIEN (denormalized VegCore doesn't distinguish between taxon occurrences and taxon observations of them)
- 05:46 PM Revision 11876: bugfix: inputs/ARIZ/~.clean_up.sql: prevent "column already exists" errors when there is an input column of the same name as an output column
- 05:44 PM Revision 11875: bugfix: lib/runscripts/datasrc_dir.run: import(): don't run `sql/install` if the schema already exists, because this will try to rerun all the schema-creation queries. note that this idempotent functionality was *not* provided by the `make .../install` target that was previously used (idempotency is new with new-style import).
- 05:26 PM Revision 11874: bugfix: schemas/vegbien.sql: updated for renamed county_centroids column names
- 04:16 PM Revision 11873: inputs/.geoscrub/import_order.txt: added county_centroids so that it would be installed by new-style import
- 03:54 PM Revision 11872: bugfix: lib/runscripts/datasrc_dir.run: import(): can't run `datasrc_make reinstall` anymore because this now defers to the runscript for new-style import datasources (which was done so that `make .../install` properly reinstalls all the datasources). instead, call the applicable make targets manually (there are just 2 of them).
- 03:37 PM Revision 11871: inputs/FIA/TREE/run: documented import() runtime (1.5 h), which includes table cleanup runtime (1 h)
- 03:09 PM Revision 11870: bugfix: bin/pg_dump_limit: support errexit by ignoring the nonzero exit status that grep returns when it doesn't match anything
- 02:43 PM Revision 11869: inputs/GBIF/raw_occurrence_record_plants/run: updated import() runtime (same), documented table cleanup runtime (1.5 h)
- 02:38 PM Revision 11868: inputs/GBIF/raw_occurrence_record_plants/postprocess.sql: CREATE INDEX ... specimenHolderInstitutions: documented runtime (45 min)
- 02:28 PM Revision 11867: inputs/GBIF/raw_occurrence_record_plants/postprocess.sql: Remove institutions that we have direct data for: documented runtime (3.5 min)
- 02:27 PM Revision 11866: /README.TXT: Datasource setup: added steps to backup e-mails
Also available in: Atom