Project

General

Profile

Activity

From 09/23/2013 to 10/22/2013

10/22/2013

10:22 PM Revision 11409: schemas/VegCore/ERD/VegCore.ERD.mwb: relationships: HAS-A: added HAS-MANY going in the opposite direction, because every HAS-A has an opposite HAS-MANY
Aaron Marcuse-Kubitza
10:17 PM Revision 11408: schemas/VegCore/ERD/VegCore.ERD.mwb: relationships: IS-A, HAS-A: added directional arrows
Aaron Marcuse-Kubitza
10:10 PM Revision 11407: schemas/VegCore/ERD/VegCore.ERD.mwb: field order box: removed spacing between top of text box and bottom of outer box label
Aaron Marcuse-Kubitza
10:04 PM Revision 11406: schemas/VegCore/ERD/VegCore.ERD.mwb: regenerated exports and udpated image map
Aaron Marcuse-Kubitza
10:00 PM Revision 11405: schemas/VegCore/ERD/VegCore.ERD.mwb: reordered columns according to the field order convention
Aaron Marcuse-Kubitza
09:49 PM Revision 11404: schemas/VegCore/ERD/VegCore.ERD.mwb: added label documenting the field order convention:
1) inherited
2) required
3) identifying
4) foreign key
5) extenders
6) others
Aaron Marcuse-Kubitza
08:34 PM Revision 11403: web/links/index.htm: updated to Firefox bookmarks. added links for EER models, data management plans. put PostgreSQL before MySQL because we have found PostgreSQL to be a much more capable database system, even though it lacks some of MySQL's user-friendly features.
Aaron Marcuse-Kubitza
06:38 PM Revision 11402: planning/timeline/timeline.2013.xls: updated for progress
Aaron Marcuse-Kubitza
06:21 PM Revision 11401: fix: schemas/vegbien.sql: analytical_stem_view: renamed specimens columns to use the VegCore names, where these differ from DwC, so that the now-VegCore staging table column names are the same as the analytical_stem_view column names
Aaron Marcuse-Kubitza
06:16 PM Revision 11400: schemas/vegbien.sql: regenerated using `make schemas/remake`. note that analytical_stem_view column renamings need this step after a search-and-replace of the column names, in order to remove excess "" around all-lowercase names and reset generated index names.
Aaron Marcuse-Kubitza
06:10 PM Revision 11399: fix: schemas/vegbien.sql: analytical_stem_view: renamed specimens columns to use the VegCore names, where these differ from DwC, so that the now-VegCore staging table column names are the same as the analytical_stem_view column names
Aaron Marcuse-Kubitza
01:20 PM Revision 11398: added planning/goals/web_interface/phpPgAdmin.select_interface.png for use at wiki.vegpath.org/Proposed_enhancements
Aaron Marcuse-Kubitza
09:39 AM Revision 11397: inputs/CVS/_src/: added refresh from Mike Lee
Aaron Marcuse-Kubitza

10/21/2013

07:14 PM Revision 11396: fix: bin/map: put template: comment out the "Put template:" label so that the output is valid XML, and displays properly in a browser rather than showing a syntax error
Aaron Marcuse-Kubitza
05:16 PM Revision 11395: planning/timeline/timeline.2013.xls: usability testing: added subtask to provide scientists with their requested data
Aaron Marcuse-Kubitza

10/20/2013

05:51 PM Revision 11394: planning/timeline/timeline.2013.xls: updated for progress
Aaron Marcuse-Kubitza
05:21 PM Revision 11393: bugfix: bin/import_all: need to publish datasources that won't be published by `make .../import`, so that the per-datasource import XPaths that refer to TNRS/geoscrub will link up with the TNRS/geoscrub source entry instead of creating a new entry without the metadata (because the entry with the metadata was named TNRS.new/geoscrub.new)
Aaron Marcuse-Kubitza
05:09 PM Revision 11392: schemas/vegbien.sql: datasource_publish(): use parameter names instead of $# because this is a PL/pgSQL function
Aaron Marcuse-Kubitza
05:07 PM Revision 11391: bugfix: schemas/vegbien.sql: datasource_publish(): if the datasource to publish already has the published name, don't datasource_rm() it
Aaron Marcuse-Kubitza
04:55 PM Revision 11390: bin/import_all: removed no longer needed import of geoscrub data, because analytical_stem_view is now joined to the geoscrub_output table directly, instead of using the imported canon_place entries
Aaron Marcuse-Kubitza
04:52 PM Revision 11389: schemas/vegbien.sql: analytical_stem_view: join to the geoscrub_output table directly, instead of using the imported canon_place entries. this avoids the need to import geoscrub_output into VegBIEN (which is expected to take 2+ hours after the refresh), as well as the need to then refresh any datasources whose geoscrubbing input data has changed.
Aaron Marcuse-Kubitza
04:37 PM Revision 11388: inputs/.geoscrub/geoscrub_output/postprocess.sql: added nullable unique index on the inputs, for use by analytical_stem_view. note that it must be nullable in order to create a match when not all of the input fields are populated. this uses array[] to create a nullable index, which is much better than column-based import and VegBIEN's use of COALESCE() because the expression is the same for every type and no NULL sentinel value is needed.
Aaron Marcuse-Kubitza
03:50 PM Revision 11387: schemas/VegCore/VegCore.ERD.mwb: fixed lines
Aaron Marcuse-Kubitza
03:41 PM Revision 11386: schemas/VegCore/ERD/VegCore.ERD.mwb: regenerated exports and udpated image map
Aaron Marcuse-Kubitza
03:34 PM Revision 11385: schemas/VegCore/ERD/VegCore.ERD.mwb: person: allow to have multiple organizations
Aaron Marcuse-Kubitza
03:28 PM Revision 11384: schemas/VegCore/ERD/VegCore.ERD.mwb: split "2b. GNRS" label into two labels, one for each table GNRS is applied to
Aaron Marcuse-Kubitza
03:23 PM Revision 11383: schemas/VegCore/ERD/VegCore.ERD.mwb: georeferencing: merged into geoplace, since this is actually information attached to a specific plot, etc. relating to the coordinates used in its geoplace subclass
Aaron Marcuse-Kubitza
02:59 PM Revision 11382: schemas/VegCore/ERD/VegCore.ERD.mwb: geovalidatable_place: changed parent geoplace pointer to parent_boundary_WKT, since the immediate parent may not have an associated boundary to use for geovalidation (i.e. it may not be an official GADM geoplace), although ancestors further up likely will be
Aaron Marcuse-Kubitza
02:40 PM Revision 11381: schemas/VegCore/ERD/VegCore.ERD.mwb: place.name: made it required because it's needed for the unique constraint to be populated properly (including for subclasses such as geoplace, which need to generate this from the coordinates)
Aaron Marcuse-Kubitza
02:30 PM Revision 11380: schemas/VegCore/ERD/VegCore.ERD.mwb: place.rank: made it required, because every place should have some kind of rank indicating what type of place it is, including lower ranks (e.g. plot, individual)
Aaron Marcuse-Kubitza
02:24 PM Revision 11379: schemas/VegCore/ERD/VegCore.ERD.mwb: place: added unique constraint on parent, rank, name
Aaron Marcuse-Kubitza
02:24 PM Revision 11378: schemas/VegCore/ERD/VegCore.ERD.mwb: place.locality: moved to geopath, because this is actually a rank of place (i.e. below municipality) rather than a field that every place could have
Aaron Marcuse-Kubitza
02:07 PM Revision 11377: schemas/VegCore/ERD/VegCore.ERD.mwb: place.locality: moved to geopath, because this is actually a rank of place (i.e. below municipality) rather than a field that every place could have
Aaron Marcuse-Kubitza
01:46 PM Revision 11376: schemas/VegCore/ERD/VegCore.ERD.mwb: geoplace.official_name: renamed to name to merge with inherited field from place. documented that for geoplaces, this is the official, scrubbed name.
Aaron Marcuse-Kubitza
12:48 PM Revision 11375: inputs/.geoscrub/geoscrub_output/postprocess.sql: added geovalid derived column, for use by analytical_stem_view
Aaron Marcuse-Kubitza

10/19/2013

06:56 PM Revision 11374: bin/with_all: $all: renamed to $hidden_srcs for clarity, since this now just adds the hidden (.*) datasources, rather than always using all datasources
Aaron Marcuse-Kubitza
06:50 PM Revision 11373: bugfix: bin/with_all: in $all mode, just prepend the .* datasources to the user-selected (or default) @inputs, so that using $all to add these datasources doesn't inadvertently cause the action to be performed for *all* datasources
Aaron Marcuse-Kubitza
04:33 PM Revision 11372: web/links/index.htm: updated to Firefox bookmarks. PostgreSQL: ALTER TABLE: added documentation about disabling of foreign key triggers, which is only possible by the superuser. note that marking a foreign key constraint as NOT VALID does *not* disable the trigger, so NOT VALID cannot be used for this purpose. this would be used to add fkeys from core VegBIEN tables to validation results tables such as the geoscrubbing results, without needing to import the validation results directly into core VegBIEN (which is time-consuming and currently must be done *before* input data is loaded, requiring a datasource reload to add geoscrubbing results).
Aaron Marcuse-Kubitza
02:15 PM Revision 11371: bin/import_all: usage: documented that this can now be run with a custom datasources list (each of the form inputs/src/)
Aaron Marcuse-Kubitza
02:02 PM Revision 11370: bin/with_all: added support for providing a custom list of inputs to run the command on
Aaron Marcuse-Kubitza
01:29 PM Revision 11369: inputs/.geoscrub/geoscrub_output/postprocess.sql, run: updated runtimes
Aaron Marcuse-Kubitza
01:13 AM Revision 11368: inputs/.geoscrub/geoscrub_output/run: documented full load_data() runtime (9 min @starscream)
Aaron Marcuse-Kubitza
01:12 AM Revision 11367: inputs/.geoscrub/geoscrub_output/postprocess.sql: updated runtimes for refreshed data, which now has 4x as many rows (1,707,970->6,747,650)
Aaron Marcuse-Kubitza
12:54 AM Revision 11366: inputs/.geoscrub/geoscrub_output/: refreshed geoscrub data. removed +header.csv because the extract now contains the header in the first row of the file.
Aaron Marcuse-Kubitza
12:52 AM Revision 11365: bugfix: lib/sh/local.sh: psql(): $is_root: use `` around case statement instead of $(), because it contains an embedded unbalanced )
Aaron Marcuse-Kubitza
12:27 AM Revision 11364: bugfix: inputs/.geoscrub/geoscrub_output/geoscrub.csv.run: include only the columns that Jim provided in his extract (the geoscrub table contains additional internal columns that are not part of the geovalidation data for VegBIEN). documented runtime (30 s) and upload time (1.5 min).
Aaron Marcuse-Kubitza

10/18/2013

10:33 PM Revision 11363: inputs/.geoscrub/geoscrub_output/geoscrub.csv.run: removed no longer needed setting of $local_server, $local_user (and use of $local_pg_database instead of $database) because the use_local bug in local.sh has been fixed
Aaron Marcuse-Kubitza
10:32 PM Revision 11362: bugfix: lib/sh/local.sh: psql(): don't default the connection vars using use_local if running as the postgres user. in that case, connection must happen via a socket, with server="", and as the user running the command (postgres), with user="".
Aaron Marcuse-Kubitza
09:55 PM Revision 11361: bugfix: inputs/.geoscrub/geoscrub_output/geoscrub.csv.run: need to manually set local_server, local_user to "" so that they do not default to their bien-user values
Aaron Marcuse-Kubitza
09:54 PM Revision 11360: bugfix: lib/sh/db.sh: avoid outputting to /dev/fd/# when running as sudo on Linux, because this causes a "Permission denied" error (due to the /dev/fd/# file being owned by a different user). this is not a problem with normal redirects (>&#), because they do not use /dev/fd/# files which can have access permissions.
Aaron Marcuse-Kubitza
09:52 PM Revision 11359: bugfix: lib/runscripts/util.run: to_top_file(): need to pass "$@" to to_file
Aaron Marcuse-Kubitza
08:17 PM Revision 11358: lib/runscripts/util.run: to_top_file: added function for this (in addition to alias), so that this can be run from sudo in a wrap_fn command
Aaron Marcuse-Kubitza
07:50 PM Revision 11357: lib/sh/db.sh: pg_as_root(): run sudo with echo_run to help debug
Aaron Marcuse-Kubitza
06:29 PM Revision 11356: bugfix: lib/sh/db.sh: pg_cmd(): only set PG* connection/login env vars when the corresponding var is *non-empty*. there are some situations in which these must be unset (in order to use the default value), and other situations when the var must be set to something (i.e. "") to avoid it being defaulted to a value in local.sh > connection vars.
Aaron Marcuse-Kubitza
06:27 PM Revision 11355: backups/TNRS.backup.md5: updated
Aaron Marcuse-Kubitza
06:13 PM Revision 11354: bugfix: inputs/.geoscrub/geoscrub_output/geoscrub.csv.run: need to set $local_pg_database instead of $database because use_local (in psql()) does not currently avoid clobbering already-set versions of the applicable env vars
Aaron Marcuse-Kubitza
06:11 PM Revision 11353: bugfix: lib/sh/local.sh: pg_as_root(): need to use -E (preserve environment) option to sudo, so that $schema, $table get passed through
Aaron Marcuse-Kubitza
06:05 PM Revision 11352: bugfix: lib/sh/local.sh: psql(): only \set schema, table if $schema, $table are non-empty, because otherwise, you will get a "zero-length delimited identifier" error
Aaron Marcuse-Kubitza
05:30 PM Revision 11351: added inputs/.geoscrub/geoscrub_output/geoscrub.csv.run to export the geoscrub table (must be run on vegbiendev)
Aaron Marcuse-Kubitza
05:29 PM Revision 11350: lib/sh/local.sh: added require_remote()
Aaron Marcuse-Kubitza
05:29 PM Revision 11349: lib/sh/db.sh: added pg_as_root()
Aaron Marcuse-Kubitza
05:28 PM Revision 11348: lib/runscripts/util.run: added $wrap_fn to run any function via sudo, etc.
Aaron Marcuse-Kubitza
05:23 PM Revision 11347: Added instructions for dependencies in the README.
Paul Sarando
05:23 PM Revision 11346: Added indexes to speed up geonames-to-gadm.sql.
Without these indexes, these queries could take hours to complete.
With them, the times more closely matched the time...
Paul Sarando
05:23 PM Revision 11345: Fixed a couple of syntax errors in geovalidate.sh.
Fixed a sql syntax error and a bash syntax error in the next line. Paul Sarando
04:03 PM Revision 11344: planning/timeline/timeline.2013.xls: "geoscrubbing automated pipeline": scheduled for after Paul's current set of tasks on the geoscrubbing re-run is complete. i'm budgeting several weeks for this since my understanding is that Paul is doing this part-time.
Aaron Marcuse-Kubitza
03:57 PM Revision 11343: planning/timeline/timeline.2013.xls: moved "geoscrubbing automated pipeline" under "simplify import process for easier maintainability"
Aaron Marcuse-Kubitza
03:53 PM Revision 11342: planning/timeline/timeline.2013.xls: geoscrubbing re-run: added subtask to spot-check reloaded geoscrubbing data
Aaron Marcuse-Kubitza
03:52 PM Revision 11341: planning/timeline/timeline.2013.xls: geoscrubbing re-run: added separate subtask for "geoscrubbing data reload", since apparently it was not clear that of course the new data will need to be imported into VegBIEN before the results of the re-run are available. this is currently scheduled to happen in the next full-database import, which is the week of 10/28 in order to include further validations fixes.
Aaron Marcuse-Kubitza
02:06 PM Revision 11340: planning/timeline/timeline.2013.xls: CVS validation: use timespan dot ◦ for supertask
Aaron Marcuse-Kubitza
02:05 PM Revision 11339: planning/timeline/timeline.2013.xls: CVS validation: added subtasks that are similar to for FIA validation (create validation subset, create extract)
Aaron Marcuse-Kubitza
02:03 PM Revision 11338: planning/timeline/timeline.2013.xls: FIA validation: split apart into subtasks, including "decide which columns to validate", which has to happen ahead of time before the extract can be generated
Aaron Marcuse-Kubitza
01:15 PM Revision 11337: planning/timeline/timeline.2013.xls: fixed check marks for past (hidden) weeks, which had gotten duplicated when rows were copied together with their check marks
Aaron Marcuse-Kubitza
01:04 PM Revision 11336: planning/timeline/timeline.2013.xls: fixed line heights
Aaron Marcuse-Kubitza
12:57 PM Revision 11335: planning/timeline/timeline.2013.xls: fixed column width so the dates display properly in MS Excel
Aaron Marcuse-Kubitza
12:48 PM Revision 11334: planning/timeline/timeline.2013.xls: right-aligned legend so it isn't too close to the "During week of:" label
Aaron Marcuse-Kubitza
12:46 PM Revision 11333: planning/timeline/timeline.2013.xls: added legend:
• task
◦ timespan
✓ task progress
☑ timespan progress
Aaron Marcuse-Kubitza
10:25 AM Revision 11332: planning/timeline/timeline.2013.xls: attribution/conditions of use: made it a subtask of "add missing columns" because this is related to data needed for published analyses. added dots because this is an ongoing task, that depends on data providers getting their use conditions to us.
Aaron Marcuse-Kubitza
10:08 AM Revision 11331: planning/timeline/timeline.2013.xls: reload core & analytical database: moved next reload ahead to last week of October so that we can include the updated geovalidation data for the 10/31 deadline. added additional reload so that they are spaced <= 1 month apart.
Aaron Marcuse-Kubitza
09:54 AM Revision 11330: planning/timeline/timeline.2013.xls: receive feedback from documentation tester: added an extra week to receive additional feedback from them in response to documentation fixes made
Aaron Marcuse-Kubitza
09:51 AM Revision 11329: planning/timeline/timeline.2013.xls: attribution/conditions of use: made this a top-level task instead of a subtask of "data provider metadata", to avoid including lower-priority tasks (i.e. in the later column) in the same section as higher-priority tasks
Aaron Marcuse-Kubitza
09:47 AM Revision 11328: planning/timeline/timeline.2013.xls: datasource validations: regrouped by subtask instead of by datasource, so that the high-priority subtasks get done for all datasources before moving on to lower-priority subtasks for any datasources
Aaron Marcuse-Kubitza
08:54 AM Revision 11327: planning/timeline/timeline.2013.xls: reduced width of Milestone column to make room to fit an additional week on the printed page
Aaron Marcuse-Kubitza
08:42 AM Revision 11326: planning/timeline/timeline.2013.xls: attribution/conditions of use: removed "(Brad/Brian/Bob/etc.)" because these are from everyone who provided or obtained data, not just Brad/Brian/Bob
Aaron Marcuse-Kubitza
08:40 AM Revision 11325: planning/timeline/timeline.2013.xls: rescheduled tasks to accommodate the separate non-critical feature requests subtasks
Aaron Marcuse-Kubitza
08:37 AM Revision 11324: planning/timeline/timeline.2013.xls: datasource validations: split "fix feature requests" into separate "fix critical feature requests" and "fix non-critical feature requests" tasks. rescheduled non-critical feature requests until after the other validation tasks have been completed.
Aaron Marcuse-Kubitza
08:11 AM Revision 11323: planning/timeline/timeline.2013.xls: add globally-unique occurrenceID: moved up to next week because we would like to be able to get this done for the 10/31 deadline
Aaron Marcuse-Kubitza
07:52 AM Revision 11322: planning/timeline/timeline.2013.xls: updated for progress
Aaron Marcuse-Kubitza
07:48 AM Revision 11321: planning/timeline/timeline.2013.xls: moved "data provider metadata" before "datasource validations (spot-checking)" because conditions of use are necessary for scientists who want to publish papers based on the data (which is a key use case)
Aaron Marcuse-Kubitza
07:43 AM Revision 11320: planning/timeline/timeline.2013.xls: moved "usability testing" before "datasource validations (spot-checking)" because this is most important towards reaching our goal of a useful information resource
Aaron Marcuse-Kubitza
07:41 AM Revision 11319: planning/timeline/timeline.2013.xls: moved "geoscrubbing re-run", "add globally-unique occurrenceID" back under "usability testing" > "add missing columns" because these are in fact part of the usability testing
Aaron Marcuse-Kubitza
01:26 AM Revision 11318: planning/timeline/timeline.2013.xls: updated for progress
Aaron Marcuse-Kubitza

10/17/2013

11:59 PM Revision 11317: planning/timeline/timeline.2013.xls: updated for progress
Aaron Marcuse-Kubitza
10:32 PM Revision 11316: planning/timeline/timeline.2013.xls: "flatten the datasources to a common schema": moved to later column because the complex tasks "switching to new-style import" and "create interactive scripts for each import step" are also scheduled then. (it's unlikely we would have much time over winter break anyway, considering that there is ~1 week's worth of holidays then.)
Aaron Marcuse-Kubitza
10:24 PM Revision 11315: planning/timeline/timeline.2013.xls: scheduled "simplify import process for easier maintainability"
Aaron Marcuse-Kubitza
10:20 PM Revision 11314: planning/timeline/timeline.2013.xls: tasks performed by someone else (geoscrubbing re-run): changed solid check marks ✓ to open check marks ☑ to match the solid • vs. open ◦ dot convention
Aaron Marcuse-Kubitza
10:17 PM Revision 11313: planning/timeline/timeline.2013.xls: documentation testing: added supertask dots. removed later dots for scheduled tasks.
Aaron Marcuse-Kubitza
10:15 PM Revision 11312: planning/timeline/timeline.2013.xls: scheduled "documentation testing"
Aaron Marcuse-Kubitza
10:02 PM Revision 11311: planning/timeline/timeline.2013.xls: scheduled "simplify process of mapping/adding a new datasource"
Aaron Marcuse-Kubitza
10:01 PM Revision 11310: planning/timeline/timeline.2013.xls: "add globally-unique occurrenceID": moved it up to the first week when we're no longer fixing existing issues in datasources, since this has similar priority to adding missing columns discovered during usability testing (which is scheduled as an ongoing task)
Aaron Marcuse-Kubitza
09:54 PM Revision 11309: planning/timeline/timeline.2013.xls: usability testing: did task breakdown (find scientists who want to use BIEN3 data, etc.) and scheduled subtasks
Aaron Marcuse-Kubitza
09:34 PM Revision 11308: planning/timeline/timeline.2013.xls: moved "add missing columns" to its own supertask. used outline check mark ☑ (analogous to open circle ◦) to mark supertasks as completed which were split up into subtasks.
Aaron Marcuse-Kubitza
09:29 PM Revision 11307: planning/timeline/timeline.2013.xls: later column: removed dots from scheduled items
Aaron Marcuse-Kubitza
09:06 PM Revision 11306: planning/timeline/timeline.2013.xls: moved "switching to new-style import"-related steps (other than for CVS) to separate "simplify import process for easier maintainability" supertask, since this is not part of the "simplify process of mapping/adding a new datasource" task
Aaron Marcuse-Kubitza
08:39 PM Revision 11305: planning/timeline/timeline.2013.xls: add any missing columns: added and scheduled step to add globally-unique occurrenceID
Aaron Marcuse-Kubitza
08:34 PM Revision 11304: planning/timeline/timeline.2013.xls: geoscrubbing re-run: added dots ◦ for this for the time when it can be worked on asynchronously by Paul Sarando
Aaron Marcuse-Kubitza
08:30 PM Revision 11303: planning/timeline/timeline.2013.xls: data provider metadata: added dots ◦ for the portion of "attribution and conditions of use" that can be worked on asynchronously by Brad/Brian/Bob
Aaron Marcuse-Kubitza
08:24 PM Revision 11302: planning/timeline/timeline.2013.xls: scheduled "aggregated validations" during the last 2 weeks of "datasource validations (spot-checking)", because these weeks are only spent fixing issues uncovered in the remaining datasources, so there may be extra time then
Aaron Marcuse-Kubitza
07:20 PM Revision 11301: planning/timeline/timeline.2013.xls: scheduled other tasks after "datasource validations (spot-checking)" is complete
Aaron Marcuse-Kubitza
07:14 PM Revision 11300: planning/timeline/timeline.2013.xls: datasource validations (spot-checking): each datasource's validation supertask: added open circles ◦ spanning the length of the subtasks
Aaron Marcuse-Kubitza
07:12 PM Revision 11299: planning/timeline/timeline.2013.xls: use an open circle ◦ instead of a bullet • for supertasks that have been fully split into subtasks (not just itemizing a few subtasks), so that these don't count towards the bullets (estimated workload) in each week
Aaron Marcuse-Kubitza
07:09 PM Revision 11298: planning/timeline/timeline.2013.xls: use an open circle ◦ instead of a bullet • for tasks that are performed by someone other than me, so that these don't count towards the bullets (estimated workload) in each week
Aaron Marcuse-Kubitza
07:04 PM Revision 11297: planning/timeline/timeline.2013.xls: datasource validations (spot-checking): split each datasource into subtasks and scheduled them
Aaron Marcuse-Kubitza
06:00 PM Revision 11296: planning/timeline/timeline.2013.xls: moved "move denormalized validations to stage II", "move stage III validations to stage II" outside of "switching to new-style import" because the "switching to new-style import" step refers just to the per-datasource switching steps, not to the additional refactorings that would be needed to avoid dependency on the complex XPath mappings (mappings/VegCore-VegBIEN.csv)
Aaron Marcuse-Kubitza
05:57 PM Revision 11295: planning/timeline/timeline.2013.xls: datasource validations (spot-checking): added subtasks for each of the remaining datasources (wiki.vegpath.org/2013-10-17_conference_call#validation-order)
Aaron Marcuse-Kubitza
05:43 PM Revision 11294: planning/timeline/timeline.2013.xls: moved non-validation-related tasks after the 10/31 deadline so that these are not taking time away from the validation
Aaron Marcuse-Kubitza
05:40 PM Revision 11293: planning/timeline/timeline.2013.xls: moved "flatten the datasources to a common schema" under "simplify process of mapping/adding a new datasource" because this is also needed separately for datasources where the left-joining is not part of the validation
Aaron Marcuse-Kubitza
05:30 PM Revision 11292: planning/timeline/timeline.2013.xls: extended "revisions to VegBIEN schema" to length of "datasource validations (spot-checking)" because schema changes are expected as we add missing fields
Aaron Marcuse-Kubitza
05:23 PM Revision 11291: planning/timeline/timeline.2013.xls: crossed out and hid completed tasks ("find out amount remaining in BIEN3 budget")
Aaron Marcuse-Kubitza
05:17 PM Revision 11290: planning/timeline/timeline.2013.xls: datasource validations (spot-checking): extended through the end of November because data providers' fixes on the remaining 10 datasources (wiki.vegpath.org/2013-10-17_conference_call#validation-order) are likely to add significantly to the issues and feature requests associated with these datasources (e.g. the 2nd-round VegBank validation added 4 issues and 5 feature requests). there is also expected to be wait time while data providers are responding (most likely in multiple rounds of feedback).
Aaron Marcuse-Kubitza
04:59 PM Revision 11289: planning/timeline/timeline.2013.xls: data provider metadata: removed "iPlant can do" because this actually requires Brad/Brian/Bob/other data providers to provide this info. however, this info may be findable on the web for some datasources.
Aaron Marcuse-Kubitza
04:53 PM Revision 11288: planning/timeline/timeline.2013.xls: moved "data provider metadata" right after "datasource validations" because this is part of the completed database itself rather than the tools to maintain it
Aaron Marcuse-Kubitza
04:47 PM Revision 11287: planning/timeline/timeline.2013.xls: split "revisions to schema" into "revisions to VegBIEN schema" (part of datasource validations) and "revisions to normalized VegCore" (part of documentation)
Aaron Marcuse-Kubitza
04:44 PM Revision 11286: bin/import_all: use just import_scrub, not reimport_scrub, because import_scrub now automatically publishes the datasource's import (i.e. removes the temp suffix)
Aaron Marcuse-Kubitza
04:43 PM Revision 11285: bugfix: inputs/input.Makefile: import: remove the temp suffix once the import is done, so that the full database import doesn't keep the suffix attached to the datasources that import_all didn't import with reimport. removed unused import_publish target (instead use import_temp to invoke just the import without the temp suffix removal).
Aaron Marcuse-Kubitza
04:27 PM Revision 11284: planning/timeline/timeline.2013.xls: moved part of "switching to new-style import" under "datasource validations (spot-checking)" because this is necessary to validate CVS
Aaron Marcuse-Kubitza
04:24 PM Revision 11283: planning/timeline/timeline.2013.xls: moved "simplify process of mapping/adding a new datasource" and "documentation testing" after "usability testing" because these tasks were there to make it possible for people other than me to reload/add to the database, which we have now decided is a lower priority than creating the validated database itself
Aaron Marcuse-Kubitza
04:14 PM Revision 11282: planning/timeline/timeline.2013.xls: added weeks through the end of the year (12/31)
Aaron Marcuse-Kubitza
02:00 PM Revision 11281: schemas/VegBIEN/attribution/BIEN 3 data use and attribution.docx: changed dataset definition to the definition in normalized VegCore ("a collection of records from the same place, with the same attribution requirements"), following discussion with Ramona
Aaron Marcuse-Kubitza
01:13 PM Revision 11280: planning/timeline/timeline.2013.xls: updated for progress
Aaron Marcuse-Kubitza
12:34 PM Revision 11279: schemas/VegBIEN/attribution/BIEN 3 data use and attribution.docx: updated to Ramona's commented version
Aaron Marcuse-Kubitza
12:19 AM Revision 11278: inputs/CVS/plot_/map.csv: realLatitude, realLongitude: remapped to UNUSED because these columns are actually empty
Aaron Marcuse-Kubitza

10/13/2013

07:32 PM Revision 11277: inputs/CVS/taxonObservation_/map.csv: collector_ID: remapped it to UNUSED and removed the join to party via it, like in VegBank
Aaron Marcuse-Kubitza
07:03 PM Revision 11276: inputs/CVS/: deleted stemLocation_, because the CVS stemLocation table is empty (unlike VegBank)
Aaron Marcuse-Kubitza
05:26 PM Revision 11275: inputs/CVS/import_order.txt: added plantConcept_/ so it would get automapped after switching to new-style import
Aaron Marcuse-Kubitza
04:24 PM Revision 11274: inputs/CVS/taxonObservation_/map.csv: denorm_{tri,quad}*: mapped to infraspecificRank*, infraspecificEpithet*
Aaron Marcuse-Kubitza
04:02 PM Revision 11273: inputs/CVS/taxonObservation_/map.csv: infraspecific ranks: remapped to EQUIV#to:species (which is the speciesBinomial), because these actually contain the full taxonomic name at that rank, like VegBank
Aaron Marcuse-Kubitza
03:50 PM Revision 11272: inputs/CVS/taxonObservation_/map.csv: genus: documented that unlike VegBank, does *not* include genus author
Aaron Marcuse-Kubitza
03:47 PM Revision 11271: inputs/CVS/taxonObservation_/map.csv: denorm_* terms _alt-ed with normalized terms: use DUPLICATE#of instead where possible. documented where and why _alt was necessary (this applies to a few rows for division, genus).
Aaron Marcuse-Kubitza
03:42 PM Revision 11270: bugfix: inputs/CVS/taxonObservation_/map.csv: species: remapped to speciesBinomial, not specificEpithet (like for VegBank). however, note that denorm_species *is* in fact the epithet, unlike VegBank.
Aaron Marcuse-Kubitza
03:19 PM Revision 11269: fix: inputs/CVS/taxonObservation_/postprocess.sql: removed {} around denorm_genus to match the normalized genus
Aaron Marcuse-Kubitza
02:28 PM Revision 11268: inputs/CVS/taxonObservation_/map.csv: removed unnecessary _alts for terms that don't have a duplicate denorm_* or hierarchical field
Aaron Marcuse-Kubitza
02:06 PM Revision 11267: fix: inputs/CVS/taxonObservation_/postprocess.sql: fix 1 row that has denorm_kingdom != Kingdom (i.e. both NOT NULL but not the same)
Aaron Marcuse-Kubitza
12:46 PM Revision 11266: bugfix: lib/common.Makefile: $(subMake): don't enclose the target in "" because sometimes the target is empty (i.e. `all`), and nothing should be passed to the sub-make
Aaron Marcuse-Kubitza
12:10 AM Revision 11265: inputs/VegBank/plot_/create.sql: documented runtime (5 min)
Aaron Marcuse-Kubitza
12:10 AM Revision 11264: bugfix: inputs/CVS/plot_/create.sql: like for VegBank, need to compare place.*PLOT_ID*, not PLOTPLACE_ID, with plot.PLOT_ID
Aaron Marcuse-Kubitza
12:02 AM Revision 11263: /README.TXT: Single datasource import: added pointer to instructions to remake the analytical DB (also required after single datasource import)
Aaron Marcuse-Kubitza

10/12/2013

04:59 PM Revision 11262: planning/timeline/timeline.2013.xls: updated for progress
Aaron Marcuse-Kubitza
04:20 PM Revision 11261: inputs/VegBank/verify/input_cols.txt, inputs/VegBank/+taxon_observation.**.sample/create.sql: updated to match taxon_observation.** columns
Aaron Marcuse-Kubitza
04:16 PM Revision 11260: inputs/VegBank/verify/input_cols.txt, inputs/VegBank/+taxon_observation.**.sample/create.sql: updated to match taxon_observation.** columns
Aaron Marcuse-Kubitza
03:49 PM Revision 11259: /README.TXT: Maintenance: to synchronize vegbiendev, jupiter, and your local machine: run all sync_uploads on the svn working copy using --size-only, because the mtimes are based on when the files were last updated by svn and are not meaningful
Aaron Marcuse-Kubitza
03:46 PM Revision 11258: /README.TXT: Full database import: On local machine: do steps under Maintenance > "to synchronize vegbiendev, jupiter, and your local machine": removed no longer accurate indicator that these steps are above Full database import, since Full database import is now at the beginning of the file
Aaron Marcuse-Kubitza
03:05 PM Revision 11257: inputs/VegBank/verify/input_cols.txt, inputs/VegBank/+taxon_observation.**.sample/create.sql: updated to match taxon_observation.** columns
Aaron Marcuse-Kubitza
03:03 PM Revision 11256: bugfix: inputs/VegBank/+taxon_observation.**.sample/: renamed to ^taxon_observation.**.sample because a leading + has a special meaning to bash (it indicates a shell option, and you will get an error "invalid option name"), as well as to make (it indicates that a recipe command invokes make recursively)
Aaron Marcuse-Kubitza
02:14 PM Revision 11255: bugfix: inputs/VegBank/taxon_observation.**/header.csv: updated for observation_/map.csv bugfix, which added new hasobservationsynonym field. this fixes a strange test bug caused by the taxon_observation.**/map.csv column list being mismatched/misaligned with what was in the underlying tables. (column mismatches will often cause unexplainable errors in unrelated sections of code the same way that buffer overflows do in C++.)
Aaron Marcuse-Kubitza
02:01 PM Revision 11254: bugfix: inputs/VegBank/taxon_observation.**.sample/: renamed to +taxon_observation.**.sample so that the *-expansion of taxon_observation.** doesn't add taxon_observation.**.sample (which causes it to attempt to install taxon_observation.**.sample before taxon_observation.** is installed)
Aaron Marcuse-Kubitza
12:48 PM Revision 11253: bugfix: *Makefile: recursive invocation of $(MAKE): enclose targets in "" in case they contain *
Aaron Marcuse-Kubitza
12:12 PM Revision 11252: bugfix: lib/runscripts/table.run: load_data(): pass $is_view through to `make reinstall` so that DROP VIEW will be used instead of DROP TABLE where applicable
Aaron Marcuse-Kubitza
12:11 PM Revision 11251: bugfix: inputs/input.Makefile: %/uninstall: allow user to set is_view=1 flag to use DROP VIEW instead of DROP TABLE
Aaron Marcuse-Kubitza
12:08 PM Revision 11250: lib/sh/util.sh: added instructions for making an export only visible locally
Aaron Marcuse-Kubitza

10/10/2013

06:50 PM Revision 11249: bugfix: inputs/VegBank/observation_/header.csv, map.csv: updated for refresh, which inserts hasobservationsynonym at the end of the observation table
Aaron Marcuse-Kubitza
05:46 PM Revision 11248: inputs/VegBank/taxon_observation.**.sample/create.sql: reordered columns in the same order as analytical_plot, for easier validation
Aaron Marcuse-Kubitza
05:46 PM Revision 11247: bugfix: lib/runscripts/table.run: load_data(): in remaking mode, need to remake header.csv in case the columns have changed
Aaron Marcuse-Kubitza
03:07 PM Revision 11246: web/links/index.htm: updated to Firefox bookmarks. updated favicons.
Aaron Marcuse-Kubitza
02:57 PM Revision 11245: web/links/index.htm: updated to Firefox bookmarks. sudo: added instructions to turn off incorrect password e-mails.
Aaron Marcuse-Kubitza
02:40 PM Revision 11244: inputs/VegBank/taxon_observation.**.sample/create.sql: include only the subset of columns that is imported to VegBIEN
Aaron Marcuse-Kubitza
02:32 PM Revision 11243: inputs/VegBank/taxon_observation.**.sample/test.xml.ref: updated inserted row count (which was most likely generated before the output column names had been set to the input column names)
Aaron Marcuse-Kubitza
01:55 PM Revision 11242: added inputs/VegBank/verify/input_cols.include.txt, with runscript to generate it
Aaron Marcuse-Kubitza
01:26 PM Revision 11241: inputs/VegBank/verify/input_cols.unmapped.txt*: renamed to input_cols.exclude.txt* because this now includes mapped columns as well
Aaron Marcuse-Kubitza
01:18 PM Revision 11240: inputs/VegBank/verify/input_cols.unmapped.txt*: renamed to input_cols.exclude.txt* because this now includes mapped columns as well
Aaron Marcuse-Kubitza
01:11 PM Revision 11239: inputs/VegBank/verify/input_cols.unmapped.txt.run: remove unmapped join columns, since these *would* be included in the extract
Aaron Marcuse-Kubitza
01:09 PM Revision 11238: inputs/VegBank/verify/input_cols.unmapped.txt.run: take input directly from input_cols.txt to avoid needing to first copy and paste it into input_cols.unmapped.txt
Aaron Marcuse-Kubitza
01:03 PM Revision 11237: inputs/VegBank/verify/input_cols.unmapped.txt.run: added back deliberately excluded columns (DUPLICATE#of:..., etc.) so that the # of rows in the file can be subtracted from the total # of columns to get the # of input columns that would be included in the extract
Aaron Marcuse-Kubitza
12:43 PM Revision 11236: bugfix: inputs/input.Makefile: %/VegBIEN.csv: `ln -s` to create VegBIEN.csv: enclose the filenames in "" since they may contain * (e.g. taxon_observation.**)
Aaron Marcuse-Kubitza
12:23 PM Revision 11235: added inputs/VegBank/verify/input_cols.txt, input_cols.unmapped.txt (with runscript to filter input_cols.unmapped.txt)
Aaron Marcuse-Kubitza
11:53 AM Revision 11234: schemas/VegBIEN/attribution/BIEN 3 data use and attribution.docx: made Ramona's corrections with track changes turned on. note that you have to use MS Word for this, not LibreOffice, because LibreOffice can't save the table of contents properly in .docx or .doc format (although it can save it in .odt format).
Aaron Marcuse-Kubitza
08:18 AM Revision 11233: inputs/VegBank/stratum/postprocess.sql: added pkey
Aaron Marcuse-Kubitza
08:05 AM Revision 11232: inputs/VegBank/taxonobservation_/postprocess.sql: added __parent index on locationID to facilitate the LEFT JOINs used to create the validation input
Aaron Marcuse-Kubitza
07:54 AM Revision 11231: inputs/VegBank/observation_/postprocess.sql: added __parent index on locationID to facilitate the LEFT JOINs used to create the validation input
Aaron Marcuse-Kubitza
07:45 AM Revision 11230: inputs/VegBank/import_order.txt: added taxon_observation.**.sample so it will automatically be kept up to date
Aaron Marcuse-Kubitza
07:32 AM Revision 11229: inputs/VegBank/taxon_observation.**.sample/create.sql: set runtime (1 s)
Aaron Marcuse-Kubitza
07:30 AM Revision 11228: inputs/VegBank/: added taxon_observation.**.sample subset of plots to use in the validation. this avoids the need to import all of VegBank just to validate a few of the plots.
Aaron Marcuse-Kubitza

10/09/2013

10:12 PM Revision 11227: bin/map: usage: documented that verbosity > 3 in commit mode turns on debug_temp mode, which creates real tables instead of temp tables
Aaron Marcuse-Kubitza
09:41 PM Revision 11226: schemas/VegCore/ERD/VegCore.ERD.mwb: georeferencing: documented that this stores location determinations (= VegBIEN.vegpath.org?public.locationplace)
Aaron Marcuse-Kubitza
06:31 PM Revision 11225: inputs/VegBank/taxon_observation.**/: updated for data refresh
Aaron Marcuse-Kubitza
06:25 PM Revision 11224: inputs/VegBank/plantconcept_/: mapped columns, since this is now included in import_order.txt and therefore gets processed by the column-renaming runscripts. note that this means that in taxonobservation_/map.csv, the plantconcept_ input column names need to be changed to what they are mapped to.
Aaron Marcuse-Kubitza
06:16 PM Revision 11223: inputs/VegBank/taxonobservation_/create.sql: updated runtime (20 s)
Aaron Marcuse-Kubitza
05:54 PM Revision 11222: schemas/VegCore/ERD/VegCore.ERD.mwb: regenerated exports and udpated image map
Aaron Marcuse-Kubitza
05:51 PM Revision 11221: fix: schemas/VegCore/ERD/VegCore.ERD.mwb: project: allow it to *have* a dataset rather than *being* one, because projects do not always have separate attribution requirements (e.g. in Madidi they don't). this also fixes a bug where projects whose datasets had a parent dataset (e.g. VegBank) would need a dummy event for that parent dataset so that the fkeys on both dataset.parent->dataset *and* event.parent->event would be satisfied. added project__unique constraint on name within dataset.
Aaron Marcuse-Kubitza
05:38 PM Revision 11220: schemas/VegCore/ERD/VegCore.ERD.mwb: taxa_sampling_event.parent (inherited): require this to be a taxa_sampling_event. documented that this is the taxa_sampling_event for the parent stratum.
Aaron Marcuse-Kubitza
05:15 PM Revision 11219: schemas/VegCore/ERD/VegCore.ERD.mwb: fixed lines
Aaron Marcuse-Kubitza
05:03 PM Revision 11218: schemas/VegCore/ERD/VegCore.ERD.mwb: fixed lines
Aaron Marcuse-Kubitza
05:00 PM Revision 11217: schemas/VegCore/ERD/VegCore.ERD.mwb: regenerated exports and udpated image map
Aaron Marcuse-Kubitza
04:55 PM Revision 11216: schemas/VegCore/ERD/VegCore.ERD.mwb: place_visit.parent (inherited): added fkey constraint that this must be a place_visit. documented that this is the concurrent observation of the parent place.
Aaron Marcuse-Kubitza
04:47 PM Revision 11215: schemas/VegCore/ERD/VegCore.ERD.mwb: project: inherit from dataset, since projects generally confer attribution and conditions of use
Aaron Marcuse-Kubitza
04:26 PM Revision 11214: fix: schemas/VegCore/ERD/VegCore.ERD.mwb: place_visit: fixed synonyms list to "VegX.vegpath.org?plotObservation, VegBank.vegpath.org?observation (which was confusingly named) + some of VegBank.vegpath.org?plot (which is actually 1:1 with observation)"
Aaron Marcuse-Kubitza
04:22 PM Revision 11213: schemas/VegCore/ERD/VegCore.ERD.mwb: place_observation: renamed to place_visit for clarity, since other tables that point to this use it as a general container, not just a link to the observations about the place
Aaron Marcuse-Kubitza
04:16 PM Revision 11212: schemas/VegCore/ERD/VegCore.ERD.mwb: merged stratum_event into taxa_sampling_event since they serve the same purpose (separating taxa samplings within the same place_observation by subsetting methodology). merged layer into stratum since strata as we think of them are horizontal, although there are other definitions of stratum (e.g. size class) which are more similar to our method table.
Aaron Marcuse-Kubitza
03:51 PM Revision 11211: schemas/VegCore/ERD/VegCore.ERD.mwb: event subclasses: subject: added applicable fkey constraints to the inherited field. renamed existing table-specific subject fields to subject to merge with the inherited field.
Aaron Marcuse-Kubitza
03:38 PM Revision 11210: schemas/VegCore/ERD/VegCore.ERD.mwb: event: added unique constraints for every way that the scoping fields can meaningfully be combined. these correspond to the various unique constraints offered by VegBIEN.locationevent. (the event table's constraints are always tricky because there are many ways to define an event. it is important to include all of them so that there is almost always a way to generate a natural key, without resorting to an artificial key from traceable.id_by_source.)
Aaron Marcuse-Kubitza
03:28 PM Revision 11209: schemas/VegCore/ERD/VegCore.ERD.mwb: event.place: renamed to subject and allow this to be any traceable, not just a place. this allows taxon_observations to be scoped within their taxon_occurrences just as subplot events are scoped within parent plot events.
Aaron Marcuse-Kubitza
02:46 PM Revision 11208: schemas/VegCore/ERD/VegCore.ERD.mwb: traceable.authors: documented that this makes traceable mutually recursive with party_list
Aaron Marcuse-Kubitza
02:45 PM Revision 11207: schemas/VegCore/ERD/VegCore.ERD.mwb: traceable.authors: documented that this makes traceable mutually recursive with party_list
Aaron Marcuse-Kubitza
02:42 PM Revision 11206: schemas/VegCore/ERD/VegCore.ERD.mwb: fixed inheritance connectors
Aaron Marcuse-Kubitza
02:38 PM Revision 11205: schemas/VegCore/ERD/VegCore.ERD.mwb: regenerated exports and udpated image map
Aaron Marcuse-Kubitza
02:35 PM Revision 11204: schemas/VegCore/ERD/VegCore.ERD.mwb: record.authors: moved to traceable because actually, different sections of a denormalized source record may have separate authors (e.g. participants for taxa_sampling_event, identifiers for taxon_determination)
Aaron Marcuse-Kubitza
02:24 PM Revision 11203: schemas/VegCore/ERD/VegCore.ERD.mwb: record.dataset: split this into the scoping_dataset (which scopes the id_within_dataset) and the attribution_dataset (which provides attribution information)
Aaron Marcuse-Kubitza
01:55 PM Revision 11202: schemas/VegCore/ERD/VegCore.ERD.mwb: regenerated exports and udpated image map
Aaron Marcuse-Kubitza
01:45 PM Revision 11201: schemas/VegCore/ERD/VegCore.ERD.mwb: record.authors: changed this to refer *only* to original data creators such as collectors and identifiers. this way, there is no need to generate a combined party_list of owners+authors for every record, because the record-specific authors list can just point to the same party_list as the collectors or event participants. this is also closer to Brad's meaning, which is specifically "a person or institution responsible for *creating* the dataset" [+emph].
Aaron Marcuse-Kubitza
01:31 PM Revision 11200: schemas/VegCore/ERD/VegCore.ERD.mwb: party_list: added array field, which stores an array combining all the list entries. if specified, a trigger will autogenerate party_list_entries from this field. if not specified, it will be autopopulated whenever a party_list_entry is added.
Aaron Marcuse-Kubitza
12:59 PM Revision 11199: schemas/VegCore/ERD/VegCore.ERD.mwb: added data_owner subclass of party_list_entry, which stores use_conditions for each data_owner of a dataset. dataset: added access_rights (public, private, etc.), use_conditions (citation requirements, etc.).
Aaron Marcuse-Kubitza
12:25 PM Revision 11198: schemas/VegCore/ERD/VegCore.ERD.mwb.run: record.authors: clarified that this should include *all* parties who helped create the data record, making it a *superset* of dataset.data_owners, who receive attribution. clarified that *record-specific* authors such as collectors and identifiers generally do not receive attribution, although *dataset-level* authors do (this field includes both).
Aaron Marcuse-Kubitza
12:07 PM Revision 11197: schemas/VegCore/ERD/VegCore.ERD.mwb.run: record.id_within_dataset: removed instruction to also put this in source.name, because source.name is now optional, and this would not make sense because source does not have a parent pointer
Aaron Marcuse-Kubitza
12:03 PM Revision 11196: schemas/VegCore/ERD/VegCore.ERD.mwb.run: documented that schema COMMENTs cannot contain ' (used in contractions such as "don't"), because MySQL Workbench cannot escape these properly
Aaron Marcuse-Kubitza
12:02 PM Revision 11195: schemas/VegCore/ERD/VegCore.ERD.mwb: record: added authors, which stores the people who helped create the data record. this is Brad's data author term (Brad.vegpath.org?attribution.data_author), but actually belongs to the individual data *record* rather than to an entire data*set* because records may have different authors (collectors, identifiers, etc.). note that schema COMMENTs apparently cannot contain ' (used in contractions such as "don't"), because MySQL Workbench cannot escape these properly.
Aaron Marcuse-Kubitza
11:38 AM Revision 11194: schemas/VegCore/ERD/VegCore.ERD.mwb: dataset.data_owners: documented that this includes all aggregators through which the data has passed, so that this is equivalent to Brad's data indexers (Brad.vegpath.org?attribution.data_indexer)
Aaron Marcuse-Kubitza
11:31 AM Revision 11193: fix: schemas/VegCore/ERD/VegCore.ERD.mwb: dataset.parent: fixed example to indicate that the GBIF.ARIZ dataset is contained within a specific GBIF *dataset* (the parent), but was obtained from the GBIF *organization* (the dataset_source). ARIZ is then the first_publisher, not the dataset_source.
Aaron Marcuse-Kubitza
11:22 AM Revision 11192: schemas/VegCore/ERD/VegCore.ERD.mwb: dataset: renamed data_source to dataset_source for clarity
Aaron Marcuse-Kubitza
11:14 AM Revision 11191: schemas/VegCore/ERD/VegCore.ERD.mwb: dataset.contacts: documented that this is the contact person(s) to ask about attribution questions. there should always be at least one contact person. if unknown, the person loading the data should set this to themselves.
Aaron Marcuse-Kubitza
11:12 AM Revision 11190: added schemas/VegCore/Brad_Boyle/BIEN 3 data use and attribution.docx symlink to schemas/VegBIEN/attribution/BIEN 3 data use and attribution.docx, so that all of Brad's attribution-related terms are in one place
Aaron Marcuse-Kubitza
11:07 AM Revision 11189: schemas/VegCore/ERD/VegCore.ERD.mwb: dataset.contacts: make this required, so that anyone adding data to a VegCore DB always provides at least one contact person, possibly themselves, to ask about attribution questions (as suggested by Brad in BIEN 3 data use and attribution.docx)
Aaron Marcuse-Kubitza
11:00 AM Revision 11188: schemas/VegCore/ERD/VegCore.ERD.mwb: dataset.first_publisher: allow this to be any kind of source, rather than just a party (i.e. an organization). this is possible now that party inherits from source.
Aaron Marcuse-Kubitza
10:52 AM Revision 11187: schemas/VegCore/ERD/VegCore.ERD.mwb: dataset: linked to Brad's synonyms for dataset_source (Brad.vegpath.org?attribution.proximate_data_provider), first_publisher (Brad.vegpath.org?attribution.data_publisher, Brad.vegpath.org?data_provenance.primarydataprovider)
Aaron Marcuse-Kubitza
10:43 AM Revision 11186: bugfix: web/people/Brad_Boyle/.htaccess: only forward requests for a term defined by Brad to the attribution/ subdir, so that Brad.vegpath.org shows a directory listing instead of forwarding right to the attribution doc
Aaron Marcuse-Kubitza
10:33 AM Revision 11185: bugfix: web/people/Brad_Boyle/.htaccess: set %{ENV:innermost_dir} to Brad_Boyle
Aaron Marcuse-Kubitza
10:28 AM Revision 11184: added web/people/Brad_Boyle/data_provenance/ (BIEN database entities related to data provenance and ownership.docx)
Aaron Marcuse-Kubitza
09:56 AM Revision 11183: bugfix: web/people/Brad_Boyle/attribution/.htaccess: \-escape the %s so that they aren't treated as backreferences. set the [redirect] flag so that the downloaded file has the correct filename (BIEN 3 data use and attribution.docx instead of attribution).
Aaron Marcuse-Kubitza
09:49 AM Revision 11182: bugfix: web/people/Brad_Boyle/attribution/.htaccess: use %20 instead of + because the Apache server apparently doesn't translate + to space correctly
Aaron Marcuse-Kubitza
09:47 AM Revision 11181: bugfix: web/people/Brad_Boyle/attribution/.htaccess: link to the VegBIEN subdir in /exchange_schemas instead of the main /VegBIEN because there are actually two VegBIEN subdirs, and the main one instead points to /datasources/VegBIEN
Aaron Marcuse-Kubitza
09:45 AM Revision 11180: added web/people/Brad_Boyle/ with attribution/ subdir for BIEN 3 data use and attribution.docx
Aaron Marcuse-Kubitza
09:15 AM Revision 11179: schemas/VegCore/ERD/VegCore.ERD.mwb: dataset: renamed data_source to dataset_source for clarity
Aaron Marcuse-Kubitza
08:54 AM Revision 11178: inputs/VegBank/plantconcept_/create.sql: documented runtime (21 min)
Aaron Marcuse-Kubitza
08:28 AM Revision 11177: bugfix: inputs/VegBank/plantconcept_/: added new-style import files
Aaron Marcuse-Kubitza
08:27 AM Revision 11176: bugfix: inputs/VegBank/import_order.txt: added plantconcept_, because new-style import needs it to be explicitly listed in import_order.txt in order to run it
Aaron Marcuse-Kubitza
08:24 AM Revision 11175: inputs/VegBank/run: refresh(): added usage
Aaron Marcuse-Kubitza
08:22 AM Revision 11174: inputs/VegBank/run: refresh(): documented that this should be run on *vegbiendev*
Aaron Marcuse-Kubitza
07:58 AM Revision 11173: inputs/VegBank/_archive/2012-8-30/: svn:ignore the data exports
Aaron Marcuse-Kubitza
07:57 AM Revision 11172: inputs/VegBank/run: added refresh() target
Aaron Marcuse-Kubitza
07:54 AM Revision 11171: inputs/VegBank/: refreshed VegBank so that all of Mike Lee's sample plots would be included in the input data. (VegBank was last refreshed from the live DB on 2012-8-30.) split vegbank.sql into vegbank.schema.sql and vegbank.data.sql so that the schema can be examined and imported separately, like for MySQL datasources. inputs/VegBank/vegbank.~.clean_up.sql: commented out setting comminterpretation.commname to NOT NULL, because after the refresh it is now NULL in 10 rows, where commconcept_id is also NULL.
Aaron Marcuse-Kubitza

10/08/2013

05:31 PM Revision 11170: lib/runscripts/datasrc_dir.run: import(): added remake (rm=1) mode that reinstalls the datasource before continuing with the subdirs' import actions
Aaron Marcuse-Kubitza
05:29 PM Revision 11169: lib/runscripts/in_datasrc_dir.run: added datasrc_make(), which runs make in the datasrc dir
Aaron Marcuse-Kubitza
12:36 PM Revision 11168: inputs/.geoscrub/_src/README.TXT: added e-mail from John Donoghue with general description of the BIEN2 geovalidation workflow
Aaron Marcuse-Kubitza

10/03/2013

04:04 PM Revision 11167: bugfix: schemas/vegbien.sql: plot.**: removed ORDER BY because a SELECT on the view with its own ORDER BY will actually sort the rows *twice* instead of optimizing away plot.**'s default ORDER BY
Aaron Marcuse-Kubitza
03:44 PM Revision 11166: schemas/vegbien.sql: added plot.** subset view of analytical_stem_view, used for VegBank validation
Aaron Marcuse-Kubitza
01:41 PM Revision 11165: planning/timeline/timeline.2013.xls: moved reload core & analytical database out a week because it shouldn't be done until after CVS validation is complete
Aaron Marcuse-Kubitza
12:25 PM Revision 11164: inputs/.geoscrub/_src/README.TXT: added link to geovalidation description in wiki
Aaron Marcuse-Kubitza
11:03 AM Revision 11163: added schemas/VegBIEN/attribution/BIEN 3 data use and attribution.docx from Brad
Aaron Marcuse-Kubitza
10:03 AM Revision 11162: planning/timeline/timeline.2013.xls: updated to reflect that the next week should be spent doing data validations, as decided in the conference call
Aaron Marcuse-Kubitza

10/02/2013

05:59 AM Revision 11161: bugfix: schemas/vegbien.sql: locationevent__communities(): need to use commdescription instead of commname as the communityName because that's where communityName has been mapped to in mappings/VegCore-VegBIEN.csv. (commname instead stores the communityID.)
Aaron Marcuse-Kubitza
05:56 AM Revision 11160: inputs/VegBank/: added observation__community/
Aaron Marcuse-Kubitza
05:49 AM Revision 11159: inputs/VegBank/vegbank.~.clean_up.sql: commclass.commcode,commname: rename to prevent collisions
Aaron Marcuse-Kubitza
04:26 AM Revision 11158: inputs/VegBank/vegbank.~.clean_up.sql: indicate required column comminterpretation.commname
Aaron Marcuse-Kubitza
04:20 AM Revision 11157: inputs/VegBank/vegbank.~.clean_up.sql: commconcept.commname: rename to prevent collision with commname.commname
Aaron Marcuse-Kubitza
04:15 AM Revision 11156: planning/timeline/: svn:ignore: added .~* (lock files for .xls)
Aaron Marcuse-Kubitza
03:58 AM Revision 11155: schemas/vegbien.sql: analytical_stem_view: added instructions for what to do after updating this
Aaron Marcuse-Kubitza
03:55 AM Revision 11154: schemas/vegbien.sql: analytical_stem_view, analytical_plot: added communities using new locationevent__communities__array()
Aaron Marcuse-Kubitza
03:52 AM Revision 11153: schemas/vegbien.sql: added locationevent__communities*()
Aaron Marcuse-Kubitza
03:10 AM Revision 11152: bugfix: schemas/vegbien.sql: locationevent: locationevent__pull_forward_from_parent() trigger: only pull forward the location_id when there is a stratum_id, so that empty locationevents that should be auto-forwarded are not unintentionally made valid by populating the location_id field (which causes a check constraint to become satisfied, and prevents auto-forwarding from working). this fixes a bug in some datasources' automated tests, such as `make inputs/SALVIAS/plotObservations/test`.
Aaron Marcuse-Kubitza
02:45 AM Revision 11151: lib/sql_io.py: put_table(): default param: documented that this will be used for *all* missing rows, regardless of which error caused them not to be inserted. this means that auto-forwarding (wiki.vegpath.org/Auto-forwarding) can be used with *any* type of constraint violation, not just NOT NULL constraints (which it is typically used with).
Aaron Marcuse-Kubitza

10/01/2013

10:57 PM Revision 11150: /Makefile: mk_db: config/users.sql: unset ON_ERROR_STOP so user-exists errors don't prevent further user-adds
Aaron Marcuse-Kubitza

09/30/2013

07:47 AM Revision 11149: planning/timeline/timeline.2013.xls: updated for progress
Aaron Marcuse-Kubitza
07:38 AM Revision 11148: bugfix: schemas/vegbien.sql: analytical_stem_view: plot__collectors: to include plot__collectors for nested stratum events, need to calculate the locationevent__contributors__array() for the locationevent and parent_event *separately*, and then COALESCE those, instead of just running locationevent__contributors__array() once on the COALESCE of the two IDs
Aaron Marcuse-Kubitza
07:34 AM Revision 11147: bugfix: schemas/vegbien.sql: locationevent: added pull-forward trigger to populate location_id from the parent event. a pull-forward trigger *must* be used for this (not a COALESCE() in analytical_stem_view) because the locationevent cannot even be found by analytical_stem_view if it does not have an associated location.
Aaron Marcuse-Kubitza
06:37 AM Revision 11146: schemas/VegCore/ERD/VegCore.ERD.mwb: source: added required url field, which points to the source data and uniquely identifies the source. note that this differs from the id, which is a *self*-refential pkey (as described at wiki.vegpath.org/Web_interface#Clickable-fkeys).
Aaron Marcuse-Kubitza
05:47 AM Revision 11145: schemas/VegCore/ERD/VegCore.ERD.mwb: fixed lines
Aaron Marcuse-Kubitza
05:38 AM Revision 11144: schemas/VegCore/ERD/VegCore.ERD.mwb: regenerated exports and udpated image map
Aaron Marcuse-Kubitza
05:32 AM Revision 11143: schemas/VegCore/ERD/VegCore.ERD.mwb: source: removed canon, because this would create too much complexity as a result of having multiple copies of the same source with different names. instead, source names will be scrubbed *upon insert* so that the row always contains the most accurate data, and there is only one copy of each source. this scrubbing may use the assistance of a thesaurus table w/ standard abbreviations, etc.
Aaron Marcuse-Kubitza
05:07 AM Revision 11142: schemas/VegCore/ERD/VegCore.ERD.mwb: plot.name: merged with inherited name field
Aaron Marcuse-Kubitza
04:43 AM Revision 11141: schemas/VegCore/ERD/VegCore.ERD.mwb: regenerated exports and udpated image map
Aaron Marcuse-Kubitza
04:40 AM Revision 11140: schemas/VegCore/ERD/VegCore.ERD.mwb: source: added canon field. when there are multiple names for a source, this points to the canonically-named source. by default, points recursively to itself.
Aaron Marcuse-Kubitza
04:04 AM Revision 11139: schemas/VegCore/ERD/VegCore.ERD.mwb: taxon_observation.taxon_occurrence: documented that when not provided (and not fillable from another field), a taxon_occurrence will be created with the same id as the taxon_observation. this operation should eventually be perfomed by a trigger.
Aaron Marcuse-Kubitza
03:58 AM Revision 11138: schemas/VegCore/ERD/VegCore.ERD.mwb: geovalidatable_place.parent_geoplace: renamed to parent so it will merge with the inherited parent field. this is the parent geoplace.
Aaron Marcuse-Kubitza
03:27 AM Revision 11137: schemas/VegCore/ERD/VegCore.ERD.mwb: regenerated exports and udpated image map
Aaron Marcuse-Kubitza
03:20 AM Revision 11136: schemas/VegCore/ERD/VegCore.ERD.mwb: fixed lines
Aaron Marcuse-Kubitza
03:18 AM Revision 11135: schemas/VegCore/ERD/VegCore.ERD.mwb: individual_observation.specimenholder_institutions: documented that instead, the taxon_occurrence for the individual_observation should have an autogenerated current_observation that merges together all the individual_observations for these institutions' specimens. in this case, each institution's specimen will still have its *own* individual_observation containing what *it* believes to be the specimenholder institutions.
Aaron Marcuse-Kubitza
03:03 AM Revision 11134: schemas/VegCore/ERD/VegCore.ERD.mwb: individual_observation.specimenholder_institutions: documented that when provided, the source (i.e. traceable.source) should be set to the value of this field, because these institutions together provided the data for this entity
Aaron Marcuse-Kubitza
02:59 AM Revision 11133: schemas/VegCore/ERD/VegCore.ERD.mwb: party_list: inherit from source so that a party_list can be used as the source for a row (e.g. a list of institutions that together provided the data for an individual_observation that has specimens at multiple institutions)
Aaron Marcuse-Kubitza
02:55 AM Revision 11132: schemas/VegCore/ERD/VegCore.ERD.mwb: reobservable_presence: merged into taxon_determination, so that voucher is directly available to taxon_determination to autopopulate fields when a redetermination is made on the voucher
Aaron Marcuse-Kubitza
02:37 AM Revision 11131: schemas/VegCore/ERD/VegCore.ERD.mwb: fixed lines
Aaron Marcuse-Kubitza
02:36 AM Revision 11130: schemas/VegCore/ERD/VegCore.ERD.mwb: aggregate_observation.sampling_event: merged with inherited sampling_event field
Aaron Marcuse-Kubitza
02:33 AM Revision 11129: schemas/VegCore/ERD/VegCore.ERD.mwb: taxon_observation.parent: require that this is an fkey to another taxon_observation, since this stores the *taxon_observation* of the same taxon_occurrence in the parent sampling_event
Aaron Marcuse-Kubitza
02:22 AM Revision 11128: schemas/VegCore/ERD/VegCore.ERD.mwb: subplot.parent_plot: renamed to parent so it will merge with the inherited parent field. this is the parent plot.
Aaron Marcuse-Kubitza
02:16 AM Revision 11127: schemas/VegCore/ERD/VegCore.ERD.mwb: individual_count.taxon_presence: renamed to parent so it will merge with the inherited parent field. this is the taxon_presence for the taxon as a whole (not subdivided by size_class, etc.).
Aaron Marcuse-Kubitza
02:13 AM Revision 11126: schemas/VegCore/ERD/VegCore.ERD.mwb: taxa_sampling_event.method: renamed to subsetting_method for clarity
Aaron Marcuse-Kubitza
02:06 AM Revision 11125: schemas/VegCore/ERD/VegCore.ERD.mwb: stratum_event.parent_event: renamed to parent so it will merge with the inherited parent field
Aaron Marcuse-Kubitza
01:58 AM Revision 11124: schemas/VegCore/ERD/VegCore.ERD.mwb: traceable: added alternate pkey id_by_source, which allows referencing a traceable by its source information even when the subclass chose a different value for the pkey
Aaron Marcuse-Kubitza
01:49 AM Revision 11123: schemas/VegCore/ERD/VegCore.ERD.mwb: taxon_observation.observation_in_parent_place: merged with inherited parent field, documenting that for this table, this contains the observation of the same taxon_occurrence in the parent sampling_event
Aaron Marcuse-Kubitza
01:33 AM Revision 11122: schemas/VegCore/ERD/VegCore.ERD.mwb: fixed lines
Aaron Marcuse-Kubitza
01:24 AM Revision 11121: schemas/VegCore/ERD/VegCore.ERD.mwb: individual_observation.place_observed_at: merged with inherited place field, documenting that for this table, this contains the place the individual was observed at
Aaron Marcuse-Kubitza
01:16 AM Revision 11120: schemas/VegCore/ERD/VegCore.ERD.mwb: event: reordered date_range, participants, place in the order of how likely they are to be provided. (place was considered more optional than the other two because a project does not usually have a place.)
Aaron Marcuse-Kubitza
01:09 AM Revision 11119: schemas/VegCore/ERD/VegCore.ERD.mwb: taxon_observation.sampling_event: made it required so that there is always a sampling_event to scope the primary_collector/collector_number when provided. (for specimens, this will have the same event information (date_range, participants, place) as the taxon_observation.)
Aaron Marcuse-Kubitza
12:35 AM Revision 11118: schemas/VegCore/ERD/VegCore.ERD.mwb: taxon_observation: added unique constraint on sampling_event, primary_collector, collector_number. reordered these columns in constraint order (path order).
Aaron Marcuse-Kubitza

09/29/2013

11:35 PM Revision 11117: schemas/VegCore/ERD/VegCore.ERD.mwb: taxon_observation: added primary_collector, to scope the collector_number. collectors: renamed to all_collectors for clarity, now that there is a primary_collector field.
Aaron Marcuse-Kubitza
11:14 PM Revision 11116: schemas/VegCore/ERD/VegCore.ERD.mwb: specimen.current_collection, owner_collection: moved to specimen_observation because these may differ each time the specimen is reobserved
Aaron Marcuse-Kubitza
11:08 PM Revision 11115: schemas/VegCore/ERD/VegCore.ERD.mwb: specimen.code_in_individual: renamed to id_within_individual to match the id_within_* naming convention using elsewhere
Aaron Marcuse-Kubitza
10:58 PM Revision 11114: schemas/VegCore/ERD/VegCore.ERD.mwb: individual_observation: removed code because this duplicates taxon_observation.collector_number and record.id_within_dataset
Aaron Marcuse-Kubitza
10:53 PM Revision 11113: schemas/VegCore/ERD/VegCore.ERD.mwb: specimen.specimenholder_institutions: moved to individual_observation because the list of institutions which store a specimen from the plant actually relates to the plant, not a particular specimen (this list should be the same for all specimens taken from the plant)
Aaron Marcuse-Kubitza
10:34 PM Revision 11112: schemas/VegCore/ERD/VegCore.ERD.mwb: specimen: removed collection_event because this is the same as individual_observation
Aaron Marcuse-Kubitza
10:11 PM Revision 11111: schemas/VegCore/ERD/VegCore.ERD.mwb: specimen: removed separate fkey to individual because this is linked to from individual_observation
Aaron Marcuse-Kubitza
09:55 PM Revision 11110: schemas/VegCore/ERD/VegCore.ERD.mwb: place_observation: link to project. taxa_sampling_event: removed link to project since this now happens via place_observation.
Aaron Marcuse-Kubitza
09:35 PM Revision 11109: schemas/VegCore/ERD/VegCore.ERD.mwb: taxa_sampling_event: merged required within_place into place_observation, which already contains a required fkey to place
Aaron Marcuse-Kubitza
09:29 PM Revision 11108: schemas/VegCore/ERD/VegCore.ERD.mwb: taxa_sampling_event: added place_observation fkey for the concurrent place_observation, which contains all the info about the place (communities, soil data, etc.). this allows a taxon_observation to be traced to its associated location measurements.
Aaron Marcuse-Kubitza
08:58 PM Revision 11107: bugfix: mappings/VegCore-VegBIEN.csv: nest all taxonoccurrences inside a stratum event, so that the parent locationevent is *always* fully populated before child locationevents point to it. (previously, a stub parent event was created when the child event was imported first, which blocked the fully-populated parent event from being inserted later on.) this uses auto-folding (for VegBank/CVS) and auto-forwarding (for other datasources) to prune empty stratum events for taxonoccurrences that don't have strata. (see wiki.vegpath.org/Auto-folding, wiki.vegpath.org/Auto-forwarding for more info about these normalization techniques.) note that the inserted row counts stay *exactly the same* for all datasources except VegBank (which was being fixed), indicating that this signficant change to the mappings did *not* change the semantics of the import of taxonoccurrences.
Aaron Marcuse-Kubitza
08:37 PM Revision 11106: inputs/VegBank/observationcontributor_/test.xml.ref: updated inserted row count
Aaron Marcuse-Kubitza

09/28/2013

10:40 PM Revision 11105: bugfix: mappings/VegCore-VegBIEN.csv: stratum's locationevent: link this to the parent locationevent, so that the parent locationevent's information (such as locationeventcontributors) is accessible to the stratum's locationevent
Aaron Marcuse-Kubitza
09:08 PM Revision 11104: bugfix: inputs/VegBank/taxon_observation.**/postprocess.sql: inlined _join() so that taxon_observation.** wouldn't get cascadingly deleted whenever the util schema (where this normally resides) gets reinstalled
Aaron Marcuse-Kubitza
08:23 PM Revision 11103: bugfix: schemas/vegbien.sql: locationeventcontributor: added missing unique constraint (locationeventcontributor_unique)
Aaron Marcuse-Kubitza
08:21 PM Revision 11102: bugfix: schemas/vegbien.sql: locationeventcontributor.role: made it optional because some locationeventcontributors may not have an associated role
Aaron Marcuse-Kubitza
06:45 PM Revision 11101: schemas/vegbien.sql: analytical_stem_view, analytical_plot: added plot__collectors, which uses locationevent__contributors__array()
Aaron Marcuse-Kubitza
06:42 PM Revision 11100: schemas/vegbien.sql: locationevent__contributors(): split into set-returning and array-returning functions. sort results by fullname so there is a predictable ordering (VegBank does not include a sort_order in the observationContributor table).
Aaron Marcuse-Kubitza
05:56 PM Revision 11099: schemas/vegbien.sql: added locationevent__contributors() aggregating function for use in analytical_stem_view
Aaron Marcuse-Kubitza
06:53 AM Revision 11098: added inputs/VegBank/observationcontributor_/
Aaron Marcuse-Kubitza
06:51 AM Revision 11097: mappings/VegCore-VegBIEN.csv: mapped event__participant
Aaron Marcuse-Kubitza
05:51 AM Revision 11096: mappings/VegCore.htm: regenerated from wiki. added EventParticipant table containing event__participant.
Aaron Marcuse-Kubitza
05:04 AM Revision 11095: bugfix: inputs/VegBank/: taxonOccurrenceID: include the aggregateOrganismObservationID in this so that there is one taxonoccurrence for *each* stratum's taxonImportance. this allows the different strata to have separate taxonoccurrences that are associated with the stratum-specific locationevents, rather than all being lumped into one taxonoccurrence, with only one locationevent.
Aaron Marcuse-Kubitza
12:06 AM Revision 11094: schemas/VegCore/ERD/VegCore.ERD.mwb: taxon_observation.observation_in_parent_place: renamed to observation_in_parent_event because the taxon_observation might be connected to a stratum_event which is nested within another sampling_event before it's connected to any sampling_event for a parent plot. this is the case for VegBank data with associated strata.
Aaron Marcuse-Kubitza

09/27/2013

11:14 PM Revision 11093: bugfix: schemas/vegbien.sql: locationevent: locationevent_unique_within_creator unique index on sourceaccessioncode: need to add stratum_id, because the addition of stratum_id to *this* table instead of a separate stratum_event table (which would have required more schema changes) means we need to use the awkward construction of appending stratum_id to *every* unique constraint, including ones that ordinarily would just pass through the datasource's pkey
Aaron Marcuse-Kubitza
06:31 PM Revision 11092: web/links/index.htm: updated to Firefox bookmarks. added Java client- and server-side links. web: split into client- and server-side folders.
Aaron Marcuse-Kubitza
06:29 PM Revision 11091: planning/timeline/timeline.2013.xls: updated for progress
Aaron Marcuse-Kubitza
03:43 PM Revision 11090: /README.TXT: Datasource setup: added link to Example steps for a datasource (wiki.vegpath.org/Import_process_for_Madidi)
Aaron Marcuse-Kubitza
03:23 PM Revision 11089: /README.TXT: Full database import: To remake analytical DB: added runtime (13 h)
Aaron Marcuse-Kubitza

09/26/2013

10:15 AM Revision 11088: bugfix: schemas/postgresql.Mac.conf: allow external client logins like for vegbiendev, by listening on all addresses rather than just localhost
Aaron Marcuse-Kubitza

09/24/2013

05:53 PM Revision 11087: planning/timeline/timeline.2013.xls: updated for progress
Aaron Marcuse-Kubitza
04:40 PM Revision 11086: schemas/vegbien.ERD.mwb: fixed lines
Aaron Marcuse-Kubitza
02:54 PM Revision 11085: schemas/vegbien.sql: analytical_plot: added stratum__name
Aaron Marcuse-Kubitza
02:48 PM Revision 11084: schemas/vegbien.ERD.mwb: updated to PostgreSQL schema
Aaron Marcuse-Kubitza
02:47 PM Revision 11083: schemas/vegbien.sql: analytical_stem_view: added stratum table (stratum__name)
Aaron Marcuse-Kubitza
02:14 PM Revision 11082: mappings/VegCore-VegBIEN.csv: mapped stratum__name
Aaron Marcuse-Kubitza
02:09 PM Revision 11081: schemas/vegbien.sql: added source_id to allow different datasources to have their own strata
Aaron Marcuse-Kubitza
02:03 PM Revision 11080: schemas/vegbien.ERD.mwb: updated to PostgreSQL schema
Aaron Marcuse-Kubitza
01:51 PM Revision 11079: schemas/vegbien.sql: stratum: added unique constraint on source_id/stratumname, and require stratumname, so that strata are identified by name within their datasource
Aaron Marcuse-Kubitza
01:45 PM Revision 11078: schemas/vegbien.sql: added source_id to allow different datasources to have their own strata
Aaron Marcuse-Kubitza
01:22 PM Revision 11077: schemas/vegbien.ERD.mwb: regenerated exports
Aaron Marcuse-Kubitza
01:18 PM Revision 11076: schemas/vegbien.ERD.mwb: updated to PostgreSQL schema
Aaron Marcuse-Kubitza
01:15 PM Revision 11075: schemas/vegbien.sql: locationevent: added stratum_id to unique constraints, so that there can be multiple locationevents for the same underlying event but with different strata
Aaron Marcuse-Kubitza
01:01 PM Revision 11074: schemas/vegbien.sql: link locationevent to stratum instead of the other way around (similar to how it's represented in normalized VegCore as stratum_event->stratum, rather than stratum->taxa_sampling_event). this works around column-based import's lack of support for linking two tables together via two separate routes (in this case locationevent->taxonoccurrence->stratum and locationevent->stratum).
Aaron Marcuse-Kubitza
12:54 PM Revision 11073: web/links/index.htm: updated to Firefox bookmarks. added PostgreSQL links.
Aaron Marcuse-Kubitza
12:09 PM Revision 11072: schemas/vegbien.sql: stratum: added stratumname
Aaron Marcuse-Kubitza
 

Also available in: Atom