*{.sh,run}: runscript targets: use begin_target instead of echo_func so the target name is properly echoed. note that this requires using with_rm so that $rm is properly progagated to applicable invoked targets. (previously, $rm was progagated to all invoked targets. note that with_rm only works inside a runscript target that starts with begin_target.)
fix: schemas/VegCore/mk_derived: added `EOF` at end to avoid (benign) "here-document delimited by end-of-file" warnings on Linux
*{.sh,run}: use new begin_target instead of `echo_func; set_make_vars`
added schemas/VegCore/Brad_Boyle/bien3_data_provenance_use_cases.docx* from e-mail from Brad
schemas/VegCore/vegbien_identifier_examples.xlsx: changed to symlink to Brad_Boyle/vegbien_identifier_examples.xlsx, to avoid duplicate copies
moved everything into /trunk/ to create the standard svn layout, for use with tools that require this (eg. git-svn). IMPORTANT: do NOT do an `svn up`. instead, re-use your working copy's existing files with `svn switch` (http://svnbook.red-bean.com/en/1.6/svn.ref.svn.c.switch.html).
schemas/VegCore/ERD/VegCore.ERD.mwb: traceable.id_by_source: support multiple ids_by_source per traceable, because the same entity may be present in multiple datasources (e.g. if one got data from the other), and we would like to remove that duplicate
schemas/VegCore/ERD/VegCore.ERD.mwb: connecting lines: inherits from traceable: added arrow to indicate what this label refers to
schemas/VegCore/ERD/VegCore.ERD.mwb: regenerated exports and udpated image map
schemas/VegCore/ERD/VegCore.ERD.mwb: HAS-A/IS-A box: renamed to "connecting lines" for clarity
schemas/VegCore/ERD/VegCore.ERD.mwb: relationships: HAS-A: added HAS-MANY going in the opposite direction, because every HAS-A has an opposite HAS-MANY
schemas/VegCore/ERD/VegCore.ERD.mwb: relationships: IS-A, HAS-A: added directional arrows
schemas/VegCore/ERD/VegCore.ERD.mwb: field order box: removed spacing between top of text box and bottom of outer box label
schemas/VegCore/ERD/VegCore.ERD.mwb: reordered columns according to the field order convention
schemas/VegCore/ERD/VegCore.ERD.mwb: added label documenting the field order convention:1) inherited2) required3) identifying4) foreign key5) extenders6) others
schemas/VegCore/VegCore.ERD.mwb: fixed lines
schemas/VegCore/ERD/VegCore.ERD.mwb: person: allow to have multiple organizations
schemas/VegCore/ERD/VegCore.ERD.mwb: split "2b. GNRS" label into two labels, one for each table GNRS is applied to
schemas/VegCore/ERD/VegCore.ERD.mwb: georeferencing: merged into geoplace, since this is actually information attached to a specific plot, etc. relating to the coordinates used in its geoplace subclass
schemas/VegCore/ERD/VegCore.ERD.mwb: geovalidatable_place: changed parent geoplace pointer to parent_boundary_WKT, since the immediate parent may not have an associated boundary to use for geovalidation (i.e. it may not be an official GADM geoplace), although ancestors further up likely will be
schemas/VegCore/ERD/VegCore.ERD.mwb: place.name: made it required because it's needed for the unique constraint to be populated properly (including for subclasses such as geoplace, which need to generate this from the coordinates)
schemas/VegCore/ERD/VegCore.ERD.mwb: place.rank: made it required, because every place should have some kind of rank indicating what type of place it is, including lower ranks (e.g. plot, individual)
schemas/VegCore/ERD/VegCore.ERD.mwb: place: added unique constraint on parent, rank, name
schemas/VegCore/ERD/VegCore.ERD.mwb: place.locality: moved to geopath, because this is actually a rank of place (i.e. below municipality) rather than a field that every place could have
schemas/VegCore/ERD/VegCore.ERD.mwb: geoplace.official_name: renamed to name to merge with inherited field from place. documented that for geoplaces, this is the official, scrubbed name.
schemas/VegCore/ERD/VegCore.ERD.mwb: georeferencing: documented that this stores location determinations (= VegBIEN.vegpath.org?public.locationplace)
fix: schemas/VegCore/ERD/VegCore.ERD.mwb: project: allow it to have a dataset rather than being one, because projects do not always have separate attribution requirements (e.g. in Madidi they don't). this also fixes a bug where projects whose datasets had a parent dataset (e.g. VegBank) would need a dummy event for that parent dataset so that the fkeys on both dataset.parent->dataset and event.parent->event would be satisfied. added project__unique constraint on name within dataset.
schemas/VegCore/ERD/VegCore.ERD.mwb: taxa_sampling_event.parent (inherited): require this to be a taxa_sampling_event. documented that this is the taxa_sampling_event for the parent stratum.
schemas/VegCore/ERD/VegCore.ERD.mwb: fixed lines
schemas/VegCore/ERD/VegCore.ERD.mwb: place_visit.parent (inherited): added fkey constraint that this must be a place_visit. documented that this is the concurrent observation of the parent place.
schemas/VegCore/ERD/VegCore.ERD.mwb: project: inherit from dataset, since projects generally confer attribution and conditions of use
fix: schemas/VegCore/ERD/VegCore.ERD.mwb: place_visit: fixed synonyms list to "VegX.vegpath.org?plotObservation, VegBank.vegpath.org?observation (which was confusingly named) + some of VegBank.vegpath.org?plot (which is actually 1:1 with observation)"
schemas/VegCore/ERD/VegCore.ERD.mwb: place_observation: renamed to place_visit for clarity, since other tables that point to this use it as a general container, not just a link to the observations about the place
schemas/VegCore/ERD/VegCore.ERD.mwb: merged stratum_event into taxa_sampling_event since they serve the same purpose (separating taxa samplings within the same place_observation by subsetting methodology). merged layer into stratum since strata as we think of them are horizontal, although there are other definitions of stratum (e.g. size class) which are more similar to our method table.
schemas/VegCore/ERD/VegCore.ERD.mwb: event subclasses: subject: added applicable fkey constraints to the inherited field. renamed existing table-specific subject fields to subject to merge with the inherited field.
schemas/VegCore/ERD/VegCore.ERD.mwb: event: added unique constraints for every way that the scoping fields can meaningfully be combined. these correspond to the various unique constraints offered by VegBIEN.locationevent. (the event table's constraints are always tricky because there are many ways to define an event. it is important to include all of them so that there is almost always a way to generate a natural key, without resorting to an artificial key from traceable.id_by_source.)
schemas/VegCore/ERD/VegCore.ERD.mwb: event.place: renamed to subject and allow this to be any traceable, not just a place. this allows taxon_observations to be scoped within their taxon_occurrences just as subplot events are scoped within parent plot events.
schemas/VegCore/ERD/VegCore.ERD.mwb: traceable.authors: documented that this makes traceable mutually recursive with party_list
schemas/VegCore/ERD/VegCore.ERD.mwb: fixed inheritance connectors
schemas/VegCore/ERD/VegCore.ERD.mwb: record.authors: moved to traceable because actually, different sections of a denormalized source record may have separate authors (e.g. participants for taxa_sampling_event, identifiers for taxon_determination)
schemas/VegCore/ERD/VegCore.ERD.mwb: record.dataset: split this into the scoping_dataset (which scopes the id_within_dataset) and the attribution_dataset (which provides attribution information)
schemas/VegCore/ERD/VegCore.ERD.mwb: record.authors: changed this to refer only to original data creators such as collectors and identifiers. this way, there is no need to generate a combined party_list of owners+authors for every record, because the record-specific authors list can just point to the same party_list as the collectors or event participants. this is also closer to Brad's meaning, which is specifically "a person or institution responsible for creating the dataset" [+emph].
schemas/VegCore/ERD/VegCore.ERD.mwb: party_list: added array field, which stores an array combining all the list entries. if specified, a trigger will autogenerate party_list_entries from this field. if not specified, it will be autopopulated whenever a party_list_entry is added.
schemas/VegCore/ERD/VegCore.ERD.mwb: added data_owner subclass of party_list_entry, which stores use_conditions for each data_owner of a dataset. dataset: added access_rights (public, private, etc.), use_conditions (citation requirements, etc.).
schemas/VegCore/ERD/VegCore.ERD.mwb.run: record.authors: clarified that this should include all parties who helped create the data record, making it a superset of dataset.data_owners, who receive attribution. clarified that record-specific authors such as collectors and identifiers generally do not receive attribution, although dataset-level authors do (this field includes both).
schemas/VegCore/ERD/VegCore.ERD.mwb.run: record.id_within_dataset: removed instruction to also put this in source.name, because source.name is now optional, and this would not make sense because source does not have a parent pointer
schemas/VegCore/ERD/VegCore.ERD.mwb.run: documented that schema COMMENTs cannot contain ' (used in contractions such as "don't"), because MySQL Workbench cannot escape these properly
schemas/VegCore/ERD/VegCore.ERD.mwb: record: added authors, which stores the people who helped create the data record. this is Brad's data author term (Brad.vegpath.org?attribution.data_author), but actually belongs to the individual data record rather than to an entire data*set* because records may have different authors (collectors, identifiers, etc.). note that schema COMMENTs apparently cannot contain ' (used in contractions such as "don't"), because MySQL Workbench cannot escape these properly.
schemas/VegCore/ERD/VegCore.ERD.mwb: dataset.data_owners: documented that this includes all aggregators through which the data has passed, so that this is equivalent to Brad's data indexers (Brad.vegpath.org?attribution.data_indexer)
fix: schemas/VegCore/ERD/VegCore.ERD.mwb: dataset.parent: fixed example to indicate that the GBIF.ARIZ dataset is contained within a specific GBIF dataset (the parent), but was obtained from the GBIF organization (the dataset_source). ARIZ is then the first_publisher, not the dataset_source.
schemas/VegCore/ERD/VegCore.ERD.mwb: dataset: renamed data_source to dataset_source for clarity
schemas/VegCore/ERD/VegCore.ERD.mwb: dataset.contacts: documented that this is the contact person(s) to ask about attribution questions. there should always be at least one contact person. if unknown, the person loading the data should set this to themselves.
added schemas/VegCore/Brad_Boyle/BIEN 3 data use and attribution.docx symlink to schemas/VegBIEN/attribution/BIEN 3 data use and attribution.docx, so that all of Brad's attribution-related terms are in one place
schemas/VegCore/ERD/VegCore.ERD.mwb: dataset.contacts: make this required, so that anyone adding data to a VegCore DB always provides at least one contact person, possibly themselves, to ask about attribution questions (as suggested by Brad in BIEN 3 data use and attribution.docx)
schemas/VegCore/ERD/VegCore.ERD.mwb: dataset.first_publisher: allow this to be any kind of source, rather than just a party (i.e. an organization). this is possible now that party inherits from source.
schemas/VegCore/ERD/VegCore.ERD.mwb: dataset: linked to Brad's synonyms for dataset_source (Brad.vegpath.org?attribution.proximate_data_provider), first_publisher (Brad.vegpath.org?attribution.data_publisher, Brad.vegpath.org?data_provenance.primarydataprovider)
schemas/VegCore/ERD/VegCore.ERD.mwb: source: added required url field, which points to the source data and uniquely identifies the source. note that this differs from the id, which is a self-refential pkey (as described at wiki.vegpath.org/Web_interface#Clickable-fkeys).
schemas/VegCore/ERD/VegCore.ERD.mwb: source: removed canon, because this would create too much complexity as a result of having multiple copies of the same source with different names. instead, source names will be scrubbed upon insert so that the row always contains the most accurate data, and there is only one copy of each source. this scrubbing may use the assistance of a thesaurus table w/ standard abbreviations, etc.
schemas/VegCore/ERD/VegCore.ERD.mwb: plot.name: merged with inherited name field
schemas/VegCore/ERD/VegCore.ERD.mwb: source: added canon field. when there are multiple names for a source, this points to the canonically-named source. by default, points recursively to itself.
schemas/VegCore/ERD/VegCore.ERD.mwb: taxon_observation.taxon_occurrence: documented that when not provided (and not fillable from another field), a taxon_occurrence will be created with the same id as the taxon_observation. this operation should eventually be perfomed by a trigger.
schemas/VegCore/ERD/VegCore.ERD.mwb: geovalidatable_place.parent_geoplace: renamed to parent so it will merge with the inherited parent field. this is the parent geoplace.
schemas/VegCore/ERD/VegCore.ERD.mwb: individual_observation.specimenholder_institutions: documented that instead, the taxon_occurrence for the individual_observation should have an autogenerated current_observation that merges together all the individual_observations for these institutions' specimens. in this case, each institution's specimen will still have its own individual_observation containing what it believes to be the specimenholder institutions.
schemas/VegCore/ERD/VegCore.ERD.mwb: individual_observation.specimenholder_institutions: documented that when provided, the source (i.e. traceable.source) should be set to the value of this field, because these institutions together provided the data for this entity
schemas/VegCore/ERD/VegCore.ERD.mwb: party_list: inherit from source so that a party_list can be used as the source for a row (e.g. a list of institutions that together provided the data for an individual_observation that has specimens at multiple institutions)
schemas/VegCore/ERD/VegCore.ERD.mwb: reobservable_presence: merged into taxon_determination, so that voucher is directly available to taxon_determination to autopopulate fields when a redetermination is made on the voucher
schemas/VegCore/ERD/VegCore.ERD.mwb: aggregate_observation.sampling_event: merged with inherited sampling_event field
schemas/VegCore/ERD/VegCore.ERD.mwb: taxon_observation.parent: require that this is an fkey to another taxon_observation, since this stores the taxon_observation of the same taxon_occurrence in the parent sampling_event
schemas/VegCore/ERD/VegCore.ERD.mwb: subplot.parent_plot: renamed to parent so it will merge with the inherited parent field. this is the parent plot.
schemas/VegCore/ERD/VegCore.ERD.mwb: individual_count.taxon_presence: renamed to parent so it will merge with the inherited parent field. this is the taxon_presence for the taxon as a whole (not subdivided by size_class, etc.).
schemas/VegCore/ERD/VegCore.ERD.mwb: taxa_sampling_event.method: renamed to subsetting_method for clarity
schemas/VegCore/ERD/VegCore.ERD.mwb: stratum_event.parent_event: renamed to parent so it will merge with the inherited parent field
schemas/VegCore/ERD/VegCore.ERD.mwb: traceable: added alternate pkey id_by_source, which allows referencing a traceable by its source information even when the subclass chose a different value for the pkey
schemas/VegCore/ERD/VegCore.ERD.mwb: taxon_observation.observation_in_parent_place: merged with inherited parent field, documenting that for this table, this contains the observation of the same taxon_occurrence in the parent sampling_event
schemas/VegCore/ERD/VegCore.ERD.mwb: individual_observation.place_observed_at: merged with inherited place field, documenting that for this table, this contains the place the individual was observed at
schemas/VegCore/ERD/VegCore.ERD.mwb: event: reordered date_range, participants, place in the order of how likely they are to be provided. (place was considered more optional than the other two because a project does not usually have a place.)
schemas/VegCore/ERD/VegCore.ERD.mwb: taxon_observation.sampling_event: made it required so that there is always a sampling_event to scope the primary_collector/collector_number when provided. (for specimens, this will have the same event information (date_range, participants, place) as the taxon_observation.)
schemas/VegCore/ERD/VegCore.ERD.mwb: taxon_observation: added unique constraint on sampling_event, primary_collector, collector_number. reordered these columns in constraint order (path order).
schemas/VegCore/ERD/VegCore.ERD.mwb: taxon_observation: added primary_collector, to scope the collector_number. collectors: renamed to all_collectors for clarity, now that there is a primary_collector field.
schemas/VegCore/ERD/VegCore.ERD.mwb: specimen.current_collection, owner_collection: moved to specimen_observation because these may differ each time the specimen is reobserved
schemas/VegCore/ERD/VegCore.ERD.mwb: specimen.code_in_individual: renamed to id_within_individual to match the id_within_* naming convention using elsewhere
schemas/VegCore/ERD/VegCore.ERD.mwb: individual_observation: removed code because this duplicates taxon_observation.collector_number and record.id_within_dataset
schemas/VegCore/ERD/VegCore.ERD.mwb: specimen.specimenholder_institutions: moved to individual_observation because the list of institutions which store a specimen from the plant actually relates to the plant, not a particular specimen (this list should be the same for all specimens taken from the plant)
