Project

General

Profile

Statistics
| Revision:
  • svn:ignore: *

# Date Author Comment
13357 04/30/2014 05:46 PM Aaron Marcuse-Kubitza

bugfix: **/postprocess.sql: don't use the public schema, because this creates an unsatisfied dependency while the database is being installed, and breaks `make install`

12757 03/18/2014 05:41 PM Aaron Marcuse-Kubitza

inputs/CVS/plot_/postprocess.sql: add unique constraint on locationName (analogous to the unique constraint in plot), for use by inputs/VegBank/plot/postprocess.sql in removing inter-datasource duplication

11970 01/20/2014 11:33 AM Aaron Marcuse-Kubitza

moved everything into /trunk/ to create the standard svn layout, for use with tools that require this (eg. git-svn). IMPORTANT: do NOT do an `svn up`. instead, re-use your working copy's existing files with `svn switch` (http://svnbook.red-bean.com/en/1.6/svn.ref.svn.c.switch.html).

11842 12/05/2013 12:27 PM Aaron Marcuse-Kubitza

inputs/CVS/cvs.~.clean_up.sql: remove plot.realLatitude/realLongitude, since this is private data that should not be publicly visible

11804 12/03/2013 08:00 AM Aaron Marcuse-Kubitza

fix: inputs/CVS/plot_/postprocess.sql: locality: include the site name (authorLocation), because this is part of the unique specification of the place that was sampled, and Bob wants this to be included in VegBIEN

11796 12/02/2013 02:46 PM Aaron Marcuse-Kubitza

fix: inputs/CVS/plot_/map.csv: PARENT_ID: remapped to UNUSED, to clarify that subplots are not implemented through this field

11760 11/26/2013 05:54 AM Aaron Marcuse-Kubitza

inputs/CVS/: don't import joined tables, because they are now imported in the taxon_observation.** left-join instead

11754 11/25/2013 09:43 PM Aaron Marcuse-Kubitza

inputs/CVS/: prepended the table name to each column name to prevent column collisions, using the steps at http://wiki.vegpath.org/Left-joining_a_datasource

11749 11/25/2013 07:52 PM Aaron Marcuse-Kubitza

inputs/CVS/: switched to new-style import, using the steps at http://wiki.vegpath.org/Adding_new-style_import_to_a_datasource

11745 11/25/2013 06:51 PM Aaron Marcuse-Kubitza

fix: inputs/CVS/plot_/map.csv: removed filter-less collisions. note that the name county_ is assigned in plot_/create.sql, not cvs.~.clean_up.sql as one might expect, because this is a generated column.

11744 11/25/2013 06:42 PM Aaron Marcuse-Kubitza

fix: inputs/CVS/plot_/map.csv: removed filter-less collisions

11743 11/25/2013 06:41 PM Aaron Marcuse-Kubitza

fix: inputs/CVS/plot_/map.csv: removed filter-less collisions

11729 11/21/2013 05:20 PM Aaron Marcuse-Kubitza

inputs/CVS/plot_/: translated column filters to postprocessing derived columns, using the steps at http://wiki.vegpath.org/Adding_new-style_import_to_a_datasource#1-Translate-filters-to-postprocessing-derived-columns

11727 11/21/2013 04:27 PM Aaron Marcuse-Kubitza

inputs/CVS/plot_/postprocess.sql: added pkey from the primary joined table

11726 11/21/2013 04:11 PM Aaron Marcuse-Kubitza

inputs/CVS/plot_/map.csv: documented assumptions about the units of fields

11725 11/21/2013 03:52 PM Aaron Marcuse-Kubitza

inputs/CVS/plot_/map.csv: documented assumptions about the units and meaning of numeric codes for fields

11396 10/21/2013 07:14 PM Aaron Marcuse-Kubitza

fix: bin/map: put template: comment out the "Put template:" label so that the output is valid XML, and displays properly in a browser rather than showing a syntax error

11278 10/17/2013 12:19 AM Aaron Marcuse-Kubitza

inputs/CVS/plot_/map.csv: realLatitude, realLongitude: remapped to UNUSED because these columns are actually empty

11264 10/13/2013 12:10 AM Aaron Marcuse-Kubitza

bugfix: inputs/CVS/plot_/create.sql: like for VegBank, need to compare place.*PLOT_ID*, not PLOTPLACE_ID, with plot.PLOT_ID

11107 09/29/2013 08:58 PM Aaron Marcuse-Kubitza

bugfix: mappings/VegCore-VegBIEN.csv: nest all taxonoccurrences inside a stratum event, so that the parent locationevent is always fully populated before child locationevents point to it. (previously, a stub parent event was created when the child event was imported first, which blocked the fully-populated parent event from being inserted later on.) this uses auto-folding (for VegBank/CVS) and auto-forwarding (for other datasources) to prune empty stratum events for taxonoccurrences that don't have strata. (see wiki.vegpath.org/Auto-folding, wiki.vegpath.org/Auto-forwarding for more info about these normalization techniques.) note that the inserted row counts stay exactly the same for all datasources except VegBank (which was being fixed), indicating that this signficant change to the mappings did not change the semantics of the import of taxonoccurrences.

10866 09/04/2013 11:06 PM Aaron Marcuse-Kubitza

inputs/*/*/test.xml.ref: updated source.shortname for new datasource name, which now starts out with .new suffix

10257 07/11/2013 12:09 PM Aaron Marcuse-Kubitza

inputs/*/*/map.csv: added distinguishing #... suffix (e.g. UNUSED#institutionID) to the special terms OMIT, PRIVATE, UNUSED (VegCore.vegpath.org#Special-terms) to avoid creating a collision in the staging table renaming

8801 05/02/2013 08:53 PM Aaron Marcuse-Kubitza

inputs/input.Makefile: SVN: add, %/add: */logs: also svn:ignore *.gz, used for compressed log files

8206 03/27/2013 08:23 PM Aaron Marcuse-Kubitza

mappings/VegCore-VegBIEN.csv: subplotID,subplot -> location.sourceaccessioncode: Fixed bug where need /_first to handle the case where both subplotID and subplot are provided

8176 03/25/2013 09:01 PM Aaron Marcuse-Kubitza

inputs/input.Makefile: %/.map.csv.last_cleanup: Run fix_line_endings after canon/translate to standardize Python's \r\n line endings back to \n. This prevents issues with mixed line endings because LibreOffice (and probably Excel) treat all cell-internal line endings as \n but row line endings as whatever the file had, while text editors like jEdit translate all line endings to whatever the autodetected line ending is. (This creates spurious line ending diffs when a map spreadsheet containing multiline cells is edited in a text editor.)

7703 02/26/2013 04:27 PM Aaron Marcuse-Kubitza

mappings/VegCore-VegBIEN.csv: _avg(): Use numeric param names to work with SQL functions

7464 02/05/2013 03:40 PM Aaron Marcuse-Kubitza

mappings/VegCore-VegBIEN.csv: locationID->location.sourceaccessioncode: Removed restriction that this mapping can't occur if geovalidation information is present. The locationID is no longer mapped to the place.sourceaccessioncode, so this filter is not necessary.

7404 01/31/2013 04:01 PM Aaron Marcuse-Kubitza

mappings/VegCore.csv: Regenerated from wiki

7216 01/14/2013 01:53 PM Aaron Marcuse-Kubitza

input.Makefile: Maps validation: %/new_terms.csv: Filter out terms that map to UNUSED, because these are not mappings that are useful as VegCore synonyms

7091 01/07/2013 04:13 PM Aaron Marcuse-Kubitza

mappings/VegCore.csv: Regenerated from wiki

7009 12/21/2012 12:07 PM Aaron Marcuse-Kubitza

mappings/VegCore-VegBIEN.csv: locationID/locationName + subplot -> location.sourceaccessioncode mapping: Fixed bug where subplot was incorrectly being mapped to this field even when there was no location*. (This field can only be populated if both location* and subplot are specified.) Also only map locationID for this, to avoid inconsistencies where one table supplies locationID+subplot, while another table supplies locationName+subplot, but they both get mapped to the same field, preventing plots from being matched up with their observations when creating the analytical_stem.

6992 12/20/2012 02:26 PM Aaron Marcuse-Kubitza

mappings/VegCore-VegBIEN.csv: authortaxoncode mappings: Only use authorTaxonCode if there is no plant ID, because an individual plant gets its own taxonoccurrence and thus needs the taxonoccurrence's IDs to be unique to the plant, regardless of what the author designates as the taxonoccurrence code

6989 12/20/2012 01:23 PM Aaron Marcuse-Kubitza

mappings/VegCore-VegBIEN.csv: Mapped authorTaxonCode

6834 12/14/2012 03:18 AM Aaron Marcuse-Kubitza

mappings/VegCore.csv: Terms: Removed namespace prefixes (dcterms:), because VegCore terms are globally unique within VegCore and there should not be multiple versions of the same VegCore term with different namespaces. Provenance is instead indicated in the Sources column, which contains not just a namespace but a full URL to each source term.

6831 12/14/2012 02:46 AM Aaron Marcuse-Kubitza

mappings/VegCore.csv: Term names: Changed special characters to _ because Redmine doesn't support special characters in HTML anchors (it removes everything except letters, numbers, _, and -)

6742 12/11/2012 01:53 AM Aaron Marcuse-Kubitza

inputs/CVS/: Refreshed data with new export from Bob