Project

General

Profile

« Previous | Next » 

Revision 11707

Added by Aaron Marcuse-Kubitza about 11 years ago

bugfix: inputs/CVS/stemCount_/map.csv: ensure the aggregateoccurrence.sourceaccessioncode is always populated, because this is a required field when using sourceaccessioncodes. without it, the import will exclude rows which lack a value in this field because it cannot deduplicate on it for these rows, leading to the dropping of large numbers of occurrences. this shows up when comparing provider_count to the input table's row count, and produces the following error in the .errors table:
---
ERROR: duplicate key value violates unique constraint "aggregateoccurrence_taxonoccurrence_1_to_1"
DETAIL: Key (taxonoccurrence_id)=(1504108) already exists.
CONTEXT: SQL function "INSERT INTO aggregateoccurrence#1" statement 1
condition: sourceaccessioncode IS NULL
translated condition: "CVS.stemCount_.STEMCOUNT_ID" IS NULL
---
note that it uses the wrong unique constraint aggregateoccurrence_taxonoccurrence_1_to_1, because `sourceaccessioncode IS NULL` was true for some rows

View differences:

inputs/CVS/stemCount_/map.csv
1 1 
CVS,VegCore,Filter,Comments
2 
taxonImportance_ID,OMIT#taxonImportance_ID,,Omitted because STEMCOUNT_ID is more specific
2 
taxonImportance_ID,aggregateOrganismObservationID,/_join/1,
3 3 
taxonObservation_ID,taxonOccurrenceID,,
4 4 
stratum_ID,*stratum_id,,
5 5 
cover,coverPercent,,
......
16 16 
cust_cvs_corner4Presence,*cust_cvs_corner4Presence,,
17 17 
cust_cvs_corner5Presence,*cust_cvs_corner5Presence,,
18 18 
cust_cvs_originalCoverCode,*cust_cvs_originalCoverCode,,
19 
STEMCOUNT_ID,aggregateOrganismObservationID,,
19 
STEMCOUNT_ID,aggregateOrganismObservationID,/_join/2,
20 20 
stemDiameter,diameterBreastHeight_cm,,
21 21 
stemDiameterAccuracy,*stemdiameteraccuracy,,
22 22 
stemHeight,height_m,,
inputs/CVS/stemCount_/VegBIEN.csv
4 4 
cover,/location/locationevent/locationevent:[@fkey=parent_id]/taxonoccurrence/aggregateoccurrence/cover_fraction/_percent_to_fraction/value,
5 5 
stemDiameter,/location/locationevent/locationevent:[@fkey=parent_id]/taxonoccurrence/aggregateoccurrence/plantobservation/stemobservation/diameterbreastheight_m/_cm_to_m/value,
6 6 
stemHeight,/location/locationevent/locationevent:[@fkey=parent_id]/taxonoccurrence/aggregateoccurrence/plantobservation/stemobservation/height_m,
7 
STEMCOUNT_ID,/location/locationevent/locationevent:[@fkey=parent_id]/taxonoccurrence/aggregateoccurrence/sourceaccessioncode,
7 
taxonImportance_ID,/location/locationevent/locationevent:[@fkey=parent_id]/taxonoccurrence/aggregateoccurrence/sourceaccessioncode/_join/1,
8 
STEMCOUNT_ID,/location/locationevent/locationevent:[@fkey=parent_id]/taxonoccurrence/aggregateoccurrence/sourceaccessioncode/_join/2,
8 9 
taxonObservation_ID,"/location/locationevent/locationevent:[@fkey=parent_id]/taxonoccurrence/iscultivated/_alt/1/_if[@name=""if TaxonOccurrence""]/cond/_exists",
9 10 
taxonObservation_ID,/location/locationevent/locationevent:[@fkey=parent_id]/taxonoccurrence/sourceaccessioncode/_first/1,
10 
STEMCOUNT_ID,"/location/locationevent/locationevent:[@fkey=parent_id]/taxonoccurrence/sourceaccessioncode/_first/2/_if[@name=""if organism in AggregateOccurrence""]/cond/_exists",
11 
taxonImportance_ID,"/location/locationevent/locationevent:[@fkey=parent_id]/taxonoccurrence/sourceaccessioncode/_first/2/_if[@name=""if organism in AggregateOccurrence""]/cond/_exists/_join/1",
12 
STEMCOUNT_ID,"/location/locationevent/locationevent:[@fkey=parent_id]/taxonoccurrence/sourceaccessioncode/_first/2/_if[@name=""if organism in AggregateOccurrence""]/cond/_exists/_join/2",
11 13 
taxonObservation_ID,"/location/locationevent/locationevent:[@fkey=parent_id]/taxonoccurrence/taxondetermination[!isoriginal=true]/_if[@name=""if has taxondetermination""]/cond/_exists",
12 14 
basalArea,,** No join mapping for *basalarea **
13 15 
biomass,,** No join mapping for *biomass **
......
32 34 
stemHeightAccuracy,,** No join mapping for *stemheightaccuracy **
33 35 
stemTaxonArea,,** No join mapping for *stemtaxonarea **
34 36 
stratum_ID,,** No join mapping for *stratum_id **
35 
taxonImportance_ID,,** No non-empty join mapping for OMIT#taxonImportance_ID ** Omitted because STEMCOUNT_ID is more specific
36 37 
taxonImportance_dba_src_ID,,** No join mapping for *taxonImportance_dba_src_ID **
37 38 
taxonImportance_entry_herbLineID,,** No join mapping for *taxonImportance_entry_herbLineID **
38 39 
taxonImportance_entry_treeLineID,,** No join mapping for *taxonImportance_entry_treeLineID **
inputs/CVS/stemCount_/test.xml.ref
19 19 
                                                <height_m>$stemHeight</height_m>
20 20 
                                            </stemobservation>
21 21 
                                        </plantobservation>
22 
                                        <sourceaccessioncode>$STEMCOUNT_ID</sourceaccessioncode>
22 
                                        <sourceaccessioncode>
23 
                                            <_join>
24 
                                                <1>$taxonImportance_ID</1>
25 
                                                <2>$STEMCOUNT_ID</2>
26 
                                            </_join>
27 
                                        </sourceaccessioncode>
23 28 
                                    </aggregateoccurrence>
24 29 
                                    <sourceaccessioncode>$taxonObservation_ID</sourceaccessioncode>
25 30 
                                </taxonoccurrence>
inputs/CVS/stemCount_/new_terms.csv
1 
taxonImportance_ID,OMIT#taxonImportance_ID,,Omitted because STEMCOUNT_ID is more specific
1 
taxonImportance_ID,aggregateOrganismObservationID,/_join/1,
2 2 
cover,coverPercent,,
3 
STEMCOUNT_ID,aggregateOrganismObservationID,,
3 
STEMCOUNT_ID,aggregateOrganismObservationID,/_join/2,
4 4 
stemDiameter,diameterBreastHeight_cm,,

Also available in: Unified diff