Analytical database¶

Crossed out steps are completed

From Brad's *Overview of the the BIEN2 Analytical Database*:

• Scripts it refers to are on nimoy in /home/bien/bien2_scripts/ or /home/bien/bien_web_scripts/bien_web/
• Steps have been reordered in priority order

1. Step 2: Create & populate table geoscrub
   • add to analytical DB: See geovalid column
2. Step 3: Prepare political division authority tables
3. Step 4: Geographic name resolution
   • need GNRS authority files
4. Step 5: Index New World observations: See isNewWorld statistics
   • uses GNRS-scrubbed names
5. Step 6: Geovalidation
   • need VegBIEN's place names and lat/longs
6. Step 12: Index higher taxa: See higherPlantGroup statistics
   • deal with genus homonyms in NCBI: prepend family to genus?
     homonyms:
     

     SELECT name_txt, count(*), array_agg(rank) FROM "NCBI".nodes GROUP BY name_txt HAVING count(*) > 1
     

7. Step 8: Detection and flagging of cultivated specimens: This is now parsed from the locality field
   • unwrap SQL from PHP scripts in nimoy:/home/bien/bien2_scripts/geoscrub/cultivated/ :
     • cult_by_locality/
     • cult_by_taxon/
     • cult_by_herbaria/
8. Step 9: Detection and flagging of FIA plots from plantations and logged areas
9. Step 10: Transfer geovalidation results to bien2 database
10. Step 11: Taxonomic resolution
11. Step 13: Index threatened and endangered species: See threatened statistics
12. Step 14: Aggregate species abundances from plot observations
13. Step 15: Standardize plot metadata regarding plot methodology
14. Step 16: Index data providers: See provider_count and Data providers .
    • cite subproviders in aggregators as well as the top-level datasources: done for specimens data
    • VegBIEN subproviders (DwC institutionCode):
      

      SELECT organizationname FROM "public.2012-11-08-21-59-04.r6098".party WHERE organizationname IS NOT NULL ORDER BY organizationname
      

15. Step 17: Determine a "concensusHabit" (growth form) for each species
    • parse specimen description
    • determine if can use existing scripts: yes, see Barbara's scripts under Growth forms
    • Barbara's scripts may be on nimoy in /home/bdobrin/HabitTables/Scripts/