Import process¶

• Backup benchmarks
• Denormalizing a datasource
• Import steps
  • General import steps
    • Adding a single-table flat-file datasource
    • Connecting to vegbiendev to run import
    • Datasource import steps
    • Datasource left-joining steps
    • Datasource mapping steps
    • Setting up a VegBIEN VM from scratch
  • Specific import steps
    • Adding ACAD--a Darwin Core datasource
    • Adding Cyrille traits--a traits datasource
    • Adding Madidi--a flat-file plots datasource
    • Mapping a new table in VegBank--a SQL plots datasource
    • Refreshing ACAD--a Darwin Core datasource
    • Refreshing CVS--an MS Access plots datasource
    • Refreshing VegBank--a SQL plots datasource
• Individual datasource refresh
• New-style import
  • Db process comments
  • Importing to a denormalized VegCore schema
  • Refactor-in-place
  • Stage II vs stage IV validations
  • Staging table column renaming
    • Adding new-style import to a datasource
• Normalization techniques
  • Auto-folding
  • Auto-forwarding
  • Breadcrumb alternatives
  • Dup-select
    • Column-based import
    • Dup-return
  • Pull-forward
  • Trigger-based import
• Result filtering
• Row-based import benchmarks
• TNRS workflow

• Table of contents
• Import process
  • Steps
  • Instructions
  • Proposed refactoring
    • Staging table preprocessing

Steps¶

see Import steps

Instructions¶

See README.TXT > Datasource setup, Single datasource import, Full database import

Proposed refactoring¶

The proposed refactoring of the import process (below) would reduce the import time by as much as 14 hours, and avoid the need to maintain countless lines of code in Python and Makefiles:

Staging table preprocessing¶