BIEN geovalidation notes

========================

[Also see comments embedded in specific scripts in this directory.]

The bash and SQL statements contained in the files as ordered below

should be applied to carry out geographic name scrubbing and

geovalidation on a given corpus of BIEN location records.

That said, given the tight deadline under which this was done in order

to produced a geovalidated BIEN3 corpus in advance of the Nov 2013

working group meeting, and the corresponding manner in which much of

this was actually executed piecemeal in an iterative and interactive

fashion within a bash shell and psql session, I can't guarantee that the

code in its current state could be run end-to-end without intervention.

It's close, but probably not bulletproof.

1. geovalidate.sh

   - creates postgis DB and loads GADM2 data

2. geonames.sh

   - loads geonames.org data and adds some custom mapping logic

3. geonames-to-gadm.sql

   - contains SQL statements that build linkages between geonames.org

     names and GADM2 names

4. load-geoscrub-input.sh

   - dumps geoscrub_input from vegbien and loads it into the geoscrub db

5. geonames.sql

   - contains SQL statements that scrub asserted names and (to the

     extent possible) map them to GADM2

6. geovalidate.sql

   - contains (postgis-extended) SQL statements that score the validity

     of GADM2-scrubbed names against given point coordinates

The resulting 'geoscrub' table is what contains the scrubbed (i.e.,

GADM2-matched) names and various geovalidation scores.

Notes/Caveats/Todos:

* Clearly the SQL statements used in this procedure suffer from a lot of

  redundancy, and it might be worth trying to refactor once we're happy

  with the particular approach taken.

* Need to pull out more known notes/caveats/todos and highlight them :)