schemas/functions.sql: _nullIf(): Removed NOT NULL constraint on null param, to support use a (nullable) column rather than a literal as the null-equivalent value
xml_func.py: Simplifying functions: Added _nullIf(), to remove calls with no null value
xml_dom.py: Added prune_parent()
schemas/functions.sql: Added _or()
schemas/functions.sql: Added _merge_words()
schemas/vegbien.sql: analytical_*: Renamed geosourceValid to geovalid. (It had gotten renamed in the reference -> source rename.)
mappings/VegCore.csv: Renamed georeferenceValid to geovalid
inputs/import.stats.xls: Updated import times. This now includes the Canadensys plants-related datasources HIBG, JBM, QFA, TRT, TRTE, UBC, VASCAN, and WIN.
inputs/import.stats.xls: Updated import times
Added inputs/HIBG/
Added inputs/JBM/
Added inputs/VASCAN/
Added inputs/WIN/
Added inputs/UBC/
Added inputs/TRTE/Specimen/
Added inputs/QFA/
Added inputs/TRT/
schemas/vegbien.sql: Allow bien_read to SELECT from all tables in the public schema
schemas/vegbien.sql: Allow bien_read to SELECT from analytical_aggregate, analytical_stem
lib/PostgreSQL-MySQL.csv: Removed GRANT/REVOKE because SCHEMA GRANTs are not supported in MySQL
pg_dump_vegbien: non-$owners mode: Removed --no-privileges in order to include GRANTs to other users
root Makefile: PostgreSQL: $(postgresReload-Linux): Making schemas/*.conf world-readable: Fixed bug where need to do this as the bien user, which owns the files
root Makefile: PostgreSQL: $(postgresReload-*): Make schemas/*.conf world-readable so it's readable by the postgres user, which the .conf installation is run as
root Makefile: PostgreSQL: $(postgresReload-*): Also install pg_hba.conf
root Makefile: PostgreSQL: Added postgres_reload to reload postgresql.conf and restart the DB
root Makefile: PostgreSQL: postgres-*: Factored postgresql.conf installation out in to $(postgresReload-*)
schemas/: Synced pg_hba.conf and pg_hba.Mac.conf's bien entries, which adds phpPgAdmin support (template1 access) on the Mac and bien_read access on Linux
root Makefile: VegBIEN DB: DB and users: Also create bien_read user for read-only access to the DB
schemas/pg_hba.Mac.conf: Allow access to the bien group rather than just the bien user, which will include bien_read
schemas/pg_hba.Mac.conf: Fixed bug where also need to allow password-based logins from the same machine, in order to work with pgAdmin
schemas/vegbien.ERD.poster.pdf: Updated to 33x51in poster size and 0.25in margins
README.TXT: Schema changes: Creating a poster of the ERD: Added section with the State St FedEx Kinkos' rates for posters ($10.25/sq ft laminated)
README.TXT: Schema changes: Creating a poster of the ERD: Changed "Measure the fractional height of the text onscreen" to "Determine the poster size"
Added schemas/vegbien.ERD.poster.pdf
Added schemas/vegbien.ERD.poster.core.print_options.png
Added schemas/vegbien.ERD.poster.core.pdf
schemas/pg_hba.Mac.conf: Fixed bug where needed ident entry for postgres superuser
Added config/bien_read_password
README.TXT: Schema changes: Added instructions to calculate the minimum VegBIEN poster size (to make the text as least as big as on the VegBank ERD poster), which is 35x54in portrait
schemas/vegbien.sql: analytical_stem_view: cultivated: Use location.iscultivated when taxonoccurrence.iscultivated is not available
Added inputs/FIA/FIA_COND_unique/, which contains the oldgrowth flag
mappings/VegCore-VegBIEN.csv: Mapped oldGrowth
schemas/functions.sql: Added _not()
mappings/VegCore.csv: Added oldGrowth
mappings/VegCore-VegBIEN.csv: Remapped cultivated to location when a TaxonOccurrence is not provided, indicating that the record is a plot
schemas/vegbien.sql: location: Added iscultivated for cases when entire plots rather than individual taxonoccurrences are marked as cultivated
inputs/FIA/: Added FIA_COND table from nimoy.geoscrub and code to generate a unique plot table from it, including the oldgrowth calculated field
Added inputs/FIA/Organism/postprocess.sql to cast PlotCD to a bigint
my2pg: Also remove (#) after bigint
Regenerated vegbien.ERD exports
schemas/vegbien.ERD.mwb: Fixed lines
schemas/vegbien.sql: source: Renamed fulltext to citation because according to the VegBank data dictionary <http://vegbank.org/vegbank/views/dba_tabledescription_detail.jsp?view=detail&wparam=reference&entity=dba_tabledescription&where=where_tablename#fulltext> this is actually the full text of the reference citation, not of the reference itself (it would be unusual to store that in VegBank)
schemas/vegbien.sql: Removed no longer needed sourcejournal, which can be stored in source and pointed to via parent_id instead of sourcejournal_id. sourcejournal.journal maps to source.fulltext, issn to isbn, and abbreviation to shortname.
mappings/VegCore-VegBIEN.csv: Mapped acceptedCounty, county to the matched place
schemas/vegbien.sql: source: Added matched_source_id
sql.py: parse_exception(): function MissingCastException: If 1st param is hstore, only perform the cast on the value param. This fixes a bug in _map() calls whose value is a non-text type, such as SALVIAS.plotMetadata.AccessCode.
sql_io.py: cast(): Use sql_gen.Cast() to generate the cast, in order to take advantage of its support for casts to unknown
sql_gen.py: Cast: Support casts to unknown by casting to text first
schemas/postgresql.conf: Turn on the error log
schemas/pg_hba.conf: Also grant the bien user access to template1, which is accessed by phpPgAdmin
schemas/vegbien.sql: source: Added parent_id for nested sources, e.g. an article in a journal
lib/forwarding.Makefile: $(subdirs): Also exclude .archive/
Renamed inputs/_archive/ to .archive/ so it wouldn't be treated as a datasource
README.TXT: Documentation: Redmine-formatted list of steps for column-based import: Use ACAD instead of QMOR, which was removed
inputs/Makefile: Import logs: $(rsyncLogs): Include log files at any depth in the directory tree rather than just 1-2 levels deep. This adds log files whose containing directories have been moved to _archive/ directories.
Added inputs/_archive/
Removed inputs/QMOR/ because it's an insect collection
schemas/vegbien.sql: projectcontributor: Removed surname, since this information is stored in party_id->party.surname
schemas/vegbien.sql: projectcontributor: Removed cheatrole, since there is already a role field and this field was unused in VegBank
schemas/vegbien.sql: role: Added values from projectcontributor.ROLE_ID <http://vegbank.org/vegbank/views/dba_tabledescription_detail.jsp?view=detail&wparam=projectcontributor&entity=dba_tabledescription&where=where_tablename#ROLE_ID>
schemas/vegbien.sql: sourcecontributor: role: Changed type to role
schemas/vegbien.sql: role enum: Added VegBank data dictionary values from <http://vegbank.org/vegbank/views/dba_fielddescription_detail.jsp?view=detail&wparam=1331&entity=dba_fielddescription¶ms=1331>
schemas/vegbien.sql: sourcecontributor: Renamed position to order for consistency with the ERD definition <http://vegbank.org/vegbank/views/dba_tabledescription_detail.jsp?view=detail&wparam=referencecontributor&entity=dba_tabledescription&where=where_tablename#position> and disambiguation from other meanings of position which are similar to role
schemas/vegbien.sql: sourcecontributor: Renamed roletype to role for consistency with the ERD definition <http://vegbank.org/vegbank/views/dba_tabledescription_detail.jsp?view=detail&wparam=referencecontributor&entity=dba_tabledescription&where=where_tablename#roleType>
inputs/.geoscrub/geoscrub_output/map.csv: Mapped to county, acceptedCounty
mappings/VegCore.csv: Added acceptedCounty
schemas/pg_hba.Mac.conf: Changed to match schemas/pg_hba.conf
schemas/pg_hba.conf: Fixed bug where also need an IPv6 bien entry with md5 authentication, because the IPv4 md5 authentication does not apply to "localhost", which is translated to the IPv6 address ::1
schemas/pg_hba.conf: Fixed bug where also need a local bien entry with md5 authentication, because the host-based md5 authentication applies only to literal IP addresses, not "localhost"
Added schemas/pg_hba.Mac.conf
schemas/pg_hba.conf: Restrict all accesses to the server except the bien user accessing vegbien using ident or a password, and the postgres superuser logging in using ident
inputs/.geoscrub/geoscrub_output/map.csv: Mapped countyvalidity to latLongInCounty
schemas/functions.sql: _map(): Fixed bug where entries that map to NULL were incorrectly being treated as if the entry didn't exist. Note that -> returns NULL both if the entry's value is NULL and if the entry doesn't exist, so ? must be used to recheck the presence of the key in the hstore.
mappings/VegCore-VegBIEN.csv: Mapped latLongInCounty
mappings/VegCore.csv: Added latLongInCounty
schemas/vegbien.sql: Added distance_to_county_m. Note that this can also be used to store latLongInCounty by mapping true to 0 and false to -1.
schemas/pg_hba.conf: Changed trust authentication back to ident/md5. Not sure how it got set to trust since I used md5 when enabling remote access to the DB for the bien user.
Added schemas/pg_hba.conf
schemas/vegbien.sql: place: Removed placecode to prevent datasources from creating duplicate entries for the same place, with different placecodes. This was a problem with the original BIEN2 geoscrub dataset, which contained duplicates.
schemas/vegbien.sql: analytical_stem_view: Fixed bug where need to join taxonoccurrence.collector_id to party because it's now an fkey rather than a literal name
schemas/vegbien.sql: analytical_*: Added coordinateUncertaintyInMeters
schemas/vegbien.sql: analytical_stem_view: Join to newWorldCountries on 2-digit ISO code instead of country name, to increase (BIEN2) newWorldCountries and GADM overlap
psql_vegbien: Run with sh because it no longer needs bash support
