fix: inputs/VASCAN/Source/map.csv: observationGranularity should not be specimen because this datasource contains only taxa
bugfix: bin/import_all: now that always using log files to fix output clutter, need to delete created logs if logging is turned off
bugfix: bin/import_all: don't errexit if a background process is Ctrl-C'd
bugfix: bin/import_all: was run without initial "." test: don't exit nonzero because this will close the subshell
bugfix: bin/import_all: ensure that this is run in a subshell, which is needed so errexits don't close the terminal window
bin/import_all: documented that this must be run in a subshell (obtained by running `$0`)
bugfix: bin/import_all: need to always use log files for background processes
fix: bin/import_all: Source/import: don't use by_col=1 for this because it's slower for small #s of rows. by_col mode is no longer needed for metadata-only tables because these tables now have a single empty row so that they also work in row-based mode.
fix: bin/import_all: hidden srcs: use with_all for this to avoid needing to list every source, and to display the backgrounded command with the variables substituted
bin/import_all: TNRS, geoscrub: integrated into the list of metadata sources
bin/import_all: TNRS, geoscrub: use import rather than publish because the non-imported tables have now been excluded
fix: inputs/.geoscrub/geoscrub_output/: added _no_import because these tables are metadata that is used in the analytical DB. this is better than relying on bin/import_all not to import these.
fix: bin/import_all: updated for new metadata datasource names (see issue #940)
fix: config/VirtualBox_VMs/vegbiendev/README.TXT: updated for new metadata datasource names (see issue #940)
fix: inputs/IUCN/: renamed to inputs/.IUCN/ (see issue #940)
fix: inputs/newWorld/: renamed to inputs/.newWorld/ (see issue #940)
bugfix: inputs/IRMNG/: renamed to inputs/.IRMNG/ so that this comes before TNRS, which depends on it (this is a metadata datasource, so it can start with "."). part of issue #940.
config/VirtualBox_VMs/vegbiendev/README.TXT: to test the import: set up the VM: added steps to create a VM snapshot when done
config/VirtualBox_VMs/vegbiendev/README.TXT: `make`: documented runtime (15 min)
fix: config/VirtualBox_VMs/vegbiendev/README.TXT: removed non-applicable step about the MySQL root password, which is not used for this procedure
added config/VirtualBox_VMs/vegbiendev/README.TXT with instructions to test the import
inputs/GBIF/_MySQL/.rsync_ignore: don't exclude GBIFPortalDB-*.data.sql.gz, even though this is an intermediate file, because it's better to have a backup of it locally. this was excluded in r13316 (2014-4-24) to free up disk space on the local machine.
fix: inputs/Madidi/LocationObservation/postprocess.sql: parse Spanish dates
validation/completeness/BIENDatesPopulated.xlsx: updated: Martha added Priority column
schemas/util.sql: replace_words(): use map_words() to enable inlining the concatenated hstore, so that it doesn't need to be concatenated on each function call
schemas/util.sql: added map_words()
schemas/util.sql: added date_from_spanish()
schemas/util.sql: added replace_words()
schemas/util.sql: added words(text)
schemas/util.sql: spanish_date_words(): switched to using hstore because this is better for the algorithm that will translate these
schemas/util.sql: added spanish_date_words()
schemas/util.sql: added `replacement` type
bugfix: mk_set_relation_metadata(): need to include col comments as well so that these are re-created properly by recreate()
schemas/util.sql: added mk_set_comments(regclass)
schemas/util.sql: show_set_comment(regclass): renamed to mk_set_comment() to match other mk_set_comment() functions
schemas/util.sql: added mk_set_col_comments(regclass)
schemas/util.sql: added cols(regclass)
schemas/util.sql: added mk_set_comment(col_ref)
added validation/completeness/BIENDatesPopulated.xlsx from Google spreadsheets
fix: mappings/VegCore-VegBIEN.csv: mapped DwC year/month/day, which are split-date alternatives to eventDate. this fixes the missing eventDate in FIA.
bugfix: inputs/FIA/*/VegBIEN.csv: regenerated, replacing these with symlinks to the file used by new-style import
inputs/FIA/taxon_observation.**/test.xml.ref: updated
inputs/FIA/TREE/test.xml.ref: updated
inputs/FIA/REF_RESEARCH_STATION/test.xml.ref: updated
bugfix: /README.TXT: to synchronize vegbiendev, jupiter, and your local machine: updated to new TWiki/ location
bugfix: /README.TXT: to synchronize vegbiendev, jupiter, and your local machine: updated to new VegBIEN/ location
fix: inputs/bien2_traits/TraitObservation/map.csv: mapped VisitingDate to eventDate
fix: inputs/Madidi/LocationObservation/postprocess.sql: populated missing eventDate from PlotInventoryName (authorEventCode)
schemas/util.sql: mk_set_comment(table_ regclass, comment text): use new util.mk_set_comment(text, text)
schemas/util.sql: added mk_set_comment(col col_ref, comment text)
schemas/util.sql: added sql(col_ref)
schemas/util.sql: added mk_set_comment(on_ text, comment text)
schemas/util.sql: added comment(col_ref)
schemas/util.sql: added col_num(col_ref)
removed no longer needed inputs/.TNRS/grants.sql, since the grants in schema.sql are now being run
bugfix: inputs/input.Makefile: sql/install: schema.sql should not be passed through pg_dump_limit because it contains GRANT statements that need to be run
schemas/public_.sql: iucn_red_list_view: documented how to regenerate iucn_red_list from this
exports/2014-6-4.Iara_Lacher.reserve_prioritization.csv.run: documented runtime (7.5 min) and rowcount (3.5 million)
bugfix: schemas/public_.sql: 2014-6-4.Iara_Lacher.reserve_prioritization: fixed slowdown in materializing the view, which was caused by source__observation_type(), by replacing this with a LEFT JOIN as was done for is_threatened_iucn
schemas/util.sql: in_south_america(): optimized by using BETWEEN instead of util.contained_within__no_dateline(). this reduces the filter time for 2014-6-4.Iara_Lacher.reserve_prioritization by about 10%.
fix: schemas/public_.sql: 2014-6-4.Iara_Lacher.reserve_prioritization: is_threatened_iucn: generate this much faster by doing a LEFT JOIN, which can reuse the same in-memory hash index for every row
bugfix: schemas/public_.sql: wrapper functions can't use util-schema types because these will cause the wrapper function to be cascadingly dropped when the util schema is reinstalled
bugfix: in_south_america(): must use util.contained_within__no_dateline() instead of util.contained_within_approx() to ensure that the more accurate geometry logic is used
bugfix: schemas/public_.sql: source__observation_type(): don't display NOTICEs about the search_path, because this function will be called millions of times
bugfix: schemas/public_.sql: in_iucn_red_list(): don't display NOTICEs about the search_path, because this function will be called millions of times
schemas/public_.sql: 2014-6-4.Iara_Lacher.reserve_prioritization: added is_threatened_iucn
schemas/public_.sql: added in_iucn_red_list()
schemas/public_.sql: added iucn_red_list, materialized from iucn_red_list_view
bugfix: schemas/vegbien.sql: iucn_red_list_view: need to include only names with an accepted name, and distinctify on the accepted names
schemas/public_.sql: added iucn_red_list
schemas/util.sql: materialize_view(): use util.copy() instead of util.materialize_query() so that all view metadata is transferred
schemas/public_.sql: iucn_red_list: renamed to iucn_red_list_view since this will be materialized
schemas/public_.sql: 2014-6-4.Iara_Lacher.reserve_prioritization: added collection_type (using source__observation_type())
schemas/public_.sql: added source__observation_type()
schemas/public_.sql: 2014-6-4.Iara_Lacher.reserve_prioritization: added accepted_subspecies, and removed accepted_taxon_name_with_author which had been substituting for it
schemas/public_.sql: added subspecies() wrapper
schemas/util.sql: added subspecies()
schemas/public_.sql: 2014-6-4.Iara_Lacher.reserve_prioritization: include only coordinates in South America
schemas/util.sql: added south_america(), in_south_america()
fix: schemas/util.sql: OPERATOR ~(geocoord, postgis.geometry): renamed to because this is not an approximate comparison for geometry
(geocoord, postgis.geometry): renamed to
fix: schemas/util.sql: contained_within_approx(geocoord, postgis.geometry): renamed to contained_within__no_dateline(__) because this is not an approximate comparison for geometry
schemas/util.sql: lat_long_in_new_world(): renamed to just in_new_world() because the lat/long is implied by the param type
schemas/util.sql: lat_long_in_new_world(): take a geocoord param instead of separate lat/long params
backups/TNRS.backup.md5: updated
schemas/util.sql: added contained_within_approx(geocoord, geometry) and corresponding OPERATOR ~@(geocoord, geometry)
schemas/util.sql: added OPERATOR ~@(geocoord, geography)
schemas/util.sql: lat_long_in_new_world(): use new contained_within_approx(geocoord, geography)
schemas/util.sql: added contained_within_approx(geocoord, postgis.geography), which enables specifying just `(lat, long)` without the ::util.geocoord type specifier
schemas/util.sql: OPERATOR (postgis.geography, postgis.geography): renamed to ~ because it's approximate
(postgis.geography, postgis.geography): renamed to ~
schemas/util.sql: contained_within(): renamed to contained_within_approx() because the latitude lines of geography type bounding boxes bulge outward, creating false positives above and below the bounding box
schemas/util.sql: added contained_within__no_dateline(geometry, geometry) and corresponding operator @
schemas/util.sql: geometry(geocoord): documented that it is not possible to create a cast for this, as a bug in pg_dump prevents the cast from being exported, even when no export filters are applied
schemas/util.sql: point(geocoord): renamed to geometry(geocoord) since this is now a cast
schemas/util.sql: point(): return geometry instead of geography to support using points with geometry arithmetic
schemas/util.sql: point(): take a single util.geocoord param instead of separate lat/long
schemas/util.sql: added geocoord type
schemas/public_.sql: 2014-6-4.Iara_Lacher.reserve_prioritization: include only georeferenced occurrences (lat/long NOT NULL)
schemas/util.sql: bounding_box(): use bounding_box__no_dateline() to construct the postgis.geometry object