inputs/.TNRS/schema.sql: added new derived columns to derived views

fix: schemas/util.sql: derived_col_update(): also need steps to drop column, because DROP __ CASCADE doesn't work when there are dependent views

inputs/.TNRS/schema.sql: _accepted_infraspecific{rank,epithet}: use array slice of new _accepted{genus,specific_epithet,infra_{rank,epithet}}, which is simpler than using remove_prefix() in __accepted_infraspecific_label

inputs/.TNRS/schema.sql: "[accepted_]genus__@DwC__@vegpath.org": don't need to use *Accepted_name anymore because _accepted{genus,specific_epithet,infra_{rank,epithet}} is now generated from *Accepted_name

inputs/.TNRS/schema.sql: taxon_match."__accepted_{genus,specific_epithet}": renamed to "__accepted_{genus,specific_epithet,infra_{rank,epithet}}" since this now includes these other ranks as well

bugfix: inputs/.TNRS/schema.sql: taxon_match."__accepted_{genus,specific_epithet}": use "*Accepted_name" instead of "Accepted_species[_binomial]__@TNRS__@vegpath.org" (from "*Accepted_name_species") because Accepted_name_species apparently sometimes does not match the Accepted_name and uses malformed Unicode characters

inputs/.TNRS/schema.sql: taxon_match: `inputs/.TNRS/data.sql.run refresh`: documented runtime (1 min)

bugfix: inputs/.TNRS/schema.sql: taxon_match: use "Accepted_species[_binomial]__@TNRS__@vegpath.org" instead of "*Accepted_name_species". this fixes a bug in __accepted_infraspecific_label where Accepted_name_species with trailing whitespace could not be prefix-removed from names that contained just a species binomial.

fix: inputs/.TNRS/schema.sql: taxon_match: added derived column "Accepted_species[_binomial]__@TNRS__@vegpath.org", which removes trailing whitespace

inputs/.TNRS/schema.sql: added steps to remove a column and to add a non-derived column

inputs/.TNRS/schema.sql: taxon_match: to remove columns or add columns at the end: merged into "to add a new derived column"

inputs/.TNRS/schema.sql: to add columns in the middle: renamed to "to move a column to the middle" for clarity

inputs/.TNRS/schema.sql: to populate a new column: updated to use util.derived_col_update()

fix: inputs/.TNRS/schema.sql: taxon_match: to remove columns or add columns: also need to run util.recreate_view()

inputs/.TNRS/schema.sql: taxon_match: to remove columns or add columns at the end: don't need to run `rm=1 inputs/.TNRS/data.sql.run` because this is now run by `make schemas/remake`

inputs/.TNRS/schema.sql: added new derived columns to derived views

schemas/util.sql: remove_prefix(), remove_suffix(): support case-insensitive matching

bugfix: inputs/.TNRS/schema.sql: taxon_match.__accepted_infraspecific_label: need to use case-insensitive matching of the removed prefix because TNRS lowercases part of the Accepted_name

schemas/util.sql: remove_prefix(), remove_suffix(): support case-insensitive matching

bugfix: inputs/.TNRS/schema.sql: taxon_match: use wrapper for util.remove_prefix() so CHECK constraints that use it don't get dropped when the util schema is reinstalled

inputs/.TNRS/schema.sql: taxon_match: COMMENT: added steps to port derived column changes to vegbiendev

bugfix: inputs/.TNRS/schema.sql: taxon_match: derived columns: use new "matched~Name[_no_author]___@TNRS__@vegpath.org" instead of "*Name_matched" so that "No suitable matches found." is removed before concatenating with other fields

inputs/.TNRS/schema.sql: taxon_match: added derived column "matched~Name[_no_author]___@TNRS__@vegpath.org", which removes the "No suitable matches found." string

inputs/.TNRS/schema.sql: reordered derived columns in dependency order

bugfix: inputs/.TNRS/schema.sql: "[accepted_]morphospecies[_binomial]__@Brad__.TNRS@vegpath.org": need to use "[accepted_]genus__@DwC__@vegpath.org" rather than "*Accepted_name" for this for rank = genus

inputs/.TNRS/schema.sql: taxon_match: added derived column "[scrubbed_]morphospecies[_binomial]__@Brad__.TNRS@vegpath.org"

bugfix: inputs/.TNRS/schema.sql: "[accepted_]genus__@DwC__@vegpath.org": need to populate this for rank = genus

inputs/.TNRS/schema.sql: taxon_match: added derived column "[scrubbed_]taxonomicStatus__@DwC__@vegpath.org"

bugfix: inputs/.TNRS/schema.sql: derived columns: use "Accepted_family__@TNRS__@vegpath.org" instead of "*Accepted_name_family" because "*Accepted_name_family" is sometimes missing

fix: inputs/.TNRS/schema.sql: taxon_match: added derived column "Accepted_family__@TNRS__@vegpath.org", which is needed because "*Accepted_name_family" isn't always populated

bugfix: inputs/.TNRS/schema.sql: taxon_match: to add columns in the middle: also need to run util.derived_cols_repopulate() since the dependency order has changed

fix: inputs/.TNRS/schema.sql: taxon_match: COMMENT: to add columns in the middle: also need to run util.derived_cols_update()

fix: inputs/.TNRS/schema.sql: taxon_match: COMMENT: updated util.derived_cols_sync() to util.derived_cols_update()

inputs/.TNRS/schema.sql: reordered derived columns in dependency order

inputs/.TNRS/schema.sql: reordered derived columns in dependency order

bugfix: schemas/util.sql: derived_cols_trigger_update(): need to clear derived cols first so old values won't be used in calculations

inputs/.TNRS/schema.sql: derived_cols_trigger_update(): put expr on same line as var, to save space

inputs/.TNRS/schema.sql: derived_cols_trigger_update(): removed lines between statements, which add too much space

fix: schemas/util.sql: derived_cols_trigger_update(): don't indent expr because may have multiple lines

bugfix: schemas/util.sql: derived_col_update(): set_comment(): vars can't have same name as params, which will be substituted

inputs/.TNRS/schema.sql: taxon_match: added derived column "[accepted_]morphospecies[_binomial]__@Brad__.TNRS@vegpath.org"

inputs/.TNRS/schema.sql: taxon_match: added derived column "[parsed_]morphospecies[_suffix]__@Brad__.morphosp@vegpath.org"

inputs/.TNRS/schema.sql: added new derived columns to derived views

fix: schemas/util.sql: derived_col_update(): column comment: formula: put on own line to support multiline exprs

bugfix: schemas/util.sql: derived_col_update(): column comment: need to use expr from DB, not as provided by user, to avoid this changing when derived_col_update() is rerun

inputs/.TNRS/schema.sql: taxon_match: added derived columns scrubbed_*

inputs/.TNRS/schema.sql: taxon_match: put matched-taxon derived columns before accepted-taxon derived columns

inputs/.TNRS/schema.sql: taxon_match: added derived column "[matched_]species[_binomial]~(Accepted_)__@TNRS__@vegpath.org"

inputs/.TNRS/schema.sql: _matched_has_accepted: renamed to matched_has_accepted because the leading _ did not create vertical alignment anyway, due to different quoting syntaxes

schemas/util.sql: derived_col_update(): add steps to rename column

bugfix: schemas/util.sql: derived_col_update(): don't set_comment() until CHECK constraint exists so that we can use its canon-ed formula

inputs/.TNRS/schema.sql: taxon_match: added derived column _matched_has_accepted

inputs/.TNRS/schema.sql: added new derived columns to derived views

inputs/.TNRS/schema.sql: taxon_match: added derived columns "[matched_]scientificName[_with_author]__@DwC__@vegpath.org", "[accepted_]scientificName[_with_author]__@DwC__@vegpath.org"

inputs/.TNRS/schema.sql: taxon_match: added derived columns "[accepted_]Infraspecific_rank[_abbr]__@TNRS__@vegpath.org", "[accepted_]infraspecificEpithet__@DwC__@vegpath.org"

inputs/.TNRS/schema.sql: taxon_match: added derived column "__accepted_infraspecific_{rank,epithet}"

inputs/.TNRS/schema.sql: taxon_match: added derived column __accepted_infraspecific_label

bugfix: schemas/util.sql: derived_col_update(): steps to modify: also need to run util.derived_cols_populate()

inputs/.TNRS/schema.sql: taxon_match: added derived columns "[accepted_]genus__@DwC__@vegpath.org", "[accepted_]specificEpithet__@DwC__@vegpath.org"

inputs/.TNRS/schema.sql: taxon_match: ran derived_cols_update()

inputs/.TNRS/schema.sql: taxon_match: documented that whenever columns are renamed, util.derived_cols_sync() should be run

inputs/.TNRS/schema.sql: taxon_match__fill_derived(): updated using new util.derived_cols_sync()

fix: inputs/.TNRS/schema.sql: taxon_match: to populate a new column: also need to recluster table so rows are exported in sorted order

fix: inputs/.TNRS/schema.sql: taxon_match: cluster the table so the rows are always exported in the same order, even when an UPDATE statement is used to run triggers

inputs/.TNRS/schema.sql: added _accepted{genus,specific_epithet} derived column, using the steps at wiki.vegpath.org/Postgres_queries#maintaining-derived-column-formulas-across-column-renames

fix: inputs/.TNRS/schema.sql: taxon_match: added back * prefixes on TNRS-namespace column names

inputs/.TNRS/schema.sql: MatchedTaxon: taxon_best_match: don't alias to `s` since this is no longer a nested select

inputs/.TNRS/schema.sql: MatchedTaxon: nested select: use just taxon_best_match since this no longer performs renamings

inputs/.TNRS/schema.sql: MatchedTaxon: renamed output columns to match input columns, so that the nested select would not be performing any renamings

fix: inputs/.TNRS/schema.sql: MatchedTaxon: added all columns from taxon_match (some of them apparently hadn't been included in MatchedTaxon)

fix: inputs/.TNRS/schema.sql: reordered columns to match taxon_match

fix: inputs/.TNRS/schema.sql: MatchedTaxon: added all columns from taxon_match (some of them apparently hadn't been included in MatchedTaxon)

fix: inputs/.TNRS/schema.sql: added back Family_matched, which had gotten merged with Name_matched_accepted_family when TNRS temporarily stopped populating Name_matched_accepted_family

inputs/.TNRS/schema.sql: MatchedTaxon: taxonomicStatus: moved outside the inner SELECT so that the inner SELECT would consist solely of renamings

inputs/.TNRS/schema.sql: MatchedTaxon: removed unneeded "Name_matched." prefix on source-specific names (only the * is needed because there is only one table)

inputs/SALVIAS/Source/header.csv: updated

inputs/.TNRS/Source/test.xml.ref: updated

inputs/VegBank/stemlocation_/test.xml.ref: updated

inputs/TEX/Specimen2/test.xml.ref: updated

inputs/GBIF/raw_occurrence_record_plants/test.xml.ref: updated

fix: inputs/.IRMNG/*/: added _no_import because this datasource is only used as a lookup table and does not need to be imported/scrubbed

bugfix: added nodes/_no_import, because the NCBI import doesn't work due to a Postgres 9.3 bug (#859). _no_import used to not be needed because we didn't import the hidden sources.

inputs/.TNRS/Source/map.csv: datasetURL: updated to indicate that this uses the TNRS dev server (the actual URL is private)

fix: inputs/VASCAN/Source/map.csv: observationGranularity should not be specimen because this datasource contains only taxa

fix: inputs/.geoscrub/geoscrub_output/: added _no_import because these tables are metadata that is used in the analytical DB. this is better than relying on bin/import_all not to import these.

fix: inputs/IUCN/: renamed to inputs/.IUCN/ (see issue #940)

fix: inputs/newWorld/: renamed to inputs/.newWorld/ (see issue #940)

bugfix: inputs/IRMNG/: renamed to inputs/.IRMNG/ so that this comes before TNRS, which depends on it (this is a metadata datasource, so it can start with "."). part of issue #940.

inputs/GBIF/_MySQL/.rsync_ignore: don't exclude GBIFPortalDB-*.data.sql.gz, even though this is an intermediate file, because it's better to have a backup of it locally. this was excluded in r13316 (2014-4-24) to free up disk space on the local machine.

fix: inputs/Madidi/LocationObservation/postprocess.sql: parse Spanish dates

fix: mappings/VegCore-VegBIEN.csv: mapped DwC year/month/day, which are split-date alternatives to eventDate. this fixes the missing eventDate in FIA.

bugfix: inputs/FIA/*/VegBIEN.csv: regenerated, replacing these with symlinks to the file used by new-style import

inputs/FIA/taxon_observation.**/test.xml.ref: updated

inputs/FIA/TREE/test.xml.ref: updated

inputs/FIA/REF_RESEARCH_STATION/test.xml.ref: updated

fix: inputs/bien2_traits/TraitObservation/map.csv: mapped VisitingDate to eventDate

fix: inputs/Madidi/LocationObservation/postprocess.sql: populated missing eventDate from PlotInventoryName (authorEventCode)

removed no longer needed inputs/.TNRS/grants.sql, since the grants in schema.sql are now being run

bugfix: inputs/input.Makefile: sql/install: schema.sql should not be passed through pg_dump_limit because it contains GRANT statements that need to be run

inputs/publishable datasources.xlsx: updated