web/vegpath/index.php: Factored Redmine svn URLs out into $Redmine_svn

Added schemas/VegCore/Brad_Boyle/ with Brad's VegCore IDs terms spreadsheets

web/vegpath/index.php: URL fragment redirect: Removed loading indicator (blinking of resource being loaded) because for file downloads, the download will not redirect to a new page and the resource name will instead remain blinking

web/vegpath/index.php: URL fragment redirect: Fixed bug where need to use parse_mixed_path() instead of parse_dot_path() to support XPaths

web/vegpath/index.php: Terms: Added CTFS

web/vegpath/index.php: URL patterns header: Reworded comment to clarify that the elements that can be left out are in italics

web/vegpath/index.php: URL fragment redirect: Only perform fragment redirect if URL shortener requires it (as indicated by the $alias ending in #)

web/vegpath/index.php: nimoy nodes: Factored nimoy phpMyAdmin() calls out into nimoy_db() function

web/vegpath/index.php: nimoy nodes: Factored nimoy URL prefix out into $nimoy

web/vegpath/index.php: SALVIAS: Added subnodes for each SALVIAS database on nimoy

web/vegpath/index.php: VegCore: Changed suffix from .table.column to .term because VegCore terms are globally unique and don't need a table prefix

web/vegpath/index.php: BIEN2: Added subnodes for each BIEN2 database on nimoy

web/vegpath/index.php: Inlined VegBank() because it's only used once

web/vegpath/index.php: VegBank(): Use custom_separator()'s new $main_url param

web/vegpath/index.php: custom_separator(): Added $main_url param like in by_prefix(), for use when there is no path

web/vegpath/index.php: VegBank(): Use new custom_separator()

web/vegpath/index.php: VegBank(): Use new custom_separator()

web/vegpath/index.php: VegBank(): Use new custom_separator()

web/vegpath/index.php: Added custom_separator()

web/vegpath/index.php: Add # suffix to $alias instead of adding it in add_ref(), because not all aliases will use a # suffix

web/vegpath/util.php: Added array_non_empty(), join_non_empty()

web/vegpath/index.php: add_ref(): redirect: Don't display "Loading" as the page itself is not shown to the user when using a Location: header

web/vegpath/index.php: URL fragment redirect: Fixed bug where can't prefix the fragment with / because this won't work if VegPath is not at the top-level of the domain (and in any case is unnecessary to form a relative path)

web/vegpath/index.php: URL fragment redirect: Blink the namespace being loaded, rather than displaying "Loading" at the bottom of the page, to make it clearer what's being loaded

web/vegpath/index.php: add_ref(): Don't display the URL pattern in monospace, because fixed-width font is not needed and variable-width font is easier to read. This requires decreasing the header level because variable-width font is bigger than fixed-width font.

web/vegpath/main.css: Added entries for h6

web/vegpath/index.php: parse_dot_path() calls: Fixed bug where need to test result components using \`!== ""\` instead of just using the value as a boolean because PHP incorrectly interprets the string "0" as false

web/vegpath/util.php: Path: Made $tail default to ""

web/vegpath/index.php: URL fragment redirect: Prepend / instead of ? because filesystem paths are now redirected using .htaccess, removing the need to set the query string

web/vegpath/index.php: phpMyAdmin(): Fixed bug where can't use tbl_alter as the column target page because it requires a token param, which is only obtained after the user logs in

web/vegpath/util.php: partition(): Fixed bug where strpos() actually does take the search-for param second rather than first (but returns false on no match)

web/vegpath/util.php: partition(): Fixed bug where strpos() returns false rather than -1 on no match

web/vegpath/util.php: partition(): Fixed bug where strpos() takes the search-for param first rather than second

web/vegpath/index.php: php*Admin(): Append $target to $url in separate step from returning $url, to make it easier to debug-print the value of $url before returning it

web/vegpath/index.php: BIEN2: Fixed to use phpMyAdmin with correct URL instead of vegbiendev phpPgAdmin placeholder

web/vegpath/index.php: Added phpMyAdmin()

web/vegpath/.htaccess: mod_rewrite: RewriteBase: Changed to / now that VegPath is hosted at vegbiendev.nceas.ucsb.edu

web/vegpath/.htaccess: unknown paths: Don't redirect to index.php if the path already starts with index.php, to help avoid infinite redirect recursion

web/vegpath/.htaccess: unknown paths: Removed no longer accurate VegRef from comment header

vegpath/.htaccess: DirectorySlash: Lowercased On for consistency with other boolean directives

Added web/vegpath.conf with Apache VirtualHost config

vegpath/index.php: VegBank(): Fixed bug where name had incorrectly been replaced with $name

vegpath/index.php: phpPgAdmin(): Factored out appending of "&subject="

vegpath/: Archived no longer used vegpath/index.htm

vegpath/index.*: IH: URL-encode ' in search URL to avoid needing to backslash-escape it

vegpath/index.php: IH: Fixed search URL to undo incorrect search-and-replace of + with .

vegpath/index.*: Changed "Supported URLs" to "Supported URL patterns" because the provided short URLs take parameters that are filled into the destination URL

vegpath/index.*: Removed tagline from window title because the window title (which becomes the bookmark name) only needs to be long enough to identify what the page is, and should not be cluttered with descriptions

vegpath/index.*: Changed tagline to emphasize that the shortened URLs are especially intended to be human-readable, e.g. for use as column names

VegRef/: Renamed to VegPath to make it clearer that the purpose of the web service is to provide short, globally unique paths to resources (e.g. for use as global IDs <https://projects.nceas.ucsb.edu/nceas/projects/bien/wiki/Global_IDs&gt;)

VegRef/.htaccess: URL resolution: mod_rewrite: Fixed bug where also need to consider if %{REQUEST_FILENAME} is a dir or symlink, which occurs e.g. when redirecting to the VegRef/ dir itself

web/VegRef/.htaccess: Turned off RewriteEngine temporarily because it was preventing the internal VegRef redirection from working

Added web/VegRef/, which provides a URL-shortening service for vegetation references. VegRef URLs are short and human-readable enough to be used as database column names. This also causes Google spreadsheets to display a clickable link for each column name, which takes the user to the data dictionary definition for the term. String row IDs (such as global IDs <https://projects.nceas.ucsb.edu/nceas/projects/bien/wiki/Global_IDs&gt;) can also be clickable VegRef URLs.

inputs/NCU/Specimen/map.csv: cultivated: Map false (0) to NULL because cultivated can only be true or NULL, not false

inputs/CTFS/_src/_README.TXT: Added e-mails from Brad Boyle about issues with species aggregation methods

inputs/UNCC/Specimen/map.csv: Populated specimenHolderInstitutions

Added planning/goals/BIEN3_derived_data_products.docx.url

Added planning/goals/BIEN3_derived_data_products.docx

inputs/ARIZ/omoccurrences/map.csv: cultivationStatus: Map to NULL if it's false, because cultivated can only be true or NULL

schemas/vegbien.sql: analytical_stem_view: Merged taxonName_verbatim and scientificNameAuthorship_verbatim into scientificName_verbatim

schemas/vegbien.sql: analytical_plot: Added specimens fields at the end, for use by plots data that also tracks specimens (e.g. Madidi, SALVIAS)

README.TXT: Full database import: In PostgreSQL: Removed tests related to unscrubbed_taxondetermination_view and TNRS taxondeterminations, which are no longer needed because TNRS is joined directly to analytical_stem_view

mappings/VegCore.htm: Regenerated from wiki. Renamed specimenDonor to specimenDonorInstitution to clarify that this is an institution, not a collector.

inputs/UNCC/Specimen/map.csv: Mapped flower, fruit -> reproductiveCondition using translation is from e-mail conv between Jim Matthews at Habitat Assessment and Bob Peet on 2010-12-2~3

make_analytical_db: Support the $log flag, which toggles output to the terminal or a log file

inputs/import.stats.xls: Updated import times. The core import time has dropped by more than half (!) to ~12 hours, now that the TNRS scrubbing is added using a simple LEFT JOIN, instead of being pushed through the normalized schema. Not since October has the import been this fast!

import_times: When there are no matching logs, exit with status 0 instead of 1 because this is a normal condition

import_times: Fixed bug where sed would wait for user input from stdin when there were no matching logs

README.TXT: Full database import: record the import times in inputs/import.stats.xls: If the row counts do not match up: Added step to get the datasource names from the log file footers

inputs/UNCC/Specimen/map.csv: cultivated: assume anything non-NULL to be cultivated, so that we err on the side of excluding possibly-cultivated specimens

inputs/UNCC/_src/_README.TXT: Added e-mail conv between Jim Matthews at Habitat Assessment and Bob Peet about the meanings of the Leaf, Flower, Fruit, and Root codes

README.TXT: Full database import: Added steps to manually back up the DB. (These steps are specific to the import process, and include filters to exclude previous imports. There are also general backup steps in the Backups > Full DB section.)

inputs/ACAD/Specimen/map.csv: institutionCode: Replace "Acadia University" with the IH code ACAD

inputs/.TNRS/data.sql: Re-ran TNRS on the test_taxonomic_names so that the sample data would contain all the accepted names under the current acceptedScientificName formula, and would not produce any entries for unscrubbed names in tnrs_input_name

schemas/vegbien.sql: tnrs_input_name: taxonlabel.taxonomicname subquery: Don't exclude taxonlabels with a matched_label_id, because matched_label_id is no longer used to indicate that a name has a scrubbed name (this was used when all of TNRS was imported at the beginning of the import)

schemas/vegbien.sql: tnrs_input_name: accepted_name subquery: Renamed to the VegCore table name AcceptedTaxon

schemas/vegbien.sql: tnrs_input_name: taxonlabel.taxonomicname subquery: Use new MatchedTaxon view, which use VegCore column names where possible

schemas/vegbien.sql: tnrs_input_name: accepted_name subquery: Use new MatchedTaxon, AcceptedTaxon views, which use VegCore column names where possible

README.TXT: Full database import: Added instructions to run TNRS and view the TNRS log

README.TXT: Full database import: Removed no longer necessary command to scrub unscrubbed taxondeterminations

*.sql: concat_ws() calls: Fixed bug where need to surround it with NULLIF because concat_ws() returns '' instead of the required NULL when supplied with only NULL args

schemas/vegbien.sql: analytical_stem_view: concat_ws() calls: Fixed bug where arguments were still enclosed in ARRAY[], because the name had been changed from concat_delim() but PostgreSQL removed the VARIADIC keyword without also removing the ARRAY[] wrapper

schemas/vegbien.sql: Removed no longer needed unscrubbed_taxondetermination_view. Use taxonlabel joined to ScrubbedTaxon instead.

inputs/.TNRS/schema.sql: ScrubbedTaxon: Include only matches with a match score of at least 0.8 (score_ok())

inputs/.TNRS/schema.sql: Added back score_ok() and tnrs.tnrs_score_ok index

*.sql: Replaced concat_delim() with concat_ws(), which is built-in as of PostgreSQL 9.1 (http://stackoverflow.com/questions/1943433/postgresql-concat-ws-like-function)

inputs/U/Specimen/map.csv: Populated specimenHolderInstitutions

mappings/VegCore.htm: Regenerated from wiki. Renamed specimenHolders to specimenHolderInstitutions to make it obvious that this is a list of institutions, such as would be in institutionCode in a DwC export.

inputs/SpeciesLink/Specimen/map.csv: Fixed embedded line endings

inputs/NY/Ecatalog_all/map.csv: PlantFungDescription->occurrenceRemarks: Removed "plant fungus" label because this field actually contains the specimenDescription

inputs/.TNRS/import_order.txt: Removed public.unscrubbed_taxondetermination_view

inputs/.TNRS/: Moved no longer needed public.unscrubbed_taxondetermination_view/ to _archive/

root Makefile: Datasources: scrub: Removed no longer needed scrub-remake, which has been replaced by direct use of TNRS.ScrubbedTaxon

schemas/vegbien.sql: analytical_stem_view: taxondetermination: Fixed bug where need to include only the is_datasource_current taxondetermination

schemas/vegbien.sql: analytical_stem_view: Use new TNRS.ScrubbedTaxon directly instead of the scrubbed taxondeterminations added to the normalized DB

inputs/.TNRS/schema.sql: MatchedTaxon: Renamed matchedTaxonomicStatus to preferred name taxonomicStatus

mappings/VegCore.htm: Regenerated from wiki. Taxon: Moved scrubbing-related terms to MatchedTaxon.

inputs/.TNRS/schema.sql: Added matched* prefix where needed

inputs/.TNRS/schema.sql: MatchedTaxon: Added missing Name_submitted column

inputs/.TNRS/schema.sql: MatchedTaxon: Name_matched: Fixed mapping to matchedTaxonName