schemas/functions.sql: Added col_names()

root Makefile: Installation: Fixed bug where need to run schemas/public/install separately because schemas/install installs only the util schemas

root Makefile: Installation: install util schemas (temp functions py_functions) before inputs, so that inputs can use util functions in their postprocess.sql or create.sql scripts. (However, they must not use util functions in views or index functions, because these would be cascadingly deleted whenever the util schemas are reinstalled before an import.)

README.TXT: Single datasource import: Added by_col=1 to all commands

mappings/VegCore-VegBIEN.csv: locationRemarks: Remapped to locationnarrative because location.notespublic is a boolean field

lib/sql_io.py: mk_errors_table(): Create a unique index on the MD5 of the value and error instead of on the values directly, because some strings are too long to index (e.g. row 2537268 of MO.Specimen causes an error "index row size 3032 exceeds maximum 2712 for index [...] Values larger than 1/3 of a buffer page cannot be indexed")

inputs/import.stats.xls: Updated import times

bin/map: No mappings warning: Added explanation that this could also be due to no column name matches, and hint to check if you are importing the correct input table

inputs/MO/: Renamed Specimen.2/ -> now available Specimen/

inputs/MO/: Removed old import in Specimen/

Refreshed MO

csvs.py: TsvReader.next(): Fixed bug where empty line needs to be separately returned as [], because csv.reader would interpret it as EOF since the line ending has already been removed

csvs.py: sniff(): TSVs: Turn off quoting because TSVs use \-escapes instead of quotes to escape delimeters, newlines, etc.

csvs.py: InputRewriter.readline(): Surround function in a try block that prints all exceptions, so that debugging information is available if an error occurs when this stream is used as input for psycopg's copy_expert() (COPY FROM)

Populated inputs/MO/import_order.txt

Refreshed SALVIAS

Added web/main/CTFS/

inputs/SALVIAS/: Regenerated salvias_*.schema.sql from the MySQL version, to take advantage of my2pg improvements. The placeholder *_index columns which take the place of MySQL's inline index definitions have now been replaced by no-op CHECK constraints, so that there are no longer lots of dummy *_index columns in the map spreadsheets.

Added web/main/Redmine/ alias to VegBIEN/Redmine/

Added web/main/VegBIEN/Redmine/

web/main/VegBIEN/.htaccess: Forward to new db/ subdir

Added web/main/VegBIEN/db/

web/main/**/.htaccess: Removed RewriteCond -l tests because one of the -d or -f tests will always also pass, making the -l test unnecessary

web/main.conf: Added tolower RewriteMap

web/main/.htaccess: use separate lowercase version when available: Also support input strings in mixed case which is not the default capitalization, in addition to all-lowercase strings

web/main/.htaccess: use separate lowercase version when available: Generate the new dirname with a separate RewriteCond so its value can be used both in the -d test and in the replacement string, rather than separately for each

web/main/.htaccess: translate dotpaths: Allow an unescaped . at the beginning of a filename, because this will never be a . separator. This adds support for hidden files in dir paths, which now won't be interpreted as dotpaths. However, regular files with extensions still need to have the filename escaped because it will otherwise be interpreted as a dotpath.

web/main/.htaccess: Set Options +FollowSymLinks. It should be on by default ("All options except for MultiViews. This is the default setting." <http://httpd.apache.org/docs/2.2/mod/core.html#options&gt;), but this makes sure it will always be enabled.

web/main/.htaccess: Name the lowercased versions of dirs with a leading . (to make them hidden) instead of a trailing _ , to avoid having each dir listed twice in a row in the dir index

Added web/main/TNRS/

Added web/main/VegBank/

Added web/main/BIEN2/

web/main/index.php: Replaced - with . in namespaces to conform to new dotpath naming convention, which allows nesting of namespaces

web/main/SALVIAS/.htaccess: Forward to new dd/ subdir

Added web/main/SALVIAS/dd/

web/main/DwC/.htaccess: Forward to new terms/ subdir

Added web/main/DwC/terms/

web/main/**/.htaccess: don't redirect subdir paths: Fixed bug where can only match non-empty string, because otherwise the rule would match this directory, which should still have its redirects processed

web/main/index.php: Added back smaller spacing between the table columns

web/main/main.css: blockquote: Removed right margin so there isn't a big space between the table columns in index.php, which results from nesting right-padded blockquotes inside one another

web/main/index.php: Changed Brad-Boyle to just Brad because people's names only have to be unique within VegPath

web/main/.htaccess: Added fallback redirect to VegCore for paths without a namespace. This can be used to link to specific VegCore terms without needing to include the VegCore namespace.

Added web/main/VegBIEN/

Added web/main/servers/vegbiendev/

Added web/main/.phpPgAdmin/

web/main/.phpMyAdmin/.htaccess: Set [redirect] flag in case the dest server is on the same machine as VegPath itself

web/main/*/ lowercase versions: Renamed with _ suffix to avoid svn conflicts on case-insensitive filesystems such as Mac HFS+

web/main/.htaccess: Support lowercase versions of mixed-case dirnames without breaking case-insensitive filesystems such as Mac HFS+

web/main/: Added lowercase symlinks for mixed-case dirs to work with subdomain translation, which uses subdomains lowercased by DNS

web/main/index.php: Use absolute URLs for dependencies to work with subdomain translation, which adds components to the URL path

web/main/**/.htaccess: Use RewriteRule instead of RedirectMatch to handle incremental redirects internally instead of issuing a (much slower) redirect to the web browser each time. This also handles edge cases better, as [last] RewriteRules can be used to control when to forward control to a subdir, and doesn't require prepending the path to the dir the .htaccess file is in. Note that this requires all gateway dirs (dirs with subdirs) to contain special RewriteRules to avoid redirecting subdir paths and handle DirectoryIndex; see web/main/DwC/.htaccess. Also note that the regexp of a catch-all RewriteRule must exactly follow the template for internal or external redirects; see web/main/SALVIAS/db/.htaccess for internal redirects and web/main/DwC/history/.htaccess for external redirects.

web/main/.htaccess: translate dotpaths: Allow the part before the [] escape to contain [], to support labels that end in [] (like PHP array vars in the query string) labels with a simple array-subscript syntax (a[b]). This also shortens the regexp and makes it more readable without the \[\] in [^.\[\]/] . Note that this also allows invalid combinations of [] exprs (e.g. more than one per level or unbalanced []), which will still be translated but will probably not have the desired result.

web/main/.htaccess: translate dotpaths: Inline the [] escape regexp into the main regexp, because it is now approximately the same length as the []-matching portion of the main regexp and this greatly simplifies the code by removing the extra RewriteCond. Note that the translation rule is now a plain regexp (run repeatedly until no match), which can be used in any programming language that supports Perl-compatible regular expressions, not just mod_rewrite.

web/main/.htaccess: translate dotpaths: discardpath explanation: Clarified that the infinite loop resulted from reappending PATH_INFO (the Apache-matched filename)

web/main/.htaccess: translate dotpaths: Require any [] escape to have the ] at the end of the level, to simplify the [] regexp

web/main/.htaccess: translate dotpaths: Use a lookahead assertion to ensure that at least one character is matched as the head of the dotpath. This ensures that (.*/)? + the rest of the regexp does not match a path with a trailing /, which is a sealed /-path and not subject to dotpath translation.

web/main/.htaccess: translate dotpaths: Only support one [] escape per dot-level to (greatly) shorten the [] regexp. This does not pose a problem for encoding . because the entire level can simply be enclosed in [].

web/main/.htaccess: Uncommented ErrorDocument

web/main/.htaccess: translate dotpaths: Removed separate sealing of the /-path, which is now performed by the main RewriteRule because it appends a / even if there is no . suffix. This does not cause an infinite loop because a character is always added (/), which prevents the previously-matched head (after the last / but before any .) from being matched again.

web/main/.htaccess: translate dotpaths: Fixed bug where it's actually the portion before the . (but after the last /) that should be subject to []-unescaping, rather than the portion after the . . Fixed bug where [] escapes were not being unescaped because the wildcard .* group matched the whole head portion instead of allowing the []-captures to match.

web/main/.htaccess: translate subdomain to path: Don't use expr RewriteConds because they are not supported by Apache 2.2. Instead issue an external redirect with the subdomain part of the hostname removed, for the purpose of changing HTTP_HOST so that the replacement is not performed again if the mod_rewrite rules are run more than once.

web/main/**/.htaccess: Use RewriteEngine, inheriting the web/main/.htaccess rules, in order to translate dotpaths that follow /-paths to existing dirs

web/main/DwC/.htaccess: Moved DwC.history redirect to web/main/DwC/history/.htaccess

web/main/SALVIAS/db/.htaccess: Removed trailing / because for DB redirects, this is apparently necessary

web/main/.phpMyAdmin/.htaccess: Prepend http:// to the dest URL stem, instead of requiring the dest URL to provide the protocol, because the two // are replaced with one / by Apache when mod_rewrite is on, creating an invalid URL

web/main/.htaccess: translate dotpaths: Fixed bug where can't remove all [] escapes, because this would also remove [] from protected parts of the URL (i.e. before the last /). Instead just remove up to two [] escapes per dot-level, e.g. a.[b.c]=[d.e] -> a/b.c=d.e . Note that this removes the restriction against URLs containing "[]", because these [] will be left as-is as long as they are followed at some point by a / .

Removed web/.htaccess because it is not part of any site (web/main/.htaccess is the top-level .htaccess file for the default site)

web/main/.htaccess: translate dotpaths: replace all unescaped . with / : Allow empty components on either side of the . , to ensure that all unescaped dots are consistently replaced with / . Note that hidden directories that start with . will have the . preserved, because they have (or get) a trailing slash, which prevents dotpath from operating on them.

web/main/.htaccess: translate dotpaths: remove all [] escapes: Also remove empty [] to allow permalinking clients to easily generate escaped strings by URL-encoding the value and enclosing it in [], which will now also work even if the value is empty. This change will break any destination URLs that use "[]", such as PHP scripts that use this to indicate an array variable in the query string. In these URLs, the [] must now be %-encoded.

web/main/.htaccess: RewriteRules with [redirect] and http:// : Removed [redirect] because it is implied when the replacement is an absolute URL

web/main/.htaccess: RewriteRules with [redirect]: Added last because apparently redirect will not immediately cause the redirect, and will instead continue rewriting the URL ("You will almost always want to use [R] in conjunction with [L] (that is, use [R,L]) because on its own, the [R] flag prepends http://thishost[:thisport] to the URI, but then passes this on to the next rule in the ruleset, which can often result in 'Invalid URI in request' warnings." <http://httpd.apache.org/docs/2.2/rewrite/flags.html#flag_r&gt;)

web/main/.htaccess: RewriteRules: Added [discardpath,noescape] flags to all rules, because these are needed for correct interpretation of the replacement and should be the default setting. For some rules they may be optional, but it is better to include them on all rules to ensure there is never an unexpected effect from omitting them.

web/main/.htaccess: Added "RewriteOptions inherit" because this¹ should be the default setting, to ensure that global filter rules also occur in subdirs. Note that the presence of this directive in this .htaccess file does not solve this problem, but putting it there will remind writers of subdir .htaccess files that they need to include that directive....

web/main/SALVIAS/.htaccess: RedirectMatch for below top-level: Use /(.*?)/?$ suffix for consistency with other sources' RedirectMatch patterns

web/main/TCS,VegX/.htaccess: Fixed bug where need to match dirs by themselves in addition to when followed by a filename

web/main/DwC/.htaccess: Fixed bug where need to match dirs by themselves (/DwC/, /DwC/history/) in addition to when followed by a filename

web/main/.htaccess: mod_rewrite: Removed unneeded RewriteBase, because the default setting for RewriteBase is actually the dir of the .htaccess file itself, rather than always / . ("RewriteRule can be used in per-directory config files (.htaccess). There it will act locally, i.e., the local directory prefix is stripped at this stage of processing and your rewriting rules act only on the remainder. At the end it is automatically added back to the path." <http://httpd.apache.org/docs/1.3/mod/mod_rewrite.html#RewriteBase&gt;)

web/main/SALVIAS/*/.htaccess: Added trailing / to Redirect paths, which works now that a trailing / is automatically added to all paths by dotpath translation

web/main/SALVIAS/: Moved db/ and users/ actions to their corresponding subdirs. The rules will still be run in the appropriate order, because if a .htaccess exists in a subdir, it will be used in preference to the .htaccess in the parent dir.

my2pg: Removed SETs for standard_conforming_strings, escape_string_warning because these are also prepended by my2pg.data, which this script's output is piped to

my2pg.data: \0 removal: Fixed bug where need to handle all backslashes before the 0, because they might escape just each other (an even number of \) rather than also encoding the \0 (an odd number of \)

web/main/.htaccess: translate dotpaths to /-paths: Added examples

web/main/.htaccess: translate dotpaths to /-paths: Translate all .-separated components in the path to / instead of just those that exist in the filesystem. This allows per-source rules to match just a / for the path-element separator instead of sometimes / and other times . (depending on whether that portion of the dotpath had been translated). Support [] escapes that preserve any . they surround, e.g. a.[b.c] -> a/b.c . (Supporting []-escapes requires that the /-path be "sealed" by appending a / , to prevent the . -> / translation rule from reinterpreting a newly-unescaped [] sequence as a dotpath. This also requires that the translation rule ignore anything before the last /, because it could have been []-unescaped by a previous round of mod_rewrite, e.g. in another dir. All the per-source rules need to be retrofitted to support the new trailing / .) Note also the discardpath flag (http://httpd.apache.org/docs/2.2/rewrite/flags.html#flag_dpi) on all RewriteRules that perform a replacement on the input string (i.e. that have a pattern other than ^.*$). This flag is needed to avoid infinite loops, because otherwise, a critical bug in mod_rewrite causes it to reappend the filename portion of the input string (PATH_INFO) to the result, causing it to be present twice, e.g. a/b -> a/b/b (https://issues.apache.org/bugzilla/show_bug.cgi?id=38642).

web/main/nimoy/: Moved to web/main/servers/ so it can be grouped with other resources that are servers

web/main/index.php: $root: Fixed bug where need to use gethostbyaddr($_SERVER["SERVER_ADDR"]) instead of $_SERVER["SERVER_NAME"] because SERVER_NAME is actually set to whichever hostname the server is accessed through (e.g. path.vg or vegbiendev.nceas.ucsb.edu), rather than always being set to the reverse-DNS lookup of SERVER_ADDR, as is specified in the Apache docs ("If no ServerName is specified, then the server attempts to deduce the hostname by performing a reverse lookup on the IP address" <http://httpd.apache.org/docs/2.2/mod/core.html#servername&gt;)

web/main/.htaccess: Added back "don't rewrite existing paths" to ensure that rules after it will not rewrite any actual files (web page dependencies)

web/main/nimoy/.htaccess: Removed no longer needed trailing /

web/main/.phpMyAdmin/.htaccess: Add trailing / to provided URL in case the user didn't add it

web/main/nimoy/.htaccess: Removed unnecessary index.php from end of phpMyAdmin URL

web/main/DwC,SALVIAS,TCS,VegX: Added .htaccess redirects and dot-path-matching empty dirs, which will replace the index.php redirects with a simpler syntax (http://httpd.apache.org/docs/2.2/mod/mod_alias.html#redirect)

web/main/.htaccess: translate dot-paths to dirs: Documented that this only occurs when the dirs exist in the filesystem, and that each subdir Redirect directive needs a corresponding tree of (empty) dirs

Removed no longer used web/main/_archive/index.htm. Use web/main/index.php instead, which issues the Location: header to support clients other than web browsers.

web/vegpath/: Renamed to web/main/ because the content here is the main site hosted on vegbiendev. It will eventually contain internal links to VegBIEN resources in addition to external links (persistent URLs) to sources used in creating VegBIEN.

web/vegpath/.htaccess: Removed "don't rewrite existing paths" RewriteRule because there are no longer any rules after it that require it

web/vegpath/.htaccess: translate dot-paths to dirs: translate all levels of dot-paths present in the filesystem rather than just the first level, by using the [next] RewriteRule flag (http://httpd.apache.org/docs/current/rewrite/flags.html#flag_n). This avoids the need to translate further levels of dot-paths in each subdir's .htaccess.

web/vegpath/.htaccess: translate subdomain to path: Simplified using new subdomain2path RewriteMap, which avoids the need for a separate RewriteRule+RewriteConds for each number of nested subdomains

web/vegpath.conf: subdomain2path RewriteMap: Fixed bug where need `RewriteEngine on` in the VirtualHost config for a prg: RewriteMap to take effect ("External rewriting programs are not started if they're defined in a context that does not have RewriteEngine set to on." <http://httpd.apache.org/docs/current/rewrite/rewritemap.html#prg&gt;)

web/vegpath.conf: Added subdomain2path RewriteMap

Added web/subdomain2path for use by RewriteMap

web/vegpath.conf: Added spacing between sections and reordered directives