Activity
From 05/14/2014 to 06/12/2014
06/12/2014
- 08:03 PM Revision 13739: web/username_prefix.php: username/password prompt message: added instructions to always append "." when visiting the homepage of the site ("vegpath.org.")
- 07:10 PM Revision 13738: schemas/VegBIEN/VegBIEN data dictionary.xlsx: moved to schemas/VegBIEN/data_dictionary/
- 05:42 PM Revision 13737: schemas/VegBIEN/VegBIEN data dictionary.xlsx: updated: analytical_stem_view: approved by/to discuss: renamed to * (removing "approved by") because the participants in today's conference call indicated that they were happy with the definitions, and the issues raised before the call had been addressed
- 05:32 PM Revision 13736: schemas/VegBIEN/VegBIEN data dictionary.xlsx: updated: added all tables used to create analytical_stem_view, not just those that Brian M listed. (Brian M generated his tables list from the analytical_stem_view columns rather than the joins, and renamed plot.** to plot, which is why his list ended up missing tables and in the wrong order.)
- 03:56 PM Revision 13735: added exports/2014-6-12.Jeff_Ott.climatic_range_determinants.csv.run
- 03:53 PM Revision 13734: schemas/vegbien.sql: added 2014-6-12.Jeff_Ott.climatic_range_determinants (wiki.vegpath.org/Data_requests)
- 03:38 PM Revision 13733: web/username_prefix.php: deletion comments: require -...- strikethrough on both () to avoid ambiguity when the deleted part starts or ends with -
- 11:51 AM Revision 13732: schemas/VegBIEN/VegBIEN data dictionary.xlsx: updated: resized columns to fit laptop screen
- 10:43 AM Revision 13731: schemas/VegBIEN/VegBIEN data dictionary.xlsx: updated: added my and Brian McGill's notes from the data dictionary conference call
- 10:11 AM Task #937 (New): fix missing date fields
- * -do a search for null date fields to figure out which datasources this applies to- _see "missing dates spreadsheet"...
- 08:00 AM Revision 13730: schemas/VegBIEN/VegBIEN data dictionary.xlsx: updated: added analytical_stem_view table definitions
- 07:38 AM Revision 13729: removed schemas/vegbien.ERD.MySQL_Workbench_6.1.6.mwb, because vegbien.ERD.mwb is now targeted to the correct version of MySQL Workbench (and is more up-to-date)
06/10/2014
- 07:35 PM Revision 13728: schemas/public_.sql: analytical_stem_view: applied data dictionary renamings, using the steps at wiki.vegpath.org/VegBIEN_schema_refactoring#apply-data-dictionary-renamings-to-database
- 07:20 PM Revision 13727: schemas/VegBIEN/VegBIEN data dictionary.xlsx: updated: added "to discuss" column requested by Martha. resized columns to avoid truncating the viewFullOccurrence term names.
- 03:40 PM Revision 13726: schemas/VegBIEN/VegBIEN data dictionary.xlsx: updated: finished populating definitions of terms
- 03:05 PM Revision 13725: web/username_prefix.php: deletion comments: also support strikethrough syntax for clarity: (-c_-)x@url -> c_x@url
- 02:30 PM Revision 13724: added schemas/DwC/hist symlink to history/ to help fit URLs in the 63-char limit
- 11:48 AM Revision 13723: web/username_prefix.php: insertion comments: documented that the semantic meaning of the [] portion also applies, even though it's not included in the linked term name
- 11:38 AM Revision 13722: schemas/VegBIEN/VegBIEN data dictionary.xlsx: updated: finished populating viewFullOccurrence->analytical_stem_view and analytical_stem_view->viewFullOccurrence mappings
- 11:12 AM Revision 13721: schemas/VegBIEN/VegBIEN data dictionary.xlsx: updated: use new deletion comments (with () ) to indicate portions of external terms whose semantic meaning does not apply to our usage
- 11:09 AM Revision 13720: web/username_prefix.php: added support for deletion comments of the form (c_)x(_d)@url -> c_x_d@url . deletion indicates that the semantic meaning of the () portion does not apply, even though it's included in the linked term name.
- 10:59 AM Revision 13719: web/username_prefix.php: comments: use [] instead of () because in writing, [] denotes insertion, while () (as an editing mark) denotes something to remove
- 10:58 AM Revision 13718: web/username_prefix.php: comments: use [] instead of () because in writing, [] denotes insertion, while () (as an editing mark) denotes something to remove
- 10:15 AM Revision 13717: schemas/VegBIEN/VegBIEN data dictionary.xlsx: updated: removed extra rows and columns so that they aren't included when copying and pasting
- 10:11 AM Revision 13716: schemas/VegBIEN/VegBIEN data dictionary.xlsx: updated: added tab to store viewFullOccurrence->analytical_stem_view mapping
- 09:55 AM Revision 13715: schemas/VegBIEN/VegBIEN data dictionary.xlsx: updated: added viewFullOccurrence column
06/09/2014
- 05:26 PM Revision 13714: bugfix: web/index.php: fixed bug where full directory index would not display correctly if invoked as /index.php instead of / (caused by incorrect URL)
- 04:19 PM Revision 13713: schemas/VegBIEN/VegBIEN data dictionary.xlsx: updated: added links to editable and clickable versions of the spreadsheet, to enable switching back and forth between them. populated more definitions and links.
- 12:49 PM Revision 13712: web/username_prefix.php: word-wraps: renamed to linewraps
- 12:48 PM Revision 13711: web/username_prefix.php: added support for word-wraps of the form x_-_.y@url -> x.y@url (the _ are needed to work in Google spreadsheets)
- 12:12 PM Revision 13710: bugfix: web/username_prefix.php: comments: need to use syntax that doesn't use : because Firefox will not update the "password" for the website with the new value after the : . the new syntax use () instead.
- 11:33 AM Revision 13709: web/username_prefix.php: comment prefixes: support embedded ":" in comment
06/07/2014
- 07:35 AM Revision 13708: schemas/VegBIEN/VegBIEN data dictionary.xlsx: updated: populated some definitions from VegCore data dictionary where available
- 07:20 AM Revision 13707: bugfix: schemas/VegBIEN/VegBIEN data dictionary.xlsx: updated: restored "" that had been stripped out when the data was copied out and pasted back, due to a bug in Google spreadsheets. prepended : before initial " as a workaround to prevent the " getting stripped out again.
- 07:15 AM Revision 13706: schemas/VegBIEN/VegBIEN data dictionary.xlsx: updated
- 07:04 AM Revision 13705: schemas/public_.sql: analytical_stem_view: applied data dictionary renamings, using the steps at wiki.vegpath.org/VegBIEN_schema_refactoring#apply-data-dictionary-renamings-to-database
- 12:22 AM Revision 13704: web/username_prefix.php: user2path(): comment prefixes: documented how to put the comment on a separate line when word-wrapping (c-_:x@url). this format works in Google spreadsheets.
- 12:20 AM Revision 13703: web/username_prefix.php: user2path(): comment prefixes: documented how to put the comment on a separate line when word-wrapping (c-_:x@url). this format works in Google spreadsheets.
- 12:12 AM Revision 13702: web/username_prefix.php: user2path(): added support for comment prefixes, which can be used to include term categories. *note*: when URL is used without http:// , you must include a non-letter in the comment (eg. c_:x@url) to work in Firefox.
06/06/2014
- 11:52 PM Revision 13701: web/username_prefix.php: user2path(): visually separated steps for easier readability
- 11:30 PM Revision 13700: fix: schemas/VegBIEN/VegBIEN data dictionary.xlsx: updated: restored "" that had been stripped out when the data was pasted in. set column widths to fit on a laptop screen.
- 11:20 PM Revision 13699: added schemas/VegBIEN/VegBIEN data dictionary.xlsx
- 10:47 PM Revision 13698: bugfix: web/username_prefix.php: user2path(): padding used to visually separate 1st element of URL: require all padding to be __ because sometimes terms contain a leading or trailing _
- 10:37 PM Revision 13697: bugfix: web/username_prefix.php: user2path(): padding used to visually separate 1st element of URL: support embedded @ inside highlighted part: also need to match end-padding when followed by "."
- 10:34 PM Revision 13696: web/username_prefix.php: user2path(): padding used to visually separate 1st element of URL: support embedded @ inside highlighted part
- 10:18 PM Revision 13695: web/username_prefix.php: user2path(): added support for padding used to visually separate 1st element of URL (_x_@url -> x@url)
- 08:32 PM Revision 13694: bugfix: web/username_prefix.php: need to handle : in login (which would normally indicate a password)
- 08:45 AM Task #934: Data Dictionary
- added links where link creation could be automated, and populated the example terms from each source
- 08:43 AM Revision 13693: schemas/public_.sql: analytical_stem_view: example terms from each source: added location in the VegBIEN core DB
- 08:15 AM Revision 13692: schemas/public_.sql: analytical_stem_view: terms in normalized schemas: put the source name right next to the term name (instead of after the table name) so it's easy to find
- 08:08 AM Revision 13691: schemas/public_.sql: analytical_stem_view: VegX terms: populated definitions
- 07:52 AM Revision 13690: schemas/public_.sql: analytical_stem_view: VegX terms: referenced to the specific table/subpath that they are in
- 07:32 AM Revision 13689: bugfix: schemas/public_.sql: analytical_stem_view: locationName: re-sourced to VegX because this is actually not a DwC term
- 07:24 AM Revision 13688: schemas/public_.sql: analytical_stem_view: populated definitions of linked DwC terms from DwC data dictionary
- 07:09 AM Revision 13687: schemas/public_.sql: analytical_stem_view: renamed terms to link URLs, using the steps at wiki.vegpath.org/VegBIEN_schema_refactoring#data-dictionary-rename-terms-to-link-URLs
- 06:47 AM Revision 13686: inputs/.TNRS/schema.sql: unsafe_taxon_names(): removed the name with "spp." now that this TNRS bug (https://pods.iplantcollaborative.org/jira/browse/TNRS-193) has been fixed
- 06:44 AM Revision 13685: schemas/public_.sql: analytical_stem_view: populated links from unrenamed term names, using the steps at wiki.vegpath.org/VegBIEN_schema_refactoring#data-dictionary-populate-links-from-unrenamed-term-names
- 06:02 AM Revision 13684: fix: schemas/public_.sql: analytical_stem_view: links: removed = before them because the link is in fact one member in a list of definitions (the = would apply to all definitions, not just the link, and since it apply to all definitions, it's redundant because of course the comment is what the column equals)
- 05:27 AM Revision 13683: web/.htaccess: auto-detect dotpath: need to explicitly set usernames_disabled=1, because for some reason the `%{QUERY_STRING} =""` check doesn't detect a query string
- 05:24 AM Revision 13682: web/.htaccess: auto-detect dotpath: removed handling of no longer used @ prefix (used to be used by username-based prefix subpaths)
- 05:18 AM Revision 13681: bugfix: web/.htaccess: username-based prefix subpaths: usernames_disabled: also need to support it being set in the current round
- 05:16 AM Revision 13680: bugfix: web/.htaccess: username-based prefix subpaths: removed support for query strings because these are often used at the root level, and should then be treated as a regular dotpath rather than something with a username-based prefix
- 05:07 AM Revision 13679: bugfix: web/.htaccess: forced mod_autoindex listing: use $usernames_disabled to ensure that usernames stay disabled even when /all path is removed
- 05:06 AM Revision 13678: bugfix: web/.htaccess: username-based prefix subpaths: added back usernames_disabled env var, because it *is* needed if a RewriteRule *in a previous round* creates an empty path (mod_rewrite runs the RewriteRules repeatedly until no replacements are made)
- 05:01 AM Revision 13677: bugfix: web/.htaccess: username-based prefix subpaths: REDIRECT_REQUEST_URI: also allow it to be unset, as it is when using usernames
- 04:57 AM Revision 13676: fix: web/index.php: directory index: don't append QUERY_STRING, as this is not used by /all
- 04:57 AM Revision 13675: web/.htaccess: username-based prefix subpaths: removed usernames_disabled env var because the bug that appeared to require this has now been fixed
- 04:56 AM Revision 13674: bugfix: web/.htaccess: username-based prefix subpaths: also need to check original URL (REDIRECT_REQUEST_URI), in addition to current URL (REQUEST_URI) to see if usernames should be disabled
- 04:39 AM Revision 13673: web/.htaccess: username-based prefix subpaths: allow explicitly disabling in a RewriteRule to handle special cases
- 04:28 AM Revision 13672: web/.htaccess: auto-detect dotpath: removed handling of no longer used @ prefix (used to be used by username-based prefix subpaths)
- 04:20 AM Revision 13671: bugfix: web/index.php: directory index: use QUERY_STRING, not HTTP_AUTHORIZATION (HTTP_AUTHORIZATION was probably left over from a previous attempt to provide username-based prefix subpaths)
- 02:44 AM Revision 13670: fix: schemas/public_.sql: analytical_stem_view: vegpath.org links: use ; instead of : between link and definition, because it makes more grammatical sense
- 02:42 AM Revision 13669: schemas/public_.sql: analytical_stem_view: vegpath.org links: removed no longer needed // suffix
- 02:38 AM Revision 13668: web/.htaccess, username_prefix.php: username-based prefix subpaths: don't require the // at the end , to avoid clutter. **IMPORTANT**: in order to access the home page after visiting a URL with a username, you must now append "." to the host ("vegpath.org.") (other pages are not affected by this problem). since vegpath.org is a URL shortener, this is a worthwhile tradeoff to avoid needing the // suffixes everywhere.
- 02:19 AM Revision 13667: bugfix: web/.htaccess: subdomains: each path element must be non-empty, to avoid matching a trailing . incorrectly
- 01:18 AM Revision 13666: web/.htaccess: username-based prefix subpaths: updated documentation to use leading // instead of trailing ? or ?@
- 01:14 AM Revision 13665: web/username_prefix.php: $_SERVER["PHP_AUTH_USER"] test: factored $_SERVER["SCRIPT_URL"] out of both branches of if statement
- 12:36 AM Revision 13664: web/username_prefix.php: don't remove @ from query string because that is no longer a special character
- 12:08 AM Revision 13663: schemas/public_.sql: analytical_stem_view: added links and definitions for half a dozen of the terms from various sources
06/05/2014
- 11:46 PM Revision 13662: fix: web/.htaccess: username-based prefix subpaths: changed format to subpath@host//path because the trailing ? makes it look like the sentence the URL is in is a question
- 08:38 PM Revision 13661: web/.phpPgAdmin/.htaccess: action redirect: only append ? if adding to the query string
- 01:17 PM Revision 13660: inputs/publishable datasources.xlsx: updated
- 01:17 PM Revision 13659: inputs/publishable datasources.xlsx: updated
- 12:27 PM Revision 13658: inputs/publishable datasources.xlsx: updated
- 12:12 PM Revision 13657: inputs/publishable datasources.xlsx: updated
- 11:33 AM Revision 13656: web/.phpPgAdmin/.htaccess: view: default to displaying the columns rather than the definition
- 09:04 AM Task #934: Data Dictionary
- indicated provenance of the @analytical_stem_view@ (viewFullOccurrence) terms
- 09:02 AM Revision 13655: schemas/public_.sql: analytical_stem_view: indicated provenance of the rest of the terms
- 08:32 AM Revision 13654: schemas/public_.sql: analytical_stem_view: flagged the DwC terms
- 08:20 AM Revision 13653: web/.phpPgAdmin/.htaccess: support viewing the columns of a view rather than the definition
- 08:16 AM Revision 13652: web/.phpPgAdmin/.htaccess: RewriteRules: use overridable action var
- 08:14 AM Revision 13651: web/.phpPgAdmin/.htaccess: support overriding action param in a later RewriteRule
- 08:04 AM Revision 13650: web/.phpPgAdmin/.htaccess: removed skip= because this creates a confusing control flow, and prevents using chain
- 07:51 AM Revision 13649: backups/TNRS.backup.md5: updated
- 07:30 AM Revision 13648: schemas/public_.sql: added plot.**.modify()
- 07:24 AM Revision 13647: inputs/.TNRS/schema.sql: *_modify(): removed the need to manually maintain copies of the dependent view definitions with the *s in place, because the *s are now added automatically by view_def_to_orig()
- 07:21 AM Revision 13646: inputs/.TNRS/schema.sql: *_modify(): removed the need to manually maintain copies of the dependent view definitions with the *s in place, because the *s are now added automatically by view_def_to_orig()
- 07:15 AM Revision 13645: fix: schemas/util.sql: view_def_to_orig(): also need to merge .* expressions resulting from a SELECT * of a join, to avoid duplicated columns
- 06:52 AM Revision 13644: fix: schemas/util.sql: view_def_to_orig(): require at least 6 cols to avoid false positives in the expansion match pattern
- 06:49 AM Revision 13643: bugfix: schemas/util.sql: view_def_to_orig(): 1st col: can't prepend \y because it considers only \w chars, not "
- 06:35 AM Revision 13642: bugfix: schemas/util.sql: view_def_to_orig(): don't match whitespace in the middle of a "" identifier, as this could throw off the parser
- 06:34 AM Revision 13641: schemas/util.sql: eval(): restore user's intent by running util.view_def_to_orig() on the query to unexpand expanded * expressions
- 05:50 AM Revision 13640: schemas/util.sql: pg_get_viewdef(): use util.view_def_to_orig()
- 05:50 AM Revision 13639: schemas/util.sql: added view_def_to_orig()
- 05:46 AM Revision 13638: fix: schemas/util.sql: pg_get_viewdef(): should be STABLE because it references system catalogs
- 05:44 AM Revision 13637: schemas/util.sql: added pg_get_viewdef() wrapper, which unexpands expanded * expressions
- 04:42 AM Task #936 (New): in TNRS client, detect Internal Server Errors when they occur instead of after the pauses have timed out
- the problem is that an "is it ready" check can produce a variety of errors, including Internal Server Errors, but all...
- 04:30 AM Revision 13636: lib/tnrs.py: max_names: raised back up to 500 now that a workaround for the Internal Server Errors is in place (https://github.com/iPlantCollaborativeOpenSource/TNRS/issues/7)
- 03:57 AM Revision 13635: fix: schemas/public_.sql: tnrs_input_name: added ORDER BY to ensure a stable sort order. this also creates better query plans by enabling merge joins.
- 03:53 AM Task #935: TNRS crashes when scrubbing names with two infraspecific epithets
- a workaround has been implemented that filters out unsafe taxon names. however, the unsafe names are hardcoded, so a ...
- 03:50 AM Revision 13634: bugfix: schemas/public_.sql: tnrs_input_name: need to exclude unsafe taxon names as a workaround to #935 (TNRS crashes when scrubbing names with two infraspecific epithets)
- 03:45 AM Revision 13633: schemas/public_.sql: taxonlabel: added index on taxonomicname to facilitate retrieving rows from tnrs_input_name
- 03:39 AM Revision 13632: inputs/.TNRS/schema.sql: added taxon_name_is_safe()
- 03:30 AM Revision 13631: inputs/.TNRS/schema.sql: added unsafe_taxon_names()
06/04/2014
- 10:23 PM Task #935 (Resolved): TNRS crashes when scrubbing names with two infraspecific epithets
- h3. problem
the error occurs on this name:
Lamiaceae Agastache pallidiflora (Heller) Rydb. spp. pallidiflora va... - 06:00 PM Task #917: TNRS: Instructions for new version with TPL
- The taxon names Brad sent for testing the rescrubbing are now attached.
- 03:01 PM Revision 13630: fix: lib/tnrs.py: max_names: lowered to 50 because the dev TNRS server is now always crashing with an Internal Server Error when scrubbing 500 names at a time (https://github.com/iPlantCollaborativeOpenSource/TNRS/issues/7)
- 11:41 AM Revision 13629: web/.htaccess: auto-detect dotpath in query string: also auto-prepend & for query strings appended to a username-based prefix, so that these don't need to have & manually prepended to make them appendable
- 11:14 AM Revision 13628: web/.htaccess: auto-detect dotpath in query string: also auto-detect dotpath for query strings appended to a username-based prefix, so that these don't need to have . prepended to make them appendable
- 02:25 AM Revision 13627: web/username_prefix.php: no prefix: use $_SERVER["SCRIPT_URL"] instead of rtrimming $_SERVER["REQUEST_URI"]
- 02:24 AM Revision 13626: web/username_prefix.php: has prefix: use same expr whether or not it has a query string, because the with-query-string expr works for both cases
- 02:20 AM Revision 13625: web/username_prefix.php: use if statements instead of `? :` for clarity
- 02:10 AM Revision 13624: web/username_prefix.php: use $subpath instead of $_SERVER["PHP_AUTH_USER"] for clarity
- 02:09 AM Revision 13623: bugfix: **/.htaccess: redirects with fragment: qsappend does not support fragment, so append it separately
- 01:59 AM Revision 13622: web/.htaccess, username_prefix.php: username-based prefix subpaths: support dotpath in query string (which will be appended verbatim to the prefix). use "__?@__" instead of "__?__&" to signal username-based prefix, so that the query string does not end in a dangling & .
- 01:53 AM Revision 13621: bugfix: inputs/.TNRS/.htaccess: qsappend does not support fragment, so append it separately
- 01:06 AM Revision 13620: bugfix: web/.htaccess: auto-detect dotpath in query string: don't exclude dotpaths that come before a normal query string (&k=v...)
- 01:05 AM Revision 13619: web/.htaccess: auto-detect dotpath in query string: documented what each RewriteCond does
- 01:03 AM Revision 13618: web/.htaccess: auto-detect dotpath in query string: use %{QUERY_STRING} instead of %0 for clarity
- 12:48 AM Revision 13617: web/.htaccess, web/username_prefix.php: username-based prefix subpaths: support query strings in addition to paths
- 12:18 AM Revision 13616: web/index.php: removed no longer needed redirection of username-based prefix subpaths, which is now handled by web/.htaccess
- 12:16 AM Revision 13615: web/.htaccess: handle username-based prefix subpaths (of the form "subpath@url?") for all paths rather than just for the root (which were handled by index.php)
- 12:11 AM Revision 13614: added web/username_prefix.php, for use with username-based prefix redirects
06/03/2014
- 11:13 PM Revision 13613: web/index.php: subpath@host? redirects: also support multiple @ and nested . in prefix: a@b.c@host? -> host?b.c.a
- 11:01 PM Revision 13612: web/index.php: support prefix redirect of the form subpath@host? , which is better than subpath.host because case is preserved and special chars allowed. (must require trailing ? for redirect, to avoid needing login to view the page itself.)
- 10:47 PM Revision 13611: bugfix: web/index.php: full directory index: appending query string: need to use $_SERVER["QUERY_STRING"], not $_SERVER["HTTP_AUTHORIZATION"] for this
- 03:24 PM Task #934 (New): Data Dictionary
- Define the Data Dictionary terms for VegBIEN and associated tables.
# -First define the (approximately 60) terms nee... - 11:36 AM Revision 13610: schemas/public_.sql: sync_*(): renamed to *_modify() to facilitate finding these functions when modifying the corresponding view (using the new naming convention for a view's on-modify function)
- 10:36 AM Revision 13609: bugfix: inputs/.TNRS/schema.sql: MatchedTaxon_modify(): updated to include taxon_scrub derived fields
- 10:25 AM Revision 13608: bugfix: schemas/util.sql: mk_drop_from_create(): need to match first rather than last CREATE
- 10:21 AM Revision 13607: inputs/.TNRS/schema.sql: *_modify(): allow running without a view_query, as recreate_view() now supports this
- 09:52 AM Revision 13606: schemas/util.sql: recreate_view(): support omitting the view_query if the view has already been modified (eg. for public.*_view, which allow changing the view as a separate step)
- 08:56 AM Revision 13605: fix: schemas/public_.sql: sync_*(): use util.copy() instead of CREATE TABLE AS so that table and column comments are also copied. this avoids the need to separately add the same comments to the view *and* its materialized table.
- 08:52 AM Revision 13604: bugfix: schemas/util.sql: recreate(): need to handle case where util.mk_drop_from_create() is NULL
- 08:45 AM Revision 13603: bugfix: schemas/util.sql: mk_drop_from_create(): only match CREATE if no custom DROP came before it
- 08:29 AM Revision 13602: bugfix: schemas/public_.sql: sync_geoscrub_input_to_view(): `CREATE TABLE geoscrub_input AS __`: needs `LIMIT 0`
- 08:19 AM Revision 13601: fix: schemas/util.sql: explain2notice_msg_if_can(): also need to catch invalid_cursor_definition ("cannot open multi-query plan as cursor")
- 08:15 AM Revision 13600: schemas/public_.sql: sync_analytical_stem_to_view(): removed DROP TABLE IF EXISTS because this is now done automatically by util.recreate()
- 07:39 AM Revision 13599: schemas/util.sql: added copy()
- 07:36 AM Revision 13598: schemas/util.sql: added copy_data()
06/02/2014
- 04:24 PM Revision 13597: fix: lib/tnrs.py: Constrain by Source: turn it on so that the download settings reflect what TNRS actually used, while this is broken
- 06:19 AM Revision 13596: fix: lib/tnrs.py: max_names: reduced back to 500 because even 5000 crashes the dev TNRS server
- 05:52 AM Revision 13595: lib/tnrs.py: max_names: reduced to 5000 because 100,000 causes an internal server error
- 05:25 AM Revision 13594: bugfix: /README.TXT: Full database import: To run TNRS: to rescrub all names: also need to re-create public-schema views that were cascadingly deleted
- 05:23 AM Revision 13593: /README.TXT: Full database import: To run TNRS: added steps to rescrub all names
- 04:59 AM Revision 13592: backups/TNRS.backup.md5: updated
- 04:50 AM Revision 13591: lib/tnrs.py: switched to downloading all matches per name, as is needed to implement #917. note that this will break the parts of the schema that use the tnrs table, until Brad's match-picking algorithm can be implemented, but this tradeoff is necessary to be able to begin scrubbing sooner (Martha; wiki.vegpath.org/2014-05-29_conference_call#TNRS)
- 04:35 AM Revision 13590: schemas/vegbien.sql: tnrs_input_name: don't scrub accepted names, as using multiple matches per name no longer provides a single accepted name to scrub. instead, the Accepted_* fields can be whitespace-split to generate the same columns that would have been generated by the scrubbing (and without the overhead of the extra TNRS call).
- 04:27 AM Revision 13589: fix: inputs/.TNRS/schema.sql: added back index on Name_submitted, which is needed for tnrs_input_name to work properly (now that there is no automatic index created by a unique constraint)
- 04:08 AM Revision 13588: schemas/vegbien.sql: tnrs_input_name: don't scrub accepted names, as using multiple matches per name no longer provides a single accepted name to scrub. instead, the Accepted_* fields can be whitespace-split to generate the same columns that would have been generated by the scrubbing (and without the overhead of the extra TNRS call).
- 03:43 AM Revision 13587: fix: inputs/.TNRS/schema.sql: tnrs: removed unique constraint on Name_submitted, Name_matched because there can be more than one match with the same Name_matched (but different accepted names, etc.)
06/01/2014
- 09:00 PM Revision 13586: fix: inputs/.TNRS/schema.sql: tnrs.tnrs__valid_match index: made it non-unique to allow multiple matches per name, as is needed to implement #917
- 05:00 AM Revision 13585: bugfix: inputs/.TNRS/schema.sql: tnrs__match_num__fill(): only fill if not set, to support case where tnrs is being restored from a .sql file (where match_num is already set)
- 04:36 AM Revision 13584: inputs/.TNRS/schema.sql: tnrs: documented runtime to add a constraint (3 min)
- 04:35 AM Revision 13583: inputs/.TNRS/schema.sql: unique constraint on Name_submitted: added Name_matched to allow multiple matches per name, as is needed to implement #917
- 03:44 AM Revision 13582: inputs/.TNRS/schema.sql: tnrs: documented how to populate a new column
- 03:41 AM Revision 13581: inputs/.TNRS/schema.sql: tnrs: pkey: use match_num instead of Name_number to allow multiple matches per name, as is needed to implement #917
05/31/2014
- 10:31 PM Revision 13580: inputs/.TNRS/schema.sql: tnrs.match_num: made it NOT NULL now that it's populated
- 10:28 PM Revision 13579: inputs/.TNRS/schema.sql: tnrs: populate match_num
- 10:25 PM Revision 13578: inputs/.TNRS/schema.sql: tnrs: populate match_num
- 09:50 PM Revision 13577: inputs/.TNRS/schema.sql: tnrs: documented how to add and remove columns
- 08:58 PM Revision 13576: inputs/.TNRS/schema.sql: made COMMENTs start on their own line, using the steps at wiki.vegpath.org/Postgres_queries#make-COMMENTs-start-on-their-own-line
- 08:58 PM Revision 13575: inputs/.TNRS/schema.sql: made COMMENTs start on their own line, using the steps at wiki.vegpath.org/Postgres_queries#make-COMMENTs-start-on-their-own-line
- 08:11 PM Revision 13574: inputs/test_taxonomic_names/_scrub/*: updated to TNRS schema
- 08:10 PM Revision 13573: inputs/.TNRS/schema.sql: tnrs: added match_num
- 08:06 PM Revision 13572: inputs/.TNRS/data.sql.run: refresh(): documented runtime (1 min)
- 08:03 PM Revision 13571: schemas/Makefile: added back rename/%, which *is* used by `inputs/.TNRS/data.sql.run refresh`. updated it to use schema bundles.
- 06:44 PM Revision 13570: inputs/.TNRS/schema.sql: added tnrs__match_num__next()
- 06:29 PM Revision 13569: inputs/.TNRS/schema.sql: added tnrs__batch_begin() trigger to populate the match_num (match sort order)
- 05:06 AM Revision 13568: exports/2014-3-11.Jeff_Ott.climatic_range_determinants.csv.run: documented export_() runtime (11 min) and rowcount (11 million matching the filter criteria)
05/30/2014
- 06:34 PM Revision 13567: inputs/.TNRS/schema.sql: added tnrs__batch_begin() trigger to populate the match_num (match sort order)
- 06:22 PM Revision 13566: schemas/util.sql: added seq__reset()
- 06:15 PM Revision 13565: schemas/util.sql: added seq__create()
- 06:11 PM Revision 13564: fix: schemas/util.sql: try_cast(), is_castable(): also catch invalid_schema_name, thrown by `'pg_temp.__'::regclass`
- 02:49 PM Revision 13563: backups/TNRS.backup.md5: updated
- 07:50 AM Revision 13562: lib/tnrs.py: max_names: increased to 100000 because the dev server can handle more names (no simultaneous users), as decided in the conference call (wiki.vegpath.org/2014-05-29_conference_call#TNRS)
- 07:42 AM Revision 13561: schemas/vegbien.ERD.mwb: regenerated exports
- 07:39 AM Revision 13560: schemas/vegbien.ERD.mwb: re-updated to schemas/vegbien.my.sql, which now recognizes the broken tables. fixed sync issues. vegbien.ERD.mwb is now fully in sync with vegbien.my.sql.
- 07:37 AM Revision 13559: fix: lib/PostgreSQL-MySQL.csv: need to replace "double precision" with "double" to work with MySQL Workbench 5.2.47
- 06:58 AM Revision 13558: schemas/vegbien.ERD.mwb: updated to schemas/vegbien.my.sql. some tables weren't recognized (likely due to bugs in MySQL Workbench 5.2.47), and have been left as-is (unsynced). note that downgrading to 5.2.35 is not an option, because that is fatally broken by a system upgrade.
- 05:58 AM Revision 13557: fix: schemas/vegbien.ERD.mwb: use schemas/vegbien.my.sql instead of schemas/vegbien.my.sql.changes.sql as the sync source
- 05:27 AM Revision 13556: schemas/vegbien.ERD.mwb: switched back to MySQL Workbench 6.1.6 version, which also works with MySQL Workbench 5.2.47
- 05:17 AM Revision 13555: schemas/vegbien.ERD.mwb: restored version for MySQL Workbench 5.2.35 (undid r13549), as 6.1.6 has bugs in the DDL file sync
- 05:05 AM Revision 13554: schemas/vegbien.ERD.mwb: renamed to vegbien.ERD.MySQL_Workbench_6.1.6.mwb to differentiate the versions for different versions of MySQL Workbench
- 04:49 AM Revision 13553: removed no longer used config/VirtualBox_VMs/VegCore ERD/. use the Ubuntu 14.04 VM instead, which now has the VegCore ERD setup.
- 04:48 AM Revision 13552: config/VirtualBox_VMs/Ubuntu 14.04/Ubuntu 14.04.vbox: added VegCore ERD setup
- 03:30 AM Revision 13551: schemas/: svn:ignore *.changes.sql, needed for MySQL Workbench 6.1.6
- 03:30 AM Revision 13550: schemas/vegbien.ERD.mwb: updated
- 03:16 AM Revision 13549: schemas/vegbien.ERD.mwb: updated layout for MySQL Workbench 6.1.6, which uses a different line spacing
05/29/2014
- 11:53 AM Revision 13548: lib/tnrs.py: commented out the value of max_names that is not active, for clarity
05/28/2014
- 03:54 PM Revision 13547: config/VirtualBox_VMs/vegbiendev/vegbiendev.vbox: updated: merged post-bootloader installation steps into one snapshot now that eth1 addition is successful
- 03:49 PM Revision 13546: config/VirtualBox_VMs/vegbiendev/vegbiendev.vbox: updated, which adds eth1
- 03:47 PM Revision 13545: config/VirtualBox_VMs/Ubuntu 12.04/Ubuntu 12.04.vbox: updated
05/27/2014
- 11:12 PM Revision 13544: lib/tnrs.py: sources: updated to list/sort order in issue #917
- 10:39 PM Revision 13543: added exports/2014-3-11.Jeff_Ott.climatic_range_determinants.csv.run
- 10:38 PM Revision 13542: schemas/public_.sql: added 2014-3-11.Jeff_Ott.climatic_range_determinants
- 10:17 PM Revision 13541: schemas/public_.sql: analytical_stem_view: added scrubbed_taxon_name_with_author, needed by Jeff Ott's analysis (wiki.vegpath.org/Data_requests)
- 10:13 PM Revision 13540: inputs/.TNRS/schema.sql: taxon_scrub.scrubbed_unique_taxon_name.*: added scrubbed_taxon_name_with_author, needed by Jeff Ott's analysis (wiki.vegpath.org/Data_requests)
- 10:09 PM Revision 13539: schemas/public_.sql: added scrubbed_specific_epithet, scrubbed_species_binomial, which are needed by Jeff Ott's analysis (wiki.vegpath.org/Data_requests)
- 10:08 PM Revision 13538: fix: schemas/public_.sql: sync_analytical_stem_to_view(): removed fkey to source.shortname because this prevents reloading individual datasources
- 09:24 PM Task #927 (Resolved): move morphospecies name formation to normalized DB
- see [[TNRS derived columns#scrubbed_morphospecies_binomial]] > @scrubbed_morphospecies_binomial@
- 09:19 PM Revision 13537: added downloads/
- 03:27 AM Revision 13536: schemas/public_.sql: added scrubbed_specific_epithet, scrubbed_species_binomial, which are needed by Jeff Ott's analysis (wiki.vegpath.org/Data_requests)
- 03:21 AM Revision 13535: fix: schemas/util.sql: mk_drop_from_create(): also support CREATE queries that include the SELECT statement on the same line as the CREATE
- 12:33 AM Revision 13534: schemas/public_.sql: analytical_stem_view: scrubbed_morphospecies_binomial: use new taxon_scrub.scrubbed_morphospecies_binomial
- 12:28 AM Revision 13533: inputs/.TNRS/schema.sql: taxon_scrub: added scrubbed_morphospecies_binomial, analogous to accepted_morphospecies_binomial for scrubbed_*
- 12:13 AM Revision 13532: inputs/.TNRS/schema.sql: taxon_scrub: added scrubbed_morphospecies_binomial, analogous to accepted_morphospecies_binomial for scrubbed_*
05/26/2014
- 11:54 PM Revision 13531: inputs/.TNRS/schema.sql: taxon_scrub: documented how to modify it
- 11:51 PM Revision 13530: inputs/.TNRS/schema.sql: added taxon_scrub_modify()
- 11:41 PM Revision 13529: schemas/util.sql: create_if_not_exists(): print message if already exists, so the function doesn't inexplicably appear not to have run at all
- 11:20 PM Revision 13528: inputs/.TNRS/schema.sql: added taxon_scrub_modify()
05/23/2014
- 06:17 PM Revision 13527: inputs/.TNRS/schema.sql: MatchedTaxon_modify(): use simpler util.recreate_view()
- 06:15 PM Revision 13526: inputs/.TNRS/schema.sql: MatchedTaxon_modify(): documented usage
- 06:09 PM Revision 13525: schemas/util.sql: added recreate_view(), a special case of util.recreate()
- 04:24 PM Revision 13524: fix: schemas/util.sql: recreate(): usage: use `schema` instead of `schemas`
- 04:20 PM Revision 13523: config/VirtualBox_VMs/vegbiendev/vegbiendev.vbox: updated
- 03:57 PM Revision 13522: bugfix: schemas/public_.sql: _plots_20_tnrs_names: verbatim_name_with_author: use taxonverbatim.taxonomicname rather than taxonlabel.taxonomicname
- 02:16 AM Revision 13521: config/VirtualBox_VMs/Ubuntu */*.vbox: updated
05/22/2014
- 04:43 PM Task #932 (New): Discovering BIEN data through iPlant
iPlant will assign DOIs to datasets made public through its infrastructure. For the BIEN db data, the plan is:
* t...- 04:32 PM Task #931 (New): BIEN db at iPlant
- Parent issue for tasks related to moving BIEN3 relational database and associated analytical views to iPlant.
- 04:20 PM Revision 13520: bugfix: config/VirtualBox_VMs/: switched from symlinks to hard links, because svn does not follow symlinks
- 02:07 PM Revision 13519: added config/VirtualBox_VMs/, containing the .vbox settings and password.txt (a non-empty password is needed for some system commands)
05/21/2014
- 07:30 PM Revision 13518: inputs/.TNRS/schema.sql: MatchedTaxon_modify(): removed no longer needed DROP VIEW statement
- 07:30 PM Revision 13517: schemas/util.sql: recreate(): perform the correct DROP VIEW in the function itself so that the caller does not have to worry about forming it properly
- 07:19 PM Revision 13516: bugfix: schemas/util.sql: mk_drop_from_create(): added `DROP`
- 07:18 PM Revision 13515: schemas/util.sql: added mk_drop_from_create()
- 06:52 PM Revision 13514: schemas/util.sql: added regexp_match()
- 05:58 PM Revision 13513: planning/meetings/BIEN conference call availability.xlsx: updated
- 05:50 PM Revision 13512: fix: schemas/util.sql: force_recreate(): renamed to just recreate(), because "force" normally implies that things will be deleted, which this function does not do
- 05:49 PM Revision 13511: fix: schemas/public_.sql: _plots_20_tnrs_names: fixed order of tnrs_taxonomic_status, tnrs_accepted_name_family
- 04:45 PM Revision 13510: schemas/public_.sql: _plots_20_tnrs_names: added tnrs_taxonomic_status, as requested in issue #915
- 04:34 PM Revision 13509: schemas/public_.sql: analytical_stem_view: taxonomic_status: don't filter with map_taxonomic_status() anymore since this is now done by "TNRS"."MatchedTaxon"
- 04:25 PM Revision 13508: fix: inputs/.TNRS/schema.sql: MatchedTaxon.taxonomicStatus: filter using map_taxonomic_status() so that the corrected value is available in the normalized DB, not just analytical_stem
- 04:05 PM Revision 13507: inputs/.TNRS/schema.sql: MatchedTaxon: to modify: use new MatchedTaxon_modify(), which eliminates the work of putting together the dependent views
- 03:53 PM Revision 13506: inputs/.TNRS/schema.sql: added MatchedTaxon_modify()
- 03:44 PM Revision 13505: schemas/util.sql: show_create_view(): use new mk_set_relation_metadata()
- 03:41 PM Revision 13504: schemas/util.sql: added mk_set_relation_metadata()
- 04:13 AM Revision 13503: bugfix: inputs/.TNRS/schema.sql: map_taxonomic_status(): need to use accepted name instead of scrubbed name (which also includes no-opinion names), as described at http://wiki.vegpath.org/2013-11-14_conference_call#taxonomic-fields. this used to be the accepted name, but got switched when the concatenated name was also used to store the matched name for no-opinion names.
- 01:50 AM Revision 13502: schemas/public_.sql: _plots_20_tnrs_names: use accepted_morphospecies_binomial, which is now available directly in the TNRS results, instead of just the morphospecies_suffix, as requested in issue #915
- 01:27 AM Revision 13501: inputs/.TNRS/schema.sql: MatchedTaxon: documented how to modify it (using util.force_recreate())
- 01:23 AM Revision 13500: fix: lib/PostgreSQL-MySQL.csv: also remove left-behind lines such as `$$);`
- 12:59 AM Revision 13499: schemas/util.sql: force_recreate(): documented usage, which is somewhat complex and has several important parts
05/20/2014
- 05:46 PM Revision 13498: inputs/.TNRS/schema.sql: MatchedTaxon, etc.: added accepted_morphospecies_binomial derived field
- 05:40 PM Revision 13497: bugfix: schemas/util.sql: restore_views(): need to specify view name for a manual existence check, in case the view definition becomes invalid, which would produce a nonstandard (uncatchable) exception
- 05:33 PM Revision 13496: bugfix: schemas/util.sql: explain(): don't display any log messages printed by IMMUTABLE functions in the query, which EXPLAIN apparently runs
- 05:21 PM Revision 13495: schemas/util.sql: create_if_not_exists(): added relation param, which can be used to ensure that a standard exception is always generated if the relation exists, even if the table definition would be invalid (which generates a variety of exceptions)
- 04:26 PM Task #929 (New): change TNRS client to store metadata
- * see details in [[2013-06-13 conference call#include-TNRS-version-and-settings-in-TNRS-cache]]
* after implementi... - 04:17 PM Task #928 (New): switch to new TNRS setup
- # #927 (part of #915)
# #917
# #916 (using rescrubbed names)
# #915
# -#929 (metadata)- _postponed_ - 03:26 PM Task #927 (Resolved): move morphospecies name formation to normalized DB
- * currently, it's only in the analytical_stem view
- 02:39 PM Revision 13494: /README.TXT: to back up the vegbiendev databases: added step to review diff before performing backup, in case the backup scripts break and try to synchronize things incorrectly
05/19/2014
- 09:08 PM Revision 13493: bugfix: schemas/util.sql: try_cast(), is_castable(): also need to handle syntax_error_or_access_rule_violation, which is thrown by ::regclass
- 08:52 PM Revision 13492: schemas/util.sql: added relation_exists()
- 08:27 PM Revision 13491: schemas/util.sql: save_drop_views()/restore_views(): store the view path in addition to the definition so it can be used in restoring
- 08:24 PM Revision 13490: bugfix: schemas/util.sql: in_reverse(): can't use `SELECT *` for composite types because this expands them
- 08:21 PM Revision 13489: bugfix: schemas/util.sql: array_reverse(): can't use `SELECT *` for composite types because this expands them
- 07:50 PM Revision 13488: schemas/util.sql: save_drop_views()/restore_views(): use opaque return type so it can be changed without changing the local var type in functions that use this
- 07:22 PM Revision 13487: schemas/util.sql: force_recreate(): recreate_users_cmds: renamed to restore_views_info since this is now whatever is needed by util.restore_views()
- 07:16 PM Revision 13486: schemas/util.sql: added restore_views() and use it in force_recreate()
- 02:07 AM Revision 13485: bugfix: schemas/util.sql: save_drop_views(): views must be *dropped* in reverse dependency order, but *returned* in forward dependency order
- 01:36 AM Revision 13484: bugfix: schemas/util.sql: force_recreate(): dependent views need to be re-created in *forward* dependency order. this is done by reversing the dependencies only for save_drop_views().
- 12:48 AM Revision 13483: bugfix: schemas/util.sql: mk_set_comment(): need to use VIEW instead of TABLE for views
- 12:29 AM Revision 13482: bugfix: schemas/util.sql: show_create_view(): also need to include view comment
- 12:24 AM Revision 13481: fix: schemas/util.sql: mk_set_comment(): added ;
- 12:20 AM Revision 13480: fix: schemas/util.sql: show_set_comment(): don't display a COMMENT ON statement if no comment, because this will be appended to table defs, etc. and would create clutter
- 12:16 AM Revision 13479: bugfix: schemas/util.sql: mk_set_comment(): need to handle NULL comment properly
- 12:06 AM Revision 13478: schemas/util.sql: added show_set_comment(regclass)
05/18/2014
- 11:57 PM Revision 13477: schemas/util.sql: set_comment(): use util.mk_set_comment()
- 11:48 PM Revision 13476: schemas/Makefile: added vegbien/install, vegbien/uninstall which also handle TNRS
05/17/2014
- 06:26 PM Revision 13475: fix: schemas/Makefile: vegbien.sql: also remake inputs/.TNRS/schema.sql since its contents change along with vegbien.sql
- 06:22 PM Revision 13474: schemas/util.sql: added mk_set_comment()
- 06:22 PM Revision 13473: bugfix: lib/runscripts/util.run: $is_runscript: unexport so don't pass it to invoked scripts
- 06:01 PM Revision 13472: backups/pg_snapshot: documented runtime for initial backup (4 h)
- 05:36 PM Revision 13471: bugfix: schemas/util.sql: force_recreate(): recreate_users_cmds: use create_if_not_exists() rather than eval(), because cmd might manually re-create a deleted dependent view, causing it to already exist
- 05:27 PM Revision 13470: bugfix: schemas/util.sql: save_drop_view(): util.show_create_view(): don't include `OR REPLACE` because that causes nonuniform errors (eg. invalid_table_definition), instead of the standard duplicate_table exception caught by util.create_if_not_exists()
- 05:25 PM Revision 13469: fix: schemas/util.sql: show_create_view(): removed extra trailing ; because pg_get_viewdef() now includes one
- 05:24 PM Revision 13468: schemas/util.sql: show_create_view(): made inclusion of `OR REPLACE` configurable
- 05:10 PM Revision 13467: schemas/util.sql: save_drop_view(): use util.drop_view()
- 03:40 PM Revision 13466: /README.TXT: Maintenance: added steps to back up the vegbiendev databases
- 02:15 PM Revision 13465: bugfix: lib/runscripts/util.run: run_args_cmd(): don't prepend main to args if no args, because for a non-runscript, all args will be passed to main(), leading `main` to be doubled
- 01:30 PM Revision 13464: lib/tnrs.py: use the TNRS dev server (with private URL in tnrs.url) instead of the live server, since that contains datasources that we need
- 01:29 PM Revision 13463: lib/streams.py: added file_get_contents()
- 01:14 PM Revision 13462: lib/tnrs.py: configure the server separately from the base URL
- 01:12 PM Revision 13461: lib/: svn:ignore tnrs.url so the TNRS dev server URL does not become public
05/16/2014
05/15/2014
- 11:57 PM Revision 13459: bugfix: web/.phpPgAdmin/.htaccess: custom pages: need to handle in query string because .php filename would be treated as path element
- 11:32 PM Revision 13458: web/.phpPgAdmin/.htaccess: support custom .php pages
- 10:30 AM Revision 13457: planning/meetings/BIEN conference call availability.xlsx: updated
- 09:43 AM Task #917: TNRS: Instructions for new version with TPL
- Aaron, for now (next couple of months, this must be done on the development TNRS web app since TPL is only on the dev...
Also available in: Atom