Project

General

Profile

Activity

From 12/13/2012 to 01/11/2013

01/11/2013

05:05 PM Revision 7205: README.TXT: Data import: Deleting imports before the last: Added instructions to keep a previous import instead of deleting it
Aaron Marcuse-Kubitza
04:22 PM Revision 7204: input.Makefile: Staging tables installation: $(logInstall): Always log the installation, regardless of the $log env var, because $log is set by default on development machines but an install log should still be created
Aaron Marcuse-Kubitza
01:03 PM Revision 7203: schemas/vegbien.ERD.mwb: Regenerated exports
Aaron Marcuse-Kubitza
10:19 AM Revision 7202: schemas/vegbien.sql: unscrubbed_taxondetermination_view: Fixed bug where need to handle the case where (SELECT source.source_id FROM source WHERE source.shortname = 'TNRS') is NULL because no TNRS names have been imported yet
Aaron Marcuse-Kubitza
09:44 AM Revision 7201: **/new_terms.csv, **/unmapped_terms.csv: Regenerated using `make missing_mappings`
Aaron Marcuse-Kubitza
09:19 AM Revision 7200: mappings/VegCore-VegBIEN.csv: morphoname: Remapped to the original rather than current taxondetermination because this is the *original* name applied by the author
Aaron Marcuse-Kubitza
09:16 AM Revision 7199: inputs/SALVIAS*/Organism/map.csv: Remapped voucher_string/coll_number to recordNumber instead of catalogNumber, because this number is actually applied by the collector rather than by a herbarium
Aaron Marcuse-Kubitza
09:11 AM Revision 7198: mappings/VegCore-VegBIEN.csv: Mapped recordNumber to new specimenreplicate.collectionnumber
Aaron Marcuse-Kubitza
09:02 AM Revision 7197: mappings/VegCore-VegBIEN.csv: Also map recordNumber (collectionnumber) to the indirect voucher's specimenreplicate
Aaron Marcuse-Kubitza
08:48 AM Revision 7196: inputs/*/*/map.csv: Remapped recordNumber to new individualCode where applicable
Aaron Marcuse-Kubitza
08:44 AM Revision 7195: mappings/VegCore-VegBIEN.csv: Mapped individualCode. authortaxoncode: Prefer tag over recordNumber (collectionnumber), because this applies to the plant rather than the specimen.
Aaron Marcuse-Kubitza
08:17 AM Revision 7194: mappings/VegCore-VegBIEN.csv: Mapped morphoname
Aaron Marcuse-Kubitza
08:16 AM Revision 7193: mappings/VegCore.csv: Regenerated from wiki
Aaron Marcuse-Kubitza
08:14 AM Revision 7192: mappings/VegCore.csv: Regenerated from wiki
Aaron Marcuse-Kubitza
08:04 AM Revision 7191: schemas/vegbien.sql: taxonverbatim: Added morphoname (which is different from the morphospecies suffix)
Aaron Marcuse-Kubitza
07:33 AM Revision 7190: schemas/vegbien.sql: plantobservation: Renamed collectionnumber to authorplantcode since this number, which identifies the *plant*, is actually different from the collectionnumber that identifies the *specimen* collected from it. This distinction is meaningful for plots data, but generally not for specimens data.
Aaron Marcuse-Kubitza
07:28 AM Revision 7189: schemas/vegbien.sql: plantobservation: Renamed collectionnumber to authorplantcode since this number, which identifies the *plant*, is actually different from the collectionnumber that identifies the *specimen* collected from it. This distinction is meaningful for plots data, but generally not for specimens data.
Aaron Marcuse-Kubitza
07:23 AM Revision 7188: schemas/vegbien.sql: specimenreplicate: Added collectionnumber
Aaron Marcuse-Kubitza
07:17 AM Revision 7187: schemas/vegbien.sql: taxonlabel: Removed no longer used matched_label_fit_fraction. Use taxondetermination.taxonfit instead.
Aaron Marcuse-Kubitza
07:02 AM Revision 7186: inputs/*/*/test.xml.ref: Restored inserted row counts, which had gotten auto-accepted from a test run on a non-empty DB
Aaron Marcuse-Kubitza
06:55 AM Revision 7185: schemas/vegbien.ERD.mwb: Expanded analytical_stem to fit the width of all fields
Aaron Marcuse-Kubitza
06:53 AM Revision 7184: schemas/vegbien.sql: taxondetermination: taxondetermination_computer_min_fit CHECK constraint: Fixed bug where need to use CASE instead of OR when a branch of an OR shouldn't be evaluated, because PostgreSQL doesn't support short-circuit OR
Aaron Marcuse-Kubitza
06:38 AM Revision 7183: README.TXT: Debugging: Added instructions for "binary chop" debugging, which requires syncing the DB schema to the svn working copy
Aaron Marcuse-Kubitza
06:08 AM Revision 7182: mappings/VegCore-VegBIEN.csv: Removed no longer used mappings for verbatimScientificName in _if conditions
Aaron Marcuse-Kubitza
06:08 AM Revision 7181: inputs/.NCBI/nodes/test.xml.ref: Restored inserted row counts, which had gotten auto-accepted from a test run on a non-empty DB
Aaron Marcuse-Kubitza
06:06 AM Revision 7180: sql_io.py: put_table(): DuplicateKeyException: Uniquifying input table to avoid internal duplicate keys: Also filter out duplicate rows in the out_table, so that they don't create duplicate key errors and the resulting index holes
Aaron Marcuse-Kubitza
06:01 AM Revision 7179: sql.py: distinct_table(): Added support for custom joins used in creating the new table. This can then be used by sql_io.put_table() to filter out duplicate rows in the out_table, so that they don't create duplicate key errors and the resulting index holes.
Aaron Marcuse-Kubitza
05:53 AM Revision 7178: README.TXT: Documentation: Redmine-formatted list of steps for column-based import: Added step to reinstall public schema first, to reset the sequences so that they don't create a diff when the new steps.by_col.log.sql is committed
Aaron Marcuse-Kubitza
05:48 AM Revision 7177: Added inputs/ACAD/Specimen/logs/steps.by_col.log.sql
Aaron Marcuse-Kubitza
05:45 AM Revision 7176: sql_gen.py: Join: Added support for mapping values which are lists, for use in USING joins
Aaron Marcuse-Kubitza
05:40 AM Revision 7175: inputs/SALVIAS/*/test.xml.ref: Restored SALVIAS* inserted row counts, which had gotten auto-accepted from a test run on a non-empty DB
Aaron Marcuse-Kubitza
05:01 AM Revision 7174: schemas/vegbien.sql: analytical_stem: Added locationName (authorPlotCode), subplot, individualCode (authorPlantCode) for use in validation
Aaron Marcuse-Kubitza
04:57 AM Revision 7173: schemas/vegbien.sql: sync_analytical_stem_to_view(): Drop and re-create dependent objects to avoid errors that analytical_stem can't be dropped because of dependents
Aaron Marcuse-Kubitza
04:56 AM Revision 7172: schemas/vegbien.sql: sync_analytical_stem_to_view(): Changed to PL/pgSQL function to allow adding PL/pgSQL commands
Aaron Marcuse-Kubitza
03:26 AM Revision 7171: schemas/vegbien.ERD.mwb: Moved family_higher_plant_group to leave room for analytical_stem to expand
Aaron Marcuse-Kubitza
03:08 AM Revision 7170: mappings/VegCore-VegBIEN.csv: Removed no longer used mappings for verbatimScientificName in _if conditions
Aaron Marcuse-Kubitza
02:59 AM Revision 7169: mappings/VegCore-VegBIEN.csv: Removed taxonlabel for original taxondetermination, because the original taxondetermination is not scrubbed by scrub.make (only the most current taxondetermination gets scrubbed, because only a single scrubbed determination is added by scrub.make). This still leaves the original taxondetermination's taxonverbatim, which stores the taxonomic information for historical purposes.
Aaron Marcuse-Kubitza
02:44 AM Revision 7168: mappings/VegCore-VegBIEN.csv: Removed no longer used accepted and verbatim (parsed) taxonlabels, which have been replaced by a single accepted or matched taxondetermination created by scrub.make
Aaron Marcuse-Kubitza
02:34 AM Revision 7167: Removed no longer used inputs/.TNRS/tnrs_accepted, tnrs_other. Use the tnrs_canon view instead.
Aaron Marcuse-Kubitza
02:22 AM Revision 7166: Removed no longer used inputs/.TNRS/tnrs_accepted, tnrs_other. Use the tnrs_canon view instead.
Aaron Marcuse-Kubitza
02:18 AM Revision 7165: Added inputs/.TNRS/_archive/
Aaron Marcuse-Kubitza
02:18 AM Revision 7164: Added inputs/.TNRS/tnrs/cleanup.sql to prevent running the default cleanup operations, which don't work on tables which have views referencing them (as is the case for tnrs, which is referenced by tnrs_canon)
Aaron Marcuse-Kubitza
02:07 AM Revision 7163: import_all: Removed no longer needed TNRS import, which has been replaced by scrub.make (which adds TNRS taxondeterminations after the import instead of creating taxonlabel links before it)
Aaron Marcuse-Kubitza
02:03 AM Revision 7162: mappings/VegCore-VegBIEN.csv: Removed TNRS input taxonlabels meant to cross-link to taxonlabels added by the TNRS import, because TNRS taxondeterminations are now created instead
Aaron Marcuse-Kubitza
01:42 AM Revision 7161: schemas/vegbien.sql: analytical_stem_view: Use just the main taxonlabel created by scrub.make instead of all the additional taxonlabels created by the TNRS import
Aaron Marcuse-Kubitza
01:11 AM Revision 7160: mappings/VegCore-VegBIEN.csv: main taxonverbatim.morphospecies "if has verbatim name" condition: Fixed bug where need to remove the taxonIsCanonical flag, because the TNRS.public.unscrubbed_taxondetermination_view table (which uses this flag) *should* include this field (although not other places where the morphospecies is stored by other TNRS tables)
Aaron Marcuse-Kubitza
12:49 AM Revision 7159: schemas/vegbien.sql: taxondetermination: taxondetermination_set_iscurrent() trigger: Also run on delete, to mark another taxondetermination as the current one when a current taxondetermination is deleted
Aaron Marcuse-Kubitza
12:18 AM Revision 7158: inputs/.TNRS/schema.sql: tnrs_canon: Annotations: Always use value from the matched name, because the accepted name does not have this
Aaron Marcuse-Kubitza
12:05 AM Revision 7157: mappings/VegCore-VegBIEN.csv: primary taxonlabel's parent taxonlabel: Fixed bug where a taxonverbatim was incorrectly being created solely to store the taxonRank, even though it was already stored in the taxonlabel's rank field
Aaron Marcuse-Kubitza

01/10/2013

11:52 PM Revision 7156: mappings/VegCore-VegBIEN.csv: Don't map morphospecies to the parsed taxonlabel's taxonepithet, because this causes an extra, parsed taxonlabel to be created for TNRS.public.unscrubbed_taxondetermination_view. It is not needed by the other TNRS tables.
Aaron Marcuse-Kubitza
11:45 PM Revision 7155: inputs/.TNRS/public.unscrubbed_taxondetermination_view/map.csv: Omit Infraspecific_rank to help avoid creating a separate, parsed taxonlabel. Don't map to taxonRank because Name_matched_rank is populated more often.
Aaron Marcuse-Kubitza
11:34 PM Revision 7154: inputs/.TNRS/public.unscrubbed_taxondetermination_view/scrub.make: Reduced $maxPause to 4 hr, because new taxondeterminations are being added throughout the import, so it is unlikely that more than more than 4 hr would pass between successive imports of taxondeterminations (causing scrub.make to stop prematurely)
Aaron Marcuse-Kubitza
11:23 PM Revision 7153: inputs/.TNRS/schema.sql: Removed no longer used tnrs+accepted. Use tnrs_canon or a self-join of tnrs instead
Aaron Marcuse-Kubitza
11:22 PM Revision 7152: schemas/vegbien.sql: tnrs_input_name: Use TNRS.tnrs directly instead of the now-deprecated tnrs+accepted
Aaron Marcuse-Kubitza
11:12 PM Revision 7151: schemas/vegbien.sql: Use new TNRS.tnrs_canon instead of tnrs+accepted to avoid creating additional taxonlabels for the parsed, matched, and accepted names and instead just use the most-canonicalized name of the names output by TNRS (the accepted name if available, or the matched name otherwise)
Aaron Marcuse-Kubitza
10:50 PM Revision 7150: mappings/VegCore-VegBIEN.csv: "if has verbatim name" _if statements that filter something out for TNRS mappings: Also assume true if taxonIsCanonical is specified, because some TNRS tables (eventually such as public.unscrubbed_taxondetermination_view) do not specify a separate "verbatim" taxondetermination but do provide taxonIsCanonical as a flag to turn various mappings on and off
Aaron Marcuse-Kubitza
09:06 PM Revision 7149: mappings/VegCore-VegBIEN.csv: Remapped matched*Fit_fraction to taxondetermination.taxonfit when a taxondetermination, not just a taxonlabel, is provided
Aaron Marcuse-Kubitza
09:03 PM Revision 7148: bin/map: map_table(): Resolving prefixes: Fixed bug where need to use list instead of tuple for metadata value mappings
Aaron Marcuse-Kubitza
08:16 PM Revision 7147: schemas/vegbien.sql: taxondetermination: Added CHECK constraint to allow only taxondeterminations with a minimum fit fraction of 80%, analogous to taxonlabel's taxonlabel_1_matched_label_min_fit() trigger
Aaron Marcuse-Kubitza

01/09/2013

05:34 PM Revision 7146: mappings/VegCore-VegBIEN.csv: Don't create a separate TNRS input taxonlabel if taxonIsCanonical exists
Aaron Marcuse-Kubitza
05:24 PM Revision 7145: inputs/.TNRS/schema.sql: tnrs_canon: Fixed bug where need to always use Unmatched_terms from tnrs rather than tnrs_accepted
Aaron Marcuse-Kubitza
05:07 PM Revision 7144: inputs/.TNRS/schema.sql: Added tnrs_canon, which stores the most canonicalized name output by TNRS
Aaron Marcuse-Kubitza
04:17 PM Revision 7143: schemas/vegbien.sql: analytical_stem_view: accepted_taxonverbatim: Fixed bug where need to join only to the taxonverbatim whose morphospecies is NULL, to avoid joining to multiple taxonverbatims at once. This extra filter is now needed because there can be multiple taxonverbatims for a taxonlabel with different morphospecies.
Aaron Marcuse-Kubitza
03:59 PM Revision 7142: mappings/VegCore-VegBIEN.csv: taxonlabel.taxonomicname: Prepend the family to the rest of the name using new _merge_prefix() instead of _join_words()/_nullIf(), so that any input taxonomic name that includes the family will not have the family duplicated in the combined taxonomic name. Previously, the duplication was removed only when the rest of the input name was *equal to* the family. This change fixes a bug in the new TNRS import where a pre-concatenated taxonomic name (Accepted_scientific_name) which includes the family is now used instead of Accepted_name, which only includes it when it's equal to the family.
Aaron Marcuse-Kubitza
03:52 PM Revision 7141: xml_func.py: Simplifying functions: Merging: Added _merge_prefix() passthru
Aaron Marcuse-Kubitza
03:33 PM Revision 7140: schemas/functions.sql: Added _merge_prefix()
Aaron Marcuse-Kubitza
02:42 PM Revision 7139: inputs/.TNRS/schema.sql: tnrs_populate_accepted_scientific_name(): Fixed bug where Accepted_name_family shouldn't be prefixed to Accepted_name if Accepted_name is itself the family, to avoid duplicating the family in the Accepted_scientific_name
Aaron Marcuse-Kubitza
02:18 PM Revision 7138: inputs/.TNRS/schema.sql: tnrs+accepted: Added new Accepted_scientific_name column and mapped it in public.unscrubbed_taxondetermination_view
Aaron Marcuse-Kubitza
11:06 AM Revision 7137: schemas/vegbien.sql: tnrs_input_name: Fixed bug where need to filter out tnrs+accepted rows with NULL Accepted_scientific_name, because inputs to tnrs_db must be strings
Aaron Marcuse-Kubitza
10:53 AM Revision 7136: schemas/vegbien.sql: tnrs_input_name: Prepend TNRS accepted names that have not yet been parsed. This allows parsing TNRS accepted names without first needing to import them into taxonlabels, which may not occur until the next import.
Aaron Marcuse-Kubitza
10:09 AM Revision 7135: inputs/.TNRS/schema.sql: tnrs+accepted: Use new Accepted_scientific_name to join to tnrs_accepted.Name_submitted
Aaron Marcuse-Kubitza
10:05 AM Revision 7134: inputs/.TNRS/schema.sql: tnrs: Added tnrs_populate_accepted_scientific_name() trigger
Aaron Marcuse-Kubitza
09:57 AM Revision 7133: inputs/.TNRS/schema.sql: tnrs: Added Accepted_scientific_name field which will contain the joined-together accepted name that gets re-parsed by TNRS
Aaron Marcuse-Kubitza
09:13 AM Revision 7132: inputs/.TNRS/: Changed tnrs+accepted to a view (defined in schema.sql) so accepted names would automatically be populated as they are parsed by TNRS, rather than needing to run `make inputs/.TNRS/tnrs+accepted/reinstall` to populate them
Aaron Marcuse-Kubitza
08:16 AM Revision 7131: mappings/VegCore-VegBIEN.csv: Also map the morphospecies to the accepted taxonverbatim when an accepted name is provided
Aaron Marcuse-Kubitza
08:01 AM Revision 7130: schemas/vegbien.sql: taxonverbatim: taxonverbatim_unique: Added morphospecies so that there can be multiple taxonverbatims for the same taxonlabel, each with different morphospecies suffixes
Aaron Marcuse-Kubitza
04:17 AM Revision 7129: inputs/.TNRS/public.unscrubbed_taxondetermination_view/map.csv: Mapped Accepted_name.*
Aaron Marcuse-Kubitza
03:02 AM Revision 7128: schemas/vegbien.sql: unscrubbed_taxondetermination_view: Use new tnrs+accepted instead of tnrs so that the accepted name can be imported at the same time
Aaron Marcuse-Kubitza
02:23 AM Revision 7127: import_all: Reinstall tnrs+accepted, for eventual use by unscrubbed_taxondetermination_view
Aaron Marcuse-Kubitza
02:20 AM Revision 7126: Added inputs/.TNRS/tnrs+accepted/, which self-joins the TNRS results to their parsed accepted names
Aaron Marcuse-Kubitza
02:02 AM Revision 7125: import_all: Directly import just the TNRS tables that should be imported, because some TNRS tables are included in import_order.txt so that they are part of the automated testing, but should not be imported at the same time as tnrs_accepted/tnrs_other
Aaron Marcuse-Kubitza
12:45 AM Revision 7124: inputs/import.stats.xls: Updated import times
Aaron Marcuse-Kubitza

01/08/2013

11:24 PM Revision 7123: with_all: $all mode: Fixed bug where need " " before # for it to be interpreted as a comment (unlike in a Makefile, where the " " often needs to be left out to avoid it being treated as part of a variable value)
Aaron Marcuse-Kubitza
10:55 PM Revision 7122: bin/map: Made $redo flag default to off, because redo mode is slow (all tables have to be truncated) and is only needed when running tests on a public schema with data in it, which would not be the case on a development machine where tests are usually run
Aaron Marcuse-Kubitza
10:19 PM Revision 7121: import_all: Made temporary vars local, so they wouldn't affect the calling shell
Aaron Marcuse-Kubitza
09:45 PM Revision 7120: schemas/vegbien.sql: unscrubbed_taxondetermination_view: Sort by taxondetermination.taxonoccurrence_id instead of taxondetermination_id to allow scanning the taxondetermination_single_current_determination index, which includes only current determinations and avoids needing to scan past many non-current determinations. Note that using taxonoccurrence_id does not create sort order ambiguity between taxondeterminations with the same taxonoccurrence_id, because there is only one current determination per taxonoccurrence.
Aaron Marcuse-Kubitza
09:32 PM Revision 7119: schemas/vegbien.sql: unscrubbed_taxondetermination_view: Inner-join to taxonverbatim and taxonlabel instead of LEFT JOINing, because only taxondeterminations with a taxonlabel can have accepted taxondeterminations (otherwise there would be no name to scrub)
Aaron Marcuse-Kubitza
09:30 PM Revision 7118: schemas/vegbien.sql: unscrubbed_taxondetermination_view: Inner-join to tnrs instead of LEFT JOINing, because only taxondeterminations whose taxonlabels have already been scrubbed by TNRS should have accepted taxondeterminations added. Removed now-unneeded filter by tnrs.Name_submitted IS NOT NULL, which is replaced by the inner join.
Aaron Marcuse-Kubitza
08:46 PM Revision 7117: sql_io.py: put_table(): ensure_cond(): Fixed bug where need to wrap strings used in the tracked error message in strings.ustr()
Aaron Marcuse-Kubitza
08:33 PM Revision 7116: xml_dom.py: replace_with_text(): Fixed bug where need to use scalar.is_nonnull_scalar() instead of is_scalar() to avoid converting None values to the string 'None'
Aaron Marcuse-Kubitza
08:32 PM Revision 7115: scalar.py: Added is_nonnull_scalar()
Aaron Marcuse-Kubitza

01/07/2013

08:17 PM Revision 7114: README.TXT: Data import: Fixed bug where `make inputs/upload` needs to be run on local machine, not vegbiendev
Aaron Marcuse-Kubitza
08:16 PM Revision 7113: sql.py: create_table(): Support creating a table like a view
Aaron Marcuse-Kubitza
08:04 PM Revision 7112: sql.py: Added InvalidTypeException and parse it in parse_exception()
Aaron Marcuse-Kubitza
07:39 PM Revision 7111: mappings/VegCore.csv: Regenerated from wiki
Aaron Marcuse-Kubitza
07:34 PM Revision 7110: schemas/vegbien.sql: taxondetermination_set_iscurrent(): Fixed bug where need to sort scrubbed determinations first for scrub.make to work. (Otherwise, a datasource determination might remain iscurrent even after a scrubbed determination was added, causing scrub.make to repeatedly attempt to re-add it.)
Aaron Marcuse-Kubitza
07:20 PM Revision 7109: inputs/.TNRS/public.unscrubbed_taxondetermination_view/map.csv: Set dateIdentified to _now()
Aaron Marcuse-Kubitza
07:20 PM Revision 7108: inputs/.TNRS/public.unscrubbed_taxondetermination_view/scrub.make: Unset $n to avoid limiting the # rows/iteration
Aaron Marcuse-Kubitza
07:15 PM Revision 7107: schemas/py_functions.sql: parse_date_range(): Don't parse strings containing a time, because - and ' ' don't have the same meaning as in a date range
Aaron Marcuse-Kubitza
07:03 PM Revision 7106: xml_dom.py: replace_with_text(): Support any scalar type recognized by scalar.is_scalar()
Aaron Marcuse-Kubitza
06:54 PM Revision 7105: scalar.py: is_scalar(): Added datetime.datetime
Aaron Marcuse-Kubitza
06:43 PM Revision 7104: schemas/functions.sql: Added _now()
Aaron Marcuse-Kubitza
06:39 PM Revision 7103: import_all: Make $dump_opts, $public_import local vars, so they will be automatically unset if the script is aborted
Aaron Marcuse-Kubitza
06:31 PM Revision 7102: mappings/VegCore-VegBIEN.csv: identificationType: Fixed bug in mapping where extra *_id/ needed to be removed
Aaron Marcuse-Kubitza
06:25 PM Revision 7101: inputs/.TNRS/public.unscrubbed_taxondetermination_view/map.csv: Set taxonOccurrenceID to dummy value 0 to enable the taxonoccurrence CHECK constraint to pass. This is needed because the constraint must pass before the pkey (which should already exist) is even checked.
Aaron Marcuse-Kubitza
06:19 PM Revision 7100: inputs/.TNRS/public.unscrubbed_taxondetermination_view/map.csv: Set identificationType to computer
Aaron Marcuse-Kubitza
06:18 PM Revision 7099: mappings/VegCore-VegBIEN.csv: Mapped identificationType
Aaron Marcuse-Kubitza
06:15 PM Revision 7098: mappings/VegCore.csv: Regenerated from wiki
Aaron Marcuse-Kubitza
05:39 PM Revision 7097: schemas/vegbien.sql: unscrubbed_taxondetermination_view: Use `SELECT source_id FROM source WHERE shortname = ...` instead of source_by_shortname() so that the source table is updated to point to the same schema as the view rather than pointing to whichever version (usually public) is first in the search_path
Aaron Marcuse-Kubitza
05:23 PM Revision 7096: schemas/vegbien.sql: unscrubbed_taxondetermination_view: Fixed bug where need to include only those taxondeterminations that already have a match in TNRS.tnrs, to avoid adding empty TNRS taxondeterminations. As the concurrent tnrs daemon runs, these taxondeterminations will gradually acquire matches in tnrs and then be processed by scrub.
Aaron Marcuse-Kubitza
05:00 PM Revision 7095: import_all: Make $import_source a local var, so it will be automatically unset if the script is aborted
Aaron Marcuse-Kubitza
04:49 PM Revision 7094: vegbien_dest: Schema override for referring to a table in the $public schema: Only process the override when $!schemaVar and $!tableVar are non-*empty*, to allow setting $schema=""
Aaron Marcuse-Kubitza
04:47 PM Revision 7093: schemas/Makefile: DDL generation: vegbien.sql: Unset $dump_opts so that pg_dump does not use env vars left after running import_all
Aaron Marcuse-Kubitza
04:44 PM Revision 7092: schemas/Makefile: DDL generation: vegbien.sql: Unset $version so that pg_dump always uses the public schema, even after running import_all
Aaron Marcuse-Kubitza
04:13 PM Revision 7091: mappings/VegCore.csv: Regenerated from wiki
Aaron Marcuse-Kubitza
04:13 PM Revision 7090: README.TXT: Testing: Added commands to put in .profile on a development machine
Aaron Marcuse-Kubitza
04:10 PM Revision 7089: import_all: Added command to add scrubbed taxondeterminations
Aaron Marcuse-Kubitza
04:09 PM Revision 7088: mappings/VegCore.csv: Regenerated from wiki
Aaron Marcuse-Kubitza
04:08 PM Revision 7087: import_all: Start tnrs-remake *after* starting the inputs, so that for subset imports (e.g. n=2), there will already be names to scrub when tnrs-remake starts up and it won't enter pause mode to wait for new rows (the pause is calibrated for full imports, and is too long for subset imports)
Aaron Marcuse-Kubitza
04:01 PM Revision 7086: with_all: Also exclude .archive/ from the subdirs to forward commands to
Aaron Marcuse-Kubitza
03:40 PM Revision 7085: inputs/.TNRS/public.unscrubbed_taxondetermination_view/scrub.make: Added option to wait for new rows, in the same way tnrs_db does
Aaron Marcuse-Kubitza
03:38 PM Revision 7084: inputs/.TNRS/public.unscrubbed_taxondetermination_view/scrub.make: Factored new rows added test out into rowsAdded() function
Aaron Marcuse-Kubitza
03:09 PM Revision 7083: Added inputs/.TNRS/public.unscrubbed_taxondetermination_view/scrub.make, which adds scrubbed taxondeterminations to VegBIEN
Aaron Marcuse-Kubitza
02:00 PM Revision 7082: root Makefile: Removed $(subMake), which is now defined properly by lib/common.Makefile
Aaron Marcuse-Kubitza
01:59 PM Revision 7081: lib/common.Makefile: $(subMake): Removed `--makefile=../input.Makefile`, which is specific just to inputs/Makefile
Aaron Marcuse-Kubitza
01:43 PM Revision 7080: input.Makefile: Import to VegBIEN: $(import): Print the date at the beginning of the import, so successive imports to the same version can be distinguished
Aaron Marcuse-Kubitza
01:37 PM Revision 7079: input.Makefile: Import to VegBIEN: \$(import): Fixed bug where 2>&1 needs to come after >>$(log_) rather than before
Aaron Marcuse-Kubitza
01:35 PM Revision 7078: inputs/.TNRS/tnrs/tnrs.make: Usage: Added tnrs_db's $wait flag
Aaron Marcuse-Kubitza
01:34 PM Revision 7077: inputs/.TNRS/tnrs/tnrs.make: Fixed Usage message to use make, which is needed to set the PATH correctly
Aaron Marcuse-Kubitza
11:47 AM Revision 7076: Makefiles: Changed "Usage: `make -s ...`" to "Run with `make -s` to avoid echoing make commands"
Aaron Marcuse-Kubitza
11:44 AM Revision 7075: input.Makefile: Import to VegBIEN: Added %/log_file to view the import log file path
Aaron Marcuse-Kubitza
11:28 AM Revision 7074: input.Makefile: Import to VegBIEN: $(import): Append to the log file instead of replacing it, to avoid overwriting the log for a previous import to the same versioned schema. This allows a datasource to be (re-)imported multiple times, and is needed by the new method for linking taxonoccurrences to scrubbed taxonomic names.
Aaron Marcuse-Kubitza
11:22 AM Revision 7073: input.Makefile: Import to VegBIEN: $(import): Always output just to log file if $(log) is on, rather than also copying output to the terminal when $(n) is set. When $(log) is on, the output can still be viewed by tailing the log.
Aaron Marcuse-Kubitza
11:16 AM Revision 7072: input.Makefile: Import to VegBIEN: $(import): Merged consecutive $(if $(n),...)
Aaron Marcuse-Kubitza
11:14 AM Revision 7071: input.Makefile: Import to VegBIEN: $(import): Merged consecutive $(if $(log),...)
Aaron Marcuse-Kubitza
11:07 AM Revision 7070: Added inputs/.TNRS/public.unscrubbed_taxondetermination_view/
Aaron Marcuse-Kubitza
11:05 AM Revision 7069: mappings/VegCore-VegBIEN.csv: Mapped taxonOccurrencePkey
Aaron Marcuse-Kubitza
10:58 AM Revision 7068: mappings/VegCore.csv: Regenerated from wiki
Aaron Marcuse-Kubitza
10:41 AM Revision 7067: input.Makefile: Staging tables installation: Added %_view/install, to prevent trying to edit a view during installation
Aaron Marcuse-Kubitza
10:31 AM Revision 7066: vegbien_dest: Added schema override support for referring to a table in the $public schema
Aaron Marcuse-Kubitza
10:29 AM Revision 7065: input.Makefile: Staging tables installation: $(cleanup): Moved setting of $schema, $table before vegbien_dest is run, so it can modify them if needed
Aaron Marcuse-Kubitza
09:42 AM Revision 7064: inputs/.TNRS/tnrs/tnrs.make: Removed unnecessary setting of $prefix, which now defaults to ""
Aaron Marcuse-Kubitza
09:40 AM Revision 7063: schemas/vegbien.sql: Added unscrubbed_taxondetermination_view
Aaron Marcuse-Kubitza

01/04/2013

10:10 PM Revision 7062: inputs/import.stats.xls: Moved CTFS to Deleted section
Aaron Marcuse-Kubitza
10:03 PM Revision 7061: make_analytical_db: ANALYZE each table after its created so that queries use index scans instead of seq scans
Aaron Marcuse-Kubitza
09:40 PM Revision 7060: schemas/vegbien.sql: sync_analytical_*_to_view(): Added datasource fkey to source.shortname so removing a datasource will also remove the corresponding rows in the analytical views
Aaron Marcuse-Kubitza
09:36 PM Revision 7059: schemas/vegbien.sql: Regenerated analytical_stem using sync_analytical_stem_to_view()
Aaron Marcuse-Kubitza
09:19 PM Revision 7058: input.Makefile: Editing import: rm: Time the command
Aaron Marcuse-Kubitza
09:15 PM Task #549 (Resolved): add covering indexes on fkeys
Aaron Marcuse-Kubitza
07:09 PM Task #549: add covering indexes on fkeys
Added for direct fkeys which are in use Aaron Marcuse-Kubitza
05:56 PM Task #549 (Resolved): add covering indexes on fkeys
* use instructions for *[[Postgres queries#Adding covering indexes on foreign keys|adding covering indexes on foreign... Aaron Marcuse-Kubitza
09:14 PM Revision 7057: schemas/vegbien.sql: Added covering indexes where needed, as described at <https://projects.nceas.ucsb.edu/nceas/issues/549>
Aaron Marcuse-Kubitza
09:11 PM Revision 7056: schemas/vegbien.sql: Fixed fkey constraint names
Aaron Marcuse-Kubitza
09:09 PM Revision 7055: schemas/vegbien.sql: Added covering indexes where needed, as described at <https://projects.nceas.ucsb.edu/nceas/issues/549>
Aaron Marcuse-Kubitza
06:59 PM Revision 7054: schemas/vegbien.sql: fkeys to source: Added covering indexes where needed, as described at <https://projects.nceas.ucsb.edu/nceas/issues/549>
Aaron Marcuse-Kubitza
06:22 PM Revision 7053: schemas/vegbien.sql: commconcept: Renamed source_id back to reference_id (it was previously renamed to source_id in a bulk rename)
Aaron Marcuse-Kubitza
06:20 PM Revision 7052: schemas/vegbien.sql: taxondetermination: Added back reference_id, which is different than the scoping source_id (reference_id was previously renamed to source_id in a bulk rename)
Aaron Marcuse-Kubitza
06:04 PM Revision 7051: schemas/vegbien.sql: Renamed taxonconcept_concept_source_id_fkey back to taxonconcept_concept_reference_id_fkey
Aaron Marcuse-Kubitza
06:02 PM Revision 7050: schemas/vegbien.sql: Renamed *_reference_id_fkey fkeys to *_source_id_fkey
Aaron Marcuse-Kubitza
05:32 PM Revision 7049: inputs/CTFS/_no_import: Temporarily remove CTFS from the public DB per Rick Condit's request (due to validation issues)
Aaron Marcuse-Kubitza
05:25 PM Revision 7048: import_all: Run import with $public_import set in order to exclude excluded datasources
Aaron Marcuse-Kubitza
05:23 PM Revision 7047: input.Makefile: Import to VegBIEN: %/import: Don't run the import if $public_import flag is set and the datasource contains a _no_import file. This allows just excluding a datasource from the public DB, without also removing it from automated testing.
Aaron Marcuse-Kubitza
05:17 PM Revision 7046: lib/common.Makefile: Added $(and), $(or), $(not)
Aaron Marcuse-Kubitza
04:30 PM Revision 7045: mappings/VegCore.csv: Regenerated from wiki
Aaron Marcuse-Kubitza
04:12 PM Revision 7044: schemas/vegbien.sql: taxondetermination: Added scoping source_id field to allow other datasources (e.g. TNRS) to make taxondeterminations. (Repurposed existing non-scoping source_id.)
Aaron Marcuse-Kubitza

01/03/2013

08:25 AM Revision 7043: make_analytical_db: Fixed bug where can't give public_ select access to all analytical_db views because this apparently adds access rather than passing through the underlying table's permissions
Aaron Marcuse-Kubitza
08:18 AM Revision 7042: make_analytical_db: Give public_ select access to analytical_db views. This causes the actual access to depend on the underlying table's permissions.
Aaron Marcuse-Kubitza
07:43 AM Revision 7041: mappings/VegCore.csv: Regenerated from wiki
Aaron Marcuse-Kubitza
07:39 AM Revision 7040: mappings/Makefile: VegCore.csv: Include only terms that start with a lowercase letter or are all caps. This also avoids the need to filter out VegCore.tables.csv.
Aaron Marcuse-Kubitza
07:31 AM Revision 7039: mappings/VegCore.csv: Changed line endings to \n in preparation for not running filter_out_cs on the file (which changes line endings to \r\n)
Aaron Marcuse-Kubitza
02:31 AM Revision 7038: import_all: `make backups/vegbien.$version.backup/test`: Documented that this uses $dump_opts. $dump_opts must be manually set when running this command outside of import_all.
Aaron Marcuse-Kubitza
02:21 AM Revision 7037: backups/Makefile: Synchronization: %/download: Download the .md5 file first, so that the user is prompted right away for their password rather than after the main file has finished downloading, by which time the password prompt has timed out
Aaron Marcuse-Kubitza
12:02 AM Revision 7036: mappings/VegCore.csv: Regenerated from wiki
Aaron Marcuse-Kubitza

01/02/2013

11:34 PM Revision 7035: mappings/VegCore.csv: Regenerated from wiki
Aaron Marcuse-Kubitza
11:32 PM Revision 7034: mappings/Makefile: VegCore.csv: Fixed bug where need to filter out VegCore.tables.csv case-sensitively so that field names which are the same as a table name don't get filtered out
Aaron Marcuse-Kubitza
11:23 PM Revision 7033: Added filter_out_cs
Aaron Marcuse-Kubitza
09:21 PM Revision 7032: README.TXT: Data import: Added step to ensure there are no local modifications using `svn st`
Aaron Marcuse-Kubitza
08:56 PM Revision 7031: make_analytical_db: Also grant USAGE on the analytical_db schema itself to bien_read, public_
Aaron Marcuse-Kubitza
07:14 PM Revision 7030: README.TXT: Data import: After import: Also check that the provider_count table contains entries for all inputs
Aaron Marcuse-Kubitza
07:03 PM Revision 7029: Added inputs/.geoscrub/_src/geovalidity-table.txt, which was attached to Jim's geovalidation e-mail (provided in README.TXT)
Aaron Marcuse-Kubitza
06:43 PM Revision 7028: inputs/import.stats.xls: Updated import times
Aaron Marcuse-Kubitza
06:39 PM Revision 7027: README.TXT: Data import: recording the import times in inputs/import.stats.xls: Updated column group header to "By column"
Aaron Marcuse-Kubitza
06:36 PM Revision 7026: backups/Makefile: Removed no longer used $(psqlVerbose)
Aaron Marcuse-Kubitza
06:36 PM Revision 7025: backups/Makefile: Removed %.backup/rm_indexes, which is no longer needed because archived imports are now backed up instead of being stored without indexes in the live DB
Aaron Marcuse-Kubitza
06:31 PM Revision 7024: backups/Makefile: %.backup/remove: Fixed bug where need to use $no_search_path option to psql_script_vegbien
Aaron Marcuse-Kubitza

12/21/2012

03:34 PM Revision 7023: import_all: Allow caller to override $dump_opts
Aaron Marcuse-Kubitza
03:33 PM Revision 7022: pg_dump_vegbien: Renamed $opts env var to $dump_opts to avoid conflicting with other commands' vars of the same name
Aaron Marcuse-Kubitza
03:22 PM Revision 7021: mappings/VegCore.csv: Regenerated from wiki
Aaron Marcuse-Kubitza
03:20 PM Revision 7020: mappings/VegCore.csv: Regenerated from wiki
Aaron Marcuse-Kubitza
02:14 PM Revision 7019: schemas/vegbien.sql: location: Removed location_unique_within_parent_by_sourceaccessioncode, which duplicates location_unique_within_creator_by_sourceaccessioncode because the sourceaccessioncode is globally unique
Aaron Marcuse-Kubitza
02:10 PM Revision 7018: schemas/vegbien.sql: analytical_stem_view: projectID: Use project.projectname if project.sourceaccessioncode isn't provided
Aaron Marcuse-Kubitza
02:02 PM Revision 7017: schemas/vegbien.sql: location: location_unique_within_parent: Split into *_by_sourceaccessioncode and *_by_authorlocationcode_position, with each ID being matched separately. This way, if the initial import of a subplot's location provides both fields, but fkey references use only one field, the fkey references will still match the existing location because only one of the fields needs to match.
Aaron Marcuse-Kubitza
01:26 PM Revision 7016: schemas/vegbien.sql: analytical_stem_view: elevationInMeters: Use parent_location.elevation_m when location.elevation_m not provided
Aaron Marcuse-Kubitza
01:17 PM Revision 7015: schemas/vegbien.sql: analytical_stem_view: scientificName: Fixed bug where need to use accepted_taxon*label*.taxonomicname instead of accepted_taxonverbatim.taxonomicname, because taxonverbatim's name component fields aren't populated if the name doesn't match a scrubbed name. The datasource's own taxonverbatim can't be used for this because the canon_label_id refers to the concatenated taxonomic name owned by the TNRS datasource.
Aaron Marcuse-Kubitza
01:00 PM Revision 7014: inputs/NVS/Plot/map.csv: Corrected Plot ID mapping to go to subplotID instead of locationID, because each subplot gets its own ID in this field
Aaron Marcuse-Kubitza
12:50 PM Revision 7013: schemas/vegbien.sql: location: location_unique_within_parent: Also apply this constraint when sourceaccessioncode is provided, because it may be a concatenated value populated for use by the analytical DB but which is not used as an fkey by the datasource itself
Aaron Marcuse-Kubitza
12:30 PM Revision 7012: schemas/vegbien.sql: analytical_stem_view: locationID: Concatenate parent location's and subplot's IDs using '; ' instead of ' '
Aaron Marcuse-Kubitza
12:22 PM Revision 7011: schemas/vegbien.sql: analytical_*: Renamed locationName to locationID because it's now globally unique (within the datasource) and can be used as a sourceaccessioncode
Aaron Marcuse-Kubitza
12:19 PM Revision 7010: schemas/vegbien.sql: analytical_stem_view: locationName: For subplots without their own sourceaccessioncode (globally unique ID), prepend the parent location's unique ID so that locationName is globally unique
Aaron Marcuse-Kubitza
12:07 PM Revision 7009: mappings/VegCore-VegBIEN.csv: locationID/locationName + subplot -> location.sourceaccessioncode mapping: Fixed bug where subplot was incorrectly being mapped to this field even when there was no location*. (This field can only be populated if both location* *and* subplot are specified.) Also only map locationID for this, to avoid inconsistencies where one table supplies locationID+subplot, while another table supplies locationName+subplot, but they both get mapped to the same field, preventing plots from being matched up with their observations when creating the analytical_stem.
Aaron Marcuse-Kubitza
11:31 AM Revision 7008: xml_func.py: Simplifying functions: Logic: _and(), _or(): Evaluate an expression of only constant values
Aaron Marcuse-Kubitza
11:30 AM Revision 7007: lists.py: Added and_(), or_()
Aaron Marcuse-Kubitza
11:28 AM Revision 7006: xml_func.py: is_scalar(): Fixed bug where need to check if value is a string before calling is_var_name()
Aaron Marcuse-Kubitza
10:15 AM Revision 7005: inputs/NVS/StemObservation/map.csv: Remapped Verbatim Code to authorTaxonCode, because as it's used this is actually an identifier for the taxon, not the stem, despite Nick Spencer's revised mapping
Aaron Marcuse-Kubitza

12/20/2012

05:21 PM Revision 7004: schemas/vegbien.ERD.mwb: Regenerated exports
Aaron Marcuse-Kubitza
05:21 PM Revision 7003: README.TXT: Schema changes: Update graphical ERD exports: Added step to commit changes
Aaron Marcuse-Kubitza
05:02 PM Revision 7002: inputs/NVS/*/map.csv: Remapped with Nick Spencer's suggested changes
Aaron Marcuse-Kubitza
04:41 PM Revision 7001: xml_func.py: _first(): Fixed bug where need to choose the first *non-empty* param, by first pruning empty child nodes
Aaron Marcuse-Kubitza
04:38 PM Revision 7000: mappings/VegCore-VegBIEN.csv: authortaxoncode mappings: Only using authorTaxonCode if there is no plant ID: Added individualID, stemID to the terms that cause authorTaxonCode not to be mapped to VegBIEN authortaxoncode
Aaron Marcuse-Kubitza
04:03 PM Revision 6999: mappings/VegCore-VegBIEN.csv: authortaxoncode mappings: Only using authorTaxonCode if there is no plant ID: Added individualID, stemID to the terms that cause authorTaxonCode not to be mapped to VegBIEN authortaxoncode
Aaron Marcuse-Kubitza
03:59 PM Revision 6998: schemas/vegbien.sql: analytical_*: Renamed individualID to individualObservationID because this actually corresponds to plantobservation.sourceaccessioncode, which is an observation *of* an individual
Aaron Marcuse-Kubitza
03:56 PM Revision 6997: mappings/VegCore.csv: Regenerated from wiki
Aaron Marcuse-Kubitza
03:53 PM Revision 6996: README.TXT: Data import: Recording the import times: Changed <version> back to $version because these commands are actually run on vegbiendev, where $version is set. (Modifications to import.stats.xls would be made on your local machine.)
Aaron Marcuse-Kubitza
03:50 PM Revision 6995: README.TXT: Data import: Added step to unset $version before starting the import, to avoid importing on top of the last import's data
Aaron Marcuse-Kubitza
02:47 PM Revision 6994: README.TXT: Data import: Replaced $version with <version> where it needs to be manually filled in
Aaron Marcuse-Kubitza
02:40 PM Revision 6993: README.TXT: Data import: On nimoy: Added command to set $version
Aaron Marcuse-Kubitza
02:36 PM Task #548 (New): remove benign errors from the data provider feedback tables
h3. Examples... Aaron Marcuse-Kubitza
02:26 PM Revision 6992: mappings/VegCore-VegBIEN.csv: authortaxoncode mappings: Only use authorTaxonCode if there is no plant ID, because an individual plant gets its own taxonoccurrence and thus needs the taxonoccurrence's IDs to be unique to the plant, regardless of what the author designates as the taxonoccurrence code
Aaron Marcuse-Kubitza
01:47 PM Revision 6991: Generated inputs/NVS/new_terms.csv
Aaron Marcuse-Kubitza
01:47 PM Revision 6990: input.Makefile: SVN: $(svnFilesGlob): Also match *terms.csv in top-level dir
Aaron Marcuse-Kubitza
01:23 PM Revision 6989: mappings/VegCore-VegBIEN.csv: Mapped authorTaxonCode
Aaron Marcuse-Kubitza
01:12 PM Revision 6988: mappings/VegCore.csv: Regenerated from wiki
Aaron Marcuse-Kubitza
01:12 PM Revision 6987: README.TXT: Maintenance: VegCore data dictionary: Added step to commit updated mappings/VegCore.csv
Aaron Marcuse-Kubitza
12:13 PM Revision 6986: schemas/Makefile: %/publish: Fixed bug where commands were not being run transactionally, because --single-transaction requires `--file -` to work properly
Aaron Marcuse-Kubitza
11:36 AM Revision 6985: input.Makefile: Editing import: Removed rotate because appending the current svn revision doesn't make sense, since this is not related to the revision used to import the datasource
Aaron Marcuse-Kubitza
11:34 AM Revision 6984: input.Makefile: Editing import: Added rename/% and use it in rotate
Aaron Marcuse-Kubitza
11:21 AM Revision 6983: inputs/import.stats.xls: Updated import times
Aaron Marcuse-Kubitza
11:21 AM Revision 6982: schemas/Makefile: Use $* instead of $(@D) for clarity. $(@D) is only needed when the dir part of the target includes a prefix in addition to the % stem.
Aaron Marcuse-Kubitza
10:45 AM Revision 6981: make_analytical_db: Automatically call export_analytical_db when finished
Aaron Marcuse-Kubitza
10:35 AM Revision 6980: schemas/vegbien.sql: make_family_higher_plant_group(): Added `taxonepithet IS NOT NULL` filter, to allow make_analytical_db to proceed even when the NCBI import fails (leaving some nodes with rank = 'family' but no associated taxonepithet). The most recent NCBI import failed due to the search_path/DuplicateException bug resulting from the import schema and public being in the search_path together.
Aaron Marcuse-Kubitza
10:14 AM Revision 6979: schemas/Makefile: Fixed bug where need `SHELL := /bin/bash` for \$(confirmRmPublicSchema) to work correctly
Aaron Marcuse-Kubitza
10:12 AM Revision 6978: lib/common.Makefile: $(confirm): Added comment that this requires `SHELL := /bin/bash` to work correctly
Aaron Marcuse-Kubitza
10:09 AM Revision 6977: import_all: after_import(): Added `make backups/vegbien.$version.backup/test`
Aaron Marcuse-Kubitza
10:05 AM Revision 6976: sql.py: DbConn._db(): search_path: Don't append the existing search_path, because it usually includes the public schema, which is now different from the schema being imported into. This fixes a bug where sql.function_exists() would find public-schema functions in both the public schema and the import's schema because both were in the search_path, causing a DuplicateException "more than one function named ...". Note that the elements of the existing search_path are no longer needed now that vegbien_dest's $schemas includes $public. Also note that if an instance of DbConn does not specify the schemas param, the existing search_path will be left as-is rather than overwritten with an empty list.
Aaron Marcuse-Kubitza
09:54 AM Revision 6975: README.TXT: Data import: recording the import times in inputs/import.stats.xls: Added step to determine the import date using import_date
Aaron Marcuse-Kubitza
09:52 AM Revision 6974: import_date: Added note that Mac and Linux differ in the order they sort the logs in
Aaron Marcuse-Kubitza
09:50 AM Revision 6973: README.TXT: Data import: recording the import times in inputs/import.stats.xls: Updated pattern for new log filename format
Aaron Marcuse-Kubitza
09:47 AM Revision 6972: README.TXT: Data import: recording the import times in inputs/import.stats.xls: Removed extra ./ before bin/import_times
Aaron Marcuse-Kubitza
09:46 AM Revision 6971: import_date: Added note that the time this outputs is the time the first special input *finished* importing. The import itself generally starts a few minutes before that, and the exact time is in that import's public schema comment.
Aaron Marcuse-Kubitza
09:41 AM Revision 6970: import_date: Removed duplicate Usage message at top of file, which is repeated in the Usage message provided when the program is run with no arguments
Aaron Marcuse-Kubitza
09:40 AM Revision 6969: Added import_date
Aaron Marcuse-Kubitza
09:38 AM Revision 6968: Added mtime
Aaron Marcuse-Kubitza
09:29 AM Revision 6967: lib/common.Makefile: System: Added $(mtime)
Aaron Marcuse-Kubitza
09:27 AM Revision 6966: lib/common.Makefile: $(date): Factored date format out into $(dateFmt)
Aaron Marcuse-Kubitza
09:25 AM Revision 6965: backups/Makefile: Factored $(isMac) out into lib/common.Makefile
Aaron Marcuse-Kubitza
08:51 AM Task #446: fix deadlock in INSERT IGNORE replacement
Added additional bug occurrence Aaron Marcuse-Kubitza
08:30 AM Revision 6964: README.TXT: Data import: tailing logs: Updated pattern for new log filename format
Aaron Marcuse-Kubitza

12/19/2012

02:02 PM Revision 6963: schemas/Makefile: Installation: %/publish: Fixed bug where need quotes around source schema name
Aaron Marcuse-Kubitza
01:57 PM Revision 6962: README.TXT: Data import: Moved deletion previous imports before the import, so that full DB backup can be automated
Aaron Marcuse-Kubitza
01:55 PM Revision 6961: README.TXT: Data import: `make backups/vegbien.$version.backup/test`: Added --exclude-schema=public to leave out the previous (now published) import so it doesn't bloat the backup. Note that public is included in the vegbien.$version.backup for the previous import, named according to its version.
Aaron Marcuse-Kubitza
01:49 PM Revision 6960: import_all: after_import(): Added `make backups/TNRS.backup-remake`
Aaron Marcuse-Kubitza
01:46 PM Revision 6959: README.TXT: Data import: Added step to publish the import to the public schema
Aaron Marcuse-Kubitza
01:42 PM Revision 6958: import_all: after_import(): Added export_analytical_db
Aaron Marcuse-Kubitza
01:36 PM Revision 6957: README.TXT: Data import: bin/export_analytical_db: Removed `env public=$version` because export_analytical_db now uses $version as $public when provided
Aaron Marcuse-Kubitza
01:35 PM Revision 6956: README.TXT: Data import: To remake analytical DB: Removed `env public=...` because $version (which replaces $public) is now set automatically by import_all
Aaron Marcuse-Kubitza
01:32 PM Revision 6955: schemas/Makefile: Installation: py_functions/install: Removed `env public=`, which is not needed since $(psqlAsAdminVegbien) does not use psql_script_vegbien (which uses $public)
Aaron Marcuse-Kubitza
01:28 PM Revision 6954: export_analytical_db: Use vegbien_dest to set the default value for $public
Aaron Marcuse-Kubitza
01:21 PM Revision 6953: README.TXT: Data import: If many inputs have errors: Updated command to `make schemas/$version/uninstall` because the current import's schema is now named $version
Aaron Marcuse-Kubitza
01:15 PM Revision 6952: schemas/Makefile: Installation: $(schemas), $(schemasReversed) (used e.g. by `make schemas/reinstall`): Removed public so that when `make schemas/reinstall` is run before an import, it will not remove any active (published) import which resides in the public schema
Aaron Marcuse-Kubitza
01:10 PM Revision 6951: README.TXT: Schema changes: Reinstall public separately from the other schemas so that it will still be reinstalled when schemas/reinstall excludes the public schema to avoid removing any active (published) import
Aaron Marcuse-Kubitza
01:01 PM Revision 6950: vegbien_dest callers: Removed no longer needed explicit setting $prefix to "", because this is now the default value
Aaron Marcuse-Kubitza
01:00 PM Revision 6949: vegbien_dest: Changed default $prefix to "", so that the majority of callers don't need to manually set $prefix to "" to avoid it defaulting to out_
Aaron Marcuse-Kubitza
12:45 PM Revision 6948: README.TXT: Data import: Use env var $version, which is now set by import_all, instead of manually inserting the version for <version>
Aaron Marcuse-Kubitza
12:40 PM Revision 6947: vegbien_dest: Also export $version
Aaron Marcuse-Kubitza
12:30 PM Revision 6946: import_all: Run the import directly into a new, already-versioned public schema. This removes the need to manually rename the schema after import, and allows the backup commands to use the stored $version shell variable to refer to the last import.
Aaron Marcuse-Kubitza
12:25 PM Revision 6945: schemas/Makefile: %/publish: Added instruction to run `unset version` after the command, to clear the $version shell variable which will be set by import_all
Aaron Marcuse-Kubitza
12:12 PM Revision 6944: README.TXT: Data import: Replaced <import_name> with <version> because the import name is now just the version
Aaron Marcuse-Kubitza
12:10 PM Revision 6943: README.TXT: Data import: Replaced r<revision> with <version> because the version string is now equal to r<revision>
Aaron Marcuse-Kubitza
12:09 PM Revision 6942: README.TXT: Backups: Replaced <date> with <version> because the date is no longer included in the version string
Aaron Marcuse-Kubitza
12:08 PM Revision 6941: README.TXT: Name archived imports without the "public." prefix so that their backups will work with the new `make backups/%.backup/remove` command, which does not add back the prefix
Aaron Marcuse-Kubitza
11:56 AM Revision 6940: backups/Makefile; $(public*): Don't add a "public." prefix to get the name of the public schema
Aaron Marcuse-Kubitza
11:40 AM Revision 6939: backups/Makefile: Removed no longer used $(rmSchema)
Aaron Marcuse-Kubitza
11:39 AM Revision 6938: backups/Makefile: Use \$(rmSchemaCmd) from lib/common.Makefile instead of \$(rmSchema)
Aaron Marcuse-Kubitza
11:20 AM Revision 6937: vegbien_dest: Use $version as $public when $public not provided. When neither is provided, continue to use "public" and also set $version to that.
Aaron Marcuse-Kubitza
11:11 AM Revision 6936: schemas/Makefile: Installation: rotate: Use just the version, without the "public." prefix
Aaron Marcuse-Kubitza
11:04 AM Revision 6935: schemas/Makefile: Installation: `public/install public%/install`: Generalized to %/install to allow public schema versions with any name. This requires moving `%/install: %.sql` before it to override it.
Aaron Marcuse-Kubitza
11:00 AM Revision 6934: schemas/Makefile: Installation: Merged public/install and public%/install
Aaron Marcuse-Kubitza
10:54 AM Revision 6933: schemas/Makefile: Installation: Moved %/uninstall to beginning of section because it applies to all schemas
Aaron Marcuse-Kubitza
10:52 AM Revision 6932: schemas/Makefile: Installation: public: Generalized public%/publish to %/publish so that public schema versions don't have to start with public_
Aaron Marcuse-Kubitza
10:34 AM Revision 6931: schemas/Makefile: Installation: %/uninstall: Also display schema delete confirmation for schemas whose name is just the version suffix (r<revision #>)
Aaron Marcuse-Kubitza
10:32 AM Revision 6930: schemas/Makefile: Merged public%/uninstall and %/uninstall
Aaron Marcuse-Kubitza
09:49 AM Revision 6929: lib/common.Makefile: Added version target, which prints the current $(version) value
Aaron Marcuse-Kubitza
09:36 AM Revision 6928: schemas/Makefile: Installation: public: public%/uninstall: Fixed bug where need to remove the *specified* version of the public schema, not public itself. Generalized $(confirmRmPublicSchema) so it could also be used for named versions of the public schema. Inlined $(rmPublicSchema) since it's now only used in one place.
Aaron Marcuse-Kubitza
09:26 AM Revision 6927: lib/common.Makefile: Revisions: $(version): Use just the revision # to avoid cluttering the schema and log file names with long datetime strings
Aaron Marcuse-Kubitza
09:25 AM Revision 6926: schemas/Makefile: public%/install: schema comment: Include current date/time after version
Aaron Marcuse-Kubitza
09:20 AM Revision 6925: lib/common.Makefile: Replaced no longer used $(date) with function to generate human-readable text date (rather than date to put in filename). Removed leading zeros from date and hour. Added timezone.
Aaron Marcuse-Kubitza
09:07 AM Revision 6924: backups/Makefile: Removed no longer used $(dateFmt), $(mtime)
Aaron Marcuse-Kubitza
08:59 AM Revision 6923: backups/Makefile: Removed %.backup/rotate, because this incorrectly causes the current time rather than the version to be used in the backup filename. The version should instead be specified in the backup filename when it's created.
Aaron Marcuse-Kubitza
08:51 AM Revision 6922: schemas/Makefile: Installation: public: Added public%/publish to replace the current public schema with the given version
Aaron Marcuse-Kubitza
08:37 AM Revision 6921: schemas/Makefile: Installation: public: public/uninstall: Added public%/uninstall as a target to allow uninstalling versions of the public schema
Aaron Marcuse-Kubitza
08:30 AM Revision 6920: schemas/Makefile: Installation: public: public%/install: Add a comment on the schema containing the versioned schema name, so that if the schema is later renamed to just public (i.e. "published" as the current version), it will still be possible to tell which version the public schema came from
Aaron Marcuse-Kubitza
08:22 AM Revision 6919: schemas/Makefile: Installation: public: Added public%/install, to install a version of the public schema
Aaron Marcuse-Kubitza
07:59 AM Revision 6918: schemas/Makefile: Removed unused $(os)
Aaron Marcuse-Kubitza
07:58 AM Revision 6917: schemas/Makefile: Removed unused $(SED)
Aaron Marcuse-Kubitza
06:22 AM Revision 6916: Moved schemas-related commands from root Makefile to schemas/Makefile
Aaron Marcuse-Kubitza
06:15 AM Revision 6915: Makefiles: Factored out common vars/functions into lib/common.Makefile
Aaron Marcuse-Kubitza
05:59 AM Revision 6914: root Makefile: $(psqlNoSearchPath): Merged $(psqlAsBien) into it because it's the only place $(psqlAsBien) is used
Aaron Marcuse-Kubitza
05:56 AM Revision 6913: root Makefile: $(psqlAsBien): Use psql_script_vegbien instead of psql_vegbien, which adds $(psqlOpts) itself
Aaron Marcuse-Kubitza
05:50 AM Revision 6912: schemas/Makefile: Include lib/common.Makefile
Aaron Marcuse-Kubitza
05:23 AM Revision 6911: inputs/import.stats.xls: Reformatted so the first by column import and the comparison by row import will fit on the same page when printed on portrait-mode letter paper
Aaron Marcuse-Kubitza
05:10 AM Revision 6910: inputs/import.stats.xls: Changed import type labels to By row/By column so they would fit into one field, leaving the extra field free to contain the revision #
Aaron Marcuse-Kubitza
05:02 AM Revision 6909: lib/common.Makefile: Revisions: Allow $(version) to be overridden in the environment, so that the public schema and all log files share the same, pregenerated version
Aaron Marcuse-Kubitza
04:16 AM Revision 6908: schemas/vegbien.sql: Merged provider_view, provider_count, and owner_count into provider_count, using the combining query for Brad's data providers page at <http://bien.nceas.ucsb.edu/bien/people/data-providers/>
Aaron Marcuse-Kubitza
01:23 AM Revision 6907: schemas/vegbien.sql: sync_taxon_trait_to_view(): Changed pkey to index because there can be multiple values of the same taxon's trait from different observations
Aaron Marcuse-Kubitza
01:16 AM Revision 6906: mappings/Makefile: VegCore.csv: Filter out the VegCore tables so they are not matched as terms. This is necessary because some terms have the same name as a table, but the term should be the match rather than the table.
Aaron Marcuse-Kubitza
12:29 AM Revision 6905: sql.py: DbConn.col_info(): raising sql_gen.NoUnderlyingTableException: Fixed bug where also need to catch DoesNotExistException, which is thrown by ::regclass
Aaron Marcuse-Kubitza
12:26 AM Revision 6904: sql.py: DbConn.col_info(): Fixed bug where need to run run_query() recoverably, because this query throws an exception if the column's table does not exist (the information_schema query just returned no rows)
Aaron Marcuse-Kubitza
12:22 AM Revision 6903: sql.py: DbConn.col_info(): Fixed bug where need to use pg_get_expr() on pg_attrdef.adbin instead of shortcut field adsrc, because adsrc does not include schema qualifiers on table names (including strings passed to `nextval('..._seq'::regclass)`)
Aaron Marcuse-Kubitza

12/18/2012

11:42 PM Revision 6902: sql.py: DbConn.col_info(): Fixed bug where need to pass through cacheable param to run_query()
Aaron Marcuse-Kubitza
11:41 PM Revision 6901: sql.py: DbConn.col_info(): Fixed bug where need to use .to_str(self) instead of self.esc_value() because self.esc_value() expects a value, not a sql_gen.Literal instance
Aaron Marcuse-Kubitza
11:34 PM Revision 6900: sql.py: DbConn.col_info(): Fixed bug where self needs to be used everywhere that db normally is, because this is a DbConn method rather than a global function
Aaron Marcuse-Kubitza
11:31 PM Revision 6899: sql.py: DbConn.col_info(): For PostgreSQL, use pg_catalog tables directly instead of their views in information_schema. This allows using ::regclass to look up the table in the search_path, and fixes a bug in imports with an explicit public schema where column types were looked up in public instead of public.<version>. Also don't wrap default using sql_gen.as_Code() when it's None (indicating no default value, aka default=NULL), because this value is interpreted specially by sql_gen.TypedCol.
Aaron Marcuse-Kubitza
11:06 PM Revision 6898: inputs/Makefile: Input data: $(rsyncSrcs): Also include log files other than install.log.sql
Aaron Marcuse-Kubitza
09:41 PM Revision 6897: import_all: Run all imports (not just the main datasources' import) with $import_source turned off, so that the Source tables will not be imported a second time when the datasource's main tables are imported. Note that it's not necessary to wait for asynchronous commands after the jobs for the main import are started (so that $import_source is not unset until after they are started), because with_all does not return until all jobs are started and have noted the $import_source setting in effect in the shell environment.
Aaron Marcuse-Kubitza
09:32 PM Revision 6896: import_all: Source tables import: Fixed bug where need to use $all option to with_all to also include special datasources starting with "."
Aaron Marcuse-Kubitza
09:23 PM Revision 6895: make_analytical_db: Also create taxon_trait materialized view
Aaron Marcuse-Kubitza

12/17/2012

08:17 PM Revision 6894: inputs/*/*/map.csv: Reverted special OMIT mappings for input columns that have the same name as a VegCore table and have not yet been mapped to a VegCore term
Aaron Marcuse-Kubitza
08:06 PM Revision 6893: mappings/Makefile: VegCore.csv: Filter out the VegCore tables so they are not matched as terms. This is necessary because some terms have the same name as a table, but the term should be the match rather than the table.
Aaron Marcuse-Kubitza
08:04 PM Revision 6892: mappings/VegCore.csv: Changed line endings to \r\n to match the output of filter_out_ci
Aaron Marcuse-Kubitza
05:51 PM Revision 6891: inputs/CTFS/TaxonOccurrence/map.csv: Mapped SpeciesAuthority
Aaron Marcuse-Kubitza
04:59 PM Revision 6890: backups/Makefile: Synchronization: $(remote): Fixed bug where need trailing / at end of path
Aaron Marcuse-Kubitza
04:32 PM Revision 6889: backups/Makefile: Synchronization: $(remote): Updated path to backups
Aaron Marcuse-Kubitza
04:30 PM Revision 6888: README.TXT: Data import: On jupiter: Updated path to backups
Aaron Marcuse-Kubitza
04:25 PM Revision 6887: README.TXT: Installation: Added command to change to the directory of the checked out files
Aaron Marcuse-Kubitza
04:24 PM Revision 6886: README.TXT: Installation: Added command to check out files from svn
Aaron Marcuse-Kubitza
03:51 PM Revision 6885: schemas/vegbien.sql: Added taxon_trait materialized view
Aaron Marcuse-Kubitza
02:43 PM Revision 6884: mappings/Veg+-VegCore.csv: Sources: Removed redundant bien2_ prefix from bien2_staging subnamespace
Aaron Marcuse-Kubitza
02:21 PM Revision 6883: schemas/vegbien.sql: trait: trait_unique: Removed value and units because there should only be one value of a trait for each taxonoccurrence
Aaron Marcuse-Kubitza
02:18 PM Revision 6882: schemas/vegbien.sql: Reattached trait to taxonoccurrence instead of taxonlabel, because the TraitObservation traits data is actually associated with a particular occurrence (plant observation complete with location, date, etc.), rather than just a taxon
Aaron Marcuse-Kubitza
01:31 PM Revision 6881: Added inputs/bien2_traits/
Aaron Marcuse-Kubitza
01:29 PM Revision 6880: mappings/VegCore-VegBIEN.csv: Mapped traits-related DwC terms measurementType, measurementValue, measurementUnit
Aaron Marcuse-Kubitza
12:34 PM Revision 6879: schemas/vegbien.ERD.mwb: Added trait table to ERD
Aaron Marcuse-Kubitza
12:25 PM Revision 6878: schemas/vegbien.sql: trait: Added trait_unique unique index
Aaron Marcuse-Kubitza
12:19 PM Revision 6877: schemas/vegbien.sql: trait: Added units field
Aaron Marcuse-Kubitza
12:14 PM Revision 6876: schemas/vegbien.sql: trait: Renamed type to name because TraitObservation stores trait names rather than types
Aaron Marcuse-Kubitza
12:07 PM Revision 6875: schemas/vegbien.sql: trait: Linked to taxonlabel instead of stemobservation, because TraitObservation's traits are taxon-level and stem-level traits currently go in named fields instead of a stem traits table
Aaron Marcuse-Kubitza
11:45 AM Revision 6874: inputs/.TNRS/tnrs_*/map.csv: Remapped Source to OMIT so it won't match to the Source table
Aaron Marcuse-Kubitza
11:37 AM Revision 6873: inputs/.TNRS/tnrs_other/map.csv: Updated for new VegCore terms, which include Source as a table name. This field will need to be remapped so it doesn't collide with the table name.
Aaron Marcuse-Kubitza
10:04 AM Revision 6872: inputs/import.stats.xls: Updated import times
Aaron Marcuse-Kubitza
10:01 AM Revision 6871: README.TXT: Data import: Added step to check that the source table contains entries for all inputs
Aaron Marcuse-Kubitza

12/14/2012

01:01 PM Revision 6870: Regenerated vegbien.ERD exports
Aaron Marcuse-Kubitza
12:52 PM Revision 6869: make_analytical_db: Also populate owner_count
Aaron Marcuse-Kubitza
12:51 PM Revision 6868: make_analytical_db: Generate provider_count before analytical_aggregate because it's much faster
Aaron Marcuse-Kubitza
12:50 PM Revision 6867: schemas/vegbien.sql: Added materialized view owner_count, generated from owner_count_view
Aaron Marcuse-Kubitza
12:21 PM Revision 6866: make_analytical_db: Also populate provider_count
Aaron Marcuse-Kubitza
12:20 PM Revision 6865: schemas/vegbien.sql: Added materialized view provider_count, generated from provider_count_view
Aaron Marcuse-Kubitza
12:09 PM Revision 6864: schemas/vegbien.sql: Added provider_count_view for counts of occurrences per top-level provider
Aaron Marcuse-Kubitza
11:56 AM Revision 6863: Regenerated mappings/VegCore.htm
Aaron Marcuse-Kubitza
11:52 AM Revision 6862: Regenerated mappings/VegCore.htm
Aaron Marcuse-Kubitza
11:10 AM Revision 6861: Regenerated mappings/VegCore.htm
Aaron Marcuse-Kubitza
10:39 AM Revision 6860: schemas/vegbien.sql: provider_view: Sort NULL sourcetype last
Aaron Marcuse-Kubitza
10:36 AM Revision 6859: schemas/vegbien.sql: Added provider_view, which combines source and sourcename
Aaron Marcuse-Kubitza
10:31 AM Revision 6858: schemas/vegbien.sql: sourcename: Gave public_ SELECT permissions
Aaron Marcuse-Kubitza
10:17 AM Revision 6857: Regenerated mappings/VegCore.htm
Aaron Marcuse-Kubitza
10:15 AM Revision 6856: README.TXT: Maintenance: VegCore data dictionary: Regenerate everything in mappings/ that changes when VegCore.htm changes (such as VegCore.tables.redmine) instead of just VegCore.csv
Aaron Marcuse-Kubitza
09:29 AM Revision 6855: inputs/*/Source/map.csv without mappings: Added referenceType, etc. mappings. This also ensures that the source table entry for the datasource will be created before the herbaria list is imported, causing all top-level datasources to sort at the top of the source table.
Aaron Marcuse-Kubitza
09:02 AM Revision 6854: schemas/vegbien.sql: Granted the public_ user read-only access to the contents of the source table
Aaron Marcuse-Kubitza
08:53 AM Revision 6853: root Makefile: PostgreSQL: $(editPhppgadmin): Ignore errors if patch has already been applied
Aaron Marcuse-Kubitza
08:52 AM Revision 6852: lib/phpPgAdmin.config.inc.php.diff: Remove context so segment matching would depend only on the $conf['extra_login_security'] line itself
Aaron Marcuse-Kubitza
08:29 AM Revision 6851: mappings/Makefile: Added VegCore.tables.redmine, which contains the Redmine-formatted list of VegCore tables to paste into <https://projects.nceas.ucsb.edu/nceas/projects/bien/wiki/VegCore#Tables>
Aaron Marcuse-Kubitza
08:18 AM Revision 6850: mappings/: Removed no longer used VegCore.redmine. VegCore.csv is now generated from the Redmine page instead of the other way around.
Aaron Marcuse-Kubitza
08:12 AM Revision 6849: mappings/Makefile: Added VegCore.tables.csv, which contains all the tables in the VegCore data dictionary
Aaron Marcuse-Kubitza
06:59 AM Revision 6848: README.TXT: Data import: backups/fix_perms: Run using sudo to also change permissions on files owned by the bien user, and to change the owner of files owned by you to the bien user
Aaron Marcuse-Kubitza
06:45 AM Revision 6847: Regenerated mappings/VegCore.csv, which adds categories
Aaron Marcuse-Kubitza
05:47 AM Revision 6846: README.TXT: Maintenance: Added instructions to regenerate mappings/VegCore.csv whenever the VegCore data dictionary page is changed
Aaron Marcuse-Kubitza
05:41 AM Revision 6845: mappings/Makefile: Generate VegCore.csv from the VegCore data dictionary page by extracting all HTML anchors (in Redmine, each section heading, and therefore each VegCore term, gets its own anchor)
Aaron Marcuse-Kubitza
05:34 AM Revision 6844: mappings/VegCore.csv: Changed line endings to \n to match what sed generates from the VegCore data dictionary page
Aaron Marcuse-Kubitza
05:31 AM Revision 6843: mappings/VegCore.csv: Removed informational columns, because this information is now maintained on the VegCore data dictionary page at <https://projects.nceas.ucsb.edu/nceas/projects/bien/wiki/VegCore>
Aaron Marcuse-Kubitza
05:11 AM Revision 6842: mappings/Veg+-VegCore.csv.csv: Removed hypothetical terms which are not in use by any VegBIEN datasource
Aaron Marcuse-Kubitza
05:00 AM Revision 6841: mappings/Veg+-VegCore.csv: habit: Remapped to growthForm, which replaces verbatimGrowthForm
Aaron Marcuse-Kubitza
04:59 AM Revision 6840: mappings/Veg+-VegCore.csv.csv: Removed hypothetical terms which are not in use by any VegBIEN datasource
Aaron Marcuse-Kubitza
04:50 AM Revision 6839: mappings/VegCore.csv: BIEN2 terms: Added sub-namespaces (bien_web, geoscrub, etc.) to source URLs
Aaron Marcuse-Kubitza
04:15 AM Revision 6838: dict2redmine: redmine_add_links(): Hyperlink just the source name, not also the () around it
Aaron Marcuse-Kubitza
03:54 AM Revision 6837: dict2redmine: RedmineDictWriter: Use h2 instead of h3 for the term name so that the term will be normal-sized instead of smaller in the Redmine table of contents
Aaron Marcuse-Kubitza
03:52 AM Revision 6836: dict2redmine: Renamed redmine_url() to redmine_link() because it generates links, not URLs
Aaron Marcuse-Kubitza
03:49 AM Revision 6835: dict2redmine: redmine_add_links(): Put citations in () instead of [] to avoid conflicting with the Redmine syntax for internal links ( [[...]] )
Aaron Marcuse-Kubitza
03:18 AM Revision 6834: mappings/VegCore.csv: Terms: Removed namespace prefixes (dcterms:), because VegCore terms are globally unique within VegCore and there should not be multiple versions of the same VegCore term with different namespaces. Provenance is instead indicated in the Sources column, which contains not just a namespace but a full URL to each source term.
Aaron Marcuse-Kubitza
03:00 AM Revision 6833: dict2redmine: Hyperlink each term to its anchor in the data dictionary, rather than to its first source, which is not necessarily the definitive definition of the term. This also allows clicking the term to get its permalink in the address bar, rather than having to click the small, light gray paragraph mark next to the term name that Redmine provides.
Aaron Marcuse-Kubitza
02:57 AM Revision 6832: dict2redmine: redmine_add_links(): Fixed bug where need to avoid matching internal links ( [[...]] ) as citations ( [...] )
Aaron Marcuse-Kubitza
02:46 AM Revision 6831: mappings/VegCore.csv: Term names: Changed special characters to _ because Redmine doesn't support special characters in HTML anchors (it removes everything except letters, numbers, _, and -)
Aaron Marcuse-Kubitza
02:42 AM Revision 6830: mappings/Makefile: .Veg+-VegCore.csv.last_cleanup: Also canon the output (VegCore) column to the VegCore.csv vocabulary. ? prefixes are not a problem because there are always at least two alternatives listed for these terms, so canon will not modify the output field.
Aaron Marcuse-Kubitza
01:49 AM Revision 6829: psql_script_vegbien: Run psql_vegbien with `nice -n +5` to prevent CPU-intensive operations from slowing down the shell/UI
Aaron Marcuse-Kubitza
01:46 AM Revision 6828: inputs/import.stats.xls: Updated import times
Aaron Marcuse-Kubitza
01:37 AM Revision 6827: Regenerated inputs/CVS/taxonObservation_/new_terms.csv. Note that it includes mappings to terms which are not in mappings/VegCore-VegBIEN.csv, which are prefixed with *.
Aaron Marcuse-Kubitza
01:34 AM Revision 6826: input.Makefile: Maps validation: %/new_terms.csv: Undid incorrect change of column to filter terms out of. This actually needs to be the input column, even though unmapped_terms.csv is generated from the output column, because it's possible to have a mapping to a term which is not in mappings/VegCore-VegBIEN.csv, and such a term would show up in unmapped_terms.csv but should not be filtered out of new_terms.csv.
Aaron Marcuse-Kubitza
01:17 AM Revision 6825: lib/phpPgAdmin.login.php.diff: public_ user's password message: Print as its own message instead of appending it to $msg. Print it before any error message so it always appears at the top of the page.
Aaron Marcuse-Kubitza
12:51 AM Revision 6824: root Makefile: PostgreSQL: phpPgAdmin: Edit config file to allow passwordless logins. Edit login page to fill in public_ as the default username and add a message to leave the password blank for that user.
Aaron Marcuse-Kubitza
 

Also available in: Atom