Project

General

Profile

Statistics
| Revision:
Name Size Revision Age Author Comment
  _archive 1598 almost 13 years Aaron Marcuse-Kubitza Moved _archive/tapir2flatClient/trunk/client/ t...
  analysis 3076 over 12 years Aaron Marcuse-Kubitza Added top-level analysis dir for range modeling
  backups 4751 over 12 years Aaron Marcuse-Kubitza backups/Makefile: Backups: Full DB: Specify the...
  bin 5324 over 12 years Aaron Marcuse-Kubitza tnrs_db: Moved "Processing # taxonconcepts" log...
  config 272 about 13 years Aaron Marcuse-Kubitza Moved bien_password to new config dir
  inputs 5339 over 12 years Aaron Marcuse-Kubitza mappings/VegCore-VegBIEN.csv: verbatim* taxonco...
  lib 5377 over 12 years Aaron Marcuse-Kubitza sql_io.py: put_table(): Replaced limit_ref inte...
  mappings 5339 over 12 years Aaron Marcuse-Kubitza mappings/VegCore-VegBIEN.csv: verbatim* taxonco...
  schemas 5328 over 12 years Aaron Marcuse-Kubitza lib/PostgreSQL-MySQL.csv: COMMENT statement: Fi...
  to_do 4524 over 12 years Aaron Marcuse-Kubitza to_do/timeline.doc: Updated to reflect addition...
  validation 4523 over 12 years Aaron Marcuse-Kubitza Added validation/
Makefile 9.99 KB 4752 over 12 years Aaron Marcuse-Kubitza root Makefile: PostgreSQL: postgres-Linux: Adde...
README.TXT 13.1 KB 5321 over 12 years Aaron Marcuse-Kubitza README.TXT: Schema changes: files to update wit...
map 989 Bytes 5158 over 12 years Aaron Marcuse-Kubitza root map: Removed no longer needed public schem...
new_terms.csv 30.4 KB 4887 over 12 years Aaron Marcuse-Kubitza Regenerated root unmapped_terms.csv, new_terms.csv
unmapped_terms.csv 5.8 KB 4887 over 12 years Aaron Marcuse-Kubitza Regenerated root unmapped_terms.csv, new_terms.csv

Latest revisions

# Date Author Comment
5377 10/10/2012 02:30 AM Aaron Marcuse-Kubitza

sql_io.py: put_table(): Replaced limit_ref integer with ignore_all_ref boolean, because it is no longer used as a select statement limit

5376 10/10/2012 02:29 AM Aaron Marcuse-Kubitza

sql_io.py: put_table(): remove_all_rows(): Corrected "just create an empty pkeys table" comment to "just return the default value column"

5375 10/10/2012 02:27 AM Aaron Marcuse-Kubitza

sql_io.py: put_table(): mk_main_select(): Removed setting limit to limit_ref0, because an empty pkeys table is no longer created when ignoring all rows

5374 10/10/2012 02:19 AM Aaron Marcuse-Kubitza

sql_io.py: put_table(): Setting pkeys of missing rows: Removed "limit_ref0 == 0" check because this code is never reached in that case

5373 10/10/2012 02:16 AM Aaron Marcuse-Kubitza

sql_io.py: put_table(): Ignoring all rows for unrecoverable errors: Even in multi-row mode, just return whatever the default value or column was, instead of creating an output table containing the default value filled in for every row. This also assists the optimization to skip empty levels of taxonconcepts, because it folds the empty level to that level's parent level rather than creating a whole new temp table with ultimately the same contents.

5372 10/10/2012 01:57 AM Aaron Marcuse-Kubitza

sql_gen.py: not_false_re, not_true_re: Appended \b to ensure that true/false is only matched as a single word

5371 10/10/2012 01:56 AM Aaron Marcuse-Kubitza

sql_gen.py: simplify_expr(): Also simplify "NOT false" to true

5370 10/10/2012 01:53 AM Aaron Marcuse-Kubitza

sql_gen.py: simplify_expr(): Also simplify "NOT true" to false

5369 10/10/2012 01:24 AM Aaron Marcuse-Kubitza

sql_io.py: put_table(): ignore_cond(): Changed "Ignoring rows where" message with the negated (filter-out) condition to "Ignoring rows that don't satisfy" with the filter condition for clarity

5368 10/10/2012 01:22 AM Aaron Marcuse-Kubitza

sql_io.py: put_table(): ignore_cond(): If cond simplifies to false, remove all rows instead of filtering out individual rows which will all be filtered out. This optimization should improve import times of tables, such as taxonconcept, which use a check constraint instead of NOT NULL constraints to prevent empty rows. The taxonomic schema refactoring caused the creation of many more levels of taxonconcepts, many of which (such as variety, forma, cultivar) are empty for most datasources, so this optimization should also reduce overall import times for datasources that have any empty levels of taxonconcept. Note that this optimization is only possible now that sql_gen.simplify_expr() is able to simplify all the way to a single boolean value for the taxonconcept_required_key constraint.

View all revisions | View revisions

Also available in: Atom