Project

General

Profile

Statistics
| Revision:
Name Size Revision Age Author Comment
  _archive 1598 almost 13 years Aaron Marcuse-Kubitza Moved _archive/tapir2flatClient/trunk/client/ t...
  analysis 3076 over 12 years Aaron Marcuse-Kubitza Added top-level analysis dir for range modeling
  backups 4751 over 12 years Aaron Marcuse-Kubitza backups/Makefile: Backups: Full DB: Specify the...
  bin 5324 over 12 years Aaron Marcuse-Kubitza tnrs_db: Moved "Processing # taxonconcepts" log...
  config 272 about 13 years Aaron Marcuse-Kubitza Moved bien_password to new config dir
  inputs 5339 over 12 years Aaron Marcuse-Kubitza mappings/VegCore-VegBIEN.csv: verbatim* taxonco...
  lib 5373 over 12 years Aaron Marcuse-Kubitza sql_io.py: put_table(): Ignoring all rows for u...
  mappings 5339 over 12 years Aaron Marcuse-Kubitza mappings/VegCore-VegBIEN.csv: verbatim* taxonco...
  schemas 5328 over 12 years Aaron Marcuse-Kubitza lib/PostgreSQL-MySQL.csv: COMMENT statement: Fi...
  to_do 4524 over 12 years Aaron Marcuse-Kubitza to_do/timeline.doc: Updated to reflect addition...
  validation 4523 over 12 years Aaron Marcuse-Kubitza Added validation/
Makefile 9.99 KB 4752 over 12 years Aaron Marcuse-Kubitza root Makefile: PostgreSQL: postgres-Linux: Adde...
README.TXT 13.1 KB 5321 over 12 years Aaron Marcuse-Kubitza README.TXT: Schema changes: files to update wit...
map 989 Bytes 5158 over 12 years Aaron Marcuse-Kubitza root map: Removed no longer needed public schem...
new_terms.csv 30.4 KB 4887 over 12 years Aaron Marcuse-Kubitza Regenerated root unmapped_terms.csv, new_terms.csv
unmapped_terms.csv 5.8 KB 4887 over 12 years Aaron Marcuse-Kubitza Regenerated root unmapped_terms.csv, new_terms.csv

Latest revisions

# Date Author Comment
5373 10/10/2012 02:16 AM Aaron Marcuse-Kubitza

sql_io.py: put_table(): Ignoring all rows for unrecoverable errors: Even in multi-row mode, just return whatever the default value or column was, instead of creating an output table containing the default value filled in for every row. This also assists the optimization to skip empty levels of taxonconcepts, because it folds the empty level to that level's parent level rather than creating a whole new temp table with ultimately the same contents.

5372 10/10/2012 01:57 AM Aaron Marcuse-Kubitza

sql_gen.py: not_false_re, not_true_re: Appended \b to ensure that true/false is only matched as a single word

5371 10/10/2012 01:56 AM Aaron Marcuse-Kubitza

sql_gen.py: simplify_expr(): Also simplify "NOT false" to true

5370 10/10/2012 01:53 AM Aaron Marcuse-Kubitza

sql_gen.py: simplify_expr(): Also simplify "NOT true" to false

5369 10/10/2012 01:24 AM Aaron Marcuse-Kubitza

sql_io.py: put_table(): ignore_cond(): Changed "Ignoring rows where" message with the negated (filter-out) condition to "Ignoring rows that don't satisfy" with the filter condition for clarity

5368 10/10/2012 01:22 AM Aaron Marcuse-Kubitza

sql_io.py: put_table(): ignore_cond(): If cond simplifies to false, remove all rows instead of filtering out individual rows which will all be filtered out. This optimization should improve import times of tables, such as taxonconcept, which use a check constraint instead of NOT NULL constraints to prevent empty rows. The taxonomic schema refactoring caused the creation of many more levels of taxonconcepts, many of which (such as variety, forma, cultivar) are empty for most datasources, so this optimization should also reduce overall import times for datasources that have any empty levels of taxonconcept. Note that this optimization is only possible now that sql_gen.simplify_expr() is able to simplify all the way to a single boolean value for the taxonconcept_required_key constraint.

5367 10/10/2012 12:55 AM Aaron Marcuse-Kubitza

Moved expression transforming functions from sql.py to sql_gen.py because they do not manipulate an actual database and merely generate SQL

5366 10/10/2012 12:38 AM Aaron Marcuse-Kubitza

sql.py: Added true_expr, false_expr and use them where their values are used

5365 10/10/2012 12:34 AM Aaron Marcuse-Kubitza

sql.py: simplify_expr(): Also simplify "AND true" expressions

5364 10/10/2012 12:30 AM Aaron Marcuse-Kubitza

sql.py: simplify_expr(): Also simplify "AND false" expressions

View all revisions | View revisions

Also available in: Atom