Revision 5368
Added by Aaron Marcuse-Kubitza over 12 years ago
lib/sql_io.py | ||
---|---|---|
481 | 481 |
|
482 | 482 |
not_cond = sql_gen.NotCond(sql_gen.CustomCode(cond)) |
483 | 483 |
log_debug('Ignoring rows where '+strings.as_tt(not_cond.to_str(db))) |
484 |
sql.delete(db, insert_in_table, not_cond) |
|
484 |
assert cond != sql_gen.true_expr # we know the constraint failed |
|
485 |
if cond == sql_gen.false_expr: remove_all_rows() |
|
486 |
else: sql.delete(db, insert_in_table, not_cond) |
|
485 | 487 |
|
486 | 488 |
not_null_cols = set() |
487 | 489 |
def ignore(in_col, value, e): |
Also available in: Unified diff
sql_io.py: put_table(): ignore_cond(): If cond simplifies to false, remove all rows instead of filtering out individual rows which will all be filtered out. This optimization should improve import times of tables, such as taxonconcept, which use a check constraint instead of NOT NULL constraints to prevent empty rows. The taxonomic schema refactoring caused the creation of many more levels of taxonconcepts, many of which (such as variety, forma, cultivar) are empty for most datasources, so this optimization should also reduce overall import times for datasources that have any empty levels of taxonconcept. Note that this optimization is only possible now that sql_gen.simplify_expr() is able to simplify all the way to a single boolean value for the taxonconcept_required_key constraint.