Project

General

Profile

Statistics
| Revision:
Name Size Revision Age Author Comment
  _archive 1598 over 12 years Aaron Marcuse-Kubitza Moved _archive/tapir2flatClient/trunk/client/ t...
  analysis 3076 over 12 years Aaron Marcuse-Kubitza Added top-level analysis dir for range modeling
  bin 3270 over 12 years Aaron Marcuse-Kubitza csv2db: Don't cache per-row INSERT queries beca...
  config 272 almost 13 years Aaron Marcuse-Kubitza Moved bien_password to new config dir
  inputs 3261 over 12 years Aaron Marcuse-Kubitza inputs/import.stats.xls: Updated with stats fro...
  lib 3269 over 12 years Aaron Marcuse-Kubitza sql.py with_explain_comment(), DbConn: Fixed bu...
  mappings 3229 over 12 years Aaron Marcuse-Kubitza mappings/VegX-VegBIEN.stems.csv: Sort the plant...
  schemas 3262 over 12 years Aaron Marcuse-Kubitza schemas/vegbien.sql: taxondetermination: taxond...
  to_do 2547 over 12 years Aaron Marcuse-Kubitza to_do/timeline.doc: Updated to reflect the mont...
Makefile 10.5 KB 3249 over 12 years Aaron Marcuse-Kubitza root Makefile: VegBIEN DB: Schemas: Added schem...
README.TXT 2.96 KB 3205 over 12 years Aaron Marcuse-Kubitza README.TXT: Data import: Import data into VegBI...
map 1.21 KB 3140 over 12 years Aaron Marcuse-Kubitza top-level map: Added support for custom public ...

Latest revisions

# Date Author Comment
3270 07/09/2012 02:01 PM Aaron Marcuse-Kubitza

csv2db: Don't cache per-row INSERT queries because this bloats the cache (there aren't repeated identical INSERTs that shouldn't be re-run like in row-based import)

3269 07/09/2012 01:57 PM Aaron Marcuse-Kubitza

sql.py with_explain_comment(), DbConn: Fixed bug where with_explain_comment() was being run in per-row imports (row-based import and csv2db with INSERT), causing the overhead of an EXPLAIN query for every single INSERT and filling up the cache with EXPLAIN query results, by adding autoexplain mode, only running with_explain_comment() in autoexplain mode, and only enabling autoexplain mode for column-based import

3268 07/09/2012 01:11 PM Aaron Marcuse-Kubitza

db_xml.py: put_table(): Turn on autoanalyze mode to help the query planner avoid sequential scans on tables that now contain data. (Don't do this in row-based import because it creates too much overhead per insert.)

3267 07/09/2012 12:24 PM Aaron Marcuse-Kubitza

sql.py: Run all EXPLAIN queries with log_level=4 since the EXPLAIN information is now usually generated when the query is generated rather than when it's run, so the log_level is not known

3266 07/09/2012 12:21 PM Aaron Marcuse-Kubitza

sql.py: Added with_explain_comment() to query generating functions so that nested queries will also have EXPLAIN information

3265 07/09/2012 12:11 PM Aaron Marcuse-Kubitza

sql.py: Added with_explain_comment() and use it in run_query()

3264 07/09/2012 12:01 PM Aaron Marcuse-Kubitza

sql.py: run_query(): EXPLAIN output: Run explain() with log_level 1 higher than the query's log_level, so that low-level queries' EXPLAIN queries are not output when the queries themselves are not output. This also ensures that only level 2 (major) queries have the EXPLAIN logged (to introduce the query that is being run), to avoid cluttering the log output.

3263 07/09/2012 11:54 AM Aaron Marcuse-Kubitza

sql.py: explain(): Support custom log_level

3262 07/09/2012 11:48 AM Aaron Marcuse-Kubitza

schemas/vegbien.sql: taxondetermination: taxondetermination_taxonoccurrence_id_fkey manual fkey constraint: Fixed bug where needed to raise foreign_key_violation instead of unique_violation

3261 07/09/2012 11:23 AM Aaron Marcuse-Kubitza

inputs/import.stats.xls: Updated with stats from latest import

View all revisions | View revisions

Also available in: Atom