Project

General

Profile

Statistics
| Revision:
Name Size Revision Age Author Comment
  _archive 1598 over 12 years Aaron Marcuse-Kubitza Moved _archive/tapir2flatClient/trunk/client/ t...
  analysis 3076 over 12 years Aaron Marcuse-Kubitza Added top-level analysis dir for range modeling
  bin 3271 over 12 years Aaron Marcuse-Kubitza csv2db: verbosity defaults to 3 so that detaile...
  config 272 almost 13 years Aaron Marcuse-Kubitza Moved bien_password to new config dir
  inputs 3261 over 12 years Aaron Marcuse-Kubitza inputs/import.stats.xls: Updated with stats fro...
  lib 3272 over 12 years Aaron Marcuse-Kubitza sql.py: DbConn.with_savepoint(): Profile (neste...
  mappings 3229 over 12 years Aaron Marcuse-Kubitza mappings/VegX-VegBIEN.stems.csv: Sort the plant...
  schemas 3262 over 12 years Aaron Marcuse-Kubitza schemas/vegbien.sql: taxondetermination: taxond...
  to_do 2547 over 12 years Aaron Marcuse-Kubitza to_do/timeline.doc: Updated to reflect the mont...
Makefile 10.5 KB 3249 over 12 years Aaron Marcuse-Kubitza root Makefile: VegBIEN DB: Schemas: Added schem...
README.TXT 2.96 KB 3205 over 12 years Aaron Marcuse-Kubitza README.TXT: Data import: Import data into VegBI...
map 1.21 KB 3140 over 12 years Aaron Marcuse-Kubitza top-level map: Added support for custom public ...

Latest revisions

# Date Author Comment
3272 07/09/2012 02:16 PM Aaron Marcuse-Kubitza

sql.py: DbConn.with_savepoint(): Profile (nested) transactions so that the run time for groups of commands (e.g. csv2db INSERTs) is known

3271 07/09/2012 02:04 PM Aaron Marcuse-Kubitza

csv2db: verbosity defaults to 3 so that detailed queries with profiling stats are included in the log file, to assist in optimization

3270 07/09/2012 02:01 PM Aaron Marcuse-Kubitza

csv2db: Don't cache per-row INSERT queries because this bloats the cache (there aren't repeated identical INSERTs that shouldn't be re-run like in row-based import)

3269 07/09/2012 01:57 PM Aaron Marcuse-Kubitza

sql.py with_explain_comment(), DbConn: Fixed bug where with_explain_comment() was being run in per-row imports (row-based import and csv2db with INSERT), causing the overhead of an EXPLAIN query for every single INSERT and filling up the cache with EXPLAIN query results, by adding autoexplain mode, only running with_explain_comment() in autoexplain mode, and only enabling autoexplain mode for column-based import

3268 07/09/2012 01:11 PM Aaron Marcuse-Kubitza

db_xml.py: put_table(): Turn on autoanalyze mode to help the query planner avoid sequential scans on tables that now contain data. (Don't do this in row-based import because it creates too much overhead per insert.)

3267 07/09/2012 12:24 PM Aaron Marcuse-Kubitza

sql.py: Run all EXPLAIN queries with log_level=4 since the EXPLAIN information is now usually generated when the query is generated rather than when it's run, so the log_level is not known

3266 07/09/2012 12:21 PM Aaron Marcuse-Kubitza

sql.py: Added with_explain_comment() to query generating functions so that nested queries will also have EXPLAIN information

3265 07/09/2012 12:11 PM Aaron Marcuse-Kubitza

sql.py: Added with_explain_comment() and use it in run_query()

3264 07/09/2012 12:01 PM Aaron Marcuse-Kubitza

sql.py: run_query(): EXPLAIN output: Run explain() with log_level 1 higher than the query's log_level, so that low-level queries' EXPLAIN queries are not output when the queries themselves are not output. This also ensures that only level 2 (major) queries have the EXPLAIN logged (to introduce the query that is being run), to avoid cluttering the log output.

3263 07/09/2012 11:54 AM Aaron Marcuse-Kubitza

sql.py: explain(): Support custom log_level

View all revisions | View revisions

Also available in: Atom