Project

General

Profile

Statistics
| Revision:
Name Size Revision Age Author Comment
  _archive 1598 over 12 years Aaron Marcuse-Kubitza Moved _archive/tapir2flatClient/trunk/client/ t...
  analysis 3076 over 12 years Aaron Marcuse-Kubitza Added top-level analysis dir for range modeling
  bin 3149 over 12 years Aaron Marcuse-Kubitza csv2db: Fixed bug where CREATE TABLE statement ...
  config 272 almost 13 years Aaron Marcuse-Kubitza Moved bien_password to new config dir
  inputs 3133 over 12 years Aaron Marcuse-Kubitza input.Makefile: Added import/steps.by_col.sql t...
  lib 3148 over 12 years Aaron Marcuse-Kubitza sql.py: add_index(): Don't generate a unique na...
  mappings 2529 over 12 years Aaron Marcuse-Kubitza mappings/DwC2-VegBIEN.specimens.csv: Removed _t...
  schemas 3070 over 12 years Aaron Marcuse-Kubitza schemas/vegbien.sql: Added location.location_co...
  to_do 2547 over 12 years Aaron Marcuse-Kubitza to_do/timeline.doc: Updated to reflect the mont...
Makefile 10.2 KB 2983 over 12 years Aaron Marcuse-Kubitza main Makefile: Added explicit schemas/temp/rein...
README.TXT 2.9 KB 3133 over 12 years Aaron Marcuse-Kubitza input.Makefile: Added import/steps.by_col.sql t...
map 1.21 KB 3140 over 12 years Aaron Marcuse-Kubitza top-level map: Added support for custom public ...

Latest revisions

# Date Author Comment
3149 06/28/2012 11:22 PM Aaron Marcuse-Kubitza

csv2db: Fixed bug where CREATE TABLE statement was cached, causing it not to be re-executed after a rollback due to a failed COPY FROM. Avoid re-creating the table after a failed COPY FROM, and instead just remove any existing rows.

3148 06/28/2012 11:09 PM Aaron Marcuse-Kubitza

sql.py: add_index(): Don't generate a unique name for the index because the database does that automatically

3147 06/28/2012 11:00 PM Aaron Marcuse-Kubitza

csv2db: Vacuum table instead of just reanalyzing it because for some reason reanalyzing it isn't enough to fix the cached row count (causing pgAdmin3 to report that the table needs to be vacuumed)

3146 06/28/2012 10:54 PM Aaron Marcuse-Kubitza

csv2db: Don't add indexes on the created table because they use up more disk space than the table itself and currently aren't used. (The import process adds indexes on each iteration's column subset instead.)

3145 06/28/2012 10:21 PM Aaron Marcuse-Kubitza

db_xml.py: partition_size: Turning partitioning back on (with a larger limit), since the largest datasources' temp tables are still too big

3144 06/28/2012 10:20 PM Aaron Marcuse-Kubitza

sql_io.py: put_table(): Fixed bug where if there were multiple unique constraints that were violated, only the distinct temp table for the last one would get garbage-collected

3143 06/28/2012 09:01 PM Aaron Marcuse-Kubitza

db_xml.py: partition_size: Set to sys.maxint to disable partitioning. The last bugfix, which avoided returning a large result set to the client which was never read, seems to have fixed the disk space leak, so it's worth reattempting a full simultaneous import.

3142 06/28/2012 08:30 PM Aaron Marcuse-Kubitza

db_xml.py: put_table(): Subsetting in_table: Truncate in_table when finished with it, to avoid temp table disk space leaks

3141 06/28/2012 07:56 PM Aaron Marcuse-Kubitza

sql.py: insert_select(): If caller is only interested in the rowcount (if returning == None), keep the NULL rows for each insert on the server using CREATE TABLE AS. (CREATE TABLE AS sets rowcount to # rows in query, so rowcount will still be set correctly.)

3140 06/28/2012 04:59 PM Aaron Marcuse-Kubitza

top-level map: Added support for custom public schema, to be able to run imports and tests simultaneously (e.g. on a dev machine)

View all revisions | View revisions

Also available in: Atom