Project

General

Profile

Statistics
| Revision:
Name Size Revision Age Author Comment
  _archive 1598 over 12 years Aaron Marcuse-Kubitza Moved _archive/tapir2flatClient/trunk/client/ t...
  analysis 3076 about 12 years Aaron Marcuse-Kubitza Added top-level analysis dir for range modeling
  bin 3147 about 12 years Aaron Marcuse-Kubitza csv2db: Vacuum table instead of just reanalyzin...
  config 272 over 12 years Aaron Marcuse-Kubitza Moved bien_password to new config dir
  inputs 3133 about 12 years Aaron Marcuse-Kubitza input.Makefile: Added import/steps.by_col.sql t...
  lib 3145 about 12 years Aaron Marcuse-Kubitza db_xml.py: partition_size: Turning partitioning...
  mappings 2529 about 12 years Aaron Marcuse-Kubitza mappings/DwC2-VegBIEN.specimens.csv: Removed _t...
  schemas 3070 about 12 years Aaron Marcuse-Kubitza schemas/vegbien.sql: Added location.location_co...
  to_do 2547 about 12 years Aaron Marcuse-Kubitza to_do/timeline.doc: Updated to reflect the mont...
Makefile 10.2 KB 2983 about 12 years Aaron Marcuse-Kubitza main Makefile: Added explicit schemas/temp/rein...
README.TXT 2.9 KB 3133 about 12 years Aaron Marcuse-Kubitza input.Makefile: Added import/steps.by_col.sql t...
map 1.21 KB 3140 about 12 years Aaron Marcuse-Kubitza top-level map: Added support for custom public ...

Latest revisions

# Date Author Comment
3147 06/28/2012 11:00 PM Aaron Marcuse-Kubitza

csv2db: Vacuum table instead of just reanalyzing it because for some reason reanalyzing it isn't enough to fix the cached row count (causing pgAdmin3 to report that the table needs to be vacuumed)

3146 06/28/2012 10:54 PM Aaron Marcuse-Kubitza

csv2db: Don't add indexes on the created table because they use up more disk space than the table itself and currently aren't used. (The import process adds indexes on each iteration's column subset instead.)

3145 06/28/2012 10:21 PM Aaron Marcuse-Kubitza

db_xml.py: partition_size: Turning partitioning back on (with a larger limit), since the largest datasources' temp tables are still too big

3144 06/28/2012 10:20 PM Aaron Marcuse-Kubitza

sql_io.py: put_table(): Fixed bug where if there were multiple unique constraints that were violated, only the distinct temp table for the last one would get garbage-collected

3143 06/28/2012 09:01 PM Aaron Marcuse-Kubitza

db_xml.py: partition_size: Set to sys.maxint to disable partitioning. The last bugfix, which avoided returning a large result set to the client which was never read, seems to have fixed the disk space leak, so it's worth reattempting a full simultaneous import.

3142 06/28/2012 08:30 PM Aaron Marcuse-Kubitza

db_xml.py: put_table(): Subsetting in_table: Truncate in_table when finished with it, to avoid temp table disk space leaks

3141 06/28/2012 07:56 PM Aaron Marcuse-Kubitza

sql.py: insert_select(): If caller is only interested in the rowcount (if returning == None), keep the NULL rows for each insert on the server using CREATE TABLE AS. (CREATE TABLE AS sets rowcount to # rows in query, so rowcount will still be set correctly.)

3140 06/28/2012 04:59 PM Aaron Marcuse-Kubitza

top-level map: Added support for custom public schema, to be able to run imports and tests simultaneously (e.g. on a dev machine)

3139 06/27/2012 10:56 PM Aaron Marcuse-Kubitza

csv2db: Fixed bug where table needed to be a sql_gen.Table object with the proper schema, so that errors_table would be created in the correct schema. Removed no longer needed changing of the search_path.

3138 06/27/2012 10:55 PM Aaron Marcuse-Kubitza

csv2db: Fixed bug where table needed to be a sql_gen.Table object with the proper schema, so that errors_table would be created in the correct schema. Removed no longer needed changing of the search_path.

View all revisions | View revisions

Also available in: Atom