Project

General

Profile

Statistics
| Revision:
Name Size Revision Age Author Comment
  _archive 1598 almost 13 years Aaron Marcuse-Kubitza Moved _archive/tapir2flatClient/trunk/client/ t...
  analysis 3076 over 12 years Aaron Marcuse-Kubitza Added top-level analysis dir for range modeling
  bin 3103 over 12 years Aaron Marcuse-Kubitza Moved error tracking from sql.py to sql_io.py
  config 272 about 13 years Aaron Marcuse-Kubitza Moved bien_password to new config dir
  inputs 3072 over 12 years Aaron Marcuse-Kubitza inputs/SpeciesLink/src: Added custom header tha...
  lib 3114 over 12 years Aaron Marcuse-Kubitza db_xml.py: put_table(): Partition in_table if l...
  mappings 2529 over 12 years Aaron Marcuse-Kubitza mappings/DwC2-VegBIEN.specimens.csv: Removed _t...
  schemas 3070 over 12 years Aaron Marcuse-Kubitza schemas/vegbien.sql: Added location.location_co...
  to_do 2547 over 12 years Aaron Marcuse-Kubitza to_do/timeline.doc: Updated to reflect the mont...
Makefile 10.2 KB 2983 over 12 years Aaron Marcuse-Kubitza main Makefile: Added explicit schemas/temp/rein...
README.TXT 2.76 KB 2976 over 12 years Aaron Marcuse-Kubitza README.TXT: Changed documentation to use make s...
map 978 Bytes 1979 over 12 years Aaron Marcuse-Kubitza root map: Fill in defaults for inputs from VegB...

Latest revisions

# Date Author Comment
3114 06/27/2012 05:32 PM Aaron Marcuse-Kubitza

db_xml.py: put_table(): Partition in_table if larger than a threshold. The threshold is initially set to disable partitioning. Partitioning will hopefully eliminate the excessive disk usage for large input tables, which has caused the system to run out of disk space due to what may be a bug in PostgreSQL.

3113 06/27/2012 05:27 PM Aaron Marcuse-Kubitza

db_xml.py: put_table(): Set in_table's default srcs to in_table itself instead of sql_gen.src_self, so that any copies of in_table will inherit the same srcs instead of being treated as a top-level table. This ensures that the top-level table's errors table will always be used.

3112 06/27/2012 05:17 PM Aaron Marcuse-Kubitza

sql_io.py: cast(): Always convert exceptions to warnings if the input is a column or expression, even if there is no place to save the errors, so that invalid data does not need to be handled by the caller in a (much slower) extra exception-handling loop

3111 06/27/2012 04:47 PM Aaron Marcuse-Kubitza

sql_io.py: put_table(): MissingCastException: When casting, handle InvalidValueException by filtering out invalid values with invalid2null() in a loop

3110 06/27/2012 04:45 PM Aaron Marcuse-Kubitza

sql_io.py: cast_temp_col(): Run sql.update() in recover mode in case expr produces errors. Don't cache sql.update() in case this function will be called again after error recovery.

3109 06/27/2012 04:40 PM Aaron Marcuse-Kubitza

sql.py: Generalized FunctionValueException to InvalidValueException so that it will match all invalid-value errors, not just those occurring in user-defined functions

3108 06/27/2012 04:22 PM Aaron Marcuse-Kubitza

sql_io.py: put_table(): Removed no longer used sql.FunctionValueException handling, because type casting functions now do their own invalid value handling

3107 06/27/2012 03:44 PM Aaron Marcuse-Kubitza

db_xml.py: put_table(): Subsetting in_table: Call put_table() recursively using put_table_() to ensure that limit and start are reset to their default values, in case the table gets partitioned (which needs up-to-date limit and start values)

3106 06/27/2012 03:14 PM Aaron Marcuse-Kubitza

sql_io.py: put_table(): mk_main_select(): Fixed bug where the table of each cond needed to be changed to insert_in_table because mk_main_select() uses the distinct table rather than the full input table

3105 06/27/2012 03:12 PM Aaron Marcuse-Kubitza

sql_gen.py: with_table(): Support columns that are wrapped in a FunctionCall object

View all revisions | View revisions

Also available in: Atom