Project

General

Profile

Statistics
| Revision:

# Date Author Comment
2778 06/12/2012 07:14 PM Aaron Marcuse-Kubitza

schemas/postgresql.nimoy.conf: shared_buffers: Fixed syntax error where decimals were not supported

2777 06/12/2012 07:03 PM Aaron Marcuse-Kubitza

sql.py: truncate(): Re-added support for string tables using sql_gen.as_Table(). This fixes empty_db(), which relied on this functionality.

2776 06/12/2012 07:01 PM Aaron Marcuse-Kubitza

sql_gen.py: as_Table(): Added schema param to use as default schema

2775 06/12/2012 06:54 PM Aaron Marcuse-Kubitza

inputs/SALVIAS: Switched to using CSV exports of the DB, so that staging tables could be created for column-based import

2774 06/12/2012 06:48 PM Aaron Marcuse-Kubitza

sql.py: run_query_into(): Added add_indexes_ param which causes the function to add indexes on the created table

2773 06/12/2012 06:18 PM Aaron Marcuse-Kubitza

sql.py: create_table(): Use new add_indexes()

2772 06/12/2012 06:15 PM Aaron Marcuse-Kubitza

sql.py: Added add_indexes()

2771 06/12/2012 05:38 PM Aaron Marcuse-Kubitza

sql.py: get_cur_query(): Fixed bug where strings.ustr() needed to be used instead of str() when ensuring that get_cur_query() returns a string

2770 06/12/2012 05:35 PM Aaron Marcuse-Kubitza

sql.py: cast(): Removed conditional checks for save_errors, since it's now always true if the function got passed the `not save_errors` special case

2769 06/12/2012 05:32 PM Aaron Marcuse-Kubitza

sql.py: cast(): Only convert errors to warnings if errors will be saved in errors_table, so that import will always be aborted if user supplied invalid values in the mappings, even if these values are passed through a relational function

2768 06/12/2012 04:58 PM Aaron Marcuse-Kubitza

sql.py: put_table(): Support inserting tables with all default values, by providing the pkey's default value for all rows so that the SELECT query has at least one column

2767 06/12/2012 04:57 PM Aaron Marcuse-Kubitza

sql_gen.py: is_table_col(): Check that input is a Col object

2766 06/12/2012 04:31 PM Aaron Marcuse-Kubitza

sql.py: put_table(): Assert that mapping is non-empty

2765 06/12/2012 04:23 PM Aaron Marcuse-Kubitza

sql.py: mk_select(): Assert that fields list is non-empty

2764 06/12/2012 04:18 PM Aaron Marcuse-Kubitza

sql.py: DbConn.DbCursor.execute(): Set _is_insert only if query starts with INSERT, so that function definitions containing INSERT are not cached as INSERT statements (exceptions only) themselves

2763 06/12/2012 04:10 PM Aaron Marcuse-Kubitza

sql.py: DbConn.DbCursor.execute(): Fixed bug where params == None would apparently turn off the mogrifier completely, causing "%"s to be excessively escaped, by just setting params to None if it was [] or () and not using strings.esc_for_mogrify() at all

2762 06/12/2012 03:51 PM Aaron Marcuse-Kubitza

sql.py: DbConn.DbCursor.execute(): If not using params, escape the query using strings.esc_for_mogrify() in case any literals contained "%"s

2761 06/12/2012 03:49 PM Aaron Marcuse-Kubitza

strings.py: Added esc_for_mogrify()

2760 06/12/2012 03:05 PM Aaron Marcuse-Kubitza

sql.py: create_table(): Add indexes on all non-pkey columns, unless turned off or deferred using new param col_indexes

2759 06/12/2012 02:46 PM Aaron Marcuse-Kubitza

csv2db: Add column indexes on errors table. Use typed_cols and `.to_Col()` to iterate over columns to add indexes on, for the main and errors tables.

2758 06/11/2012 11:13 PM Aaron Marcuse-Kubitza

sql.py: Added track_data_error(). put_table(): ignore(): Take extra e param for the exception. Use track_data_error() to store the invalid value in the errors table.

2757 06/11/2012 11:11 PM Aaron Marcuse-Kubitza

sql_gen.py: Join.to_str(): Add newline before and after right table if it's been renamed (and therefore takes up multiple lines)

2756 06/11/2012 11:10 PM Aaron Marcuse-Kubitza

exc.py: ExceptionWithCause: Store the cause in an instance variable for later use

2755 06/11/2012 10:47 PM Aaron Marcuse-Kubitza

sql.py: mk_track_data_error(): Rename the errors_table to make the generated SQL less verbose

2754 06/11/2012 10:42 PM Aaron Marcuse-Kubitza

sql.py: mk_insert_select(): Run sql_gen.remove_table_rename() on table to get just the actual name in the DB

2753 06/11/2012 10:42 PM Aaron Marcuse-Kubitza

sql_gen.py: Added remove_table_rename()

2752 06/11/2012 10:40 PM Aaron Marcuse-Kubitza

sql_gen.py: Col: Run `.to_Table()` on table to get just the reference to the table, not any SQL code that defines it

2751 06/11/2012 10:10 PM Aaron Marcuse-Kubitza

sql.py: Added mk_track_data_error() and use it in cast(). This also ensures that if only one source column's row in the CROSS JOIN violates a unique constraint, other source columns' rows are still inserted.

2750 06/11/2012 09:59 PM Aaron Marcuse-Kubitza

sql_gen.py: with_default_table(): Added overwrite param to overwrite the table (if it isn't a NamedCol)

2749 06/11/2012 09:57 PM Aaron Marcuse-Kubitza

sql_gen.py: Join.to_str(): join(): Get just the table name of left_table and right_table using `.to_Table()`. Moved order switching of tables inside join() because the order reversing only applies to an individual condition.

2748 06/11/2012 09:34 PM Aaron Marcuse-Kubitza

sql_gen.py: Renamed set_default_table() to with_default_table() and copy col before modifying it so don't modify input

2747 06/11/2012 09:14 PM Aaron Marcuse-Kubitza

sql_gen.py: Added set_default_table(). as_ValueCond(): Use set_default_table() instead of as_Col() so that any name-only column also gets its table set. Join.to_str(): Parse left side using set_default_table() instead of as_Col() so that any name-only column also gets its table set.

2746 06/11/2012 07:31 PM Aaron Marcuse-Kubitza

sql_gen.py: Join: mapping param defaults to {} for e.g. CROSS JOINs. to_str(): Omit join_cond if mapping is empty, rather than if join is a specific type.

2745 06/11/2012 07:24 PM Aaron Marcuse-Kubitza

sql_gen.py: NamedValues: Change cols to Col objects with the table set to `name`

2744 06/11/2012 07:23 PM Aaron Marcuse-Kubitza

sql_gen.py: Added set_cols_table()

2743 06/11/2012 07:00 PM Aaron Marcuse-Kubitza

sql.py: mk_insert_select(): returning: Use sql_gen.to_name_only_col()

2742 06/11/2012 06:52 PM Aaron Marcuse-Kubitza

sql_gen.py: NamedTable: cols: Use sql_gen.Col objects or name strings instead of pre-rendered SQL code

2741 06/11/2012 06:35 PM Aaron Marcuse-Kubitza

sql_gen.py: NamedTable: Wrap nested code in Expr if needed

2740 06/11/2012 06:27 PM Aaron Marcuse-Kubitza

sql_gen.py: Added NamedValues

2739 06/11/2012 06:04 PM Aaron Marcuse-Kubitza

sql_gen.py: Values: Support multiple rows

2738 06/11/2012 05:54 PM Aaron Marcuse-Kubitza

sql.py: insert(): Use new sql_gen.Values

2737 06/11/2012 05:53 PM Aaron Marcuse-Kubitza

sql_gen.py: Added Values and default

2736 06/11/2012 05:26 PM Aaron Marcuse-Kubitza

sql_gen.py: Join.to_str(): Don't add join condition for CROSS JOINs

2735 06/11/2012 05:03 PM Aaron Marcuse-Kubitza

sql.py: put_table(): Factored out errors_table name setting so it can be used by ignore()

2734 06/11/2012 04:38 PM Aaron Marcuse-Kubitza

bin/map: If doing full import, clear errors table

2733 06/11/2012 04:37 PM Aaron Marcuse-Kubitza

sql.py: truncate(): Support sql_gen.Table objects

2732 06/11/2012 04:21 PM Aaron Marcuse-Kubitza

sql.py: Moved truncate() to Database structure queries section

2731 06/11/2012 04:11 PM Aaron Marcuse-Kubitza

sql.py: tables(): Run query with log_level=4 because it's a low-level structure-determining query

2730 06/11/2012 04:07 PM Aaron Marcuse-Kubitza

sql.py: table_exists(): Use new tables() exact param so that LIKE special chars in table name are not interpreted specially

2729 06/11/2012 04:07 PM Aaron Marcuse-Kubitza

sql.py: tables(): Added exact param to check for exact matches only

2728 06/11/2012 04:03 PM Aaron Marcuse-Kubitza

sql.py: put_table(): MissingCastException: Use new errors_table()

2727 06/11/2012 04:02 PM Aaron Marcuse-Kubitza

csv2db: Use new sql.errors_table()

2726 06/11/2012 04:02 PM Aaron Marcuse-Kubitza

sql.py: Added table_exists() and errors_table()

2725 06/08/2012 09:59 PM Aaron Marcuse-Kubitza

sql.py: DbConn.print_notices(): Fixed bug where it should not do anything for a MySQL connection, because that doesn't store notices the way Postgres does

2724 06/08/2012 09:54 PM Aaron Marcuse-Kubitza

sql.py: put_table(): MissingCastException: Debug message: Added Redmine formatting

2723 06/08/2012 09:48 PM Aaron Marcuse-Kubitza

schemas/functions.sql, vegbien.sql: Removed no longer needed cast functions, which are now created on the fly by column-based import

2722 06/08/2012 09:45 PM Aaron Marcuse-Kubitza

schemas/functions.sql: _nullIf(): Ignore uncastable value, because a value that's invalid for the given type is still well-defined as not matching the nullif() criterion

2721 06/08/2012 09:25 PM Aaron Marcuse-Kubitza

sql.py: put_table(): MissingCastException: Debug message: Removed "'s" so it wouldn't mess up syntax highlighting when pasting debug output into a SQL file

2720 06/08/2012 09:08 PM Aaron Marcuse-Kubitza

sql.py: cast(): Made errors table also store SQLSTATE in error_code column

2719 06/08/2012 08:40 PM Aaron Marcuse-Kubitza

sql.py: cast(): Documented that the value and error are inserted for each source column (hence the CROSS JOIN)

2718 06/08/2012 08:39 PM Aaron Marcuse-Kubitza

sql.py: cast(): Version the function name if using an errors table, to avoid collisions with other cast functions when the function name is truncated (or, more rarely, collisions with casts to the same type and on the same input columns but of a different table)

2717 06/08/2012 08:25 PM Aaron Marcuse-Kubitza

sql.py: cast(): function_name: Fixed bug where sql_gen.FunctionCall()'s parameters needed to be passed with *args syntax

2716 06/08/2012 08:23 PM Aaron Marcuse-Kubitza

sql.py: mk_flatten_mapping(): Propagate the original columns' sources to the flattened columns so they won't be lost in the flattening

2715 06/08/2012 08:20 PM Aaron Marcuse-Kubitza

sql.py: put_table(): MissingCastException: Use in_tables0's source table to locate the errors table in case in_tables0 has been subset into a temp table (which removes the schema name)

2714 06/08/2012 08:19 PM Aaron Marcuse-Kubitza

db_xml.py: put_table(): Track in_table's source so its original schema can be obtained and auxiliary tables located

2713 06/08/2012 08:17 PM Aaron Marcuse-Kubitza

sql_gen.py: Derived.set_srcs(): Added optional overwrite param so that a default srcs value can be set only if one isn't already set

2712 06/08/2012 08:04 PM Aaron Marcuse-Kubitza

sql_gen.py: Made Table a Derived element so that a row-subset temp table could retain the schema of the table it came from, and any auxiliary tables in that schema could be located properly

2711 06/08/2012 08:01 PM Aaron Marcuse-Kubitza

sql_gen.py: Moved srcs-related functionality from Col to new superclass Derived

2710 06/08/2012 07:53 PM Aaron Marcuse-Kubitza

sql.py: cast(): save_errors: Fixed bug where srcs needed to have their names extracted before being wrapped in sql_gen.Literals. Fixed bug where errors table INSERT needed to prefix the CROSS JOIN-ed VALUES statements with SELECT * FROM because the CROSS JOIN makes it a whole SELECT query, not just a VALUES statement.

2709 06/08/2012 07:40 PM Aaron Marcuse-Kubitza

schemas/vegbien.ERD.mwb: Adjusted lines to make less intersections happen on the first page

2708 06/08/2012 07:27 PM Aaron Marcuse-Kubitza

sql.py: put_table(): MissingCastException: Fixed bug where errors_table needed to have the same schema as in_tables0 as well as part of the same name

2707 06/08/2012 07:23 PM Aaron Marcuse-Kubitza

sql_gen.py: Added suffixed_table()

2706 06/08/2012 07:14 PM Aaron Marcuse-Kubitza

sql.py: MissingCastException: Print log message that it's casting the column, to introduce the SQL function definition that follows

2705 06/08/2012 07:06 PM Aaron Marcuse-Kubitza

sql.py: put_table(): MissingCastException: Use new cast() instead of relying on existing cast functions in the database

2704 06/08/2012 07:04 PM Aaron Marcuse-Kubitza

sql.py: Added cast()

2703 06/08/2012 06:00 PM Aaron Marcuse-Kubitza

sql_gen.py: Added wrap() and use it in wrap_in_func()

2702 06/08/2012 05:36 PM Aaron Marcuse-Kubitza

sql.py: put_table(): Track the input column(s) a column is derived from, so that error messages can be attributes to the proper input column(s)

2701 06/08/2012 05:35 PM Aaron Marcuse-Kubitza

sql_gen.py: Col: Support tracking the column(s) a column is derived from, so that error messages can be attributes to the proper input column(s)

2700 06/08/2012 05:33 PM Aaron Marcuse-Kubitza

objects.py: BasicObject: Allow subclasses to customize which attrs are compared on, by adding _compare_on() method

2699 06/08/2012 04:45 PM Aaron Marcuse-Kubitza

lists.py: uniqify(): Document that it will work on any iterable, not just lists

2698 06/08/2012 04:06 PM Aaron Marcuse-Kubitza

sql.py: mk_insert_select(): embeddable: Use sql_gen.TempFunction and sql_gen.FunctionCall

2697 06/08/2012 04:04 PM Aaron Marcuse-Kubitza

sql_gen.py: Added TempFunction

2696 06/08/2012 03:26 PM Aaron Marcuse-Kubitza

schemas/functions.sql, vegbien.sql: Cast functions: Removed `RETURN new;` at end (artifact of when they were relational functions). Made the EXCEPTION block the main block of the function, to avoid unnecessary nesting.

2695 06/08/2012 02:26 PM Aaron Marcuse-Kubitza

csv2db: Errors table: index_cols: Remove no longer needed sql_gen.Col() (now done by EnsureNotNull)

2694 06/08/2012 02:25 PM Aaron Marcuse-Kubitza

sql_gen.py: EnsureNotNull: Run value through as_Col() so FunctionCall won't default it to a Literal

2693 06/08/2012 02:19 PM Aaron Marcuse-Kubitza

csv2db: Use sql_gen.EnsureNotNull instead of the ensure_not_null() function in the functions schema to avoid a dependency on the functions schema, which would cause the UNIQUE index to be dropped whenever the functions schema is reinstalled

2692 06/08/2012 02:18 PM Aaron Marcuse-Kubitza

sql_gen.py: Added EnsureNotNull

2691 06/08/2012 02:17 PM Aaron Marcuse-Kubitza

sql_gen.py: Added InternalFunction

2690 06/08/2012 02:08 PM Aaron Marcuse-Kubitza

sql_gen.py: FunctionCall: Ensure all args are Code objects using as_Value()

2689 06/08/2012 01:51 PM Aaron Marcuse-Kubitza

csv2db: Errors table: Add UNIQUE index on all columns

2688 06/08/2012 01:50 PM Aaron Marcuse-Kubitza

sql.py: add_index(): Support multiple column(s) or expression(s). Support separate table (not extracted from cols). Support UNIQUE indexes.

2687 06/07/2012 09:41 PM Aaron Marcuse-Kubitza

sql.py: add_index(): Fixed bug where expr needed to be deep copied so that any column nested in it (e.g. inside a FunctionCall) wouldn't be modified when col.table is set to None

2686 06/07/2012 09:35 PM Aaron Marcuse-Kubitza

sql.py: add_pkey(): Support multiple, custom columns

2685 06/07/2012 09:24 PM Aaron Marcuse-Kubitza

csv2db: Vacuum the created table

2684 06/07/2012 09:23 PM Aaron Marcuse-Kubitza

sql.py: Added vacuum()

2683 06/07/2012 09:23 PM Aaron Marcuse-Kubitza

sql.py: DbConn: Added with_autocommit()

2682 06/07/2012 08:58 PM Aaron Marcuse-Kubitza

csv2db: Create errors table for use by column-based import

2681 06/07/2012 08:57 PM Aaron Marcuse-Kubitza

sql.py: create_table(): Added has_pkey param to disable making the first column the primary key

2680 06/07/2012 08:21 PM Aaron Marcuse-Kubitza

csv2db: Use verbosity-based logging like bin/map. Use sql.create_table(). Add indexes on the columns to speed up column-based import and to speed up searching the table for particular values.

2679 06/07/2012 08:00 PM Aaron Marcuse-Kubitza

sql.py: create_table(): Don't add indexes on columns, because that shouldn't happen until after the table's rows have been inserted