moved everything into /trunk/ to create the standard svn layout, for use with tools that require this (eg. git-svn). IMPORTANT: do NOT do an `svn up`. instead, re-use your working copy's existing files with `svn switch` (http://svnbook.red-bean.com/en/1.6/svn.ref.svn.c.switch.html).
xml_func.py: Removed no longer used _range(). Use SQL functions._range() instead.
xml_func.py: Removed no longer used _avg(). Use SQL functions._avg() instead.
schemas/functions.sql: Added _avg()
Moved parse_range() from xml_func.py to units.py
xml_func.py: Simplifying functions: Merging: Added _merge_prefix() passthru
xml_func.py: Simplifying functions: Logic: _and(), _or(): Evaluate an expression of only constant values
xml_func.py: is_scalar(): Fixed bug where need to check if value is a string before calling is_var_name()
xml_func.py: _first(): Fixed bug where need to choose the first non-empty param, by first pruning empty child nodes
xml_func.py: Simplifying functions: Added _eq()
xml_func.py: Added is_scalar()
xml_func.py: process(): row-based mode: preserving complex funcs: Fixed bug where functions with no params would crash reduce() because it requires at least one value when no initial value is specified
xml_func.py: Simplifying functions: Added passthru entries for _and, _or
xml_func.py: _env(): Fixed bug where need to retrieve actual string value of name param using xml_dom.NodeTextEntryIter instead of NodeEntryIter
xml_func.py: _env(): Fixed bug where need to use xml_dom.replace_with_text() instead of xml_dom.replace() because replace() requires a DOM node
xml_func.py: Simplifying functions: Added _env()
xml_func.py: Simplifying functions: Added _nullIf(), to remove calls with no null value
xml_func.py: Removed no longer used _map(), which has been replaced by a corresponding DB function
xml_func.py: process(): Pass on_error through to sql_io.put(). This fixes a bug in row-based import where DB errors in the xml_func.process() phase would abort the entire import instead of being tracked and having the return value set to None.
xml_func.py: Simplifying functions: Added _min, _max as passthroughs
xml_func.py: process(): Fixed bug where need to preserve complex functions that have unevaluated XML nodes as arguments, because XML nodes are not accepted by sql_io.put() (they are handled by db_xml.put())
xml_func.py: Removed no longer used _name(). Use _join_words() instead.
xml_func.py: Simplifying functions: Merging: Added _join_words()
xml_func.py: simplify(): Run xml_dom.prune_empty() on function nodes that don't have an explicit simplifying function. This allows single-arg functions with no arg to be pruned rather than called with no args (causing errors if the single param does not have a default value).
xml_func.py: _if(): Documented that it must be run to remove conditions that functions._if() can't handle
xml_func.py: Added _first() simplifying function
xml_func.py: Added helper functions variadic_args() and map_names()
xml_func.py: Added _join() simplifying function
xml_func.py: passthru(): Use xml_dom.prune() so that after empty children are removed, the node itself is also removed if it's empty. This enables further pruning of any node that contains the pruned node.
xml_func.py: Removed no longer used prune() (use xml_dom.prune_children() instead)
xml_func.py: Use new xml_dom.prune_children()
xml_func.py: simplify(): Removed no longer needed pass-through optimizations for XML functions, which are now handled by each function's own simplifying function
xml_func.py: Added _name simplifying function
xml_func.py: Added _alt, _merge simplifying functions
xml_func.py: passthru(): First prune the node
xml_func.py: simplify(): Use new passthru()
xml_func.py: Added passthru()
xml_func.py: simplify(): Use new prune()
xml_func.py: Added prune()
xml_func.py: simplify(): Removed no longer needed hardcoded _if simplifying code now that there is an _if() simplifying function
xml_func.py: Added _if() simplifying function
xml_func.py: Added is_var_name() and is_var()
xml_func.py: Added _exists()
xml_func.py: simplify(): Added support for custom simplifying functions, which are not hard-coded in simplify()
xml_func.py: simplify(): Also remove _if statements with only a condition. This is a required transformation, because such _if statements can't be handled by functions._if() due to there being no argument to provide the anyelement type.
xml_func.py: simplify(): Added pruning optimization that removes empty children. Empty children are created when some mappings don't apply to the current datasource.
xml_func.py: simplify(): Only generate children list if node is a function
xml_func.py: simplify(): Refactored to support processing nodes that are not functions. Changed var names for clarity.
xml_func.py: simplify(): Also consider _name() to be an aggregate function
xml_func.py: Removed no longer used _collapse()
xml_func.py: Removed no longer needed _if(), which has been translated to a SQL function
xml_func.py: XML function specification documentation: Updated parameters
xml_func.py: Removed no longer needed _eq(), which has been translated to a SQL function
xml_func.py: simplify(): Apply pass-through optimizations for _if statements with no condition (which means false). This faciliates automated testing after an _if statement has been added, because the put template provided as part of the automated test will only change for those datasources that actually have a condition entry for the _if statement, which greatly reduces the number of tests that need to be accepted. (Note that the path before the _if will still be included as an empty path if there are no other mappings to that table, because the _if statement does not surround it.)
xml_func.py: Added simplify()
xml_func.py: _map(): Don't remove None params, because are valid values and must be supported. This will become an issue once empty strings in text nodes are considered equivalent to None.
xml_func.py: _units(): Don't remove None params, because are valid values and must be supported. This will become an issue once empty strings in text nodes are considered equivalent to None.
xml_func.py: _name(): Fixed bug where needed to pass None values through and handle no name parts to properly support NULL propagation
xml_func.py: process(): Removed support for no longer used structural functions
xml_func.py: Removed no longer used structural functions
xml_func.py: _if(): Changed documentation about name param for distinguishing separate _if statements to use @name attribute instead, so that the XML/SQL function mechanism doesn't have to deal with code that's solely for XPath merging
lib/xml_func.py: _if(): Made then param optional, so that user can just map to the else branch as a shortcut for logically inverting the condition. (Note that a _not() XML function does not exist yet, so this is also a workaround.)
xml_func.py: _if(): Documented that can add `name` param to distinguish separate _if statements
xml_func.py: _if(): Made cond optional. When it's not specified or None, it is treated as False. This supports cases where all elements of the condition are required but not mapped to.
xml_func.py: Removed no longer used _simplifyPath(), which is now a built-in function of db_xml.put()
xml_func.py: _eq(): Documented that '' (empty node) is returned if a value was not mapped to, not if a value was None, since None arguments are no longer removed by process() (now XML functions do this manually with conv_items())
xml_func.py: _ref(): Only display "XPath reference target missing" warning if target node does not exist, not if it exists but is empty
xml_func.py: process(): In row-based mode, when trying to evaluate function using DB, preserve unknown funcs because these might be built-in functions of db_xml.put(). The sql.DoesNotExistException should be raised again when db_xml.put() is run and it verifies whether the function is built-in or not (e.g. _simplifyPath is now built-in, for column-based support). See db_xml.put_special_funcs for built-in functions.
xml_func.py: Removed no longer needed local XML functions that have been translated to SQL functions
Use new exc.reraise() where exc.raise_() was used, so that the stack trace is preserved when the exception is rethrown
xml_func.py: process(): If local XML function can't be found, just replace with last param instead of returning an error. This allows DB-only functions to be ignored in XML output mode.
xml_func.py: Use conv_items() in every XML function that needs empty (NULL) entries removed, so that they are not dependent on what process() does to the items
xml_func.py: process(): In row-based mode, if function is not explicitly a relational function but does not exist as a local XML function, treat it as a relational function. This will help in merging sql_io.put() and put_table(), since put() did not support SQL functions but put_table() does, and this ensures that a SQL function is always used if the local XML function has been removed in favor of it.
xml_func.py: process(): Changed rel_funcs param to a callback is_rel_func, so that caller can specify any dynamic function to determine if a name is a relational function rather than having to list out all known relational functions
xml_func.py: process(): Merge mergeable siblings recursively so that newly-mergeable children are also merged
xml_func.py: process(): Recombining pieces of nodes that were split apart in the mappings: Only combine nodes that are explicitly marked as mergeable, to avoid unwanted merges. Refactored to use new xml_dom.merge_by_name().
xml_func.py: process(): List out the xml_dom.NodeElemIter iterator so that it won't be affected by concurrent changes to the DOM tree
xml_func.py: process(): Merging children: Support merging non-adjacent children by using new xml_dom.merge_same_name(). This should eliminate the need to sort children of the same name next to each other in the mappings so that they will be merged.
xml_func.py: process(): Fixed bug where an evaluated XML function might create a node of the same name as an existing node, but these nodes would not be merged even though they referred to the same object, by merging siblings of a newly-evaluated (replaced) node if they have the same name
Moved Heuristic queries from sql.py to new sql_io.py
xml_func.py: _noCV: Fixed bug where assumed items was an iterator when it's now a list
xml_func.py: process(): Added pass-through optimization for aggregating functions with one arg
xml_func.py: process(): Call list() on the items' iterator so that its length can be determined wherever it's used without needing to call list() on it separately
Removed extra trailing whitespace
xml_func.py: process(): Refactored to emphasize special handling for row-based and column-based modes. In row-based mode, always use a DB relational function over a local XML function when possible, to faciliate testing of DB relational functions in row-based mode. (The shadowed local XML version will still be tested in non-DB modes, such as outputting to intermediate XML files.)
xml_func.py: Removed no longer used strip()
xml_func.py: process(): Added strip()'s functionality via strip option
xml_func.py: Replaced xpath.get_1() with xpath.get_value() where possible, for simplicity
xml_func.py: strip(): Evaluate structural functions like _ignore and _ref by process() instead of removing them. Store structural functions' names in structural_funcs module var. This ensures that _ref targets are still expanded in column-based import.
xml_func.py: strip(): Added preserve param for XML functions not to remove
xml_func.py: Added is_func*() and is_xml_func*() and use them where their definitions were used
xml_func.py: process(): If DB with relational functions available (passed in via db param), call any non-local XML functions as relational funcs
xml_func.py: Added _ref to retrieve a value from another XML node
xml_func.py: Made all functions take a 2nd node param, which contains the func node itself
xml_func.py: _label: return None on empty input
xml_func.py: Added _collapse to collapse a subtree if the "value" element in it is NULL
xml_func.py: _simplifyPath: Also treat the elem as empty if the required node exists but is empty
xml_func.py: strip(): Remove _ignore XML funcs completely instead of replacing them with their values
xml_func.py: Added strip(). pop_value(): Support custom name of value param.
