Project

General

Profile

Statistics
| Revision:
Name Size Revision Age Author Comment
  _archive 1598 almost 13 years Aaron Marcuse-Kubitza Moved _archive/tapir2flatClient/trunk/client/ t...
  analysis 3076 over 12 years Aaron Marcuse-Kubitza Added top-level analysis dir for range modeling
  backups 4751 over 12 years Aaron Marcuse-Kubitza backups/Makefile: Backups: Full DB: Specify the...
  bin 5028 over 12 years Aaron Marcuse-Kubitza csv2db: Removed no longer used has_row_num param
  config 272 about 13 years Aaron Marcuse-Kubitza Moved bien_password to new config dir
  inputs 5030 over 12 years Aaron Marcuse-Kubitza bin/map, db_xml.put_table() (row-based and colu...
  lib 5027 over 12 years Aaron Marcuse-Kubitza sql_io.py: import_csv(): Don't add a row number...
  mappings 5014 over 12 years Aaron Marcuse-Kubitza mappings/VegCore-VegBIEN.csv: Use new, more gen...
  schemas 5009 over 12 years Aaron Marcuse-Kubitza schemas/functions.sql: Added _join_words()
  to_do 4524 over 12 years Aaron Marcuse-Kubitza to_do/timeline.doc: Updated to reflect addition...
  validation 4523 over 12 years Aaron Marcuse-Kubitza Added validation/
Makefile 9.99 KB 4752 over 12 years Aaron Marcuse-Kubitza root Makefile: PostgreSQL: postgres-Linux: Adde...
README.TXT 11.3 KB 5016 over 12 years Aaron Marcuse-Kubitza README.TXT: Data import: `make test by_col=1`: ...
map 1.28 KB 4981 over 12 years Aaron Marcuse-Kubitza root map: Fixed custom public schema override t...
new_terms.csv 30.4 KB 4887 over 12 years Aaron Marcuse-Kubitza Regenerated root unmapped_terms.csv, new_terms.csv
unmapped_terms.csv 5.8 KB 4887 over 12 years Aaron Marcuse-Kubitza Regenerated root unmapped_terms.csv, new_terms.csv

Latest revisions

# Date Author Comment
5030 09/27/2012 12:28 AM Aaron Marcuse-Kubitza

bin/map, db_xml.put_table() (row-based and column-based import): Don't sort the input table by its pkey, in order to support input tables with no pkey. Note that reading the input table in table order and having this match the input flat file's order is only possible with sql_io.import_csv()'s truncation of the table on a failed import, which ensures that the rows will be stored in inserted order.

5029 09/27/2012 12:19 AM Aaron Marcuse-Kubitza

input.Makefile: Staging tables installation: Removed no longer used $(isJoinedTable). Note that it is no longer necessary for joined tables to be suffixed with ".src" to prevent the creation of a row_num column, which collided during joins.

5028 09/27/2012 12:17 AM Aaron Marcuse-Kubitza

csv2db: Removed no longer used has_row_num param

5027 09/27/2012 12:14 AM Aaron Marcuse-Kubitza

sql_io.py: import_csv(): Don't add a row number column to the created table because it is now added automatically to the temp table by column-based import (row-based import now also does not require a pkey for DB inputs)

5026 09/26/2012 11:49 PM Aaron Marcuse-Kubitza

bin/map, db_xml.put_table() (row-based and column-based import): Don't sort the input table by its pkey, in order to support input tables with no pkey. Note that reading the input table in table order and having this match the input flat file's order is only possible with sql_io.import_csv()'s truncation of the table on a failed import, which ensures that the rows will be stored in inserted order.

5025 09/26/2012 11:34 PM Aaron Marcuse-Kubitza

sql_io.py: import_csv(): Only do the import in a savepoint if using COPY FROM, to allow autocommits after each insert and thus make rows visible immediately after they are inserted

5024 09/26/2012 10:53 PM Aaron Marcuse-Kubitza

db_xml.py: put_table(): Subsetting in_table: Add a row number column if in_table does not already have a pkey

5023 09/26/2012 10:48 PM Aaron Marcuse-Kubitza

db_xml.py: put_table(): Subsetting in_table: Copy all of in_table's structure, rather than just the column types, by using sql.copy_table_struct() and sql.insert_select(). This preserves pkeys and NOT NULL constraints, which are useful for column-based import.

5022 09/26/2012 10:47 PM Aaron Marcuse-Kubitza

db_xml.py: put_table(): Subsetting in_table: Create in_table as a completely new sql_gen.Table instead of copying full_in_table and relying on sql.run_query_into() to set is_temp and remove the schema

5021 09/26/2012 10:40 PM Aaron Marcuse-Kubitza

sql.py: add_row_num(): Use if_not_exists in order to abort if the column already exists rather than adding a version #

View all revisions | View revisions

Also available in: Atom