Project

General

Profile

Statistics
| Revision:
Name Size Revision Age Author Comment
  _archive 1598 almost 13 years Aaron Marcuse-Kubitza Moved _archive/tapir2flatClient/trunk/client/ t...
  analysis 3076 over 12 years Aaron Marcuse-Kubitza Added top-level analysis dir for range modeling
  backups 3701 over 12 years Aaron Marcuse-Kubitza backups/Makefile: Added synchronization of back...
  bin 3924 over 12 years Aaron Marcuse-Kubitza join: Added map_1_core_only option that uses on...
  config 272 about 13 years Aaron Marcuse-Kubitza Moved bien_password to new config dir
  inputs 3919 over 12 years Aaron Marcuse-Kubitza VegBIEN mappings: Wrapped dates in _date() and ...
  lib 3920 over 12 years Aaron Marcuse-Kubitza lib/xml_func.py: _if(): Made then param optiona...
  mappings 3922 over 12 years Aaron Marcuse-Kubitza mappings: Made VegCSV-VegBIEN.{plots,organisms,...
  schemas 3911 over 12 years Aaron Marcuse-Kubitza schemas/vegbien.sql: aggregateoccurrence: Added...
  to_do 2547 over 12 years Aaron Marcuse-Kubitza to_do/timeline.doc: Updated to reflect the mont...
Makefile 10.1 KB 3764 over 12 years Aaron Marcuse-Kubitza root Makefile, input.Makefile: Maps validation:...
README.TXT 9.03 KB 3845 over 12 years Aaron Marcuse-Kubitza README.TXT: After a new import: Added steps to ...
map 1.22 KB 3475 over 12 years Aaron Marcuse-Kubitza root map: Run bin/map with a nice increment of ...

Latest revisions

# Date Author Comment
3924 08/09/2012 02:45 PM Aaron Marcuse-Kubitza

join: Added map_1_core_only option that uses only columns 0 and 1 of map_1. This is useful for one-time refactoring joins where the Source column, mappings comments, etc. shouldn't be part of the datasource's via map (although they will be part of the autogenerated VegBIEN map)

3923 08/09/2012 02:33 PM Aaron Marcuse-Kubitza

join: Use opts.env_usage() for usage message

3922 08/09/2012 02:04 PM Aaron Marcuse-Kubitza

mappings: Made VegCSV-VegBIEN.{plots,organisms,stems}.csv symlinks to VegCSV-VegBIEN.specimens.csv

3921 08/09/2012 01:46 PM Aaron Marcuse-Kubitza

mappings/Makefile: VegCSV-VegBIEN.specimens.csv: Commented out combining with DwC2-VegBIEN mappings, because merging DwC and VegX/VegCSV into one map is a lower priority than replacing all datasource VegX mappings with VegCSV (which does not require the merging but does require XPaths that don't collide, which is not yet the case)

3920 08/09/2012 01:40 PM Aaron Marcuse-Kubitza

lib/xml_func.py: _if(): Made then param optional, so that user can just map to the else branch as a shortcut for logically inverting the condition. (Note that a _not() XML function does not exist yet, so this is also a workaround.)

3919 08/09/2012 01:29 PM Aaron Marcuse-Kubitza

VegBIEN mappings: Wrapped dates in _date() and _dateRangeStart()/_dateRangeEnd(), to assist in importing date and date range values that PostgreSQL cannot parse. This will increase the import time, but hopefully also decrease the # of invalid values in the errors tables. (These functions can later be optimized to reduce the impact on import time.)

3918 08/09/2012 01:25 PM Aaron Marcuse-Kubitza

sql_io.py: put_table(): is_literals: is_function: Fixed bug where function call needed to be recreated in each iteration of the main loop, because the arguments to the function, which are based on mapping, may change as the result of error handling replacing invalid values with NULL

3917 08/09/2012 01:13 PM Aaron Marcuse-Kubitza

sql_io.py: put_table(): is_literals: Fixed bug where sql.select() that calls the function needed to be run recoverably, to auto-rollback errors. Made sql.select() cacheable because SQL functions are immutable, so it should be idempotent.

3916 08/09/2012 01:03 PM Aaron Marcuse-Kubitza

mappings/DwC2-VegBIEN.specimens.csv: Remapped taxonRemarks to taxondetermination.notes because http://rs.tdwg.org/dwc/terms/#taxonRemarks indicates that these notes are "about the taxon", not the specimen/plant in general

3915 08/09/2012 12:56 PM Aaron Marcuse-Kubitza

mappings/DwC2-VegBIEN.specimens.csv: Remapped eventDate to new aggregateoccurrence.collectiondate, which is a more accurate place than locationevent.obsstartdate/obsenddate because the date refers to a specific specimen. This also makes eventDate compatible with plots data.

View all revisions | View revisions

Also available in: Atom