Project

General

Profile

Statistics
| Revision:
Name Size Revision Age Author Comment
  _archive 1598 almost 13 years Aaron Marcuse-Kubitza Moved _archive/tapir2flatClient/trunk/client/ t...
  bin 1878 over 12 years Aaron Marcuse-Kubitza bin/map: Share on_error with the pool
  config 272 about 13 years Aaron Marcuse-Kubitza Moved bien_password to new config dir
  inputs 1854 over 12 years Aaron Marcuse-Kubitza input.Makefile: Editing import: Use $(datasrc) ...
  lib 1882 over 12 years Aaron Marcuse-Kubitza collection.py: rmap(): Fixed bugs: Made it recu...
  mappings 1843 over 12 years Aaron Marcuse-Kubitza mappings: Build VegX-VegBIEN.organisms.csv from...
  schemas 1751 over 12 years Aaron Marcuse-Kubitza vegbien.ERD.mwb: Fixed lines
  to_do 811 almost 13 years Aaron Marcuse-Kubitza Added to_do/milestones.doc
Makefile 7.72 KB 1845 over 12 years Aaron Marcuse-Kubitza root Makefile: python-Linux: Added `sudo pip in...
README.TXT 2.58 KB 1774 over 12 years Aaron Marcuse-Kubitza README.TXT: Schema changes: Split updating grap...
map 867 Bytes 1299 almost 13 years Aaron Marcuse-Kubitza map: On nimoy, use bien2_staging unless otherwi...

Latest revisions

# Date Author Comment
1882 04/16/2012 04:40 PM Aaron Marcuse-Kubitza

collection.py: rmap(): Fixed bugs: Made it recursive. Use iters.is_iterable() instead of isinstance(value, list) to work on all iterables. Use value and not nonexistent var list_.

1881 04/16/2012 04:38 PM Aaron Marcuse-Kubitza

iters.py: Added is_iterable()

1880 04/16/2012 04:11 PM Aaron Marcuse-Kubitza

parallel.py: prepickle(): Pickle all objects in vars_id_dict_ by ID, not just unpicklable ones. This ensures that a DB connection created in the main process will be shared with subprocesses by reference (id()) instead of by value, so that each process can take advantage of e.g. shared caches in the connection object. Note that this may require some synchronization.

1879 04/16/2012 04:06 PM Aaron Marcuse-Kubitza

parallel.py: MultiProducerPool.main_loop(): Got rid of no longer correct doc comment

1878 04/16/2012 04:05 PM Aaron Marcuse-Kubitza

bin/map: Share on_error with the pool

1877 04/16/2012 04:05 PM Aaron Marcuse-Kubitza

parallel.py: MultiProducerPool: Pickle objects by ID if they're accessible to the main_loop process. This should allow e.g. DB connections and pools to be pickled, if they were defined in the main process.

1876 04/14/2012 09:31 PM Aaron Marcuse-Kubitza

Added dicts.py with id_dict() and MergeDict

1875 04/14/2012 09:30 PM Aaron Marcuse-Kubitza

Added collection.py with rmap()

1874 04/14/2012 07:38 PM Aaron Marcuse-Kubitza

db_xml.py: put(): Moved pool.apply_async() from put_child() to put_(), and don't use lambdas because they can't be pickled

1873 04/14/2012 07:35 PM Aaron Marcuse-Kubitza

parallel.py: MultiProducerPool.apply_async(): Prepickle all function args. Try pickling the args before the queue pickles them, to get better debugging output.

View all revisions | View revisions

Also available in: Atom