• Table of contents
• Guzman updates
  • 2014-02-03

Guzman updates¶

2014-02-03¶

• I tested the memory management functions to get better memory usage per node using rasterOptions. Tested running code on different types of nodes. I was able to run high memory tiles using ivy bridge nodes that have 64GBs. Memory usage running 12 cores was about 50GB and took about an hour to finish. As I explain on the "obstacles blocking progress" section below changing rasterOptions settings doesn't help with memory usage.
• Did some research on different parameters for rasterOptions.
• Making smaller dbs for different testing regions.

• Working with nas to implement X11 server when running non-interactive sessions. Functions such as png() need to have some type of X11 running to make the plots.
• I'm running the code for the other stages on the test server so I can have global covariate layers and climate stations ready for stage 4.
• Working with nas to see if we can get better memory usage from R.
• Running small subsets for different areas of the globe.

• Changing raster options seems to have an effect on running time but not on memory usage. I tested different settings anywhere from rasterOptions(maxmemory = 1e+09) to rasterOptions(maxmemory = 1e+01) and this option didn't reduce memory when running predict(). I also tried changing the block.size parameter on the predict() function but parameter had no effect. For now we're stuck with using Ivy Bridge nodes which increases the amount of time it takes to get a job through the queue.
• Computing resources have experienced a high usage from other projects.

What's next *