Project

General

Profile

Activity

From 04/16/2012 to 05/15/2012

05/15/2012

11:22 AM Task #411 (In Progress): Insert DEM adaptive smoothing procedure into terrain workflow
I developed two alternative implementations of the adaptive DEM smoothing procedure (@multiscalesmooth9a_clean.aml@):... Jim Regetz
09:52 AM Task #411 (In Progress): Insert DEM adaptive smoothing procedure into terrain workflow
The original Oregon terrain processing workflow, implemented as a set of AML scripts, included the application of Joh... Jim Regetz
09:57 AM Task #208: Evaluate existing E&O AML scripts
I recently reorganized all AML scripts into a single directory in the code repo, here:
source:terrain/research/orego...
Jim Regetz

05/14/2012

10:00 AM Task #363: Assemble all GHCN data into a single database
Quick example comparing SQLite vs PostgreSQL timings on a simple aggregate query that doesn't use any indexes. SQLite... Jim Regetz

05/13/2012

11:16 PM Task #363: Assemble all GHCN data into a single database
Vaguely concerned about performance and manageability of such a large SQLite database, I wrote a second script to loa... Jim Regetz
11:14 PM Task #363: Assemble all GHCN data into a single database
Belated update. The TMIN/TMAX data loaded in ~6 hours total, yielding a 25GB SQLite file with nearly 600 million rows. Jim Regetz
10:54 PM Revision c7235eae: added R script to load GHCN data into a PostgreSQL DB
Jim Regetz
10:50 PM Revision a7663038: further optimized functions in R for some minor extra speedup
Jim Regetz

05/11/2012

10:03 AM Revision aac7c36c: Added 3 new models and nesting to GAM, task #361
Benoit Parmentier

05/10/2012

03:03 PM Revision 0d1cde0a: Changes in the diagnostics plots, barplots.
Benoit Parmentier
02:49 PM Revision 25a68ae3: Transform aspect in Eastness and Northness variables in the GAM.
Benoit Parmentier
02:42 PM Revision 4180b123: Transformed the aspect variable and added GAM model closer to PRISM
Benoit Parmentier
02:28 PM Revision 2a89fa5f: GAM prediction added to the loop with RMSE calculation
Benoit Parmentier
02:13 PM Revision ec229367: Streamlining and slight changes to the loop to create subset dataset for different dates
Benoit Parmentier
02:07 PM Revision af42291a: Additional changes to the loop and format
Benoit Parmentier
01:47 PM Revision bc59c6bb: Start of Task #361, modified code to subset for 10 dates
Benoit Parmentier
11:59 AM Revision d7b0ef36: Minor format changes and clean up to code
Benoit Parmentier
11:18 AM Revision e2d673e4: Corrected errors in RMSE calculation and added plots to view results
Benoit Parmentier
08:49 AM Task #363: Assemble all GHCN data into a single database
Data loading slowed to a crawl overnight, thanks to a record count query tucked inside in the bulk insert function th... Jim Regetz
08:34 AM Revision 3b8dd010: dropped unnecessary record-count query from bulk insert function
Jim Regetz

05/09/2012

11:49 PM Task #363: Assemble all GHCN data into a single database
Committed an R script that orchestrates the processing and loading of all 75000+ *.dly files into a SQLite database. ... Jim Regetz
11:10 AM Task #363 (In Progress): Assemble all GHCN data into a single database
We decided to organize and load _all_ available GHCN data (i.e., all dates and stations, globally) into SQLite databa... Jim Regetz
11:06 PM Revision d4e63f5d: superficial code reformatting
Jim Regetz
10:29 PM Revision 38aa5c07: replaced reshape with faster manual split-rbind approach
Jim Regetz
04:11 PM Revision 669c150d: added grep pre-filtering of rows in the *.dly files
Jim Regetz
03:45 PM Revision 2d08ed05: replaced read.fortran with faster system call to awk/tr
Jim Regetz
03:41 PM Revision dd85f6d3: added initial R script to load GHCN data into a SQLite DB
Jim Regetz
03:40 PM Task #408: Capturing LST spatial structure using spatial eigenvectors
I started writing the R code using the two functions from spdep package: ME() and SpatialFiltering()
I am trying to ...
Benoit Parmentier
03:32 PM Task #364: Integrate spatial variables and structure in the GAM methodology
I am currently exploring a two stage regression involving:
1) Step 1: GAM models using Lat, long, ELEV_SRTM, DISTOC,...
Benoit Parmentier

05/08/2012

04:31 PM Task #406: OR-GAM predictions-model assessment over 365 dates, year 2010
RMSE were calculated per month over the year 2010. Results show that contrary to my expectation RMSE are higher in S... Benoit Parmentier

05/04/2012

09:35 AM Revision 6c9ca794: simplified code by using 'a' flag for r.neighbors (improves 3abda63)
Jim Regetz
09:20 AM Revision 5af6ef96: reorganized AML terrain scripts to reduce superficial redundancy
Jim Regetz

05/02/2012

11:57 AM Task #393: Identify Focal Regions
We are currently proposing the following regions (and MODIS tiles):
Oregon: H08V04, H09V04, H08V05, H09V0...
Adam Wilson

05/01/2012

02:24 PM Task #361: Test and compare the GAM method on several days (10 days for now) for Oregon
Many models have been explored using the GAM methodology. At this stage, more detailed diagnostics is needed so for t... Benoit Parmentier
02:16 PM Task #409 (New): Effect of sampling on GAM: hold out proportions and sampling observations
This task examines the effect of the sampling and hold out proportions on tmax predictions. This task will help in de... Benoit Parmentier
02:08 PM Task #364: Integrate spatial variables and structure in the GAM methodology
Kriging has been done so far by using variograms fitted from stations' locations. The next step is to examine the fit... Benoit Parmentier
02:03 PM Task #408 (New): Capturing LST spatial structure using spatial eigenvectors
Spatial filtering is explored to assess its usefulness in capturing the LST spatial structure by creating spatial var... Benoit Parmentier
01:56 PM Task #406 (New): OR-GAM predictions-model assessment over 365 dates, year 2010
Models have so far been assessed on 10 dates spread over the year for the Oregon study area. This task provides an as... Benoit Parmentier
01:20 PM Task #375: Assemble monthly mean MODIS LST values for the complete record (2000-2012) for Oregon
The NASA Ames team is willing to generate the monthly LST climatologies, but we'll need to be specific about what we ... Adam Wilson
01:20 PM Task #393: Identify Focal Regions
Do we want to look at a mediterranean-type climate? Guess western part of South Africa qualifies. Or interior tempe... Rob Guralnick
01:13 PM Task #393: Identify Focal Regions
The folks at NASA Ames suggested that scaling up to the globe will much easier to do if we process the data using the... Adam Wilson

04/27/2012

11:05 AM Task #393: Identify Focal Regions
Natalie is ready to help MODIS data assembly once we have decided this. Can we plan to talk about this on the Tuesda... Rob Guralnick

04/24/2012

02:00 PM Task #375: Assemble monthly mean MODIS LST values for the complete record (2000-2012) for Oregon
I just noticed that the IRI data library [[http://iridl.ldeo.columbia.edu/SOURCES/.USGS/.LandDAAC/.MODIS/.1km/.8day/.... Adam Wilson
01:14 PM Task #393 (New): Identify Focal Regions
We need to identify the small number (3-5?) of focal regions that we will use for further validation after we finish... Adam Wilson

04/19/2012

01:56 PM Revision 92c36e33: made minor changes to PRISM variables
Benoit Parmentier
01:51 PM Revision 3f649df0: changed RMSE and AIC code in GAM script
Benoit Parmentier
01:39 PM Revision 7f643c83: updated GAM script for tmax
Benoit Parmentier
01:23 PM Revision 89c6eee2: added initial R script with GAM regression for tmax (one date)
Benoit Parmentier

04/18/2012

09:59 AM 20120320_NCEAS_update.pdf
Presented at 4/17/2012 phone conference Adam Wilson
09:59 AM Document: Anomaly Approach for Interpolations
Adam Wilson
 

Also available in: Atom