Activity
From 04/30/2012 to 05/29/2012
05/29/2012
- 04:49 PM Revision fbc95c44: GAM, modification of nesting of models and clean up, task #361
- 04:16 PM Revision 328528e2: Kriging, adding LST for co-kriging: task #364
- 03:53 PM Revision eec8b7e0: Kriging, slight modification and clean up, task #364
- 03:37 PM Revision 88248d6c: Kriging, modification to save kriged surfaces in geotiff and png files, task #364
- 03:24 PM Revision 3be0f72b: Kriging, modifying code to save training and testing samples in shapefiles: task #364
- 03:07 PM Revision 7da7872a: Kriging for OR, this is the first script, task #364
- 12:56 PM Revision 15a39f25: GWR modified code to save plots and images, task #364
- 12:49 PM Revision defe53e5: GWR assessment by visualizing residuals using kriging
- 12:16 PM Revision 5f312940: GWR in adding loop for 10 dates
- 11:57 AM Revision 23ed3053: GWR first script for one date in OR, task #364
05/25/2012
- 03:21 PM Task #363: Assemble all GHCN data into a single database
- The daily data files currently in the database were obtained back in Oct 2010. Benoit and I discussed simply augmenti...
- 09:05 AM Task #419 (In Progress): Assessment of results in the context of the literature: Literature review and accuracy
- Documenting the new products will necessitate assembling a list of references on climate interpolation to place the r...
05/24/2012
05/23/2012
- 01:55 PM Revision 2a6b57de: Reorganization: moved mindmap to it's own directory to isolate temporary files created when exporting to other formats
- 01:44 PM Revision 5891fba5: Add initial version of the 'mind map' for the interpolation project. This file contains an outline of the various stages of the project and links back to issues and code in the blessed repository.
- Created during Benoit's visit to Yale May 21-22.
- 01:43 PM Revision 147da66d: Adding some MOD06 (cloud product) processing routines. Still a work in progress
- 01:35 PM Revision ee6357dc: Initial commit
05/22/2012
- 10:14 AM Task #363: Assemble all GHCN data into a single database
- If useful for some specific purpose, we could generate relevant synoptic views or subsetted table dumps pivoted back ...
- 08:25 AM Task #363: Assemble all GHCN data into a single database
- Perhaps it would make sense to keep the data in the "one-row-per-month" format in the database and transform it on th...
- 07:52 AM Task #418 (New): Explore Landcover - LST interactions
- Use the full gridded datasets (rather than just station locations) to explore relationships between LST and various (...
05/17/2012
- 11:38 PM Task #416: Scope out workflow for calculating monthly LST climatologies
- I've now written some Python+GRASS code to do a good bit of what I think we want. For a user-specified _tile_, _year_...
- 03:05 PM Task #416 (In Progress): Scope out workflow for calculating monthly LST climatologies
- Assess feasibility of running LST climatologies ourselves by implementing a basic scripted workflow and estimating ov...
- 03:44 PM Task #375: Assemble monthly mean MODIS LST values for the complete record (2000-2012) for Oregon
- See task #416 for progress on developing and evaluating a procedure for doing this ourselves.
But I will add a com... - 02:16 PM Revision 01b3830e: added function documentation and removed two obsolete functions
- 02:04 PM Revision a05f49eb: automated determination of which days to aggregate for any given month
- 01:53 PM Revision 96a70efc: tweaked climatology calc function to return list of GRASS map names
- 01:43 PM Revision bd9bc508: switched from truncating to rounding mean LST back to integer
- 12:31 PM Revision 5fb982c8: wrote function to download HDF files for given tile/year/day-range
- 10:05 AM Revision b9e1a7f9: wrote function to look up local HDF files for given tile/year/day-range
- 09:12 AM Revision eaf6c10b: wrote function to encapsulate LST loading and QC-adjustment
- 08:45 AM Revision 47cdfebe: added initial Python+GRASS code to download and aggregate daily LST
- 08:45 AM Revision 934d6ab7: added script to reproduce calculation of 8-day LST from daily LST
05/16/2012
- 01:55 PM Document: Update IPLANT meeting roundup 05152012
- This document summarizes ongoing work for the IPLANT meeting on 05152012:
-GAM assessment over year 2010: average pe... - 01:49 PM Document: Update IPLANT meeting roundup 05012012
- This document provides updates presented at the IPLANT meeting 05012012:
- Results from GAM models run for the year ... - 01:41 PM Document: Update IPLANT meeting roundup 04172012
- This presentation presents mean monthly LST averages derived from daily MOD11A1. Results from Kriging and co-kriging ...
- 12:01 PM Task #207: Produce global fused DEM layer
- A global 90m product is now available, though the individual pieces have not been mosaiced together due to file size....
- 06:40 AM Task #415: Process MOD06_L2 Cloud data
- See presentation on current progress here [[https://projects.nceas.ucsb.edu/nceas/documents/18]]
- 06:40 AM Task #415 (New): Process MOD06_L2 Cloud data
- Download and process the MODIS cloud product (MOD06_L2) and produce monthly 1km summaries of key variables.
- 06:37 AM Document: Wilson update 5/15/2012
- Overview of processing the MOD06 cloud product
05/15/2012
- 11:22 AM Task #411 (In Progress): Insert DEM adaptive smoothing procedure into terrain workflow
- I developed two alternative implementations of the adaptive DEM smoothing procedure (@multiscalesmooth9a_clean.aml@):...
- 09:52 AM Task #411 (In Progress): Insert DEM adaptive smoothing procedure into terrain workflow
- The original Oregon terrain processing workflow, implemented as a set of AML scripts, included the application of Joh...
- 09:57 AM Task #208: Evaluate existing E&O AML scripts
- I recently reorganized all AML scripts into a single directory in the code repo, here:
source:terrain/research/orego...
05/14/2012
- 10:00 AM Task #363: Assemble all GHCN data into a single database
- Quick example comparing SQLite vs PostgreSQL timings on a simple aggregate query that doesn't use any indexes. SQLite...
05/13/2012
- 11:16 PM Task #363: Assemble all GHCN data into a single database
- Vaguely concerned about performance and manageability of such a large SQLite database, I wrote a second script to loa...
- 11:14 PM Task #363: Assemble all GHCN data into a single database
- Belated update. The TMIN/TMAX data loaded in ~6 hours total, yielding a 25GB SQLite file with nearly 600 million rows.
- 10:54 PM Revision c7235eae: added R script to load GHCN data into a PostgreSQL DB
- 10:50 PM Revision a7663038: further optimized functions in R for some minor extra speedup
05/11/2012
05/10/2012
- 03:03 PM Revision 0d1cde0a: Changes in the diagnostics plots, barplots.
- 02:49 PM Revision 25a68ae3: Transform aspect in Eastness and Northness variables in the GAM.
- 02:42 PM Revision 4180b123: Transformed the aspect variable and added GAM model closer to PRISM
- 02:28 PM Revision 2a89fa5f: GAM prediction added to the loop with RMSE calculation
- 02:13 PM Revision ec229367: Streamlining and slight changes to the loop to create subset dataset for different dates
- 02:07 PM Revision af42291a: Additional changes to the loop and format
- 01:47 PM Revision bc59c6bb: Start of Task #361, modified code to subset for 10 dates
- 11:59 AM Revision d7b0ef36: Minor format changes and clean up to code
- 11:18 AM Revision e2d673e4: Corrected errors in RMSE calculation and added plots to view results
- 08:49 AM Task #363: Assemble all GHCN data into a single database
- Data loading slowed to a crawl overnight, thanks to a record count query tucked inside in the bulk insert function th...
- 08:34 AM Revision 3b8dd010: dropped unnecessary record-count query from bulk insert function
05/09/2012
- 11:49 PM Task #363: Assemble all GHCN data into a single database
- Committed an R script that orchestrates the processing and loading of all 75000+ *.dly files into a SQLite database. ...
- 11:10 AM Task #363 (In Progress): Assemble all GHCN data into a single database
- We decided to organize and load _all_ available GHCN data (i.e., all dates and stations, globally) into SQLite databa...
- 11:06 PM Revision d4e63f5d: superficial code reformatting
- 10:29 PM Revision 38aa5c07: replaced reshape with faster manual split-rbind approach
- 04:11 PM Revision 669c150d: added grep pre-filtering of rows in the *.dly files
- 03:45 PM Revision 2d08ed05: replaced read.fortran with faster system call to awk/tr
- 03:41 PM Revision dd85f6d3: added initial R script to load GHCN data into a SQLite DB
- 03:40 PM Task #408: Capturing LST spatial structure using spatial eigenvectors
- I started writing the R code using the two functions from spdep package: ME() and SpatialFiltering()
I am trying to ... - 03:32 PM Task #364: Integrate spatial variables and structure in the GAM methodology
- I am currently exploring a two stage regression involving:
1) Step 1: GAM models using Lat, long, ELEV_SRTM, DISTOC,...
05/08/2012
- 04:31 PM Task #406: OR-GAM predictions-model assessment over 365 dates, year 2010
- RMSE were calculated per month over the year 2010. Results show that contrary to my expectation RMSE are higher in S...
05/04/2012
- 09:35 AM Revision 6c9ca794: simplified code by using 'a' flag for r.neighbors (improves 3abda63)
- 09:20 AM Revision 5af6ef96: reorganized AML terrain scripts to reduce superficial redundancy
05/02/2012
- 11:57 AM Task #393: Identify Focal Regions
- We are currently proposing the following regions (and MODIS tiles):
Oregon: H08V04, H09V04, H08V05, H09V0...
05/01/2012
- 02:24 PM Task #361: Test and compare the GAM method on several days (10 days for now) for Oregon
- Many models have been explored using the GAM methodology. At this stage, more detailed diagnostics is needed so for t...
- 02:16 PM Task #409 (New): Effect of sampling on GAM: hold out proportions and sampling observations
- This task examines the effect of the sampling and hold out proportions on tmax predictions. This task will help in de...
- 02:08 PM Task #364: Integrate spatial variables and structure in the GAM methodology
- Kriging has been done so far by using variograms fitted from stations' locations. The next step is to examine the fit...
- 02:03 PM Task #408 (New): Capturing LST spatial structure using spatial eigenvectors
- Spatial filtering is explored to assess its usefulness in capturing the LST spatial structure by creating spatial var...
- 01:56 PM Task #406 (New): OR-GAM predictions-model assessment over 365 dates, year 2010
- Models have so far been assessed on 10 dates spread over the year for the Oregon study area. This task provides an as...
- 01:20 PM Task #375: Assemble monthly mean MODIS LST values for the complete record (2000-2012) for Oregon
- The NASA Ames team is willing to generate the monthly LST climatologies, but we'll need to be specific about what we ...
- 01:20 PM Task #393: Identify Focal Regions
- Do we want to look at a mediterranean-type climate? Guess western part of South Africa qualifies. Or interior tempe...
- 01:13 PM Task #393: Identify Focal Regions
- The folks at NASA Ames suggested that scaling up to the globe will much easier to do if we process the data using the...
Also available in: Atom