<img src="http://i.imgur.com/RZbujnI.png" alt="plot of chunk unnamed-chunk-4"/> </p>

<p><img src="http://i.imgur.com/aAmbMaz.png" alt="plot of chunk unnamed-chunk-5"/> </p>

<p><img src="http://i.imgur.com/AeJ0AFq.png" alt="plot of chunk unnamed-chunk-6"/> </p>

<p><img src="http://i.imgur.com/qIlsdWo.png" alt="plot of chunk unnamed-chunk-7"/> </p>

<p><img src="http://i.imgur.com/Ve0mC1C.png" alt="plot of chunk unnamed-chunk-8"/> </p>

<h2>New Solution 2 (also relatively easy)</h2>

<ol>

<li>Create 12 monthly LST climatologies if at least 5 or 10 observations are available for each pixel-month.</li>

<li>Fill in missing months using temporal interpolation:

<ul>

<li>Fit function through all available months (rather than just +/- 1 month) to capture seasonal variabilty.</li>

<li>Use some function that can capture the cyclical nature (sin, spline, polynomial, etc.)</li>

</ul></li>

</ol>

<p>Remember from above that the LST curves are fairly smooth (caveat: we&#39;ve only looked at locations in the CONUS dataset).  So let&#39;s try fitting a simple sin function to estimate missing months:</p>

<pre><code class="r">### Curve fitting function Set parameter starting values and limits

parms = list(phi = 5, A = 0.05)  #starting parameters

low = list(phi = -999, A = 0)  #lower bounds - constrain s3 to be positive 

high = list(phi = 999, A = 999)  #lower bounds - constrain s3 to be positive

## define a simple model estimating a sin function with variable shift and

## amplitude

fit.model <- function(parms, x) {

    fit = sin(parms[["phi"]] + (x * 2 * pi)) * abs(parms[["A"]])

    fit[fit == Inf] = .Machine$double.xmax  #if Inf, replace with maximum value

    fit[fit == -Inf] = .Machine$double.xmin  #if Inf, replace with maximum value

    list(fit = fit)

}

## function to minimize the RMSE of predictions

minimize.model <- function(parms, x, y) {

    ss = sum((y - fit.model(parms, x)$fit)^2)

    ss = min(ss, .Machine$double.xmax)  #if Inf, replace with maximum value

    ss = max(ss, .Machine$double.xmin)  #if -Inf, replace with maximum value

    list(ss = ss)

}

## Process station data and make predictions for each month

makepreds = function(dat, drop = NULL) {

    dat = na.omit(dat)

    dat = dat[dat$variable == "lst", ]

    ## drop

    if (nrow(dat) == 0) {

        return(NULL)

    }

    ## drop some proportion of data if drop is specified

    if (!is.null(drop)) 

        dat = dat[sample(1:nrow(dat), round(nrow(dat) * (1 - drop))), ]

    dat = dat[order(dat$month), ]

    dat$time = (1:nrow(dat))/nrow(dat)

    xbar = mean(dat$value, na.rm = T)

    fit = optim(unlist(parms), minimize.model, x = dat$time, y = dat$value - 

        xbar, control = list(trace = 0, maxit = 10000), method = "L-BFGS-B", 

        lower = low, upper = high)

    dat$pred = xbar + sin(fit$par[["phi"]] + (dat$time * 2 * pi)) * fit$par[["A"]]

    dat$A = fit$par[["A"]]

    dat$phi = fit$par[["phi"]]

    dat

}

## apply the function

d4 = ddply(dlst2ls, .(station), .fun = makepreds)

d4l = melt(d4, id.vars = c("station", "month"), measure.vars = c("value", "pred"))

</code></pre>

<p>Let&#39;s see what that looks  for the 10 example stations above. The blue line is the observed LST value and the pink line is the predicted LST from the sinusoidal function.

<img src="http://i.imgur.com/oNteXlf.png" alt="plot of chunk unnamed-chunk-10"/> </p>

<p>And a histogram of the residuals for these stations:

<img src="http://i.imgur.com/1z4IHQs.png" alt="plot of chunk unnamed-chunk-11"/> </p>

<p>Not bad&hellip; Now let&#39;s drop 25% of the observations from each station and do it again:

<img src="http://i.imgur.com/6zkd5CB.png" alt="plot of chunk unnamed-chunk-12"/> </p>

<p>And the  of residuals for these 10 stations with 25% of the observations removed:

<img src="http://i.imgur.com/c1Wyh9J.png" alt="plot of chunk unnamed-chunk-13"/> </p>

<h3>Apply this function to the full CONUS dataset described above.</h3>

<p>Now look at the distributions of the residuals for the full conus dataset.

<img src="http://i.imgur.com/od4iR2S.png" alt="plot of chunk unnamed-chunk-14"/> </p>

<p>And summarize the residuals into RMSE&#39;s by station:

<img src="http://i.imgur.com/KIV2rBk.png" alt="plot of chunk unnamed-chunk-15"/> </p>

<p>So the vast majority of stations will have a RMSE of &lt;5 using this simple method even if 25% of the data are missing.</p>

<h1>Next steps</h1>

<p>We should pick 3-4 problematic tiles and 1 tile with little missing data and explore how these various options differ in infilling LST.  It would also be ideal if we could complete the interpolation using these different datasets and compare the resulting estimates of TMax over these tiles. </p>