Evaluating the sensitivity of agricultural model performance to different climate inputs

Abstract

Projections of future food production necessarily rely on models, which must themselves be validated through historical assessments comparing modeled to observed yields. Reliable historical validation requires both accurate agricultural models and accurate climate inputs. Problems with either may compromise the validation exercise. Previous studies have compared the effects of different climate inputs on agricultural projections, but either incompletely or without a ground truth of observed yields that would allow distinguishing errors due to climate inputs from those intrinsic to the crop model. This study is a systematic evaluation of the reliability of a widely-used crop model for simulating U.S. maize yields when driven by multiple observational data products. The parallelized Decision Support System for Agrotechnology Transfer (pDSSAT) is driven with climate inputs from multiple sources - reanalysis, reanalysis bias-corrected with observed climate, and a control dataset - and compared to observed historical yields. The simulations show that model output is more accurate when driven by any observation-based precipitation product than when driven by un-bias-corrected reanalysis. The simulations also suggest, in contrast to previous studies, that biased precipitation distribution is significant for yields only in arid regions. However, some issues persist for all choices of climate inputs: crop yields appear oversensitive to precipitation fluctuations but undersensitive to floods and heat waves. These results suggest that the most important issue for agricultural projections may be not climate inputs but structural limitations in the crop models themselves.

Fig. 1

Interannual variations in 1980–2009 U.S. precipitation (a) and maize yields (b) for DSSAT simulations with various climate inputs. All data is weighted by harvested maize hectares and averaged over the growing season, and all yields are detrended and normalized. Detrended NASS survey yields are shown in black. Total dollar deviations in the right axis are calculated using 1980–2009 national average maize harvested hectares and assuming a fixed price of maize around 2010 values ($204/tonne). For reference, 2010 NASS yields were ~9.6 tonnes/ha, with a total production value of ~$64 billion (~$70 billion in 2014 dollars). Variations in yield are largely correlated with variations in precipitation. Simulations do not capture yield losses from 1993 flood. (See Section c for discussion.)

Fig. 2

Correlations between annual modeled and observed yield over years 1980–2009. Aggregation is at the county level, and gray regions indicate counties with less than 0.1% of land harvested for maize. Correlations between CFSR and surveyed yield (left) are neutral to weakly positive, especially in maize-growing regions (with black outline marking counties with ≥1/4 of land harvested for maize). Bias correction (right) significantly improves estimates in the corn-belt, where most maize is rainfed.

Fig. 3

Differences in 1980–2009 growing-season-averaged bias-corrected and original reanalysis climate indices (AgCFSR - CFSR) and resulting yields. Yields here are not detrended or normalized. Corn-belt is outlined in black, where most maize is rainfed. Gray regions denote counties with less than 0.1% of land harvested for maize. Spatial patterns in yield differences are similar to precipitation differences; a wetter AgCFSR Midwest correlates with higher yields. Yield differences are inconsistent with minimum temperature and solar radiation distributions.

Fig. 4

1980–2009 mean growing season rainy days for station observations and differences from AgCFSR/CPC datasets. Rainy days are tallied for daily precipitation ≥0.1 mm over the growing seasons defined by AgCFSR yield simulations. Dots are colored and scaled by magnitude. Stations are selected only where ≤1% of growing season rainfall is missing; the average across all 157 stations ~40 rainy days (a). AgCFSR underestimates the number of rainy days (b), and CPC overestimates the number of rainy days (c). Average error of CPC is about triple that of AgCFSR.

Fig. 5

Coefficients of determination (R²) of differences in growing season county yield and number of rainy days in AgCFSR and AgCFSR_+p. AgCFSR_+p uses CPC precipitation; all other climate variables are identical to AgCFSR. Counties are binned by AgCFSR growing season precipitation in 2 cm increments for the eastern U.S. only (where each year within a given county is considered a separate ‘event’). Yield is defined as potential rainfed yield to highlight the model response to rainy days. Total rainfall between both simulations is similar (right, light blue). Differences in number of rainy days describes most of the differences in local annual yield only when total rainfall is low (left, black), even though differences in estimated number of rainy days are largest when total rainfall is high (right, dark blue). National average yields in AgCFSR and AgCFSR_+p are insensitive to differences in the number of rainy days because most counties experience rainfall totals of ~20–60 cm/season (left, red).

Fig. 6

Probability of detection of bottom five yielding years over the 30 year period for each county in the NASS observational record. For comparison to NASS, all yields are detrended. Here we identify the number of years in each county that a yield simulation correctly identifies as a bottom five yielding year (where bottom-five-year-order is unimportant). a) PDF of all counties weighted by 30-year average NASS production, and equivalent number of unweighted counties on right axis. b) AgCFSR accurately identifies 1–2 more bottom five yielding years than CFSR in the corn-belt region (outlined in black) where most maize is grown. c) Bottom five yielding years for all counties, production-weighted and binned by year. Tan boxes denote the total counties identified as a bottom five yield year in the NASS record, and colored boxes denote correct detections for each model simulation. Perfect detection would mean tan and colored boxes are the same. Simulations driven with improved precipitation estimates are better able to detect yield reductions caused by Midwest drought.

Fig. 7

1980–2009 U.S. modeled and NASS survey maize yields. All yields are normalized and detrended to remove technological and management changes present in the survey data, and yields in bottom panel are variance-adjusted to remove errors in the modeled yields’ sensitivity to changes in weather. (See Eq. 1 for variance adjustment methodology.) Top panel is identical to Fig. 1b, and shown here only for comparison. Variance-adjusting model output significantly reduces differences between simulated and observed yields and between simulation scenarios.