TY - JOUR

T1 - Estimating model prediction error

T2 - Should you treat predictions as fixed or random?

AU - Wallach, Daniel

AU - Thorburn, Peter

AU - Asseng, Senthold

AU - Challinor, Andrew J.

AU - Ewert, Frank

AU - Jones, James W.

AU - Rotter, Reimund

AU - Ruane, Alex

N1 - Publisher Copyright:
© 2016 Elsevier Ltd

PY - 2016/10/1

Y1 - 2016/10/1

N2 - Crop models are important tools for impact assessment of climate change, as well as for exploring management options under current climate. It is essential to evaluate the uncertainty associated with predictions of these models. We compare two criteria of prediction error; MSEPfixed, which evaluates mean squared error of prediction for a model with fixed structure, parameters and inputs, and MSEPuncertain(X), which evaluates mean squared error averaged over the distributions of model structure, inputs and parameters. Comparison of model outputs with data can be used to estimate the former. The latter has a squared bias term, which can be estimated using hindcasts, and a model variance term, which can be estimated from a simulation experiment. The separate contributions to MSEPuncertain (X) can be estimated using a random effects ANOVA. It is argued that MSEPuncertain (X) is the more informative uncertainty criterion, because it is specific to each prediction situation.

AB - Crop models are important tools for impact assessment of climate change, as well as for exploring management options under current climate. It is essential to evaluate the uncertainty associated with predictions of these models. We compare two criteria of prediction error; MSEPfixed, which evaluates mean squared error of prediction for a model with fixed structure, parameters and inputs, and MSEPuncertain(X), which evaluates mean squared error averaged over the distributions of model structure, inputs and parameters. Comparison of model outputs with data can be used to estimate the former. The latter has a squared bias term, which can be estimated using hindcasts, and a model variance term, which can be estimated from a simulation experiment. The separate contributions to MSEPuncertain (X) can be estimated using a random effects ANOVA. It is argued that MSEPuncertain (X) is the more informative uncertainty criterion, because it is specific to each prediction situation.

KW - Crop model

KW - Input uncertainty

KW - Model structure uncertainty

KW - Parameter uncertainty

KW - Prediction error

KW - Uncertainty

UR - http://www.scopus.com/inward/record.url?scp=84981303090&partnerID=8YFLogxK

U2 - 10.1016/j.envsoft.2016.07.010

DO - 10.1016/j.envsoft.2016.07.010

M3 - Article

AN - SCOPUS:84981303090

SN - 1364-8152

VL - 84

SP - 529

EP - 539

JO - Environmental Modelling and Software

JF - Environmental Modelling and Software

ER -