TY - JOUR

T1 - Model averaging in ecology

T2 - a review of Bayesian, information-theoretic, and tactical approaches for predictive inference

AU - Dormann, Carsten F.

AU - Calabrese, Justin M.

AU - Guillera-Arroita, Gurutzeta

AU - Matechou, Eleni

AU - Bahn, Volker

AU - Bartoń, Kamil

AU - Beale, Colin M.

AU - Ciuti, Simone

AU - Elith, Jane

AU - Gerstner, Katharina

AU - Guelat, Jérôme

AU - Keil, Petr

AU - Lahoz-Monfort, José J.

AU - Pollock, Laura J.

AU - Reineking, Björn

AU - Roberts, David R.

AU - Schröder, Boris

AU - Thuiller, Wilfried

AU - Warton, David I.

AU - Wintle, Brendan A.

AU - Wood, Simon N.

AU - Wüest, Rafael O.

AU - Hartig, Florian

N1 - Publisher Copyright:
© 2018 by the Ecological Society of America

PY - 2018/11

Y1 - 2018/11

N2 - In ecology, the true causal structure for a given problem is often not known, and several plausible models and thus model predictions exist. It has been claimed that using weighted averages of these models can reduce prediction error, as well as better reflect model selection uncertainty. These claims, however, are often demonstrated by isolated examples. Analysts must better understand under which conditions model averaging can improve predictions and their uncertainty estimates. Moreover, a large range of different model averaging methods exists, raising the question of how they differ in their behaviour and performance. Here, we review the mathematical foundations of model averaging along with the diversity of approaches available. We explain that the error in model-averaged predictions depends on each model's predictive bias and variance, as well as the covariance in predictions between models, and uncertainty about model weights. We show that model averaging is particularly useful if the predictive error of contributing model predictions is dominated by variance, and if the covariance between models is low. For noisy data, which predominate in ecology, these conditions will often be met. Many different methods to derive averaging weights exist, from Bayesian over information-theoretical to cross-validation optimized and resampling approaches. A general recommendation is difficult, because the performance of methods is often context dependent. Importantly, estimating weights creates some additional uncertainty. As a result, estimated model weights may not always outperform arbitrary fixed weights, such as equal weights for all models. When averaging a set of models with many inadequate models, however, estimating model weights will typically be superior to equal weights. We also investigate the quality of the confidence intervals calculated for model-averaged predictions, showing that they differ greatly in behaviour and seldom manage to achieve nominal coverage. Our overall recommendations stress the importance of non-parametric methods such as cross-validation for a reliable uncertainty quantification of model-averaged predictions.

AB - In ecology, the true causal structure for a given problem is often not known, and several plausible models and thus model predictions exist. It has been claimed that using weighted averages of these models can reduce prediction error, as well as better reflect model selection uncertainty. These claims, however, are often demonstrated by isolated examples. Analysts must better understand under which conditions model averaging can improve predictions and their uncertainty estimates. Moreover, a large range of different model averaging methods exists, raising the question of how they differ in their behaviour and performance. Here, we review the mathematical foundations of model averaging along with the diversity of approaches available. We explain that the error in model-averaged predictions depends on each model's predictive bias and variance, as well as the covariance in predictions between models, and uncertainty about model weights. We show that model averaging is particularly useful if the predictive error of contributing model predictions is dominated by variance, and if the covariance between models is low. For noisy data, which predominate in ecology, these conditions will often be met. Many different methods to derive averaging weights exist, from Bayesian over information-theoretical to cross-validation optimized and resampling approaches. A general recommendation is difficult, because the performance of methods is often context dependent. Importantly, estimating weights creates some additional uncertainty. As a result, estimated model weights may not always outperform arbitrary fixed weights, such as equal weights for all models. When averaging a set of models with many inadequate models, however, estimating model weights will typically be superior to equal weights. We also investigate the quality of the confidence intervals calculated for model-averaged predictions, showing that they differ greatly in behaviour and seldom manage to achieve nominal coverage. Our overall recommendations stress the importance of non-parametric methods such as cross-validation for a reliable uncertainty quantification of model-averaged predictions.

KW - AIC weights

KW - ensemble

KW - model averaging

KW - model combination

KW - nominal coverage

KW - prediction averaging

KW - uncertainty

UR - http://www.scopus.com/inward/record.url?scp=85050108631&partnerID=8YFLogxK

U2 - 10.1002/ecm.1309

DO - 10.1002/ecm.1309

M3 - Review article

AN - SCOPUS:85050108631

SN - 0012-9615

VL - 88

SP - 485

EP - 504

JO - Ecological Monographs

JF - Ecological Monographs

IS - 4

ER -