TY - JOUR

T1 - Certified Dimension Reduction for Bayesian Updating with the Cross-Entropy Method

AU - Ehre, Max

AU - Flock, Rafael

AU - Fußeder, Martin

AU - Papaioannou, Iason

AU - Straub, Daniel

N1 - Publisher Copyright:
© 2023 Society for Industrial and Applied Mathematics and American Statistical Association.

PY - 2023

Y1 - 2023

N2 - In inverse problems, the parameters of a model are estimated based on observations of the model response. The Bayesian approach is powerful for solving such problems; one formulates a prior distribution for the parameter state that is updated with the observations to compute the posterior parameter distribution. Solving for the posterior distribution can be challenging when, e.g., prior and posterior significantly differ from one another and/or the parameter space is high-dimensional. We use a sequence of importance sampling measures that arise by tempering the likelihood to approach inverse problems exhibiting a significant distance between prior and posterior. Each importance sampling measure is identified by cross-entropy minimization as proposed in the context of Bayesian inverse problems in Engel et al. [J. Comput. Phys., 473 (2023), 111746]. To efficiently address problems with high-dimensional parameter spaces, we set up the minimization procedure in a low-dimensional subspace of the original parameter space. The principal idea is to analyze the spectrum of the second-moment matrix of the gradient of the log-likelihood function to identify a suitable subspace. Following Zahm et al. [Math. Comp., 91 (2022), pp. 1789-1835], an upper bound on the Kullback-Leibler divergence between full-dimensional and subspace posterior is provided, which can be utilized to determine the effective dimension of the inverse problem corresponding to a prescribed approximation error bound. We suggest heuristic criteria for optimally selecting the number of model and model gradient evaluations in each iteration of the importance sampling sequence. We investigate the performance of this approach using examples from engineering mechanics set in various parameter space dimensions.

AB - In inverse problems, the parameters of a model are estimated based on observations of the model response. The Bayesian approach is powerful for solving such problems; one formulates a prior distribution for the parameter state that is updated with the observations to compute the posterior parameter distribution. Solving for the posterior distribution can be challenging when, e.g., prior and posterior significantly differ from one another and/or the parameter space is high-dimensional. We use a sequence of importance sampling measures that arise by tempering the likelihood to approach inverse problems exhibiting a significant distance between prior and posterior. Each importance sampling measure is identified by cross-entropy minimization as proposed in the context of Bayesian inverse problems in Engel et al. [J. Comput. Phys., 473 (2023), 111746]. To efficiently address problems with high-dimensional parameter spaces, we set up the minimization procedure in a low-dimensional subspace of the original parameter space. The principal idea is to analyze the spectrum of the second-moment matrix of the gradient of the log-likelihood function to identify a suitable subspace. Following Zahm et al. [Math. Comp., 91 (2022), pp. 1789-1835], an upper bound on the Kullback-Leibler divergence between full-dimensional and subspace posterior is provided, which can be utilized to determine the effective dimension of the inverse problem corresponding to a prescribed approximation error bound. We suggest heuristic criteria for optimally selecting the number of model and model gradient evaluations in each iteration of the importance sampling sequence. We investigate the performance of this approach using examples from engineering mechanics set in various parameter space dimensions.

KW - Bayesian inverse problems

KW - certified dimension reduction

KW - cross-entropy method

KW - high dimensions

KW - importance sampling

UR - http://www.scopus.com/inward/record.url?scp=85149275077&partnerID=8YFLogxK

U2 - 10.1137/22M1484031

DO - 10.1137/22M1484031

M3 - Article

AN - SCOPUS:85149275077

SN - 2166-2525

VL - 11

SP - 358

EP - 388

JO - SIAM-ASA Journal on Uncertainty Quantification

JF - SIAM-ASA Journal on Uncertainty Quantification

IS - 1

ER -