Stochastic black-box optimization using multi-fidelity score function estimator

Optimizing parameters of physics-based simulators is crucial in the design process of engineering and scientific systems. This becomes particularly challenging when the simulator is stochastic, computationally expensive, black-box and when a high-dimensional vector of parameters needs to be optimized, as e.g. is the case in complex climate models that involve numerous interdependent variables and uncertain parameters. Many traditional optimization methods rely on gradient information, which is frequently unavailable in legacy black-box codes. To address these challenges, we present SCOUT-Nd (Stochastic Constrained Optimization for N dimensions), a gradient-based algorithm that can be used on non-differentiable objectives. It can be combined with natural gradients in order to further enhance convergence properties. and it also incorporates multi-fidelity schemes and an adaptive selection of samples in order to minimize computational effort. We validate our approach using standard, benchmark problems, demonstrating its superior performance in parameter optimization compared to existing methods. Additionally, we showcase the algorithm’s efficacy in a complex real-world application, i.e. the optimization of a wind farm layout.