(104a) Multi-Fidelity Surrogate Models to Generate Low-Fidelity Data for Data-Driven Branch-and-Bound Optimization

High-Fidelity (HF) Simulations are essential in quantitative analysis and decision making in many engineering fields. However, some of the challenges when optimizing with embedded HF simulations are: (a) the lack of objective function and constraint equations and derivatives, and (b) high-computational cost (van de Berg et al., 2022). To optimize such systems, black-box optimization methods are used. We recently proposed a data-driven equivalent of spatial branch-and-bound algorithm (DDSBB) based on the concept of constructing underestimators to simulations from the HF data (Zhai & Boukouvala, 2022). The algorithm uses a linear programming formulation to fit quadratic underestimators that satisfy constraints that enforce their validity based on collected samples and estimated Lipschitz constants. This algorithm has shown promising results in identifying globally optimal solutions with less dependence on the initialization, and provides bounds on the identified optimal solutions (i.e., worst-best case optimal solutions) at any point of the optimization process. However, the algorithm requires a large amount of samples to converge, especially with increasing dimensionality.

One way to reduce the required HF sampling cost is to utilize the HF data to construct Low-fidelity (LF) surrogates, and generate LF data to improve the underestimators. In our prior work, we have shown that utilizing LF predictions lead to improved performance (Zhai & Boukouvala, 2022). Recent advances in hybrid surrogate modeling and physics-informed machine learning (Bradley et al., 2022), provide new opportunities for further improving this framework and we intend to explore many such techniques in this paper. We start by quantifying the effect of different LF hybrid surrogates on performance of our algorithm, by studying a large set of non-linear benchmarking problems. While this approach provides an advantage as is, we will devise techniques to further improve the LF data accuracy by utilizing the concept of multi-fidelity surrogate models (MFSMs). We will quantify the improvements in black-box optimization, when taking advantage of the feature of MFSMs to “learn better”, by exploring correlations between the LF and HF data (Guo, Manzoni, Amendt, Conti, & Hesthaven, 2022). The performance and implementation of this work will be shown in the presentation through our existing python package (PyDDSBB).

References

Bradley, W., Kim, J., Kilwein, Z., Blakely, L., Eydenberg, M., Jalvin, J., . . . Boukouvala, F. (2022). Perspectives on the integration between first-principles and data-driven modeling. Computers & Chemical Engineering, 166, 107898. doi:https://doi.org/10.1016/j.compchemeng.2022.107898

Guo, M., Manzoni, A., Amendt, M., Conti, P., & Hesthaven, J. S. (2022). Multi-fidelity regression using artificial neural networks: efficient approximation of parameter-dependent output quantities. Computer methods in applied mechanics and engineering, 389, 114378-114378.

van de Berg, D., Savage, T., Petsagkourakis, P., Zhang, D., Shah, N., & del Rio-Chanona, E. A. (2022). Data-driven optimization for process systems engineering applications. Chemical Engineering Science, 248, 117135.

Zhai, J., & Boukouvala, F. (2022). Data-driven spatial branch-and-bound algorithms for box-constrained simulation-based optimization. Journal of Global Optimization, 82(1), 21-50.