(575b) Reduced Space Formulation for Global Optimization with Gaussian Processes (Kriging) Embedded

A Gaussian process (GP), also known as Kriging, is a stochastic process where any finite collection of random variables has a multivariate Gaussian distribution [1]. Thus, GP predictions provide an estimate and its variance. GPs originate from geostatistics [2] but are nowadays applied as interpolating surrogate models across various disciplines including chemical engineering [3,4,5], bioengineering [6], and chemistry [7]. In many applications, GPs are trained on a data set and are subsequently embedded in an optimization problem, e.g., to obtain new query points in Bayesian optimization [8,5].

In process systems engineering, GPs are commonly used to emulate unit operations or complete flowsheets from data. Subsequently, a hybrid data-driven/mechanistic (flowsheet) model with trained GPs embedded is optimized. Most of the previous works on optimizations with GPs embedded rely on local optimization techniques (e.g., [3,4]). Deterministic global optimization of problems with GPs embedded was performed by Cozad et al. (2014) [9], Boukouvala et al. (2017) [10], and Keßler et al. (2019) [11] using the state-of-the-art solvers BARON and ANTIGONE. However, these works observed that GPs satisfy accuracy requirements but are difficult-to-solve for deterministic global solvers [9,10]. In our previous work, we overcame similar problems when artificial neural networks are embedded in optimization problems using a reduced-space formulation [12].

Herein, we present a method for deterministic global optimization of optimization problems with GPs embedded in a reduced variable space. In particular, we propagate McCormick relaxations through GPs [13,14]. This method allows the branch-and-bound algorithms to operate only on the degrees of freedom and hides the complex GP structure. The optimization problems with GPs embedded are solved by our in-house global deterministic solver MAiNGO [15]. The performance and scaling of the method are demonstrated on a set of illustrative case studies. The results show that the reduced-space method reduces the problem size significantly compared to the full-space formulation. The proposed method performs favorably compared BARON and allows efficient deterministic global optimization with (trained) GPs embedded.

Acknowledgment: The authors gratefully acknowledge the financial support of the Kopernikus project SynErgie by the Federal Ministry of Education and Research (BMBF) and the project supervision by the project management organization Projektträger Jülich (PtJ).

[1] Rasmussen, C. E. (2003). Gaussian processes in machine learning. In Summer School on Machine Learning (pp. 63-71). Springer, Berlin, Heidelberg.

[2] Krige, D. G. (1951). A statistical approach to some basic mine valuation problems on the Witwatersrand. Journal of the Southern African Institute of Mining and Metallurgy, 52(6), 119-139.

[3] Caballero, J. A., & Grossmann, I. E. (2008). An algorithm for the use of surrogate models in modular flowsheet optimization. AIChE journal, 54(10), 2633-2650.

[4] Quirante, N., Javaloyes, J., Ruiz-Femenia, R., & Caballero, J. A. (2015). Optimization of Chemical Processes Using Surrogate Models Based on a Kriging Interpolation. In Computer Aided Chemical Engineering (Vol. 37, pp. 179-184). Elsevier.

[5] Boukouvala, F., & Ierapetritou, M. G. (2012). Feasibility analysis of black-box processes using an adaptive sampling Kriging-based method. Computers & Chemical Engineering, 36, 358-368.

[6] Bradford, E., Schweidtmann, A. M., Zhang, D., Jing, K., & del Rio-Chanona, E. A. (2018). Dynamic modeling and optimization of sustainable algal production with uncertainty using multivariate Gaussian processes. Computers & Chemical Engineering, 118, 143-158.

[7] Schweidtmann, A. M., Clayton, A. D., Holmes, N., Bradford, E., Bourne, R. A., & Lapkin, A. A. (2018). Machine learning meets continuous flow chemistry: Automated optimization towards the Pareto front of multiple objectives. Chemical Engineering Journal, 352, 277-282.

[8] Jones, D. R., Schonlau, M., & Welch, W. J. (1998). Efficient global optimization of expensive black-box functions. Journal of Global optimization, 13(4), 455-492.

[9] Cozad, A., Sahinidis, N. V., & Miller, D. C. (2014). Learning surrogate models for simulation‐based optimization. AIChE Journal, 60(6), 2211-2227.

[10] Boukouvala, F., & Floudas, C. A. (2017). ARGONAUT: AlgoRithms for Global Optimization of coNstrAined grey-box compUTational problems. Optimization Letters, 11(5), 895-913.

[11] Keßler, T., Kunde, C., McBride, K., Mertens, N., Michaels, D., Sundmacher, K., & Kienle, A. (2019). Global optimization of distillation columns using explicit and implicit surrogate models. Chemical Engineering Science, 197, 235-245.

[12] Schweidtmann, A. M., & Mitsos, A. (2019). Deterministic global optimization with artificial neural networks embedded. Journal of Optimization Theory and Applications, 180(3), 925-948.

[13] Mitsos, A., Chachuat, B., & Barton, P. I. (2009). McCormick-based relaxations of algorithms. SIAM Journal on Optimization, 20(2), 573-601.

[14] McCormick, G. P. (1976). Computability of global solutions to factorable nonconvex programs: Part I—Convex underestimating problems. Mathematical programming, 10(1), 147-175.

[15] Bongartz, D., Najman, J., Sass, S., Mitsos, A. (2018). MAiNGO: McCormick based Algorithm for mixed integer Nonlinear Global Optimization. Technical report.