(374c) Guaranteed Error-Bounded Surrogate Modeling for Process Simulation, Optimization, and Control

The accurate prediction, simulation, optimization, and control of complex systems often requires the use of rigorous mathematical models that capture the underlying equilibrium, transport, and thermodynamic relationships. However, these first-principles-based models are often large and nonlinear, resulting in significant computational challenges when employed in process synthesis and intensification frameworks [1]. To alleviate the computational burden, data-driven surrogate modeling approaches such as artificial neural networks, polynomial response surfaces, Kriging, or Gaussian processes can be employed [2-3]. Nevertheless, one of the major challenges of these models is the non-deterministic nature of their approximation types and overall model prediction error. For example, current ANN-based models cannot always guarantee the desired approximation types (underestimation or overestimation), and purely data-driven surrogates cannot provide bounds on the prediction error over the entire domain of interest [4].

To overcome these challenges, we focus on two key aspects: developing simple-yet-accurate surrogates with guaranteed error-bounds, and using them to solve a generic system of equations that may constitute difficult-to-converge simulation problems. We propose a data-driven surrogate modeling approach that quantifies and ensures a guaranteed global maximum bound on the model prediction error [5-7]. The technique utilizes the upper bound on the diagonal Hessian elements of the model to construct edge-concave underestimators and edge-convex overestimators [8]. The simplices formed by the edge-concavity/convexity-based estimators are able to provide guaranteed under-or-over approximation based only on data samples at select points [9]. A non-interpolating surrogate is then obtained by solving a parameter estimation problem with additional novel constraints, such as, enforcing the surrogate approximation to be bounded between the vertex polyhedral under- and over-estimators of the original model, thereby providing guaranteed error bounds. Repeated application of the parameter-estimation problem within a branch-and-bound framework ultimately yields a guaranteed solution to the system of equations.

To facilitate the use of our approach, we have also developed a package called GEMS (Guaranteed Error-bounded Modeling of Surrogates) that integrates all the associated tasks, including the selection of the location of sample evaluations, Hessian bounds estimation, and generation of the surrogate forms such as QRS, kriging, or ANN [7]. GEMS utilizes the minimum number of sample points while ensuring sufficient exploration of the entire domain. Using in-house automatic differentiation schemes and interval-arithmetic rules, GEMS can also calculate the guaranteed upper bound on the Hessian of high dimensional models. These bounds are then utilized to formulate the novel constraints within the parameter-estimation problem to obtain the surrogates. To demonstrate the applicability of the surrogate modeling technique, we consider a process simulation case study that involves recycling loops. We extract the underlying model equations and utilize the GEMS framework to solve the simulation problem, i.e., find roots of the model equations. We demonstrate that the solution returned by GEMS is a valid solution to the simulation problem. The quantified global prediction error of the surrogate progressively becomes smaller as the branching occurs, leading to an efficient solution to large-scale computationally expensive simulation problems.

Keywords: Surrogate modeling, Underestimation, Error bounds, Hessian bounds, Edge-concavity, Data-driven Modeling.

References:

[1] Demirel, S. E., Li, J., Hasan, M. M. F. (2017). Systematic process intensification using building blocks. Computers & Chemical Engineering, 105, 2–38.

[2] Schweidtmann, A.M., Mitsos, A. 2019. Deterministic global optimization with artificial neural networks embedded. Journal of Optimization Theory and Applications,180(3), pp. 925–948.

[3] Tsay, C., Kronqvist, J., Thebelt, A., Misener, R. 2021. Partition-Based Formulations for Mixed-Integer Optimization of Trained ReLU Neural Networks. Advances in Neural Information Processing Systems, 34, pp. 3068—3080.

[4] Bhosekar, A., Ierapetritou, M. 2018. Advances in surrogate based modeling, feasibility analysis, and optimization: A review. Computers & Chemical Engineering, 108, pp. 250–267.

[5] Iftakher, A., Aras, C.M., Monjur, M.S., Hasan, M. M. F., 2022. Data-driven approximation of thermodynamic phase equilibria. AIChE Journal, e17624.

[6] Iftakher, A., Aras, C.M., Monjur, M.S., Hasan, M. M. F., 2022. GEMS: Guaranteed Error-bounded Modeling of Surrogates. Proceedings of the 14th International Symposium on Process Systems Engineering – PSE 2021+, 49, pp. 1831–1836.

[7] Iftakher, A., Aras, C.M., Monjur, M.S., Hasan, M. M. F., 2022. A Framework for Guaranteed Error-bounded Surrogate Modeling. Proceedings of the 2022 American Control Conference (ACC), pp. 4814-4819.

[8] Hasan, M. M. F., 2018, An Edge-concave Underestimator for the Global Optimization of Twice-differentiable Nonconvex Problems. Journal of Global Optimization, 71(4), pp. 735–752.

[9] Bajaj, I.; Hasan, M. M. F., 2020. Deterministic Global Derivative-free Optimization of Black-Box Problems with Bounded Hessian. Optimization Letters, 14, pp. 1011–1026.