Reliable prediction, optimization, and control of complex systems often involve rigorous mechanistic models that represent the underlying equilibrium, transport, and thermodynamic relationships. Such first principles-based mathematical modeling almost always results in large nonlinear and nonconvex systems of equations. These models, when employed in process synthesis and intensification frameworks, pose significant computational challenges. To alleviate the computational burden, data-driven surrogate modeling approaches (e.g., artificial neural networks (ANN), quadratic response surface (QRS), kriging or Gaussian processes, etc.) can be an alternative [1-2]. However, major challenges of surrogate models include the nondeterministic natures of the type of approximation and the overall model prediction error. For example, current ANN-based models cannot always guarantee the desired approximation types (underestimation or overestimation) [3]. Also, purely data-driven surrogates cannot provide bounds on the prediction error over the entire domain of interest. In this work, we address these issues by developing a data-driven surrogate modeling approach that provides guaranteed under-or-over estimation and quantifies a global maximum bound on the model prediction error [4-6]. Utilizing the upper bound on the diagonal Hessian elements of the model, the technique constructs edge-concave underestimators and edge-convex overestimators [7]. The simplices formed by the edge-concavity/convexity-based estimators are able to provide guaranteed under-or-over approximation only based on data samples at select points [8]. An interpolating or non-interpolating surrogate model is then obtained by enforcing the approximation to be bounded between the vertex polyhedral under- and over-estimators of the original model thereby providing guaranteed error bounds. We also develop a package named GEMS (Guaranteed Error-bounded Modeling of Surrogates) that integrates all the associated tasks which include the selection of the location of sample evaluations, Hessian bounds estimation, and generation of the surrogate forms (e.g., QRS, kriging, or ANN) [6]. GEMS utilizes the maximum likelihood of improvement to minimize the number of sample points while ensuring sufficient exploration of the entire domain. Using the inbuilt automatic differentiation schemes, GEMS can also calculate the upper bound on the Hessian of high dimensional models. We demonstrate the applicability of the surrogate modeling technique in a thermodynamic case study by utilizing it in our SPICE framework for extractive-distillation-based refrigerant separation tasks [9-10]. We observe that the suggested surrogate by GEMS, when used in place of computationally expensive high-fidelity thermodynamic models, exhibits excellent agreement in terms of the key predicted properties. We anticipate that such quantified global prediction error obtained from edge-concavity-based tight relaxations would lead to efficient branch-and-bound based global optimization of large-scale optimization problems.
Keywords: Surrogate Modeling, Guaranteed Error-bound, Simulation-based Optimization, Data-driven Modeling.
References:
[1] Schweidtmann, A.M., Mitsos, A. 2019. Deterministic global optimization with artificial neural networks embedded. Journal of Optimization Theory and Applications,180(3), pp. 925–948.
[2] Tsay, C., Kronqvist, J., Thebelt, A., Misener, R. 2021. Partition-Based Formulations for Mixed-Integer Optimization of Trained ReLU Neural Networks. Advances in Neural Information Processing Systems, 34, pp. 3068—3080.
[3] Bhosekar, A., Ierapetritou, M. 2018. Advances in surrogate based modeling, feasibility analysis, and optimization: A review. Computers & Chemical Engineering, 108, pp. 250–267.
[4] Iftakher, A., Aras, C.M., Monjur, M.S., Hasan, M. M. F., 2022. Data-driven approximation of thermodynamic phase equilibria. AIChE Journal, e17624.
[5] Iftakher, A., Aras, C.M., Monjur, M.S., Hasan, M. M. F., 2022. GEMS: Guaranteed Error-bounded Modeling of Surrogates. Proceedings of the 14th International Symposium on Process Systems Engineering – PSE 2021+, Accepted.
[6] Iftakher, A., Aras, C.M., Monjur, M.S., Hasan, M. M. F., 2022. A Framework for Guaranteed Error-bounded Surrogate Modeling. Proceedings of the 2022 American Control Conference (ACC), Accepted.
[7] Hasan, M. M. F., 2018, An Edge-concave Underestimator for the Global Optimization of Twice-differentiable Nonconvex Problems. Journal of Global Optimization, 71(4), pp. 735–752.
[8] Bajaj, I.; Hasan, M. M. F., 2020. Deterministic Global Derivative-free Optimization of Black-Box Problems with Bounded Hessian. Optimization Letters, 14, pp. 1011–1026.
[9] Monjur, M.S., Iftakher, A. and Hasan, M. M. F., 2022. Separation Process Synthesis for High-GWP Refrigerant Mixtures: Extractive Distillation using Ionic Liquids. Industrial & Engineering Chemistry Research, 61(12), pp. 4390–4406.
[10] Monjur, M.S., Iftakher, A. and Hasan, M. M. F., Ionic Liquid-based Energy Efficient Separation Pathways for High-GWP Refrigerant Mixtures: Solvent Screening and Process Intensification. Under Review.
