(578a) The Melon Toolbox: Machine Learning Models for Optimization

Data-driven models are increasingly popular in process systems engineering and are believed to become even more important through the digitalization of industry [1]. In the previous literature, there have been numerous efforts to integrate trained surrogate models into optimization problems. For instance, Gaussian processes (GPs) and artificial neural networks (ANNs) have been used in multiple studies for process modeling and optimal design (e.g. GPs in [2-5], ANNs in [6-8]). In these works, data-driven models learn unit operations and are aggregated to hybrid mechanistic/data-driven process models, which are finally optimized. However, previous studies were often limited to local optimization or stochastic global solution approaches (e.g., genetic algorithm). In addition, most studies relied on tedious manual implementation of trained models into optimization modeling environments such as GAMS.

Recently, we have shown that a reduced-space formulation is favorable for deterministic global optimization with data-driven models embedded [9,10]. We propagate McCormick relaxations [11,12] through the model equations which allows operating in the degrees of freedom. Thus, the branch-and-bound algorithms branches only on the degrees of freedom and the sizes of the resulting subproblems are significantly reduced. This leads to significant speedups compared to a conventional full-space formulation and other solvers. For instance, we observed speedup factors of over 300 for ANNs and over 700 for GPs.

We propose the “Machine Learning Models for Optimization” (MeLOn) toolbox. The MeLOn toolbox provides a model library that includes various data-driven models from the machine-learning community. It interfaces with the state-of-the-art training toolboxes in TensorFlow, scikit-learn, and Matlab. Further, the extension to other toolboxes is possible. MeLOn [13] allows the user to easily integrate trained data-driven models into optimization problems. The tool is coupled to our open-source global solver MAiNGO which can solve problems in the reduced-space formulation. Therein, we have implemented convex and concave envelopes of relevant functions (e.g., activation functions of ANNs or covariance functions of GPs). This further accelerates optimization. In addition, we can parse optimization problems to GAMS. In this talk we provide an overview about recent advances in global optimization with data-driven models embedded. We also introduce the MeLOn toolbox and highlight a few engineering case studies.

The toolbox is open-source available at https://git.rwth-aachen.de/avt.svt/public/MeLOn/

[1] Lee, J. H., Shin, J., & Realff, M. J. (2018). Machine learning: Overview of the recent progresses and implications for the process systems engineering field. Computers & Chemical Engineering, 114, 111-121.

[2] Cozad, A., Sahinidis, N. V., & Miller, D. C. (2014). Learning surrogate models for simulation‐based optimization. AIChE Journal, 60(6), 2211-2227.

[3] Quirante, N., Javaloyes, J., Ruiz-Femenia, R., & Caballero, J. A. (2015). Optimization of Chemical Processes Using Surrogate Models Based on a Kriging Interpolation. In Computer Aided Chemical Engineering (Vol. 37, pp. 179-184). Elsevier.

[4] Boukouvala, F., & Ierapetritou, M. G. (2012). Feasibility analysis of black-box processes using an adaptive sampling Kriging-based method. Computers & Chemical Engineering, 36, 358-368.

[5] Bradford, E., Schweidtmann, A. M., Zhang, D., Jing, K., & del Rio-Chanona, E. A. (2018). Dynamic modeling and optimization of sustainable algal production with uncertainty using multivariate Gaussian processes. Computers & Chemical Engineering, 118, 143-158.

[6] Meireles, M., Almeida, P., Simoes, M.G.: A comprehensive review for industrial applicability of artificial neural networks. IEEE Transactions on Industrial Electronics 50(3), 585-601 (2003).

[7] Nascimento, C.A.O., Giudici, R., Guardani, R.: Neural network based approach for optimization of industrial chemical processes. Computers & Chemical Engineering 24(9-10), 2303-2314 (2000).

[8] Henao, C.A., Maravelias, C.T.: Surrogate-based superstructure optimization framework. AIChE Journal 57(5), 1216-1232 (2011).

[9] Schweidtmann, A. M., & Mitsos, A. (2019). Deterministic global optimization with artificial neural networks embedded. Journal of Optimization Theory and Applications, 180(3), 925-948.

[10] Schweidtmann, A. M., Bongartz, D., Grothe, D., Kerkenhoff, T., Lin, X., Najman, J., & Mitsos, A. (2020). Deterministic global optimization with Gaussian processes embedded in a reduced variable space. In Preparation.

[11] Mitsos, A., Chachuat, B., & Barton, P. I. (2009). McCormick-based relaxations of algorithms. SIAM Journal on Optimization, 20(2), 573-601.

[12] McCormick, G. P. (1976). Computability of global solutions to factorable nonconvex programs: Part I—Convex underestimating problems. Mathematical programming, 10(1), 147-175.

[13] Bongartz, D., Najman, J., Sass, S., Mitsos, A. (2018). MAiNGO: McCormick based Algorithm for mixed integer Nonlinear Global Optimization. Technical report.