(193b) Improvements to EAGO.jl: Consolidation and Performance

Since its introduction in 2017, we have continued to improve and develop the open-source deterministic global optimizer EAGO.jl [1,2]. At a fundamental level, EAGO was constructed as a research platform: nearly all subroutines are implemented such that they can be extended to accommodate novel use cases, such as testing custom relaxations or alternative bounding routines, and a core design feature of EAGO is its ability to solve optimization problems embedded with complex user-defined functions. This deterministic global optimizer utilizes a reduced-space approach (as opposed to “expanded-space” auxiliary variable methods), like the global optimizer MAiNGO [3], which has remained an active area in optimization research. Due to EAGO’s flexibility, we have been able to seamlessly incorporate many such developments behind the scenes, which enables our users to take advantage of the latest tools and techniques in the global optimization community. To date, this includes recent advances in reduced-space McCormick relaxations [4-7], domain reduction techniques [8-10], improved linearization methods [11], and improved handling of bilinear terms [12], as well as methods to address semi-infinite programs [4,13], dynamic optimization problems [14], and problems with embedded implicit functions [13,15].

In this talk, we will discuss our recent efforts to improve the performance and user-accessibility of EAGO. This includes efforts to consolidate routines built for different problem types into a single catch-all function---to enable a simplified experience for typical users---as well as our successes in implementing multi-core CPU methods and our latest GPU-based global optimization methods [16-18] into EAGO’s base functionality. Many of these recent---as well as historical---developments are made possible by the unique features and high performance of the Julia programming language [19], and by the efforts of other active members in the community who continue to develop and maintain projects for general optimization (e.g., JuMP-dev [20] and MathOptInterface [21]), machine learning (e.g., SciML [22,23] and Flux [24,25]), and other general packages for numerical analysis. Our talk will cover use-case examples of EAGO that make use of some of these packages to showcase the power and capability that this tool can offer to researchers in PSE and beyond.

References:

1. Matthew E. Wilhelm and Matthew D. Stuber. Easy advanced global optimization (EAGO): An open-source platform for robust and global optimization in Julia. In AIChE Annual Meeting. AIChE, 2017.
2. Matthew E. Wilhelm and Matthew D. Stuber. EAGO.jl: easy advanced global optimization in Julia. Optimization Methods and Software, 37(2):425–450, aug 2022. doi: 10.1080/10556788.2020.1786566.
3. Dominik Bongartz, Jaromil Najman, Susanne Sass, and Alexander Mitsos. MAiNGO - McCormick-based Algorithm for mixed-integer Nonlinear Global Optimization. Technical report, RWTH-Aachen, 2018. URL https://www.avt.rwth-aachen.de/global/show_document.asp?id=aaaaaaaaabclahw.
4. Alexander Mitsos, Benoît Chachuat, and Paul I. Barton. McCormick-based relaxations of algorithms. SIAM Journal on Optimization, 20(2):573–601, jan 2009. doi: 10.1137/080717341.
5. Joseph K. Scott, Matthew D. Stuber, and Paul I. Barton. Generalized McCormick relaxations. Journal of Global Optimization, 51(4):569–606, feb 2011. doi: 10.1007/s10898-011-9664-7.
6. Angelos Tsoukalas and Alexander Mitsos. Multivariate McCormick relaxations. Journal of Global Optimization, 59(2-3):633–662, apr 2014. doi: 10.1007/s10898-014-0176-0.
7. Kamil A. Khan, Harry A. J. Watson, and Paul I. Barton. Differentiable McCormick relaxations. Journal of Global Optimization, 67(4):687–729, may 2016. doi: 10.1007/s10898-016-0440-6.
8. Stefan Vigerske. Decomposition in Multistage Stochastic Programming and a Constraint Integer Programming Approach to Mixed-Integer Nonlinear Programming. PhD thesis, Humboldt-Universität zu Berlin, Mathematisch-Naturwissenschaftlichen Fakultät II, 2013.
9. Achim Wechsung, Joseph K. Scott, Harry A. J. Watson, and Paul I. Barton. Reverse propagation of McCormick relaxations. Journal of Global Optimization, 63(1):1–36, apr 2015. doi: 10.1007/s10898-015-0303-6.
10. Ambros M. Gleixner, Timo Berthold, Benjamin Müller, and Stefan Weltge. Three enhancements for optimization-based bound tightening. Journal of Global Optimization, 67(4):731–757, jun 2017. doi: 10.1007/s10898-016-0450-4.
11. Jaromił Najman, Dominik Bongartz, and Alexander Mitsos. Linearization of McCormick relaxations and hybridization with the auxiliary variable method. Journal of Global Optimization, feb 2021. doi: 10.1007/s10898-020-00977-x.
12. Matthew E. Wilhelm and Matthew D. Stuber. Improved convex and concave relaxations of composite bilinear forms. Journal of Optimization Theory and Applications, mar 2023. doi: 10.1007/s10957-023-02196-2.
13. Matthew D. Stuber, Joseph K. Scott, and Paul I. Barton. Convex and concave relaxations of implicit functions. Optimization Methods and Software, 30(3):424–460, jun 2015. doi: 10.1080/10556788.2014.924514.
14. Matthew E. Wilhelm, Anne V. Le, and Matthew D. Stuber. Global optimization of stiff dynamical systems. AIChE Journal, 65(12), nov 2019. Doi: 10.1002/aic.16836
15. Matthew D. Stuber and Paul I. Barton. Semi-infinite optimization with implicit functions. Industrial & Engineering Chemistry Research, 54(1):307–317, dec 2015. doi: 10.1021/ie5029123.
16. Robert X. Gottlieb and Matthew D. Stuber. Global dynamic optimization using hardware-accelerated programming. In AIChE Annual Meeting 2022, Phoenix, AZ, November 2022.
17. Robert X. Gottlieb, Pengfei Xu, and Matthew D. Stuber. Automatic source code generation of complicated models for deterministic global optimization with parallel architectures. In FOCAPO-CPC 2023, San Antonio, TX, January 2023.
18. Robert X. Gottlieb, Pengfei Xu, and Matthew D. Stuber. Automatic Source Code Generation for Deterministic Global Optimization with Parallel Architectures. Under Review.
19. Jeff Bezanson, Alan Edelman, Stefan Karpinski, and Viral B. Shah. Julia: A fresh approach to numerical computing, 2014.
20. Iain Dunning, Joey Huchette, and Miles Lubin. JuMP: A modeling language for mathematical optimization. SIAM Review, 59(2):295–320, jan 2017. doi: 10.1137/15m1020575.
21. Benoît Legat, Oscar Dowson, Joaquim Dias Garcia, and Miles Lubin. MathOptInterface: a data structure for mathematical optimization problems. arXiv preprint arXiv:2002.03447, 2020. doi: 10.48550/ARXIV.2002.03447.
22. Yingbo Ma, Shashi Gowda, Ranjan Anantharaman, Chris Laughman, Viral Shah, and Chris Rackauckas. ModelingToolkit: A composable graph transformation system for equation-based modeling. arXiv preprint arXiv:2103.05244, 2021. doi: 10.48550/ARXIV.2103.05244.
23. Christopher Rackauckas and Qing Nie. DifferentialEquations.jl – a performant and feature-rich ecosystem for solving differential equations in Julia. Journal of Open Research Software, 5, may 2017. doi: 10.5334/jors.151.
24. Michael Innes, Elliot Saba, Keno Fischer, Dhairya Gandhi, Marco Concetto Rudilosso, Neethu Mariya Joy, Tejan Karmali, Avik Pal, and Viral Shah. Fashionable modelling with Flux. arXiv preprint arXiv:1811.01457, 2018. doi: 10.48550/ARXIV.1811.01457.
25. Michael Innes. Flux: Elegant machine learning with Julia. Journal of Open Source Software, 3(25):602, may 2018. doi: 10.21105/joss.00602.