(193h) Infiniteexamodels.Jl: Accelerating Infinite-Dimensional Optimization Problems on Cpus and GPUs

Infinite-dimensional optimization (InfiniteOpt) problems are ubiquitous in process systems engineering and entail decision variables defined over continuous domains [1]. Application areas include optimal control, stochastic optimization, and partial differential equation (PDE)-constrained optimization. The infinite-dimensional nature of these problems typically mandates the use of discretization schemes (e.g., orthogonal collocation over finite elements) to solve these problems by numerical means [2]. Relying on a manual approach, given the intricate nature of the discretization procedures, is susceptible to errors and often falls short of employing the most effective discretization strategies [3]. Hence, modeling environments such as pyomo.dae [3], InfiniteOpt.jl [1], and GEKKO [4] enable the use of sophisticated discretization schemes for a variety of InfiniteOpt problem types. Building on the popular algebraic modeling language, JuMP.jl, InfiniteOpt.jl is uniquely able to model InfiniteOpt formulations with both stochastic modeling elements (e.g., risk measures) and differential & algebraic equations (DAEs).

The discretized formulations produced by these frameworks are often cast as large-scale nonlinear programs, potentially involving millions of variables which can surpass the capabilities of existing algorithms and software tools. Discretized InfiniteOpt problems typically exhibit a highly recurrent structure which is not exploited by most existing modeling environments. For example, a reaction kinetics equality constraint can be enforced over thousands of random scenarios [5]. Leveraging this inherent structure has great potential to enable efficient (parallelizable) solution routines for discretized InfiniteOpt problems. For instance, exploiting the recurrent structure can significantly reduce the computational burden of expensive automatic-differentiation (AD) routines on complicated nonlinear expressions and enable the use of solution routines that rely on GPUs.

In [6], Dr. Shin and colleagues present a single-instruction, multiple-data (SIMD) abstraction for nonlinear programs (NLPs) which is implemented in the open-source Julia package ExaModels.jl. Contrary to general purpose algebraic modeling languages like JuMP.jl [7], Pyomo [8], and Gravity [9] which do not save repeated structure, ExaModels.jl preserves the parallelizable structure in the model, and in turn, exploits that structure for more efficient computations on multi-threaded CPUs or GPU accelerators. The authors in [6] demonstrate how ExaModels.jl leverages the repeated structure in large-scale AC optimal power flow (ACOPF) problems to speed up AD by 1 to 2 orders-of-magnitude relative to JuMP.jl. Moreover, an order-of-magnitude speed on the overall solution time for ACOPF problems can be achieved on GPUs using ExaModels.jl and MadNLP.jl in contrast to state-of-the-art CPU solvers [6]. The utility of these modeling/solution approaches has not yet been investigated on discretized InfiniteOpt problems.

Thus, we present InfiniteExaModels.jl [10], as a new backend for InfiniteOpt.jl to automatically discretize InfiniteOpt problems into the SIMD-NLP representation used by ExaModels.jl. This backend facilitates an intuitive modeling environment for InfiniteOpt problems that gives ExaModels.jl direct access to the repeated structures exhibited by discretized InfiniteOpt models which enable highly efficient AD routines that are parallelizable and amenable for MadNLP.jl’s state-of-the-art GPU NLP solver. This framework is general and can significantly accelerate the solution of nonlinear InfiniteOpt models across an array of applications.

References

[1] J. L. Pulsipher, W. Zhang, T. J. Hongisto, V. M. Zavala, 2022, A unifying modeling abstraction for infinite-dimensional optimization, Computers & Chemical Engineering 156, 107567

[2] L. T. Biegler, 2010, Nonlinear programming: concepts, algorithms, and applications to chemical processes, SIAM, Philadelphia, USA

[3] B. Nicholson, J. D. Siirola, J.-P. Watson, V. M. Zavala, L. T. Biegler, 2018, pyomo. dae: A modeling and automatic discretization framework for optimization with differential and algebraic equations, Mathematical Programming Computation 10, 187–223

[4] L. D. Beal, D. C. Hill, R. A. Martin, J. D. Hedengren, 2018, Gekko optimization suite, Processes 6 (8), 106

[5] W. Chen, L. T. Biegler, S. G. Muñoz, 2016, An approach for simultaneous estimation of reaction kinetics and curve resolution from process and spectral data, Journal of Chemometrics 30 (9), 506–522

[6] S. Shin, F. Pacaud, M. Anitescu, 2023, Accelerating optimal power flow with gpus: Simd abstraction of nonlinear programs and condensed-space interior-point methods, arXiv preprint arXiv:2307.16830

[7] M. Lubin, O. Dowson, J. D. Garcia, J. Huchette, B. Legat, J. P. Vielma, 2023, Jump 1.0: recent improvements to a modeling language for mathematical optimization, Mathematical Programming Computation, 1–9

[8] W. E. Hart, C. D. Laird, J.-P. Watson, D. L. Woodruff, G. A. Hackebeil, B. L. Nicholson, J. D. Siirola, et al., 2017, Pyomo-optimization modeling in python, Vol. 67, Springer

[9] H. Hijazi, G. Wang, C. Coffrin, 2018, Gravity: A mathematical modeling language for optimization and machine learning, NIPS 2018 Workshop MLOSS, Montreal, Canada

[10] J. L. Pulsipher, S. Shin, 2024, Scalable Modeling of Infinite-Dimensional Nonlinear Programs with InfiniteExaModels.jl, ESCAPE-PSE 2024, Florence, Italy