(182h) Dynamic Trajectory Optimization Via Dynamic Surrogate Modeling

Dynamic optimization has long been and will continue to be a fundamental task in process development and control. In addition to specifying optimal initial conditions of a process, selecting the time-varying controls to achieve a particular state path trajectory is key not only for economic reasons but may also be crucial for safety and feasibility reasons. However, numerical solutions of the dynamic optimization of engineering simulations for moderately-sized systems often become intractable as the number of states/control variables increase and as the modeled phenomena become more nonlinear. Many surrogate modeling-methods have been proposed for dynamic optimization including orthogonal decomposition approaches such as PCA or POD [1], spline-based such as DRSM [2], and universal approximators such as Neural Networks and Gaussian Processes [3]. However, none of these methods take into account the dynamic structure of the system.

Early work demonstrated a highly versatile approach for developing surrogate models for dynamic systems using Neural Networks [4]. These Neural Differential Equations (NDEs) have become increasingly flexible and efficient in recent years due to improvements in differential programming [5, 6] and have enabled the solution of complex problems in dynamic control [7]. By considering the dynamics of the system, a surrogate model with NDEs has increased interpretability while also offering a promising route to reduce the cost of dynamic simulations. With these advantages in mind, we propose a direct transcription methodology for control optimization based on NDEs. Specifically, we investigate the trajectory optimization of dynamic systems described by strongly nonlinear differential equations subject to path constraints. We also weigh the merits of using dynamic surrogate models versus non-dynamic surrogate models (e.g. Neural Networks, splines) for this task. This presentation will share the results of a systematic comparison between: (a) deterministic dynamic optimization techniques that use the original equations of the dynamic simulation directly (when feasible), (b) black-box surrogate-based optimization methods that employ steady-state surrogate models, and (c) the proposed technique that employs NDEs. This systematic comparison is performed on dynamic models with varying dimensionality and nonlinearity, with an emphasis on outlining which systems are most benefitted by NDE-approximations. Often, the optimal solution lies in regions beyond the range of data used to train the surrogate model. To avoid poor predictions due to surrogate extrapolation, an approach for the adaptive design of simulation experiments will be investigated. Finally, methods for ensuring the reliability of the optimized trajectory will be given by applying standard techniques in sensitivity analysis and uncertainty quantification of the converged solution.

References

1. Khowaja, K., M. Shcherbatyy, and W. Hardle. Surrogate Models for Optimization of Dynamical Systems. 2021.
2. Wang, Z., N. Klebanov, and C. Georgakis, DRSM Model for the Optimization and Control of Batch Processes. IFAC-PapersOnLine, 2016. 49(7): p. 55-60.
3. Belie, F., T. Lefebvre, and G. Crevecoeur, Optimizing state trajectories using surrogate models with application on a mechatronic example. 2016.
4. Psichogios, D.C. and L.H. Ungar, A hybrid neural network-first principles approach to process modeling. AIChE Journal, 1992. 38(10): p. 1499-1511.
5. Chen, R.T.Q., et al. Neural Ordinary Differential Equations. arXiv e-prints, 2018.
6. Rackauckas, C., et al. Universal Differential Equations for Scientific Machine Learning. arXiv e-prints, 2020. arXiv:2001.04385.
7. Luthje, J.T., et al. Adaptive Learning of Hybrid Models for Nonlinear Model Predictive Control of Distillation Columns. 2020.