(250h) Meta-Adaptive Koopman Operators for Learning-Based Model Predictive Control of Parametrically Uncertain Nonlinear Systems

Parametric uncertainties are common in nonlinear systems, often arising from factors such as variations in payload and operating conditions [1]. The presence of these uncertainties can cause performance degradation and instability and pose great challenges to the design of control systems. The endeavor to ensure desired control system tracking performance and system stability has driven the advancement of adaptive control methods for nonlinear systems with parametric uncertainties over recent decades.

Model predictive control (MPC) is a popular advanced control technique [2]. It optimizes the future predicted behavior of the system by utilizing a dynamic model along with current measurement data. Adaptive model predictive control (AMPC) has been proposed to address uncertainties [3]; however, the results on nonlinear systems have been limited. More results on nonlinear AMPC can be found in [4,5]. Although these results represent significant advancements, first-principle models are typically required as the foundation of control system designs. The theoretical assumption of linear dependency on uncertain parameters further limits its applicability to general nonlinear processes.

Recently, the Koopman operator theory has gained substantial research attention, owing to its capability to represent the dynamics of complex nonlinear processes in a linear manner [6]. Several algorithms have been developed to construct linear Koopman models from process data. These include dynamic mode decomposition (DMD) [7] and extended dynamic mode decomposition (EDMD) [8]. DMD and EDMD represent the observables using a predetermined set of basis functions, and then solve least-squares problems to approximate the linear Koopman operator. To streamline the design of the observable functions, researchers have leveraged deep learning and proposed Deep-DMD Koopman modeling methods [9–11]. However, these existing approaches have primarily targeted addressing a specific control task with fixed model parameters.

The learning and control of parametrically uncertain systems can be conceptualized as a multi-task problem, which may be addressed using the meta-learning concept [12]. For instance, meta-reinforcement learning (meta-RL), a fusion of meta-learning and RL, has been developed for learning-based control in multi-task scenarios. Meta RL controllers utilize previously acquired knowledge and real-time data to adapt to new tasks, wherein the system dynamics, objectives, or distribution of noises and disturbances can vary. [13] introduced a meta-learning-based MPC framework capable of finetuning the meta-trained neural network (NN) model using online data. This method was applied to control a legged robot in the presence of changed payloads, terrains, and even a disabled leg. In [14], a novel offline meta-RL strategy was proposed for tuning proportional-integral (PI) controllers in process control systems. However, the online adaptation of deep NNs is inefficient and computationally demanding. Furthermore, it is generally challenging for these meta-RL approaches to offer stability and ensure closed-loop performance.

Based on these observations, we aim to integrate meta-learning with Koopman operator theory to create a learning-based adaptive control framework for parametrically uncertain nonlinear systems. Within the Koopman operator framework, we proposed a meta-adaptive Koopman operator (MAKO) modeling approach. This approach learns from a multi-modal dataset to construct a meta-model for online adaptation. An adaptation scheme is developed to update the meta-model using online data while ensuring convergence. Based on the adaptive meta-model, a predictive control scheme is proposed for the underlying uncertain nonlinear systems. We prove the stability of the closed-loop system in the presence of previously unseen parameter settings. The contributions of this work include: 1) Meta-learning and Koopman operator theory are integrated for the first time to establish a learning-based adaptive MPC framework applicable to a general class of parametrically uncertain nonlinear systems; 2) We rigorously analyze and prove the stability of both the model online adaptation and the closed-loop system; 3) Based on three benchmark systems from various fields, MAKO demonstrates good modeling accuracy and robust tracking control performance in the presence of parameter uncertainties, and it outperforms competitive baselines.

References

[1] Tao, “Multivariable adaptive control: A survey,” Automatica, vol. 50, no. 11, pp. 2737–2764, 2014.

[2] Q. Mayne, J. B. Rawlings, C. V. Rao, and P. O. Scokaert, “Constrained model predictive control: Stability and optimality,” Automatica, vol. 36, no. 6, pp. 789–814, 2000.

[3] Fukushima, T.-H. Kim, and T. Sugie, “Adaptive model predictive control for a class of constrained linear systems based on the comparison model,” Automatica, vol. 43, no. 2, pp. 301–308, 2007.

[4] DeHaan and M. Guay, “Adaptive robust MPC: A minimally-conservative approach,” in 2007 American Control Conference, 2007, pp. 3937-3942.

[5] Xu, A. Suleman, and Y. Shi, “multi-rate hierarchical fault-tolerant adaptive model predictive control framework: Theory and design for quadrotors,” Automatica, vol. 153, p. 111015, 2023.

[6] O. Koopman, “Hamiltonian systems and transformation in Hilbert space,” Proceedings of the National Academy of Sciences, vol. 17, no. 5, pp. 315–318, 1931.

[7] J. Schmid, “Dynamic mode decomposition of numerical and experimental data,” Journal of Fluid Mechanics, vol.656, pp. 5–28, 2010.

[8] Li, F. Dietrich, E. M. Bollt, and I. G. Kevrekidis, “Extended dynamic mode decomposition with dictionary learning: A data-driven adaptive spectral decomposition of the Koopman operator,” Chaos: An Interdisciplinary Journal of Nonlinear Science, vol. 27, no. 10, p. 103111, 2017.

[9] Morton, A. Jameson, M. J. Kochenderfer, and F. Witherden, “Deep dynamical modeling and control of unsteady fluid flows,” Advances in Neural Information Processing Systems, vol. 31, 2018.

[10] Han, Z. Li, X. Yin, and X. Yin, “Robust learning and control of time-delay nonlinear systems with deep recurrent Koopman operators,” IEEE Transactions on Industrial Informatics, vol. 20, no. 3, pp. 4675–4684, 2024.

[11] Shi and M. Q.-H. Meng, “Deep Koopman operator with control for nonlinear systems,” IEEE Robotics and Automation Letters, vol. 7, no. 3, pp. 7700–7707, 2022.

[12] Hospedales, A. Antoniou, P. Micaelli, and A. Storkey, “Meta-learning in neural networks: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 9, pp. 5149–5169, 2021.

[13] Nagabandi, I. Clavera, S. Liu, R. S. Fearing, P. Abbeel, S. Levine, and C. Finn, “Learning to adapt in dynamic, realworld environments through meta-reinforcement learning,” in International Conference on Learning Representations, 2018.

[14] G. McClement, N. P. Lawrence, J. U. Backstr¨om, P. D. Loewen, M. G. Forbes, and R. B. Gopaluni, “Metareinforcement learning for the tuning of PI controllers: An offline approach,” Journal of Process Control, vol. 118, pp. 139–152, 2022.