(433e) Robust Learning and Predictive Control of Time-Delay Nonlinear Processes with Deep Recurrent Koopman Operators

Within the context of advanced control of large-scale industrial processes (e.g., chemical processes, pharmaceutical processes, wastewater treatment), time delays can arise from various sources, such as the duration required by lab analyzers to provide sample measurements, complex commutation, and information exchange (especially when networked communication is involved) [1-3]. While time delays may be considered small most of the times considering the slow dynamics of large-scale industrial processes, their presence has indeed posed significant challenges to the modeling and control performance of such systems, therefore considerable effort has long been devoted to overcoming these challenges [1,3]. The control of nonlinear systems with unknown time delays has been a challenging problem over the last decades. An attempt was initially made in [4] to extend the backstepping method to time-delay nonlinear systems. To circumvent the loop design issue produced by backstepping method [5], certain growth restrictions were enforced on the time-delay nonlinearities in [6]. By using the dynamic scaling change technique and adding a power integrator proposed in [7,8] addressed the global stabilization of feedforward nonlinear time-delay systems.

As the scale of the nonlinear processes and the complexity of the tasks grow, machine learning-based modeling and control have drawn many interests. However, a general and reliable data-driven framework for the modeling and control of nonlinear uncertain systems with time delays has yet to come. In past decades, the success of machine learning has motivated its applications in system identification and control [9]. The first class of these advancements trains artificial neural networks with data to approximate the dynamics of target systems [10]. The modeling problems are formulated as supervised learning problems, and neural networks are trained to infer the system dynamical behaviors based on the states/measurements and control inputs. However, with the complex and over-parameterized neural networks, the learned models become restrictive when it comes to dynamics analysis and controller design. Another group of works is model-free, where control laws are learned out of data directly, e.g. reinforcement learning, adaptive dynamic programming, and iterative learning control.

Recently, the Koopman operator theory has attracted much attention, owing to its ability to model and represent the dynamics of complex nonlinear systems as a linear system on a high dimensional observable space [11]. The transformation of nonlinearity to linearity makes it possible to apply linear control theories to the analysis and control of nonlinear systems. Motivated by this insight, practical algorithms like dynamic mode decomposition (DMD) and extended dynamic mode decomposition (EDMD), were developed and applied to the modeling of various engineering systems, including chemical processes [12,13]. The observable functions determine which features to be extracted from the state information and used for future state prediction, thus setting up a proper group of observable functions in the Koopman method is crucial yet difficult. In order to automate the design of observable functions, deep learning techniques were naturally introduced and formed the Deep-DMD methods [14-16]. However, these methods focus on modeling nominal systems with clean data sets, while noise and time delays were left out of consideration.

In this work, we present a novel learning control framework called Deep Recurrent Koopman Operator (DRKO) for uncertain nonlinear systems with time delays. We train a deep recurrent neural network to map the original state/measurement trajectory to a high-dimensional probabilistic distribution and then learn a set of linear operators to predict future states. Within the proposed DRKO framework, a computationally efficient robust predictive control scheme is developed. The stability of the closed-loop system is guaranteed by the robust control framework even in the presence of modeling errors. Finally, the performance of DRKO is illustrated via the application of a reactor-separator process example, and the superiority of the proposed method is demonstrated in comparison with baselines.

References

1. Ellis M., Christofides, P. D., Economic model predictive control of nonlinear time-delay systems: Closed-loop stability and delay compensation. AIChE Journal, 2015, 61(12): 4152-4165.
2. Rashedi M., Liu J., Huang B., Communication delays and data losses in distributed adaptive high‐gain EKF. AIChE Journal, 2016, 62(12): 4321-4333.
3. Liu J., Chen X, de la Peña D. M., Christofides P. D., Iterative distributed model predictive control of nonlinear systems: Handling asynchronous, delayed measurements. IEEE Transactions on Automatic Control, 2011, 57(2): 528-534.
4. Nguang S. K., Robust stabilization of a class of time-delay nonlinear systems. IEEE Transactions on Automatic Control, 2000, 45(4): 756-762.
5. Zhou S., Feng G., Nguang S. K., Comments on “robust stabilization of a class of time-delay nonlinear systems”. IEEE Transactions on Automatic Control, 2002, 47(9): 1586.
6. Hua C. C., Wang Q. G., Guan X. P., Memoryless state feedback controller design for time delay systems with matched uncertain nonlinearities. IEEE Transactions on Automatic Control, 2008, 53(3): 801-807.
7. Qian C., Lin W., Non-Lipschitz continuous stabilizers for nonlinear systems with uncontrollable unstable linearization. Systems & Control Letters, 2001, 42(3): 185-200.
8. Zhang X., Liu Q., Baron L, et al. Feedback stabilization for high order feedforward nonlinear time-delay systems. Automatica, 2011, 47(5): 962-967.
9. Hewing L., Wabersich K. P., Menner M., et al. Learning-based model predictive control: Toward safe learning in control. Annual Review of Control, Robotics, and Autonomous Systems, 2020, 3: 269-296.
10. Nagabandi A., Kahn G., Fearing R. S., et al. Neural network dynamics for model-based deep reinforcement learning with model-free fine-tuning, IEEE international conference on robotics and automation (ICRA). 2018: 7559-7566.
11. Koopman B. O., Hamiltonian systems and transformation in Hilbert space. Proceedings of the National Academy of Sciences, 1931, 17(5): 315-318.
12. Narasingam A., Kwon J. S.-I., Koopman Lyapunov‐based model predictive control of nonlinear chemical process systems. AIChE Journal, 2019, 65(11): e16743.
13. Narasingam A., Kwon J. S.-I., Application of Koopman operator for model-based control of fracture propagation and proppant transport in hydraulic fracturing operation. Journal of Process Control, 2020, 91: 25-36.
14. Lusch B., Kutz J. N., Brunton S. L., Deep learning for universal linear embeddings of nonlinear dynamics. Nature Communications, 2018, 9(1): 4950.
15. Morton J., Jameson A., Kochenderfer M. J., et al. Deep dynamical modeling and control of unsteady fluid flows. Advances in Neural Information Processing Systems, 2018, 31.
16. Han Y., Hao W., Vaidya U., Deep learning of Koopman representation for control. 2020 59th IEEE Conference on Decision and Control (CDC). 2020: 1890-1895.