(644c) Deep Learning-Based Approximate Economic Model Predictive Control with Offset-Free Asymptotic Performance Guarantees Using a Modifier-Adaptation Scheme

Model predictive control (MPC) is a popular optimization-based control strategy due to its ability to directly account for multivariable dynamics and constraints [1]. Since MPC can be flexibly adapted by modifying the objective and/or constraints in the underlying dynamic optimization problem, it has been increasingly used in settings that go beyond traditional setpoint tracking such as nonlinear and economic control tasks [2]. An important challenge in nonlinear forms of MPC is the potentially large computational (and memory) burden associated with repeatedly solving a generally non-convex optimal control problem in real-time. Explicit MPC is an alternative to this real-time optimization approach wherein the solution to the optimal control problem is determined offline for all feasible measured state values [3]. However, not only does the size of the mapping grow exponentially with problem size (in the worst-case), but explicit MPC is not easily extended to nonlinear problems due to increased complexity in the optimality conditions [4].

To overcome the challenges with online optimization and explicit MPC, there is increasing interest in “approximate” MPC policies that learn an explicit representation using any of the readily available function approximation techniques such as polynomials [5], radial basis functions [6], and neural networks [7-11]. The basic idea behind approximate MPC is as follows. First, the optimal control problem is solved offline for a large number of state realizations to obtain the corresponding optimal control input. Then, using this as training data, a parametric function is trained, which can then be used online to cheaply evaluate the control action. This process can be interpreted as a form of expert-based supervised learning or imitation learning [12]. Although good performance has been obtained with approximate MPC in several applications [13,14], errors in the approximation are inevitable due to the choice of “hyperparameters” that specify the functional form of the policy and insufficient training data in certain regions of the state space leading to poor generalization. These approximation errors lead to asymptotic losses in closed-loop performance.

In this presentation, we first propose a closed-loop training procedure for approximating economic MPC using deep neural networks (DNNs) in which the training samples are generated from a control-oriented representation of the state-space that is defined in terms of a tube of closed-loop trajectories. To account for the closed-loop asymptotic performance losses using the approximate policy, we then propose a Karush-Kuhn-Tucker (KKT)-derived online adaptive correction scheme, where the main idea is to add a corrective term to the approximate policy that ensures that the limit point of closed-loop system satisfies the KKT conditions of the corresponding steady-state optimization problem. Finally, to ensure convergence of the online adaptive correction scheme that is not guaranteed in the design of the DNN, we propose an offline performance verification method. The scenario-based performance verification scheme provides probabilistic guarantees that the closed-loop control system converges to a stable equilibrium point. The performance of the proposed approach is demonstrated using a benchmark Williams-Otto reactor example.

References

[1] D.Q. Mayne, “Model predictive control: Recent developments and future promise,” Automatica, vol. 50, no. 12, pp. 2967–2986, 2014.

[2] M. Diehl, H. G. Bock, J. P. Schloder, R. Findeisen, Z. Nagy, and F. Allgower, “Real-time optimization and nonlinear model predictive control of processes governed by differential-algebraic equations,” Journal of Process Control, vol. 12, pp. 577–585, 2002.

[3] A. Bemporad, M. Morari, V. Dua, and E. N. Pistikopoulos, “The explicit linear quadratic regulator for constrained systems,” Automatica, vol. 38, no. 1, pp. 3–20, 2002.

[4] A. Alessio, and A. Bemporad. "A survey on explicit model predictive control." Nonlinear model predictive control. Springer, Berlin, Heidelberg, 2009. 345-369.

[5] A. Chakrabarty, V. Dinh, M. J. Corless, A. E. Rundell, S. H. Z ̇ak, and G. T. Buzzard, “Support vector machine informed explicit nonlinear model predictive control using low-discrepancy sequences,” IEEE Transactions on Automatic Control, vol. 62, pp. 135–148, 2017.

[6] L. H. Cseko ̋, M. Kvasnica, and B. Lantos, “Explicit MPC-based RBF neural network controller design with discrete-time actual Kalman filter for semiactive suspension,” IEEE Transactions on Control Systems Technology, vol. 23, no. 5, pp. 1736–1753, 2015.

[7] S. Chen, K. Saulnier, N. Atanasov, D. D. Lee, V. Kumar, G. J. Pappas, and M. Morari, “Approximating explicit model predictive control using constrained neural networks,” in Proceedings of the American Control Conference, (Milwaukee), pp. 1520–1527, 2018.

[8] B. Karg, and S. Lucia. “Efficient representation and approximation of model predictive control laws via deep learning.” IEEE Transactions on Cybernetics, vol. 50, no. 9, pp. 3866-3878, 2020.

[9] J. A. Paulson and A. Mesbah. “Approximate closed-loop robust model predictive control with guaranteed stability and constraint satisfaction,” IEEE Control Systems Letters, vol. 4, no. 3, pp. 719–724, 2020.

[10] A.D. Bonzanini, J. A. Paulson, G. Makrygiorgos, and A. Mesbah. “Fast approximate learning-based multistage nonlinear model predictive control using Gaussian processes and deep neural networks,” Computers & Chemical Engineering, vol. 145, pp. 107174.

[11] R.J. Lovelett, F. Dietrich, S. Lee, and I.G. Kevrekidis. “Some manifold learning considerations toward explicit model predictive control,” AIChE Journal, vol. 66, pp. e16881.

[12] D.P. Bertsekas, “Reinforcement learning and optimal control,” Athena Scientific Belmont, MA., 2019

[13] A.D. Bonzanini, J. A. Paulson, D.B. Graves, and A. Mesbah. “Toward safe dose delivery in plasma medicine using projected neural network-based fast approximate NMPC,” In Proceedings of the IFAC World Congress, 5353-5359, 2020, Berlin, Germany.

[14] J. Drgona, D. Picard, M. Kvasnica, and L. Helsen. “Approximate model predictive building control via machine learning,” Applied Energy, vol. 218, pp. 199–216, 2018.