(299f) Fast Approximate Multistage NMPC with Online Scenario Tree Generation Using Active Deep Learning

Nonlinear model predictive control (NMPC) has received increasing attention in recent years due to its ability to directly handle constrained nonlinear systems and more general control objectives that go beyond traditional setpoint tracking tasks [1]. Two obstacles encountered in NMPC design and implementation are: (i) the accuracy of the model and (ii) the large computational cost required to solve a non-convex optimization problem in real-time. In particular, model uncertainty leads to not only performance losses but also possible violation of critical safety, operational, and/or quality constraints. This issue has motivated the development of so-called robust NMPC schemes [2], which search for control actions that guarantee the model satisfies constraints under all possible uncertainties. Traditional min-max approaches [3], however, do not account for the fact that new information will be available in the future, which leads to highly conservative solutions. Closed-loop robust NMPC avoids this conservatism by optimizing over control policies instead of a sequence of control inputs, but the representation of control policies as arbitrary functions makes this formulation generally intractable. Multistage NMPC (msNMPC) [4] represents the uncertainty evolution in the form of a scenario tree composed of discrete realizations of the uncertainty. A critical challenge in msNMPC is that its computational complexity grows exponentially with the number of scenarios, leading to an even further increase in online computational cost (noted in the second challenge above) that must be addressed to achieve non-conservative performance.

A promising approach for overcoming the large computational cost in msNMPC is to approximate the implicitly defined control law with a deep neural network (DNN) [5-8]. It is important to note that these methods assume the uncertainty realizations are known a priori such that, even though the DNN is cheap to evaluate online, it cannot account for changes in the uncertainties over time (e.g., unknown parameters and their associated confidence regions can be estimated with new data collected online). The selection of these scenarios as well as their adaption have received relatively little attention in the literature, as recently noted in [9,10]. Thus, our main contribution presented in this talk is the development of a framework for dynamically updating the scenario tree in approximate msNMPC by treating the location of the scenarios (and their associated probabilities) as input parameters to the DNN. We also demonstrate that, when the considered uncertainties are related to structural plant-model mismatch, the scenario tree can be adapted in terms of Gaussian process (GP) models that can be learned online (e.g., [11]).

The higher-dimensional input space in the proposed DNN structure can complicate the training process. In particular, traditional supervised learning techniques are likely to become expensive and time-consuming, as a large number of samples are often needed to train DNNs with many inputs. Therefore, we also look to leverage active learning (AL) methods that sequentially enrich the training data by adding samples that are most likely to increase the accuracy of the DNN model (e.g., add samples from a pool that maximizes the variance in the DNN prediction). A Monte Carlo dropout-based technique can be used to quickly obtain an estimate of this uncertainty, which has shown to be effective in AL applications [12]. To tailor the training to the control task at hand, we introduce a new AL strategy that combines the Monte Carlo dropout-based approach with a closed-loop simulation phase where warm starting can be used to accelerate the generation of training data. The advantages of the proposed AL-based approximate msNMPC are illustrated for control of a kHz-excited atmospheric pressure plasma jet (APPJ) in helium [13]. The main goal is to deliver a target thermal dose as quickly as possible while respecting safety-critical constraints with actuation on the millisecond scale. We will show that the proposed controller can meet these requirements through a combination of the fast DNN evaluation and the online generated scenario tree that is able to adapt to variability seen during each run of the APPJ.

References

[1] M. Diehl, H. G. Bock, J. P. Schloder, R. Findeisen, Z. Nagy, and F. Allgower, “Real-time optimization and nonlinear model predictive control of processes governed by differential-algebraic equations,” Journal of Process Control, vol. 12, pp. 577–585, 2002.

[2] A. Bemporad and M. Morari, “Robust model predictive control: A survey,” in Robustness in Identification and Control. London, U.K., Springer, 1999, pp. 207–226.

[3] P. J. Campo and M. Morari, “Robust model predictive control,” in Proceedings of the American Control Conference, pp. 1021–1026, 1987.

[4] S. Lucia, T. Finkler, and S. Engell, “Multi-stage nonlinear model predictive control applied to a semi-batch polymerization reactor under uncertainty,” Journal of Process Control, vol. 23, pp. 1306–1319, 2013.

[5] J. A. Paulson and A. Mesbah, “Approximate closed-loop model predictive control with guaranteed stability and constraint satisfaction,” IEEE Control Systems Letters, 2020 (accepted).

[6] S. Lucia and B. Karg. “A deep learning-based approach to robust nonlinear model predictive control,” IFAC-PapersOnLine, vol. 51, pp. 511–516, 2018.

[7] M. Hertneck, J. Köhler, S. Trimpe, and F. Allgöwer, “Learning an approximate model predictive controller with guarantees,” IEEE Control Systems Letters, vol. 2, pp. 543–548, 2018.

[8] S. Chen et al., “Approximating explicit model predictive control using constrained neural networks,” in Proceedings of the American Control Conference, pp. 1520–1527, 2018.

[9] M. Thombre, D. Krishnamoorthy, and J. Jäschke, “Data-driven online adaption of the scenario-tree in multistage model predictive control,” IFAC-PapersOnLine, vol. 52, pp. 461–467, 2019.

[10] F. Holtorf, A. Mitsos, and L. T. Biegler, “Multistage NMPC with on-line generated scenario trees: Application to a semi-batch polymerization process,” Journal of Process Control, vol. 80, pp. 167–179, 2019.

[11] A. D. Bonzanini, J. A. Paulson and A. Mesbah, “Safe Learning-based Model Predictive Control under State-dependent Uncertainty using Scenario Trees,” in Proceedings of the IEEE Conference on Decision and Control, 2020 (submitted).

[12] E. Tsymbalov, M. Panov, and A. Shapeev, “Dropout-based active learning for regression,” International Conference on Analysis of Images, Social Networks, and Texts, 2018

[13] D. Gidon, D. B. Graves and A. Mesbah, "Effective dose delivery in atmospheric pressure plasma jets for plasma medicine: A model predictive control approach," Plasma Sources Science and Technology, pp. 85005-85019, 2017.