(194d) Advancing Control Performance through Adversarially Robust Reduced Order Model Predictive Control

Model Predictive Control (MPC) is a powerful technique for controlling process systems, but it faces a notable trade-off between control performance and computational efficiency. At one extreme, one may choose to employ accurate nonlinear models and solve a nonlinear MPC problem to gain in performance at the expense of computational efficiency. At the other extreme, one may opt for simple linear models to solve a linear MPC problem sacrificing performance for efficiency gains. This trade-off is exacerbated for large-scale systems, which are ubiquitous in plant-wide process control. In response to this challenge, Reduced Order Models (ROMs) emerge as a promising solution, as they preserve model fidelity (and thus control performance) but with reduced dimensionality allowing for a smaller and thus more efficient optimal control problem to be solved. However, state-of-the-art data-driven ROMs, such as autoencoders, often exhibit sensitivity to disturbances and noise when applied in control settings like MPC. This diminishes control performance, limiting their practical use. This work explores the integration of adversarial machine learning techniques to fortify ROMs against such perturbations, aiming to achieve enhanced robustness without compromising performance or efficiency.

The performance-efficiency trade-off inherent in MPC is illustrated in Figure 1. The x-axis represents “control efficiency” as measured by metrics such as the MPC solution time while the y-axis represents “control performance” gauged by metrics such as the set-point tracking error. On the right-hand side, symbolised by the red circle, we find solutions that prioritise performance at the cost of efficiency, such as nonlinear MPC. These solutions, although offering high performance, may not be practically implementable [1]. Conversely, on the left-hand side, denoted by the yellow circle, the opposite situation is found. While these solutions, such as linear MPC, are implementable and thus commonly employed in practice, the significant drop in performance might not be acceptable. Instead, the ideal solution lies in the middle ground, as indicated by the green circle, where we strike a balance between performance and efficiency. Indeed, ROMs provide a means to navigate this middle ground as evidenced by the literature [2, 3]. This is because accurate nonlinear ROMs can maintain good control performance due to minimal plant-model mismatch but with reduced dimensionality allowing for a smaller optimal control problem to be solved. Recognising this benefit, ROMs have been increasingly utilised in predictive control settings, including Reinforcement Learning (RL) and MPC [2, 3, 4]. Various ROM approaches exist, ranging from Singular Value Decomposition (SVD)-based linear methods like Proper Orthogonal Decomposition (POD) to nonlinear machine learning-based techniques such as autoencoders. Linear ROMs have been used quite heavily in the control literature as they provide desirable theoretical guarantees. However, their linear nature can lead to significant plant-model mismatch, resulting in poor control performance. On the other hand, machine learning-based ROMs offer improved performance as they are nonlinear in nature. Nonetheless, they tend to exhibit higher sensitivity to the training data characteristics, which may pose challenges in out-of-sample scenarios or in the presence of disturbances and noise.

Indeed, a common challenge encountered with data-driven ROMs is their sensitivity to the data used for training. Should the training data fail to encompass the expected noise and disturbance characteristics inherent of the true underlying process, ROMs might generate inaccurate or nonsensical predictions when deployed in control scenarios. This, in turn, can lead to erroneous control actions and ultimately a degradation in control performance. This sensitivity is particularly evident in machine learning-based ROMs, given their nonlinear nature. Although robustification algorithms have been proposed for linear ROMs [5], no analogous solutions have been developed for the more powerful machine learning-based ROMs.

To tackle this limitation, we draw insights from the adversarial machine learning literature, which confronts similar challenges [6]. In safety-critical machine learning applications, such as the algorithms employed by autonomous vehicles, resilience to “adversarial perturbations” within the input dataset is paramount. These adversarial perturbations could encompass deliberately crafted perturbations devised by malicious actors or, more relevantly to our context, environmental noise and disturbances that lead to inaccurate model outputs. The field of adversarial machine learning emerged in response to address this crucial challenge. As a result, adversarial training methods have been developed to enhance model resilience, preventing performance degradation in the presence of disturbances and noise. Adversarial training involves augmenting the training data with adversarial examples—deliberate perturbations engineered to deceive the model—forcing the model to learn robust features that generalise well beyond the training dataset and are resilient to disturbance and noise-corrupted data.

Building upon the principles of adversarial machine learning, we leverage adversarial training to improve the robustness of ROMs for MPC applications. This strategy aims to enhance the ROMs' ability to withstand disturbances and noise, thus preventing significant inaccuracies that could impair control performance. For our ROM, we opt for an autoencoder-LSTM architecture, drawing inspiration from the work of [4]. This architecture amalgamates the low-dimensional feature extraction capabilities of autoencoders with the temporal modelling of LSTMs, enabling the ROM to capture both static and dynamic system behaviours effectively.

To showcase the effectiveness of our approach, we utilise the plant-wide system and MPC set-point tracking problem outlined in [7]. We explore five different modelling approaches for MPC. As benchmarks, we employ a first-principles nonlinear dynamical model of the system to solve an NMPC problem and linearize this model to solve an LMPC problem. Additionally, we utilise the SVD-based POD to construct a reduced-order model of the system. Given the system's nonlinearity, POD also yields a nonlinear reduced-order model. To tackle this, we consider one scenario where we disregard the nonlinear term (referred to as "POD" in the results) and another where we interpolate the nonlinear term using the Discrete Empirical Interpolation Method (DEIM) (referred to as "POD-DEIM" in the results). Lastly, we employ our autoencoder-LSTM to compare against the aforementioned approaches. For each of these cases, we solve the equivalent MPC set-point tracking problem 10 times and report the control performance in terms of the average summed normalized integral of time weighted absolute error (ITAE) and the control efficiency in terms of the average MPC solution time.

Before considering the impact of disturbances, we first demonstrate the advantages of employing ROMs for MPC as depicted in Figure 2. These results illustrate the scenario where no disturbances affect the system during MPC. As anticipated, LMPC and NMPC represent the extremes of the performance-efficiency trade-off. Conversely, the POD-DEIM and autoencoder-LSTM ROMs demonstrate the advantages of reduced dimensionality, showcasing commendable control performance without compromising significantly on efficiency. It is noteworthy that the standard POD ROM exhibits poor performance as it does not capture the nonlinear effects of the system nor is it as accurate as the linearized first-principles model due to the limited data used for training.

While these results are encouraging, our primary concern lies in assessing how these ROMs respond to disturbances entering the system during MPC. This aspect is depicted in Figure 3. We repeat the same experiment as Figure 2 but now allow disturbances in the form of feed flowrate and composition fluctuations. As expected, all MPC solutions experience deterioration in control performance. However, the ROMs experience significant degradation compared to LMPC and NMPC, with the autoencoder-LSTM showing the most pronounced decline.

However, after subjecting the autoencoder-LSTM to adversarial training, a remarkable improvement in its performance under disturbance conditions is observed, as depicted in Figure 4. The adversarially trained autoencoder-LSTM now achieves performance comparable to NMPC, highlighting the effectiveness of adversarial machine learning in fortifying ROMs for MPC applications.

To further demonstrate this point, Figure 5 reports the absolute degradation in performance for all the considered ROMs. It is evident that prior to adversarial training, the autoencoder-LSTM exhibited the most substantial degradation in performance. However, after adversarial training (“Autoencoder ARO” in Figure 5), the degradation experienced by the autoencoder-LSTM was only marginally worse than that observed for NMPC, which represents the unattainable best-case scenario of MPC with a perfect model.

In conclusion, adversarially robust reduced order MPC represents a compelling solution to the performance-efficiency trade-off inherent in MPC for large-scale systems. By integrating adversarial machine learning techniques, ROMs can achieve enhanced robustness without sacrificing the performance-efficiency advantages they offer. This work paves a path towards employing powerful machine learning-based ROMs for MPC applications, addressing the robustness issue that previously hindered their application in control contexts.

[1] S. J. Qin and T. A. Badgwell, “An overview of industrial model predictive control technology,” Control Engineering Practice, 2003.

[2] Ma, Z., et al. (2010) “Reduced-order models for control of fluids using the eigensystem realization algorithm”, Theoretical and Computational Fluid Dynamics.

[3] Lorenzetti, J. et al. (2019) “Reduced order model predictive control for setpoint tracking”, 18th European Control Conference.

[4] D. Ha, J. Schmidhuber, “World Models,” arXiv preprint, arXiv: 1803.10122, 2018.

[5] Brunton, S.L., et al. (2013) “Reduced-order unsteady aerodynamic models at low Reynolds Numbers”, Journal of Fluid Mechanics.

[6] T. Bai, et al. “Recent advances in adversarial training for adversarial robustness,” arXiv preprint, arXiv: 2102.01356, 2021.

[7] T. Tosukhowong, et al. “An introduction to a dynamic plant-wide optimization strategy for an integrated plant,” Computers Chemical Engineering, 2004.