(433d) Adversarially Robust Real-Time Optimization and Control (ARRTOC)

A practical challenge in process control is the handling of disturbances, noise and implementation errors which enter the system while still obtaining good set-point tracking. Moreover, all of this must be achieved within the tight time-constraints placed on controllers by the process dynamics. This can be handled by introducing additional complexity at the control layers, but this may not be practically implementable. In this work, we propose a novel algorithm, which favours practicality over complexity, by extending the role of Real-Time Optimization (RTO) to implicitly handle disturbance rejection and noise attenuation for the control layers.

RTO is a critical task for the efficient operation of a process system. It allows optimal set-points to be found for the lower-level control layers in the process operation hierarchy while at the same time satisfying operating constraints. All of this must be handled in real-time due to changes in things such as equipment and costs amongst many other factors. However, compared to the lower-level control layers, there is still typically significant time between executions of the RTO operation. Once the set-points are calculated, they are sent to the lower-level control layers. Here the goals of the controllers can be broadly classified into three main objectives. Firstly, the controller must track the set-point, usually optimally. Beyond this, the controller must be able to handle disturbances which enter the system by having adequate disturbance rejection. Finally, imperfect sensor measurements mean that feedback controllers must also attenuate any noise entering the system. These three objectives are all in conflict with each other - A fundamental insight from linear control theory. For nonlinear systems, these conflicts are further exacerbated. All of these objectives must be adequately satisfied while at the same time ensuring the controller operates at a high enough frequency compared to the fastest dynamics acting on the system i.e. tight time-constraints are placed on the controller. Consequently, when designing controllers, a trade-off is usually sought in order to satisfy this constraint. For example, for MPC, while it may be possible to design a fully nonlinear MPC solution which is robust to certain disturbance and noise characteristics and other forms of uncertainty, this may not be practically implementable.

Alternatively, to handle this challenge, we propose a means of exploiting the interaction between the upper-level RTO layer and the lower-level control layer. We argue that RTO need not only be concerned with finding optimal set-points for the controllers. Instead, it can use its better time resources to take on more responsibilities from the controllers, avoiding the need to rely on any complex, time-consuming algorithms at the resource-starved lower layers. In particular, we construct an RTO algorithm which implicitly handles disturbance rejection and noise attenuation for the control layers. This is done by finding optimal and adversarially robust set-points for the controllers. These are set-points which are, by design, robust to adversaries which in the context of RTO and control are disturbances and noisy measurements as well as other sources of implementation errors. This is chiefly achieved using a modified version of the Adversarially Robust Optimization algorithm [1,2,3].

In essence, our algorithm, which we refer to as the Adversarially Robust Real-Time Optimization and Control (ARRTOC) algorithm, is concerned with finding not only optimal set points but “operable” set-points. These are targets which allow the controller to operate at a set-point safely and reliably in the face of disturbances and noise. This goal of finding operable set-points is not new in the RTO literature. Currently, this is achieved via some form of pre or post analysis of the optimization solution space [4]. This, as attested to by several authors, may be computationally-intensive, dependent on unverified assumptions, suffer from the curse of dimensionality and thus may not be practical in some cases [5, 6]. Instead, our proposed ARRTOC algorithm handles the dual problem of optimality and operability seamlessly by finding regions of state-space which are inherently adversarially robust as part of an online optimization solution. The algorithm proceeds via two main steps. Firstly, given some initial point, the algorithm performs a neighbourhood exploration which searches for the directions in the neighbourhood where disturbances have the biggest impact on the objective. Thereafter, given this information, the algorithm selects a direction and step size within the neighbourhood which moves away from these worst-case disturbance directions. This repeats until a point is found where no feasible directions exist implying that an adversarially robust optimum has been found [1, 2].

The performance of the algorithm is best depicted via a visual illustration. Consider the example of controlling a CSTR with cooling water. The RTO objective is to maximise the operating profit. This is shown as a contour plot overlaying the concentration-temperature state-space in Fig 1 with the adversarially robust optimum (red cross), found using the ARRTOC algorithm, and the global optimum (blue cross) depicted. One should notice that the global optimum is on a "narrow" peak of the objective function. This alludes to the idea that this point is sensitive to perturbations. In the context of RTO and control, this has a direct analogy to the operability of this set-point.

The controller, in this case a simple PI controller, attempts to steer the system to the proposed set-point while rejecting disturbances and attenuating noise. In Fig 2, we show the behaviour of the controller near the set-point in state-space for two scenarios. In scenario 1 (blue dashed line) the controller tracks the global optimum while in scenario 2 (red dashed line) the controller tracks the adversarially robust optimum. Both scenarios are subject to the exact same disturbance and noise characteristics.

The dynamic performance of these two scenarios is shown in Fig 3 which shows the real-time, average, and standard deviation of the profit. For scenario 1, it is clear to see that, as a result of the disturbances acting on the system, the real-time profit (blue dashed line) rises and falls significantly, and at times almost drops to zero. The mean profit over time (solid blue line) is much lower than the expected global optimum (black dashed-starred line) suggesting that the system struggles to operate at this point. The standard deviation of the real-time profit (blue shaded region) is also quite significant. Crucially, all of this shows that, by design, the global optimum is not robust to adversarial perturbations. On the other hand, for scenario 2, one can clearly see the real-time profit (red dashed line) is far more stable over time as compared to scenario 1. In this case, as the set-point is robust by design, the mean profit (red solid line) is much closer to the robust optimum profit (green dashed-starred line) and the standard deviation (red shaded region) is much smaller compared to scenario 1. In fact, due to the inherent robustness associated with this point, the mean profit for scenario 2 is larger than the mean profit of scenario 1 by almost 25%, showing that the robust optimum has better operability properties. That is, perversely, by operating at the adversarially robust optimum, the system is more profitable, on average, than if the system operated at the global optimum.

Fundamentally, we have taken the challenge of robustifying our controller and achieved it at the RTO level instead. This allows us to use more computational time resources at the RTO level (where they are abundant) and avoid this at the control level (where they are lacking). In fact, in our work, we demonstrate via case studies that the integration of the RTO and control layers as we have suggested allows practitioners to more easily balance the robustness of both layers in a unified and holistic manner. Specifically, the controllers can be designed to an adequate level of robustness (which fits within their computational time budget) and then the remaining robustness, if required, can be handled at the RTO level by the ARRTOC algorithm.

To conclude, our proposed ARRTOC algorithm demonstrates that the RTO and control layers can be integrated in an efficient, scalable, and practical manner which satisfies both the RTO and control objectives mentioned hitherto. It favours practicality over complexity and can be extended to cases where the underlying RTO objective is not known through the use of Gaussian Process Regression [1].

[1] Bogunovic, I. et al. (2018) “Adversarially Robust Optimization with Gaussian Processes,” Advances in Neural Information Processing Systems 31 (NeurIPS 2018).

[2] Bertsimas, D. et al. (2010) “Robust optimization for unconstrained simulation-based problems,” Operations Research, 58(1), pp. 161–178.

[3] Paulson, J.A. et al. (2022) “Adversarially robust bayesian optimization for efficient auto‐tuning of generic control structures under uncertainty,” AIChE Journal, 68(6).

[4] Gazzaneo, V. et al. (2019) “Process operability algorithms: Past, present, and future developments,” Industrial & Engineering Chemistry Research, 59(6), pp. 2457–2470.

[5] Lima, F.V. et al. (2009) “Similarities and differences between the concepts of operability and flexibility: The steady-state case,” AIChE Journal.

[6] Dinh, S. and Lima, F.V. (2023) “Dynamic operability analysis for process design and control of Modular Natural Gas Utilization Systems,” Industrial & Engineering Chemistry Research, 62(5), pp. 2052–2066.