(139e) MPC-Based Adaptive Dual Control

Maintaining a good model of a controlled plant is an important challenge in the process industries, model quality being one determining factor for the performance of a model predictive controller (MPC). Performing experiments to generate data suitable for system identification is not always practical due to factors such as time constraints, the expertise needed, and expensive operational disruption. Model parameters are therefore commonly estimated using data collected during normal operation; however, recorded process data may be insufficiently informative for system identification or it can be difficult to locate the informative portions of a large data set.

Feldbaum (1961) was the first to recognize that an optimal controller for a system with unknown parameters has two conflicting tasks: directing the output toward a reference, and exciting the system for learning purposes so that better control decisions can be made in the future. A dual controller is optimal in the sense that it finds the best trade-off or balance between control and excitation.

Dual control laws can be computed for very simple systems (Åström and Helmersson, 1986), but the problem quickly becomes intractable as the number of unknown parameters increases. Many approximations to dual control laws are available in the literature, many of which rely on persistent excitation. Since persistent excitation may lead to excessive deterioration of performance, a milder form of excitation is desired.

We present and compare three variations of a novel approach to the problem of controlling a system with unknown model parameters while simultaneously exciting the system so that the parameter estimates can be improved and their error variances reduced. Excitation is explicitly included in the cost functions and balanced against standard control objectives. All three approaches use a recursive least-squares (RLS) algorithm (Ljung, 1999) for estimation of unknown model parameters. The most recent parameter estimate is used by the controller to predict future plant output, resulting in a certainty equivalence (CE) control approach.

Even though we consider linear models, the inclusion of the excitation equations in the open-loop optimization problem leads to a nonlinear programming (NLP) problem being solved at every discrete time instant; that is, the balanced excitation feature gives a nonlinear predictive controller for a linear system. We formulate our moving horizon control objective by augmenting a standard quadratic cost function with a function of a matrix containing the parameter estimate error covariances (the excitation function). An example of such a function is the trace of the error covariance matrix, corresponding to the A-optimal measure in optimal input design (Mehra, 1974). We minimize the excitation function over a horizon that may be different from the prediction horizon.

The first approach (Heirung et al., 2012) minimizes a weighted combination of future predicted outputs, control inputs, and the future trace of the error covariance matrix with the length of the excitation horizon set to that of the prediction/control horizon. The use of the error covariance matrix in the cost formulation leads to a large number of nonlinear equality constraints in the open-loop optimization problem solved online.

Our second approach (Heirung et al., 2013a) is based on the information matrix, which is the inverse of the error covariance matrix, instead of the error covariance matrix itself. Working with inverse has the advantage of a simplified formulation of the excitation equations in the NLP. This variation uses a prediction horizon of length 1. The resulting formulation leads to an optimization problem that easier to solve and a problem formulation that is easier to analyze. However, it is more difficult to formulate a suitable objective function in the information matrix than in the covariance matrix.

The third variation of the method (Heirung et al., 2013b) involves an exact reformulation of the expected value of the output error one time step ahead. The resulting expression is a function of the covariance matrix one time step ahead, and the input and output signals. The disadvantage that comes with using the covariance matrix in the objective function is the same as in the first approach, but with an excitation horizon of length one the added complexity is less significant.

All three algorithms can be implemented with minor modifications to an existing MPC framework. We present numerical results for each of the three approaches and compare some advantages and disadvantages. The results show that parameter identification can be greatly enhanced with minimal effect on the system output by allowing the controllers to slightly increase the system excitation in a smart way, and that this can be achieved without requiring persistent excitation. We also discuss some ongoing work and future directions.

References

K. J. Åström and A. Helmersson. Dual control of an integrator with unknown gain. Computers and Mathematics with Applications, 12(6):653--662, 1986.
A. A. Feldbaum. Dual-control theory. I. Automation and Remote Control, 21(9):874--880, 1961.
T. A. N. Heirung, B. E. Ydstie, and B. Foss. Towards Dual MPC. In 4th IFAC Nonlinear Model Predictive Control Conference, pages 502--507, Noordwijkerhout, the Netherlands, 2012.
T. A. N. Heirung, B. E. Ydstie, and B. Foss. An adaptive model predictive dual controller. In 11th IFAC International Workshop on Adaptation and Learning in Control and Signal Processing (ALPCOSP), Caen, France, 2013a.
T. A. N. Heirung, B. E. Ydstie, and B. Foss. An MPC approach to dual control. In 10th International Symposium on Dynamics and Control of Process Systems (DYCOPS) (Submitted), Mumbai, India, 2013b.
L. Ljung. System Identification. Theory for the User. Prentice-Hall, second edition, 1999.
R. K. Mehra. Optimal input signals for parameter estimation in dynamic systems -- Survey and new results. IEEE Transactions on Automatic Control, 19(6):753--768, 1974