(284d) Set-Membership Nonlinear Regression Approach to Parameter Estimation

Most commonly, regression problems in nonlinear parameter
estimation are posed as optimization problems, where selected
parameter values are tuned so that the model predictions match the
available measurements as close as possible, for instance in the
maximum-likelihood sense. Incorporation of measurement uncertainty
into the regression exercise has spawned several different schools of
thought. Statistical approaches can be broadly classified as
Frequentist or Bayesian. The former seek to
determine confidence regions around the regressed parameter values,
considered as the `true' parameter values [1].
They are capable of dealing with large-scale models, but confidence
analysis based on model linearization can be misleading, especially
for highly nonlinear models or when the available measurements are
scarce. Bayesian approaches meanwhile construct a probability
distribution of the parameters based on the available observations,
from which credibility regions can be inferred [2].
These approaches are appealing, but they rely on a so-called prior
for the parameter estimates, and existing algorithms such as
Markov-chain Monte-Carlo (MCMC) can be time consuming for problems
having upwards of 5-10 parameters. An alternative to these
statistical approaches is set-membership estimation, which
seeks to determine the set of all possible parameter values that are
consistent with bounded-error measurements [3].
Such problems have been shown to be tractable using complete-search
methods with ca. 10 parameters, e.g., based on interval analysis or
higher-order inclusion techniques [4,5].
Issues may arise however, as model mismatch or outlying measurements
can lead to situations where the resulting parameter set is empty.

We propose a new approach to nonlinear
regression, called set-membership nonlinear regression
(SMNLR). Given a residual function

,
where

stands for the (unknown) model parameters and

the measurement errors, the problem addressed involves characterizing
the set P given by

(1)

where

is a (non-empty) prior set for the model parameters; and

a given measurement-error set (e.g., bounds on the measurement
errors, level-sets of a joint probability distribution, etc).
We study the properties of SMNLR, with a view to relating problem (1)
with existing parameter estimation approaches. In the context of
bounded-error measurement estimation, SMNLR presents the advantage
that the set P is always non-empty, regardless of the model
mismatch or measurement error. In the context of statistical
inference, P may be interpreted as parameter estimability
sets for a given confidence level-set E on the measurement
errors [6]. Moreover, we derive
conditions under which the SMNLR regions are asymptotically
equivalent to the classical Wald confidence regions.

Notwithstanding, SMNLR is a particular arduous problem as it
involves finding all global minimizers to all
parameter estimation problems with

.
We describe how this problem can be tackled using complete-search
methods (branch-and-prune), and we discuss several simplifications
for reducing the computational burden. Comparisons are made with the
classical set-membership estimation approach as well as classical
confidence analysis methods for selected parameter estimation
problems.

Bibliography

[1] S.P. Asprey, S. Macchietto, 2000. Statistical tools for optimal dynamic model building. Computers & Chemical Engineering 24:1261-1267.

[2] D. Alspach, H. Sorenson, 1972. Nonlinear Bayesian estimation using Gaussian sum approximations. IEEE Transactions on Automatic Control 17(4):439-448.

[3] E. Walter, 1990. Special Issue on Parameter Identification with Error Bounds. Mathematics & Computers in Simulation 32:447-607.

[4] L. Jaulin and E. Walter, 1993. Set inversion via interval analysis for nonlinear bounded-error estimation. Automatica 29(4):1053-1064.

[5] B. Chachuat, B. Houska, R. Paulen R, N.D. Peric, J. Rajyaguru, M.E. Villanueva, 2015. Set-theoretic approaches in analysis, estimation and control of nonlinear systems. IFAC-PapersOnLine 48(8):981-995.

[6] K.A. McLean and K.B. McAuley, 2012. Mathematical modelling of chemical processes - obtaining the best model predictions and parameter estimates using identifiability and estimability procedures. The Canadian Journal of Chemical Engineering 90(2):351-366.