(372w) Design of Experiments Based on Precise Data-Quality Assessment Using Finite-Sample Data

Design of experiments (DoE) for identification [1] enables us to efficiently collect the data for modeling dynamic processes. It intends to maximize the data quality by optimizing the experimental conditions. The objective function representing the data quality is often a scalar measure of the Fisher information matrix F, such as the determinant in D-optimal design or the trace in A-optimal design [1, 2].

However, the objective function based on F does not precisely quantify the quality of finite-sample data. F corresponds to the inverse of the covariance matrix of the model parameters derived from asymptotic theory, where the number of data samples is assumed to be infinite [3]. Hence, the gap between the actual situation and the situation considered in the asymptotic theory expands as the number of available data samples decreases. Therefore, the objective function should be a data-quality index that exactly quantifies the quality of finite-sample data.

Recently, the sign-perturbed-sums (SPS) method was developed [4]. It provides the exact confidence region of the system parameters from the finite-sample input-output data using a mild assumption on the noise innovation, namely that, the noise innovation has a symmetric distribution. Kolumban et al. [5] proposed DoE based on SPS, where they minimized the expected volume of the confidence region. However, the targeted system of the proposed method is restricted to single-input, single-output, finite-impulse-response systems.

This research generalizes the DoE based on the SPS method so that it can be used for systems as complex as multivariate autoregressive exogenous input (ARX) systems. The proposed optimization problem is defined by Eqs. (1) and (2). The evaluation of the objective function requires an outcome of a data-acquisition experiment since the SPS algorithm for autoregressive systems depends on a realization of the noise innovation [4]. Hence, the proposed design problem is solved using Bayesian optimization, which is suitable when the objective function is expensive to evaluate [6, 7].

The proposed method was compared with the existing D-optimal design in two case studies: a simulation case study using a 2-input, 3-output first-order ARX system and an experiment case study using the three-tank system shown in Fig 1. Open-loop experiments for data acquisition were performed, and the cutoff frequencies of the bandpass filter for the input signals were optimized using the proposed and existing methods. Ten iterations were performed using Bayesian optimization. The first two iterations randomly determined the optimized variables. The results of the simulation and experiment case studies are shown in, respectively, Figs. 2 and 3. It is shown that the proposed method provides a better model in terms of the parameter error and control performance of the model predictive control (MPC) system than the conventional method. This implies that using a precise performance index of the quality of finite-sample data improves the performance of DoE.

References

[1] G. C. Goodwin, Dynamic system identification : experiment design and data analysis, vol. 136. Academic Press, 1977.

[2] Y. A. W. Shardt, Statistics for Chemical and Process Engineers-A Modern Approach. Springer Cham, 2022.

[3] L. Ljung, System Identification: Theory for the User, 2nd ed. Pearson, 1998.

[4] B. C. Csáji, M. C. Campi, and E. Weyer, “Sign-perturbed Sums (SPS): A Method for Constructing Exact Finite-sample Confidence Regions for General Linear Systems,” Proceedings of the IEEE Conference on Decision and Control, pp. 7321–7326, 2012.

[5] S. Kolumban and B. C. Csáji, “Towards D-Optimal Input Design for Finite-Sample System Identification,” 2018, pp. 215–220.

[6] B. Shahriari, K. Swersky, Z. Wang, R. P. Adams, and N. de Freitas, “Taking the Human Out of the Loop: A Review of Bayesian Optimization,” Proceedings of the IEEE, vol. 104, no. 1, pp. 148–175, 2016.

[7] S. Greenhill, S. Rana, S. Gupta, P. Vellanki, and S. Venkatesh, “Bayesian Optimization for Adaptive Experimental Design: A Review,” IEEE Access, vol. 8, pp. 13937–13948, 2020.