(305d) Low-Dimensional Vector Autoregressive Modeling for Dynamics Prediction with Maximum Likelihood Estimation

While industrial internet of things and big data have great potential of improving
predictions and decision-making, challenges are met in analyzing high dimensional
data with complex dynamics and uncertainties [1]. With operational data and
rich sensor measurements, the high dimensional time series usually do not have full-
dimensional dynamics, which calls for reduced-dimensional analytics. For example,
high collinearity usually exists in most routine operation time-series data. Therefore,
it is crucial to develop parsimonious modeling to extract reduced-dimensional
dynamics [2, 3].

In reduced dimensional dynamic data modeling, principal component analysis,
canonical correlation analysis, and partial least squares have been extended to include
dynamics in the reduced dimensional latent space. The work [4] proposed dynamic
PCA (DPCA) to extract the auto-correlations in time series via an augmented data
matrix. To maximize the predictability in dynamic latent variables (DLV), Dong and
Qin proposed dynamic-inner PCA (DiPCA) [5] and dynamic-inner CCA (DiCCA) [6]
algorithms. Furthermore, Qin developed a latent vector autoregressive modeling algorithm
with a CCA objective (LaVAR-CCA) [7]. A latent state space (LaSS-CCA)
model [8] is subsequently developed by extending the LaVAR model.

So far, most of the developed methods are formulated and solved in the least
squares sense, which does not produce uncertainty and covariance estimates for the
modeling errors and estimated parameters. In this work, we propose a probabilistic
model to partition the measurement space into a signal subspace admitting the low
dimensional DLV dynamics and a static noise subspace, which do not need to be
orthogonal to each other. Specifically, we use two expectation-maximization (EM)
steps to estimate the DLV dynamics and the signal subspace. Moreover, we apply a
statistical constraint to characterize the relationship between the estimated signal and
static noise subspaces and use it to estimate the oblique projection. The contributions
in this work are as follows.
1) We develop a probabilistic reduced-dimensional vector regressive (PredVAR)
model with oblique projections. Our study of using VAR to capture latent dynamics
Preprint submitted to AIChE Annual Meeting April 3, 2023
should initiate exploring more dynamic models.
2) We develop an iterative algorithm to update the DLV dynamics and oblique
projection estimations alternately. It uniquely uses a statistic constraint together
with an EM procedure to select the oblique projection given DLV dynamics.
3) We conduct a simulated case study to demonstrate the strength of our approach
compared to two benchmarks, a one-shot algorithm that first identifies the oblique
projection and then estimates the DLV dynamics and a counterpart that focuses on
the orthogonal projection.

References
[1] M. Sznaier, “Control oriented learning in the era of big data,” IEEE Control Syst.
Lett., vol. 5, pp. 1855–1867, 2020.
[2] S. J. Qin, Y. Dong, Q. Zhu, J. Wang, and Q. Liu, “Bridging systems theory and
data science: A unifying review of dynamic latent variable analytics and process
monitoring,” Annu Rev Control, vol. 50, pp. 29–48, 2020.
[3] G. C. Reinsel, R. P. Velu, and K. Chen, Multivariate Reduced-Rank Regression:
Theory, Methods and Applications, vol. 225. Springer Nature, 2023.
[4] W. Ku, R. H. Storer, and C. Georgakis, “Disturbance detection and isolation by
dynamic principal component analysis,” Chemometrics Intell. Lab. Syst., vol. 30,
pp. 179–196, 1995.
[5] Y. Dong and S. J. Qin, “A novel dynamic PCA algorithm for dynamic data
modeling and process monitoring,” J. Process Control, vol. 67, pp. 1–11, 2018.
[6] Y. Dong and S. J. Qin, “Dynamic latent variable analytics for process operations
and control,” Comput. Chem. Eng., vol. 114, pp. 69–80, 2018.
[7] S. J. Qin, “Latent vector autoregressive modeling and feature analysis of high
dimensional and noisy data from dynamic systems,” AIChE J., p. e17703, 2022.
[8] J. Yu and S. J. Qin, “Latent state space modeling of high-dimensional time series
with a canonical correlation objective,” IEEE Control Syst. Lett., vol. 6, pp. 3469–
3474, 2022.