Bayesian Identification of Nonlinear, Sparse, Dynamic Models

Samuel Adeyemo, Debangsu Bhattacharyya

Department of Chemical and Biomedical Engineering, West Virginia University, Morgantown, WV 26506, USA

Data-driven models are useful for both derivative-free and derivative-based optimizations, online control of dynamic processes, feasibility analysis, parameter estimation and sensitivity analysis⁴. Popular among several data-driven approaches for modeling of dynamic processes are the artificial neural networks (ANNs) and polynomial models. Despite the great success in the performance of ANNs, one significant drawback of the ANNs is the interpretability of the resulting model, and possibility of complicated and large models. Recently, different approaches have been developed to address the challenge of the interpretability of developed data-driven models, these including the Sparse Identification of Nonlinear Dynamics (SINDy)^5,6 and Algebraic Learning Via Elastic Net (ALVEN)⁷. However, there is scarcity of work on approaches that will both explicitly account for the presence of noise in the data and presence of correlated data. If the noise is not necessarily Gaussian and zero-mean, then many existing approaches can lead to biased estimates of model parameters.

A Bayesian inference algorithm is developed for optimal learning of basis functions and maximum a posteriori estimate of the parameter vector. An expectation maximization (EM) algorithm is developed where current user belief is cast in the form of informed priors. Using the Bayes law, posterior densities are estimated based on likelihood densities and informed priors. Hyperparameters that are used to parameterize the error covariance matrix are also optimally estimated once the maximum likelihood estimate of the underlying noise free output is obtained. An efficient branch and bound (B&B) algorithm that employs bi-directional pruning strategies is developed for optimal selection of basis functions whose elements are linear and nonlinear transformations of the measured input and output variables. An information-theoretic criterion is used as the optimization objective for model selection. To penalize uncertainties in parameter estimate, the objective function employs a covariance complexity penalty term, that yields a measure of the information loss due to the assumption that the parameter estimates are independent.

The algorithm is thoroughly tested for a highly nonlinear chemical solvent-based post-combustion CO₂ capture system¹ for which large amount of pilot plant data are available^1,2. The process involves complex chemical reactions kinetics, highly nonlinear vapor-liquid equilibrium, and complicated mass transfer mechanisms due to the presence of ionic species³ making the development of first principles model quite involved. Sparse identification of the models is tested by corrupting the pilot plant data with noise of known and unknown characteristics. Performance of the developed algorithm is compared with the leading algorithms. It is found that the algorithm yields considerably superior results especially when the data are corrupted with non-Gaussian and correlated noise.

References

1. Pashaei, H., Ghaemi, A., Nasiri, M. & Karami, B. Experimental Modeling and Optimization of CO2 Absorption into Piperazine Solutions Using RSM-CCD Methodology. ACS Omega 5, 8432–8448 (2020).
2. Chinen, A. S., Morgan, J. C., Omell, B., Bhattacharyya, D. & Miller, D. C. Dynamic Data Reconciliation and Validation of a Dynamic Model for Solvent-Based CO 2 Capture Using Pilot-Plant Data. Ind. Eng. Chem. Res. 58, 1978–1993 (2019).
3. Morgan, J. C. et al. Thermodynamic modeling and uncertainty quantification of CO2-loaded aqueous MEA solutions. Chem. Eng. Sci. 168, 309–324 (2017).
4. McBride, K. & Sundmacher, K. Overview of Surrogate Modeling in Chemical Process Engineering. Chemie-Ingenieur-Technik 91, 228–239 (2019).
5. Brunton, S. L., Proctor, J. L., Kutz, J. N. & Bialek, W. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proc. Natl. Acad. Sci. U. S. A. 113, 3932–3937 (2016).
6. Mangan, N. M., Kutz, J. N., Brunton, S. L. & Proctor, J. L. Model selection for dynamical systems via sparse regression and information criteria. Proc. R. Soc. A Math. Phys. Eng. Sci. 473, (2017).
7. Sun, W. & Braatz, R. D. ALVEN : Algebraic learning via elastic net for static and dynamic nonlinear model identification. Comput. Chem. Eng. 143, 107103 (2020).