(199e) Identification of Mass-Constrained Bayesian Sparse Models from Noisy Data

Over the last few decades, artificial intelligence (AI) has attained great success in the development of data-driven models for chemical systems. However, major limitations of many such models including those represented by artificial neural networks (ANNs) are the lack of interpretability, possibility of unsatisfactory and unexpected predictions when the model is extrapolated, and large data requirement for training especially when the model is not necessarily sparse. To address the challenge of sparsity and interpretability, algorithms such as the Sparse Identification of Nonlinear Dynamics (SINDy)^1,2 and Automatic Learning of Algebraic Models (ALAMO)³ have been proposed. One limitation of many existing sparse identification approaches especially for dynamic systems is that they may lead to biased estimation of basis functions and model parameters when the training dataset is corrupted with noise as these identification approaches do not explicitly account for presence of noise in the data. On the other hand, incorporation of physical laws into training of neural networks seeks to bring both interpretability and enhancement of extrapolation capabilities into resulting physics-informed machine learning models^4–6. However, most of these methods require a significant knowledge of the system dynamics to accurately specify such constraints, and the resulting algorithms and models are not usually generalizable. In particular, the Bayesian Physics-Informed Neural Network (B-PINN)⁷ can quantify model uncertainties while imposing physics constraints but requires accurate knowledge of the fidelity of the measurement network, but these information are often not available for industrial systems. One critical limitation of these identified models including B-PINN is that they do not necessarily satisfy mass constraints, which is essential for a satisfactory model of species (e.g., species concentration, species flowrate, etc.) for chemical processes. In fact, for many such identification techniques, satisfying mass constraints exactly is difficult. Often, physics-constraints are imposed as l₁ or other forms of penalty in the objective function while identifying the model, but these formulations do not necessarily exactly satisfy mass balances, nor the resulting model is guaranteed to satisfy mass balances for arbitrary changes in the input variables.

In this work, we will present the development of an algorithm for learning of robust, interpretable, and sparse models for dynamic chemical systems while guaranteeing that the models always satisfy mass balance constraints. Large number of candidate basis functions is available for model building. An expectation maximization (EM) algorithm is developed for Bayesian inference of the model parameter. The proposed approach explicitly takes noise into account while being much less computationally involved than typical fully Bayesian methods. Parsimonious model selection is ensured by considering an information-theoretic criterion that explicitly penalizes uncertainty in parameter estimates. A branch and bound algorithm with effective pruning strategies is developed for selecting the optimal set of basis functions. For satisfying mass balance during the forward and inverse problems, two algorithms are developed. In one of the algorithms, a reconciliation step is incorporated into the EM algorithm enforcing mass balance requirements. In the other algorithm, a set of equality constraints are imposed on the parameter space for exactly satisfying the mass constraints.

The proposed algorithm is evaluated for a pH neutralization process, the nonisothermal Van de Vusse reactor system, and solvent-based post-combustion CO₂ capture system. Model performances are tested by corrupting the data with varying characterization of noise generation process. It was found that the identified models exactly satisfy mass constraints even when they have been trained using data with bias and high noise to signal ratio. Sparsity, accuracy for the training and test data, model interpretability, and computational expense for training for our proposed algorithm are compared with that for some of the leading algorithms by simulating a number of scenarios.

References

1. Brunton, S. L., Proctor, J. L., Kutz, J. N. & Bialek, W. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proc. Natl. Acad. Sci. U. S. A. 113, 3932–3937 (2016).
2. Mangan, N. M., Kutz, J. N., Brunton, S. L. & Proctor, J. L. Model selection for dynamical systems via sparse regression and information criteria. Proc. R. Soc. A Math. Phys. Eng. Sci. 473, (2017).
3. Cozad, A., Sahinidis, N. V. & Miller, D. C. A combined first-principles and data-driven approach to model building. Comput. Chem. Eng. 73, 116–127 (2015).
4. Karniadakis, G. E. et al. Physics-informed machine learning. Nat. Rev. Phys. 3, 422–440 (2021).
5. Sansana, J. et al. Recent trends on hybrid modeling for Industry 4.0. Comput. Chem. Eng. 151, 107365 (2021).
6. Bradley, W. et al. Perspectives on the integration between first-principles and data-driven modeling. Comput. Chem. Eng. 166, 107898 (2022).
7. Yang, L., Meng, X. & Karniadakis, G. E. B-PINNs: Bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data. J. Comput. Phys. 425, 1–32 (2021).