(578a) Fast Training Physics Informed Machine Learning Models Using Gaussian Processes

Machine learning has become a powerful tool for solving complex problems across various domains. However, traditional data-driven machine learning approaches often treat the physical systems as black boxes, ignoring the underlying governing laws and principles. This can lead to models that may not generalize well or provide physically inconsistent predictions, especially when extrapolating beyond the training data distribution. Physics-informed machine learning (PIML) is an emerging field that aims to bridge the gap between machine learning and physical modeling by incorporating known physical laws and principles into the learning process. PIML methods leverage the synergy between data and physics-based models, resulting in more accurate, interpretable, and generalizable solutions[1].

One such technique under the PIML umbrella is Data-Driven Parameterization. Physical models often contain parameters that need to be calibrated or tuned based on experimental data. Machine learning can be used to learn the functional forms of these parameters from data, improving the accuracy of the physics-based models. The problem that arises is that estimating multiple, interrelated parameter functions within a single system (typically done through Physics-informed neural networks or PINNs) is computationally expensive, requires significant data due to the neural network architecture, does not allow for insertion of prior information about the parameters, and requires additional processing for non-native uncertainty calculations[1,2]. This work focuses on the development of a new technique using Gaussian processes built from Bayesian Basis Functions to be able to train these PIML models in an adaptable and computationally resourceful way.

Building Gaussian processes using a Bayesian Basis Function approach has training time of O(n), faster than traditional neural network training. However, the main concern in this approach is choosing and calculating the basis functions themselves. This methodology uses FoKL-GPs[3] (Forward variable selection Karhunen-Loeve decomposed Gaussian Processes), meaning the method uses a forward variable selection approach with basis functions created from a Karhunen-Loeve decomposition of a smoothing spline type GP kernel. Taking advantage of the ordered nature of basis functions, multiple Bayesian Basis Function models are created algorithmically in order of increasing complexity and a BIC (Bayesian Information Criteria) is used to identify the best model. These Gaussian processes are inserted into a physics-informed model and all FoKL-GPs are built and evaluated simultaneously to assess the interconnected nature of all the parameters of consideration.

To show an implementation of this methodology, this work takes on the case study of a CSTR with a single, reversible synthesis reaction and uses this methodology to predict both the forward and reverse reaction rate coefficient functional forms from only process simulated data. Additional use cases will also be discussed to continue to explore the applications of this PIML methodology and derive more fine-tuned physics informed models for chemical process systems.

References

[1] Karniadakis, G. E.; Kevrekidis, I. G.; Lu, L.; Perdikaris, P.; Wang, S.; Yang, L. Physics-Informed Machine Learning. Nat Rev Phys 2021, 3 (6), 422–440. https://doi.org/10.1038/s42254-021-00314-5.

[2] Raissi, M.; Perdikaris, P.; Karniadakis, G. E. Physics-Informed Neural Networks: A Deep Learning Framework for Solving Forward and Inverse Problems Involving Nonlinear Partial Differential Equations. Journal of Computational Physics 2019, 378, 686–707. https://doi.org/10.1016/j.jcp.2018.10.045.

[3] Hayes, K.; Fouts, M. W.; Baheri, A.; Mebane, D. S. Forward Variable Selection Enables Fast and Accurate Dynamic System Identification with Karhunen-Loeve Decomposed Gaussian Processes. arXiv February 23, 2023. http://arxiv.org/abs/2205.13676 (accessed 2024-03-31).