(149v) Physics-Informed Neural Networks (PINNs) for Modeling Dynamic Processes Based on Limited Physical Knowledge and Data

Dynamic operation of chemical and biotechnological processes is essential for efficient and sustainable production of many important products. Hybrid models have become a crucial tool for numerous tasks related to process operation, as often both extensive amounts of measurement data and a complete mechanistic description are lacking, rendering both entirely data-driven (black-box) and purely physics-based (white-box) modeling impractical. Various types of hybrid models have been proposed over the years, such as the sequential, parallel, and structured approaches [1, 2, 3]. Physics-informed neural networks (PINNs) can be seen as a conceptually different type of hybrid model, where a deep network acts as the sole prediction model but is informed by governing physical laws during training through additional residual terms in the loss function [4]. The origins of physics-based regularization of neural networks date back to the works of Lagaris et al. [5], but recently Karniadakis and co-workers [6, 7] showed that PINNs can predict the solutions to various forward and inverse problems defined by (partial) differential equations with high accuracy. Even more recently, Antonello et al. [8] have extended the PINN concept by adding control inputs and initial conditions as inputs to the network and thus have made the approach suitable for control applications. Antonello et al. [8], however, assume complete mechanistic knowledge of the processes and do not utilize experimental data for training the PINN.

In the present work, we investigate PINNs for modeling dynamic processes when the governing physico-chemical laws are only partially known, and availability of observational data is limited. By studying continuously stirred tank reactor (CSTR) models from the literature [9, 10], we follow an in-silico approach, i.e., we use known models to generate synthetic observational data for PINN training. Being able to effectively control the amount and diversity of training data allows us to run extensive numerical experiments on fitting and generalization capabilities of both the PINN models and purely data-driven benchmark models, i.e., deep feed-forward multilayer perceptrons. To facilitate application in control, all models take controls and initial conditions as inputs. By deliberately hiding parts of the mechanistic ground truth model from the computation of the residual loss term, we simulate the degree of mechanistic knowledge available for model development. To assess the extrapolation capabilities, we test the PINN model with input data not seen during training. We show that PINN models predict and extrapolate better than purely data-driven benchmarks when the available data is scarce. Moreover, we find that PINN models can infer hidden states, i.e., states for which neither observational data nor constitutive equations are available, with an acceptable accuracy. With their capability of modeling processes when relatively little experimental data and only partially known mechanistic descriptions are available, PINNs constitute a promising avenue for hybrid modeling in chemical engineering that warrants further investigation.

References:

[1] Psichogios, D. C. and Ungar, L. H. (1992). A hybrid neural network-first principles approach to process modeling. AIChE Journal, 38(10):1499–1511.

[2] Su, H.-T., Bhat, N., Minderman, P., and McAvoy, T. (1992). Integrating neural networks with first principles models for dynamic modeling. IFAC Proceedings Volumes, 25(5):327–332.

[3] Kahrs, O. and Marquardt, W. (2007). The validity domain of hybrid models and its application in process optimization. Chemical Engineering and Processing: Process Intensification, 46(11):1054–1066.

[4] Nabian, M. A. and Meidani, H. (2020). Physics-driven regularization of deep neural networks for enhanced engineering design and analysis. Journal of Computing and Information Science in Engineering, 20(1):011006.

[5] Lagaris, I. E., Likas, A., & Fotiadis, D. I. (1998). Artificial neural networks for solving ordinary and partial differential equations. IEEE Transactions on Neural Networks, 9(5), 987-1000.

[6] Raissi, M., Perdikaris, P., and Karniadakis, G. (2019). Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686–707.

[7] Raissi, M., Yazdani, A., & Karniadakis, G. E. (2018). Hidden fluid mechanics: A Navier-Stokes informed deep learning framework for assimilating flow visualization data. arXiv preprint arXiv:1808.04327.

[8] Antonelo, E. A., Camponogara, E., Seman, L. O., de Souza, E. R., Jordanou, J. P., & Hubner, J. F. (2021). Physics-Informed Neural Nets for Control of Dynamical Systems. arXiv preprint arXiv:2104.02556.

[9] Flores-Tlacuahuac, A., & Grossmann, I. E. (2006). Simultaneous cyclic scheduling and control of a multiproduct CSTR. Industrial & Engineering Chemistry Research, 45(20), 6698-6712.

[10] Van de Vusse, J. G. (1964). Plug-flow type reactor versus tank reactor. Chemical Engineering Science, 19(12), 994-996.