(377e) Transfer Learning-Based Modeling and Predictive Control of Nonlinear Process Network

Recurrent neural networks (RNNs) have been widely used in modeling nonlinear dynamic systems using sequential or time-series data [1]. However, in practice, it can be difficult and expensive to collect sufficient data for some real-world systems to build RNN models. Transfer learning that transfers knowledge from a source process with a large amount of data to a target process with very limited data provides an effective tool to overcome data scarcity in modeling chemical processes [2] and makes knowledge easily transferable to new processes [3]. Despite the increasing popularity and application of transfer learning in many fields of process systems engineering [4], the application of transfer learning in process modeling is still in its infancy. Specifically, the applicability and benefits of transfer learning for modeling nonlinear processes through the transfer of knowledge from a related task remain an open question. Furthermore, the traditional transfer learning method focuses on the modeling of a single process, and is challenging to be generalized to a process network with multistage processes, since it requires a similar process network as the source domain, which is difficult to obtain in practice. Incorporating domain knowledge into transfer learning is a potential solution, and the research in this area is still in its infancy.

In this work, a framework of transfer learning for modeling a target nonlinear process is developed by taking advantage of the knowledge learned in a different but similar source process. Specifically, transfer learning uses a pre-trained model developed based on a source domain as the starting point, and adapts the model to a target process with similar configurations. The generalization error for TL-based RNN (TL-RNN) is first derived to demonstrate the generalization capability on the target process. The theoretical error bound that depends on model capacity and the discrepancy between source and target domains is then utilized to guide the development of pre-trained models for improved model transferability. Subsequently, the TL-RNN model is utilized as the prediction model in model predictive controller for the target process. A simulation study of chemical reactors via Aspen Plus Dynamics is used to demonstrate the benefits of transfer learning Furthermore, to generalize the transfer learning method to a nonlinear process network, we integrate transfer learning with physics-informed machine learning methods to improve the overall prediction performance of the entire process network by incorporating domain knowledge such as process structural knowledge and the first-principles model into the training process. An example of multistage processes is used to illustrate the effectiveness of the proposed transfer learning strategy for chemical process networks.

References:

[1] Ren, Y. M., Alhajeri, M. S., Luo, J., Chen, S., Abdullah, F., Wu, Z., & Christofides, P. D. (2022). A tutorial review of neural network modeling approaches for model predictive control. Computers & Chemical Engineering, 107956.

[2] Jiang, J., Shu, Y., Wang, J., & Long, M. (2022). Transferability in deep learning: A survey. arXiv preprint arXiv:2201.05867.

[3] Neyshabur, B., Sedghi, H., & Zhang, C. (2020). What is being transferred in transfer learning?. Advances in neural information processing systems, 33, 512-523.

[4] Zhuang, F., Qi, Z., Duan, K., Xi, D., Zhu, Y., Zhu, H., Xiong, H. & He, Q. (2020). A comprehensive survey on transfer learning. Proceedings of the IEEE, 109(1), 43-76.

[5] Zhang, Y., Liu, T., Long, M., & Jordan, M. (2019, May). Bridging theory and algorithm for domain adaptation. In International conference on machine learning (pp. 7404-7413). PMLR.

[6] Karniadakis, G. E., Kevrekidis, I. G., Lu, L., Perdikaris, P., Wang, S., & Yang, L. (2021). Physics-informed machine learning. Nature Reviews Physics, 3(6), 422-440.