(712c) Physics-Informed Neural Networks with Hard Linear Equality Constraints

High-fidelity physical models are typically restricted by the computational burden from their underlying differential and algebraic equations, though they can serve as digital representations of real-world systems. This impedes their use in applications where it is essential to simulate a system repeatedly in a timely manner. To address this, data-driven methods have sought to substitute a high-fidelity physical model with a surrogate model that can be evaluated more efficiently [1-3]. This approach provides a more practical means of inferring a system’s responses under a great variety of conditions.

Neural networks have received substantial attention as surrogate models recently, which facilitate their deployment in time-sensitive and large-scale applications [4-8]. One of the fundamental problems is that neural networks are black-box models whose parameters are not physics-informed. To mitigate this drawback, the physics-informed neural network (PINN) was developed to leverage physical constraints in the loss function, which aims to minimize the prediction errors and violation of physics simultaneously at the cost of each other [9, 10]. However, the PINN method only offers “soft constraints” and cannot guarantee exact constraint satisfaction, which is still unsuitable for certain applications. For instance, PINN cannot strictly enforce conservation laws when trained to represent a chemical unit. In practical scenarios, it is common to model distinct units for the reconstruction of a large system [11]. Although the impact of violating constraints might be negligible in an isolated unit, the errors of intermediate variables can propagate throughout the entire system and be magnified over time [4]. Therefore, the violation of mass balance becomes unacceptable when multiple surrogate models are interconnected. This is detrimental to their use in high-stakes decision-making problems, where even minor constraint violations could potentially lead to catastrophic losses.

This work develops a novel PINN architecture with two non-trainable layers that embed hard linear equality constraints rather than soft constraints. The two layers equivalently represent an orthogonal projection of model predictions onto a feasible region of predefined linear equality constraints. This projection can be formulated as a quadratic program (QP) and analytically solved by the KKT conditions within the architecture. We hence refer to it as KKT-hPINN, as it is grounded in an analytical solution that always satisfies hard linear equality constraints in both training and testing processes. It does not require additional hyperparameters and does not increase computational cost.

The proposed model is tested on several case studies to approximate Aspen models at the unit, subsystem, and plant levels. Our numerical experiments indicate that the proposed model outperforms the traditional neural network and soft-constrained PINN, in terms of both constraint satisfaction and predictive capability.

[1] R. Misener and L. Biegler, "Formulating data-driven surrogate models for process optimization," Computers & Chemical Engineering, vol. 179, pp. 108411-108411, 2023, doi: https://doi.org/10.1016/j.compchemeng.2023.108411.

[2] A. Bhosekar and M. Ierapetritou, "Advances in surrogate based modeling, feasibility analysis, and optimization: A review," Computers & Chemical Engineering, vol. 108, pp. 250-267, 2018, doi: https://doi.org/10.1016/j.compchemeng.2017.09.017.

[3] A. Cozad, N. V. Sahinidis, and D. C. Miller, "Learning surrogate models for simulation-based optimization," AIChE Journal, vol. 60, no. 6, pp. 2211-2227, 2014/6// 2014, doi: https://doi.org/10.1002/aic.14418.

[4] K. Ma et al., "Data-driven strategies for optimization of integrated chemical plants," Computers & Chemical Engineering, vol. 166, pp. 107961-107961, 2022, doi: https://doi.org/10.1016/j.compchemeng.2022.107961.

[5] K. Ma, N. V. Sahinidis, R. Bindlish, S. J. Bury, R. Haghpanah, and S. Rajagopalan, "Data-driven strategies for extractive distillation unit optimization," Computers & Chemical Engineering, vol. 167, pp. 107970-107970, 2022.

[6] L. S. Dias and M. G. Ierapetritou, "Integration of planning, scheduling and control problems using data-driven feasibility analysis and surrogate models," Computers & Chemical Engineering, vol. 134, pp. 106714-106714, 2020, doi: https://doi.org/10.1016/j.compchemeng.2019.106714.

[7] S. H. Kim and F. Boukouvala, "Surrogate-based optimization for mixed-integer nonlinear problems," Computers & Chemical Engineering, vol. 140, pp. 106847-106847, 2020, doi: https://doi.org/10.1016/j.compchemeng.2020.106847.

[8] D. Goldstein, M. Heyer, D. Jakobs, E. S. Schultz, and L. T. Biegler, "Multilevel surrogate modeling of an amine scrubbing process for CO2capture," AIChE Journal, vol. 68, no. 6, pp. e17705-e17705, 2022, doi: https://doi.org/10.1002/aic.17705.

[9] W. Bradley et al., "Perspectives on the integration between first-principles and data-driven modeling," Computers & Chemical Engineering, vol. 166, pp. 107898-107898, 2022, doi: https://doi.org/10.1016/j.compchemeng.2022.107898.

[10] G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang, and L. Yang, "Physics-informed machine learning," in Nature Reviews Physics vol. 3, ed: Springer Nature, 2021, pp. 422-440.

[11] C. A. Henao and C. T. Maravelias, "Surrogate-based superstructure optimization framework," AIChE Journal, vol. 57, no. 5, pp. 1216-1232, 2011/5// 2011, doi: https://doi.org/10.1002/aic.12341.