(674f) Coupling Mechanistic and Data-Driven Models By Means of Neural Differential Equations to Incorporate Unmodeled Dynamics

Good predictive models are essential in modern chemical industry for control and optimization. Mechanistic models, incorporating physical and empirical knowledge, are dominant. However, the incorporated knowledge is inherently a simplification of reality, resulting in model structure uncertainty. Moreover, gathering the necessary knowledge is time-consuming and might be unfeasible for complex poorly-understood processes. With an ever-increasing amount of data available, data-driven methods are becoming more attractive. In these models, the structure is not explicitly specified, but rather determined by searching for relationships in the available data. Given sufficient and representative data, these models can make highly accurate predictions, unconstrained by any assumptions made. Therefore, they are particularly powerful for learning complex and poorly understood dynamics. The downside is their complete lack of interpretability. Moreover, as representative data is needed, they fail to extrapolate into regions not seen before.

Combining both approaches into a hybrid model aims at taking the best of both worlds. The mechanistic component leverages the available knowledge to model the well-understood parts. The data-driven component models the poorly-understood dynamics to fill in the knowledge gaps. Combinations of both approaches are usually achieved through an uncoupled approach, whereby first a mechanistic model is set-up and calibrated and subsequently the data-driven model is calibrated on the residual error of the mechanistic model (i.e., the difference between the model prediction and data). The data-driven component thus corrects the errors of the mechanistic component. This so-called parallel architecture leverages knowledge and data, resulting in an increased predictive power at lower data requirements [1]. However, as the data-driven model operates on the states of the system, it does not learn its dynamics (i.e., the differentials of the states), but rather the effects of those dynamics. Therefore, an uncoupled hybrid model does not generalize well [2]. Moreover, as the data-driven model is calibrated on the residual error of the mechanistic model, the parameters of both models are uncoupled. Therefore, no synergies between the data-driven and mechanistic models are possible.

Here, we propose a novel framework for coupled hybrid models based on neural differential equations [3]. Neural differential equations define the right-hand side of a differential equation as a neural network. The neural network is subsequently calibrated using backpropagation on the gradient obtained through automatic differentiation, directly learning the dynamics of the system. We combine these neural differential equations with mechanistic differential equations and calibrate the parameters simultaneously, resulting in a coupled hybrid model.

We show the power of this coupled approach on two case studies with incomplete knowledge of the dynamics of the system. In a first study, a simple reactor is modeled whose kinetics are not completely known. A known forward reaction, converting chemical A into chemical B, is modeled mechanistically. However, unknown to the modeler, a reversible reaction occurs, converting chemical B into chemical A. This reaction is thus not incorporated into the mechanistic model, resulting in a model that does not fully capture the dynamics of the system. We show that the coupled hybrid model successfully retrieves the missing kinetics, resulting in a model with high predictive power. Although this result seems trivial on this toy problem, it highlights the possibilities of applying this approach to larger, more complex systems.

Second, we model an enzyme microreactor with a pulsating feed (e.g., resulting from a positive displacement pump), inducing complex hydrodynamics. We generate synthetic data of this system using a transient CFD solver and log the outlet concentrations as a timeseries. The system is subsequently modelled using a simple mechanistic model that only includes kinetics. As the hydrodynamics are not incorporated, the model does not fully capture the dynamics of the system. Again, we show that the data-driven component of the coupled hybrid model successfully retrieves the hydrodynamics, resulting in a model that truly represents the system.

In conclusion, we introduce an architecture for coupled hybrid models based on neural differential equations. The mechanistic component incorporates all available knowledge of the modeled system, while the data-driven component captures any unmodeled dynamics, including (but not limited to) unknown kinetics and complex hydrodynamics.