The Neural Ordinary Differential Equation (NODE) is a recent family of deep learning models that combines neural network models with ordinary differential equation solvers to create a continuous approximation of the output [1]. In the field of chemical engineering, accurate modeling of process systems is a critical component of process optimization and control. With the development of machine learning modeling techniques, neural network-based modeling and control of process systems has evolved into a popular research area due to the strong performance of neural network models and the increased accessibility to developing such models. Recurrent neural networks (RNNs) have been demonstrated to be an effective black-box approach, particularly in the case of regressing time-series data, and have been utilized in the design of model predictive control (MPC) systems in various research works [2, 3]. However, RNNs are usually interpreted as a discrete-time model that approximates a continuous process with a finite number of intermediate steps. This discrete structure requires a consistent step size between data sequence within the training data set of RNN-based model, which makes the RNN models less robust when operating with irregularly sampled data, such as missing points due to sensor failure or other types of errors. The NODE model, which computes its output with an ODE solver and only needs a single data point as the initial state to compute its prediction, can be an alternative neural network model for time-series processes without the discrete and uniform step limitation of RNN models. Moreover, the NODE model is developed with an integrated ODE solver using the adjoint sensitivity method, which does not require a uniform step-size data sequence to train. Therefore, the NODE model will be a more robust alternative to the RNN model in terms of the format of the data structure for both training and implementation.
This paper aims to investigate the development of the NODE-based MPC. The development of the NODE model and the establishment of an MPC based on this continuous neural network model will be demonstrated. Furthermore, noisy data is a critical challenge to deal with in practical scenarios, and the performance of the NODE model under the effect of Gaussian and non-Gaussian types of noise will be investigated. The subsampling method is applied in this study to account for noise corruption and is found to be an effective method of suppressing non-Gaussian noise. Finally, a chemical process example is used to demonstrate the implementation of NODE-based MPC using the subsampling method to account for noisy data.
References:
[1] Chen, R.T., Rubanova, Y., Bettencourt, J., Duvenaud, D.K., Neural ordinary differential equations. Advances in neural information processing systems, 2018, 31.
[2] Wu, Z.; Tran, A.; Rincon, D.; Christofides, P. D. Machine learning-based predictive control of nonlinear processes. Part II: Computational implementation. AIChE J. 2019, 65, e16734.
[3] Wong W.C., Chee E., Li J., Wang X. Recurrent neural network-based model predictive control for continuous pharmaceutical manufacturing. Mathematics. 2018 Nov 7;6(11):242.
