(415a) Modeling Complex Nonlinear Systems Using Concatenated Static-Dynamic Neural Networks

Developing accurate first-principles models for complex nonlinear dynamic systems can be time consuming and difficult and may be infeasible to construct for certain systems due to lack of knowledge. It is also difficult to adapt first-principles models for time-varying systems. Data-driven or black-box models are easier to develop, simulate, and adapt online¹. However, it can be difficult to accurately represent complex, nonlinear dynamical system using data-driven models. Typical approach in the use of concatenated network for representing nonlinear dynamic system is to use a linear dynamic model integrated with a nonlinear static neural network^2–5. These models may fail to accurately represent the nonlinearities in the process dynamics. If both the static and dynamic models are nonlinear, it becomes considerably challenging to synthesize optimal hybrid network and estimate the network parameters. This work proposes concatenated static-dynamic neural networks with nonlinearities in both static and dynamic models. Efficient algorithms are also developed for optimal synthesis of the network and estimation of parameters.

Classical backpropagation algorithms for solving static and dynamic neural networks use first order methods, but these methods may require significant tuning of hyper parameters, can suffer from slow convergence, and may lead to difficulty in convergence in presence of inequality constraints. On the other hand, while second order methods can address some of the issues mentioned above, they can be computationally expensive due to Hessian calculation, may be limited in terms of candidate architectures, and may only be used for parameter estimating of small or medium size networks without incurring excessive computational expense. Therefore, applying the second order methods for the entire nonlinear static-dynamic network can be computationally expensive. Furthermore, the most efficient algorithm for solving the static network can be different than the most efficient algorithm for solving the dynamic network. Therefore, a novel strategy is developed where the static and dynamic networks are solved independently by efficient algorithms for those respective networks while solving an outer layer optimization for estimating the connection weights between the static and dynamic networks. Several outer layer optimization algorithms are proposed for efficient solution of the concatenated network. The developed algorithms are flexible for incorporating flexible network architectures and can include inequality constraints.

Various architectures are proposed for both the static and dynamic networks. In addition, various neuronal models with several candidate basis functions are considered to develop flexible networks that offer tradeoffs between computational expense and accuracy for highly nonlinear systems.

The proposed algorithms are applied to modeling the widely used Van de Vusse reactor and pH neutralization reactors as well as a highly complex superheater system with spatio-temporal variation, where complex dynamics associated with reactive-diffusive processes leading to oxide scale formation in the superheater tube banks coupled with mass and heat transfer makes it a challenging system. It is observed the concatenated static-dynamic neural network results in superior performance compared to the existing conventional static or dynamic networks taken separately or linear dynamic-nonlinear static networks⁴. Computational expense and convergence performance of the proposed algorithms are found to be far superior compared to the first order-only and second order-only methods especially for the superheater system. Impact of various basis functions on the computational expense and accuracy of the algorithms are also evaluated. Overall, proposed algorithms show promise for solving large nonlinear dynamic network problems.

References

1. Su, H.-T., Bhat, N., Minderman, P. A. & McAvoy, T. J. Integrating Neural Networks With First Principles Models for Dynamic Modeling. Dynamics and Control of Chemical Reactors, Distillation Columns and Batch Processes (IFAC, 1993). doi:10.1016/b978-0-08-041711-0.50054-4
2. Chen, L., Hontoir, Y., Huang, D., Zhang, J. & Morris, A. J. Combining first principles with black-box techniques for reaction systems. Control Eng. Pract. 12, 819–826 (2004).
3. Sentoni, G. B., Biegler, L. T., Guiver, J. B. & Zhao, H. T. State-Space Nonlinear Process Modeling: Identification and Universality AIChE J. 44, 2229–2239 (1998).
4. Mahapatra, P., Ma, J. & Zitney, S. E. Nonlinear Model Predictive Control Using Decoupled A-B Net Formulation for Carbon Capture Systems - Comparison with Algorithmic Differentiation Approach. Proc. Am. Control Conf. 2018-June, 6439–6444 (2018).
5. Zhao, H., Guiver, J., Neelakantan, R. & Biegler, L. T. A nonlinear industrial model predictive controller using integrated PLS and neural net state-space model. Control Eng. Pract. 9, 125–133 (2001).