(2dw) New Data-Driven Modeling Paradigms in Systems Engineering Using Novel Neural Network Structures

Systems engineering is an interdisciplinary field concentrated on the design and management of complex systems. This poster discusses my research vision in the context of three new paradigms in data-driven modeling approaches using artificial intelligence tools in the field of systems engineering.

Novel Simpler Hybrid Network Structures and Algorithms for Efficient Process Modeling

Developing accurate first-principles models for nonlinear dynamic systems can be computationally expensive. Data-driven models are relatively easier to develop but have their disadvantages in accurately representing complex nonlinear dynamic systems. For many nonlinear dynamic systems, it can be difficult to adequately model them using a simple standalone static, dynamic, or hybrid linear-time-invariant dynamic model integrated with a nonlinear static neural network¹ (NN) without suffering from the curse of dimensionality. Therefore, it is desired to consider both static and dynamic models in the hybrid structure to be nonlinear, thus making it challenging to synthesize optimal hybrid networks. The primary aspect of my PhD research proposes efficient sequential algorithms for training hybrid series / parallel static-dynamic networks with nonlinearities in both static and dynamic models².

Complex deep neural networks using Long Short-Term Memory (LSTM) or Gated Recurrent Unit (GRU) type of recurrent neural networks (RNNs) may need large number of parameters and complex structures. Therefore, the primary motivation of this work is to develop relatively simpler hybrid network structures with significantly lesser number of parameters that can still adequately model highly nonlinear dynamic systems. The proposed sequential training algorithms for the hybrid networks provide flexibility for solving the static and dynamic models independently by different optimization algorithms. We have also shown that these decomposition-based algorithms exploit the structure of the hybrid models leading to 50 – 100 times faster computation than monolithic training algorithms². Future work includes incorporation of probabilistic networks in the hybrid structures for more efficient process modeling in presence of uncertainties in training data. I envision development of novel block-oriented configurations of appropriate machine learning models with improved predictive and extrapolative capabilities based on the proposed framework.

Mass, Energy and Other Physics – Constrained Networks in Tackling Inverse / Forward Problems

The measurement / experimental data available for training and validating (i.e., inverse and forward problems) the NNs for any chemical engineering process may not necessarily satisfy mass and/or energy conservation and other physics of the system. Another aspect of my PhD thesis aims at developing fully data-driven algorithms where mass and energy balance constraints are exactly satisfied during inverse and forward problems, even though the corresponding training data violate the same. Recent years have seen the development of physics-informed neural networks (PINNs) which aim to impose certain physics constraints by penalizing the objective function of typical NN training algorithms, thus only approximately satisfying such constraints³. But in most chemical engineering applications, it is expected that certain mathematical relationships are exactly satisfied. Moreover, typical PINNs are highly system-specific. Through our research, a novel class of neural network models is proposed, namely Mass-Energy-Constrained-Neural-Networks (MECNNs), that exactly satisfies mass and energy balance constraints using only a subset of input and output boundary conditions. The mass/energy conservation laws, expressed as species molar/atom or enthalpy balance equations, are posed as equality constraints in the nonlinear parameter estimation problem, thus providing flexibility to apply this algorithm to model any generic nonlinear chemical system with uncertainties.

Unlike steady-state, developing a fully data-driven dynamic modeling approach by exactly satisfying mass/energy balance equations can be significantly challenging, since conservation of mass/energy during transience is difficult to check due to insufficient information about the holdup of a system. Therefore, in addition to steady-state modeling, efficient parameter estimation algorithms have also been developed for dynamic MECNNs. The proposed structures and algorithms have been tested by modeling various nonlinear dynamic chemical processes. It is observed that the outputs from the MECNN exactly satisfy mass/energy conservation, even though the data used for training the network violates the same. The optimal MECNNs developed in this work have also been shown to accurately capture the system truth, provided the data for model training is sufficiently rich. We are currently extending the proposed algorithms to consider additional constraints in terms of thermodynamics-based equations expressed as functions of input and output boundary conditions. We believe that this work will open a new paradigm in systems engineering research of developing physics-constrained network models without requiring any rigorous understanding of the system under consideration.

Hybrid First-Principles Artificial Intelligence (AI) Models

The synergistic integration of physics-based (first-principles) models with machine learning approaches have found numerous implementations⁴ in the form of lumped parameter models, residual modeling and digital-twin development. These models can lead to improved interpretability and extrapolation capabilities. However, there are significant opportunities for exploiting synergistic hybridization of first-principles models with data-driven models. Several novel coupling approaches have been developed as part of my PhD research including series, parallel, and cross-coupling of first-principles and data-driven models, which have been successfully applied to model different chemical systems including industrial boiler components and carbon-capture pilot-plants. For such hybrid models to be successful and yield desired outcomes, various other aspects need to be considered other than coupling approaches. In particular, it is important to consider what information need to be exchanged between first-principles models and data-driven models and at what interval, how to select the specific data-driven model for the desired outcome, and how to adapt the hybrid model.

Conventional hybrid first-principles AI models do not guarantee the satisfaction of conservation laws of the hybrid model, since in such frameworks, the AI model is used mostly as a corrective step or to address certain phenomena that is difficult to model using first-principles models alone. Another aspect of my PhD thesis considers developing algorithmic capabilities for integrating first-principles models with physics-constrained neural network models such that the hybrid model also exactly satisfies the physics conservation laws of the system. The inclusion of stochastic networks like Bayesian neural networks in the hybrid structure trained with the novel sequential algorithms has been shown to make the proposed models more robust and efficient while handling complex nonlinear dynamic data with uncertainties. I can anticipate these conceptualization approaches being extended beyond traditional process systems engineering into advanced data-analytics and high-impact research areas like energy systems optimization and sustainability.

Future Research Goals

The significance of computer simulations and data is consistently expanding as scientific exploration increasingly depend on simulations and availability of extensive data is made possible by advanced instrumentation. My post-doctoral research interests primary circles around energy systems – one of the most active research areas, specifically decarbonization, hydrogen economy, renewables integration, life cycle assessment (LCA) and sustainability. Based on my experience of collaborating with several fellow graduate students and post-doctoral scholars within WVU working on diverse research areas, I would also like to explore carbon capture and climate change, supply chain optimization and cyber security after completion of my PhD curriculum. Through my ongoing PhD project, I have been fortunate to engage in a collaboration with Southern Company, one of the prominent power generating companies in the United States and the Electric Power Research Institute (EPRI). This collaboration has given me firsthand experience in working with external organizations, which I strongly believe will be immensely beneficial for my future career in academia. As I thrive to become a future researcher and faculty member, I am also currently collaborating with my research advisor to develop research proposals in several of these areas mentioned above which are actively pursued and have a high likelihood of receiving funding from agencies such as ARPA-E, EERE, NETL, DOE, and NSF, among others.

Teaching Interests:

I am qualified to teach all undergraduate chemical engineering core courses, and especially interested in teaching process control, design, transport phenomena, mathematics, and statistics. In addition to the conventional chalk-and-duster mode of teaching, I am committed to design course materials enabling students to validate classroom learnings through various computational software such as MATLAB, Python, and Aspen, thus helping them draw meaningful parallels between textbook concepts and their practical applications.

I am also excited to teach graduate courses in process optimization, numerical methods, and process design. Furthermore, I am eager to develop grad-level elective courses on energy systems and sustainability as well as applications of machine learning in chemical engineering, with examples from real-world applications. Drawing from my previous experience of mentoring undergraduate students in the field of AI/ML, my goal is to consistently enhance my teaching materials by incorporating feedback from students, thus ensuring the most effective and relevant instruction possible.

References

1. Schoukens, M. & Tiels, K. Identification of block-oriented nonlinear systems starting from linear approximations: A survey. Automatica 85, 272–292 (2017).
2. Mukherjee, A. & Bhattacharyya, D. Hybrid Series/Parallel All-Nonlinear Dynamic-Static Neural Networks: Development, Training, and Application to Chemical Processes. Ind. Eng. Chem. Res. 62, 3221–3237 (2023).
3. Raissi, M. et al. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378, 686–707 (2019).
4. Bradley, W. et al. Perspectives on the integration between first-principles and data-driven modeling. Comput. Chem. Eng. 166, 107898 (2022).