(143c) Framework for Hybrid Machine Learning with Open-Source Python Seeq Sysid Package

Increased computational resources and machine learning methods have triggered a new era of data science that has transformative potential across many fields, but these possibilities have yet to come to fruition because physics-based and other a priori information is not incorporated into the current machine learning algorithms. This work presents a new open-source Seeq add-on to identify input and output relationships to create digital twin models for various analytics and machine learning tasks. The System Identification Add-on supports the construction of a variety of dynamic models. White box models are primarily based on physics-based principles such as conservation laws and reaction kinetics for the chemical process. Development of these models requires extensive domain knowledge and requires domain knowledge to build and maintain. Machine learning models are largely data-driven. Greybox models combine the strengths of both approaches; they simplify the white-box model by lumping several parameters into fewer parameters that can be identified from data with higher degrees of confidence. These models are capable of capturing the primary dynamics of the system but are valid over larger ranges of operation than purely data-driven models. Currently supported model identification options are ARX, ARIMAX, FIR, transfer function, subspace, and neural network. Identified models can be a blend of time-series, state space (continuous or discrete), nonlinear differential and algebraic equations (DAEs), continuous and integer variables, and machine learned models (Neural network, Gaussian Processes, and Support Vector Regressors) [1] within the Gekko underlying modeling platform [2]. Future model integration development includes constrained SysID [3-5], Transformer Neural Networks [6], Transfer learning, and Physics-Informed Neural Networks [6,7].

The SysID Python package integrates with Seeq Workbench for data exploration and analysis. Cleaned and contextualized data from Seeq workbench is used to identify dynamic models and the models are returned back into Seeq workbench or exported to other environments. This framework supports both batch historical data investigations as well as streaming predictions. The system identification framework is used for a variety of tasks including simulation, digital twin construction, advanced process control, and real-time optimization modeling.

References

[1] Gunnell, L., Manwaring, K., Lu, X., Reynolds, J., Vienna, J., Hedengren, J.D., Machine Learning with Gradient-based Optimization of Nuclear Waste Vitrification with Uncertainties and Constraints, Processes, MDPI, 2022, accepted for publication. https://gekko.readthedocs.io/en/latest/ml.html

[2] Beal, L.D.R., Hill, D., Martin, R.A., and Hedengren, J.D., GEKKO Optimization Suite, Processes, Volume 6, Number 8, 2018, doi: 10.3390/pr6080106.

[3] Park, J., Lyman, J., Darby, M., Lima, L., Nelson, C., and Hedengren, J.D., Hybrid Machine Learning and Fundamental Modeling for Real-Time Optimization of a Fluidized Bed Roaster, 2020 Spring Meeting & 16th Global Congress on Process Safety, AIChE, Houston, TX, 29 March-2 April, 2020.

[4] Park, J., Hansen, B., Gates, N., Darby, M., Hedengren, J.D., Use of Nonlinear and Machine Learning Techniques for Improved APC Modeling, AIChE Spring Meeting, New Orleans, LA, April 2019.

[5] Nikbakhsh, S., Hedengren, J.D., Darby, M., Udy, J., Constrained Model Identification Using Open-Equation Nonlinear Optimization, AIChE Spring Meeting, Houston, TX, April 2016.

[6] Park, J., Hybrid Machine Learning and Physics-based Modeling Approaches for Process Control and Optimization, Ph.D. Dissertation, Brigham Young University, 2022.

[7] Knotts, T., Hedengren, J.D., Babaei, M.R., Physics-Informed Deep Learning for Prediction of Thermophysical Properties: The Parachor Method for Surface Tension, AIChE Annual Meeting, Phoenix, AZ, Nov 13-18, 2022.