(504f) Beyond First-Principles: Harnessing Time-Series Data for Improved Process Modelling.

The pharmaceutical industry’s path towards Industry 4.0 underscores a pivotal transition to integrating advanced AI-based tools in various research domains, with an emphasis on process control. This paper introduces an exploratory study that contributes towards reducing the reliance on state-space or first-principles knowledge. Such conventional and foundational strategies have their limitations in real-time control implementations owing to their extensive computation needs, or inability to capture non-linearities that are usually observed in process data. With continuous manufacturing now producing abundant, information-dense data from improved sensor techniques, it is vital to obtain insights into any underlying long-term time-dependencies to increase prediction accuracy in a dynamic system.

Proof of concept and model-building was carried out from temperature data obtained using a jacketed static mixer system, which is a critical aspect of the pharmaceutical industry. The experiment introduced an impulse temperature change into one of the inlet streams in the system and consequently monitored the altered trajectory of the jacket and mixer outlet temperatures. The predictive performance of this response trajectory was reviewed using a host of models. The baseline outlet temperature predictions were created using two different methods. The first was a simple first-principles model using a heat exchanger approach for a double pipe system, while the second was a classical statistics based time-series approach such as ARIMA (Autoregressive Integrated Moving Average) and VECM (Vector Error Correction Model). These baseline models were compared against advanced machine learning (ML) and deep learning (DL) techniques, specifically XGBoost and LSTM. A combination of window lengths for historical data and future forecast horizons were analyzed when using AI-ML models to ensure a thorough probing of each model’s capabilities and limitations. Additionally, the differences in utilizing raw data and feature-engineered data for the AI-ML models was also examined to answer two important questions. The first, the extent to which feature engineering enhances model accuracy, and secondly, whether the improvement in accuracy, if any, justifies the need for domain-specific expertise in developing engineered features. To achieve this, a mix of physics-informed features and mathematical transformation based features were incorporated into the models and assessed accordingly.

Our initial findings highlight that LSTMs obtained the lowest mean absolute error in the outlet temperature predictions across all test data, outperforming the selected baseline and XGBoost models. On evaluating the predictive performance between the two AI-ML models for a one-step ahead forecast averaged across various history lengths, we found the following. The LSTM predictions for the outlet of the mixer have an improved accuracy of 92.690% over XGBoost. For the jacket outlet, XGBoost only slightly outperformed LSTM with an improvement of 2.425%. However, when longer forecasts such as ten step-ahead forecasts were investigated, it was found that LSTM outperforms XGBoost for both jacket outlet as well as mixer outlet predictions. The improvement in accuracy averaged over the range of history lengths showed that LSTMs improved upon XGBoost’s predictions of the jacket outlet and mixer outlet by 25.372% and 81.277% respectively. This observation proves that time-dependent relationships contain valuable information that can provide a more nuanced understanding and control of process dynamics. Moreover, of all the tested engineered features, only one feature contributed towards improving the prediction accuracy. This physics-informed feature was created from the mass flow rate contributions of both inlet streams into the static mixer and improved prediction accuracy by 5%. Although beneficial, the addition of such a feature did not markedly improve on the model built using raw sensor data. Therefore, this supports the argument that incorporating LSTMs into existing process control architecture is advantageous for continuous pharmaceutical manufacturing as it can vastly improve prediction accuracies while also reducing the reliance on process-specific experts.

Building on these insights, ongoing efforts are directed towards applying LSTM models trained on on-line UV-vis spectroscopy data for real-time prediction of reaction kinetics in flow chemistry. Further, our future work will be dedicated towards implementing this model into a Model Predictive Control (MPC) framework for flow control. We hypothesize that such models that incorporate long-term dependencies may result in increased robustness for process control as they have the capability to account for time-dependent changes in the system’s environment apart from considering process dynamics. However, this requires further investigation and as a result, is a key research question for our future work.

In conclusion, this foundational study highlights the importance of leveraging expansive time-series based sensor data to achieve improvements in predictive modelling. Its potential integration into existing control architecture thereby ensures a smooth and quick transition from ideation to implementation, making it a promising tool for the pharmaceutical industry.