(89a) How to Develop Quickly New Kinetic Models By Including a Priori Information (Part I): Transfer Learning from Previous Catalyst Generations to New One

How to develop quickly new kinetic models by including a priori information (Part I): Transfer Learning from previous catalyst generations to new one

Per Julian Becker¹, Loic Iapteff¹, and Benoit Celse¹

1 IFP Energies Nouvelles, Rond-Point de l’Echangeur de Solaize, 69360 Solaize, France

*E-mail: per.becker@ifpen.fr

HIGHLIGHTS

• Kinetic model parameter estimations with Monte-Carlo Markov Chains for new catalyst generation.
• Transfer Learning between catalyst generations by adding prior distribution of model parameters.
• Inclusion of hard & soft constraints in order to respect experimental rankings.

INTRODUCTION

Catalytic processes are immensely important to the chemical industry, with more than 95% of products using a catalyst at some part of the manufacturing process. Maximizing of product yields (and/or quality) while minimizing of costs, particularly in terms of energy consumption is becoming increasingly important considering the current global economic context of rising energy cost and requirements of reducing CO2 emissions. That is why, new catalyst generation are developed periodically to improve activity or selectivity.

Predictive reactor models are required for process design optimization, which often implies some extent of extrapolation beyond the range of available experimental data. Models must therefore be not only precise, i.e. within the acceptable range for the calibration and validation datasets, but also robust, i.e. avoiding aberrant model predictions when extrapolating. This is particularly challenging when the number of experimental data points limited and/or characterization of the reaction network is difficult due to the complexity of the feedstocks. For example, this is the case for VGO hydrocracking, where expensive and time-consuming pilot plant tests are required and the feedstocks are complex mixtures with more than 100,000s of hydrocarbon species. Typically, between 50 and 100 experimental points, with a wide variability of feedstocks (around 10) of different nature and origin are required for a good model, which is the case for mature catalysts. Model development for newly developed catalyst is particularly challenging as initially only few pilot plant points with a limited number of feeds are available. The scope of this study is then to improve the new catalyst kinetic model using the previous generations.

This work explores a methodology to include knowledge from previous catalyst generation(s) in a model for a newly developed catalyst via Bayesian Transfer Learning (TL) using Monte-Carlo Markov Chains (MCMC) [1, 2, 3] by applying a prior probability on the model parameters.

MATERIALS & METHODS

The example chosen to illustrate the technique is a Continuous Lumping (CL) model of the True Boiling Point (TBP) distribution with a single family and two reactions (cracking & isomerization); the Hydro De-Nitrogenation (HDN) reaction is solved simultaneously to account to the inhibition effect of organic nitrogen [4]. This formulation was found to produce robust, predictive models.

While machine learning methods have been used to model hydrocracking processes [2, 5], such models are generally only useful for process control, where the feeds and operating conditions remain in a narrow range and large datasets from industrial operating data is available. Purely data-driven modelling approaches, while precise, are not predictive and generally perform poorly in extrapolation.

The Bayesian Theorem gives the posterior distribution of the model parameters in terms of a prior distribution [1].

Where is the matrix of the observations (i.e. feed properties and operating conditions) and y is the matrix of output of the model, here standard cut yields and TBP distribution. The prior distribution is taken as the parameter distribution of robust model for a previous catalyst generation with a large number of available pilot plant points. While a closed form exists [1] for simple models, such as linear models, this is not the case for the mode complex CL model. The MCMC algorithm, which consists of randomly varying the parameters and applying a likelihood function, and optionally a prior, to determine whether to keep or discard the parameter variation, can be used in these cases. The prior distribution was obtained by running the MCMC algorithm on the large, mature, source dataset. This prior was then used for running the algorithm on the target dataset. The variance, or g-value, determines the adherence of the model to the value of the source parameters. The optimal g-value can be determined by performing cross-validation.

RESULTS & DISCUSSIONS

The methodology was applied to a source dataset of a previous catalyst with 91 points and 22 feeds and a target dataset from a new catalyst with 62 points and 14 feeds. A stacking test (cata1+cata2 and cata2+cata1) was used for model validation. The precision of the transferred model on the calibration and validation tests was found to be comparable to a model calibrated using a standard gradient descent algorithm without prior (transfer learning & imposed catalyst rankings); the expected catalyst rankings, in terms of diesel yield were however not respected in this case. The transferred model respects the catalyst rankings. The gradient-descent model clearly suffers from over-fitting. The MCMC algorithm can be used to produce more robust models with fewer experimental points by including information from previous catalyst generations and high-throughput test with are traditionally not used for model development.

KEYWORDS

Monte-Carlo Markov Chains, Parameter Identification, Continuous Lumping Model, Bayesian Statistics, Vacuum Gas Oil Hydrocracking

BIBLIOGRAPHY

1. Iapteff, 2022, Transfer Learning for Smart Predictive Analytics, PhD thesis
2. Tsung, et al., 2018, Statistical transfer learningn: A review and some extensions to statistical process control. Quality Engineering, pages 115-128
3. Iapteff, J. Jacques, M. Rolland, and B. Celse, 2021, Reducing the number of experiments required for modelling the hydrocracking process with kriging through bayesian transfer learning. Journal of the Royal Statistical Society: Series C (Applied Statistics).
4. J. Becker, B. Celse, D. Guillaume, V. Costa, L. Bertier, E. Guillon, and G. Pirngruber, 2016, A continuous lumping model for hydrocracking on a zeolite catalysts: model development and parameter identification, Fuel, pages 73–82.
5. Pang, et al., 2024, Data-driven prediction of product yields and control framework of hydrocracking unit. Chemical Engineering Science, 1193386
6. Bouveyron and J. Jacques, 2010, Adaptive linear models for regression: improving prediction when population has changed, Pattern Recognition Letters, pages 2237–2247.
7. P. Robert et al., 2007, The Bayesian choice: from decision-theoretic foundations to computational implementation, volume 2. Springer.