(133b) Modeling the Hydrocracking Process with Kriging through Bayesian Transfer Learning

Hydrocracking process converts heavy products from vacuum distillation into diesel and lubricants, which are more valuable products. Reactions take place in presence of a catalyst and when supplying a catalyst, a vendor must guarantee its performance. A lot of effort of the catalyst development cycle aims at predicting as accurately as possible its performance in the customer’s refinery. More specifically, we want to predict the density of the diesel cut, based on information on the feedstock, the operating conditions and some information on the output. Modeling of hydroprocessing can be based on ODE or use a machine learning algorithm. In this work, the linear model and the kriging model (Cressie 1990) are considered. The construction of predictive models is based on experimental data and experiments are very expensive: they must be performed in conditions mimicking those of refineries on a variety of feedstocks that are not commercial and diffi to obtain. New catalysts are constantly being developed so that each new generation of a catalyst requires a new model that is until now built from scratch from new experiments. The aim of this work is to build the best predictive model for a new catalyst from fewer observations and using the observations of previous generation catalysts. This task is known as transfer learning (Pan and Yang 2010 and Tsung et al. 2018).

2 Method

The method used is the transfer knowledge of parameters approach, which consists in transferring regression models from an old dataset to a new one. Obviously, the transfer

method depends on the type of regression model and two regression models are considered here, the linear model and the kriging model. The models are fi on the previous catalyst dataset, which consists of 3,177 observations. The obtained models on this large dataset are of very good quality for the prediction of processes using this previous catalyst.
In order to adapt the past knowledge to the new catalyst, a Bayesian approach is con- sidered. The Bayes Theorem gives that the posterior distribution of the model parameters θ is

π(θ|y, X) =π(θ)f (y|θ, X)/ f (y|X)

where y is the the diesel density to be predicted, X the matrix of new observations, π(θ) the prior distribution of parameters, f (y|θ, X) the likelihood and f (y|X) the marginal likelihood. The likelihood represents the knowledge about the new observations, thus the posterior distribution will be modifi when adding observations. The idea of the approach is to take as prior π(θ) a distribution centered on the previous model parameters. A pragmatic approach to chose the π(θ) variance ensuring that it is large enough to allow parameter change and small enough to retain the information is proposed.
For the kriging model, two Bayesian approaches are considered: one with a prior only for the trend parameters and one for all its parameters. When a prior is considered for all kriging parameters, no closed form exists for the posterior distribution and a MCMC1 algorithm is used to estimate the posterior distribution.

3 Results
The models for the new catalyst are fi for a varying number of new observations. These observations are selected randomly from a dataset of 1,004 new experiments. For each number of new observations selected, models are fitted using the Bayesian transfer approach and without previous knowledge. In order to evaluate the quality of the models, we focus on the RMSE score.
With the Bayesian transfer approach, the RMSE scores for the transferred models are always lower than those obtained without transfer , especially when the number of observations is low. Satisfactory models can be fitted with only five new observations.

Without transfer, reaching the same model quality requires about fi y observations. More- over, the Bayesian transfer approach yields better models than models learned from scratch, whatever the number of observations.
In this work, the observations from the new catalyst were selected randomly and Design of Experiment coupled with this Bayesian transfer approach will be considered in further works.

References
Cressie, N. (1990). “The origins of kriging”. In: Mathematical geology, pp. 239–252.
Pan, S. and Q. Yang (2010). “A Survey on Transfer Learning”. In: IEEE Transactions on Knowledge and Data Engineering, pp. 1345–1359.
Tsung, F. et al. (2018). “Statistical transfer learning: A review and some extensions to statistical process control”. In: Quality Engineering, pp. 115–128.