(14d) Modeling Online Dynamic Processes with Fast Training Gaussian Processes

Gaussian processes are invaluable tools for modeling chemical engineering systems as they provide accurate results with modest amounts of data and are inherently more explainable than other modeling techniques (such as neural networks). However, the Achilles heel of this modeling technique is that the training of a Gaussian Process is O(n3), meaning that there is a significant clock time required to create these models. For real time analysis (such as in dynamic optimization and control), online model updating is of critical importance. This means that Gaussian processes may not provide a quick enough training solution. A technique proposed to overcome this drawback is to discretize the input space and create Localized Gaussian Processes (LGPs), which greatly reduces the number of data points of each model resulting in faster training[1]. Another alternative is building Gaussian processes using a Bayesian Basis Function approach which has training time of O(n). However, the main concern in this approach is choosing and calculating the basis functions themselves. This is where a new method is proposed known as FoKL-GPs (Forward variable selection Karhunen-Loeve decomposed Gaussian Processes). This method uses a forward variable selection approach with basis functions created from a Karhunen-Loeve decomposition of a smoothing spline type GP kernel. Taking advantage of the ordered nature of basis functions, multiple Bayesian Basis Function models are created algorithmically in order of increasing complexity and a BIC (Bayesian Information Criteria) is used to identify the best model[2].

This work takes the case study of an online dynamic system and uses both LGPs and FoKL-GPs and implements them to compare their effectiveness in the context of online control. Additionally, this work combines the approaches by using LGPs with FoKL-GPs to investigate any additional benefit in the developed integrated approach. The example problem chosen is known as the Cascaded Tanks benchmark[3] which takes a variable input signal to control the rate at which water is pumped into two tanks in series. The input signal is used to predict the time dependence of the heights of the tanks. The accuracy is evaluated both before and after updates are considered and clock time is employed for evaluation of the update of all models. This analysis shows the applicability of the FoKL methodology for improved dynamic process control.

References

[1] Nguyen-Tuong, D.; Seeger, M.; Peters, J. Model Learning with Local Gaussian Process Regression. Advanced Robotics 2009, 23 (15), 2015–2034. https://doi.org/10.1163/016918609X12529286896877.

[2] Hayes, K.; Fouts, M. W.; Baheri, A.; Mebane, D. S. Forward Variable Selection Enables Fast and Accurate Dynamic System Identification with Karhunen-Loeve Decomposed Gaussian Processes. arXiv February 23, 2023. http://arxiv.org/abs/2205.13676.

[3] Wigren, T.; Schoukens, J. Three Free Data Sets for Development and Benchmarking in Nonlinear System Identification. In 2013 European Control Conference (ECC); IEEE: Zurich, 2013; pp 2933–2938. https://doi.org/10.23919/ECC.2013.6669201.