(532ea) Clarifying Trust of Machine-Learned Catalyst Predictions with Uncertainty Quantification

Trial-and-error experimental approaches to determine catalyst properties are typically expensive and time-consuming. Atomistic modeling of catalysts using density functional theory (DFT) combined with machine learning (ML) models such as deep neural networks (NNs) has emerged as a powerful approach for high-throughput catalyst screening and characterization. Machine learning using NNs enables prediction of catalysts at a lower cost and much faster speeds compared to using only experimental methods or DFT modeling. While much effort in heterogeneous catalysis have focused on improving NN model accuracy, less effort has been devoted to understanding the uncertainty of NN models. Different uncertainty quantification (UQ) techniques for NNs exist in the literature, but the best techniques for common ML applications in heterogeneous catalysis are not well-understood (e.g., adsorption energy predictions). Further analysis of UQ techniques and metrics will enable the broader use of ML for heterogeneous catalysis screening applications by quantifying when model predictions are confident or highly uncertain.

Herein we report on UQ analysis of a crystal graph convolutional neural network (CGCNN) for adsorption energy predictions. The CGCNN is tested on a large alloy catalyst dataset from the Open Catalyst Project.¹ The techniques used to quantify uncertainty are ensembling, Monte Carlo dropout, and evidential regression. We hypothesize that evidential regression with CGCNNs is a highly promising approach for UQ in heterogeneous catalysis applications because it does not require computing a distribution of models nor extensive sampling, unlike ensemble averaging or Monte Carlo dropout. These UQ techniques are compared using well-known UQ metrics: accuracy, dispersion, sharpness, and calibration. This work aims to improve UQ guidelines for common ML applications in heterogeneous catalysis (e.g., uncertainty-guided active learning for catalyst property optimization).

References

[1] Chanussot, Lowik, et al. "Open catalyst 2020 (OC20) dataset and community challenges." ACS Catalysis 11.10 (2021): 6059-6072.