How Accurate Is Automated Gap Filling of Metabolic Models?

Reaction gap filling is a computational technique for proposing the addition of reactions to genome-scale metabolic models to permit those models to run correctly. Gap filling completes a reaction network by adding reactions that enable biosynthesis of all required metabolic products from available nutrients. The models are incomplete because they are derived from annotated genomes in which not all enzymes have been identified.

We present two studies of gap-filling accuracy. In the first study [1] we compared the results of applying an automated likelihood-based gap filler (MetaFlux) within the Pathway Tools software with the results of manually gap filling the same metabolic model. Both gap-filling exercises were applied to the same genome-derived metabolic reconstruction for Bifidobacterium longum. The MetaFlux gap filler attained recall of 61.5% and precision of 66.6%, taking the manual gap-filling result as the gold standard.

In the second study [2] we generated degraded versions of the EcoCyc-20.0-GEM model by randomly removing flux-carrying reactions from a growing model. We gap-filled the degraded models using 13 variations of MetaFlux (including the use of the SCIP and CPLEX Mixed Integer Linear Programming solvers and three different gap-filling algorithms) and compared the resulting gap-filled models with the original model. The best MetaFlux variation showed a best average precision of 87% and a best average recall of 61%. Although none of the 13 algorithm variations was best in all dimensions, we found one variation that was fast, accurate, and returned more information to the user. Some gap-filling variations were inaccurate, producing solutions that were non-minimum or invalid (did not enable model growth).

[1] "How Accurate is Automated Gap Filling of Metabolic Models?" Karp PD., Latendresse M., Weaver DW., Submitted.

[2] "Evaluation of reaction gap-filling accuracy by randomization," Latendresse M., Karp PD., BMC Bioinformatics 2018 19(1):53.