Accelerating Multiscale Process Design with Bayesian Optimization: Progress, Challenges, and Opportunities

Design problems, which are pervasive in science, engineering, and manufacturing endeavors, can be generally formulated as mathematical optimization problems. In ideal situations, one is able to develop a physics-based model of the system of interest whose structure can be exploited by state-of-the-art solvers, especially in cases that the derivatives can be exactly computed. However, the development of multiscale process models for which derivative information is readily available remains a significant challenge in many real-world applications. A particularly challenging class of problems is when at least one component of the model is very expensive or time-consuming to evaluate such as high-fidelity computer simulations (e.g., thermodynamic calculations, molecular dynamic simulations, or solutions to partial differential equations) and laboratory experiments (e.g., measurement of critical material, chemical, or biological properties). Although “black-box” optimization methods can be applied in these situations, many of the available algorithms require extensive sampling and thus are not tractable for expensive-to-evaluate objectives and constraints.

Recently, Bayesian optimization (BO) has emerged as a powerful tool for optimizing expensive black-box functions due to its successes in material/drug design and hyperparameter optimization in machine learning algorithms. In this talk, we provide an overview of BO and discuss its main advantages and disadvantages in the context of process systems applications. We also discuss two new advances in BO that can deliver considerable gains in performance by effectively “peeking inside the box” (i.e., selectively exploiting problem structure whenever possible), which we refer to as “grey-box” BO methods. In particular, we show how BO can be modified to handle composite functions and functions for which multiple lower-fidelity (cheap-to-evaluate) approximations are available. We then describe applications of these methods to (i) integrated design and control of flexible building heating and cooling systems with hourly variation in weather conditions over year-long simulations and (ii) calibrating genome-scale bioreactor models to experimental data.