(676b) Human-in-the-Loop Bayesian Optimization for Expert-Guided Experimental Design of Engineering Systems

Bayesian optimization has emerged as a powerful technique for optimizing expensive-to-evaluate, black-box functions in various domains, including chemical engineering [1,2]. By efficiently balancing exploration and exploitation, Bayesian optimization enables data-efficient optimization of complex systems where gradient information is unavailable or prohibitively expensive to obtain. However, most Bayesian optimization approaches focus on fully automated optimization, neglecting the valuable insights and domain expertise possessed by human experts particularly when applied to engineering systems. This work introduces a novel methodology for expert-guided Bayesian optimization that integrates human input in a collaborative manner into the optimization loop while maintaining practicality and ease of use.

Our proposed approach exploits the hypothesis that humans are more effective and efficient at making discrete choices rather than specifying continuous throughout [3]. At each iteration, we formulate and solve a high-throughput (batch) Bayesian optimization problem across multiple potential solutions. Multi-objective optimization is applied to ensure that the utility values across the batch of solutions are maximized, whilst simultaneously maximizing the information spread across the solutions, which is characterised by the determinant of the covariance matrix. By selecting the solution at the knee point of the resulting Pareto front, our methodology ensures that the presented alternative solutions offer a balance between expected improvement and distinctiveness. We find that this approach distributes solutions evenly across local optima of the acquisition function, which practice have very similar utility values.

The expert is then tasked with selecting their preferred solution from this set of alternatives, allowing them to incorporate their domain knowledge and intuition into the decision-making process. The chosen solution is subsequently evaluated and added to the dataset, informing the next iteration of the optimization loop. By involving the expert in this discrete selection step, the methodology enables continuous integration of expert input while minimizing the time and effort required from the expert, making it practical for real-world applications. Figure 1 provides a visual overview of our methodology, and Figure 2 demonstrates a one-dimensional example, where solutions that are provided to the expert are distributed throughout distinct local optima, all with similarly high expected improvement values.

In addition to providing a number of alternative solutions to the expert at each iteration, we enable the expert to specify a number of unique solutions before optimization proceeds through an augmented design-of-experiments step. By allowing the expert to choose specific solutions at this stage we remove any time constraints that would be present if continuous solutions were to be selected ‘within the loop’. Figure 3 demonstrates this initial design of experiments approach. Our overall methodology combines the benefits of being able to specify individual solutions at a convenient stage, with the efficiency and ease of selecting between discrete alternatives throughout the optimization run.

To demonstrate the effectiveness of the expert-guided Bayesian optimization approach, we present a series of numerical case studies and real-world applications in chemical engineering. These include the optimization of bioprocess control strategies and the simultaneous optimization of operating conditions and geometry for a pulsed flow coiled tube reactor. The case studies highlight how our methodology can lead to faster convergence and improved solution quality compared to fully automated Bayesian optimization, particularly when the expert possesses relevant domain knowledge.

Our bioprocess optimization case study involves the fed-batch production of a high-value product, where the control variables are the substrate feed rate and temperature profiles over time, under the assumption that the dynamics are not able to be learned [4]. The expert-guided approach allows for the incorporation of heuristics and process understanding, such as maintaining substrate concentrations within certain ranges and avoiding drastic temperature changes. By selecting control strategies that align with their expertise, we demonstrate how human experts contribute to the identification of high-quality solutions in fewer experiments than the purely data-driven approach. Figure 4 demonstrates the convergence of standard Bayesian optimization on this problem compared to applying our approach.

In the coiled tube reactor case study, the optimization goal is to determine the optimal operating conditions (e.g., flow rate, pulsation frequency) and reactor geometry (e.g., coil diameter, pitch) to maximize the number of equivalent tanks-in-series corresponding to plug flow performance [5]. We demonstrate that our expert-guided approach leads to the discovery of novel and efficient reactor designs that may have been overlooked by a fully automated optimization routine in fewer function evaluations.

In addition to real world case studies using human experts, we analyse the impact of various problem factors on convergence, including the dimensionality of the optimization problem, the level of noise in the objective function evaluations, and the number of alternative solutions presented to the expert at each iteration. We apply a number of ‘behaviours’ that we assume a hypothetical expert to have in order to benchmark our approach, including a perfectly performing expert, a number of stochastic policies, and finally an adversarial (or misaligned) expert that consistently selects the worst solution.

Our results demonstrate that the adversarial, and the ‘correct’ expert provide a lower and upper bound on the rate of convergence respectively. As expected, standard Bayesian optimization falls between these. Importantly, for a large class of problems, when an expert selects a solution randomly from the given choices, they perform as well as standard Bayesian optimization. These results indicate that so long as the expert can select the best solution on average better than random, our collaborative approach has the potential to improve on the rate of convergence of Bayesian optimization, with minimal expert input.

These benefits become more pronounced in higher-dimensional spaces and in the presence of noisy evaluations, as the expert's input can help navigate large and complex spaces more effectively. Furthermore, the study shows that presenting a moderate number of alternatives strikes a balance between providing sufficient diversity and not overwhelming the expert with too many choices. Figure 5 demonstrates the effect of increasing problem size across a range of test functions.

By providing a framework for effectively integrating expert knowledge into the Bayesian optimization loop, our methodology addresses the limitations of fully automated approaches and enables a collaborative partnership between human experts and data-driven optimization techniques, highlighting one of the first examples of human-AI collaboration within chemical engineering. Our approach not only leads to improved solution quality but also enhances the interpretability and trustworthiness of the optimization process, as the expert remains actively involved and can provide insights into the reasoning behind the selected solutions.

Moreover, our expert-guided Bayesian optimization methodology has the potential to facilitate the adoption of advanced optimization techniques in industrial settings. By allowing experts to interact with the optimization process in a user-friendly, efficient, and intuitive manner, the approach lowers the barrier to entry and promotes the integration of domain expertise with state-of-the-art optimization algorithms. This can lead to more efficient and effective optimization workflows, ultimately driving innovation and improving the performance of engineering systems.

To conclude, this work presents a novel expert-guided Bayesian optimization methodology that enables the integration of human expertise into the optimization of expensive-to-evaluate functions. By leveraging the strengths of both human decision-making and data-driven optimization, our methodology achieves faster convergence, improved solution quality, and enhanced interpretability compared to fully automated methods. The methodology is demonstrated through a series of numerical case studies and real-world applications in chemical engineering, showcasing its potential to enhance the optimization of complex engineering systems. This work represents a significant step forward in human-AI collaboration and paves the way for more effective and efficient optimization workflows within chemical engineering.