(280f) A Robust Approach to Warped Gaussian Process-Constrained Optimization

In mathematical programming, optimization under uncertainty often focuses on parametric uncertainty [1-3]. But many application areas rely on uncertain, expensive to evaluate black-box functions, e.g., automatic chemical design, production planning, scheduling with equipment degradation, adaptive vehicle routing, automatic control and robotics, and biological systems [4-7].

Optimization problems with uncertain black-box constraints, modeled by warped Gaussian processes, have recently been considered in the Bayesian optimization setting [8]. This work introduces a new class of constraints in which the same black-box function occurs multiple times evaluated at different domain points. Such constraints are important in applications where, e.g., safety-critical measures are aggregated over multiple time periods. Our approach, which uses robust optimization, reformulates these uncertain constraints into deterministic constraints guaranteed to be satisfied with a specified probability, i.e., deterministic approximations to a chance constraint. This approach extends robust optimization methods from parametric uncertainty to uncertain functions modeled by warped Gaussian processes.

One way of including uncertain black-box function into the optimization problem is to consider the surrogate model’s parameters to be uncertain and use classical parametric uncertainty methods. Hüllen et al. [9] recently demonstrated this approach for polynomial surrogates using robust optimization. This presentation proposes a more direct approach utilizing probabilistic surrogate models to model the uncertain curves. We study optimization problems with constraints that aggregate black-box functions.

For the standard Gaussian process model, we show how the chance constraint can be exactly reformulated as a deterministic constraint using existing approaches. For the warped case, we develop a robust optimization approach that conservatively approximates the chance constraint. By constructing decision-dependent uncertainty sets from confidence ellipsoids based on the warped Gaussian process models, we obtain probabilistic constraint violation bounds. We use Wolfe duality to reformulate the resulting robust optimization problem and obtain explicit deterministic robust counterparts. This reformulation expresses uncertain constraints, modeled by Gaussian processes, as deterministic constraints with a guaranteed probability of constraint satisfaction, i.e., deterministic approximations to a chance constraint. We analyze convexity conditions of the warping function under which the Wolfe duality based reformulation is applicable. For non-convex cases, we develop a global optimization strategy using problem structure. To reduce solution conservatism, we furthermore propose an iterative a posteriori procedure of selecting the uncertainty set size that complements the obtained a priori guarantee.

We show how the proposed approach hedges against uncertainty in learned curves for two case studies: i) a production planning-inspired case study with an uncertain price-supply curve and ii) an industrially relevant drill-scheduling case study with uncertain motor degradation characteristics. For the drill-scheduling case study, we develop a custom strategy for dealing with discrete decisions.

1. Bertsimas, D., Brown, D.B., Caramanis, C.: Theory and Applications of Robust Optimization. SIAM Review 53(3), 464–501 (2011)
2. Birge, J.R., Louveaux, F.: Introduction to Stochastic Programming (2011)
3. Sahinidis, N.V.: Optimization under uncertainty: State-of-the-art and opportunities. Comput. Chem. Eng. 28(6-7), 971–983 (2004)
4. Bhosekar, A., Ierapetritou, M.: Advances in surrogate based modeling, feasibility analysis, and optimization: A review. Comput. Chem. Eng. 108, 250–267 (2018)
5. Deisenroth, M.P., Fox, D., Rasmussen, C.E.: Gaussian Processes for Data-Efficient Learning in Robotics and Control. IEEE Trans Pattern Analysis Machine Intelligence 37(2), 408–423 (2015)
6. Wiebe, J., Cecílio, I., Misener, R.: Data-Driven Optimization of Processes with Degrading Equipment. Ind. Eng. Chem. Res. 57(50), 17177–17191 (2018)
7. Ulmasov, D., Baroukh, C., Chachuat, B., Deisenroth, M.P., Misener, R.: Bayesian optimization with dimension scheduling: Application to biological systems. In: Z. Kravanja, M. Bogataj (eds.) 26th European Symposium on Computer Aided Process Engineering, Computer Aided Chemical Engineering, vol. 38, pp. 1051 – 1056. Elsevier (2016)
8. Snelson, E., Rasmussen, C.E., Ghahramani, Z.: Warped Gaussian processes. In: NIPS (2003)
9. Hüllen, G., Zhai, J., Kim, S.H., Sinha, A., Realff, M.J., Boukouvala, F.: Managing Uncertainty in Data-Driven Simulation-Based Optimization. Comput. Chem. Eng. p. 106519 (2019)