(676d) Accelerating Optimization By Exploiting the Existence of Low Dimensional Manifolds

Many multiscale problems, such as those that employ the quasi-steady state assumption in chemical reaction engineering, computational biology [1], and materials science [2], exhibit fast dynamics that quickly die out, leaving trajectories that move slowly on a low-dimensional manifold. The dynamics of traditional optimization on multiscale potentials share some of these features, and so optimization is often cumbersome and suffers from slow convergence due to the high number of decision variables.

Here, we propose a method to accelerate optimization of such multiscale systems by exploiting the underlying low-dimensional manifold. Our approach uses machine learning (here, diffusion maps) to reduce the number of decision variables by adaptively discovering good local latent variables in a data-driven fashion, foregoing the need of a priori knowledge about the system. It operates adaptively and iteratively: the process involves sampling a local patch, discovering a suitable parameterization of the manifold, optimizing on the manifold, and lifting back to the ambient space. This scheme efficiently guides the optimizer towards the minimum and extends [3] by optimizing in the reduced space as opposed to solely extrapolating the manifold.

We also extend our algorithm to multi-objective optimization (MOO) problems. Our manifold-discovery-and-extrapolation scheme can be used to discover the Pareto front when the objective functions correlate, so that the Pareto front resides on a lower-dimensional manifold. We demonstrate the effectiveness of our technique on common MOO toy examples (e.g. DTLZ).

[1] M. Alber, A. Buganza Tepole, W. R. Cannon, S. De, S. Dura-Bernal, K.Garikipati, G.Karniadakis, W.W.Lytton, P.Perdikaris, L.Petzold, and E.Kuhl. Integrating machine learning and multiscale modeling: Perspectives, challenges, and opportunities in the biological, biomedical, and behavioral sciences. NPJ Digital Medicine, 2(1):1–11, 2019.

[2] M.F. Horstemeyer. Multiscale modeling : A review. In J. Leszczynskiand and M.K. Shukla, editors, Practical Aspects of Computational Chemistry: Methods, Concepts, and Applications, 4, 87–135. Springer, Dordrecht, Netherlands, 2009.

[3] D. Pozharskiy, N. J. Wichrowski, A. B. Duncan, G. A. Pavliotis, and I. G. Kevrekidis. Manifold learning for accelerating coarse-grained optimization. Journal of Computational Dynamics, 7(2):511—536, 2020.