Incorporating gradient information during the regression of surrogate models of various processes can greatly improve their representation of the process and help simulation and optimization of processes by providing more reliable gradient behavior [3]. Recently, Cozad and Sahinidis [2] proposed an MINLP formulation of symbolic regression that can be expanded to simultaneously calculate the first and second partial derivatives of key parameters. The purpose of the current paper is to enhance the surrogate model regression procedure with gradient information to improve the bounding and model discrimination by incorporating partial first and second derivatives into the regression framework. Symbolic regression learns both the model structure and parameters to model a data set, unlike traditional regression that limits the scope of the regression to a fixed functional form and has typically been performed with genetic programming [1]. The regression only requires the specification of a set of operators and operands (+, -, *, /, exp(.), log(.), sqr(.), cub(.), √(.), etc.) to flexibly develop new functional forms that accurately represent the data. Recent developments in applying a global deterministic approach have shown improved fitting metrics, such as sum of squared error and other information criterion [2,5]. Further expanding the symbolic regression capabilities by applying chain rule evaluations of the derivatives while simultaneously determining the model structure allows for gradient information to impact the regression process and improve the surrogate representation of the data. We applied this technique to benchmarked equations to demonstrate these new capabilities.
Reference cited:
[1] Koza, J.R.: Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge, MA (1992) 24.
[2] Cozad, A. & Sahinidis, N.V. (2018). A global MINLP approach to symbolic regression. Mathematical Programming, 170, 97-119, 2018.
[3] Matias, J. & Jaschke, J (2019) Using a neural network for estimating plant gradients in real-time optimization with modifier adaptation. IFAC-PapersOnLine, 15, 808-813, 2019.
[4] J. McDermott, D.R. White, S. Luke, L. Manzoni, M. Castelli, L. Vanneschi, W. Jaśkowski, K. Krawiec, R. Harper, K.D. Jong, U.M. O’Reilly, Genetic programming needs better benchmarks, in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO) (ACM, Philadelphia, 2012)
[5] Tawarmalani, M.;Sahinidis, N. V. Global optimization of mixed-integer nonlinear programs: A theoretical and computational study, Mathematical Programming, 99, 563-591, 2004.
