(374n) Novel Parametric Gradient Calculation Method for Multistage Systems with Generalized Constraints

Sensitivity and gradient evaluations are essential for understanding the variability of a system subject to changes in input parameters, aiding in applications such as optimization, control, or decision-making processes (Castillo et al., 2008; Logsdon and Biegler, 1989, Horn and Tsai, 1967). Various approaches are available for the gradient evaluation in the simulation of large-scale steady-state systems, utilizing techniques such as automatic differentiation, sensitivity analysis, optimization or machine learning (Amaran et al., 2016). The term large-scale refers to problems with a substantial number of design variables, structural state variables, or constraint functions, or a combination thereof, necessitating significant high-performance parallel computing resources to solve within a reasonable timeframe (Kennedy and Martins, 2014).

However, the evaluation of gradients in large-scale multistage systems simulation poses significant challenges due to computational complexity, numerical instability, scalability issues, and the limitations of the traditional differentiation techniques. Additionally, model complexity, sensitivity to noise, and data requirements of machine learning-based approaches further amplify these challenges. Overcoming these obstacles necessitates the development of efficient, scalable and robust gradient evaluation techniques that can effectively handle the characteristics of large-scale systems while offering reliable insights for a wide array of applications.

This contribution focuses on re-examining and advancing the evaluation of parametric sensitivities within the context of simulating highly complex, hierarchical multiscale modular systems of very large size. The models being analyzed may necessitate sensitivity evaluations concerning their response to parametric inputs. These evaluations serve not only to test and verify their robustness, but also to integrate them into modular structures within a comprehensive optimization framework. Such an optimization framework aims to enhance system performance based on selected criteria, while simultaneously adhering to essential optimality constraints.

While gradient-free optimization methods have been successfully applied to important design problems, their applications typically involve no more than O(10²) design variables, and these methods exhibit very poor scalability with the dimensionality of the design variables (Kennedy and Martins, 2014). For large-scale, high-fidelity applications, gradient-based methods are deemed more suitable, although the challenges related to computational time and accuracy need to be addressed. To address these challenges, the use of either sensitivities or appropriately generalized adjoint equations for efficient calculation of constraint and objective functions gradients for generalized multistage systems, irrespective of whether they are dynamic in nature or they are steady-state.

The proposed approach adopts a generalized modular strategy suitable for any type of system, starting from a traditional sensitivity-based calculations initially, and subsequently developing a novel generalized adjoint-based method. The resulting algorithm comprises a sequence of forward and backward sweep computational steps, which are entirely equivalent, and serve as a generalization of the adjoint-based calculation methods for gradients of constraints. These methods find application in various numerical analysis computations related to dynamical systems, including optimal control problems.

It has to be noted that the model is regarded as a general modular representation of any coupled system, without making a distinction between dynamic or steady-state systems. In this context, a dynamic system is perceived as having state profiles as private internal variables, while interacting with its external environment through the input of initial conditions and parameter values. Its output consists of final conditions or any internal trajectory points that require reporting to the external environment during dynamic simulation. The proposed strategy using a novel adjoint scheme generalizes this approach to any multistage system model, of which the stages need not be of dynamic nature, such as in the use of adjoint equations in optimal control of multistage Differential- Algebraic Equation (DAE) systems (Morison and Sargent, 1986).

The choice between the use of the adjoint- and the sensitivity-based approach depends on the balance between the number of constraints/functions requiring gradient evaluation, and the number of states in the underlying dynamical system. The adjoint-based approach may be advantageous when dealing with a smaller number of constraints than state variables that require gradient evaluation, whereas the sensitivity-based approach could be more computationally efficient for a larger number of constraints than state variables in the modular treatment of the underlying dynamic system.

The simulation of a multistage system is demonstrated using an example consisting of steady-state feedforward blocks, employing both the sensitivity- and the proposed adjoint-based approach. The results obtained reveal that the numerical values derived from the gradient evaluation are identical for both methods. Therefore, it can be concluded that the newly introduced approach for general multistage sequential systems is entirely non-restrictive. This indicates its effectiveness and applicability, offering flexibility and robustness in gradient evaluation for such systems.

References

Amaran, S., Sahinidis, N.V., Sharda, B., Bury, S.J., 2016, Simulation optimization: a review of algorithms and applications, Annals of Operations Research 240, 351-380, https://doi.org/10.1007/s10479-015-2019-x.

Castillo, E., Minguez, R., Castillo, C., 2008, Sensitivity analysis in optimization and reliability problems, Reliability Engineering & Safety Systems 93 (12), 1788-1800, https://doi.org/10.1016/j.ress.2008.03.010.

Kennedy, G.J., Martins, J.R.R.A., 2014, A parallel finite-element framework for large-scale gradient-based design optimization of high-performance structrures, Finite Elements in Analysis and Design 87, 56-73, http://dx.doi.org/10.1016/j.finel.2014.04.011.

Logsdon, J.S., Biegler, L.T., 1989, Accurate solution of differential-algebraic optimization problems, I&EC Research 28 (11), 1628-1639, https://doi.org/10.1021/ie00095a010.

Morison, K.R., Sargent, R.W.H., 1986, Optimization of multistage processes described by differential-algebraic equations. In: Hennart, JP. (eds) Numerical Analysis. Lecture Notes in Mathematics 1230, Springer, Berlin, Heidelberg, https://doi.org/10.1007/BFb0072673.