(522a) Comparison of Uncertainty Propagation Methods for Output Moment Estimation

The steady increase in computational power enabled the widespread utilization of simulation models to assist decision making in chemical and energy process design, operations, and control. There are many sources of uncertainties in these models, e.g., in the model inputs and parameters estimated from experimental data, which leads to uncertainties in the outputs of these models. Uncertainty propagation methods are used to quantitatively characterize the uncertainty of the simulation output , where the dependence of to uncertain independent variables and parameters is defined by the simulation model, . Here, each component of is a random variable with a probability density function, , and some of the components may be correlated with each other. The order moment of a random variable is defined as , and the central moment of order is defined as where is the expected value of . The central moments of are routinely employed in aiding decision making for design, operation, and control of processes. For example, mean, the first-order moment, is implemented in stochastic programming (Birge & Louveaux). Both the first and second order central moments, mean and variance, play a key role in robust design (Raza & Liang, 2012).

The uncertainty propagation methods provide different approaches for computing the moment integrals because they cannot be analytically evaluated for most engineering applications. For example, Monte Carlo integration randomly samples the uncertain input space, evaluates the simulation model at those points, and utilizes appropriate averaging of the simulation model output values to estimate the moments (Dieck, 2007). Quasi-random Monte Carlo methods use low-discrepancy sequences to determine the location of the samples (Caflisch, 1998). The numerical integration methods, like full factorial numerical integration, compute the integral using different quadrature formulas (Lee & Chen, 2009). There are methods based on expansions like polynomial chaos expansion that approximates the output random variable as a polynomial series of standard normal random variables which, in turn, enables the computation of the integral (Ghanem and Spanos, 1982). With all these methods, the accuracy of moment estimates improves with increasing number of simulation model evaluations. However, how these estimates compare to each other for a computational budget for simulation models with different characteristics, and if and how these estimates change with respect to each other as the computational budget increases are open questions.

In this study, we computationally investigated the efficiencies of six different uncertainty propagation methods in terms of their abilities to estimate the first-four moments of simulation outputs. The uncertainty propagation methods are Latin Hypercube Sampling (LHS), Full Factorial Integration (FFNI), Univariate Dimension Reduction (UDR), Halton series, Sobol series, and Polynomial Chaos Expansion (PCE). We use a number of test functions with varying degrees of nonlinearity and with different numbers of uncertain inputs as simulation models to derive guidelines for choosing the most efficient uncertainty propagation method considering the nonlinearity or uncertain input dimensions. In this experiment, moments calculated from all the methods are compared to the ‘true’ values which are the moment values calculated by 5×10⁶ Monte-Carlo simulations. We also compared the minimum number of model evaluations required for the first four moments of the output distribution reaching and remaining inside a band whose width is equal to a pre-determined percentage of the true moment value. Experiments were coded in Python 3.6 and packages Sobol_seq (Naught101, 2017), and Chaospy (Feinberg and Langtangen, 2015) were utilized for implementing UPMs using Sobol series and polynomial chaos expansion, respectively.

The results suggest that FFNI estimates the first four moments accurately with fewer model evaluations for uncertain inputs with low dimensions (less than three), but the number of model evaluations explodes to obtain a similar accuracy at higher dimensions. The moment estimates of the UDR converge with relatively few model evaluations for lower dimensions. However the estimates generally have a bias, which is larger for higher order moments such as skewness and kurtosis. The moment estimates obtained by Halton and Sobol sampling methods are robust with respect to the changes in the number of uncertain inputs. However, the number of model evaluations required to obtain accurate estimates of higher order moments may become relatively large. The estimates of PCE are robust with respect to the nonlinearity of the model.

References

Birge, J.R. and Louveaux, F. (1997). Introduction to Stochastic Programming, Springer Verlag

Caflisch, R. (1998). Monte Carlo and quasi-Monte Carlo methods. Acta Numerica, 7, 1-49. DOI:10.1017/S0962492900002804

Dieck R.H. (2007). Measurement uncertainty: methods and applications. 4th edition.

Feinberg F., Langtangen H.P. (2015). Chaospy: An open source tool for designing methods of uncertainty quantification, Journal of Computational Science, 11, 46-57.

Ghanem RG, Spanos PD (1991) Stochastic finite elements: a spectral approach. Springer, New York.

Herzog M., Gilg A., Paffrath M., Rentrop P., Wever U. (2008) Intrusive versus Non-Intrusive Methods for Stochastic Finite Elements. In: Breitner M.H., Denk G., Rentrop P. (eds) From Nano to Space. Springer, Berlin, Heidelberg.

Lee, S., & Chen, W. (2009). A comparative study of uncertainty propagation methods for black-box-type problems. Structural and Multidis. Optimization, 37, 239-253.

Naught101. (2017). Sobol sequence implementation. GitHub repository. https://github.com/naught101/sobol_seq.

Raza M. A. & Liang W. (2012) Uncertainty-based computational robust design optimization of dual-thrust propulsion system, Journal of Engineering Design, 23:8, 618-634, DOI: 10.1080/09544828.2011.636011