(203i) A Principal Component Normalizing Flow for Modeling Renewable Electricity Generation

Wind and solar power depend on natural occurrences and are therefore volatile and uncertain. The inherent uncertainty of renewable electricity sources must be considered in decision-making such as design and operation of energy systems, e.g., through stochastic programming (SP) based on discrete scenarios [1-3]. In SP, the scenarios must accurately represent the underlying distributions of the stochastic processes to avoid skewed or infeasible decisions. Traditional scenario generation approaches rely on Gaussian Copulas [3] and autoregressive moving average (ARIMA) models [2-4]. Recent advances in deep learning have enabled scenario generation without prior assumptions using neural autoregressive models (NARX) [5] and deep generative models, e.g., generative adversarial networks (GAN) [6] or normalizing flows [7]. In contrast to NARXs and GANs, normalizing flows model the target distribution explicitly using a nonlinear transformation of a multivariate Gaussian based on invertible neural networks. The inverse of that transformation models the probability density function directly and enables training by log-likelihood maximization [8]. However, recent research from the field of image generation has shown that standard normalizing flows are unable to model manifold distributions and instead assign non-zero probabilities to out-of-distribution data, which results in the generation of noisy data [9]. We find that energy time series often reside on manifolds with a much lower dimensionality than the number of time steps due to the correlation between time steps and that standard normalizing flows should therefore not be used for scenario generation of renewable electricity generation [10]. To maintain the direct likelihood maximization but avoid the generation of noisy data, we propose a dimensionality reducing normalizing flow layer based on the principal component analysis (PCA) [10]. The PCA sets up the normalizing flow in a lower-dimensional space and excludes the space outside of the manifold, i.e., the space of zero probability. We apply the resulting principal component flow (PCF) for scenario generation of solar and wind electricity generation based on historical data from Germany in the years 2013 to 2015. A comparison between standard normalizing flows and the PCF shows a significant improvement in the distribution fit and the representation of the frequency behavior as a result of the dimensionality reduction.

References:

[1] A. Mitsos, N. Asprion, C. A. Floudas, M. Bortz, M. Baldea, D. Bonvin, A. Caspari, P. Schäfer, Challenges in process optimization for new feed-stocks and energy sources, Computers & Chemical Engineering 113 (2018) 209–221

[2] Schäfer, P., Westerholt, H. G., Schweidtmann, A. M., Ilieva, S., & Mitsos, A. (2019). Model-based bidding strategies on the primary balancing market for energy-intense processes. Computers & Chemical Engineering, 120, 4-14.

[3] J. M. Morales, A. J. Conejo, H. Madsen, P. Pinson, M. Zugno, Integrating Renewables in Electricity Markets: Operational Problems, Vol. 205, Springer Science & Business Media, 2013.

[4] Ierapetritou, M. G., Wu, D., Vin, J., Sweeney, P., & Chigirinskiy, M. (2002). Cost minimization in an energy-intensive plant using mathematical programming approaches. Industrial & engineering chemistry research, 41(21), 5262-5277.

[5] S. I. Vagropoulos, E. G. Kardakos, C. K. Simoglou, A. G. Bakirtzis, J. P. Catalao, Ann-based scenario generation methodology for stochastic variables of electric power systems, Electric Power Systems Research 134 (2016) 9–18.

[6] Y. Chen, Y. Wang, D. Kirschen, B. Zhang, Model-free renewable scenario generation using generative adversarial networks, IEEE Transactions on Power Systems 33 (3) (2018) 3265

[7] L. Zhang, B. Zhang, Scenario forecasting of residential load profiles, IEEE Journal on Selected Areas in Communications 38 (1) (2020) 84–9

[8] G. Papamakarios, E. Nalisnick, D. J. Rezende, S. Mohamed, B. Lakshminarayanan, Normalizing flows for probabilistic modeling and inference, arXiv preprint arXiv:1912.02762, 2021

[9] J. Brehmer, K. Cranmer, Flows for simultaneous manifold learning and density estimation, arXiv preprint arXiv:2003.1391, 2020

[10] E. Cramer., A. Mitsos, R. Tempone, and M. Dahmen. “Principal component density estimation for scenario generation using normalizing flows”, Manuscript in preparation, 2021.