(119c) Transformer-Based Capacity Prediction for Lithium-Ion Batteries with Data Augmentation

Lithium-ion batteries have become quite integral to the recent technological advancements witnessed in various areas such as transportation, electronics, and clean energy storage [1-2]. To ensure optimal performance and safety in the use of these batteries, capacity estimation is a crucial component in battery management systems to monitor battery health [3]. Reported capacity estimation methods can be classified into two categories: model-based and data-driven approaches [4]. The model-based approach offers precise, physics-based insights and predictive diagnostics, but its complexity and computational demands limit real-time application and broader usability [5]. On the other hand, data-driven approaches such as deep learning methods are model-free and directly utilize existing data to predict capacities [6]. In particular, the transformer architecture, known for its remarkable achievements in natural language processing, shows promising potential for improving the accuracy of capacity estimation [7]. Its ability to process sequential data and recognize long-range dependencies within an input sequence using multi-head attention offers a significant advantage in understanding and predicting the intricate behaviors of battery capacity over time [8].

In this work, we propose to use voltage, current, and temperature data from historical battery charge cycles to predict the capacity of future cycles. We standardize the cycle-specific voltage, current, and temperature readings through down-sampling for consistency in dimensionality. A moving window approach, incorporating this uniform data alongside the capacities of preceding cycles, is employed to forecast the subsequent cycle’s capacity using the transformer model. To preserve the temporal context of each cycle within the sequence, we integrate positional encoding using sine and cosine functions before processing the data through the transformer architecture, which includes elements such as multi-head attention, skip connections, fully connected layers, and a linear output layer. To further enhance the model’s accuracy and adaptability, Gaussian noise is introduced to the input data, effectively augmenting the dataset and improving the model’s capacity prediction capabilities [9].

To validate our method, we utilize data from two distinct battery collections: NASA’s Group 1 battery, consisting of four cells [10], and another four cells from the Michigan battery experiment [11]. All the cell data was collected at room temperature. We train the transformer model to predict the capacity values of the last 60 cycles for each cell while using the data from its previous cycles and all the other cells for training. We use the root mean square error (RMSE) as our evaluation metric. The transformer models based on NASA dataset with augmentation show an averaged RMSE score of 1.14, a significant improvement in performance than models based on transformer without augmentation, LSTM, and CNN (scores are 2.21, 18.14, 23.02, respectively, for these three methods). Also, tests on the Michigan battery data record RMSEs of 1.83 for the augmented transformer, 2.33 for the standard transformer, 25.94 for LSTM, and 27.68 for CNN. This advancement highlights the transformer model’s efficacy in Li-ion battery capacity estimation and the effectiveness of data augmentation in setting a new standard for predictive accuracy and model robustness in this field.

References

[1] Horiba, Tatsuo. "Lithium-ion battery systems." Proceedings of the IEEE 102, no. 6 (2014): 939-950.

[2] Diouf, Boucar, and Ramchandra Pode. "Potential of lithium-ion batteries in renewable energy." Renewable Energy 76 (2015): 375-380.

[3] Zhu, Jiangong, Yixiu Wang, Yuan Huang, R. Bhushan Gopaluni, Yankai Cao, Michael Heere, Martin J. Mühlbauer et al. "Data-driven capacity estimation of commercial lithium-ion batteries from voltage relaxation." Nature Communications 13, no. 1 (2022): 2261.Top of Form

[4] Lucaferri, Valentina, Michele Quercio, Antonino Laudani, and Francesco Riganti Fulginei. "A review on battery model-based and data-driven methods for battery management systems." Energies 16, no. 23 (2023): 7807.

[5] Lai, Xin, Wei Yi, Yifan Cui, Chao Qin, Xuebing Han, Tao Sun, Long Zhou, and Yuejiu Zheng. "Capacity estimation of lithium-ion cells by combining model-based and data-driven methods based on a sequential extended Kalman filter." Energy 216 (2021): 119233.

[6] Zhang, Ming, Dongfang Yang, Jiaxuan Du, Hanlei Sun, Liwei Li, Licheng Wang, and Kai Wang. "A review of SOH prediction of Li-ion batteries based on data-driven algorithms." Energies 16, no. 7 (2023): 3167.

[7] Vaswani, Ashish, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. "Attention is all you need." Advances in Neural Information Processing Systems 30 (2017).

[8] Chen, Daoquan, Weicong Hong, and Xiuze Zhou. "Transformer network for remaining useful life prediction of lithium-ion batteries." IEEE Access 10 (2022): 19621-19628.

[9] Fan, Yuanliang, Han Wu, Weiming Chen, Zeyu Jiang, Xinghua Huang, and Si-Zhe Chen. "A data augmentation method to optimize neural networks for predicting SOH of lithium batteries." In Journal of Physics: Conference Series, vol. 2203, no. 1 (2022): 012034.

[10] Saha, Bhaskar, and Kai Goebel. "Model adaptation for prognostics in a particle filtering framework." International Journal of Prognostics and Health Management Volume 2 (color) 61 (2011).

[11] Mohtat, Peyman, J. Siegel, A. Stefanopoulou, and Suhak Lee. "UofM pouch cell voltage and expansion cyclic aging dataset." University of Michigan—Deep Blue Data (2021).