Energy Management for Hybrid Energy Storage System in Electric Based on Deep Deterministic Policy Gradient

Shuai Xia; Chun Wang

doi:10.62051/ijcsit.v2n1.22

Authors

Shuai Xia
Chun Wang

DOI:

https://doi.org/10.62051/ijcsit.v2n1.22

Keywords:

Hybrid energy storage system, Energy management, Deep reinforcement learning, DDPG

Abstract

In this paper, an intelligent control system design scheme based on deep deterministic policy gradient (DDPG) algorithm is proposed for the complex continuous action space problem in the hybrid energy storage system of electric vehicles. Firstly, the basic principle and internal logic of DDPG algorithm are introduced, including key elements such as Actor-Critic architecture, experience playback, target network, reward signal, policy gradient and value function update. Then, how to apply the DDPG algorithm to the industrial control system is described in detail. The Actor network learns the optimal strategy, the Critic network evaluates the value of the state-action pair, and uses the experience playback and the target network to improve the system stability and performance. Finally, the effect of the intelligent control system based on DDPG algorithm in complex environment is verified by simulation experiments. The results show that the system can effectively optimize the control strategy, improve the response speed and stability of the system, and has a good engineering application prospect.

Downloads

Download data is not yet available.

References

Y.G. Liu, J.J. Liu, Y.J. Zhang, et al. Rule learning based energy management strategy of fuel cell hybrid vehicles considering multi-objective optimization[J]. Energy, 2020, 207:118212.

R. Rodriguez, J.P.F. Trovão, J. Solano. Fuzzy logic-model predictive control energy management strategy for a dual-mode locomotive[J]. Energy Conversion and Management, 2022, 253:115111.

Singh K V, Bansal H O and Singh D. Fuzzy logic and Elman neural network tuned energy management strategies for a power-split HEVs [J]. Energy, 2021. 225.

[26] Zhang Q, Wang L, Li G. A real-time energy management control strategy for battery and supercapacitor hybrid energy storage systems of pure electric vehicles [J]. Journal of Energy Storage, 2020. 31.

[27] Rahman A U, Zehra S S, Ahmad I. Fuzzy supertwisting sliding mode-based energy management and control of hybrid energy storage system in electric vehicle considering fuel economy [J]. Journal of Energy Storage, 2021. 37.

A. Prasanthi, H. Shareef, M. Asna, et al. Optimization of hybrid energy systems and adaptive energy management for hybrid electric vehicles[J]. Energy Conversion and Management, 2021, 243:114357.

Q. Zhang, L.J. Wang, G. Li, et al. A real-time energy management control strategy for battery and supercapacitor hybrid energy storage systems of pure electric vehicles[J]. Journal of Energy Storage, 2020, 31:101721.

W.H. Li, H. Cui, T. Nemeth, et al. Cloud-based health-conscious energy management of hybrid battery systems in electric vehicles with deep reinforcement learning[J]. Applied Energy, 2021, 293:116977.

H.E. He, J.F. Cao, X. Cui. Energy optimization of electric vehicle’s acceleration process based on reinforcement learning[J]. Journal of Cleaner Production, 2020, 248:119302.

Y.C. Li, H.W. He, A. Khajepour, et al. Energy management for a power-split hybrid electric bus via deep reinforcement learning with terrain information[J]. Applied Energy, 2019, 255:113762.

D.Z. Xu, Y.D. Cui, J.Y. Ye, et al. A soft actor-critic-based energy management strategy for electric vehicles with hybrid energy storage systems[J]. Journal of Power Sources, 2022, 524:231099.

W.H. Li, H. Cui, T. Nemeth, et al. Deep reinforcement learning-based energy management of hybrid battery systems in electric vehicles[J]. Journal of Energy Storage, 2021, 36:102355.

P. Wu, J. Partridge, E. Anderlini et al. Near-optimal energy management for plug-in hybrid fuel cell and battery propulsion using deep reinforcement learning[J]. International Journal of Hydrogen Energy, 2021, 46(80):40022-40040.

Lian R, Peng J, Wu Y. Rule-interposing deep reinforcement learning based energy management strategy for power-split hybrid electric vehicle [J]. Energy, 2020. 197.