Progress and Challenges in Applying Deep Reinforcement Learning to Intelligent Navigation

Yuchi Zhuang

doi:10.62051/ijcsit.v3n3.17

Authors

Yuchi Zhuang

DOI:

https://doi.org/10.62051/ijcsit.v3n3.17

Keywords:

Deep reinforcement learning, Intelligent navigation, Path planning, Deep Q network, Policy gradient

Abstract

This article mainly discusses the application methods, progress and challenges of deep reinforcement learning (DRL) in intelligent navigation. With the development of computer and artificial intelligence technology, deep learning and reinforcement learning are combined to form deep reinforcement learning. This method shows significant advantages in processing high-dimensional state spaces and complex decision-making tasks. The article first reviews traditional navigation methods, including simulated annealing methods, artificial potential field methods, and fuzzy logic methods. Then it analyzes graph-based methods such as A* algorithm, probabilistic landmark method, and rapid exploration of random trees, as well as bionic intelligence methods such as Genetic algorithm, artificial neural network, ant colony optimization and particle swarm optimization, etc. Subsequently, the article introduces in detail the basic principles of reinforcement learning and its value-oriented, strategy-oriented and combination-oriented methods, focusing on the specific application of deep reinforcement learning in navigation tasks, such as deep Q network (DQN), deep deterministic strategy gradient (DDPG) and dominant actor-critic (A2C) algorithms. Finally, the article discusses the advantages, challenges and future development directions of deep reinforcement learning in navigation applications, emphasizing the technology's path planning and decision-making optimization capabilities in complex dynamic environments.

Downloads

Download data is not yet available.

References

[1] Kulhánek, J., Derner, E., Bruin, T. D., Babuška, R.: Vision-based Navigation Using Deep Reinforcement Learning. 2019 European Conference on Mobile Robots (ECMR), pp. 1-8 (2019).

[2] Choi, J., Dance, C., Kim, J., Park, K., Han, J., Seo, J., Kim, M.: Fast Adaptation of Deep Reinforcement Learning-Based Navigation Skills to Human Preference. 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 3363-3370 (2020).

[3] Santos, I. B. D. A., Romero, R.: Deep Reinforcement Learning for Visual Semantic Navigation with Memory. 2020 Latin American Robotics Symposium (LARS), 2020 Brazilian Symposium on Robotics (SBR) and 2020 Workshop on Robotics in Education (WRE), pp. 1-6 (2020).

[4] Shi, H., Shi, L., Xu, M., Hwang, K.: End-to-End Navigation Strategy With Deep Reinforcement Learning for Mobile Robots. IEEE Transactions on Industrial Informatics, 16, pp. 2393-2402 (2020).

[5] Bird, J., Arden, D.: Indoor navigation with foot-mounted strapdown inertial navigation and magnetic sensors [Emerging Opportunities for Localization and Tracking]. IEEE Wireless Communications, 18, pp. 28-35 (2011).

[6] Erkeç, T. Y., Haciyev, C.: Traditional Methods on Relative Navigation of Small Satellites. 2019 9th International Conference on Recent Advances in Space Technologies (RAST), pp. 869-874 (2019).

[7] Ping-pin, Z.: Research on Polar Grid Navigation of Great-Circle Sailing. Command Control & Simulation (2015).

[8] Forsman, F., Sjörs-Dahlman, A., Dahlman, J., Falkmer, T., Lee, H. C.: Eye Tracking During High Speed Navigation at Sea. Journal of Transportation Technologies, 2, pp. 277-283 (2012).

[9] Shao, K., Zhao, D., Zhu, Y., Zhang, Q.: Visual Navigation with Actor-Critic Deep Reinforcement Learning. 2018 International Joint Conference on Neural Networks (IJCNN), pp. 1-6 (2018).

[10] Zeng, J., Ju, R., Qin, L., Hu, Y., Yin, Q., Hu, C.: Navigation in Unknown Dynamic Environments Based on Deep Reinforcement Learning. Sensors (Basel, Switzerland), 19 (2019).

[11] Nwaonumah, E. M., Samanta, B.: Deep Reinforcement Learning for Visual Navigation of Wheeled Mobile Robots. 2020 SoutheastCon, pp. 1-8 (2020).

[12] Chen, G., Pan, L., Chen, Y., Xu, P., Wang, Z., Wu, P., Ji, J., Chen, X.: Robot Navigation with Map-Based Deep Reinforcement Learning. 2020 IEEE International Conference on Networking, Sensing and Control (ICNSC), pp. 1-6 (2020).

[13] Hussein, A., Elyan, E., Gaber, M., Jayne, C.: Deep imitation learning for 3D navigation tasks. Neural Computing and Applications, 29, pp. 389-404 (2017).

[14] Shao, K., Zhao, D., Zhu, Y., Zhang, Q.: Visual Navigation with Actor-Critic Deep Reinforcement Learning. 2018 International Joint Conference on Neural Networks (IJCNN), pp. 1-6 (2018).

[15] Kulhánek, J., Derner, E., Bruin, T. D., Babuška, R.: Vision-based Navigation Using Deep Reinforcement Learning. 2019 European Conference on Mobile Robots (ECMR), pp. 1-8 (2019).

[16] Zhu, K., Zhang, T.: Deep reinforcement learning based mobile robot navigation: A review. Tsinghua Science & Technology, 26, pp. 674-691 (2021).

[17] Dhiman, V., Banerjee, S., Griffin, B. A., Siskind, J., Corso, J. J.: A Critical Investigation of Deep Reinforcement Learning for Navigation. ArXiv (2018).

[18] Devo, A., Mezzetti, G., Costante, G., Fravolini, M., Valigi, P.: Towards Generalization in Target-Driven Visual Navigation by Using Deep Reinforcement Learning. IEEE Transactions on Robotics, 36, pp. 1546-1561 (2020).

[19] Lin, C., Hasan, S., Bai, O.: Robotic Navigation with Human Brain Signals and Deep Reinforcement Learning. 2021 4th International Conference on Robotics, Control and Automation Engineering (RCAE), pp. 278-283 (2021).

[20] Zheng, Q., Huang, X., Dong, C., Liu, Y., Chen, D.: An improved deep reinforcement learning for robot navigation. (2023).