Gait Learning for Hexapod Robot Facing Rough Terrain Based on Dueling-DQN Algorithm

Abstract

In the handling of dangerous goods in explosive environments, robots are increasingly being used instead of human operators. Robots designed for operation in explosive environments are mostly equipped with tracked structures, which, due to their limited terrain adaptability, struggle to movement rugged landscapes. Hexapod robots, with their excellent maneuverability and adaptability, possess advantages in completing hazardous material handling tasks in such rugged terrains. One current challenge lies in enabling hexapod robots to autonomously adjust their gaits to cope with rugged terrain. This paper proposes a gait learning method based on the Dueling Deep Q-Network (Dueling-DQN) algorithm to address the gait adjustment problem of hexapod robots in sloped, terraced, and rugged terrain. The method combines lidar data and reinforcement learning to extract features from the lidar data to determine terrain types and foot coordinates. Finally, the Dueling-DQN algorithm and redundant phase strategy are employed to facilitate the motion of hexapod robots in these three types of rugged terrain. Simulation and prototype experiments are conducted to evaluate the Dueling-DQN algorithm's performance in terms of rewards and stability margins for the three types of terrain. During algorithm training on sloped, terraced, and rugged terrain, stable rewards and positive stability margins are achieved after approximately 490 iterations. The effectiveness and feasibility of the proposed method are further validated through Gazebo simulation and prototype experiments. In the context of movement in rugged terrain within explosive environments, the gait learning method based on the Dueling-DQN algorithm offers valuable insights into the control of hexapod robots.