Powered Landing Guidance Algorithms Using Reinforcement Learning Methods for Lunar Lander Case

Any future planetary landing missions, just as demonstrated by Perseverance in 2021 Mars landing missions require advanced guidance, navigation, and control algorithms for the powered landing phase of the spacecraft to touch down a designated target with pinpoint accuracy (circular error precision < 5 m radius). This requires a landing system capable to estimate the craft’s states and map them to certain thrust commands for each craft’s engine. Reinforcement learning theory is used as an approach to manage the mapping guidance algorithm and translate it to engine thrust control commands. This work compares several reinforcement-learning-based approaches for a powered landing problem of a spacecraft in a two-dimensional (2-D) environment and identifies their advantages or disadvantages. Three methods in reinforcement learning, namely Q-Learning, and its extensions such as DQN and DDQN. They are benchmarked in terms of rewards and training time needed to land the Lunar Lander. It is found that the Q-Learning method also called Heuristic produced the highest efficiency. Another contribution of this paper is to show the combination usage of online weights � between the action selection process and action evaluation process, yields a higher reward, instead of separating them, which significantly enhances their optimization performance. The simulations of the powered guidance performance in a 2-D simulation environment highlight the effectiveness of DQN to handle multiple neural networks better than DDQN.