Capturing Motion,
Crafting Stories

This work is accepted by ICRA 2024 and be selected as the finalist of three awards:

• the IEEE ICRA Best Paper Award on Unmanned Aerial Vehicles
• the IEEE ICRA Best Conference Paper Award
• the IEEE ICRA Best Student Paper Award

Designing such a drone racing flight assistance system faces several challenges. First, novice pilots lack experience, have poor maneuverability, and require guidance from experts. Second, the unpredictable nature of novice pilots' operations poses significant safety challenges. Third, it is essential to ensure that beginner pilots maintain a strong sense of participation.

In academia, a relatively mature solution for fully autonomous drone racing already exists. Without human involvement, drones rely solely on their computational platforms to achieve flight performance comparable to professional pilots, oftentimes surpassing them. However, with this solution, a pilot is not involved with controlling the drone. This paper aims to design a flight assistance system that not only enhances the performance of drone racing, but also provides human pilots with a fully immersive racing experience. This requires accurately capturing the pilot’s intent and providing real-time feedback to the drone's movements. From this perspective, assisted flight control is more challenging than fully autonomous flight. 

System framework

This system adopts a layered structure consisting of offline and online components.

1. Offline preprocessing module

Firstly, the point cloud map of the racing venue and flight path is constructed. A relatively mature global trajectory planning method is used to generate the fastest path as an expert reference. Along this reference trajectory, a dense safe flight corridor is generated to provide a sufficiently large safe area, representing safe airspace using a series of interconnected polyhedra.

2. Online replanning module

A remote-control mapping primitive is designed to quickly capture the pilot's input, and a time-based trajectory progress is customized to further capture intent. Then, a trajectory planner is used to periodically generate trajectories that are safe, smooth, feasible, and consistent with the pilot’s intent.

3. Add yaw angle replanning

The yaw angle is finally replanned on the generated trajectory. This helps the drone turn quickly towards the next target point, providing the pilot with the best perspective and reducing the difficulty of operation. This is particularly important for novice pilots.

A diagram of our flight assistive system architecture.

Experimental validation

Simulations and real-world experiments were conducted to validate the performance of the proposed system. In the real-world experiment, five participants were invited to pilot drones in a drone racing scenario. The completion time was recorded, and the final rankings were determined.

During the competition, the NOKOV motion capture system was used to obtain relevant drone position information to verify the flight trajectories of the drones during the race.

All pilot participants successfully completed the race in the real-world scenario, with speeds exceeding 3.0 m/s. The flight trajectories of all pilots and snapshots of drone flights are shown in the following figure. With the flight assistance system proposed in this paper, even novice pilots were able to complete the race successfully.

The winning pilot achieved a maximum flight speed of 6.0 m/s, with an average flight speed close to 3.0 m/s.