Overview
On June 11, Prof. Fei Gao's team at Zhejiang University published a paper titled "Precise Aggressive Aerial Maneuvers with Sensorimotor Policies" in Science Robotics.
The study demonstrates how a quadrotor can perform aggressive maneuvers through narrow tilted gaps using only onboard sensing, without external localization or trajectory planning.
During the experimental validation, the NOKOV optical motion capture system provided high-precision state estimation, supporting hardware-in-the-loop (HIL) testing, synthetic gap generation, and flight performance evaluation.
Conventional UAV gap-traversal methods typically rely on multiple modules such as state estimation, trajectory planning, and control. Information loss and accumulated errors between modules can significantly affect performance in highly constrained flight scenarios.
Inspired by biological sensorimotor coordination, the researchers developed an end-to-end sensorimotor policy that directly maps visual and inertial observations to flight control commands, enabling perception-to-control decision making without explicit trajectory planning.
Working Principle and System Implementation
In this study, the NOKOV optical motion capture system served as the ground-truth sensing infrastructure for UAV experiments.
Its primary functions included:
Providing high-precision quadrotor state estimation for HIL testing
Supporting synthetic gap image generation for policy validation
Recording ground-truth flight trajectories for quantitative evaluation
Delivering full-state feedback for baseline closed-loop control experiments
By supplying accurate ground-truth motion data, the system enabled safe, repeatable, and measurable validation of UAV perception, control, and sim-to-real transfer performance.
Tianyue Wu et al., Precise aggressive aerial maneuvers with sensorimotor policies. Sci. Robot. 11, eaeb0180 (2026). DOI: 10.1126/scirobotics.aeb0180
The research team developed a custom quadrotor platform equipped with:
Monocular camera
PX4 flight controller
NVIDIA Jetson Orin NX
For hardware-in-the-loop testing, the quadrotor executed the learned policy while synthetic gap images were generated using fused onboard IMU data and NOKOV motion capture measurements. This approach reduced collision risks while maintaining realistic flight dynamics and control conditions.
The experiments included:
Traversal through tilted gaps with unknown orientations
Reactive flight through dynamic gaps
Consecutive narrow-gap traversal
Traversal through openings with different geometries
Flight under diverse visual backgrounds
The video demonstrates: (1) real-world traversal through tilted gaps with previously unknown orientations; (2) reactive traversal through dynamic gaps; (3) traversal through closely placed consecutive gaps; (4) traversal through geometrically diverse openings; and (5) traversal through narrow gaps with visually diverse backgrounds.
Policy Learning and Sim-to-Real Transfer
The study used informed reset (IR) to improve RL exploration efficiency, together with perturbation force, response randomization, and delay randomization to reduce the sim-to-real performance gap. IR made it easier for the agent to enter feasible solution regions, increasing the average success rate for the single-gap task from about 70% to 96% and helping the policy escape local optima in consecutive-gap traversal. Randomization substantially improved the distilled vision-based policy's robustness to dynamics discrepancies and observation noise.
Performance Evaluation with Reference Baseline Systems
Baseline 1 [1] used a known gap position and pose and relied on the NOKOV optical motion capture system to provide real-time full-state feedback for closed-loop quadrotor control and reference trajectory tracking. This setup enables precise closed-loop trajectory tracking under privileged state information from motion capture systems.
Baseline 2 [2] achieved narrow-gap traversal through vision-inertial fusion, pose estimation, and rapid online replanning.
The experiments showed that, despite operating with indirect visual observations and no external localization, the proposed vision-based policy achieved success rates comparable to trajectory-tracking methods with privileged information, while alleviating explicit error propagation and model mismatch.
Key Takeaways
End-to-end sensorimotor policy for aggressive UAV flight
High-speed traversal through narrow tilted gaps using onboard sensing only
Hardware-in-the-loop validation supported by NOKOV motion capture
Ground-truth trajectory tracking and performance evaluation
High-precision motion capture for UAV state estimation and sim-to-real validation
References
[1] Z. Wang, X. Zhou, C. Xu, F. Gao, Geometrically constrained trajectory optimization for multicopters. IEEE Trans. Robot. 38, 3259–3278 (2022).
[2] D. Falanga, E. Mueggler, M. Faessler, D. Scaramuzza, “Aggressive quadrotor flight through narrow gaps with onboard sensing and computing using active vision” in Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA) (IEEE, 2017), pp. 5774–5781.
Author Profiles
Tianyue Wu (Co-first author) received his bachelor's and master's degrees from Zhejiang University and will pursue a Ph.D. at the University of Hong Kong. His research interests include dexterous manipulation intelligence for robotic hands and humanoid robots, as well as motion intelligence for aerial robots. He has published first-author papers in leading robotics journals and conferences, including Science Robotics and Robotics: Science and Systems.
Guangtong Xu (Co-first author) is an associate research professor at the School of Automation, Hangzhou Dianzi University. He received his bachelor's and Ph.D. degrees from Beijing Institute of Technology in 2015 and 2021, respectively. From 2021 to 2023, he conducted postdoctoral research in the Department of Precision Instrument at Tsinghua University. From 2023 to 2026, he worked at the Huzhou Institute of Zhejiang University, where this study was completed. His research focuses on autonomous navigation, motion planning, and swarm coordination for aerial robots. He has published papers in T-ASE, RA-L, ICRA, IROS, and other robotics journals and conferences.
Zihan Wang holds a master's degree in engineering from North China Electric Power University and was a visiting student at the Huzhou Institute of Zhejiang University during this study. His research focuses on autonomous robotic systems and decision-making and planning. One of his research works has been accepted by IROS.
Junxiao Lin holds a master's degree from Zhejiang University and will pursue a Ph.D. at the University of Hong Kong. His research focuses on robotic system design, motion control, and dexterous manipulation. He has published several papers in leading robotics journals and conferences, including RA-L and IROS.
Tianyang Chen is a master's student at the College of Control Science and Engineering, Zhejiang University. His research focuses on aerial robots and vision-language-action models.
Yuze Wu received his Ph.D. from Zhejiang University and is a young scientist at Differential Robotics Technology. He has published nearly 20 papers, including work in Nature Communications, Science Robotics, and IEEE Transactions on Robotics. His research focuses on VLA/VLN embodied navigation models, drone world models, and robotic multi-agent systems.
Zhichao Han received his Ph.D. from the College of Control Science and Engineering, Zhejiang University. His research focuses on motion planning for mobile robots, nonlinear optimization, deep learning, and reinforcement learning.
Zhiyang Liu received his Ph.D. from Zhejiang University and is the COO of Differential Robotics Technology.
Fei Gao (Corresponding author) is a tenured associate professor and doctoral supervisor at the College of Control Science and Engineering, Zhejiang University, and the founder of Differential Robotics Technology. His research interests include aerial robotics, autonomous navigation, motion planning, environmental perception, and swarm robotics.
Room820, China Minmetals Tower, Chaoyang Dist., Beijing
info@nokov.cn
+ 86-10-64922321
