Delivery, surveillance, building inspection, and maintenance are just a few of the possible uses of autonomous drones in urban areas. But their use has been limited because of the challenges they face in navigating unstructured, ever-changing city environments. To help drones safely navigate cities, researchers from the University of Zurich, ETH Zurich, and the Polytechnic University of Madrid developed DroNet, a neural network that helps drones react to obstacles and recognize dangerous situations, such as a collision.
“Cities are dynamic. They change continuously, and standard drone navigation approaches cannot handle unforeseen changes,” said study author Antonio Loquercio. “However, cars and bicycles navigate without problems in these environments. So we proposed a data-driven method to teach drones to imitate the way cars and bicycles move around the city.”
Figure 1: Drone in the city
The approach allows drones to reactively predict a steering angle and probability of collision from the image captured by the drone’s camera. These inputs are converted into control flying commands, which enable the UAV to navigate safely.
To build this neural network, the team gathered images online that were captured from a car to analyze steering angles. To help calculate collision probabilities, the researchers mounted a GoPro to the handlebars of a bicycle, and manually took photos as they approached obstacles. Frames that were far away from the obstacle were given the value of 0, as there was no chance of collision, while those close to the obstacle were given the value 1, as there was a higher chance of collision.
Figure 2: Images obtained by researchers
“This ‘zero-shot,’ map-less navigation is a feature that commercial drones still lack. However, this skill is fundamental in all cases where the application space changes frequently, like a city street,” Loquercio explained.
During tests, the researchers found DroNet generalizes well to scenarios that are visually different from the training images. This included flying in indoor environments, including a corridor and a parking ramp, and at a higher altitude with only a slight drop in performance.
Figure 3: Drone flying at higher altitude
In its current iteration, DroNet has two limitations that the researchers are looking to address. First, as the drone uses images taken from cars and bicycles to create steering angles, its agility in flight is restrained. To address this, the team will need to collect 3D collision-free image trajectories and integrate them into the neural network. Second, it’s not possible to give the drone a goal location to be reached. The system is an advanced “lane-follower,” meaning it would decide on its own which direction to take at an intersection. This problem could be solved by implementing GPS into the system.
This novel approach solves for many roadblocks that are preventing drones from taking flight in the city, while also providing a simple and comparatively inexpensive method for data collection. The primary benefit of DroNet is its ability to fly safely by avoiding obstacles and collisions, with no prior knowledge of an environment or reliance on a map. This feature is pivotal in ensuring the safety of civilians in a city. In the future, a system like DroNet could help autonomous vehicles navigate city streets by providing an efficient method for data collection and a novel approach to determine collision probabilities.
For more information on drones visit the IEEE Xplore Digital Library.
