Creating an Adaptive and Intuitive Single-Operator Multi-MAV Interface Using Augmented Reality

The goal of this research is to achieve safe and effective singular operator supervision and control of multi-micro aerial vehicle (MAV) swarms with an intuitive and easy-to-use Augmented Reality interface. Singular-operator swarm control is applicable to multiple industries and scenarios [2]. MAVs currently navigate pre-defined paths and accomplish given tasks [1]. However, there is a loss of efficiency, as a single operator must have programming knowledge & manage multiple systems. Achieving singular operator control software would allow for users to spatially task the swarm within various environments, preventing conflicts due to errors in other operators/software. The design of the augmented reality interface was done using 3D interactable objects that are mapped to the positions of the MAVs in the 3D-space. Path-planning and execution is done within an AR headset by moving 3D objects to draw the desired path each drone should follow. This is accompanied by a user interface (UI) orbiting the user with options to delete paths, upload the paths to the swarm, emergency shutoff and to execute/stop.

This project will be accomplished by integrating existing software and hardware, such as the Robot Operating System (ROS), Crazyswarm [1], Optitrack motion-capture, HoloLens 2[5] as well as the crazyflie’s Micro-Aerial-Vehicle (MAV) [6] and creating new software with the above capabilities. Control of the swarm through this augmented reality interface has been achieved through the above method using HoloLens 2 and the execution of software on a ROS-receiving PC. Further improvements in terms of safety features such as environmental obstacle avoidance, better safety features, accessibility, and integrating more of Crazyswarm’s controls, are in progress. My findings show that AR as a means of controlling MAV swarms is viable in an intuitive manner that is easy to utilize for novice users. Additionally, this project allows for further studies into human-robot interaction with MAV swarms & augmented reality.