Making robot swarms legible with Augmented Reality
ARTUISIS - Augmented Reality and Tangible User Interface to Supervise and Interact with robot Swarms
ANR-funded project (ANR-21-CE33-0006), coordinated by Jérémy Rivière, October 2021–October 2025.
At a glance
ARTUISIS asks a deceptively simple question: how can a person understand, monitor, and influence a swarm of autonomous robots without losing sight of its collective dynamics? The project brings together swarm robotics, tangible interaction, and Augmented Reality (AR) to make the local mechanisms behind a collective behaviour visible where they matter: around the swarm itself.
The scientific direction and the swarm-robotics focus were led by Jérémy Rivière. My contribution was to co-supervise Aymeric Hénard’s doctoral work and to focus on the AR visualisation strand: identifying which information a human operator needs, designing spatially localised visual cues, and evaluating whether they help people anticipate and prevent a loss of swarm cohesion.
From local rules to an intelligible collective
Robot swarms are decentralised systems made of many small, autonomous agents. Their collective behaviours—such as flocking, aggregation, expansion, coverage, or shape formation—emerge from local interactions rather than from a central controller. This makes swarms robust and scalable, but also difficult to read: the operator sees the global motion without necessarily seeing the forces, neighbourhood relations, or changes that produced it.
ARTUISIS addresses this gap through three complementary goals:
- Explain self-organisation by relating visible collective behaviours to their underlying local mechanisms.
- Support supervision by communicating the dynamics of the swarm in real time.
- Enable informed action through intuitive tangible interaction with the swarm’s spatial organisation.
The project therefore pairs Jérémy Rivière’s work on robot-swarm self-organisation with a human-centred question: what should be made visible, in what form, and at what location, for an operator to build a useful mental model of a dynamic distributed system?
Augmented visualisation for swarm monitoring
The AR work explored situated, localised visualisations: graphical cues registered to robots and to their immediate spatial organisation instead of a separate, abstract control panel. The aim is not to display every data stream, but to reveal the information that is otherwise hard to infer from robot trajectories alone—for example, inter-robot links, local virtual forces, direction and velocity, the convex envelope of the group, and signals of an emerging fragmentation.
This approach turns the physical workspace into an explanatory view of the system. It can be used in simulation and with tracked physical robots, allowing an operator to inspect swarm structure while retaining awareness of the real environment. The visualisation research was driven by a perceptual evaluation loop:
- establish what people can already perceive from a swarm’s motion;
- identify critical situations where perception is insufficient, notably a pending fragmentation;
- design visual cues that make the relevant mechanisms perceptually available; and
- test whether those cues improve monitoring and support an appropriate intervention.
The videos below illustrate this principle. The overlays make relationships and local forces visible as the swarm changes its collective organisation.
Featured publication
IEEE Transactions on Visualization and Computer Graphics · 2025
The project’s main AR result evaluates visual cues that help an operator monitor swarm cohesion and anticipate fragmentation.
Doctoral research: Aymeric Hénard
Aymeric Hénard completed a PhD in the ARTUISIS project from October 2021 to September 2024, co-supervised by Gilles Coppin, Jérémy Rivière, Sébastien Kubicki, and Étienne Peillard. His thesis, “Augmented Reality and Tangible Interaction to Supervise and Interact with a Swarm of Robots”, was defended on 5 December 2024.
The thesis established a coherent progression from the foundations of swarm self-organisation to operator support. It contributed a classification of spatial self-organisation methods, studied how observers perceive and anticipate swarm fragmentation, and developed and evaluated augmented visualisations intended to make the dynamics of a swarm more understandable. This work provides a basis for AR interfaces that assist a human operator without replacing the swarm’s decentralised autonomy.
Project team
ARTUISIS brought together complementary expertise in swarm robotics, human factors, tangible interaction, and Augmented Reality. The project was coordinated by Jérémy Rivière, with Aymeric Hénard’s doctoral research forming its main scientific thread.
Jérémy Rivière
Scientific lead of ARTUISIS, with a focus on robot-swarm self-organisation: the local mechanisms that produce collective behaviours and the ways an operator can understand and influence them.
Aymeric Hénard
Aymeric’s doctoral research formed the main scientific thread of ARTUISIS, connecting the swarm’s micro-mechanisms with the operator’s ability to perceive, understand, and supervise its collective behaviour.
- classified the methods that produce spatial self-organisation in robot swarms;
- characterised human perception and anticipation of swarm fragmentation; and
- designed and evaluated localised visual cues for AR swarm monitoring.
Étienne Peillard
Contributed the situated-visualisation perspective: selecting, designing, and evaluating AR cues that make swarm dynamics perceptible to an operator.
Gilles Coppin
Brought expertise in human–computer interaction and the human factors of supervising autonomous systems.
Sébastien Kubicki
Contributed expertise in natural human–system interaction and tangible interfaces for representing and acting on swarm spatiality.
Research engineeringJérémie DonjatProject development and technical support
Student contributions
Nolwenn Paluet
Developed the AR prototype for immersive, localised swarm-data visualisation. Her work connected the headset to the swarm gateway and introduced cues such as robot direction vectors and the swarm’s convex envelope, in both simulated and motion-capture environments.
Tristan Guichaoua
Set up the tracking and virtual-perception environment for experiments with a MONA robot swarm, providing the technical foundation for observing robot motion and testing behavioural algorithms.
Visualising self-organisation
The visual material follows the same thread as the research: first observe an emergent behaviour, then expose the local information that helps explain it.
Collective behaviours in simulation
Augmented cues around the physical swarm
Funding
ARTUISIS was funded by the French National Research Agency (ANR) under reference ANR-21-CE33-0006.
