AI “Perceptive Assistance” During Teleoperations of Rovers in Deep Space

Tue, 01 Oct 2024 00:00:00 +0000

Abstract

Future missions to the Moon and Mars will rely on remotely operated rovers for exploration and construction. Extended-Reality interfaces can help operators understand the rover’s environment, but restricted camera views, communication latency, and the absence of natural perceptual or proprioceptive feedback may reduce situational awareness and performance.

This PhD project investigates AI-based “perceptive assistance” for deep-space rover teleoperation. It combines immersive interaction, human perception, multimodal feedback, state prediction, and human-factors research to design and evaluate assistance adapted to these extreme operating conditions.

Research challenges

Situational awareness — understand how limited fields of view and non-natural head positions affect the operator’s perception of the remote environment.
Communication latency — characterize the perceptual and behavioral effects of Earth–Moon delays, including their possible contribution to cybersickness.
Missing sensory cues — determine which combinations of visual, auditory, haptic, and proprioceptive feedback best compensate for absent or degraded information.
Adaptive assistance — develop AI agents and state-prediction techniques that select or generate effective assistance according to the task and operating conditions.

Methodology and impact

The research combines controlled user studies in simulated Virtual Reality with the development of technical assistance based on sensory compensation and AI. Analogous environments, including underwater teleoperation scenarios, provide experimentally accessible settings for studying constrained perception and remote action.

Beyond space exploration, the results may support teleoperation in other environments affected by degraded communications, limited sensing, or strong physical constraints.

People involved

PhD student

Nazial Kadir

Nazial Kadir is a PhD student at IMT Atlantique and the University of Adelaide within CNRS IRL CROSSING. His research investigates AI-based perceptive assistance for deep-space rover teleoperation and is funded through the MSCA COFUND SEED programme.

Supervisors at IMT Atlantique

Guillaume Moreau

Professor of Computer Science at IMT Atlantique, Brest campus, in France, with research interests in Virtual and Augmented Reality within the Lab-STICC lab. He is also a member of the INRIA Hybrid team and currently serve as Deputy Dean for Research and Innovation at IMT Atlantique.

Gilles Coppin

Gilles Coppin is Professor of Computer Science at IMT Atlantique and member of the INUIT team. His roles involve both teaching and conducting pioneering research, particularly in the areas of human-computer interaction and artificial intelligence

Etienne Peillard

Since August 1, 2026, I have been an Associate Professor in the Department of Automatic Control, Production Engineering and Computer Science (DAPI) at IMT Atlantique, on the Nantes campus. I am a member of the PACCE team at the LS2N (UMR CNRS 6004).

University of Adelaide

The doctoral research is jointly supervised at the University of Adelaide by Anna Ma-Wyatt, whose work addresses human factors, sensorimotor control, and eye movements.

Research environment

The project draws on the immersive-interaction and eye-tracking facilities of IMT Atlantique and the University of Adelaide. It also benefits from the research environment of the CNRS International Research Laboratory CROSSING and the Andy Thomas Space Resource Centre.

Funding

This research project is funded through SEED, IMT Atlantique’s interdisciplinary, international, and intersectoral doctoral programme co-funded by the European Union under the Horizon Europe Marie Skłodowska-Curie Actions COFUND scheme (grant agreement No. 101126644).

Teleoperation | Étienne Peillard