Avatars and the augmented self in Augmented Reality
Augmented Self: TowaRds effective Avatars in augmented reaLity
Augmented Self: TowaRds effective Avatars in augmented reaLity
ANR ASTRAL (2024–2027) is a collaborative project bringing together IMT Atlantique, Inria Rennes, Centrale Nantes, and Clarté. At IMT Atlantique, it combines a reusable real-time platform for changing the visible body with experimental research on how virtual hands reshape perception and action in the physical world.
At a glance
Avatars can improve presence and performance in immersive environments and can alter users’ self-perception and behavior. Yet they remain less common in Augmented Reality (AR), where users can often see their physical body directly. ASTRAL investigates how to create compelling AR avatars and how those avatars affect users.
The project is organized around three complementary scientific axes:
- Technology — synthesizing and displaying avatars in both video see-through and optical see-through AR systems.
- Interaction — enabling users to act through an avatar while accounting for differences between the physical and virtual body and environment.
- Perception — studying embodiment, self-perception, self-confidence, performance, and the psychological and social effects of AR avatars.
Potential applications range from physical rehabilitation to sports-performance enhancement. The project combines Augmented Reality, computer vision, 3D interaction, avatar synthesis, embodied interaction, and perceptual evaluation.
A platform to experiment with the augmented body
ISMAR 2025 · Demo / poster
Hands Off, Avatars On
The technical backbone of the IMT Atlantique contribution is a modular, real-time platform for replacing the user’s hands in egocentric video see-through AR. Developed by Aymeric Hénard and Titouan Aline, it makes visual body modification a configurable research instrument rather than a one-off demonstration.
Six independently replaceable modules cover video acquisition, hand tracking and pose estimation, segmentation, hand removal, avatar animation, and final rendering. Their shared-memory architecture keeps the pipeline responsive while allowing each component to be exchanged or improved as an experiment demands.
The platform was presented at ISMAR 2025 as Hands Off, Avatars On: Towards a Modular Approach for Real-Time Body Modification in AR. It provides the practical foundation for controlled studies of embodiment, body image, and interaction—today at the hand level, with the architecture designed to grow toward more complete body transformations.
Featured publication
IEEE ISMAR-Adjunct · 2025
The platform turns real-time visual hand replacement into a configurable experimental apparatus, linking tracking, segmentation, diminished reality, and avatar rendering.
Doctoral research: Juri Yoneyama
PhD · October 2024 – September 2027
Juri Yoneyama investigates what happens when people alternate between physical and augmented hands and objects. Her thesis treats AR interaction as a continuous sensorimotor situation: virtual actions do not stop at the boundary of the display, but can carry over into the next physical action.
The work develops this question through a progressive experimental programme. An initial pick-and-place study showed that the physicality of the object is a primary driver of performance: real objects were handled faster than virtual or composite ones, whereas changing the appearance of the hand had comparatively little effect. Crucially, interacting with virtual objects also degraded the following interaction with a real object. This result motivates the thesis’ focus on the transition—not only on each interaction in isolation.
Juri’s subsequent studies examine this transition at two complementary scales:
- In Grasping Real and Virtual Objects in AR, she uses the opening and timing of the fingers before contact to measure grasp uncertainty. The study tests how pinch and physics-based virtual grasping, together with object size, shape the next real grasp.
- In Interacting with Two Dominant Hands in AR, she studies switching between a real hand and a spatially displaced virtual counterpart. Participants can generally select the appropriate hand for the task; the detailed measures reveal where hand appearance and the direction of the transition still affect movement initiation, trajectory, and embodiment.
Together, these studies make the platform and the thesis mutually productive: the platform enables precise manipulations of the visible body; Juri’s experiments reveal how those manipulations matter for perception, motor control, and the design of future AR interactions. The next stage investigates visual interaction cues that can reduce uncertainty when tactile feedback is absent from virtual-object interaction.
Research media
Project team
At IMT Atlantique, the platform work and Juri’s doctoral research form the project’s central thread. The official ASTRAL project page presents the full consortium and its partner institutions.
Doctoral research
Juri Yoneyama
Juri’s thesis is the main experimental thread at IMT Atlantique. Co-supervised by Étienne Peillard, Guillaume Moreau, and Ferran Argelaguet, it examines how virtual and physical hands and objects influence one another when people switch between augmented and real actions.
- identifies object physicality and cross-modal transitions as critical factors in mixed-object performance;
- measures carry-over effects through anticipatory grip opening and its timing; and
- characterises switching with a spatially displaced virtual hand through movement, proprioception, and embodiment measures.
Doctoral supervision
Étienne Peillard
Contributes the perspective of body perception and embodied interaction, and co-supervises the doctoral research.
Guillaume Moreau
Contributes expertise in virtual and augmented reality systems, and co-supervises the doctoral research.
Ferran Argelaguet
Co-supervises Juri’s doctoral research and contributes expertise in immersive interaction, embodiment, and perceptual evaluation.
Platform engineering
Aymeric Hénard
Leads the implementation of the modular platform for real-time visual body modification, from video acquisition to avatar rendering.
Research internship
Titouan Aline
Co-developed the modular pipeline for real-time hand replacement in video see-through augmented reality.
Consortium
ASTRAL is a multidisciplinary consortium coordinated by Clarté, with IMT Atlantique / Lab-STICC, Centrale Nantes / LS2N PACCE, and Inria Rennes / Hybrid. The partners bring complementary expertise in AR, computer vision, 3D interaction, avatar synthesis, embodiment, and immersive-system evaluation.
Funding
ASTRAL is funded by the French National Research Agency under reference ANR-23-CE33-0004. The project started in February 2024, runs for 42 months, and received an ANR grant of €613,639.