VisAR – Co-Located Shared Augmented Reality for Personalized Low-Vision Training

Loss of visual function significantly affects the ability to perform daily activities independently. This project explores a novel approach to low-vision rehabilitation through the development of a shared augmented reality (S-AR) system. In this system, both the low-vision client and therapist wear head-mounted displays, allowing for real-time observation, guidance, and interaction. The project aims to simulate everyday training tasks in a safe and adaptable environment, supporting individuals in making better use of their residual vision. Conducted in close collaboration with vision rehabilitation centers, this interdisciplinary initiative combines human-computer interaction, clinical insights, and spatial computing to reimagine how visual strategies are trained and supported in practice.

This figure presents a real-world view of the co-located S-AR task between an LV participant and an LV therapist. Both individuals are seated at a table wearing AR headsets. Virtual striped mugs are overlaid on the table in a 5×5 grid pattern. A transparent selection box highlights an additional mug floating above the table, indicating the participant’s intended target. Visual gaze cues are overlaid: a red line shows the participant’s eye gaze, a green line indicates head orientation, and a semi-transparent cone on the table represents the headset’s field of view. A red trace on the table visualizes the participant’s recent eye-gaze path over the past 0.3 seconds. Labels identify the LV participant and the LV therapist.
Illustration of the co-located S-AR task with shared-gaze cues as seen from a therapist- or observer-role headset. The red and green lines represent the projected eye gaze and head gaze of the participant, respectively. The semi-transparent cone on the table surface indicates their FoV. The red trace on the table illustrates the trace of the eye-gaze within 0.3 seconds.

This project is a collaboration between the Departments of Computer Science (Prof. Denis Lalanne), Special Education (Prof. Nicolas Ruffieux), and Psychology (Prof. Roberto Caldara), of the University of Fribourg. The recruitment of patients is in collaboration with the Swiss Federation of the Blind and Visually Impaired, as well as with the Neurorehabilitation and Neurology units of the Fribourg Hospital, and the Neuro-Ophthalmology unit of the Jules-Gonin Eye Hospital.