Autism: children achieve greater autonomy with virtual and augmented reality

The work explored innovative technological solutions to improve the autonomy of young people with autism spectrum disorders (ASD) through the use of virtual reality, augmented reality and conversational agents, focusing on urban mobility as the primary context of investigation.

Virtual Reality allows people to practise using public transport by 'immersing' themselves, using a headset, in a digital environment that simulates spaces and activities typically found when using the train and the underground.

The 5A Augmented Reality applications support users while using public transport in the real world by generating, on tablets or smartphones, visual information that is superimposed over the view of the surroundings and helps people understand how to get around and what to do. Both applications integrate a Conversational Agent who acts as a virtual companion and proactively dialogues with the user to guide them during both the simulation of public transport use and the real-world experience.

The 5A applications were co-designed by a multidisciplinary team consisting of engineers and interaction designers from the Politecnico di Milano, and autism specialists from the two clinical partners – Fondazione Sacra Famiglia and IRCCS E. Medea – Associazione La Nostra Famiglia. A rigorous trial was conducted at these partner centres, involving 27 teenagers with autism spectrum disorders and eight therapists, in order to evaluate empirically the usability and effectiveness of the 5A tools.

“The goal of 5A is to make young people with ASD as autonomous as possible in everyday life. For now, the technology we have created seems to really help them to use public transport more safely and consciously. In the future, we would like to help people with ASD to deal with complexity also in other contexts, e.g., admission to hospital, visiting museums, navigating large shopping centres,” explains Franca Garzotto, Professor of Information Processing Systems at the Politecnico di Milano and Scientific Coordinator of 5A, “A learning and support process that starts with 'training' in the virtual world, to be carried out at home, at school or at a therapy centre, and through augmented reality is extended to become a support tool that is contextualised in space and time, helping the subject at a specific time and in a specific place.” 

“As a research group dealing with innovative methodologies serving the needs of people with neurodevelopmental disorders, we truly valued the collaboration within the 5A project,” says Maria Luisa Lorusso, Head of the Unit of Neuropsychology of Neurodevelopmental Disorders at IRCCS Medea – La Nostra Famiglia. “The work and the interdisciplinary exchange were very stimulating and, in our opinion, allowed us to explore new horizons for a technology that is increasingly close to the needs of everyday life and increasingly ready to respond to people's desire to empower themselves, acquire new skills and decide on their own way of life and education.”

"The research involved young patients of ours, with the aim of increasing their autonomy in everyday activities,” says Monica Conti, Director of the Innovative Autism Services at Fondazione Sacra Famiglia. “For the young people involved, participation had positive effects on their self-esteem and personal motivation because they felt chosen to take part in a project whose development and success depended on their contribution. Their caregivers were also enthusiastic about the project, motivated by the desire to increase the opportunities for their children to enjoy experiences that are a source of personal well-being and satisfaction.

This project is in line with the commitment of the Foundation, which has always experimented with innovative ideas aimed at offering autistic people the best opportunities for growth and independence.”

The project was undertaken with the support of Fondazione TIM, chaired by Salvatore Rossi and led by General Manager, Giorgia Floriani. In particular, Fondazione TIM is an expression of the TIM Group's social commitment to promoting the culture of change and digital innovation, fostering inclusion, communication, and economic and social growth.

At the end of two years of intensive and exciting work, the final 5A project event presented and discussed the results achieved, with demonstrations of the 5A applications open for the event participants to try.