Machine learning network offers personalised autism therapy

This type of therapy works best, however, if the robot can smoothly interpret the child’s own behaviour — whether he or she is interested and excited or paying attention — during the therapy.

Researchers at the MIT Media Lab have now developed a type of personalised machine learning that helps robots estimate the engagement and interest of each child during these interactions, using data that are unique to that child.

Armed with this personalised 'deep learning' network, the robots’ perception of the children’s responses agreed with assessments by human experts, with a correlation score of 60 percent, the scientists report in Science Robotics.

It can be challenging for human observers to reach high levels of agreement about a child’s engagement and behaviour. Their correlation scores are usually between 50% and 55%. Rudovic and his colleagues suggest that robots that are trained on human observations, as in this study, could someday provide more consistent estimates of these behaviours.

“The long-term goal is not to create robots that will replace human therapists, but to augment them with key information that the therapists can use to personalise the therapy content and also make more engaging and naturalistic interactions between the robots and children with autism,” explained Oggi Rudovic, a postdoc at the Media Lab and first author of the study.

Rosalind Picard, a co-author on the paper and professor at MIT who leads research in affective computing, said that personalisation is especially important in autism therapy: A famous adage is, “If you have met one person, with autism, you have met one person with autism.”

“The challenge of creating machine learning and AI [artificial intelligence] that works in autism is particularly vexing, because the usual AI methods require a lot of data that are similar for each category that is learned. In autism where heterogeneity reigns, the normal AI approaches fail,” said Picard.

Rudovic, Picard, and their teammates have also been using personalised deep learning in other areas, finding that it improves results for pain monitoring and for forecasting Alzheimer’s disease progression.

Robot-assisted therapy for autism often works something like this: A human therapist shows a child photos or flash cards of different faces meant to represent different emotions, to teach them how to recognise expressions of fear, sadness, or joy.

The therapist then programs the robot to show these same emotions to the child, and observes the child as she or he engages with the robot. The child’s behaviour provides valuable feedback that the robot and therapist need to go forward with the lesson.

The researchers used SoftBank Robotics NAO humanoid robots in this study. Almost 2 feet tall and resembling an armored superhero or a droid, NAO conveys different emotions by changing the color of its eyes, the motion of its limbs, and the tone of its voice.

The 35 children with autism who participated in this study, 17 from Japan and 18 from Serbia, ranged in age from 3 to 13. They reacted in various ways to the robots during their 35-minute sessions, from looking bored and sleepy in some cases to jumping around the room with excitement, clapping their hands, and laughing or touching the robot.

Most of the children in the study reacted to the robot “not just as a toy but related to NAO respectfully as it if was a real person,” especially during storytelling, where the therapists asked how NAO would feel if the children took the robot for an ice cream treat, according to Rudovic.

One 4-year-old girl hid behind her mother while participating in the session but became much more open to the robot and ended up laughing by the end of the therapy. The sister of one of the Serbian children gave NAO a hug and said “Robot, I love you!” at the end of a session, saying she was happy to see how much her brother liked playing with the robot.

“Therapists say that engaging the child for even a few seconds can be a big challenge for them, and robots attract the attention of the child,” said Rudovic, explaining why robots have been useful in this type of therapy.

“Also, humans change their expressions in many different ways, but the robots always do it in the same way, and this is less frustrating for the child because the child learns in a very structured way how the expressions will be shown.”

Discover more here.

Image credit: MIT Media Lab.