Touch and sound let blind “see” computer screens

When the cursor moves across an object on the screen, force feedback, vibrations and sound cues give the user information about the object’s size, shape, texture and color to help them identify the information displayed on a computer screen. 

Working under the guidance of Brad Duerstock, associate professor of engineering practice in the School of Industrial Engineering and Weldon School of Biomedical Engineering, and Juan Wachs, associate professor in the School of Industrial Engineering, Zhang is trying to address the number of students with visual impairments who become involved in STEM studies.

A 2014 National Science Foundation publication reports that no more than 1% of people who are visually impaired are involved in advanced science and engineering research and receive doctoral degrees.

“How science is conducted and how findings are typically represented is usually quite visual, making it difficult for students with visual impairments,” Duerstock said. Conventional methods to assist such students include printing tactile representations of computerised images on expensive 3D sheets of material.

“The method is much less expensive and allows a person to directly interface with a computerised image using a haptic device and other sensory interfaces,” Duerstock said. Haptic devices are handheld devices that give users feedback with forces when used with a computer system, such as controllers common with modern home video game systems.

David Schwarte, assistive technology specialist for Information Technology at Purdue, or ITaP, has been testing the system with Duerstock and Wachs’ team. He has a visual impairment. “I think this has a possibility of making things easier for people with a visual impairment,” Schwarte said. “The big advantage is that it’s electronic and more real time.”

In addition to students of all ages, Zhang believes the system would be useful for scientists with visual impairments. “We think the technology could be extended to other users as well, such as people without disabilities needing to get details about objects that only exist in the virtual state,” Zhang said.

“New Scientist” published an article about the technology. Duerstock and Wachs’ team has a working prototype of the system and plans to do field trials this summer. The team also is looking for collaborators and investors to expedite the commercialisation of the technology.

Zhang also plans to create a startup company to market the technology. “We’re going through Purdue Foundry’s LaunchBox program right now. We’re also doing market research on the technology,” she said.