Intelligent acoustic sensor systems support The Hearing Car

Experts at the Fraunhofer Institute for Digital Media Technology IDMT in Oldenburg are working on The Hearing Car and developing AI-based system solutions for acoustic event recognition in the vicinity of vehicles. A mobile system for in-vehicle electroencephalography (EEG) for optimising human-machine interaction and the YourSound assistant for personalising sound experiences complete the system.

The researchers will provide an insight into the latest developments at the IAA MOBILITY trade show in Munich from 5— 8 September 2023, at the joint Fraunhofer booth in Hall B1, Booth D11.

Modern vehicles are insulated against external noise to meet end customers’ requirements for driving comfort, especially at higher speeds. As a result, noises that are critical for making the right driving decisions are often heard too late, or not at all. However, external noises such as ambulance sirens, wet roads, or a screw in a tire can provide key information to drivers. Conventional vehicle sensors reach their limits when it comes to detecting these noises. This is why intelligent acoustic sensor systems developed by Fraunhofer IDMT at the Oldenburg Branch for Hearing, Speech, and Audio Technology HSA are going to equip cars with a sense of hearing. In conjunction with other driver assistance systems, the smart acoustic system provides relevant information for necessary driving manoeuvres or (predictive) maintenance.

“On behalf of car manufacturers and suppliers, we are developing and testing new sensor technologies and algorithms for acoustic environment monitoring, source localisation, signal enhancement, and speech interaction on both test tracks and roads,” explains Moritz Brandes, Project Manager of The Hearing Car at Fraunhofer IDMT in Oldenburg. For research and development purposes, the experts are using their own workshops to set up demonstrators and their own test vehicle to test components and algorithms. Employees of the Oldenburg Branch for Hearing, Speech, and Audio Technology HSA are internationally qualified after having completed test-drive training courses. On behalf of their customers, they take test drives in various climate zones.

Acoustic turn assistant: like having ears in your blind spot

Nowadays, trucks and cars make use of cameras instead of mirrors to monitor traffic, including in the blind spot that occurs when turning. Complementing such camera systems, acoustic event analysis or detection of important ambient noises can crucially enhance the sensory functions in current as well as in autonomous vehicles. At the HSA branch of Fraunhofer IDMT, specific research is being conducted into algorithms for detecting and locating traffic-related noises to ensure that tomorrow’s vehicles are integrated even more safely into the traffic flow. For example, microphones in truck camera arms and mirrors offer an opportunity to expand the driver’s perception of acoustic information when turning their vehicle to help prevent accidents.

Acoustic rear-view camera: like having ears at the back of your car

Whether drivers are reversing into parking spaces, hitching trailers, or otherwise manoeuvring their vehicles, acoustic signals can significantly help to increase the safety of both drivers and passersby. Microphones installed on vehicles and intelligent software enable drivers to interact with people outside the vehicle without having to open their windows. At Fraunhofer IDMT in Oldenburg, research is being conducted into the accurate recording and reproduction of ambient noises in terms of location as well as the corresponding microphone hardware. This function will increase the driver’s attention when executing different manoeuvres and – the developers hope – will actively prevent accidents.

Mobile systems for electroencephalography (EEG) for optimising human-machine interaction

Drivers need to be attentive to traffic at all times in order to react to critical situations in time. But how much does a person’s attention and responsiveness change when the vehicle is driving autonomously? Researchers at Fraunhofer IDMT-HSA are tackling this question by investigating the changes in what is known scientifically as the vigilance state. To do so, they have developed a mobile EEG sensor system that records brain activity through electrodes placed on the head. Vigilance refers to a state of sustained attention during monotonous activities. Visitors to the Fraunhofer booth will be able to experience first-hand how EEG data is recorded.

YourSound: a personalised sound experience

The Oldenburg Branch for Hearing, Speech, and Audio Technology HSA will also be showcasing their innovative systems to be used in the vehicle interior at IAA Mobility 2023.

With YourSound technology, users can quickly and intuitively personalise the sound of audio devices such as headphones, smart speakers, or multimedia systems in vehicles, thereby creating their own personal sound experience in their car. The YourSound technology can be integrated into infotainment systems and is used to create a customised sound configuration which then serves as the basis for all future audio playback in the vehicle. Once users have set their preferences, they are provided with a better listening experience, regardless of the volume or driving situation.