New demo system combines 3D sensing with eye tracking

These important features are required to support new assisted and autonomous driving technologies, and go beyond what today´s 2D systems are capable of providing. The 3D optical sensing technology from ams provides a detailed ‘depth map’ image of the driver’s head and analyses it to recognise the driver’s unique facial features to identify the driver.

Firat Sarialtun, In-Cabin Sensing Segment Manager of ams, said: “The automotive industry has been actively evaluating 3D optical sensing technologies for driver monitoring, as it provides more accurate and robust information about the position and movement of the driver’s head and eyes compared to existing less sophisticated 2D systems. Now, by developing a 3D demonstration system for eye tracking, ams and Eyeware have shown the industry the way to integrate the 3D sensing hardware and eye-tracking software elements and produce a reliable method for detecting drowsiness and distraction.”

Bastjan Prenaj, Co-founder and Chief Business Development Officer at Eyeware, added: “Driver monitoring has the potential to make a vital contribution to improving road safety, but the systems and components used in a DMS must be ultra-reliable. That is a key benefit of the  Eyeware software: together with the ams 3D sensing technology, it performs in all lighting and usage conditions, and gives accurate data on which OEMs can build applications such as drowsiness alerts.”

Flood and dot pattern illuminators from ams are used in the ASV-based demonstrator to provide robust illumination, resisting interference from sunlight and reflection from sunglasses. This com-bines with advanced eye-tracking software from Eyeware Tech SA, to apply proprietary algorithms to the depth-mapped images of the driver’s eyes and head posture to calculate in real time the direction of the driver’s gaze 

Designers developing driver and cabin monitoring systems can use the outputs of this demonstrator to build sophisticated safety features which can detect when the driver is drowsy or asleep at the wheel. Using 3D images and 3D head pose estimation enables precise monitoring of the so called ‘micro sleeps’ for early detection of sleepy drivers, not possible with 2D systems. The DMS also detects drivers when their gaze has been distracted from the road, for instance to read a text message on their mobile phone. This can help reduce a leading cause of road accidents.

The signals from the 3D DMS demonstrator will also enable next-generation heads-up displays, detecting where the driver’s gaze is directed to control the projection of displayed information on the windshield. This enables visual applications in the car based on augmented reality. The demonstration system is based on the ams Active Stereo Vision evaluation kit, which uses the ams face image processing library and Eyeware’s eye-tracking software. Customer implementations can use a dedicated software development kit supplied by Eyeware, which is fully compatible with the ams 3D sensing components.