Enhancing the driver experience

The relentless advance of technology within cars is no secret, but as our preferred method of getting around becomes more complex, the amount and speed of information being delivered to the driver is also increasing rapidly and is becoming more graphically rich. With such complexity comes a difficult challenge for human factors designers who have to ensure that the driver isn’t overwhelmed and is presented with clear, precise graphical information.

There is an advanced graphics technology for human machine interface (HMI) graphics called ray tracing, which is capable of delivering photo realistic/hyper-realistic graphics on screen (and well within the power budgets set aside for such functions). Ray tracing has been around as a graphics technology for decades. Most high end movies and TV shows that use computer generated imaging (CGI) rely on ray tracing to deliver special effects.

Reducing the cost of ray tracing
What Imagination Technologies has done is deliver a technology as part of its PowerVR graphics family that reduces the computation needed for tracing the rays, allowing a much lower cost and power efficient solution. The patented algorithm massively reduces the computation required to deliver stunning high end graphics without compromising on power budgets. This solution can be used in mobile and automotive devices without killing the power budgets. For example, a comparative technology running on a high end graphics card would achieve one-third to one-fifth of the ray performance while consuming 250W of power versus 4-5W of the PowerVR solution.

So how does it work? Ray tracing is basically the ‘physics of light’. Rays are fired into the scene, and as they traverse it they hit objects which set off a shader (a shader is a small program that works out the lighting, material property, reflectivity, refraction, etc). The shader information along with information about whether the intersection point is in direct light or shadow, is then added to the ray. The ray then ‘bounces’ off the object, continuing to traverse the scene until it hits other objects and this, in turn, will spawn further shaders. The outcome of all of this is that each pixel will eventually end up ‘coloured’ appropriately in the scene with realistic reflections, refractions and subtle lighting effects such as global illumination. The result can end up being near photorealistic, determined by the number of ray ‘bounces’.

Above: new graphics technologies are helping to produce stunning driver experiences

Not only can ray tracing technology deliver photorealism to the instrument cluster, it can also enable simpler ways to alter the images sent to the heads up display (HUD) lenses so that they appear on a curved surface (like the windshield) as on a flat screen, thus enabling a cost reduction by reducing the need for a custom built lens. Indeed it could be used to enable a single system to deliver sharp HMIs onto any screen with just a simple alteration to the pre-warping algorithm so that one HUD device could be used in a range of cars from different manufacturers.

Ray tracing can also be used to enhance rasterised graphics (what we have today) by judicious use in a hybrid (rasterising/ray tracing) solution. The levels of use of ray tracing can range from adding realistic shadowing to the scene (to make the dials and needles look much sharper), to enabling absolute photo realistic images of the car in the infotainment system.

Other uses
Another application is in augmented reality (AR) headsets. Here foveated rendering can be used to place more detail in the centre of where the eye is looking. To explain foveated rendering, if you think about the eye, the detail that you pick up is in a 30-40° arc in the centre of view - the rest of the image through the eye delivers less detail. With ray tracing in an AR headset, the eyes can be tracked by sensors and the majority of the rays focussed in the central area that the user is looking at. This is a much more efficient use of the total ray budget and can give a much higher level of detail where it is needed.

Using such a technique, the HUD information can be placed in the centre of vision at all times. In addition, the time the eye spends gazing away from the windshield can be tracked, and appropriate warnings sounded so the driver is reminded to look at the road.

Ray tracing also gives advantages to the HMI/UI developer as it is often much easier to achieve an effect using ray tracing rather than multiple rendering techniques that can often only get close to what ray tracing can deliver. Indeed there are many effects such as reflection in reflection that can be done using ray tracing, and are impossible in standard rasterisation.

The PowerVR ray tracing technology is able to work alongside optimised PowerVR graphics architecture as an additional logic block and pipeline to Imagination’s standard graphics processors (GPUs).

From the HMI developer point of view, the new applications programming interface (API) that they will be working with should feel comfortable as it will be in the form of OpenGL ES API extensions which have been around for embedded devices for over a decade. Coupled with a software development kit (SDK) and supporting toolchain it will be straightforward for a HMI developer to pick up and run with this new capability. Any designer using Unity3D 5.4 tools can already experience the benefits of ray tracing in developing light maps much more efficiently using the Light Map Baking tool. This massively increases developer efficiency by showing the developer the effects of changes made to their lighting in seconds rather than minutes as is the case currently.

To sum up, PowerVR Ray Tracing coupled with PowerVR Graphics technology delivers a truly high end graphics experience, giving the user stunning photorealistic or hyper-realistic HMIs. Ray tracing graphics represent the logical next step of automotive graphics, following a path well-trodden already by film and TV companies. Ray tracing can also help reduce cost and complexity in delivering HUD/HMI systems. Imagination’s technology can help car manufacturers and OEMs deliver advanced graphics on multiple screens around the cockpit without overwhelming the driver.