Reversing relies on video for safety updates

The importance of automobile safety has led to several key innovations over the last 50 years, starting with the seat belt in 1966. Since then, many more innovations such as anti-lock brakes and airbags arrived on the scene as standard safety equipment. Over the last two decades alone, vehicles began offering lane-departure warning systems, and radar-based blind spot monitoring to help drivers safely change lanes. The latest safety technology to become standard equipment is the rearview camera.

Meeting Legislation

Although rearview camera systems have been available as optional equipment for many years, US and other vehicle manufacturers around the world are starting to make rearview camera systems (Figure 1) standard equipment for all new car models. This global safety trend follows the US Federal Government’s National Highway Traffic Safety Administration (NHTSA) mandate for rearview cameras. The NHTSA regulation requires that manufacturers implement rear visibility technology as standard equipment for all vehicles under 10,000 pounds.

Figure 1: The rearview camera system is now the baseline safety feature in all new car models

The legislation required the NHTSA to revise the Federal Motor Vehicle Safety Standard 111 (FMVSS-111) to expand the required field of view behind the vehicle. It also states that all new vehicles sold in the US must have a rearview camera system from May 2018. Honda and Acura were among the first automobile brands to offer back-up cameras as standard equipment on all models. According to IHS Markit research, infotainment and safety critical displays will see double digit growth through 2022.

Design challenges

The major technical challenge faced by car original equipment manufacturers (OEMs) is how to achieve compliance with the new US law, including a key piece of the NHTSA regulation, which states live rear camera video must be displayed in less than two seconds after the driver places the vehicle into reverse gear. The law aims to reduce the number of deaths and injuries caused by drivers reversing, many accidents involve young children. One key challenge to achieving FMVSS-111 compliance is the fact that the centre head unit’s main processor, or SoC, runs a complex operating system (OS), which can take several seconds to boot up.

Figure 2a: Frozen rearview camera image    

Figure 2b: Actual rearview camera image

In addition, the OS is prone to software freezes and glitches that can result in displaying a frozen image. As software complexity increases, freezes and hang-ups are becoming more prevalent, affecting system reliability and potentially leading to accidents.

Hardware bypass

The TW8844 high definition (HD) LCD video processor with built-in video decoder is optimised to ensure rearview camera reliability. It can turn on, synchronise to a video source and display that video in less than half a second, solving the fast boot reliability issue. Another major concern is the reliability of the camera displaying live video when the driver needs it. The system is running a very complicated OS and software stack, which can cause it to freeze and not show live video when the vehicle is placed in reverse gear. Figure 2 shows a frozen rearview camera image on left, and the actual rearview camera image with a pedestrian walking behind the vehicle on right. The driver thinks he or she is seeing a real-time video feed, and that the area behind the vehicle is clear, but in reality it is an older, frozen video frame. By the time the vehicle starts to move, a child could have run behind it, or someone could have walked past (see Figure 2b), introducing a major safety concern.

Figure 3: Frozen detection decouples the camera-display path from the SoC

The TW8844 full HD LCD video processor works in conjunction with the system’s main SoC, and is capable of monitoring both the camera output and the video data coming out of the SoC. If it determines the SoC’s output video is frozen, the TW8844 will reroute the camera video and bypass the SoC, sending the camera video directly to the center stack LCD panel. This happens nearly instantaneously. When the SoC recovers, the TW8844 can switch back to it.

Camera system signal flow

Under normal operating conditions, the rearview camera video is fed into the TW8844 LCD video processor, as shown in Figure 3. The TW8844 can receive either traditional analogue composite video or HD digital video. It processes the video then routes it to the SoC via a MIPI-CSI2 interface. The SoC mixes the camera video with graphical overlay content such as text, icons and coloured parking guidelines. The combined image is sent back to the TW8844, which in turn drives that video directly to the LCD panel. Throughout this process, the TW8844 is constantly performing frozen detection measurements, using multiple techniques to measure image aspects such as luminance, colour values, individual pixels, and more to determine if the video is frozen or corrupted. The system designer can program thresholds for the various measurement techniques to create a unique and robust frozen/corrupt image detection capability that meets a particular OEM’s requirements.

If the TW8844 determines that the SoC is sending frozen or corrupted video, it automatically issues an interrupt signal to the microcontroller (MCU), which instructs the TW8844 to go into bypass mode, directly driving the rear camera video to the LCD panel. At that point, the MCU can also instruct the SoC to go into soft reset so that the driver does not have to re-start the vehicle in order for the head unit software to recover. Once the software stack starts working properly again, the MCU instructs the TW8844 to change from bypass mode and go back to feeding video into the SoC again.

The same signal flow occurs during fast boot mode. When the vehicle ignition starts up and the driver places the car into reverse gear, a reverse signal is triggered, telling the system MCU the car is in reverse. If the SoC software has not completed boot up by the time the vehicle is shifted into reverse, the MCU instructs the TW8844 to send the camera video straight to the LCD while the SoC completes the boot up cycle. Once the OS is booted, the TW8844 instantly changes modes and routes the signal to the SoC to mix the video with graphics before the signal comes back to the TW8844 to route to the LCD. 

Inside the video processor

The integrated TW8844 (Figure 4) is the first full high-definition (HD) 1080p LCD video processor with analogue video decoder, two scalers and MIPI-CSI2 SoC interface designed specifically to address infotainment and rear camera display systems. The TW8844 provides the reliability automakers require to ensure their rearview camera is compliant with the FMVSS-111 standard. The TW8844 enables the migration from analogue to digital camera systems, and supports a variety of video interfaces and LCD panel resolutions up to full HD 1920 x 1080.

Figure 4: The TW8844/45 circuit diagram highlights key function blocks within the HD LCD video processor

The TW8844 video processor with MIPI-CSI2 output and its sister device, the TW8845 with traditional BT.656 output, exceed the requirements of the FMVSS-111 law. Both devices display live video with graphics overlay in less than 0.5 seconds after vehicle ignition, overcoming the fast boot reliability issues inherent with today’s more complex center-stack systems. Through its frozen/corrupt image detection capability, the TW8844/45 provides a robust rear camera architecture that adds a level of system reliability that goes beyond the specific wording of the FMVSS-111 regulation, but is in keeping with the intent of the new law.

What’s on the horizon?

Safety is a significant differentiator and has become a real factor in the consumer’s choice of vehicle. There is more on the horizon as the cost of implementing safety features continues to decline, making it possible to deliver them in more mainstream models. Moving forward, more video will be used to deliver capabilities like night vision and object recognition (video analytics) to improve vehicle safety.