Enabling dual camera streaming for 13MP cameras on NXP i.MX8M Plus

However, the processor doesn’t support simultaneous streaming of high-bandwidth data. Jump right into the article to learn an out-of-the-box technique e-con Systems’ engineering team came up with to solve this challenge.

Many embedded vision applications like AMRs, surround view systems, autonomous tractors, etc., rely on multi-camera systems for various reasons. These include enabling a wide enough field of view to cover a scene, eliminating lens distortions, and obtaining images of target objects from multiple angles/orientations.

So, it’s clear that multi-camera applications require the camera to transfer high-resolution data at high speeds to the host processor. However, this depends on the bandwidth of the camera interface and the capabilities of the host platform. Now, while the NXP i.MX8M Plus is a powerful and versatile processor well-suited for modern applications, it is only capable of supporting one stream of 4K@30fps.

In this article, you’ll find an out-of-the-box method to overcome this challenge and how to maximise the potential of the NXP i.MX8M Plus processor with e-con’s 13 MP MIPI colour camera modules.

Understanding the dual camera pipeline in an embedded vision system

First, let’s understand how the pipeline in a dual camera setup works using i.MX8M Plus. The data from the two cameras (13MP modules in this case) are transferred through a MIPI CSI-2 interface to the ISI – and from there to the memory.

This entire workflow is demonstrated in the image below:

Figure 1 – i.MX8M Plus camera data flow

4 reasons why NXP i.MX8M Plus can be very effective for dual streaming

• High performance: Features an Arm Cortex-A53 quad-core CPU, a Cortex-M7 real-time processor, and a dedicated neural processing unit (NPU) to handle complex vision tasks.
• Low power consumption: Designed to consume minimal power – making it ideal for battery-powered embedded devices that require long battery life.
• Robust connectivity: Comes with a range of connectivity options (Wi-Fi, Bluetooth, Ethernet, USB, and HDMI) to easily integrate with other systems.
• Customisation: The i.MX8M Plus is available in different configurations and can be customised to meet the specific needs of a wide range of embedded vision applications.

The dual streaming challenge while using the i.MX8M Plus processor

While the i.MX8M Plus processor can support multiple video streams, processing two streams at the same time can be a challenging task. So, while connecting two cameras, the NXP i.MX8M Plus processor is only capable of supporting one stream of 4K@30fps.

This dual streaming challenge arises because the driver architecture was structured on top of the platform’s inbuilt ISP. It results in bandwidth issues even when using the camera with an inbuilt ISP for which a platform ISP is not used.  Also, video processing requires a significant amount of computational power and memory. Hence, the i.MX8M Plus processor must split its resources between the two streams – resulting in reduced performance and potential bottlenecks.

How e-con equips i.MX8M Plus-based systems with simultaneous high-resolution streaming

Embedded vision applications where simultaneous multi-camera streaming can be a game-changer NXP i.MX8M Plus is a popular processor used in many AI-based vision applications, especially those that need to maintain a smaller form factor. And many of these systems might have to have multiple cameras while still limiting the form factor for the purposes of ease of use or reducing the overall size of the device.

Our expert engineering team came up with the perfect solution to address this problem. It involves using separate ISI chain buffers for the two MIPI CSI-2 ports. By modifying the buffer allocation at the ISI layer, simultaneous streaming of 13MP data is possible at 30 fps.

This can be path-breaking in many applications given the inability of the inbuilt ISP of the i.MX8M Plus platform to handle high bandwidth image data.

e-con Systems has designed several 13MP cameras perfectly suited for the NXP i.MX8M Plus processor. It includes:

• e-CAM131_CUiMX8 – 13MP camera for i.MX8 processors
• e-CAM133_iMX8 – autofocus camera for i.MX8 processors

Some use cases of i.MX8M Plus-based systems with e-con’s 13MP cameras

• Factory automation devices
• Multi-camera drones
• Video conference systems
• Ophthalmologic devices
• Precision farming equipment
• People counting systems
• Industrial human-machine interfaces
• High-end video surveillance systems
• Parking lot management systems