Unveiling the power of safety bubble detectors in robotics

This article focuses on implementing a safety bubble detector application using the Analog Devices EVAL-ADTF3175D-NXZ time of flight platform. The ADTF3175D module has a field of view (FoV) of 75°. To cover a wider FoV in real-world applications, multiple sensors are combined. For example, to cover 270° FoV, four modules are used. The safety bubble detection algorithm runs on the i.MX 8M Plus processor featured in the EVAL-ADTF3175D-NXZ platform. The algorithm captures the depth images from the sensor and detects any object within the safety bubble radius. To facilitate integration into robotics applications, the safety bubble detection application is implemented using the Robot Operating System (ROS) framework. The algorithm is highly optimised to achieve a frame rate of 30 FPS on this platform. Safety bubble detectors serve as a fundamental component of automated guided vehicles (AGVs) and autonomous mobile robots (AMRs). The safety zone is typically represented as a virtual circular area around an AGV/AMR, indicated by the red circle in Figure 1.

Figure 1. A safety bubble detector.

Safety bubble detectors are fundamental to any AGV/AMR system. As shown in Figure 2, a safety bubble detector has been constructed using four EVAL-ADTF3175D-NXZ modules to cover an FoV of 278°. The modules are arranged in a horizontal setup, with each time of flight (ToF) module positioned at 67.5° relative to each other. This configuration helps reduce blind spots and provides an FoV of 278°.

Figure 2. A horizontal setup. (a) Top view. (b) Front view.

To facilitate communication between the ToF modules and the host system, the ROS publisher-subscriber model is employed, as illustrated in Figure 3. In this setup, the Ethernet over USB is used for communication to ensure data integrity and enhance communication speed.

A safety bubble detection algorithm is used to detect objects within the safety bubble radius. The detection flag is transmitted as a ROS topic, allowing the host machine to subscribe to all the modules’ topics and combine the detection results. Additionally, the modules publish depth, IR, and output images for further analysis. ROS provides effective visualisation tools, such as rviz, which can visualise published topics. The application is designed to be highly configurable and passes parameters to the ROS nodes for adjusting camera position, rotation, and other configuration values.

A multithreaded architecture is implemented in the application, as shown in Figure 4. Three threads, namely input, process, and output, run in parallel. The design aims to minimise latency and ensure that the processing block continuously operates on the most recent accessible frame. The input thread reads the image from the ToF module and updates the input queue, while the process thread takes the input queue and runs the safety bubble detection algorithm, publishing the detected flag and pushing the output to the output queue. The output thread reads the output queue and publishes the topics for visualisation. In real-time scenarios where the processing block has a lower frame rate than the input thread, previous frames are discarded to prioritise the most recent frame with minimal latency.

Figure 3. Host as a subscriber and ROS nodes as publisher’s architecture.

Figure 4. A multithreaded program.

Communication between the host and ToF modules occurs via the ROS publisher-subscriber model, using the TCP/IP protocol. The host combines the published output images from the ROS nodes (ToF modules) and publishes the combined output.

Shown in Figure 5, the host machine is the NVIDIA Jetson Xavier NX, which powers and communicates with all four ToF modules using Ethernet over a USB protocol.

The default radius of the safety bubble is one meter, which can be configured in the ROS launch files. If an object is detected within this area, the object detection flag is triggered and sent to the host via a ROS topic. The host machine subscribes to the object detection topic from each ToF module. The results are combined using a simple logical operating range (OR) operation, indicating the presence of an object if any of the sensors detect one within the safety bubble.

Figure 5. Horizontal setup with an NVIDIA Jetson Xavier NX.

For visualisation, the sensors convert the acquired image to its top view and mark the objects in green and red pixels depending on whether they are inside or outside the safety bubble. This image is also published as a ROS topic by each sensor and the host machine combines them into a combined image. Figure 6 shows the combined image of all the published output image topics.

Figure 6. Combined top view of four TOF modules.

For visualisation, a square box is drawn on the top left corner to show object detection status (green: objects not detected, red: objects detected). See Figure 7.

These images can be visualised using the ROS tool rviz. In addition, the NVIDIA Jetson Xavier NX can be connected to a monitor using an HDMI cable to see the output. Visualisations such as the depth image, point cloud, and top view of the input image can be enabled for analysis. These visualisations provide more detailed information and insights into the detected objects and their spatial relationships. See Figure 8.

Figure 7. Visualisation. (a) Objects not detected. (b) Objects detected.

Figure 8. Visualisation (debug images for analysis).

SQA process used

Standard software quality assurance (SQA) methodologies are used to ensure software safety and quality.

• Unit test: ROS supports multiple levels of unit testing.
• Library unit testing: ROS-independent libraries are tested.
• ROS node unit test: Node unit tests start the node and test its external API; that is, published topics, subscribed topics, and services.
• Code coverage: Code coverage analysis is done by one of the packages of ROS, which helps to eliminate the dead code and increase the unit test quality.
• Documentation: ROS has a package called ros_doc_lite, which gives a doxygen format document for the source files.

Clang format is used to format the code and Clang-Tidy is used to maintain the ROS coding style guidelines.

The safety bubble detector reliably detects objects of various shapes, colours, and sizes, including cables as small as 5 mm thick.

The algorithm’s low latency of 30 ms ensures real-time object detection and response.

By leveraging the ROS framework extensively for interface and visualisation, the application is highly portable. It is compatible with any host machine that uses ROS and will reduce customers’ time to market.

A ToF sensor’s accuracy is lower for transparent and low reflective objects. This results in the late detection of some objects like glass bottles and plastic balls. As an example, Figure 9 depicts the distance at which objects are detected by the algorithm (the safety radius is set to 100 cm). The y-axis represents test objects. Glass bottle (12, 7) implies that the glass bottle is 12 cm tall and

7 cm wide; if only one parameter is present in brackets, the radius of the object or length of the cube is indicated. See Table 1 for a summary of the safety bubble detector specifications.

Figure 9. The detection accuracy.

Conclusion

This safety bubble detector comprised of the ADTF3175D and EVAL-ADTF3175D-NXZ ToF platform has many advantages. It is highly optimised for the i.MX 8M Plus platform, achieving a smooth performance of 30 FPS. It utilises a multithread approach to minimise latency, ensuring quick and responsive operation. It also implements SQA methodologies to ensure software safety and maintain quality standards.