AI Perception coming to ROS

“As more ROS developers leverage hardware platforms that contain additional compute capabilities designed to offload the host CPU, ROS is evolving to make it easier to efficiently take advantage of these advanced hardware resources,” said Brian Gerkey, CEO of Open Robotics.

“Working with an accelerated computing leader like NVIDIA and its vast experience in AI and robotics innovation will bring significant benefits to the entire ROS community.”

Open Robotics to extend ROS for NVIDIA AI 

Open Robotics will enhance ROS 2 to enable efficient management of data flow and shared memory across GPU and other processors present on the NVIDIA Jetson edge AI platform. The company state that this will improve the performance of applications that have to process high-bandwidth data from sensors such as cameras and lidars in real time. 

In addition, Open Robotics and NVIDIA are working to enable seamless simulation interoperability between Open Robotics’s Ignition Gazebo and NVIDIA Isaac Sim on Omniverse. Isaac Sim already supports ROS 1 and 2 out of the box and features an ecosystem of 3D content with its connection to applications, such as Blender and Unreal Engine 4. 

Ignition Gazebo brings a decades-long track record of widespread use throughout the robotics community, including in competition events such as the ongoing DARPA Subterranean Challenge. 

With the two simulators connected, ROS developers can easily move their robots and environments between Ignition Gazebo and Isaac Sim to run simulation and take advantage of each simulator’s advanced features such as high-fidelity dynamics, accurate sensor models and photorealistic rendering to generate synthetic data for training and testing of AI models. 

Software resulting from this collaboration is expected to be released in the spring of 2022.

Isaac GEMs released for ROS

Isaac GEMs for ROS are hardware accelerated packages that the company state make it easier for ROS developers to build high-performance solutions on the Jetson platform. The focus of these GEMs is on improving throughput on image processing and on DNN-based perception models that are of growing importance to roboticists. The company clain that these packages reduce the load on the host CPU while providing performance gain.

Isaac Sim features enable ROS developers

The latest release of Isaac Sim includes significant support for the ROS developer community. Some examples of this are the ROS2 Navigation stack and the MoveIt Motion Planning Framework. These examples are available today.

Isaac Sim generates synthetic datasets for training perception

In addition to being a robotic simulator, Isaac Sim claim to have a powerful set of capabilities to generate synthetic data to train and test perception models. It's clear that the better that a robot can perceive its environment the more autonomous it can be, thereby requiring less human intervention. 

Once Isaac Sim generates synthetic datasets, they can be fed directly into NVIDIA’s TAO, an AI model adaptation platform, to adapt perception models for a robot’s specific working environment. The task of ensuring that a robot’s perception stack is going to perform in a given working environment can be started well before any real-data is ever collected from the target surroundings.

Roboticists have long faced challenges in connecting and integrating the classic robotic tasks like navigation to AI-based perception stacks. Isaac Sim addresses this workflow challenge by being simultaneously a full robotics and synthetic data generation tool with streamlined integration with the TAO training platform.

More to come at ROS World and GTC 2021

NVIDIA is gearing up for ROS World on 21-22, October 2021. It is planning to release more new GEMs for Jetson developers including several DNNs. It will also announce features in Isaac Sim to support the ROS developer community. 

NVIDIA has a lineup of speakers, talks, and content at the upcoming GTC scheduled for 8-11 November. It will have a track for robotics developers including a presentation by Brian Gerkey, CEO of Open Robotics.