Low-latency platform handles CERN's colossal data needs

Massive volumes of data are collected by the experiments on CERN’s Large Hadron Collider (LHC), the world’s largest and most powerful particle accelerator. Teams from IDT and CERN will use the IDT technology to improve the quality and timeliness of this data collection, as well as the initial analysis and reconstruction work at the experiments’ data farms and the CERN Data Centre.

The LHC produces millions of collisions every second in each detector, generating approximately one PB of data per second. This data is vital to CERN’s quest to answer fundamental questions about the universe. The RapidIO technology provides a low-latency connection between clusters of computer processors, dramatically speeding the movement of data. Widely used for 4G base stations, IDT’s low-latency RapidIO products can also enable real-time data analytics and data management for high-performance computing and data centres.

As part of the mandate for the fifth phase of the CERN openlab partnership, several of the LHC experiments are exploring the possibility of moving from custom-built hardware and backplanes to fully programmable heterogeneous computing with low-latency interconnect between large clusters of processors. IDT’s current RapidIO 20Gb/s interconnect products will be used in the first stage of the collaboration with an upgrade path to RapidIO 10xN 40Gb/s technology in the future as research at CERN progresses. 

Because of the volume of real-time data CERN collects, current implementations are done in custom-built ASIC hardware. Using algorithms implemented in hardware, the data is sampled, and only 1% is selected for further analysis.

The collaboration is based on industry standard IT form factor solutions suitable for deployment in HPC clusters and data centers. Engineers will use heterogeneous servers based on specifications from RapidIO.org that are targeted towards the Open Compute Project High Performance Computing initiative that IDT co-chairs.

"This CERN collaboration is about enabling programmable real-time mission critical data analytics,” said Sailesh Chittipeddi, Vice President of Global Operations and CTO, IDT. “Since the job spans multiple processors, the interconnect between them has to be ultra-low latency, and our technology - already used across 4G wireless base station deployments worldwide - is suited to CERN’s real-time interconnect needs."

“The bottleneck for better data acquisition, selection and analytics is superior real-time interconnect,” said Alberto Di Meglio, Head of Openlab, CERN. “Our collaboration with IDT to develop a RapidIO-based computing architecture should help solve CERN’s real-time data filtering problem, enabling us to select and utilise more meaningful events from the LHC and improve efficiency of analytics in our data centre monitoring and operations.”

“We established the HPC initiative to service the unique needs of those end users with the highest compute-centric workloads in the industry,” said Corey Bell, CEO, Open Compute Project. “CERN has some of the most stringent workloads for low-latency computing, so this collaboration is a great opportunity to see the benefits of RapidIO in action.”

The computing platform used for the collaboration is based on commercially available RapidIO-enabled 1U heterogeneous servers capable of supporting industry-standard servers, GPU, FPGA and low-power 64-bit SoCs, as well as top-of-rack RapidIO switches available from Prodrive Technologies.