Neutrino research to triple computational performance

The computer system for the Super-Kamiokande collects, stores, and analyses the massive amount of neutrino observation data from the approximately 13,000 photomultiplier tubes in the Super-Kamiokande detector. Together with the detector, the system is a core part of the facility.

This new system will consist of a computational server cluster that will be composed of 85 Fujitsu Server PRIMERGY RX2530 M2x86 servers, a high-speed, distributed file system, and a data processing system located at the underground experiment site. The computational server will have a performance level of a SPECint_rate2006 benchmark result of 107,100, about triple that of the existing system, with about three times the disk space and data transmission speed.

Fujitsu has supplied the main computer system to the Kamioka Observatory since 1993. With this new system, Fujitsu will support neutrino research at the Kamioka Observatory, helping to shed light on the properties of neutrinos and the workings of the universe.

Installation background

The Kamioka Observatory operates the Super-Kamiokande facility, constructed in the Kamioka mine located in Kamioka-cho, Gifu Prefecture, and by observing neutrinos, is helping to solve the mysteries surrounding the birth of the universe and the creation of matter. In 2015, the ICRR's director, Professor Takaaki Kajita, received the Nobel Prize in Physics for the discovery of neutrino oscillations, which shows that neutrinos have mass. This discovery meant that the Standard Model, which assumed that neutrinos had no mass, required extensions. This has had a significant impact on a wide range of areas of research.

In addition to solar and atmospheric neutrinos, the Super-Kamiokande detector must always be ready to reliably observe very rare phenomena such as supernova neutrinos which only occur once every few decades for a matter of seconds. Therefore, the computer system must be able to operate with stability 24 hours a day, 365 days a year, and perform high speed analysis processing and securely store massive amounts of observation and post-analysis processing data of up to 500GB per day. It must also be able to rapidly access huge volumes of past data in order to re-analyse data in a short period of time.

With this new system, Fujitsu will further improve analysis efficiency over the current system. This includes enhanced reliability to meet these requirements, better data analysis performance, the massively strengthened storage capacity necessary for the next five years of observations, and analysis resource allocation when the load is concentrated.

Summary of the computer system for Super-Kamiokande

This system, installed in The Kamioka Observatory, consists of a computational server cluster, a high-speed distributed file system, and a data processing system located at the underground experiment site. The new system will be built entirely with Fujitsu products, including hardware using the latest CPUs and hard disks and software such as the FUJITSU Software FEFS scalable file system and the FUJITSU Software Technical Computing Suite HPC middleware, with the goal of improving analysis efficiency through a highly reliable system, leading to even more accurate observations.

The goal of the computer system is to accumulate observation data, and to analyse such factors as energy and the direction the neutrinos move. Through this analysis, the researchers aim to shed light on the properties of neutrinos, solve the mystery of how matter was created in the immediate period after the big bang, as well as verify the Grand Unified Theory by discovering the phenomenon of proton decay.

Computation server

The computation server will be a cluster system composed of 85 PRIMERGY RX2530 M2x86 servers (170 processors, 2,380 cores), which will provide calculation performance approximately triple that of the current system, with a SPECint_rate2006 result of 107,100.

High-Speed distributed file system

The high-speed distributed file system will be made up of eight FUJITSU Server PRIMERGY RX2540 M2 servers, eight FUJITSU Storage ETERNUS DX200 S3 units, and two FUJITSU Storage ETERNUS DX100 S3 storage systems with an effective hard disk capacity of nine petabytes, approximately tripling that of the current system. The FEFS scalable file system software will provide data transmission performance equivalent to approximately three times that of the current system in response to simultaneous high-volume access from the computation server.

Underground experiment site data processing system

The observation data captured by the Super-Kamiokande facility's photomultiplier tube detectors will be sent to a system installed within the mine consisting of 40 PRIMERGY RX2530 M2x86 servers and one ETERNUS DX100 S3 storage system, then transmitted to the high-speed distributed file system after undergoing such procedures as noise removal. This highly reliable system will offer the 24/7/365 stable operations necessary to record the precious observational data without any loss.

Yoshinari Hayato, Assistant Professor at the ICRR, University of Tokyo commented: “In the 20 years Super-Kamiokande has been in operation, we have accumulated a great deal of data, and we are continuing to do so. By using this data, it is becoming possible to carry out more detailed research to understand the characteristics of neutrinos.”

Hayato continued: “More accurate event reconstructions and more precise simulation data production require greater computational resources. Now, with this new system's increase in CPU and memory performance, we expect to further advance research to solve problems.”