AI technology developed to combat cyber attacks

Conventionally, an expert in cyber attack analysis then manually investigates and checks the degree of threat, to determine whether action is needed to minimise damage.

To secure the necessary training data needed to develop highly accurate AI technology, Fujitsu Laboratories has now developed a technology that identifies and extracts attack logs, which show the behaviour of a cyber attack, from huge amounts of operations logs. It also developed a technology that expands on the small number of training data extracted in a manner that does not spoil attack characteristics. This generates a sufficient amount of training data.

In simulations using these technologies, they achieved a match rate of about 95% in comparison with experts' conclusions regarding the need for action, and they did not miss any attack cases that required a response. The time necessary to reach a conclusion was also shortened from several hours to several minutes.

By using these technologies, countermeasures can quickly be put in place for cyber attacks that have been determined to require action, contributing to business continuity and the prevention of loss. Details of these technologies are being announced at the 36th Symposium on Cryptography and Information Security (SCIS 2019), being held from Tuesday, January 22nd, to Friday, January 25th, in Otsu city, Shiga prefecture, Japan.

In recent years, the number of cyber attacks against business networks has continued to increase. With targeted attacks, the attacker uses clever techniques to embed malware that can be controlled remotely in an organisation, and then remotely controls devices infected with malware to conduct intelligence activities. In defence, when a company discovers suspicious activities with such monitoring equipment as a security appliance, a security expert manually investigates the attack, and takes time to evaluate danger and risk, then determines the necessity to respond.

The decision to respond needs to be made carefully as the responses themselves may have consequences. For example, attacked business devices may need to be isolated, and the network reconstructed, resulting in operation stoppages that impact businesses.

According to statistics from Japan's Ministry of Economy, Trade and Industry, by 2020 there will be a shortage of 193,000 security professionals in Japan. That being said, AI-based automation is expected to rapidly determine the necessity to respond to attack cases, making decisions on the same level as an expert who has advanced knowledge and insight on attacks.

In order to develop an AI-based model to make determinations, the following issues regarding training on attack information needed to be addressed:

1. The operations logs for normally functioning servers, devices, and network equipment coexist with the attack operations logs, and both logs are accumulated in great abundance. To conduct proper learning with AI, it is necessary to identify the traces of targeted attacks from the large number of logs. However, distinguishing between logs is difficult because intelligence activities via targeted attacks utilise OS commands and other methods.
2. It is extremely difficult to extract attack operations logs from the huge amounts of existing logs, while securing them in large quantities as training data. For AI technologies, it is possible to increase the small amounts of training data through procedures and conversions such as noise processing; however, such simple processing of the training data of targeted attacks can cause the attack characteristics to be lost, making data expansion difficult.

Fujitsu Laboratories has developed technologies to secure sufficient amounts of training data related to targeted attacks required for the creation of highly accurate, AI determination models. Features of the developed technologies are outlined below:

Training data extraction technology
Based on the know-how Fujitsu has accumulated in its security-related business and research, as well as from about seven years' worth of actual attack analysis data, Fujitsu Laboratories has built a database of attack patterns that includes commands and parameters linked to intelligence activities of targeted attacks. By using this database, users can accurately identify and extract a series of intelligence activities from the vast amounts of logs.

Training data expansion technology
This technology generates simulations of new intelligence gathering activities-a type of targeted attack-without losing attack characteristics. The technology calculates attack levels and identifies the important commands of intelligence activities in the extracted targeted attack, then converts the parameters within the range existing in the attack pattern database. As a result, it becomes possible to expand the training data fourfold.

Effects
Fujitsu Laboratories combined the newly developed technologies with its own Deep Tensor AI technology, and ran evaluative testing on the determination model that had been trained on the new training data. Run in a simulation using about four months of data-12,000 items-the technologies made an approximate 95% match with the findings that a security expert generated through manual analysis, achieving a near equal determination of response necessity.

Furthermore, the technologies were field tested on STARDUST, the Cyber-attack Enticement Platform which is jointly operated with the National Institute of Information and Communications Technology (NICT), using real cyber attacks targeting companies. The technologies automatically determined the attack cases requiring a response, thereby confirming their effectiveness.

With these AI technologies, determinations of the necessity of action, which until now have taken an expert several hours to several days, can be automatically made with high accuracy from tens of seconds to several minutes. Furthermore, by combining these technologies with Fujitsu Laboratories' high-speed forensic technology, which rapidly analyses the whole picture of the status of damage from a targeted attack, the response sequence, from attack analysis to instructions for action, can be automated, enabling immediate responses to cyber attacks and minimising damage.

Future plans
Fujitsu aims to make use of these technologies within its Managed Security Services, as a response platform for cyber attacks.