AI technology enables logical dialogue in Japanese

Previously, Hitachi developed a basic AI technology which analysed huge volumes of English text data and presented opinions in English. This time, Hitachi incorporated this technology into a new AI technology for the Japanese language to meet the needs of Japanese enterprises.

Today, the social landscape changes rapidly and customer needs are becoming increasingly diversified. Companies are expected to continuously create new services and values. Further, driven by recent advancements in information & telecommunication and analytics technologies, interest is growing in technology that can extract valuable insight from big data which is generated on a daily basis.

Hitachi has been developing a basic AI technology that analyses huge volumes of English text data and presents opinions in English to help enterprises make business decisions. The original technology required rules of grammar specific to the English language to be programmed, to extract sentences representing reasons and grounds for opinions.

This process represented a hurdle in applying system to Japanese or any other language as it required dedicated programs correlated to the linguistic rules of the target language.

By applying deep learning, this issue was eliminated thus enabling the new technology to recognise sentences that have high probability of being reasons and grounds without relying on linguistic rules. More specifically, the AI system is presented with sentences which represent reasons and grounds extracted from thousands of articles.

Learning from the rules and patterns, the system becomes discriminating of sentences which represent reasons and grounds in new articles.

Hitachi added an "attention mechanism" which support deep learning to estimate which words and phrases are worthy of attention in texts like news articles and research reports. The "attention mechanism" helps the system to grasp the points that require attention, including words and phrases related to topics and values.

This method enables the system to distinguish sentences which have a high probability of being reasons and grounds from text data in any language.

The technology developed will be core technology in achieving a multi-lingual AI system capable of offering opinion. Hitachi will pursue further research to realise AI systems supporting business decision making by enterprises worldwide.