Software offers real-time translation for the hearing impaired

The previous version of LiveTalk only supported the Japanese language, but now, with multilingual speech recognition and support for an AI-equipped translation engine, Fujitsu Social Science Laboratory's proprietary technology includes functionality to immediately display translations as text on screen. With this addition, LiveTalk enables real-time communication not only for communicating with people with hearing disabilities, but also between multiple languages.

Fujitsu and Fujitsu Social Science Laboratory are making LiveTalk available in order to facilitate communication between diverse groups in a variety of situations, including information booths at international event venues and tourist sites, or business meetings where multiple languages are spoken.

Background
Since LiveTalk was launched in May 2015, it has been adopted as a communication-support tool for people with hearing disabilities by some 100 businesses, hospitals, government offices, and educational institutions, in order to provide reasonable accommodation for people with hearing disabilities. In November 2016, Fujitsu strengthened the functionality of LiveTalk in order to increase usability, adding support for mobile devices, so that it could more easily be used anywhere, and by adopting a cloud-based speech recognition engine with a high degree of recognition accuracy.

As Japan looks to host a variety of international events in the coming years, there is growing interest in ways to remove barriers to communication between languages. Likewise, companies are doing more business on a global level, and have a growing need for products and services that support communication among international staff.

In this context, Fujitsu and Fujitsu Social Science Laboratory have expanded the functionality of LiveTalk with multilingual speech recognition and an AI-equipped translation engine, newly providing it as the Diversity Communication Tool.

About the new features
1. Speech recognition and automatic translation for 19 languages

• Including Japanese, LiveTalk now supports speech recognition and automatic translation for a total of 19 languages: English, Chinese (simplified), Chinese (traditional), Korean, French, Spanish, Arabic, Russian, Portuguese, German, Italian, Polish, Dutch, Danish, Finnish, Swedish, Norwegian, and Catalan.
• In ten of these languages - Japanese, English, Chinese (simplified), French, Spanish, Arabic, Russian, Portuguese, German, and Italian - support for an AI-based translation engine enables more accurate translations.
• Users can simply set their preferred display language for their own device, and the speaker's words will be displayed, translated into that language. The speaker's language does not need to be set, and there are no limits on the number or combinations of languages that can be used in a group conversation.
• The system can distinguish and translate speakers even if multiple participants speak at the same time.
• The system allows for keyboard input as well as speech input, so that hearing-impaired people can participate fully, alongside speakers of other languages.

Figure 1: Example usage of automatic multilingual translation

2. Proprietary technology that displays translation results concurrently with speech, enabling smooth communications

• Fujitsu Social Science Laboratory developed a proprietary technology (patent pending) that displays translation results concurrently with speech, so people can interact at a natural conversational tempo.
• This technology provides a low-stress communication environment compared with existing translation tools, which have a time lag between when a speaker stops speaking and when the translation appears, by eliminating the time lag.

Figure 2: Comparison with previous methods of automatic translation

Both Fujitsu and Fujitsu Social Science Laboratory are committed to using ICT to help solve communications problems that divide people, and to helping their customers communicate across boundaries.