Understanding medical conversations with Google's deep learning tech

In order to help offload note-taking, many doctors have started using medical scribes as a part of their workflow. These scribes listen to the patient-doctor conversations and create notes for the EHR.

According to a recent study, introducing scribes not only improved physician satisfaction, but also medical chart quality and accuracy. But the number of doctor-patient conversations that need a scribe is far beyond the capacity of people who are available for medical scribing.

So a Google team wondered: could the voice recognition technologies already available in Google Assistant, Google Home, and Google Translate be used to document patient-doctor conversations and help doctors and scribes summarise notes more quickly?

In “Speech Recognition for Medical Conversations”, the company shows that it is possible to build Automatic Speech Recognition (ASR) models for transcribing medical conversations.

While most of the current ASR solutions in medical domain focus on transcribing doctor dictations (i.e., single speaker speech consisting of predictable medical terminology), their research shows that it is possible to build an ASR model which can handle multiple speaker conversations covering everything from weather to complex medical diagnosis.

Using this technology, Google will start working with physicians and researchers at Stanford University, who have done extensive research on how scribes can improve physician satisfaction, to understand how deep learning techniques such as ASR can facilitate the scribing process of physician notes.

In their pilot study, Google investigates what types of clinically relevant information can be extracted from medical conversations to assist physicians in reducing their interactions with the EHR. The study is fully patient-consented and the content of the recording will be de-identified to protect patient privacy.

The team hopes these technologies will not only help return joy to practice by facilitating doctors and scribes with their everyday workload, but also help the patients get more dedicated and thorough medical attention, ideally, leading to better care.