Deep learning software analyses cardiovascular risk

While most of these factors can be obtained by simply asking the patient, others factors, like cholesterol, require a blood draw. Doctors also take into account whether or not a patient has another disease, such as diabetes, which is associated with significantly increased risk of CV events.

Recently, Google has studied many examples of how deep learning techniques can help to increase the accuracy of diagnoses for medical imaging, especially for diabetic eye disease.

In “Prediction of Cardiovascular Risk Factors from Retinal Fundus Photographs via Deep Learning,” published in Nature Biomedical Engineering, Google has shown that in addition to detecting eye disease, images of the eye can very accurately predict other indicators of CV health. This discovery suggests researchers might discover even more ways to diagnose health issues from retinal images.

Using deep learning algorithms trained on data from 284,335 patients, Google was able to predict CV risk factors from retinal images with surprisingly high accuracy for patients from two independent datasets of 12,026 and 999 patients. For example, their algorithm could distinguish the retinal images of a smoker from that of a non-smoker 71% of the time.

In addition, while doctors can typically distinguish between the retinal images of patients with severe high blood pressure and normal patients, Google's algorithm could go further to predict the systolic blood pressure within 11 mmHg on average for patients overall, including those with and without high blood pressure.

In addition to predicting the various risk factors (age, gender, smoking, blood pressure, etc) from retinal images, the algorithm was accurate at predicting the risk of a CV event directly.

The algorithm used the entire image to quantify the association between the image and the risk of heart attack or stroke. Given the retinal image of one patient who (up to 5 years) later experienced a major CV event (such as a heart attack) and the image of another patient who did not, the algorithm could pick out the patient who had the CV event 70% of the time.

This performance approaches the accuracy of other CV risk calculators that require a blood draw to measure cholesterol.

Google opened the “black box” by using attention techniques to look at how the algorithm was making its prediction. These techniques allow us to generate a heatmap that shows which pixels were the most important for a predicting a specific CV risk factor.

For example, the algorithm paid more attention to blood vessels for making predictions about blood pressure, as shown in the image above. Explaining how the algorithm is making its prediction gives doctor more confidence in the algorithm itself. In addition, this technique could help generate hypotheses for future scientific investigations into CV risk and the retina.

At the broadest level, Google is excited about this work because it may represent a new method of scientific discovery. Traditionally, medical discoveries are often made through a sophisticated form of guess and test — making hypotheses from observations and then designing and running experiments to test the hypotheses.

However, with medical images, observing and quantifying associations can be difficult because of the wide variety of features, patterns, colours, values and shapes that are present in real images.

This approach uses deep learning to draw connections between changes in the human anatomy and disease, akin to how doctors learn to associate signs and symptoms with the diagnosis of a new disease. This could help scientists generate more targeted hypotheses and drive a wide range of future research.

With promising results, a lot of scientific work remains. Google's dataset had many images labeled with smoking status, systolic blood pressure, age, gender and other variables, but it only had a few hundred examples of CV events.

Discover more here.

Image credit: Google Research Blog.