App screens for arrhythmia using smartphone camera

It works because each heartbeat increases the volume of blood in the user’s vessels. Blood’s hemoglobin absorbs light, which decreases the light reflected by the skin. The Cardiio app tracks those tiny changes in reflected light, not visible to the human eye, to accurately calculate a heartbeat.

Recently, however, the startup has started using that light-reflection data to make a new app that can detect irregular heartbeat patterns of atrial fibrillation (AF), a common type of heart arrhythmia that can lead to strokes.

Called Cardiio Rhythm, the clinical app is currently being used for research only and has been validated by several academic papers. In the near future, Cardiio hopes to make the app available to consumers for early heart-arrhythmia screening.

“We aim to develop algorithms that unlock clinical insights from [biometric] data collected from commonly available devices such as smartphones and wearables,” says Cardiio co-founder and CEO Ming-Zher Poh PhD ’11, a former Media Lab research affiliate who co-developed the app. “That enables people to learn more about their health and … take more control of their health.”

Cardiio Rhythm would be intended only as an early screening tool for AF, to be used, for example, by older individuals who are more susceptible to the condition. Official diagnosis of any heart arrhythmia would still need to come from a health care professional.

According to the Centers for Disease Control and Prevention, an estimated 2.7 million to 6 million people in the United States suffer from AF, which happens when the beating in the heart’s upper chambers is irregular, so blood doesn’t flow well to the lower chambers. This can cause blood clots, which can travel to the brain and cause a stroke.

Annually, AF causes about 750,000 hospitalisations and an estimated 130,000 deaths. The condition also contributes to about 15 to 20% of ischemic strokes, the strokes caused by blood clots in the brain. “The risk of AF increases with age, and the U.S. is an aging population, so it’s a growing problem,” Poh says.

If diagnosed early, around two-thirds of AF-related strokes can be prevented through medication, Poh says. But because the condition often produces vague or no symptoms, it sometimes goes undetected.

Moreover, a type of AF called paroxysmal AF causes spontaneous irregular heartbeats that may go undiagnosed. The standard AF-detection device, an electrocardiogram (ECG), involves attaching electrodes to a patient’s chest to measure the electrical impulse of a heartbeat and detect irregularities.

“But there’s a chance the electrocardiogram won’t pick up [AF] if your heart rhythm is not irregular during a routine checkup at a clinic,” Poh says. “Sometimes people don’t discover they have this irregular heart rhythm until after they suffer a stroke.”

Cardiio Rhythm can detect irregular heartbeats by focusing the smartphone camera on someone’s face or by having them place a finger on the camera. When the person’s heart beats, the app captures the blood volume data in pulse waveforms. Machine learning algorithms analyse those waveforms to detect signature pulse patterns of AF. “We’re not trying to replace the ECG, but help people take early action,” Poh says.

In a 2016 study presented at the American Heart Association’s Scientific Sessions led by Bryan Yan, an associate professor at The Chinese University of Hong Kong, Cardiio Rhythm was used to screen 85 hospital patients for AF against an ECG, by having them face an iPhone camera immediately following an ECG reading.

The app was 95% as accurate as the ECG in detecting AF. A larger study led by Chung-Wah Siu, a cardiology professor of the University of Hong Kong, used the app to screen 1,013 patients at risk of developing AF, with similar results reported in the Journal of the American Heart Association.

“We will release more publications in the future, because validation is very important,” Poh says. Then, the startup will work toward regulatory clearance. As a PhD student in the Harvard-MIT Division of Health Sciences and Technology program, Poh conducted his thesis on a wrist wearable that monitors electrical conductance of the skin to detect potentially deadly seizures. Developed with Media Lab Professor Rosalind Picard, the device is now being commercialised through Empatica.

As a side project at the Media Lab, however, Poh developed Cardiocam, a technology that captured heart and breathing rates through a webcam. He and lab partner Daniel McDuff PhD ’14 modified a technology called photoplethysmography, which measures pulse and blood oxygen by shining a red and infrared light into a patient’s fingertip and measuring the light absorbed by oxygenated and nonoxygenated hemoglobin.

Using advanced signal-processing methods, the team developed software that could extract the same pulse information using only ambient light at a distance to measure light reflection on skin.

The first application for the technology wasn’t an app but an entire system called the Medical Mirror. This involved an LCD monitor with a built-in webcam and a connected laptop running the software.

A two-way mirror was affixed to the front of the system, so people could see their reflection and the webcam could still detect a user’s heart rate, breathing rate, and beat-to-beat variations. The LCD monitor would then project that information onto the mirror.

The mirror attracted some media attention, and Poh formed a startup to commercialise a version of the technology with Yukkee Poh, then a Brigham and Women’s Hospital postdoctoral fellow, and Chia-Hsun Lee PhD’11, his labmate at the time. But building and selling a heart-rate-reading mirror wasn’t fiscally feasible for a young company with budget constraints.

In January 2012, the startup, then named Cardiio, entered the Rock Health digital health incubator in San Francisco and released its app, which was one of the first of its kind. Since then, it has garnered millions of downloads worldwide.

Soon after the app’s release, Cardiio returned to Cambridge, inspired to connect with major hospitals to pursue its clinical app. The startup is now partnered with Massachusetts General Hospital and two hospitals in Hong Kong.

With Cardiio Rhythm on the rise, Poh says his startup hopes to serve as a type of intermediate for consumers, between widely available fitness apps and traditional medical devices that screen for disease, which are expensive and often require prescriptions to use.