Bridging the Gap Between Wearables and Healthcare

Image showing heart rate and activity on the Kardia app

Wearable devices like Fitbit® and Apple Watch® are inexpensive and widely available. The greatest challenge to using wearable devices in healthcare is drawing meaningful medical conclusions from the data they collect, and providing reliable, trustworthy information for patients and doctors to act on.

In cardiology, the gold standard for diagnosis of cardiac arrhythmias and other abnormalities is an electrocardiogram (ECG). With KardiaBand™ and SmartRhythm™ monitoring, AliveCor is releasing the world’s first cardiac health platform for the Apple Watch, that combines an FDA-cleared ECG device and analysis algorithms with state-of-the-art artificial intelligence models for tracking heart rate and activity data from the Apple Watch.

SmartRhythm monitoring involves a deep neural network that runs directly on the Apple Watch, constantly acquiring data from the Watch’s heart rate sensor and accelerometer. It compares your heart rate and changes in your heart rate over time to what it expects from your minute-by-minute level of activity, and gives you a graphical display of where your heart rate falls within the boundaries predicted by the neural network. When the network sees a pattern of heart rate and activity that it does not expect, it notifies you to take an ECG.

Image showing heart rate and activity on the Kardia app

When you feel symptoms, get a SmartRhythm notification, or any time you like, you can immediately take a 30-second ECG using the KardiaBand sensor attached to the watchband. AliveCor’s FDA-cleared algorithms run directly on Apple Watch and can instantly recognize Normal Sinus Rhythm, and check for Possible Atrial Fibrillation (AF), the most common type of serious arrhythmia.

Three watch faces showing an ECG recording, a normal sinus rhythm, and possible atrial fibrillation

How does it work?

To understand how Kardia for Apple Watch works, let’s start by talking about your heart, how the Apple Watch and other wearable devices can measure your heart rate, and how an ECG is different from the information you get from a heart rate sensor alone.

Your heart is a pump. With each beat of your heart, blood is pumped through your arteries and causes them to expand. In the time between beats, your arteries relax again. On the underside of the Apple Watch is a sensor, called a photoplethysmogram (PPG), that uses green and infrared LEDs to shine light onto your skin, and detects the small changes in the amount of light reflected back as your arteries expand and relax with each beat of your heart. Using this sensor, the Apple Watch can tell how fast your heart is beating, and how your heart rate changes over time.

But, your heart rate does not tell everything there is to know about your heart. The PPG sensor on the Apple Watch can only see what happens after each heartbeat, as blood is pumped around your body. It can’t tell you anything about what is making your heart beat, or about what happens inside your heart during each beat. An ECG is very different, and tells you a lot more!

Three hearts showing a P-Wave, QRS-Complex, and a T-Wave

An ECG measures the electrical activity in your heart muscles. It detects the small pulse of electricity from the sinoatrial node (the body’s natural pacemaker, which normally initiates each heartbeat) and the large electrical impulses produced as the lower chambers of the heart (the ventricles) contract and relax. By looking at an ECG, a doctor can discern a wealth of information about the health and activity of your heart muscle, much more than you can tell from your heart rate alone. ECGs are the required gold standard for diagnosis of arrhythmias and many cardiac abnormalities, and can even be used to see evidence of acute heart attacks and even events that have occurred in the past.

Research has shown that taking frequent ECGs increases the likelihood of detecting certain arrhythmias, and decreases the mean time to diagnosis1.

SmartRhythm Monitoring

There are ECG products that provide continuous monitoring, but they have limited lifetimes, are highly sensitive to proper placement, are uncomfortable to use, are invasive and/or expensive; for example, ECG patches or Holter monitors can only be worn for a very limited time, and implanted loop recorders require surgery. A PPG sensor like the one in Apple Watch is non-invasive, convenient and can be worn continuously. However, the PPG signal does not provide anything near the diagnostic capability of an ECG.

At AliveCor, we wanted a way to get the best of both worlds - by using PPG that can keep an eye on your heart rate all the time, and notifying you when your heart rate and other factors such as activity don’t seem to agree, so that you can easily take a high-quality, medical-grade ECG. That’s the motivation behind SmartRhythm monitoring.

How did we build SmartRhythm monitoring?

We started by thinking about what factors influence heart rate, and how we could make use of the kind of heart rate data we can get from the Apple Watch PPG sensor. The association of arrhythmias with specific patterns of heart rate and heart rate variability is well-known from earlier research at AliveCor and elsewhere.2 It has also been noted that while these patterns may be correlated to arrhythmias, they are not specific, i.e. similar patterns may occur in perfectly normal situations. Since people wear their Apple Watch while doing all sorts of things, we knew that most of the changes in their heart rates would come simply from normal daily activity. Years ago, we started working on (and patented3) using the concept of heart rate/activity discordance, i.e. a measured heart rate that is inconsistent with your current level of activity, to identify times when a user should take an ECG.

Aggregated heart rate data from number of steps and heart rate in the last 5 minutes

This graph illustrates what some of our early data analysis looked like. The Y axis represents heart rate, and the X axis represents the number of “steps” taken in the past 5 minutes, estimated by the Apple Watch accelerometer. It’s clear that there’s a direct relationship between heart rate and activity, but also a wide variation in the heart rates that correspond to any given activity level. As we looked deeper into the data we acquired, we could see interactions between all the various factors that influence heart rate, including activity, stress, time of day, consumption of caffeine, fitness level, etc. We quickly realized that we would need more sophisticated models to capture the complexity of the data we were seeing, and so we turned to approaches based on deep learning.

SmartRhythm monitoring uses an autoregressive, deep neural network that can learn the normal relationship between heart rate and activity, and notify you when it sees an unexpected pattern. This is a type of unsupervised learning. When most people think about machine learning, they imagine tasks like image recognition, where a computer is trained to identify specific, named objects in an image. This is called supervised learning. SmartRhythm monitoring uses techniques from unsupervised learning to learn for itself what a normal heart rate and activity pattern looks like, and notifies you when the actual data from the Apple Watch doesn’t match what the SmartRhythm model expects to see.

In particular, we used a specific neural network configuration called a Nonlinear Autoregressive Network (NAN). SmartRhythm monitoring runs continuously on your Apple Watch and processes heart rate data from the Apple PPG sensor, and activity data from the accelerometer.

Diagram showing how SmartRhythm works

The SmartRhythm system works by looking at your most recent 5 minutes of activity data (1), and based on the trained model of normal heart rate and activity patterns as well as your historical heart rate (2) and activity (3) data, it makes a prediction for what it expects your most recent 5 minutes of heart rate values to be.

SmartRhythm monitoring then compares the characteristics of the heart rate it predicted for you, with your actual heart rate values recorded during those 5 minutes. If your heart rate pattern differs significantly (much higher, lower, and/or differing variance) from what the network expects it to be, it notifies you to take an ECG.

All of this works without ever explicitly teaching the neural network what is “normal” and what isn’t. In fact, we trained this neural network simply by providing a large number of healthy volunteers with Apple Watches that recorded their heart rate and activity patterns while performing regular daily activities, including exercise and sleep.

Internally, the neural network uses a component called a Gaussian Mixture Model (GMM) to produce a probability distribution over expected heart rates. The diagram below shows what the network’s prediction looks like for a user during a workout.

Network prediction during exercise

Some Examples

SmartRhythm monitoring has been developed and tested in a wide variety of situations. We’ll have more to share on this in future articles, but for now, here are some examples from our engineering tools of what SmartRhythm monitoring data looks like. Activity is shown in light blue bars, the green band is a visualization of the heart rate predictions, the dark green bars show the heart rate range (1 minute per bar), and the red dotted line shows where a notification to record ECG was triggered.

An ordinary day in the office
Normal activity bars

Exercise where the activity is sensed by the watch (e.g. healthy users walking or running)
Exercise activity bars

Exercise where the activity is not accurately sensed (e.g. healthy users weightlifting, or using a stationary bike)
Exercise where activity is not accurately sensed

Occurrence of arrhythmias (From patients with episodes of cardiac arrhythmias verified by implantable loop recorder data)
Arrhythmia activity bars

As noted earlier, taking frequent ECGs can help you better manage your heart health, and capturing an ECG specifically during times of heart rate/activity discordance may be useful. However, receiving a SmartRhythm notification does not necessarily mean that something is wrong. There are many perfectly normal situations that can cause a heart/rate activity discordance, including exercise the Apple Watch can’t detect, stress or anxiety, consumption of caffeine or alcohol, and even motion artifact from wearing the watch band too loosely. Conversely, not getting a SmartRhythm notification does not necessarily mean that everything is normal, since the PPG is only measuring your heart rate and does not capture the full complexity of your heart. SmartRhythm monitoring should be used as an additional means to capture frequent ECGs. When there is a potential heart problem, either identified by Kardia’s ECG analysis algorithms or because you feel symptoms, a record of ECGs serves as actionable documentation for a medical professional to diagnose and come up with a treatment plan.

What does it look like on Apple Watch?

The SmartRhythm monitoring home screen shows your heart rate and activity over the past hour, in 1-minute intervals. Each interval bar on the graph shows you the minimum and maximum values your heart rate reached in that period. The outline around the interval bars is the prediction band. This is a simplified rendering of the output of the SmartRhythm network’s Gaussian Mixture Model, which shows you the upper and lower limits of the heart rates SmartRhythm monitoring has predicted based on your current level of activity. Whenever your heart rate doesn’t match the SmartRhythm predictions, it will highlight those interval bars in orange. This can happen briefly for a variety of reasons, but if the disagreement is especially large or lasts for a long time, we’ll notify you to take an ECG, and you’ll see an orange symbol on the chart.

Diagram detailing the UI of the Kardia App

Performing an ECG

Kardia for Apple Watch is designed to let you take an ECG anywhere, for any reason. Using your Kardia Band sensor, you can immediately take an ECG and in 30 seconds receive a result from our FDA-cleared ECG analysis algorithms. You can also forward your ECG to your doctor for a professional opinion.

EKG Sensor module for the KardiaBand

KardiaBand is a self-contained, FDA-cleared, miniaturized ECG device. Powered by an internal lithium battery with a lifetime of 1-2 years, the sensor in KardiaBand is always ready to use - with the recording screen open on your Apple Watch, simply touch your index finger to the KardiaBand sensor to start a recording. The live view shows your ECG in real time, and after a 30 second recording, our FDA-cleared machine learning algorithms run directly on Apple Watch and immediately report a result: Normal if our algorithms indicate your ECG is normal, Possible Atrial Fibrillation if our algorithms detect signs that you may be experiencing AF, Unclassified if the signal falls outside the range we can confidently classify, and Unreadable if there was too much interference or motion artifact in the signal to analyze.

If you don’t have KardiaBand, you can still use SmartRhythm monitoring on the Apple Watch: Simply use KardiaMobile to take an ECG on your iPhone instead of using your watch.

What’s Next

Kardia for Apple Watch is just the first step in bridging the gap between consumer wearable devices and the science of healthcare.

The availability of a wearable, mobile ECG device has the potential to dramatically improve early detection of arrhythmias. Many arrhythmias start as occasional, intermittent problems that usually don’t show up in a doctor’s office and are notoriously difficult to diagnose. For example, from initial symptom presentation, it takes on average 1.7 years4 to diagnose atrial fibrillation. In this period, the patient is at an elevated risk of stroke and other serious complications.

We are incredibly excited about the potential for Kardia for Apple Watch to bring these numbers down by allowing patients to immediately take a gold-standard ECG whenever and wherever they want, and for SmartRhythm monitoring to help patients identify times when they might not have thought to take an ECG otherwise.

Beyond detection of arrhythmias, AliveCor is working on other technologies that utilize the power of machine learning and wearable ECG devices like KardiaBand to expand the clinical and diagnostic significance of the ECG even further. For example, in partnership with the Mayo Clinic, we’re working towards reliable detection of life-threatening electrolyte abnormalities, like hyperkalemia, directly from an ECG, and screening tools that may one day enable detection of genetic abnormalities, like congenital long QT syndrome. The ECG has been around for more than 100 years. New techniques like machine learning may enable us to see what was formerly invisible to human eyes, and this may save millions of lives.

AliveCor®, KardiaBand™ and SmartRhythm™ are trademarks of AliveCor, Inc. Apple Watch® is a trademark of Apple Inc.
Fitbit® is a trademark of Fitbit, Inc.

  3. U.S. Patent Nos. 9,572,499 and 9,839,363, and additional pending patent applications
  4. Out of Sync: The State of AFib in America Survey. March-April, 2009