Welcome to the third part of this series examining: are fitness trackers fit to track your fitness? In this three-part series, we are reviewing the scientific data on the accuracy of such trackers for the three most popular uses: 1) activity/step count tracking, 2) heart rate tracking, and 3) sleep tracking.

In our first part, we examined activity/step count tracking noting that research suggested that most activity trackers seem to track steps relatively accurately, BUT it’s questionable whether this information prompted a change in people’s behavior or activity.

In our second part, we examined heart rate tracking and found that current wrist-based heart rate devices are inadequate to track heart rate –, particularly when the user is exercising.

Today we examine sleep tracking.

Sleep tracking by Americans using consumer wearables has become an incredibly common activity. Forrester research estimated back in 2015 that 10% of US adults use a wearable fitness/sleep tracking device regularly and up to 50% indicated they would consider one. Nearly all measures of sleep using wearables are derived from the principles of body movement developed many years ago by researchers using accelerometers (movement detection sensors). The scientific name for this is called actigraphy. Although actigraphy has been well validated with medical grade devices (often costing >$1000), actigraphy-based measurements in consumer devices are not well validated unlike the testing done on step counts or heart rate in consumer devices (and wearables).

One recent study published in 2015 in the journal Sleep (Meltzer et al) evaluated the the accuracy of Fitbit compared to medical-grade actigraphy devices, as well as to the gold standard for sleep monitoring, polysomnography (PSG). PSG is performed in a specialized sleep laboratory with over a dozen sensors and electrodes attached to a person. Researchers looked at three common measures that are often displayed or reported, namely, total sleep time (TST), wake after sleep onset (WASO), and sleep efficiency (SE – measured as a percentage of TST divided by time of lights-out to lights-on). The study foundthat PSG and medical-grade actigraphy agreed with each other. However, Fitbit Ultra, independent of how sensitive it was set, either overestimated or underestimated the three measures above by approximately 10-20%. In real numbers, Fitbit Ultra in Sensitive mode would underestimate TST by over 100 minutes and, therefore, underestimated SE by over 20% (83.4% with PSG vs 62.7% with Fitbit). This level of error could lead to misinterpretation and anxiety on the part of potential patients worried about whether they sleep well. Such anxiety is not just a theoretical risk.

Another study published in 2017 in the Journal of Clinical Sleep Medicine (Baron et al) reported three cases of patients who went for medical treatment based upon their consumer wearable sleep data. Patients had anxiety and often refused the medical treatment suggestions — like cognitive behavioral therapy — because they often felt distrust of their doctor when the accuracy of their wearable was questioned.

As one patient in this study noted to his physician, “I know you don’t like this, but my Fitbit tells me…” The study authors noted that this may have “affected the therapeutic relationship” between patient and doctor. They further described this behavior as follows:

We termed this condition “orthosomnia,” with “ortho” meaning straight or correct, and “somnia” meaning sleep, because patients are preoccupied or concerned with improving or perfecting their wearable sleep data.

In conclusion, sleep testing with consumer wearables is not well-validated to measure clinical health or medical conditions. In fact, the handful of studies that have been done suggest that not only could the data be inaccurate, but could also result in anxiety and potential negative medical consequences to the patient such as “orthosomnia”. The fact that medical-grade actigraphy devices already exist suggests that companies making consumer wearables could produce accurate devices if the market demanded the accuracy.

My recommendation: demand accuracy in your sleep tracking devices – then sleep well knowing you did what you could!