Fitness trackers vary widely in accuracy, researchers find

May 26, 2017, 1:49 a.m.

Researchers from the Stanford School of Medicine have found that popular fitness trackers such as Fitbit generally record accurate heart rates, but the accuracy with which they measure calories burned is significantly worse.

In addition to a paper recently published in the Journal of Personalized Medicine, the team has created a website to make the data more accessible and included an option for laypeople to upload their own device’s data.

Professor of Cardiovascular Medicine, Genetics and Biomedical Data Science Euan Ashley’s team of 60 volunteers tested seven different devices: Apple Watch, Basis Peak, Fitbit Surge, Microsoft Band, Mio Alpha 2, PulseOn and the Samsung Gear S2. Other paper authors included graduate student Anna Shcherbina, visiting assistant professor Mikael Mattsson and senior research scientist Daryl Waggott.

Although six out of seven devices recorded heart rate with an error margin of less than 5 percent, the devices had a wide range of accuracies for recording energy expenditure, with the most accurate device off by an average of 27 percent and the least accurate device off by 93 percent.

The results lead one to question the usefulness of the devices’ data for users seeking to make informed decisions, Ashley said. Although manufacturers may test devices, consumers often do not know how the testing process occurred or how to interpret companies’ claims.

The 31 women and 29 men participating in the study wore the seven trackers while using a stationary bike or treadmill. To accurately test heart and metabolic rates, the researchers used medical-grade electrocardiographs and an instrument that measures oxygen and carbon dioxide in breath, which accurately portrays energy expenditure.

“The heart rate measurements performed far better than we expected, but the energy expenditure measures were way off the mark,” Ashley told Stanford Medicine. “The magnitude of just how bad they were surprised me.”

Ashley’s team did not reach conclusions as to why the devices were so bad at measuring calories burned, but Shcherbina noted that each device has a particular algorithm for relating movement to energy expenditure. The devices measure heart rates directly, but the energy expenditure algorithms seem to have a wide margin for improvement, researchers said.

“My take on this is that it’s very hard to train an algorithm that would be accurate across a wide variety of people because energy expenditure is variable based on someone’s fitness level, height [and] weight,” Shcherbina said. 

Researchers plan to implement the study in a day-to-day context in which participants wear their devices during normal activity and exercise outside the lab. The same medical-grade instruments will be used to record participants’ heart rate and energy expenditure data, but researchers hope the results will illuminate the devices’ capabilities and limitations in a broader variety of settings.

 

Contact Fiona Kelliher at fionak ‘at’ stanford.edu.

Login or create an account