Understanding Tensions in Consumers’ Expectations of Sleep Tracking Devices’ Capabilities
June - September 2016
As an independent research project, I collaborated with Ruth Ravichandran, a PhD candidate at the UW Electrical Engineering Department working in Shwetak Patel's UbiComp Lab, in Summer of 2016. In this project, I learned and applied previously unexperienced research methods and became more confident as a qualitative researcher. I studied how commercially available sleep tracking tools give false hope and promises to their users even though sleep sensing technology is not up to par with clinical standards. I worked with eminent professors in the field (Drs. Shwetak Patel, Julie Kientz, and Laura Pina) as we came to understand the tensions in sleep tracking use. Our findings were written and submitted to CHI, where it was accepted for CHI 2017.
Sleep is an important aspect of our health, but it is difficult for people to track manually because it is an unconscious activity. The ability to sense sleep using technology has aimed to lower the barriers of tracking sleep. Although sleep sensors are widely available, their usefulness and potential to promote healthy sleep behaviors has not been fully realized. We found that the feedback provided by current sleep sensing technologies affects users’ perceptions of their sleep and encourages goals that are in tension with evidence-based methods for promoting good sleep health. We provided design recommendations for improving the feedback of sleep sensing technologies by bridging the gap between expert and user goals.
My primary role was as a qualitative researcher which contributed to the project greatly since Ruth had no prior experience in qualitative research. My responsibilities included scraping data from app stores and Amazon, preparing and conducting all interviews, preparing and disseminating the survey, and taking part in data analysis.
As part of my responsibilities, we contacted a well-known sleep blog who featured a story about the study. Read more here.
How are people currently using commercially available sleep sensors and making sense of feedback they provide?
What aspects of sleep sensing and feedback either facilitate or potentially undermine people’s ability to understand their sleep and achieve good sleep health?
What aspects of current sleep sensor technology designs are in line with evidence-based methods of understanding and promoting good sleep health?
Interviewed 5 sleep experts
Surveyed 87 participants
Interviewed 12 participants
6986 consumer product reviews for the most widely used commercial sleep sensing devices from Amazon and the 2 app stores.
Interviews with the sleep experts and a literature review were used to understand the capabilities of current sleep tracking technologies and the standards used in sleep labs. Other methods were used to gain an understanding of consumers’ expectations from sleep tracking at a consumer level.
App Reviews (6986 reviews)
Smart Alarm Clock (87)
Sleep Cycle (98)
Sleep Bot (98)
Sleep Cycle (84)
Sleep Tracker (1)
Sleep as Android (116)
Sleep as Android Paid (38)
Dedicated Devices (683)
Sense with Sleep Pill (290)
Withings Aura (215)
Fitness Trackers (5451)
Fitbit One (2113)
Fitbit HR (2452)
Misfit Shine (78)
Jawbone Up3 (808)
We found that the feedback provided by current sleep sensing technologies affects users’ perceptions of their sleep and encourages goals that are in tension with evidence-based methods for promoting good sleep health.
Self-trackers using sleep sensing technologies often develop broken mental models about what commercial sleep sensors are able to actually sense, how they work, and are frustrated with the lack of algorithmic transparency in sleep sensing technologies.
Self-trackers find it distracting when feedback emphasizes unconscious aspects of sleep, such as time in sleep stages, over aspects of their sleep they have the ability to control and improve.
Self-trackers can better understand and improve their overall sleep habits when feedback from sleep sensors focuses on duration, timing, and making connections to modifiable behaviors and sleep hygiene.
Our findings show that sleep sensors increase awareness in prioritizing sleep and help users address modifiable behaviors and their sleep hygiene. But, current technologies focus on sleep metrics people do not have control over to directly change (e.g., time in sleep stages) and this distracts users from focusing on aspects they have control over that improve sleep health.
Our guidelines draw from our results, and connect to evidence-based strategies that focus on sleep hygiene, modifiable behaviors, and the SATED framework for good sleep quality.
Include Subjective Sleep Quality Assessment
Contextualize Sleep Quality with Journaling
Focus on Actionable Feedback
Give Feedback in Ranges, not Single Point Values: to avoid false precision
Increase Transparency in Formulae and Algorithms
I found many learning opportunities in this project.
Scraping and applying product review data was a new method of research for me and involved countless hours reading, sorting, analyzing, and coding. As a method of elicitation, I was pleasantly surprised as to how in-depth some of the reviews are and it is something I can rely on to use again in the future if needed.
The interviews with the sleep experts were the most fascinating to me. They are at a unique position where they know these tools do not work, but using these technologies can mean users are stepping in the right direction in terms of their sleep health. It will be interesting where Ruth's research will take her, as her skills in Electrical Engineering will no doubt put our design recommendations to good use.