We discuss lessons learned on common shortcomings in applying user-centered design for digital health. Published in the July-September 2020 issue of IEEE Pervasive Computing.
The symposium, as part of GSA 2020, presents four projects on technology use of older adults and caregivers.
Abstract. Innovative technology can improve the lives of older adults, including those diagnosed with dementia, and their caregivers. Yet a lack of careful attention to preferences and needs of end-users and continuous updates to resources could leave consumers without a valuable user experience. This symposium will cover exemplar cases of innovative technologies, available resources, and current research. The first presentation will discuss virtual reality used among older adults with dementia and the opportunities to further explore it’s use as an intervention. The second presentation will share the process of seeking stakeholders’ preferences in the design specifications for a socially assistive robot and how the perspectives shaped the development of the Quori robot. The third presentation will focus on detailing the Information Quality Framework for Online Dementia Care Resources. The fourth presentation will discuss the implications of a systematic review that revealed researchers are reporting on all older adults within a category of 65 and older, thus failing to present differences among different cohorts. These presentations will all conclude with a discussion on opportunities for improvement in the respective areas.
Sefcik, J. S. & Huh-Yoo, J. (2020) Technology Use of Older Adults and Caregivers: Discoveries and Opportunities for Improvement. (Symposium Chair: Sefcik, J. S., Discussant: Huh-Yoo, J.) Gerontological Society of America 2020 Annual Scientific Meeting, Online Meeting. November 4-7.
Sefcik, J. S., Petrovsky, D. V., Cacchione, P. Z., Oh, S., & Demiris, G.
Virtual Reality Use Among Persons with Dementia: An Integrative Review
It is not well understood how virtual reality (VR) is currently used by older adults who have cognitive deficits due to dementia. The aim of this integrative review was to examine and report on published research exploring VR use among older adults with dementia. We searched 3 data bases for publications and used Whittemore and Knafl’s methodology for data extraction. Out of 122 articles we identified 24 that met our inclusion criteria, 15 published in 2012 and later. Most articles (12) used VR for assessment, and the others used VR for cognitive training (5) and as an intervention (3) (i.e., for exercise). Sample sizes were 30 or fewer persons with dementia. There is heterogeneity in the types of VR equipment, experiences, and foci of assessment through VR use. We identify opportunities to further explore VR as an intervention for persons with dementia to improve quality of life.
Pamela Z. Cacchione, Caio Mucchiani, Kristine Lima, Ross Mead, Mark Yim, Michelle J Johnson
Engaging End Users in Designing Systems and Hardware for a Socially Assistive Robot
Development of low-cost robots to assist older adults requires the input of end users: older adults, paid caregivers and clinicians. This study builds on prior work focused on the task investigation and deployment of mobile robots in a Program of All-inclusive Care for the Elderly. We identified hydration, walking and reaching as tasks appropriate for the robot and helpful to the older adults. In this study we investigated the design specifications for a socially assistive robot to perform the above tasks. Through focus groups of clinicians, older adults and paid caregivers we sought preferences on the design specifications. Using conventional content analysis, the following four themes emerged: the robot must be polite and personable; science fiction or alien like; depends on the need of the older adult; and multifaceted to meet the needs of older adults. These themes were used in the design and deployment of the Quori robot.
Smriti, Diva, DiMaria-Ghalili, Rose Ann, Gitlin, L. N., Sarcevic, A., Yan, E., Huh-Yoo, Jina
Information Quality Assessment Framework for Online Dementia Care Resources
Persons with dementia and caregivers can benefit from online resources. The quality and accessibility of these resources, however, can vary. We present work on the Information Quality Framework for Online Dementia Care Resources. To develop the framework, we first empirically examine resources being retrieved with query terms developed with a medical librarian. Searching one of the possible keyword combinations related to living with dementia on Google “Alzheimer AND financial planning” returned 18,900,000 results. Among the top 13 results on the first page of the search results, six were websites of government or non-profit organizations, four were for-profit companies, and three were advertisements. Out of eight unique organizations and companies, two provided support through online communities, but only one is active. The next steps include developing systematic ways to evaluate the credibility and accuracy of these resources, and search and test broader topics of dementia care resources online.
Choi, Hyung Wook; DiMaria-Ghalili, Rose Ann; Kelly, Mat; Poole, Alexander; Yan, Erjia; Huh-Yoo, J
Older Adults and Technology Use: A Systematic Literature Review
Researchers are increasingly interested in leveraging technology to support the physical and mental well-being of older adults. We systematically reviewed previous scholars’ criteria for sampling older adult populations, focusing on age cohorts (namely adults over 65) and their use of internet and smart technologies. We iteratively developed keyword combinations that represent older adults and technology from the retrieved literature. Between 2011 and 2020, 70 systematic reviews were identified, 26 of which met our inclusion criteria for full review. Most important, not one of the 26 papers used a sample population classification more fine-grained than “65 and older.” A knowledge gap thus exists; researchers lack a nuanced understanding of differences within this extraordinarily broad age-range. Demographics that we propose to analyze empirically include not only finer measures of age (e.g., 65-70 or 71-75, as opposed to “65 and older”), but also those age groups’ attitudes toward and capacity for technology use.
We reviewed Amazon Skill user reviews to understand usability and design requirements for designing health-supporting conversational agents. The paper will be published at EAI Pervasive Health 2020. Here’s a link to the paper–due to COVID-19, the conference has been postponed, and the organizers allowed us to publish our camera ready version without the copyright block added.
JY. Shin, J. Huh-Yoo, Designing Everyday Conversational Agents for Managing Health and Wellness: A Study of Alexa Skills Review, in: EAI Pervasive Health, 2020. In Press
In this paper, we discuss our interview results with Institutional Review Board (IRB) members in the U.S. on their perceptions on risks towards digital research data. The paper has been accepted with minor revision for ACM CSCW 2020 to be held this September. More detail will follow later.
J. Huh-Yoo, E. Rader, It’s the Wild, Wild, West: Lessons Learned From IRB Members’ Risk Perceptions Toward Digital Research Data, in: ACM CSCW, 2020.
Paper Presents empirical evaluation of using lightweight signal features from a set of complex activities during the meal (e.g., clattering sound, arm gestures of eating, human voice, TV sound) and fusing built-in sensor data of multiple mobile devices available in a family with a CRFs-based classifier.
Abstract. Monitoring the family mealtime activities enables the analysis of the previous daily routine, hence the positive changes can be made towards better relationships among family members and better physical/mental health. Moreover, the details of family mealtime activities provide important information for study in sociology and culture. This paper presents FamilyLog — a practical system to log family mealtime activities using smartphones and smartwatches. FamilyLog automatically detects and logs details of activities during the mealtime, including occurrence and duration of meal, conversations, participants, TV viewing etc., in an unobtrusive manner. Based on the sensor data collected from real families, we carefully design robust yet lightweight signal features from a set of complex activities during the meal, including clattering sound, arm gestures of eating, human voice, TV sound, etc. Moreover, FamilyLog opportunistically fuses data from built-in sensors of multiple mobile devices available in a family with a CRFs-based classifier. To evaluate the real-world performance of FamilyLog, we perform extensive experiments that consist of 77 days of sensor data from 37 subjects in 8 families with children. FamilyLog can detect those events with high accuracy across different families and home environments.
|Bi C, Xing G, Hao T, Huh J, Peng W, Ma M, Chang X. FamilyLog: Monitoring Family Mealtime Activities by Mobile Devices. IEEE Transactions on Mobile Computing. 2019 May 14.|
Paper Systematic review of facilitators and barriers to underserved populations’ consumer health technology adoption
Objectives: Underserved populations can benefit from consumer health informatics (CHI) that promotes self-management at a lower cost. However, prior literature suggested that the digital divide and low motivation constituted barriers to CHI adoption. Despite increased Internet use, underserved populations continue to show slow CHI uptake. The aim of the paper is to revisit barriers and facilitators that may impact CHI adoption among underserved populations.
Methods: We surveyed the past five years of literature. We searched PubMed for articles published between 2012 and 2017 that describe empirical evaluations involving CHI use by under- served populations. We abstracted and summarized data about facilitators and barriers impacting CHI adoption.
Results:From 645 search results, after abstract and full-text screening, 13 publications met the inclusion criteria of identify- ing barriers to and facilitators of underserved populationsâ€™ CHI adoption. Contrary to earlier literature, the studies suggested that the motivation to improve health literacy and adopt technology was high among studied populations. Beyond theÂ digital divide, barriers included: low health and computer literacy, challenges in accepting the presented information, poor usability, and unclear content. Factors associated with increased use were: user needs for information, user-access mediated by a proxy person, and early user engagement in system design.
Conclusions: While the digital divide remains a barrier, newer studies show that high motivation for CHI use exists. However, simply gaining access to technology is not sufficient to improve adoption unless CHI technology is tailored to address user needs. Future interventions should consider building larger empirical evidence on identifying CHI barriers and facilitators.
Keywords Medical informatics applications, consumer health information, ethnic groups, socioeconomic factors, minority groups, health disparities
Identified the features critical for predicting social support needs in online health communities.
Background: While online health social networks (OHSNs) serve as an effective platform for patients to fulfill their various social support needs, predicting the needs of users and providing tailored information remains a challenge.
Objective: The objective of this study is to discriminate important features for identifying users’ social support needs based on knowledge gathered from survey data. This study also provides guidelines for a technical framework which can be used to predict users’ social support needs based on raw data collected from OHSNs.
Methods: We initially conducted an online survey with 184 OHSN users. From this survey data, we extracted 34 features based on five categories: (1) demographics, (2) reading behavior, (3) posting behavior, (4) perceived roles in OHSNs, and (5) values sought in OHSNs. Features from the first four categories were used as variables for binary classification. For the prediction outcomes, we used features from the last category: the needs for emotional support, experience-based information, unconventional information, and medical facts. We compared 5 binary classifier algorithms: Gradient Boosting Tree, Random Forest, Decision Tree, Support Vector Machines, and Logistic Regression. We then calculated the scores of the area under the ROC curve (AUC) to understand the comparative effectiveness of the used features.
Results: The best performance was AUC scores of 0.89 for predicting users seeking emotional support, 0.86 for experience-based information, 0.80 for unconventional information, and 0.83 for medical facts. With the gradient boosting tree as our best performing model, we analyzed the strength of individual features in predicting one’s social support need. Among other discoveries, we discovered that users seeking emotional support tend to post more in OHSNs compared to others.
Conclusions: We developed an initial framework for automatically predicting social support needs in OHSNs using survey data. Future work should involve non-survey data to evaluate the feasibility of the framework. Our study contributes to providing personalized social support in OHSNs.
Min-Je Choi, Sung-Hee Kim, Sukwon Lee, et al. 2017. Toward Predicting Social Support Needs in Online Health Social Networks. Journal of Medical Internet Research (forthcoming). http://doi.org/10.2196/jmir.7660
The paper investigated whether conversations on online breast cancer community contribute to people choosing an aggressive, higher risk surgery.
Background: The increased uptake of contralateral prophylactic mastectomy (CPM) among breast cancer patients remains poorly understood. We hypothesized that the increased rate of CPM is represented in conversations on an online breast cancer community and may contribute to patients choosing this operation.
Methods: We downloaded 328,763 posts and their dates of creation from an online breast cancer community from August 1, 2000 to May 22, 2016. We then performed a keyword search to identify posts which mentioned breast cancer surgeries: contralateral prophylactic mastectomy (n=7,095), mastectomy (n=10,889) and lumpectomy (n=9,694). We graphed the percentage of CPM-related, lumpectomy-related and mastectomy-related conversations over time. We also graphed the frequency of posts which mentioned multiple operations over time. Finally, we performed a qualitative study to identify factors influencing the observed trends.
Results: Surgically-related posts (e.g., mentioning at least one operation) made up a small percentage (n=27,678; 8.4%) of all posts on this community. The percentage of surgically related posts mentioning CPM was found to increase over time, whereas the percentage of *Revision version w/ Markings surgically-related posts mentioning mastectomy decreased over time. Among posts that mentioned more than one operation, mastectomy and lumpectomy were the procedures most commonly mentioned together, followed by mastectomy and CPM. There was no change over time in the frequency of posts that mentioned more than one operation. Our qualitative review found that the majority of posts mentioning a single operation were unrelated to surgical decision-making; rather the operation was mentioned only in the context of the patient’s cancer history. Conversely, the majority of posts mentioning multiple operations centered around the patients’ surgical decision-making process.
Conclusions: CPM-related conversation is increasing on this online breast cancer community, while mastectomy-related conversation is decreasing. These results appear to be primarily informed by patients reporting the types of operations they have undergone, and thus appear to correspond to the known increased uptake of CPM.
R.A. Marmor*, W. Dai, X. Jiang, S. Wang, S.L. Blair, J. Huh, Increase In Contralateral Prophylactic Mastectomy Conversation Online Unrelated To Decision-Making, Journal of Surgical Research. 218 (2017).
This paper presents evaluation of the FamilyLog system with 37 subjects from 8 families. The system automatically detects activities during the mealtime, including occurrence and duration of meal, conversations, participants, and TV viewing using acoustic data from phones and smartwatches. [paper]
FamilyLog: A Mobile System for Monitoring Family Mealtime Activities
Research has shown that family mealtime plays a critical role in establishing good relationships among family members and maintaining their physical and mental health. In particular, regularly eating dinner as a family significantly reduces prevalence of obesity. However, American families with children spend only 1 hour on family meals while three hours watching TV on an average work day. Fine-grained activity logging is proven effective for increasing self-awareness and motivating people to modify their life styles for improved wellness. This paper presents FamilyLog – a practical system to log family mealtime activities using smartphones and smartwatches. FamilyLog automatically detects and logs details of activities during the mealtime, including occurrence and duration of meal, conversations, participants, TV viewing etc., in an unobtrusive manner. Based on the sensor data collected from real families, we carefully design robust yet lightweight signal features from a set of complex activities during the meal, including clattering sound, arm gestures of eating, human voice, TV sound, etc. Moreover, FamilyLog opportunistically fuses data from built-in sensors of multiple mobile devices available in a family through an HMM-based classifier. To evaluate the real-world performance of FamilyLog, we perform extensive experiments that consist of 77 days of sensor data from 37 subjects in 8 families with children. Our results show that FamilyLog can detect those events with high accuracy across different families and home environments.
C. Bi, G. Xing, T. Hao, J. Huh, W. Peng, M. Ma, FamilyLog: A Mobile System for Monitoring Family Mealtime Activities, in: IEEE International Conference on Pervasive Computing and Communications 2017, Institute of Electrical and Electronics Engineers Inc., 2017: pp. 21–30.
Our paper, ‘“How Did We Get Here?”: Topic Drift in Online Health Discussions’, has been accepted to the Journal of Medical Internet Research.
Albert Park, Andrea Hartzler, Jina Huh, Gary Hsieh, David McDonald, Wanda Pratt.“How Did We Get Here?”: Topic Drift in Online Health Discussions’. J Med Internet Res (forthcoming). doi:10.2196/jmir.6297 http://dx.doi.org/10.2196/jmir.6297
Background: Patients increasingly use online health communities to exchange health information and peer support. During the progression of health discussions a change of topic—topic drift—can occur. Topic drift is a frequently occurring phenomenon that is linked to incoherence and frustration in online communities and other forms of computer-mediated communication. For sensitive topics, such as health, such drift could have life-altering repercussions, yet topic drift has not been studied in these contexts.
Objective: Our goals were to understand topic drift in online health communities, and then to develop and evaluate an automated approach to detect both topic drift and efforts of community members to counteract such drift.
Methods: We manually analyzed 721 posts from 184 threads from seven online health communities within WebMD to understand topic drift, members’ reaction towards topic drift, and their effort to counteract topic drift. Then, we developed an automated approach to detect topic drift and counteraction effort. We detected topic drift by calculating cosine similarity between 229,156 posts from 37,805 threads and measuring change of cosine similarity scores from the threads’ first posts to their sequential posts. Using a similar approach, we detected counteractions to topic drift in threads by focusing on the irregular increase of similarity scores compared to the previous post in threads. Finally, we evaluated the performance of our automated approaches to detect topic drift and counteracting efforts by using a manually-developed gold standard.
Results: Our qualitative analyses revealed that in threads of online health communities, topics change gradually, but usually stay within the global frame of topics for the specific community. Members showed frustration when topic drift occurred in the middle of threads, but reacted positively to off-topic stories shared as separate threads. Although all types of members helped to counteract topic drift, original posters provided the most effort to keep threads on topic. Cosine similarity scores show promise for automatically detecting topical changes in online health discussions. In our manual evaluation, we achieve an F1-score of .71 and .73 for detecting topic drift and counteracting effort to stay on topic, respectively.
Conclusions: Our analyses expand our understanding of topic drift in a health context and highlight practical implications, such as promoting off-topic discussions as a function of building rapport in online health communities. Furthermore, the quantitative findings suggest that an automated tool could help detect topic drift, support counteraction efforts to bring the conversation back on topic, and improve communication in these important communities. Findings from this study have the potential to reduce topic drift and improve online health community members’ experience of computer-mediated communication.