“How Did We Get Here?”: Topic Drift in Online Health Discussions

Our paper,  ‘“How Did We Get Here?”: Topic Drift in Online Health Discussions’, has been accepted to the Journal of Medical Internet Research.

Albert Park, Andrea Hartzler, Jina Huh, Gary Hsieh, David McDonald, Wanda Pratt.“How Did We Get Here?”: Topic Drift in Online Health Discussions’. J Med Internet Res (forthcoming). doi:10.2196/jmir.6297 http://dx.doi.org/10.2196/jmir.6297


Background: Patients increasingly use online health communities to exchange health information and peer support. During the progression of health discussions a change of topic—topic drift—can occur. Topic drift is a frequently occurring phenomenon that is linked to incoherence and frustration in online communities and other forms of computer-mediated communication. For sensitive topics, such as health, such drift could have life-altering repercussions, yet topic drift has not been studied in these contexts.

Objective: Our goals were to understand topic drift in online health communities, and then to develop and evaluate an automated approach to detect both topic drift and efforts of community members to counteract such drift.

Methods: We manually analyzed 721 posts from 184 threads from seven online health communities within WebMD to understand topic drift, members’ reaction towards topic drift, and their effort to counteract topic drift. Then, we developed an automated approach to detect topic drift and counteraction effort. We detected topic drift by calculating cosine similarity between 229,156 posts from 37,805 threads and measuring change of cosine similarity scores from the threads’ first posts to their sequential posts. Using a similar approach, we detected counteractions to topic drift in threads by focusing on the irregular increase of similarity scores compared to the previous post in threads. Finally, we evaluated the performance of our automated approaches to detect topic drift and counteracting efforts by using a manually-developed gold standard.

Results: Our qualitative analyses revealed that in threads of online health communities, topics change gradually, but usually stay within the global frame of topics for the specific community. Members showed frustration when topic drift occurred in the middle of threads, but reacted positively to off-topic stories shared as separate threads. Although all types of members helped to counteract topic drift, original posters provided the most effort to keep threads on topic. Cosine similarity scores show promise for automatically detecting topical changes in online health discussions. In our manual evaluation, we achieve an F1-score of .71 and .73 for detecting topic drift and counteracting effort to stay on topic, respectively.

Conclusions: Our analyses expand our understanding of topic drift in a health context and highlight practical implications, such as promoting off-topic discussions as a function of building rapport in online health communities. Furthermore, the quantitative findings suggest that an automated tool could help detect topic drift, support counteraction efforts to bring the conversation back on topic, and improve communication in these important communities. Findings from this study have the potential to reduce topic drift and improve online health community members’ experience of computer-mediated communication.

Art therapy for me

We see many adult coloring books in the bookstores these days, which are called art therapy books. It is probably an oversimplification of art therapy, but that is for another thread of conversation.

I purchased one of those books and started coloring with my daughter. I soon found myself not liking it because I have to only color where I am supposed to. All the lines were so thick and black. They were so definitive like I was not supposed to color outside those lines. I wanted to define where I can color and what the shapes are going to be. So this is my version of my own art therapy. This soothed me.


Lessons learned for online health community moderator roles: A mixed methods study of moderators resigning from WebMD communities

Our mixed methods paper on examining what happened when all staff moderators left WebMD online health communities in 2013 has been accepted to the Journal of Medical Internet Research!

Jina Huh, Rebecca Marmor, and Xiaoqian Jiang. 2016. Lessons Learned for Online Health Community Moderator Roles: A Mixed-Methods Study of Moderators Resigning From WebMD Communities. Journal of medical Internet research 18, 9: e247. http://doi.org/10.2196/jmir.6331 [pdf]


Background: Online health community (OHC) moderators help facilitate conversations and provide information to members. However, the necessity of the moderator in helping members achieve goals in receiving the support they need remains unclear, with some prior research suggesting that moderation is unnecessary or even harmful for close-knit OHCs. Similarly, members’ perceptions of moderator roles are underexplored. Starting January of 2013, WebMD moderators stopped working for WebMD communities. This event provided an opportunity for us to study the perceived role of moderators in OHCs.

Objective: We examine OHC members’ perception of OHC moderators by studying their reactions towards the departure of moderators in their communities. We also analyzed the relative posting activity on OHCs before and after the departure of moderators from the communities among all members and those who discussed moderators’ departures.

Methods: We applied mixed methods to studying all 55 moderated WebMD communities’ posts by querying terms relating to discussions surrounding moderators’ disappearance from the WebMD community. We performed open and axial coding and affinity diagramming to thematically analyze patients’ reactions to disappeared moderators. We analyzed the number of posts and poster groups (members and moderators) over time to understand posting patterns around moderators’ departure.

Results: From 821 posts under 95 threads retrieved, a total of 166 open codes were generated. The codes were then grouped into two main themes with six total sub-themes. First, patients attempted to understand why moderators had left and what could be done to fill the void of the missing moderators. During these discussions, the posts revealed that patients believed moderators played critical roles in the communities by: making the communities vibrant and healthy, finding solutions, and giving medical information. Some patients felt personally tied with moderators, expressing they would cease their community participation. Patients also indicated that moderators were not useful or sometimes even harmful for peer interactions. The overall community’s posting activity analysis showed no significant difference before and after the moderators’ departure. The overall posting activities of the communities were declining well before the moderators’ departure. This declining posting activities might be the reason WebMD removed the moderators.

Conclusion: Compassionate moderators who provide medical expertise, control destructive member posts, and help answer questions can provide important support for patient engagement in OHCs. Moderators are in general received positively by community members and do not appear to interfere with peer interactions. Members are well aware of the possibility of misinformation spreading in OHCs. Further investigation into the attitudes of less vocal community members should be conducted.

Personas in Online Health Communities

Our paper on personas in online health communities have been accepted to the Journal of Biomedical Informatics! We will keep you updated once the paper has the final camera-ready version.

Huh J, Kwon BC, Kim S-H, et al. Personas In Online Health Communities. J Biomed Inform. 2016. In Press. doi:10.1016/j.jbi.2016.08.019.

Many researchers and practitioners use online health communities (OHCs) to influence health behavior and provide patients with social support. One of the biggest challenges in this approach, however, is the rate of attrition. OHCs face similar problems as other social media platforms where user migration happens unless tailored content and appropriate socialization is supported. To provide tailored support for each OHC user, we developed personas in OHCs illustrating users’ needs and requirements in OHC use. To develop OHC personas, we first interviewed 16 OHC users and administrators to qualitatively understand varying user needs in OHC. Based on their responses, we developed an online survey to systematically investigate OHC personas. We received 184 survey responses from OHC users, which informed their values and their OHC use patterns. We performed open coding analysis with the interview data and cluster analysis with the survey data and consolidated the analyses of the two datasets. Four personas emerged—Caretakers, Opportunists, Scientists, and Adventurers. The results inform users’ interaction behavior and attitude patterns with OHCs. We discuss implications for how these personas inform OHCs in delivering personalized informational and emotional support.


TrajectoryExplorer: Breast cancer

We are developing a tool to help patients with breast cancer make decisions. To do this, we have been conducting foundational work, including understanding how patients perceive various surgical options, such as contralateral prophylactic mastectomy (CPM), which has become a controversial topic after Angelina Jolie’s announcement.

We are comparing various sources including online health communities, national trends in uptake of surgical options, and search logs to understand how the public perceives CPM and come to understand pros and cons for various decision making points in general. We systematically developed manual annotations around characterizing patients and their posts in online breast cancer communities. These data along with clinical data will be used to develop a prediction model that will help breast cancer patients understand other patients’ trajectory in comparison with their own. We started conducting interviews with breast cancer patients to understand their needs that will be incorporated into our design.

Our students (Lead: Rebecca Marmor) were included in the final 9 teams for the Design Challenge at AMIA 2016!

Related publications:

Team members:
Rebecca Marmor, MD (Surgery resident, Project lead)
Meera Meghunathan (Medical student)
Elizabeth S. Epstein (Medical student)
Kenneth Trang (High school intern, data manager)
Xiaoqian Jiang, PhD (Machine learning)
Shuang Wang, PhD (Machine learning)
Jihoon Kim, MS (Biostatistics)
Sarah Blair, MD (Oncology)
Jina Huh, PhD (Human Computer Interaction, Social Analytics)

Preuss high school students
Mitchell Boldin (University of Michigan, MSI)