📚 Reading Group Agenda

Disha Rana

Topic: Uncertainty Signals in Generative Chatbots

Hao Yuan

My recent work focuses on exploring computational methods, which include LLM and linguistic features analysis, to find the association between PLWDs' cognitive abilities and dementia stages, AND their writing. This project idea is developed based on our previous project that investigates PLWDs’ perspectives on their relationship with family caregivers by examining their posts on the dementia subreddit. In addition to expressing needs to maintain emotional connection with their family throughout their trajectory of decline, PLWDs describe how they are experiencing progressive decline in, for example, memory and language. These self-reports suggest that PLWDs’ writing can provide a valuable source of longitudinal data that reflects how their cognitive abilities change over time. This thus motivates us to explore how to make use of computational tools to find the association between PLWDs' cognitive abilities and dementia stages AND their writing. For the LLM part, we are designing an agentic system that automatically identifies and extracts PLWDs’ self-reported cognitive and functional symptoms of decline and self-expressed emotions in their online forum posts. The extraction process is based on an expert-reviewed codebook that contains definitions of commonly-seen symptoms of decline in PLWDs and their representative examples. For the linguistic feature analysis part, we extract features that have been used in prior studies for examining associations between PLWDs’ language abilities and dementia diagnosis. We extend these speech data-based studies by exploring these features in the written text of posts made by PLWDs. In particular, we find statistically significant differences in multiple feature scores between PLWDs’ posts and healthy users’ posts. We also found statistically significant differences in linguistic feature scores between PLWDs’ posts written before and after their first self-reported symptoms of decline. Together, these two approaches demonstrate how PLWDs’ writing can be associated with cognitive abilities and potentially used to infer their stages of decline.

Recap by Zoom AI

This was the first meeting of a reading group event where graduate students presented their research projects. Lu introduced the group and its purpose of providing a regular space for members to practice reading and presentation skills. The first presenter was Disha, who shared her summer research project with Professor Tiffany Li, examining how AI chatbots affect learning when they display uncertainty signals to learners. Disha explained that despite uncertainty signals, novice learners still struggle to catch errors, and her study aims to identify the specific barriers preventing learners from verifying chatbot responses. Hao then presented his computational research on using large language models and linguistic features analysis to find associations between people living with dementia's writing and their cognitive abilities and dementia stages. Hao described his work on developing an agentic system to automatically identify and extract self-reported cognitive symptoms from online forum posts on the dementia subreddit. The meeting included detailed discussions about methodology, data collection challenges, and potential clinical applications of both presentations.

Nikitha Preetham

I am currently working on research work in collaboration with Psyche Care, a digital mental health care and coaching provider for caregivers of youth with behavioral issues and at risk. The research aims to increase caregiver engagement by developing tailored engagement strategies and real-time data collection methods. In pursuit of this work, I am exploring areas of communication that can be automated by AI powered digital tools and what human context can be lost if automated. For our future work, we plan on conducting user interviews to assess caregiver preferences and experiences regarding engagement and data collection methods as well as hope to conduct a study to understand caregiver-provider engagement with the addition of an AI chatbot to augment personalized communication and engagement.

Chukwuemeka Ugwu

Topic: Agentic AI for Dementia Care: The Next Frontier for Ubiquitous Health Intelligence

Dementia care is emerging as one of the most important yet insufficiently addressed challenges for ubiquitous health AI. Effective dementia management requires continuous interpretation of diverse and evolving signals, including speech patterns, daily behavior, caregiver observations, neuroimaging findings, and electronic health records, collected over months or even years of care. Conventional clinical AI systems have improved isolated prediction tasks, but they remain limited in their ability to translate fragmented multimodal information into clinically meaningful actions. Recent progress in agentic AI offers a new direction by enabling large language model systems to retrieve evidence, coordinate multiple analytical tasks, and generate sequential care recommendations. Despite this promise, current dementia AI efforts remain disconnected across modalities, with neuroimaging, speech, and EHR-based systems advancing independently rather than functioning as an integrated reasoning partner for clinicians and caregivers. The major barrier to clinical translation is no longer prediction accuracy alone, but the lack of clinically embedded agentic architectures that can unify passive sensing, multimodal reasoning, and human-centered trust across the relationships among clinicians, caregivers, and people living with dementia. This presentation outlines clinically embedded agentic architectures that transform passive monitoring into interpretable support for long-term dementia management.

Recap by Zoom AI

The group held a meeting where members shared their recent research progress. Lu discussed exploring mental models in relation to generative AI, explaining how users interpret AI applications through multiple schemata. Hao described refining hypotheses related to linguistic features in dementia-related writing. Ayşenur introduced herself as a new PhD student working with Pro. Tiffany Li on AI in education, focusing on how people can refine AI responses. Chukwuemeka shared his work on a literature review for dementia projects, examining specific features used to classify dementia. Nikitha presented her progress on analyzing medical datasets involving dyadic conversations between caregivers and providers to identify engagement patterns and automation challenges.

Lu Wang

Topic: From Understanding to Anchoring: Informal Caregivers' Mental Models of Generative Artificial Intelligence-based Conversational Agents for Problem-Solving

https://dl.acm.org/doi/full/10.1145/3774935.3803056

Users face challenges in understanding the capabilities of Generative Artificial Intelligence-Based Conversational Agents (GCAs), learning to interact with them, and evaluating GCA outputs. Understanding and designing around users’ mental models of GCAs could help address such challenges. This doctoral research investigates informal caregivers’ mental models of GCAs for multiple problem-solving tasks and aims to promote effective and safe use of GCAs through user modeling and adaptive interface design.

Recap by Zoom AI

Lu presented her PhD proposal titled "From Understanding to Anchoring Informal Caregivers' Mental Models of Generative AI-Based Conversation Agents for Problem Solving." She discussed her motivation for studying how informal caregivers conceptualize generative AI-based conversational agents, explaining that traditional measurement methods like self-report surveys are insufficient for capturing the dynamic nature of mental models during real-time interactions. Lu shared findings from her semi-structured interview study with 16 informal caregivers, revealing diverse understandings of AI and conversational agents, and demonstrated how different mental models influence problem-solving strategies. Nikitha provided feedback suggesting Lu could improve the presentation by spending less time on theoretical foundations and more time on her specific research findings, while also discussing potential methodological approaches including Bayesian frameworks and causal models for inferring mental states.

Hao Yuan

Title: Differential linguistic features of verbal fluency in behavioral variant frontotemporal dementia and primary progressive aphasia

https://pubmed.ncbi.nlm.nih.gov/35416098/

Frontotemporal dementia (FTD) is an early-onset neurodegenerative disorder with a heterogeneous clinical presentation. Verbal fluency is regularly used as a sensitive measure of language ability, semantic memory, and executive functioning, but qualitative changes in verbal fluency in FTD are currently overlooked. This retrospective study examined qualitative, linguistic features of verbal fluency in 137 patients with behavioral variant (bv)FTD (n = 50), or primary progressive aphasia (PPA) [25 non-fluent variant (nfvPPA), 27 semantic variant (svPPA), and 34 logopenic variant (lvPPA)] and 25 control participants. Between-group differences in clustering, switching, lexical frequency (LF), age of acquisition (AoA), neighborhood density (ND), and word length (WL) were examined in the category and letter fluency with analysis of variance adjusted for age, sex, and the total number of words. Associations with other cognitive functions were explored with linear regression analysis. The results showed that the verbal fluency performance of patients with svPPA could be distinguished from controls and other patient groups by fewer and smaller clusters, more switches, higher LF, and lower AoA (all p < 0.05). Patients with lvPPA specifically produced words with higher ND than the other patient groups (p < 0.05). Patients with bvFTD produced longer words than the PPA groups (p < 0.05). Clustering, switching, LF, AoA, and ND-but not WL-were differentially predicted by measures of language, memory, and executive functioning (range standardized regression coefficient 0.25-0.41). In addition to the total number of words, qualitative linguistic features differ between subtypes of FTD. These features provide additional information on lexical processing and semantic memory that may aid the differential diagnosis of FTD.

Recap by Zoom AI

Hao presented a paper on linguistic features in verbal fluency tests to differentiate frontotemporal dementia subtypes. Hao explained the study's methodology, which analyzed clustering, switching, and linguistic variables like lexical frequency and age of acquisition in dementia patients and control participants. The discussion that followed covered Hao's application of similar linguistic features to study online forum posts by people with dementia versus healthy users, with participants questioning the methodology and suggesting alternative comparison groups.

Olzhas Yessenbayev

Topic: Digital Bites: exploring on-screen consumption feedback as support for screen-accompanied dining

People increasingly eat in front of screens, where divided attention pulls focus from the meal, loosening the link between the process of eating and its consequences: satiety, and eating regulation. That's because satiety and eating regulation are partly cognitive: they arise not only from physiological signals but also from an awareness of how much one has eaten, one's hunger, and one's eating goals — which screen distractions erode. Existing mindful-eating interventions in HCI largely target eating *behavior* — prompting diners to eat slower or look at their plates — which is hard to sustain and competes for attention with the screen. In this work, we explore targeting the *consumption awareness* directly: we render an ongoing consumption trace as an ambient, glanceable display of how much one has eaten so far, rendered on the screen so it accompanies the ongoing activity rather than interrupting it. In a preliminary within-subjects study (N=21), the cue increased attention to and memory of the meal and reduced intake by 16\%, with no loss of fullness or meal enjoyment, while remaining unobtrusive to screen use. Interviews showed it prompted diners to weigh their intake against their hunger and personal goals, supporting flexible regulation in either direction — including under-eaters who used it to eat more. We contribute consumption awareness as a new target for eating-regulation technology, and preliminary investigation of users' interactions with it.

Recap by Zoom AI

The meeting was a research presentation where Olzhas shared his work on "Digital Bytes," a system designed to help people regulate their eating behavior while engaging with screens. The system measures chewing activity using a camera and displays it as a progress bar on the screen to help users maintain awareness of their food consumption. During the presentation, Olzhas explained the study's methodology, which involved 21 participants eating pizza while reading articles, and demonstrated that the intervention resulted in people eating 16% less food while maintaining similar levels of enjoyment and satiety. The participants provided positive feedback, suggesting improvements such as adding goal-setting features, using vertical progress bars, and potentially applying the system to track water consumption. The group also discussed the study's limitations, including its controlled nature and the challenges of conducting eating research.

Lu Wang

Paper: Kleinberg, J., Ludwig, J., Mullainathan, S., & Raghavan, M. (2024). The inversion problem: Why algorithms should infer mental state and not just predict behavior. Perspectives on Psychological Science, 19(5), 827-838.

More and more machine learning is applied to human behavior. Increasingly these algorithms suffer from a hidden—but serious—problem. It arises because they often predict one thing while hoping for another. Take a recommender system: It predicts clicks but hopes to identify preferences. Or take an algorithm that automates a radiologist: It predicts in-the-moment diagnoses while hoping to identify their reflective judgments. Psychology shows us the gaps between the objectives of such prediction tasks and the goals we hope to achieve: People can click mindlessly; experts can get tired and make systematic errors. We argue such situations are ubiquitous and call them “inversion problems”: The real goal requires understanding a mental state that is not directly measured in behavioral data but must instead be inverted from the behavior. Identifying and solving these problems require new tools that draw on both behavioral and computational science.

Recap by Zoom AI

Lu presented a paper on the "inversion problem" in user modeling, explaining how behavioral data may not perfectly reflect users' mental states, and discussed how this concept applies to her research on caregivers' mental models when using GCA for caregiving tasks. Nikitha and Lu discussed the challenges of user education in generative AI interfaces like ChatGPT and Claude. Lu suggested implementing adaptive education through interface design and service recommendations to help users understand available features without overwhelming them. They agreed that AI systems should clearly inform users about their capabilities and limitations, with Lu providing an example of how DeepSeek proactively alerts users about disabled features. The discussion focused on finding the right balance between simplicity and user education to maintain low learning curves while ensuring effective usage of AI tools.

Topic:

Recap by Zoom AI