User-Friendly Chatbot to Mitigate the Psychological Stress of Older Adults During the COVID-19 Pandemic: Development and Usability Study

Introduction

As COVID-19 spread worldwide, many governments implemented stay-at-home, social distancing, and self-isolation measures to protect the most vulnerable people, including older adults, for whom deteriorating physical health could result in both physical and mental challenges [1,2]. As one of the most vulnerable groups to experience negative outcomes of COVID-19, older adults were instructed to self-quarantine and isolate themselves from others to mitigate the risk of infection. Consequently, they missed out on their standard interactions with others and spent substantial amounts of time in isolation. Limited contact with other people can lead to loss of social support, which is especially important for older adults [1,2]. Indeed, older adults who experienced social isolation during the pandemic have been negatively impacted by feelings of increased loneliness and reduced quality of life. Furthermore, the risks faced by individuals living alone, including older adults, are particularly pronounced with more challenging economic conditions [3,4]. Loneliness is defined as a subjective feeling manifesting as uneasiness and unhappiness linked to a lack of connection or inclusiveness with others, whereas social isolation describes the objective state of an individual’s social environment and interactional patterns [5]. Growing evidence shows that “loneliness” is closely related to physical and mental health in older adults; stronger feelings of loneliness can lead to the development of unhealthy lifestyle habits (eg, smoking and drinking) and a reduction in engagement with health-positive behaviors (eg, exercise and a nutritious diet) [6,7]. Loneliness is also associated with decreased sleep and sleep efficiency and poor subjective sleep quality in middle-aged and older adults, along with correlations to cognitive dysfunction, disability, cardiovascular disease, increased mortality, and even suicide [8-10]. Loneliness is also associated with negative emotion processing in the brain [11] and is a significant risk factor for anxiety and depression in older adults [12,13].

The prevalence of anxiety disorders and depression among older adults is high. Approximately 2% of adults older than 55 years are diagnosed with major depression, and 10%-15% of older adults have depressive symptoms [14]. Anxiety disorders are the most common psychiatric diseases, with a lifetime prevalence of up to 30%. Treatment of depressive and anxiety disorders in older adults is difficult and common challenges include poor drug efficacy, adverse drug reactions, and common drug interactions caused by polypharmacy. Coping with loneliness among older adults has potential benefits for depression and anxiety, as loneliness can affect older people’s physical and mental health, triggering psychological vulnerability, causing brain alterations beyond those associated with normal aging, and reducing intellectual or psychological flexibility caused by discrete and chronic stress [15]. Given the serious impact of social isolation and loneliness on older adults’ mental and physical health, researchers must investigate ways to mitigate these adverse health outcomes. As an assistive technology, chatbots and conversational agents may provide companionship and reduce the loneliness experienced because of the social restrictions imposed during a pandemic [7].

In recent years, there have been breakthroughs in the application of deep-learning techniques for natural language processing, allowing a variety of applications to be used in medical communication, disease detection, and treatment. For instance, chatbots have been developed to assist patients with autism and to provide social-skill training for specific fields such as mental health [16]. There is also a publicly listed application that provides cognitive behavioral therapy to improve anxiety and depression symptoms. Chatbots are also used for veterans as a screening tool for posttraumatic stress syndrome, and users of such applications have compared its use to interacting with real people, highlighting the chatbot’s ability to more freely and accurately reflect their own symptoms [17]. Additionally, chatbots used for mental illness discharge preparation can offer time-saving benefits for nurses and allow users to repeat questions without time pressure [18]. Upon reviewing previous research, it is evident that most chatbot users are relatively young, with limited availability of chatbots specifically developed for older adults in psychiatric clinics. While an overwhelming number of mental health apps are available to the public, these apps seldom focus on the needs of the older adult population. Furthermore, older adults can be apprehensive about using apps containing too many complex functions. This situation highlights the need to design and develop an older adult–user-friendly chatbot geared toward the specific interests of this population, while promoting their physical and mental health.

To address this gap, this study investigated the perspectives of geriatric psychiatric outpatient users of a mental health chatbot during the COVID-19 pandemic and sought to develop a chatbot for older adult users with anxiety and depressive disorders. This chatbot aims to offer companionship and promote self-awareness regarding emotions, sleep patterns, and physical activity to improve overall health. Therefore, our primary research objective was to examine whether chatbots can mitigate the psychological stress that older adults experienced during COVID-19.

Methods

Ethical Considerations

Ethical approval was granted by the Institutional Review Board of Chang Gung Memorial Hospital (202001915B0). The participants were recruited from the psychiatric outpatient department of the hospital between September 2021 and March 2022. Informed consent was obtained from each participant before study initiation. Data were deidentified by assigning a unique identifier code to each participant; only the study moderator has access to the file linking the code to the personal information, which is stored in a locked cabinet. Participants were provided with a travel compensation of ~US $15 for completing the postintervention questionnaires; no other compensation was provided for participation in the study.

Participants

Participants comprised two groups of individuals aged either 65 years and older or younger than 65 years who were diagnosed with depressive or anxiety disorders, including persistent depressive disorder, major depressive disorder, generalized anxiety disorder, panic disorder, or adjustment disorder with anxiety or depression. Certified psychiatrists made diagnoses according to Diagnostic and Statistical Manual of Mental Disorders (Fifth Edition) (DSM-5) criteria.

During the time of the study, none of the participants had experienced suicidal ideation or had an increase in medication 3 months prior to the beginning of the study. Participants included in the study were literate, active mobile phone users with internet access, who had used the chatbot regularly for at least 4 weeks. We excluded patients who were not literate, those with severe physical illness, and those diagnosed with a major neurocognitive illness (eg, schizophrenia, bipolar disorder, brain injury, substance abuse, stroke history, and dementia). During the case referral period, only one patient was excluded due to a diagnosis of bipolar disorder. All participants completed a questionnaire regarding their demographic characteristics.

Health Promotion Chatbot

Chatbot Design

The LINE Official Account Manager was used to build the “Health Promotion” chatbot. LINE was used as it is currently the most popular chatbot for older adults in Taiwan. Previous research found that some older adults have difficulty keeping pencil-and-paper sleep diaries [19]. This difficulty can be ameliorated using mobile technologies; thus, we designed a user-friendly interface for older adults using buttons in the chatbot app. The proposed chatbot is multifunctional and includes health diary data collection and hygiene education functions. Figures 1-4 provide screenshots of the Health Promotion chatbot functions translated into English.

‎

‎

‎

‎

Monitoring and Recording Health Data

The Health Promotion chatbot sent daily messages to all participants and collected their health diary responses. The diaries tracked three key response areas, including mood, sleep, and daily activity. Figure 1 depicts the participants’ rating of their mood using five status terms: great, not bad, indifferent, sad, and angry. Sleep quality was rated as good, not bad, or sleepless (Figure 2). In addition to keeping an activity diary, the chatbot sent messages such as “Did you go out today?” or “Did you do some exercise? What kind of exercise?” (Figure 3). These messages helped guide older adult users to complete their activity information in the formatted “Message buttons.”

Providing Health Information and Advice

The main health promotion education program consisted of three topics: sleep hygiene education, food education, and exercise videos. The educational videos were provided to help the older adults practice good sleep hygiene and exercise habits. The sleep hygiene education video helped users practice “abdominal breathing” for relaxation. The exercise videos were recorded by a professional rehabilitator to guide older adults in performing different types of body movements. Overall, the app proved to be an effective tool to help the users perform training courses at home. The health promotion education programs are shown in Figure 4.

Measures

Self-Assessments

Self-assessment scales for loneliness, depression, and anxiety were administered before and after chatbot use. Using a 7-point Likert scale, we also assessed individuals’ attitudes toward the chatbot post intervention.

Loneliness was measured using the University of California Los Angeles Loneliness Scale (UCLA-LS), which has demonstrated good reliability and validity for all ethnic groups as well as for older adults [20]. Participants rated themselves on a 1-4 scale for the 20 items, with a possible total score ranging from 20 to 80. As a one-dimensional measure, higher scores indicate greater loneliness.

Depression was measured using the Geriatric Depression Scale (GDS) 15-item short form; a cutoff score of 15 indicates clinical depression [21].

Anxiety was assessed using the Hospital Anxiety and Depression Scale anxiety subscale (HADS-A), which consists of 7 items. The depression subscale, also comprising 7 items, was not used in this study owing to the duplication of targets measured by the GDS [22].

Attitudes Toward the Chatbot

Based on the measure proposed by Bickmore et al [23], the questionnaire was modified to evaluate users’ attitudes toward the chatbot. The questionnaire covered four dimensions regarding satisfaction, usability, willingness to continue use, and adherence. All participants were asked to rate a series of attitude statements using a 7-point Likert rating scale, ranging from 1 (strongly disagree) to 7 (strongly agree), to assess their attitudes toward the chatbot’s features, such as “I was satisfied with the chatbot,” “It was easy to use the chatbot,” “I would like to continue using the chatbot,” “I will follow the chatbot’s recommendations,” and others [23-26].

Utilization Rate

The participant utilization rate was determined according to the frequency of interactions with the chatbot, specifically measured as the ratio of the number of interaction days to the total test days for each participant. This score encompasses the percentage of responses to inquiries regarding emotions, sleep quality, sport participation, and daily outings. For example, if a participant responded to emotional inquiries 15 times (with only one interaction per day counted) during a 30-day testing period, their emotional inquiry response rate would be 50%.

Statistical Analysis

We compared the effectiveness of the chatbot between the age groups of ≥65 and <65 years. In Taiwan, individuals 65 years and older can enjoy old-age benefits (such as old-age pensions and bus discounts). Therefore, the division of participants into these two age groups allowed for a comparison of the effects of the chatbot on anxiety and depression in individuals who have already reached the age of eligibility for old-age benefits and those who have not. This distinction is important as it enables researchers to explore differences in psychological well-being between the two groups considering the impact of access to benefits and associated social support networks.

All statistical analyses were performed using SPSS for Windows 20.0 (IBM Corp). The Fisher exact test, Wilcoxon signed rank test, analysis of covariance, and Spearman rank correlations were applied as appropriate.

Results

Sample Characteristics

The age of the participants ranged from 55 to 82 years, with a mean age of 65.21 (SD 7.51) years; the study sample comprised 74% (n=26) female participants and 26% (n=9) male participants. There were no significant differences in the demographic variables between the groups, except that a greater percentage of participants in the older age group (≥65 years) were living alone (Table 1).

Attitude Toward the Chatbot

In the postintervention assessment, we collected the participants’ attitudes toward the chatbot using a 7-point Likert scale. The mean scores for satisfaction, usability, willingness to continue using, and adherence were 5.33, 6.33, 4.87, and 5.07, respectively, in the ≥65 years age group; the corresponding mean scores in the <65 years age group were 5.15, 6.05, 4.7, and 4.95, respectively. There were no group differences in satisfaction, usability, willingness to continue using, and adherence scores.

Use and Response Rates

The use rate was determined by the frequency of a patient’s interaction with the chatbot, specifically measured as the ratio of interacting days to the total testing days for each participant. This included the rate of response to the inquiries for mood, sleep quality, and participation, as well as the “I went out today” inquiry.

There was no significant difference in the response rate between the two age groups. However, the response rate to the sport participation inquiry was less than 50%. Furthermore, during the 1-month intervention, the participants had a high use rate, with over 82% of the participants reporting being active daily (Table 2).

Self-Assessment Scales

The improvement in the UCLA-LS score post intervention was not significant in the <65 age group (P=.98) but was significant in the ≥65 age group (P=.006). HADS-A and GDS scores showed no significant improvements in either group (Table 3).

Figures 5-7 show the changed trends in the scales before and after the intervention. The slopes flattened after the intervention.

‎

‎

‎

Discussion

Principal Findings

Usability and User Satisfaction of the Health Promotion Chatbot Among Older Adults During the COVID-19 Pandemic

This study took place during the COVID-19 pandemic, which greatly impacted people’s daily lives and led to notable reductions in interpersonal interactions and outdoor activities [27]. The pandemic also substantially affected the economy, impacting employment and raising anxiety for many who experienced both the disease itself and the protocols set in place to counter the spread of the virus [28,29].

The results of this study indicate that both age groups, ≥65 and <65 years, were equally satisfied with the chatbot. Its design was rated highly for usability and user satisfaction by both groups. The ≥65 age group provided a mean satisfaction rating of 5.33 and a mean usability rating of 6.33, whereas the corresponding scores for the <65 age group were slightly lower at 5.15 and 6.05, respectively. These high satisfaction ratings reflect the chatbot’s effectiveness in catering to a wide range of users and their positive reception of the user-friendly interface.

The use rate among the older adults was remarkably positive. Previous studies have suggested that systems that are not user-friendly for older adults are often met with resistance and ineffectiveness [30]. However, this system, which was specifically designed for older adults, features a simple and user-friendly interface to target the barriers that older adult users might face in terms of digital literacy and technical assistance [31,32]. Our Health Promotion chatbot was designed to have concise functions, aiming to provide companionship and support even to those who are less familiar with technological products. Additionally, the use and response rates showed that age did not affect the willingness to use the chatbot, further suggesting that the content and interface of the chatbot are well-received and accessible for older adult users.

Therefore, the findings of this investigation on chatbot applicability in geriatric psychiatric outpatient clinics indicate that the chatbot interface is user-friendly and meets the desired criteria.

Chatbots Mitigate Loneliness in Older Adults

Some studies have explored the potential of assistive technology in addressing social isolation and loneliness among older adults. For example, one prospective cohort study reported that humanoid robots improve patient outcomes by providing companionship and entertainment in acute hospital settings and may be an acceptable outlet for socialization among older adults [33,34]. In a systematic review, Chen and Schulz [7] found that information and communication technology interventions can effectively reduce social isolation among older adults and potentially improve their quality of life by enhancing social support, promoting social engagement, and reducing feelings of loneliness and depression. However, a cross-sectional study also showed that problematic social media use was positively associated with perceived social isolation among older adults [35].

Four possible mechanisms by which information technology alleviates social isolation and loneliness have been proposed in the literature [7,36]: connection to the outside world, gaining social support, engaging in interesting activities, and boosting self-confidence. Boosting self-confidence was interpreted based on reports of older adults feeling younger and more equipped with new skills when taught how to use communication technologies. In our study, the chatbot greeted older adults daily and encouraged them to stay active. Some participants reported benefitting from the chatbot’s exercise videos; engaging with the chatbot and performing daily exercises made them feel rejuvenated and more purposeful.

Overall, our results suggest that the chatbot helped to reduce feelings of loneliness; the average UCLA-LS score of the total cohort at baseline was 40.4, which decreased to 38.1 post intervention. For the ≥65 age group, the average baseline score was 38.9, which decreased to 34.9 post intervention, representing a statistically significant difference (P=.006). However, the <65 age group had an average baseline score of 41.5, which remained stable post intervention with a nonsignificant difference (P=.98). Therefore, although the participants had decreased UCLA-LS scores post intervention on the whole, only the older participants (≥65 years) showed a significant improvement in loneliness.

Generally, the ≥65 age group responded well to the chatbot, as reflected in the UCLA-LS score changes, indicating that this age group can benefit from chatbots. Some potential reasons to explain the lack of an effect on loneliness for the <65 age group may be related to this group experiencing elevated stress in their daily lives [4,37] due to financial burdens and concerns about the future. Evidently, the effectiveness of chatbots in alleviating such stress is limited. Moreover, the <65 age group likely also had more opportunities for interpersonal interaction, even during the pandemic. Members of this group tend to not live alone and to have more avenues (eg, different types of social media) to access information. Consequently, the influence of chatbot interventions on their lives was comparatively less pronounced.

Our study demonstrates the positive impact of chatbot interventions on older adults enrolled in psychiatric outpatient departments and with relatively stable symptoms during the pandemic. Previous studies have mentioned the importance of using technology to reduce the impact of loneliness on older adults, particularly in the context of pandemic-related restrictions [2,38,39]. Although this study was conducted during a period when the pandemic was more severe, it offers valuable insights into older adult care. To enhance the generalizability and applicability of future trials, we recommend expanding the participant pool to include older adults outside of clinical settings.

Limitations

This pilot study has some limitations. When assessing loneliness and depression, it is important to include in-depth interviews to understand users’ thoughts. However, due to the pandemic, interviews could not be conducted during the study period. We therefore aim to further enhance this aspect by including interviews in future research. In addition, given that the study was conducted during the COVID-19 pandemic, it was challenging to enroll participants, as many patients were unwilling to visit the hospital, resulting in a limited sample size. The improvement in loneliness may be due in part to the placebo effect. Additionally, the study sample was derived solely from the psychiatric outpatient department of a single medical center, potentially introducing bias in the referral process and limiting the generalizability of the findings to a broader population. Finally, the follow-up period was relatively short (within half a year); thus, determining the long-term effect of the intervention requires further research.

Conclusions

The chatbot interface developed in this study was simple and easy to use. The results demonstrated that older adults in psychiatric outpatient clinics with relatively stable depressive and anxiety symptoms could benefit from the chatbot’s care, companionship, and potential reduction in loneliness, particularly during a pandemic. Simultaneously, the accessibility and user satisfaction of chatbots, which are crucial for promoting the health of older adults through using such apps, enabled the participants to easily access health information and advice, thereby enhancing their health awareness and quality of life.

Future research can explore how more personalized mental health support can be provided by expanding on the functionality of chatbots to effectively manage depression and anxiety among older adults. Simultaneously, there should be in-depth exploration of how chatbots can be integrated with traditional health care services to establish a more comprehensive medical support system. Qualitative investigations through individual or group interviews can add further insights into their use and adaptation.