This paper is in the following e-collection/theme issue:

Formative Evaluation of Digital Health Interventions (2198) e-Mental Health and Cyberpsychology (1394) Web-based and Mobile Health Interventions (3102) Anxiety and Stress Disorders (1003) Depression and Mood Disorders; Suicide Prevention (1278) mHealth for Wellness, Behavior Change and Prevention (2733) Mobile Health (mhealth) (2753)

Published on in Vol 7 (2023)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/46458, first published .
Efficacy of a Text-Based Mental Health Coaching App in Improving the Symptoms of Stress, Anxiety, and Depression: Randomized Controlled Trial

Efficacy of a Text-Based Mental Health Coaching App in Improving the Symptoms of Stress, Anxiety, and Depression: Randomized Controlled Trial

Efficacy of a Text-Based Mental Health Coaching App in Improving the Symptoms of Stress, Anxiety, and Depression: Randomized Controlled Trial

Authors of this article:

Yee Siew Lim1 Author Orcid Image ;   Jia Hui Quek1 Author Orcid Image ;   Xiu Wei Ching1 Author Orcid Image ;   Dominic Tao Ran Lim1 Author Orcid Image ;   Kean Ghee Lim1 Author Orcid Image ;   Chandramani Thuraisingham1 Author Orcid Image ;   Parikial Philip George1 Author Orcid Image
Article Authors Cited by (1) Tweetations (2) Metrics

    Original Paper

    International Medical University (IMU), Seremban, Malaysia

    †deceased

    Corresponding Author:

    Yee Siew Lim, MBBS

    International Medical University (IMU)

    Jalan Rasah, Bukit Rasah

    Seremban, 70300

    Malaysia

    Phone: 60 67677798

    Email: lim.yeesiew@outlook.com


    Background: Stress, anxiety, and depression are major mental health concerns worldwide. A wide variety of digital mental health interventions have demonstrated efficacy in improving one’s mental health status, and digital interventions that involve some form of human involvement have been shown to demonstrate greater efficacy than self-guided digital interventions. Studies demonstrating the efficacy of digital mental health interventions within the Asian region are scarce.

    Objective: This study aimed to investigate the potential efficacy of the digital mental health intervention, ThoughtFullChat, which consists of one-on-one, asynchronous, text-based coaching with certified mental health professionals and self-guided tools, in improving self-reported symptoms of depression, anxiety, and stress. The study also aims to examine the potential differences in efficacy among occupational subgroups and between sexes.

    Methods: A randomized controlled study was conducted among housemen (trainee physicians), students, faculty members, and corporate staff at International Medical University, Malaysia. A total of 392 participants were enrolled and randomized to the intervention (n=197, 50.3%) and control (n=195, 49.7%) groups. Depression, anxiety, and stress symptoms were measured using the Depression, Anxiety, and Stress Scale–21 items at baseline and after the 3-month intervention period. The Satisfaction with Life Scale and Brief Resilience Scale were also included, along with a questionnaire about demographics.

    Results: Significant decrease was observed in depression (P=.02) and anxiety (P=.002) scores in the intervention group. A subgroup (corporate staff) also demonstrated significant decrease in stress (P=.005) alongside depression (P=.006) and anxiety (P=.002). Females showed significant improvements in depression (P=.02) and anxiety (P<.001) when compared with males.

    Conclusions: This study provides evidence that the ThoughtFullChat app is effective in improving the symptoms of depression, anxiety, and stress in individuals, particularly among corporate staff from the educational field. It also supports the notion that mobile mental health apps that connect users to mental health professionals in a discreet and cost-efficient manner can make important contributions to the improvement of mental health outcomes. The differential improvements among occupational subgroups and between sexes in this study indicate the need for future digital mental health app designs to consider an element of personalization focused on systemic components relating to occupation.

    Trial Registration: Clinicaltrials.gov NCT04944277; https://classic.clinicaltrials.gov/ct2/show/NCT04944277

    JMIR Form Res 2023;7:e46458

    doi:10.2196/46458

    Keywords

    digital mental health; mobile health; randomized control trial; mobile phone; mental health; Depression, Anxiety, and Stress Scale–21 items; DASS-21; Asia; Malaysia 


    Background

    Mental health is a subject of great concern in health care, with up to US $1 trillion each year spent on selected mental illnesses such as depression and anxiety worldwide [1] In the Asian region, pooled prevalence estimates from a meta-analysis study based in South Asia indicate depression and anxiety to reach >25% in large representative populations [2]. The countries studied were Afghanistan, Bangladesh, Bhutan, India, the Maldives, Nepal, Pakistan, and Sri Lanka.

    According to the UK Office for National Statistics, the prevalence of moderate to severe depression in working adults increased from 5.5% to 7.5% and from 18% to 19.7% during the COVID-19 pandemic. In comparison, the depression rate among nonworking adults did not fluctuate significantly (24.8% before the pandemic to 26.6% during the pandemic) [3]. The Malaysian National Health and Morbidity 2015 survey found the prevalence of mental health problems among adults in Malaysia to be almost tripled from 11.2% in 2006 to 29.2% in 2015. Regarding depression, 1 in 5 cases were reported among Malaysian university students [4].

    Odriozola-González et al [5] studied the mental well-being of Spanish university students from March 2020 to April 2020. Their results showed 34.19% to be experiencing moderate to extremely severe depression symptoms, 21.34% with severe anxiety symptoms, and 28.14% with moderate to extremely severe stress symptoms. Other important high-risk groups highlighted in the extant literature include health care workers [6,7] and academics [8-11].

    Most people experiencing mental health problems are unable to access treatment owing to a variety of reasons that include the lack of availability of services, time constraints, transportation problems, high costs, and stigma [2,12-14]. Digital mental health interventions, commonly defined as mental health interventions that use digital tools (ie, mobile apps or artificial intelligence–based software) to augment mental health care and improve its delivery, access, and effectiveness [15], have been known to be particularly useful in overcoming the barrier of stigma owing to increased privacy and anonymity [16]. Such interventions also improve access to preventive care [15], thus enabling a proactive approach to mental well-being. In terms of efficacy, digital mental health interventions have been shown to be as effective as traditional approaches [17] but without the typical barriers to accessibility that are linked to these latter approaches. Particularly in Asia, where the stigma associated with mental disorders can be considered “worse than the disease” [18], individuals may prefer digital interventions owing the level of anonymity and privacy afforded by them. Several digital mental health interventions have been proven to be effective [17,19], with evidence pointing to great effectiveness among interventions that combine a human therapist with digital assistance of some form, as compared with purely self-guided interventions [20].

    Although many studies examine the efficacy of digital mental health interventions among different occupational groups such as university students [5], health care workers [6,8], physicians [7], and faculty staffs [10], very few emphasize the differential impact of these interventions among occupational groups or even between sexes. There is some evidence indicating that females are more likely to use and benefit from digital or web-based mental health interventions as compared with males [21,22]; however, these observations emphasize use rather than efficacy. Knowing the differential impact of digital mental health interventions among different occupational groups and sexes may provide useful insights that lead to more inclusive and effective intervention designs.

    Most studies of digital mental health interventions are conducted in the West, with very little available data from Asia or other non-Western populations. Notably, 2 recent studies [23,24] in Singapore had highlighted the effectiveness of self-guided interventions in reducing stress and improving body image and self-compassion. To the best of our knowledge, there are no studies assessing the effectiveness of digital interventions that combine the services of a mental health professional and self-guided tools within an Asian sample at the time of writing. There are also no digital mental health studies in the region that emphasized improvements in symptoms of depression and anxiety.

    In this study, we speculated that the ThoughtFullChat (TFC) mobile app, a one-on-one, asynchronous, text-based mental health coaching app equipped with self-guided tools, could potentially improve symptoms of depression, anxiety, and stress as measured by the Depression, Anxiety, and Stress Scale–21 items (DASS-21) scale. They also intended to explore the differences in efficacy among different subgroups based on occupation (medical housemen, corporate staff, undergraduate students, and faculty members) and sex, as this may present implications to mental health intervention design strategies among these different groups. The study was conducted on a Malaysian sample affiliated with the International Medical University (IMU), Malaysia, making it among the first to explore this subject in an Asian context.

    Objectives

    This study examined the potential efficacy of the TFC mobile app among physicians in training (housemen), students, faculty members (academic staff), and corporate staff (non–health care) in reducing the symptoms of self-reported depression, anxiety, and stress. The study also examined the potential improvements in self-reported resilience and satisfaction with life after the intervention. Sex differences in improvements were analyzed, and feedback about user satisfaction was sought.

    The first hypothesis that was proposed was that there would be improvement in depression, anxiety, and stress symptoms in the general sample and the subgroups after the intervention, consistent with previous observations in studies. The second hypothesis was that females would show more significant improvement than males, given the nature of the app that involved asynchronous texting with a mental health professional and other self-guided tools, which have been previously observed to be more engaging with females [22].


    Study Design

    This randomized controlled trial was conducted among housemen, university students, faculty members, and corporate staff at IMU from July 26, 2021, to October 26, 2021. The study was also registered on Clinicaltrials.gov (NCT04944277).

    Ethical Considerations

    All procedures were approved by the research ethics committee of IMU on March 18, 2021 (CSc/Sem6[032021, CSc/Sem6[04]2021, CSc/Sem6[05]2021, CSc/Sem6[06]2021). CONSORT (Consolidated Standards of Reporting Trials) guidelines were followed, and the study was conducted in accordance with the Declaration of Helsinki (as revised in 2013).

    This study adheres to rigorous ethical standards, encompassing human participant research ethics review, exemptions, and approvals; ensuring informed consent; protecting privacy and confidentiality; and affirming data anonymity or deidentification, while strictly refraining from providing any form of compensation for participation in the study.

    Enrolled participants were assigned a code, which was used as the sole identifier during this study. The identity of the participants was not revealed to the mental health professionals who served them. The emergency contact details of each participant were requested and stored by ThoughtFull’s internal clinical team to be used in case of an emergency. The internal team had no access to the chat system or any communications between the mental health professional and participants. All participants were informed about the limits to privacy and the confidentiality policy before joining the study.

    The information about the study was conveyed through an email and Microsoft Teams. By replying to the email, stating their agreement to participate in this study, and completing the registration process, participants demonstrated their informed consent to participate in the study.

    Participants

    Enrollment

    Participants were contacted through email and Microsoft Teams. A webinar on mental health resilience was conducted jointly by the ThoughtFull team and the clinical research team to introduce the research project and recruit participants. Interested participants were required to complete DASS-21, Satisfaction with Life Scale (SWLS), and Brief Resilience Scale (BRS). To maximize the response rate, a written notice and verbal reminder via phone were given to the nonresponding participants after the initial distribution of the questionnaires and before the commencement of the intervention.

    Inclusion and Exclusion Criteria

    The inclusion criteria include medical practitioners (housemen) who undergo internship training in any Malaysian hospital; are current IMU faculty members, students, and corporate staff of IMU who are competent in English; and are experiencing none to severe stress, anxiety, or depression based on their DASS-21 scores. All participants were aged >18 years. Participants who had extremely severe stress, anxiety, or depression were excluded from the study and were provided with helplines and contacts for mental health support services through the study information sheet. High scores were excluded owing to the nature and focus of the asynchronous text-based intervention, which is primarily preventive. More comprehensive and direct interventions may be required to mitigate potential harm that may be caused to those who have heightened symptoms. Hence, the intervention itself, being preventive in nature, is not meant for use by those who have severe symptoms. Moreover, precise cutoffs for the different levels of symptoms are provided in Table 1.

    Table 1. Depression, Anxiety, and Stress Scale–21 items scoring guide.
    LevelsDepression (score), rangeAnxiety (score), rangeStress (score), range
    Normal0-40-30-7
    Mild5-64-58-9
    Moderate7-106-710-12
    Severe11-138-913-16
    Extremely severe≥14≥10≥17
    Sample Size Calculation

    This interventional study with TFC involved the measurement of 5 variables: Depression, Anxiety, and Stress (DASS-21), Resilience and Satisfaction of Life scores. We used anxiety as the primary outcome for our sample size estimation, as previous analysis performed by ThoughtFull (ThoughtFull World, unpublished data, December 2020) showed that the most effective improvements were to be found with this construct.

    The TFC app had an odds ratio change of 0.5 to 0.6, based on past data provided by ThoughtFull (ThoughtFull World, unpublished data, December 2020). Thus, an odds ratio of 0.54 was selected for use. This is based on individuals with anxiety moving into the normal range of scores or improving by 1 grade of severity. The sample size calculator obtained from OpenEpi [25] provided a sample size of 384, with 192 (50%) in each group. This study had 50.3% (197/392) of participants in the intervention group and 49.7% (195/392) in the control group, satisfying the sample size estimation.

    Randomization

    Block randomization was chosen to allocate participants into intervention and control groups for our study. Before the beginning of the trial, we used Research Randomizer [26], a web-based software, to assign participants to intervention and control groups. We randomized 5 participants from every 10 respondents to our questionnaire. In total, 5 unique sets of numbers were generated, and the corresponding participants were allocated to the intervention group. The procedure was repeated for every 10 respondents.

    Treatments

    This trial used a 1 × 1 factorial design (intervention group with free 3-month access to the coaching app vs control group with no access to the app). Participants in the control group were not introduced or given free access to the TFC app but were not restricted from using other apps or obtaining other forms of intervention during the study period.

    Intervention Group

    The intervention group was given a 3-month free access to the TFC app, a subscription-based mobile platform that empowers users to proactively engage with their mental health. The TFC app is a commercially available solution that is currently used as a coaching tool for the improvement of mental health outcomes in individuals with no formal diagnosis.

    Participants had full access to one-on-one, daily, asynchronous, text-based coaching with certified mental health professionals and self-guiding tools (ie, daily mood loggings, professionally curated guided lessons, and emotional health assessment using DASS-21). These mental health professionals had at least a master’s degree with ≥3 years of experience in working as counselors or clinical psychologists based in Singapore and Malaysia. Figure 1 shows the screenshots of the TFC app (version 1.6.0; latest version as of July 2021)—home tab, learn tab, chat tab, and grow tab.

    ‎
    Figure 1. Screenshots of the ThoughtFullChat app. DASS-21: Depression, Anxiety, and Stress Scale–21 items.
    Control Group

    The control group was not given access to the TFC app; however, they were not barred from obtaining and using the app (or other apps) of their own accord. They were also provided with other nondigital resources such as helplines or counseling resources to be used in case of need.

    Participants of both groups were assessed after 3 months of the intervention period, using a digital questionnaire comprising of all the instruments used (refer to the Instruments section).

    Instruments

    The instruments used in the questionnaire were DASS-21, SWLS, and BRS.

    DASS-21 Questionnaire

    DASS-21 is a self-report scale used to screen for anxiety, depression, and stress. Each of the main scales contains 7 subscales, which sum up to 21 subscales. The sum scores for each scale (anxiety, depression, and stress) were categorized into normal, mild, moderate, severe, and extremely severe based on the severity ratings (Table 1) by Lovibond and Lovibond [27]. Previous studies have shown evidence for adequate validity and strong consistency (α=.90) of DASS-21 among the Malaysian student population [28].

    Scores within the normal range indicate minimal or no depressive or anxiety or stress symptoms. Scores within the mild range indicate the presence of notable depressive or anxiety or stress symptoms that may be manageable without significant impairment. Scores within the moderate range indicate the presence of notable depressive or anxiety or stress symptoms that may be interfering with daily functioning. Scores within the severe range indicate the presence of severe depressive or anxiety or stress symptoms that may significantly affect daily functioning. Scores within the extremely severe range indicate the presence of severe depressive or anxiety or stress symptoms that may significantly affect daily functioning and require intervention. DASS-21 alone does not act as a tool for official diagnosis but is instead a screening tool.

    SWLS Questionnaire

    SWLS measures the individual’s subjective satisfaction with their overall life and consists of 5 items, with each using a 7-point Likert-type scale. This scale assesses different aspects of life satisfaction including alignment with ideals, life quality, general satisfaction, attainment of goals, and acceptance of past choices. The sum scores on each item were categorized into extremely satisfied (31-35), satisfied (26-30), slightly satisfied (21-25), neutral (20), slightly dissatisfied (15-19), dissatisfied (10-14), and extremely dissatisfied (5-9). SWLS has great validity and consistency (α=.86) among the clinical outpatients in Malaysia [29].

    BRS Questionnaire

    BRS measures the individual’s resilience, which is the ability to bounce back from adversity and cope with challenges. It consists of 6 items, with each using a 5-point Likert-type scale. This scale assesses different aspects of resilience including the ability to recover quickly, handle difficult situations, remain focused under pressure, and maintain an optimistic outlook. Scores of 6 items were summed together and divided by the number of questions answered. The score was interpreted based on the recommended range, which is low resilience (1-2.99), normal resilience (3-4.30), and high resilience (4.31-5). BRS had also been validated in the past and had shown high reliability (α=.93) among the Malaysian student population [30].

    TFC App Feedback

    Apart from measuring the symptoms of depression, anxiety, stress, resilience, and satisfaction with life, participants were also asked to indicate their general satisfaction with and perceived ease of use of the app.

    Pilot Study

    A pilot study of the questionnaire was conducted among individuals who were demographically similar to the target population (n=20). These participants were not included in the study population. The aim of the pilot study was to ensure that the questions were understood and perceived correctly. The pilot study found that all participants (20/20, 100%) understood the context well and could answer the questionnaires without any obstructions.

    Statistical Analysis

    The collected data were exported from Microsoft Forms to Excel Sheets and analyzed using the IBM SPSS version 28.0.0.0. Descriptive statistics were calculated for the participant’s sociodemographic factors and user experience of the app in the intervention group. The mean and SD for the DASS-21, SWLS, and BRS scores before and after the intervention are presented. Wilcoxon signed rank test was used to calculate the significant differences in the mean of the 3 metrics (DASS-21, SWLS, and BRS) of the 2 groups. P value <.05 is considered as statistically significant. Rank-biserial correlation value was used to calculate the effect size of each result.


    Overview

    Figure 2 explains the flow of this study. There were 495 individuals who responded to our preinterventional questionnaire. Upon screening the participants’ DASS-21 scores, 79.2% (392/495) were enrolled and randomized to the intervention and control groups. The remaining 12.7% (63/495) of the respondents did not meet the inclusion criteria owing to severe DASS-21 scores (refer to the cutoff scores for severe DASS-21 in Table 1). We excluded 0.1% (35/495) of the respondents owing to incomplete questionnaires. During the study, 15.2% (30/197) of the participants from the intervention group requested to be removed from participating. In the control group, 14.4% (28/195) of the participants did not complete the follow-up questionnaire. After 3 months, there were 167 participants in each group who completed the intervention.

    ‎
    Figure 2. CONSORT (Consolidated Standards of Reporting Trials) diagram showing the selection process of this study. BRS: Brief Resilience Scale; DASS-21: Depression, Anxiety, and Stress Scale–21 items; RCT: randomized controlled trial; SWLS: Satisfaction with Life Scale.

    Sociodemographic Descriptions

    Sociodemographic characteristics of the respondents are presented in Table 2. The sample was mostly females (215/334, 64.2%), those aged between 18 and 29 years (178/334, 53.3%), and those who are single (194/334, 58%). The respondents’ nationality is separated into 2 major categories: Malaysian (304/334, 91%) and others (30/334, 9%). Most of our respondents have a high level of education, with bachelor’s degree (192/334, 57.5%) and master’s degree (67/334, 20%). In terms of income status, most of them (178/334, 53.3%) are in the middle 40% category. We categorized the respondents into bottom 40% (<Ringgit Malaysia [RM] 4849 [US $1047.53]), middle 40% (RM 4850 [US $1047.53] to RM 10,959 [US $2367.47]), and top 20% (>RM 10,959 [US $2367.47]) of the household income classification in Malaysia by following the income range set by the Department of Statistics, Malaysia, in 2019.

    Table 2. Descriptive statistics about the sociodemographics of participants (N=334).
    CharacteristicsIntervention group (n=167), n (%)Control group (n=167), n (%)
    Age group (years)

    		18-2998 (58.7)80 (47.9)

    		30-4950 (29.9)71 (42.5)

    		>5019 (11.4)16 (9.6)
    Sex

    		Male52 (31.1)67 (40.1)

    		Female115 (68.9)100 (59.9)
    Nationality

    		Malaysian151 (90.4)153 (91.6)

    		Non-Malaysian16 (9.6)14 (8.4)
    Highest level of formal education

    		Secondary school2 (1.2)6 (3.6)

    		Preuniversity19 (11.4)24 (14.4)

    		Diploma1 (0.6)2 (1.2)

    		Bachelor’s degree93 (55.7)99 (59.3)

    		Master’s degree40 (23.9)27 (16.2)

    		PhD12 (7.2)9 (5.4)
    Family income, RMa (US $)

    		<4360 (B40b; $2367.47)35 (20.9)35 (20.9)

    		4360 to 9619 (M40c; $2367.47 to $2367.47)84 (50.3)94 (56.3)

    		>9619 (T20d; $2367.47)46 (27.5)36 (21.6)

    		Not disclosed2 (1.2)2 (1.2)
    Relationship status

    		Single99 (59.3)95 (56.9)

    		Married66 (39.5)71 (42.5)

    		Others2 (1.2)1 (0.6)

    aRM: Ringgit Malaysia.

    bB40: Bottom 40% of the household income classification in Malaysia.

    cM40: Middle 40% of the household income classification in Malaysia.

    dT20: Top 20% of the household income classification in Malaysia.

    Overall Outcomes

    Decrease in overall mean scores of depression, anxiety, and stress was observed in the intervention group, whereas the mean scores were on the rise in the control group (Table 3). The decrease in depression (P=.02) and anxiety (P=.002) in the intervention group was significant. Changes in the SWLS and BRS scores showed no significant difference.

    DASS-21, SWLS, and BRS scores of the participants in each subgroup are shown in Table 4. The overall effect size of the subgroups was generally small, except for the corporate staff group, which had a moderate effect size (Rank-biserial correlation=0.60). SWLS and BRS scores were not significant for all groups, except corporate staff.

    Table 3. Comparing means, P values, and effect sizes for outcome variables between overall intervention and control groups before and after the 3-month study period.
    Outcome variableIntervention groupControl group

    		Preintervention score, mean (SD)Postintervention score, mean (SD)P valueRank-biserial correlationPreintervention score, mean (SD)Postintervention score, mean (SD)P valueRank-biserial correlation
    Depression3.49 (3.30)2.90 (3.52).020.2443.37 (3.18)3.99 (5.12).31−0.103
    Anxiety3.39 (2.60)2.79 (3.26).0020.3143.19 (2.62)3.68 (4.69).70−0.039
    Stress5.30 (3.65)5.02 (4.50).290.1024.85 (3.49)5.20 (5.21).10−0.0006
    Satisfaction with Life24.36 (5.87)24.61 (6.40).22−0.11823.79 (5.71)24.23 (5.73).29−0.10
    Resilience3.58 (3.21)3.35 (0.62).130.1503.32 (0.58)3.34 (0.61).670.044
    Table 4. Comparing the means, P values, and effect sizes between each subgroup.

    		Intervention groupControl group

    		Preintervention score, mean (SD)Postintervention score, mean (SD)P valueRank-biserial correlationPreintervention score, mean (SD)Postintervention score, mean (SD)P valueRank-biserial correlation
    Housemen group (n=91)

    		Depression3.65 (3.33)3.58 (3.67).570.1073.54 (3.21)4.88 (3.87).06−0.358

    		Anxiety3.88 (2.57)3.77 (4.20).530.1132.95 (2.51)3.91 (3.79).16−0.280

    		Stress5.46 (4.77)5.42 (4.77).970.0074.98 (3.58)5.56 (3.89).50−0.123

    		Satisfaction with Life21.75 (5.79)22.15 (4.88).79−0.04920.88 (6.04)22.19 (5.19).17−0.262

    		Resilience3.02 (0.57)2.96 (0.42).400.1683.07 (0.51)3.12 (0.65).61−0.107
    Student group (n=73)

    		Depression3.63 (3.57)2.84 (3.45).370.1764.63 (3.21)4.94 (4.93).710.076

    		Anxiety3.03 (2.47)2.55 (2.33).010.3553.25 (2.85)3.58 (3.83).70−0.086

    		Stress4.89 (3.49)4.79 (3.57).860.0345.83 (3.25)5.25 (4.21).350.189

    		Satisfaction with Life24.66 (5.82)22.45 (7.47).040.38922.70 (5.84)23.42 (7.14).58−0.117

    		Resilience3.37 (0.89)3.12 (0.68).080.3463.36 (0.64)3.20 (0.68).170.302
    Faculty group (n=57)

    		Depression3.78 (3.24)3.17 (4.07).210.2342.50 (3.05)4.70 (10.2).66−0.165

    		Anxiety3.25 (2.73)2.56 (3.05).240.2592.26 (2.42)4.80 (9.57).37−0.308

    		Stress6.61 (4.49)6.78 (5.62).88−0.0325.25 (4.51)8.90 (10.55).20−0.345

    		Satisfaction with Life27.08 (4.94)27.39 (6.87).23−0.28028.05 (4.21)26.93 (5.96).740.094

    		Resilience4.94 (6.71)3.81 (0.51).600.1153.75 (0.64)3.85 (0.57).42−0.222
    Corporate staff (n=113)

    		Depression2.90 (3.14)1.90 (2.74).0060.5662.80 (3.02)2.70 (3.26).930.015

    		Anxiety3.30 (2.63)2.10 (2.77).0020.5973.60 (2.58)3.30 (3.36).140.227

    		Stress4.50 (3.55)3.40 (3.29).0050.5644.10 (3.13)3.80 (3.30).440.120

    		Satisfaction with Life24.70 (5.70)27.10 (4.75)<.001−0.61824.90 (4.70)25.20 (4.65).91−0.017

    		Resilience3.60 (0.69)3.60 (0.49).68−0.0803.47 (0.61)3.40 (0.45).460.118
    Age group

    		18-29 years


    		Depression3.79 (3.47)3.26 (3.55).130.1903.95 (3.22)4.74 (4.09).24−0.160


    		Anxiety3.63 (2.57)3.24 (3.51).060.2423.09 (2.63)3.80 (3.71).16−0.205


    		Stress5.33 (3.31)5.03 (4.18).440.0975.30 (3.52)5.40 (3.95).930.013


    		Satisfaction with Life23.1 (5.90)22.66 (6.04).490.08521.94 (5.90)22.63 (5.97).34−0.133


    		Resilience3.26 (0.75)3.08 (0.57).050.2543.19 (0.58)3.12 (0.61).410.126

    		>30 years


    		Depression3.03 (3.02)2.36 (3.41).040.3432.83 (3.07)3.31 (5.86).84−0.031


    		Anxiety3.04 (2.61)2.14 (2.76).010.4133.29 (2.63)3.57 (5.46).380.125


    		Stress5.26 (4.10)5.01 (4.95).490.1074.44 (3.43)5.01 (6.16).900.018


    		Satisfaction with Life26.14 (5.37)27.38 (5.88).002−0.46125.51 (4.97)25.77 (5.05).71−0.048


    		Resilience4.28 (4.89)3.72 (0.49).91−0.0183.49 (0.62)3.54 (0.54).93−0.012
    Sex

    		Male


    		Depression3.56 (3.57)3.33 (3.88).320.1783.37 (3.27)3.69 (5.69).800.041


    		Anxiety3.13 (2.54)2.83 (3.09).530.1053.03 (2.56)3.39 (5.65).460.115


    		Stress4.77 (3.38)4.79 (4.12).950.0124.54 (3.48)4.70 (5.91).390.128


    		Satisfaction with Life23.94 (6.78)23.17 (7.05).770.15624.43 (5.54)24.51 (5.52).77−0.019


    		Resilience4.12 (5.67)3.31 (0.60).300.3473.40 (0.59)3.48 (0.53).30−0.018

    		Female


    		Depression3.43 (3.19)2.71 (3.33).020.2723.36 (3.14)4.20 (4.73).13−0.198


    		Anxiety3.50 (2.62)2.77 (3.35)<.0010.4093.30 (2.67)3.88 (3.94).26−0.151


    		Stress5.54 (3.75)5.13 (4.68).240.1375.06 (3.50)5.53 (4.77).54−0.076


    		Satisfaction with Life24.55 (5.43)25.26 (6).03−0.24923.35 (5.80)24.09 (5.88).22−0.151


    		Resilience3.34 (0.73)3.36 (0.64).690.0483.26 (0.65)3.25 (0.65).580.073

    Housemen Group

    The intervention group showed improvement in DASS-21 scores, but the results were not statistically significant. The control group showed a slight increase in DASS-21 scores, but this was also not statistically significant.

    Medical Students Group

    The depression, anxiety, and stress levels decreased among the medical students in the intervention group, but the decrease was not statistically significant. However, the control group showed a slight increase in depression and anxiety but was not significant.

    Faculty Group

    The depression and anxiety levels had shown some improvement in the intervention group but worsened in the control group. However, the differences were not significant.

    Corporate Staff Group

    The intervention group had shown significant improvements in the DASS-21 scores (depression: P=.006; anxiety: P=.002; and stress: P=.005) as compared with the control group. It also showed significant improvement in the SWLS scores (P<.001).

    Age Group

    This study found that the young group (aged 18-29 years) had high baseline levels of depression, anxiety, and stress compared with those aged >30 years. Significant improvement in depression (P=.04) and anxiety (P=.01) was seen in the group of individuals aged >30 years. In contrast, both age groups in the control group showed decrease in depression, stress, and anxiety.

    Sex

    In the intervention group, depression and anxiety scores reduced for both females and males, but females showed significant improvements in depression (P=.02) and anxiety (P<.001). Females also showed significant improvements in SWLS scores. The changes in the control group were all nonsignificant.

    User Experience With the TFC App

    Regarding user experience of the app, approximately 68.3% (114/167) of the users reviewed the app favorably, with 61.2% (102/167) saying that they will recommend the app to others and 74% (69/93) claiming that the app was easy to use (Table 5).

    Table 5. User experience of the ThoughtFullChat app.
    Item and response optionsFrequency, n (%)
    How likely would you recommend ThoughtFullChat app to a friend or colleague? (n=167)

    		Not at all7 (4.2)

    		Somewhat unlikely13 (7.8)

    		Neither likely nor unlikely45 (26.9)

    		Somewhat likely72 (43.1)

    		Very likely30 (17.9)
    Review of ThoughtFullChat app (n=167)

    		Very unsatisfied6 (3.6)

    		Unsatisfied6 (3.6)

    		Neutral41 (24.6)

    		Satisfied89 (53.3)

    		Very satisfied25 (14.9)
    User-friendliness of ThoughtFullChat app (n=93; missing data: n=74)

    		Very difficult to use1 (1.1)

    		Difficult to use3 (3.2)

    		Neither easy nor difficult to use20 (21.5)

    		Easy to use47 (50.5)

    		Very easy to use22 (23.7)

    Principal Findings

    The goal of this study was to examine the potential effectiveness of a text-based mental health coaching app in improving mental well-being among adults without any known diagnosis of mental health problems. The results demonstrated partial support for the first hypothesis, that the use of the TFC app would result in reduced DASS-21 scores. There was significant decrease in scores of anxiety and depression in the overall sample, but the decrease in stress was not significant, possibly indicating a pathway to the reduction of depression and anxiety that is independent of stress scores. The analysis of occupational subgroups seemed to indicate great effectiveness of the intervention for corporate staff as opposed to the other subgroups. It is likely that the nature of the workplace environment (ie, a fixed schedule and with arguably great predictability as compared with students and housemen) may have contributed independently to the increased effectiveness observed. However, further studies, particularly of qualitative nature, may be required to further confirm this speculation. It is also worth noting that even in subgroups in which there were no significant improvements in the intervention group, deterioration was observed in the control group, likely indicating resistance to deterioration in the intervention group. This is a reasonable conjecture that requires much more in-depth analysis to be confirmed. This study highlights the importance of considering the type of occupation of the target group when choosing digital interventions as a strategy for mental health. An understanding of the differential pathways that led to improvement (ie, how the different occupational groups interacted with the app) could provide essential clues for the development of more effective interventions that are more inclusive and accommodative of occupational differences.

    The second hypothesis, that females would show more significant improvements than males, was also supported by the results. Smail-Crevier et al [21] reported differential likelihood of use of e–mental health programs between females and males. Females preferred features relating to access of information, interactive self-help, well-being exercises, and text-based communications with mental health professionals, whereas males preferred a more gamified system. The TFC app in its current form seemed to match the first, with text-based communication being the primary focus of the intervention. The design and characteristics of a mental health coaching app are crucial in improving the user experience and thus mental health status. This study supports the notion that disparities in app design may unintentionally result in unequal access between the sexes. Improvements to the design that include a more gamified approach may be relevant in engaging males through the app; however, it is important to ensure that the app does not become addictive and thus counterproductive.

    Taken together, the differential improvements among occupational subgroups and between sexes in this study indicate the need for future digital mental health app designs to consider an element of personalization. Although such personalization has often been suggested to include the use of individual psychological patterns and physiological markers [31], large systemic factors such as type of occupation or workplace environment may also prove to be essential. Although it is true that there is no one-size-fits-all approach, consideration of a variety of individual and systemic factors may lead to intervention designs that are more inclusive of and effective for a wide target.

    In general, this study is consistent with previous studies in showing that digital mental health coaching apps can help improve mental health [32]. As the participants of this project belonged to a wide range of age groups and had different occupational backgrounds, the result of this study suggests that the app can be implemented for managing depression, stress, and anxiety symptoms for a diverse group of individuals in the general population, although with somewhat different levels of efficacy. The study also contributes to the limited baseline data available about digital mental health interventions in the Asian region.

    Strengths and Limitations

    Digital mental health apps are becoming more widely available, and studies of these apps are largely conducted in Western populations. This study is among the few studies of digital mental health apps in Asia, thus adding to the limited but growing knowledge base on this subject within the region.

    Although personalization in digital mental health care is a common direction, this study advocates for the consideration of large systemic factors such as type of occupation and sex when considering the personalization of mental health care. Keeping in mind that different subgroups within a population respond differently to a uniform intervention and understanding how these differences in response emerge through the interaction of systemic factors is likely to provide useful information that could guide efforts toward effective personalization of intervention design. This study presents preliminary evidence of such differences and represents a first step in this direction within an Asian context.

    Limitations of the study include the absence of a blinding method. Participants who were assigned to the control group were aware that they will not be using the app. However, all participants from both groups were addressed using the same method, which was via email and SMS text messages. It is also important to note that the questionnaires used in this study are screening tools and not meant to confer any formal diagnosis. The study was therefore focused on prediagnosed individuals. Studies involving diagnosed individuals may provide more useful insights.

    In addition, it is noted that the sample consists only of individuals associated with IMU and, therefore, may be subject to a certain uniformity of experience relating to their occupational environment. However, the housemen affiliated with IMU were still subjected to a wide range of experiences pertaining to their physical location, as they were based in various hospitals within the country. Future studies may consider using a wide range of participants from different organizations to be inclusive of a more diverse set of occupational experiences. Furthermore, the participants of this study were allowed to use the app with flexibility, and there was insufficient monitoring of their DASS-21 scores along with app use. Hence, there was insufficient control for other factors that may have affected DASS-21 scores. This includes the presence or absence of other stress factors such as working and family environments, which may have affected the results. Finally, no direct connections were made between using digital mental health apps and the barrier of stigma and perceived accessibility and acceptance of mental health interventions. This is perhaps a helpful area for further study within the landscape.

    Conclusions

    This study provides evidence that the TFC app is effective in improving symptoms of depression, anxiety, and stress in individuals, particularly among corporate staff. It also demonstrated great improvements in females as compared with males, likely attributed to the design of the app. This is indicative of the need for a consideration of sex differences and occupational groups while designing digital mental health intervention strategies. Results also demonstrated positive user experience among participants, suggesting that similar digital tools can be implemented as an approach to tackling mental health issues within the Asian region.

    Acknowledgments

    This study was supported by the International Medical University (CSc/Sem6[03]2021, CSc/Sem6[]2021, CSc/Sem6[05]2021, and CSc/Sem6[06]2021) and ThoughtFull World Pte Ltd (TFRG0001). The authors thank late professor Philip George for his continuous guidance and expertise.

    Data Availability

    The data sets generated and analyzed during this study are not publicly available owing to internal data handling policies pertaining to privacy and confidentiality but are available from the corresponding author upon reasonable request.

    Authors' Contributions

    KGL, CT, and PPG were involved in study conception and design. YSL, JHQ, XWC, and DTRL provided administrative support. All authors were involved in the provision of study materials. YSL, JHQ, XWC, and DTRL were involved in the collection and assembly of data. YSL, JHQ, XWC, and DTRL were involved in data analysis and interpretation. All authors were involved in writing the manuscript and provided the final approval of the manuscript. The authors have completed the CONSORT (Consolidated Standards of Reporting Trials) 2010 reporting checklist. The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    CONSORT-eHEALTH checklist (V 1.6.1).

    PDF File (Adobe PDF File), 1301 KB
    1. National health and morbidity survey 2015-volume II: non-communicable diseases, risk factors & other health problems. Ministry of Health Malaysia. Jan 2015. URL: https:/​/www.​researchgate.net/​publication/​305213149_National_Health_and_​Morbidity_Survey_2015-VOLUME_II_Non-Communicable_Diseases_Risk_Factors_Other_Health_Problems [accessed 2020-09-02]
    2. Naveed S, Waqas A, Chaudhary AM, Kumar S, Abbas N, Amin R, et al. Corrigendum: prevalence of common mental disorders in south Asia: a systematic review and meta-regression analysis. Front Psychiatry. Nov 20, 2020;11:602062. [FREE Full text] [CrossRef] [Medline]
    3. Robinson J. Rate of depression in Great Britain doubled during COVID-19 pandemic, ONS figures reveal. Pharm J. Aug 19, 2020 [CrossRef]
    4. Kinman G, Wray S. Higher stress: a surveyof stress and well-being among staff in higher education. University and College Union. URL: https:/​/www.​ucu.org.uk/​media/​5911/​Higher-stress-a-survey-of-stress-and-well-being-among-staff-in-​higher-education-Jul-13/​pdf/​HE_stress_report_July_2013.​pdf [accessed 2020-09-20]
    5. Odriozola-González P, Planchuelo-Gómez Á, Irurtia MJ, de Luis-García R. Psychological effects of the COVID-19 outbreak and lockdown among students and workers of a Spanish university. Psychiatry Res. Aug 2020;290:113108. [FREE Full text] [CrossRef] [Medline]
    6. De Kock JH, Latham HA, Leslie SJ, Grindle M, Munoz SA, Ellis L, et al. A rapid review of the impact of COVID-19 on the mental health of healthcare workers: implications for supporting psychological well-being. BMC Public Health. Jan 09, 2021;21(1):104. [FREE Full text] [CrossRef] [Medline]
    7. Mohd Fauzi MF, Mohd Yusoff H, Muhamad Robat R, Mat Saruan NA, Ismail KI, Mohd Haris AF. Doctors' mental health in the midst of COVID-19 pandemic: the roles of work demands and recovery experiences. Int J Environ Res Public Health. Oct 08, 2020;17(19):7340. [FREE Full text] [CrossRef] [Medline]
    8. Espinosa-Guerra E, Rodríguez-Barría E, Donnelly C, Carrera JP. Mental health consequences of COVID-19 in house staff physicians. F1000Res. Mar 11, 2022;11:303. [CrossRef]
    9. Kabito GG, Wami SD, Chercos DH, Mekonnen TH. Work-related stress and associated factors among academic staffs at the University of Gondar, Northwest Ethiopia: an institution-based cross-sectional study. Ethiop J Health Sci. Mar 01, 2020;30(2):223-232. [FREE Full text] [CrossRef] [Medline]
    10. Carolan S. Increasing adherence to digital mental health interventions delivered in the workplace. University of Sussex. Oct 10, 2018. URL: https:/​/sussex.​figshare.com/​articles/​thesis/​Increasing_adherence_to_digital_mental_health_interventions_​delivered_in_the_workplace/​23462978 [accessed 2020-10-15]
    11. Sun W, Wu H, Wang L. Occupational stress and its related factors among university teachers in China. J Occup Health. Aug 04, 2011;53(4):280-286. [FREE Full text] [CrossRef] [Medline]
    12. Shi W, Shen Z, Wang S, Hall BJ. Barriers to professional mental health help-seeking among Chinese adults: a systematic review. Front Psychiatry. May 20, 2020;11:442. [FREE Full text] [CrossRef] [Medline]
    13. Moll S, Eakin JM, Franche RL, Strike C. When health care workers experience mental ill health: institutional practices of silence. Qual Health Res. Feb 06, 2013;23(2):167-179. [CrossRef] [Medline]
    14. Kessler RC, Frank RG. The impact of psychiatric disorders on work loss days. Psychol Med. Jul 1997;27(4):861-873. [CrossRef] [Medline]
    15. Parmenter E, Attridge M. EAP and COVID-19: can robots improve the mental health of the workforce? Benefits Mag. 2021;58(5):38-45.
    16. Framing the issues: digital mental health in Asia. Milken Institute. Nov 15, 2021. URL: https://milkeninstitute.org/report/digital-mental-health-asia [accessed 2020-09-15]
    17. Karyotaki E, Riper H, Twisk J, Hoogendoorn A, Kleiboer A, Mira A, et al. Efficacy of self-guided internet-based cognitive behavioral therapy in the treatment of depressive symptoms: a meta-analysis of individual participant data. JAMA Psychiatry. Apr 01, 2017;74(4):351-359. [FREE Full text] [CrossRef] [Medline]
    18. Research Randomiser homepage. Research Randomiser. URL: https://www.randomizer.org/ [accessed 2020-10-23]
    19. Graham AK, Greene CJ, Kwasny MJ, Kaiser SM, Lieponis P, Powell T, et al. Coached mobile app platform for the treatment of depression and anxiety among primary care patients: a randomized clinical trial. JAMA Psychiatry. Sep 01, 2020;77(9):906-914. [FREE Full text] [CrossRef] [Medline]
    20. Moshe I, Terhorst Y, Philippi P, Domhardt M, Cuijpers P, Cristea I, et al. Digital interventions for the treatment of depression: a meta-analytic review. Psychol Bull. Aug 2021;147(8):749-786. [CrossRef] [Medline]
    21. Smail-Crevier R, Powers G, Noel C, Wang J. Health-related internet usage and design feature preference for e-mental health programs among men and women. J Med Internet Res. Mar 18, 2019;21(3):e11224. [FREE Full text] [CrossRef] [Medline]
    22. Antezana G, Venning A, Smith D, Bidargaddi N. Do young men and women differ in well-being apps usage? Findings from a randomised trial. Health Informatics J. Feb 07, 2022;28(1):14604582211064825. [FREE Full text] [CrossRef] [Medline]
    23. Toh SH, Tan JH, Kosasih FR, Sündermann O. Efficacy of the mental health app intellect to reduce stress: randomized controlled trial with a 1-month follow-up. JMIR Form Res. Dec 14, 2022;6(12):e40723. [FREE Full text] [CrossRef] [Medline]
    24. Ong WY, Sündermann O. Efficacy of the mental health app "intellect" to improve body image and self-compassion in young adults: a randomized controlled trial with a 4-week follow-up. JMIR Mhealth Uhealth. Nov 21, 2022;10(11):e41800. [FREE Full text] [CrossRef] [Medline]
    25. Dean AG, Sullivan KM, Soe MM. OpenEpi: open source epidemiologic statistics for public health, version 2.3.1. ScienceOpen. URL: https://www.scienceopen.com/book?vid=61cdd360-9883-4330-8c18-3f0341b0f715 [accessed 2023-07-04]
    26. Urbaniak GC, Plous S. Research Randomizer (Version 4.0) [Computer software]. Research Randomizer. URL: http://www.randomizer.org/ [accessed 2023-08-13]
    27. Lovibond SH, Lovibond PF. Depression Anxiety Stress Scales – Short Form (DASS-21). NovoPsych. 1995. URL: https://novopsych.com.au/assessments/depression/depression-anxiety-stress-scales-short-form-dass-21/ [accessed 2023-08-13]
    28. Ahmad N, Roslan S, Othman S, Shukor SF, Bakar AY. The validity and reliability of psychometric profile for depression, anxiety and stress scale (DASS21) instrument among Malaysian undergraduate students. Int J Acad Res Bus Soc Sci. Jul 21, 2018;8(6):812-827. [CrossRef]
    29. Aishvarya S, Maniam T, Karuthan C, Sidi H, Nik Jaafar NR, Oei TP. Psychometric properties and validation of the Satisfaction with Life Scale in psychiatric and medical outpatients in Malaysia. Compr Psychiatry. Jan 2014;55 Suppl 1:S101-S106. [CrossRef] [Medline]
    30. Amat S, Subhan M, Marzuki Wan Jaafar W, Mahmud Z, Suhaila Ku Johari K. Evaluation and psychometric status of the brief resilience scale in a sample of Malaysian international students. Asian Soc Sci. Aug 24, 2014;10(18) [CrossRef]
    31. Rahmani AM, Lai J, Jafarlou S, Azimi I, Yunusova A, Rivera AP, et al. Personal mental health navigator: harnessing the power of data, personal models, and health cybernetics to promote psychological well-being. Front Digit Health. Sep 22, 2022;4:933587. [FREE Full text] [CrossRef] [Medline]
    32. Zhang R, Nicholas J, Knapp AA, Graham AK, Gray E, Kwasny MJ, et al. Clinically meaningful use of mental health apps and its effects on depression: mixed methods study. J Med Internet Res. Dec 20, 2019;21(12):e15644. [FREE Full text] [CrossRef] [Medline]

    ‎
    BRS: Brief Resilience Scale
    CONSORT: Consolidated Standards of Reporting Trials
    DASS-21: Depression, Anxiety, and Stress Scale–21 items
    IMU: International Medical University
    RM: Ringgit Malaysia
    SWLS: Satisfaction with Life Scale
    TFC: ThoughtFullChat

    Edited by A Mavragani; submitted 13.02.23; peer-reviewed by C Rodriguez-Paras, L Wah Yun, B Cieślik; comments to author 27.06.23; revised version received 18.07.23; accepted 04.08.23; published 22.09.23.

    Copyright

    ©Yee Siew Lim, Jia Hui Quek, Xiu Wei Ching, Dominic Tao Ran Lim, Kean Ghee Lim, Chandramani Thuraisingham, Parikial Philip George. Originally published in JMIR Formative Research (https://formative.jmir.org), 22.09.2023.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.