This paper is in the following e-collection/theme issue:

Formative Evaluation of Digital Health Interventions (2226) e-Mental Health and Cyberpsychology (1411) Users' and Patients' Needs for Mental Health Services (382)

Published on in Vol 7 (2023)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/35659, first published .
Personalized Help-Seeking Web Application for Chinese-Speaking International University Students: Development and Usability Study

Personalized Help-Seeking Web Application for Chinese-Speaking International University Students: Development and Usability Study

Personalized Help-Seeking Web Application for Chinese-Speaking International University Students: Development and Usability Study

Authors of this article:

Isabella Choi1, 2 Author Orcid Image ;   Gemma Mestroni3, 4 Author Orcid Image ;   Caroline Hunt3, 4 Author Orcid Image ;   Nick Glozier1, 2 Author Orcid Image
Article Authors Cited by (2) Tweetations (4) Metrics

    Original Paper

    1Central Clinical School, Faculty of Medicine and Health, The University of Sydney, Camperdown, Australia

    2Australian Research Council Centre of Excellence for Children and Families Over the Life Course, Sydney, Australia

    3School of Psychology, Faculty of Science, The University of Sydney, Sydney, Australia

    4Brain and Mind Centre, The University of Sydney, Sydney, Australia

    Corresponding Author:

    Isabella Choi, BSc, DClinPsych, PhD

    Central Clinical School

    Faculty of Medicine and Health

    The University of Sydney

    Office 4, Level 5, Professor Marie Bashir Centre

    Missenden Road

    Camperdown, 2050

    Australia

    Phone: 61 0286277240

    Email: isabella.choi@sydney.edu.au


    Background: The mental health of international students is a growing concern for education providers, students, and their families. Chinese international students have low rates of help seeking owing to language, stigma, and mental health literacy barriers. Web-based help-seeking interventions may improve the rate of help seeking among Chinese international students.

    Objective: This study aimed to describe the development of a mental well-being web app providing personalized feedback and tailored psychoeducation and resources to support help seeking among international university students whose first language is Chinese and test the web application’s uptake and engagement.

    Methods: The bilingual MindYourHead web application contains 6 in-app assessments for various areas of mental health, and users are provided with personalized feedback on symptom severity, psychoeducation tailored to the person’s symptoms and information about relevant interventions, and tailored links to external resources and mental health services. A feasibility study was conducted within a school at the University of Sydney to examine the uptake and engagement of the web application among Chinese international students and any demographic characteristics or help-seeking attitudes or intentions that were associated with its engagement.

    Results: A total of 130 Chinese international students signed up on the web application. There was an uptake of 13.4% (122/908) in the schools’ Chinese student enrollment. Most participants (76/130, 58.5%) preferred to use the web application in Chinese and used informal but not formal help for their mental health. There was considerable attrition owing to a design issue, and only 46 students gained access to the full content of the web application. Of these, 67% (31/46) of participants completed 1 or more of the in-app mental well-being assessments. The most commonly engaged in-app assessments were distress (23/31, 74%), stress (17/31, 55%), and sleep (15/31, 48%), with the majority scoring within the moderate- or high-risk level of the score range. In total, 10% (9/81) of the completed in-app assessments led to clicks to external resources or services. No demographic or help-seeking intentions or attitudes were associated with web-application engagement.

    Conclusions: There were promising levels of demand, uptake, and engagement with the MindYourHead web application. The web application appears to attract students who wished to access mental health information in their native language, those who had poor mental health in the past but relied on informal support, and those who were at moderate or high risk of poor mental well-being. Further research is required to explore ways to improve uptake and engagement and to test the efficacy of the web application on Chinese international students’ mental health literacy, stigma, and help seeking.

    JMIR Form Res 2023;7:e35659

    doi:10.2196/35659

    Keywords

    help seeking; mental health; international students; Chinese international students; mental health awareness 


    Help-Seeking Among Chinese-Speaking International Students

    Psychological distress is common among university students. Approximately one-third of students have a common mental disorder during their university education, and of those, only 25% receive treatment [1]. Before the pandemic, international students accounted for approximately 25% of university enrollments in Australia, with the majority coming from China (38%), followed by India (18%) and Nepal (7%) [2]. International students in Australia have similar levels of psychological distress as domestic students [3-5], yet they are significantly less likely to access mental health services [6-8]. A survey found that 17% of international students sought help from a health professional for a mental health problem compared with 55% of domestic students at an Australian university [4]. Similarly, another web-based survey of students at an Australian university and associated tertiary college found that only 21% of international students reported having previously used a mental health service compared with 58% of domestic students [6]. International students reported more negative attitudes and lower intentions for seeking help for mental health concerns compared with domestic students [6]. As the number of international students is likely to rebound after COVID-19, it is vital to support help seeking in this population.

    Low rates of help seeking for mental health problems are evident among Chinese-speaking international students, with 1 survey finding that more than half of Chinese-speaking international students in Australia reported high psychological distress, yet only 9% sought help from mental health services [7]. Common barriers to help seeking among Chinese-speaking international students included not recognizing their symptoms as part of mental health problems, thinking their problem was not severe or serious enough, and not being aware of the available treatment services [7]. In addition, Chinese migrants have high levels of stigmatizing attitudes toward mental illness [9], and fear of losing face can hinder their help seeking [10]. They can face practical barriers such as concerns about the cost of services or transportation difficulties, lack of time, and language barriers [7]. Although all international students attending university in Australia must possess a certain level of English proficiency, nonnative English speakers can find it difficult to understand mental health–related issues or express their concerns in a second language owing to fears of not being understood or feelings of embarrassment or shame [11,12]. As such, many Chinese-speaking international students preferred to handle the problem alone or rely on an intimate partner, family, or friends for help [13].

    Digital Help-Seeking Interventions

    Help-seeking interventions aim to improve help-seeking attitudes, intentions, and behaviors through delivering information to improve mental health literacy; reducing stigma surrounding mental disorders; providing help-seeking source information about how and where to find potential providers of help; and, to a lesser extent, providing personalized feedback about the individual’s symptoms to prompt help seeking among those with higher symptom levels [14]. A meta-analysis found that help-seeking interventions improved attitudes, intentions, and behaviors to seek formal help for mental health problems, with long-term benefits for formal help-seeking behaviors [15]. A recent systematic review also provided support that digital help-seeking interventions can improve help-seeking intentions, but more research is needed on actual help-seeking behaviors [16].

    There is some support for digital interventions improving help seeking among university students [17-22]. A web-based psychoeducation intervention provided university students with vignettes of a young person experiencing a mental health problem, alongside with a description of symptoms and treatment options, and challenging stigmatizing views [21]. There was an increase in mental health literacy and a decrease in mental health stigma in the intervention group compared with the control group [21]. In addition, the changes in the level of knowledge and stigma were associated with more positive help-seeking attitudes and intentions [21]. In the German site of the World Health Organization’s World Mental Health International College Student initiative, university students were randomized to either an intervention condition or a control condition. Students in the intervention condition additionally received personalized feedback based on symptom severity, psychoeducation tailored to the personal symptom profile, and information about mental health services [18]. Those who received personalized feedback on their mental health were more likely to report significantly higher intentions to use mental health support services in the next semester than those who did not [18]. Only 1 web-based help-seeking intervention specifically targeted international students [17]. The intervention provided information on depression symptoms, treatment, and help options, which improved help-seeking attitudes but not help-seeking intentions [17].

    Objectives

    Although these findings are promising, it is unclear whether such effects will generalize to a Chinese-speaking international student population that faces additional language and attitudinal barriers to help seeking. It is important to consider the language, culture, and context when designing interventions for people from culturally diverse backgrounds, and there is evidence that suggests that highly culturally adapted digital mental health interventions are more efficacious [23]. This paper describes the development and feasibility study of a bilingual help-seeking digital intervention that provides personalized feedback and psychoeducation to support help seeking among Chinese-speaking international university students. The feasibility study aims to examine the uptake and engagement of the intervention among Chinese-speaking international students and to explore if any participant demographic characteristics, help-seeking attitudes, and help-seeking intentions were associated with its engagement.


    Development Overview

    We developed a minimum viable product (MVP) of a bilingual mental well-being help-seeking website application, MindYourHead, for Chinese first-language international students. The design of the web application was guided by the dynamic process model of help seeking by Rickwood et al [24], whereby help seeking begins with the awareness of symptoms and appraisal of having a problem, followed by the expression of symptoms and need for support; the identification of accessible help sources; and, finally, the willingness to seek help and disclose relevant information [24]. Informed by this model, the MindYourHead web application was designed to address help-seeking barriers relevant to Chinese international students by providing (1) personalized feedback based on individual symptom severity to address low symptom awareness, (2) psychoeducation tailored to symptoms and information about interventions appropriate to their symptom level to address the barriers of low mental health literacy and beliefs about the helpfulness of interventions, (3) tailored links to external resources and mental health services to address potential barriers of insufficient knowledge about mental health services and ease of access, (4) psychoeducation addressing common questions about mental health and help seeking and issues such as cost and confidentiality, and (5) links to crisis support services. The web application was available in simplified Chinese and English to address language barriers. The web application could be accessed by a mobile phone, tablet, and desktop.

    The content for the web application was developed through a review of the relevant literature on common mental health issues faced by international students as well as consultation with the target population through user design workshops. A series of 4 user design workshops were conducted with Chinese-speaking international students. In the initial workshops, the students were presented with the objective and in-principal design of the web application. They were asked to generate areas of mental well-being they were interested in self-assessing, the type of information they wished to learn from the personalized feedback, and the type of services or resources they would find helpful. The workshop also explored the language used by Chinese international students to describe their emotional and personal problems and how to make the web application engaging for students. In the subsequent workshops, students were guided to use the prototype web application through a read-aloud format, and they were asked to comment on how useful they found the feedback and resources that were provided. They also commented on the indexes of user experience that were used to refine the layout and functionality of the web application. User feedback informed the final MVP design, but workshop findings are not presented as no ethical approval was obtained for conducting the workshops.

    We also implemented many of the recommendations identified in a literature review to help guide the development of better and more rigorous apps [25]. This included providing mental health information, providing links to crisis support services, providing tailoring interventions as per the individual’s needs, using a cognitive behavioral therapy framework, recommending helpful activities outside of the app, and creating a simple and intuitive interface [25]. Another design consideration was to address core psychological needs such as autonomy, competence, and relatedness, which can improve engagement outcomes with mental health apps and could be a critical factor to consider when developing apps [26]. The MindYourHead web application provided autonomy by allowing users the freedom to pick and choose which content to explore in any order they choose. Feelings of competence and relatedness may be increased through the simple and intuitive design of the MindYourHead web application and its availability in the user’s preferred language. An international student with Chinese as their first language studying a medicine degree was involved in translating the English content developed by a clinical psychologist (IC) and a clinical psychology registrar (GM) on the team into simplified Chinese. The translations were reviewed by IC, who was also fluent in Chinese, and by Chinese-speaking international students from the workshops.

    Description of MindYourHead MVP

    Overview

    The home page of MindYourHead presented users with a choice of 6 in-app assessments. The in-app assessments were informed by the user design workshops and used validated mental health screening questionnaires, including psychological distress, sleep, stress, alcohol consumption, smoking, and social support (Figure 1). Participants could decide which in-app assessment they wished to explore. They had the autonomy to select to complete as many or as few of the in-app assessments as they wished. On the completion of each in-app assessment, users were provided with immediate personalized feedback related to their symptom severity, brief psychoeducation tailored to symptoms and relevant interventions, and tailored links to external mental health services and web resources (Figure 2). Participants could also select whether to use the web application in English or simplified Chinese.

    ‎
    Figure 1. Home page (English and Chinese).
    ‎
    Figure 2. Example of feedback for high distress (English and Chinese).
    In-App Assessments
    Distress

    Distress was measured using the Kessler Psychological Distress Scale (K10) [27], a widely used and well-validated short-form measure for distress [28]. The K10 asks 10 questions related to how depressed and anxious the individual has been feeling over the last month, with scores ranging from 10 to 50. Participants were asked to indicate how often they have felt a particular way (eg, hopeless) on a scale from 1=all of the time to 5=none of the time. The Chinese version of the K10 also demonstrated good reliability and validity [29]. The K10 was labeled as the My Wellbeing assessment, with a score between 10 and 15 indicating low distress, 16 and 29 indicating moderate distress, and 30 and 50 indicating high distress, using cutoffs from Australian K10 scoring norms [30].

    Sleep Problems

    Sleep problems were measured using the Insomnia Severity Index (ISI) [31]. The ISI asks 7 questions related to sleep problems over the past 2 weeks, with scores ranging from 0 to 28. Participants were asked to rate the severity of the problems on a scale from 0=none to 4=very severe. The Chinese version of this scale has also been found to be reliable and valid [32]. This was labeled as the My Sleep assessment in the app, with scores between 0 and 14 indicating Low Sleep Problems, 15 and 21 indicating Moderate Sleep Problems, and 22 and 28 indicating High Sleep Problems [31].

    Stress

    Stress was labeled as My Stress assessment in the app. It was measured using the Perceived Stress Scale (PSS) [33]. Participants were asked to rate their thoughts and feelings over the past month across 10 items, such as “In the last month, how often have you been upset because of something that happened unexpectedly?” Each item was rated on a scale from 0=never to 4=very often, producing a total score ranging from 0 to 40. The simplified Chinese version of this scale also has satisfactory psychometric properties [34]. A score between 0 and 13 indicated Low Stress, 14 and 26 indicated Moderate Stress, and 27 and 40 indicated High Stress.

    Alcohol Consumption

    Alcohol consumption was measured using the Alcohol Use Disorders Identification Test–Concise (AUDIT-C) [35], a 3-item screening measure that has been used extensively to identify those at risk for an alcohol use disorder. This short-form scale has demonstrated construct validity and internal consistency in a student population, with a Cronbach α of .80 [36]. Each item has a rating between 0 and 4 points for questions relating to how often and how much alcohol the individual consumes, and total scores range from 0 to 12. The AUDIT-C is the only alcohol screening test specifically designed for international use, and the Chinese version has been demonstrated to have satisfactory psychometric properties [37]. This was labeled as My Drinking Habits, with a score between 0 and 6 indicating low-risk consumption and 7 and 12 indicating high-risk consumption [36].

    Smoking

    Smoking was measured using 3 screening questions to assess nicotine dependence taken from the Fagerstrom Test for Nicotine Dependence [38]. The questions included the following: “How soon after waking do you smoke your first cigarette?” (if within 30 minutes, indicative of nicotine dependence); “How many cigarettes do you smoke on a typical day?” (if >10 a day, indicative of nicotine dependence); and “If you have previously attempted to quit, did you experience withdrawals or cravings?” A positive response to any question indicated high risk. The simplified Chinese version has been shown to have satisfactory reliability and validity [39]. This formed the My Smoking Habits assessment.

    Social Support

    Social support was labeled as My Social Support assessment in the app. It was measured using the Multidimensional Scale of Perceived Social Support [40]. The scale includes 12 items that ask questions related to whether the individual feels that they have strong social support from family, friends, or a significant other. Responses were made on a 7-point Likert scale from 1=very strongly disagree to 7=very strongly agree, with total scores ranging from 12 to 84. This scale has demonstrated good reliability and validity in both the original version [41] and the Chinese version [42]. A total score of 12-35 indicated Low Social Support, 36-60 indicated Moderate Social Support, and 61-84 indicated High Social Support.

    Personalized Feedback

    Users received personalized risk feedback immediately after completing the questionnaires based on their score level, along with 3 levels of information: “what does this mean?” “what can I do?” and “where can I find help?” (Figure 2). Each level of feedback was tailored to the individual’s score for the relevant scale for the particular in-app assessment (eg, K10). Participants’ scores fell into one of the 2 or 3 levels (eg, high distress, moderate distress, or low distress). The feedback included information about symptoms, frequency in the student population, strategies to reduce symptoms and increase well-being, and the available evidence-based treatments. Users were provided with information and links for up to 4 relevant external mental health services or evidence-based resources (eg, university counseling service, university health service, MyCompass app [43], and ThisWayUp web-based clinic [44]). Whenever a user scored within the high-risk range on any measure, they were encouraged to seek support from the university counseling service and a general practitioner.

    Other App Sections

    The web application included an emergency help section that provided the telephone number and links to emergency and other 24-hour support services and translation services. The “Frequently Asked Questions About Getting Support” section addressed common questions about mental health, access to treatment, costs, and confidentiality, as guided by the user design workshops. Users could explore these sections freely.

    Feasibility Study

    Aims

    We assessed the feasibility of the MindYourHead web application by evaluating its uptake and engagement among Chinese-speaking international students within a school at the University of Sydney. We further aimed to explore whether any participant’s demographic characteristics and preexisting help-seeking attitudes and intentions were associated with engagement with the web application.

    Ethics Approval

    The feasibility study was approved by the Human Research Ethics Committee of the University of Sydney (project number: 2020/610).

    Recruitment

    Participants had to be Chinese-speaking international students, enrolled in a course at the University of Sydney, with access to a reliable internet connection. The recruitment was conducted through the University of Sydney’s School of Architecture, Design, and Planning to determine the uptake of the MindYourHead web application by a pool of potential students. The school had 908 Chinese international students (from China, Hong Kong, Macau, and Taiwan) during the semester in which this study was conducted. The school sent 3 recruitment emails to enrolled students between October 2020 and December 2020, inviting Chinese-speaking international students to participate in a study to test a mental well-being web application for Chinese international students. The email provided the students with a link to the landing page of the web application (Figure 3). No reimbursement was offered to students for participating in the study.

    ‎
    Figure 3. Landing page (English and Chinese).
    Measures
    Baseline Self-report Questionnaires

    Participants completed the demographic information before gaining access to the full content of the web application. Questions included their age; gender; home country; current location; and faculty, level, and year of study. Participants were asked to rate their English language proficiency (very well, well, not well, or not at all). Participants were also asked whether they had ever experienced a period of poor mental health in the past and were asked to indicate whether they had ever sought help for their mental health across several different sources (ie, mental health professionals, friends, web-based resources, etc).

    The Attitudes Toward Seeking Professional Psychological Help Scale–Short Form (ATSPPH-SF) [45] was used as a measure of participants’ attitudes toward seeking professional help for their mental health. The scale has demonstrated internal consistency ranging from 0.82 to 0.84 among samples of college students [45]. The scale has the following 2 factors: openness to seeking treatment for emotional problems and perceived value and need in seeking treatment. Scores range from 0 to 30, with higher scores indicating a more positive treatment attitude and shown to be associated with less treatment-related stigma and greater intentions to seek treatment in the future [46]. The Chinese version of the questionnaire has demonstrated strong reliability and validity [47].

    The General Help-Seeking Questionnaire (GHSQ) [48] was used to assess the intention to seek help for personal or emotional problems from different sources. The scale has demonstrated reliability and validity as a flexible measure that can be applied to various contexts (Cronbach α=.70; test-retest reliability=0.86) in both English [48] and Chinese [49]. Additional sources of help seeking relevant to the university population were included in addition to common sources of help, such as mental health websites (eg, Beyond Blue and Headspace), e–mental health apps (eg, Smiling Mind app and My Compass app), university counseling and psychology services, and university staff. Participants answered how likely they were to use various mental health services if they had a personal or emotional problem on a 7-point Likert scale from 1=extremely unlikely to 7=extremely likely.

    Engagement and Help-Seeking Measures

    The web application automatically recorded objective engagement data, including the number of assessments completed, assessment scores and symptom/risk level, and link clicks to external resources.

    Procedure

    Participants read the participant information statement and provided informed consent within the web application. Then, they completed a demographic questionnaire, which generated a unique QR code that participants could save on their devices (Figure 4). The QR code was used as a deidentified identifier to track the participant’s data and allowed the participant to log in to their account upon return. On receiving their unique QR codes, the participants completed the baseline measures (ATSPPH-SF and GHSQ). They were then given access to the home page of the MindYourHead web application. This gave them access to the in-app assessments and personalized feedback. The in-app assessments were for distress (K10), sleep (ISI), stress (PSS), alcohol use (AUDIT-C), smoking (Fagerstrom Test for Nicotine Dependence), and social support (Multidimensional Scale of Perceived Social Support; Figure 1). The participants were able to freely access the content of the web application as many times as they wished and complete as many or as few of the assessments as they wished over the 3-month study period.

    ‎
    Figure 4. Example of the QR code page after completion of demographic questionnaires.
    Statistical Analysis

    Quantitative data were analyzed using SPSS (version 23.0; IBM). Descriptive statistics were used for participant characteristics and app use data to demonstrate the uptake and engagement of the web application. Independent 2-tailed t tests and Pearson chi-square tests of independence were conducted to determine whether there were any significant demographic or help-seeking intention and attitude differences between participants who gained access to the full content of the web application and those who did not and between participants with full access to the web application who engaged with at least one assessment and those who did not engage in any. Pearson correlation coefficient tests were conducted to determine whether there were any correlations between the participant’s intentions and attitudes toward help seeking (ie, scores on the ATSPPH-SF and GHSQ) and their engagement within the web application (ie, the number of in-app assessments completed). The α level was set at .05 to determine statistical significance.


    Uptake

    A total of 149 participants consented to participate in the feasibility study. In total, 12.8% (19/149) did not meet the inclusion criteria and were excluded from the analysis, resulting in 130 eligible participants who met the inclusion criteria and completed the demographic questionnaire. Of the 130 participants, 122 (93.8%) were from the School of Architecture, Planning and Design, representing an uptake of 13.4% (122/908) of the school’s enrollment of Chinese international students. Figure 5 illustrates the participant flow.

    ‎
    Figure 5. Flowchart of participant engagement at each step within the study. AUDIT-C: Alcohol Use Disorders Identification Test–Concise; FTND: Fagerstrom Test for Nicotine Dependence; ISI: Insomnia Severity Index; K10: Kessler Psychological Distress Scale; MSPSS: Multidimensional Scale of Perceived Social Support; PSS: Perceived Stress Scale.

    Demographic Characteristics

    Table 1 presents the demographic characteristics of the participants. The mean age of the participants was 22 (SD 2.23; range 18-28) years. Most participants were women (88/130, 67.7%), from China (120/130, 92.3%), currently residing in Australia (81/130, 62.3%), studying at an undergraduate level (74/130, 56.9%), and studying full time (127/130, 97.7%). All participants indicated that they had some level of proficiency in speaking English, but 33.8% (44/130) of participants reported that they spoke English not well. More than half of the participants (76/130, 58.5%) chose to use the web application in Chinese. Approximately two-thirds of participants (81/130, 62.3%) indicated that they had experienced poor mental health in the past. When asked about help seeking for their mental health in the past, most participants (78/130, 60%) mentioned that they had sought help from a friend. Less than a quarter of the participants reported seeking help from a mental health professional (27/130, 20.8%) or a general practitioner (19/130, 14.6%). Few participants (6/130, 4.6%) had ever sought help for mental health issues through web-based information via websites or using e–mental health programs or apps.

    Table 1. Demographic characteristics and significance statistics of the participants who joined the study, those with full access, and those who completed an in-app assessment (N=130).

    		Total uptake (N=130)Full access to app content (n=46)Completed ≥1 in-app assessments (n=31)Full access vs not gained access (n=46 vs n=84)P valueCompleted ≥1 in-app assessments vs none (n=31 vs n=15)P value
    Age (years), mean (SD)22 (2.23)22.3 (2.25)22.65 (2.36)t128=1.151.25t44=1.497;.14
    Gender, n (%)

    		Woman88 (67.7)32 (69.6)23 (74.2)χ21=0.1.84χ21=1.4.30

    		Man39 (30)13 (28.3)7 (22.6)—a———

    		Nonbinary3 (2.3)1 (2.1)1 (3.2)————
    The language used in the web application, n (%)

    		English54 (41.5)8 (17.4)4 (12.9)χ21=17.1<.001χ21=1.3.41

    		Chinese76 (58.5)38 (82.6)27 (87.1)————
    Home country, n (%)

    		China120 (92.3)44 (95.6)30 (96.8)N/AbN/AN/AN/A

    		Hong Kong7 (5.4)1 (2.2)0 (0)————

    		Other (Taiwan or Singapore)3 (2.3)1 (2.2)1 (3.2)————
    Currently in Australia, n (%)

    		Yes81 (62.3)28 (60.9)19 (61.3)χ21=0.1.85χ21=0.0>.99

    		No49 (37.7)18 (39.1)12 (38.7)————
    Number of years in Australia, n (%)

    		<134 (26.2)11 (23.9)8 (25.8)χ23=1.1.78χ23=5.9.12

    		1-234 (26.2)14 (30.4)7 (22.6)————

    		2-430 (23.1)9 (19.6)5 (16.1)————

    		>432 (24.6)12 (26.1)11 (35.5)————
    Currently studying, n (%)

    		Undergraduate74 (56.9)26 (56.5)15 (48.4)χ21=0.0>.99χ21=2.6.13

    		Postgraduate56 (43.1)20 (43.5)16 (51.6)————
    Year of study, n (%)

    		First year64 (49.2)20 (43.5)16 (51.6)χ22=1.0.61χ22=2.6.27

    		Second year44 (33.8)17 (37)10 (32.3)————

    		Third year or above22 (16.9)9 (19.6)5 (16.1)————
    Study mode, n (%)

    		Full time127 (97.7)46 (100)31 (100)N/AN/AN/AN/A

    		Part time3 (2.3)0 (0)0 (0)————
    English proficiency, n (%)

    		Very well16 (12.3)5 (10.9)2 (6.5)χ22=0.7.71χ22=2.3.31

    		Well70 (53.8)27 (58.7)20 (64.5)————

    		Not well44 (33.8)14 (30.4)9 (29)————
    Poor mental health in the past, n (%)

    		Yes81 (62.3)33 (71.7)23 (74.2)χ21=2.7.13χ21=0.3.73

    		No49 (37.7)13 (28.3)8 (25.8)————
    Previously sought help from, n (%)

    		Mental health professional27 (20.8)12 (26.1)10 (32.3)χ21=1.2.37χ21=1.9;.29

    		General practitioner19 (14.6)5 (10.9)4 (12.9)χ21=0.8.44χ21=0.4>.99

    		Intimate partner30 (23.1)14 (30.4)8 (25.8)χ21=2.2.19χ21=1.0.50

    		Friend78 (60)29 (63)23 (74.2)χ21=0.3.71χ21=5.1.048

    		Family29 (22.3)12 (26.1)8 (25.8)χ21=0.1.82χ21=0.0>.99

    		Helpline5 (3.8)2 (4.3)2 (6.5)N/AN/AN/AN/A

    		Web-based resources or app6 (4.6)3 (6.5)3 (9.7)N/AN/AN/AN/A

    		Other2 (1.5)1 (2.2)1 (3.2)N/AN/AN/AN/A

    aNot available.

    bN/A: not applicable.

    Engagement

    Overall App

    There was a significant dropout of participants upon receiving the QR code, with only 35.4% (46/130) of participants proceeding to complete the baseline questionnaires (ATSPPH-SF and GHSQ). Participants who selected to use the web application in Chinese were more likely to complete the baseline measures and gain access to the full content of the app than those who selected to use the app in English (P<.001). There were no other significant differences between participants who gained access to the full content of the app and those who dropped out (Table 1). On investigation, it appeared that many participants thought they should scan the QR code instead of clicking on the go to home button (Figure 4), which may have accounted for the high level of dropout.

    In-App Assessment Completion Rates and Clicks to Resources

    Of the 46 participants who gained access to the full content of the MindYourHead web application, 31 (67%) completed ≥1 in-app assessments (Figure 5). Among the 31 participants who engaged in any in-app assessment, 11 (24%) used 1 assessment, 4 (9%) used 2 assessments, 7 (15%) used 3 assessments, 6 (13%) used 4 assessments, 1 (2%) used 5 assessments, and 2 (4%) used all 6 assessments.

    Figure 6 shows the completion rates and participant score severity for the in-app assessments. My Well-being had the highest level of engagement, with 74% (23/31) of the participants completing the K10 questionnaire and 2 participants clicking on links to external resources (1 scoring moderate risk clicked on the MyCompass app [43] and 1 scoring high risk accessed the university counseling service website). This was followed by My Stress, with 55% (17/31) completing the PSS questionnaire and 4 participants with moderate-risk scores clicking on links to external resources (1 accessed the university counseling service, 2 accessed the This Way Up web-based clinic [44], and 1 used the Happify app [50]). My Sleep was used by 48% (15/31) of the participants, with no clicks to external resources. My Social Support was used by 35% (11/31) of the participants, and 1 individual with a low-risk score clicked on a link to the university’s Chinese Student Association. My Smoking was used by 26% (8/31) of the participants, and 1 high-risk scorer clicked on the link to the My QuitBuddy app [51]. My Alcohol was used by 23% (7/31) of the participants, with no clicks to external resources.

    Of the total 81 assessments completed, there was a total of 9 (11%) link clicks to external resources. Although the moderate- and high-risk categories appeared to attract more clicks than the low-risk categories, this was not significant (χ22=3.1; P=.22; Table 2).

    ‎
    Figure 6. Number of in-app surveys completed for each area of content, with participant score severity shown (ie, low, moderate, or high). AUDIT-C: Alcohol Use Disorders Identification Test–Concise; FTND: Fagerstrom Test for Nicotine Dependence; ISI: Insomnia Severity Index; K10: Kessler Psychological Distress Scale; MSPSS: Multidimensional Scale of Perceived Social Support; PSS: Perceived Stress Scale.
    Table 2. Total clicks by risk level across all in-app assessments (n=81).

    		Total clicks, n (%)Did not click, n (%)
    High risk2 (19)14 (81)
    Moderate risk5 (16)27 (84)
    Low risk1 (3)32 (97)
    Associations With App Engagement

    There was a trend for participants to have engaged in an in-app assessment if they had previously sought help for their mental health from a friend, but this did not reach statistical significance (χ21=3.7; P=.054). There were no other significant differences between participants who engaged with the in-app assessment content and those who did not in terms of participant characteristics and help-seeking intentions or attitudes (Table 1). There was no significant correlation between the participants’ attitudes toward seeking help for mental health and their level of engagement (Table 3).

    Table 3. Correlation between help-seeking attitudes and intentions scores and the number of in-app assessments completed (n=31).
    VariablesValues, mean (SD)Number of assessments completed


    		Correlation coefficient, rP value
    Intention to seek help from...

    		Friend5.39 (1.45)0.04.83

    		Partner4.84 (1.97)0.09.64

    		Parent3.58 (1.63)0.12.52

    		Other relative2.55 (1.69)0.02.91

    		Mental health professional4.65 (1.17)0.27.14

    		Physician3.97 (1.33)0.08.69

    		University staff2.74 (1.26)0.10.59

    		Religious leader2.13 (1.67)−0.34.06

    		Helpline2.23 (1.38)0.06.76

    		Web-based resources3.48 (1.63)0.18.32

    		Apps3.23 (1.33)0.22.23

    		No one3.26 (1.63)−0.30.10
    ATSPPH-SFa score16.8 (2.17)−0.14.44

    aATSPPH-SF: Attitudes Toward Seeking Professional Psychological Help Scale–Short Form.


    Principal Findings

    This feasibility study described the MindYourHead web application and examined its uptake and engagement among Chinese-speaking international students and any demographic and help-seeking characteristics associated with its engagement. The MindYourHead web application is a bilingual web-based help-seeking intervention that contains in-app assessments for various areas of mental health. Within each in-app assessment, users are provided with personalized feedback on symptom severity, tailored psychoeducation on symptoms and relevant interventions, and tailored links to external resources and mental health services. The uptake of the web application among Chinese-speaking international students within the school was 13.4% (122/908). The majority of participants (76/130, 58.5%) who signed up on the web application and accessed its content did so in Chinese. In total, 67% (31/46) of the participants who accessed the full app content used at least one in-app assessment, with the distress, stress, and sleep in-app assessments being the most popular. There were no demographic or help-seeking intentions or attitudes associated with engagement with the web application.

    Uptake

    There is a general lack of reporting on the uptake rates of digital mental health interventions [52,53]. The uptake of digital interventions for the treatment of mental health conditions such as stress and depressive symptoms among university populations was reported to mostly range from 1% to 8% [54-59], with one study reporting 16.9% [60]. In our study, 13.4% (122/908) of the Chinese-speaking international students enrolled at the school signed up to use the MindYourHead web application through a passive recruitment strategy of 3 student email communications over the semester. This uptake is comparable with other personalized feedback interventions for university students, which are aimed at encouraging help seeking. A study that offered personalized assessments for depression and suicide risk reported an 8% uptake [55], and another study targeting suicide risk reported that 25% of invited students began the screening survey [20]. The World Mental Health International College Student initiative sent up to 6 reminder emails to college students, which resulted in 25.9% starting the screening survey and 12.3% completing the surveys and being eligible to receive feedback on their symptom level [18]. This feasibility study demonstrated a promising initial uptake of the MindYourHead web application among Chinese-speaking international students to warrant further experimental testing, although more research is needed to explore ways to improve uptake among this population and students in general.

    Engagement

    Although the MindYourHead web application was not designed to be used in a linear fashion, and it allowed users to pick and choose the in-app assessments that were relevant or of interest to them, 67% (31/46) of the participants engaged with at least one in-app assessment in the MindYourHead web application, and of those, most participants (20/31, 65%) completed >1 in-app assessment. This level of participant engagement with the web application content was comparable with other digital mental health interventions, despite it not offering any symptom intervention. Approximately 21%-88% of users engaged minimally (completing at least one module or assessment) in digital self-help interventions for depression and anxiety [52]. Specifically, studies on digital mental health treatment interventions for university students have reported that 11% of participants started a web-based gratitude intervention [61], 68% opened at least one web session of a web-based therapist-assisted–acceptance-based behavioral intervention targeting anxiety [62], 74% used at least some of the unguided web-based self-help interventions (MoodGYM) [63], 76% began a web-based stress management intervention [64], and 85% completed lesson 1 of a web-based acceptance and commitment therapy prevention program [65].

    Participants engaged most with content related to psychological distress, stress, and sleep but less with content related to alcohol consumption and smoking behavior. Most of the participants (48/81, 59%) who completed assessments scored in either moderate or high ranges, especially on distress and stress, suggesting that the web application attracted those who required support for early intervention or treatment. There was also a trend for those who clicked links to external resources to have scored at a moderate to a high level of risk in the in-app assessments, suggesting that the web application can support those who need to seek external support. However, the overall proportion of clicks was still low, and further qualitative research is needed to explore the reasons users click or do not click on external resources.

    Demand

    This feasibility study suggests that there is a demand within this target population for a web application to support help seeking. Consistent with previous research, participants demonstrated low formal help-seeking intentions and behaviors. Despite almost two-thirds of the participants (81/130, 62.3%) having experienced poor mental health in the past, only 21% (27/130) had previously sought help from a mental health professional. On average, the participants had negative attitudes toward seeking professional help for emotional problems. Instead, the participants most commonly sought help from a friend for their mental health. Reliance on friends and self and questioning the severity of their issues were barriers to seeking professional help among Chinese international students [7]. We found that there was a trend for help seeking in the past from a friend to be associated with engagement in the in-app assessments, suggesting that the MindYourHead web application may support people who traditionally relied on informal help seeking by potentially linking them with formal services and resources.

    This study also demonstrates the importance of developing mental health apps and resources for international students in their first language. Although international students in Australia, especially at the tertiary level, need to meet English language requirements to gain entry into their course and receive a student visa, one-third of the Chinese first-language international student participants (44/130, 33.8%) indicated that they spoke English not well. Furthermore, there was a strong preference for using the web application in Chinese among users. This is in line with past studies that have shown language barriers to be a significant obstacle in seeking help for mental health [7,66]. The level of uptake and engagement with MindYourHead may be owing to the web application offering mental health information in the students’ preferred language, although the low number of link clicks to external resources could be owing to the lack of multilingual resources available.

    Limitations

    A major limitation was the design issue related to the QR code, which resulted in high attrition of participants before completing the baseline questionnaires and accessing the full web application. It is not known whether the help-seeking attitudes or intentions of participants who dropped out were different from those who accessed the full app content. It is possible that without confusion regarding QR codes, the actual engagement rate of the MindYourHead web application may have been higher. However, the dropout could also reflect that participants found the baseline questionnaires onerous to complete or that they did not know that there was more content to the app. Future versions of the MindYourHead app will move away from using QR codes and provide an onboarding of the app such that the users know what to expect. More extensive testing of prototypes with user feedback should also be conducted before commencing any future studies.

    The MindYourHead web application was limited by the lack of multilingual resources available in Australia. Although care was taken to include Chinese-language mental health resources wherever possible, most links to external resources were only available in English, which may have limited the clicks to external resources from participants. However, this limitation likely reflects a more systemic issue regarding the paucity of multilingual mental health resources in Australia. Future versions of the app should aim to include more multilingual resources where possible or develop in-app self-help resources available in the target language.

    Finally, this feasibility study focused on uptake and engagement with the app. No satisfaction or usability metrics were used, and there were no postintervention measures assessing changes in participants’ help-seeking attitudes or intentions. Furthermore, the study only measured clicks to external resources, but it is unknown whether participants actually engaged with those external resources. A randomized controlled trial is needed in the future to test the efficacy of apps in improving mental health literacy, stigma, and help-seeking intentions and attitudes as well as to qualitatively explore user experience, satisfaction, and acceptability.

    Conclusions

    To the best of our knowledge, MindYourHead is the first mental well-being web application targeted at Chinese-speaking international students to support their help seeking and is available in both English and Chinese. This feasibility study of the MindYourHead web application MVP showed promising uptake and engagement among Chinese-speaking international students, comparable with other digital mental health interventions for university students. Furthermore, the web application appears to attract international students who traditionally relied on friends for mental health support by engaging them with in-app assessments and potentially directing them to formal services and resources. This study also provided useful information that can guide future versions of the web application to support international students, especially the importance of making mental health information and resources available in the target population’s native language. Further research is required to explore ways to improve uptake and engagement and to test the efficacy of the web application on Chinese international students’ help seeking in a trial.

    Acknowledgments

    The study is funded by the University of Sydney-Brain and Mind Centre Research Development Grant. The funders had no role in the study design and analysis, the interpretation of the results, or the decision to submit this work for publication. The authors would like to thank The University of Sydney Information and Communications Technology Techlab team who worked on web application build and usability testing; Cheng Sophia Jin for assisting with the Chinese translations; the University of Sydney’s Student Life program and the School of Architecture, Planning and Design for supporting the study; the workshop participants; and the study participants. At the time the final manuscript was being prepared, IC and NG were supported (partially or fully) by the Australian government through the Australian Research Council’s Centre of Excellence for Children and Families Over the Life Course (project ID CE200100025).

    Conflicts of Interest

    None declared.

    1. Bruffaerts R, Mortier P, Auerbach RP, Alonso J, Hermosillo De la Torre AE, Cuijpers P, WHO WMH-ICS Collaborators. Lifetime and 12-month treatment for mental disorders and suicidal thoughts and behaviors among first year college students. Int J Methods Psychiatr Res 2019 Jun;28(2):e1764 [FREE Full text] [CrossRef] [Medline]
    2. Ferguson H, Sherrel H. Overseas students in Australian higher education: a quick guide. Parliament of Australia.   URL: https:/​/www.​aph.gov.au/​About_Parliament/​Parliamentary_Departments/​Parliamentary_Library/​pubs/​rp/​rp2021/​Quick_Guides/​OverseasStudents [accessed 2020-07-04]
    3. Said D, Kypri K, Bowman J. Risk factors for mental disorder among university students in Australia: findings from a web-based cross-sectional survey. Soc Psychiatry Psychiatr Epidemiol 2013 Jun;48(6):935-944. [CrossRef] [Medline]
    4. Skromanis S, Cooling N, Rodgers B, Purton T, Fan F, Bridgman H, et al. Health and well-being of international university students, and comparison with domestic students, in Tasmania, Australia. Int J Environ Res Public Health 2018 Jun 01;15(6):1147 [FREE Full text] [CrossRef] [Medline]
    5. Stallman H. Psychological distress in university students: a comparison with general population data. Australian Psychologist 2010 Nov 06;45(4):249-257 [FREE Full text] [CrossRef]
    6. Clough BA, Nazareth SM, Day JJ, Casey LM. A comparison of mental health literacy, attitudes, and help-seeking intentions among domestic and international tertiary students. British J Guidance Counselling 2018 Apr 08;47(1):123-135. [CrossRef]
    7. Lu SH, Dear BF, Johnston L, Wootton BM, Titov N. An internet survey of emotional health, treatment seeking and barriers to accessing mental health treatment among Chinese-speaking international students in Australia. Counselling Psychol Q 2013 Aug 07;27(1):96-108. [CrossRef]
    8. Raunic A, Xenos S. University counselling service utilisation by local and international students and user characteristics: a review. Int J Adv Counselling 2008 Oct 4;30(4):262-267. [CrossRef]
    9. Mellor D, Carne L, Shen Y, McCabe M, Wang L. Stigma toward mental illness: a cross-cultural comparison of Taiwanese, Chinese immigrants to Australia and Anglo-Australians. J Cross Cultural Psychol 2012 Jul 20;44(3):352-364. [CrossRef]
    10. Ma S, Zhu Y, Bresnahan M. Chinese international students' face concerns, self-stigma, linguistic factors, and help-seeking intentions for mental health. Health Commun 2022 Nov 08;37(13):1631-1639. [CrossRef] [Medline]
    11. Newton DC, Tomyn AJ, LaMontagne AD. Exploring the challenges and opportunities for improving the health and wellbeing of international students: perspectives of international students. J Australian New Zealand Student Services Assoc 2021 May 24;29(1):18-34. [CrossRef]
    12. International students and their mental health and physical safety. Orygen. 2020.   URL: https:/​/www.​orygen.org.au/​Orygen-Institute/​Policy-Areas/​Employment-and-education/​Education/​International-students-and-their-mental-health-and/​International-Student-Mental-Health-and-Physical-S?ext= [accessed 2022-03-17]
    13. Lian Z, Wallace BC, Fullilove RE. Mental health help-seeking intentions among Chinese international students in the U.S. higher education system: the role of coping self-efficacy, social support, and stigma for seeking psychological help. Asian Am J Psychol 2020 Sep;11(3):147-157. [CrossRef]
    14. Gulliver A, Griffiths KM, Christensen H, Brewer JL. A systematic review of help-seeking interventions for depression, anxiety and general psychological distress. BMC Psychiatry 2012 Jul 16;12(1):81. [CrossRef]
    15. Xu Z, Huang F, Kösters M, Staiger T, Becker T, Thornicroft G, et al. Effectiveness of interventions to promote help-seeking for mental health problems: systematic review and meta-analysis. Psychol Med 2018 Dec;48(16):2658-2667. [CrossRef] [Medline]
    16. Evans-Lacko S, Hahn J, Peter L, Schomerus G. The impact of digital interventions on help-seeking behaviour for mental health problems: a systematic literature review. Curr Opin Psychiatry 2022 May 01;35(3):207-218. [CrossRef] [Medline]
    17. Clough BA, Nazareth SM, Casey LM. Making the grade: a pilot investigation of an e-intervention to increase mental health literacy and help-seeking intentions among international university students. Brit J Guidance Counselling 2019 Oct 03;48(3):347-359. [CrossRef]
    18. Ebert DD, Franke M, Kählke F, Küchler AM, Bruffaerts R, Mortier P, WHO World Mental Health - International College Student collaborators. Increasing intentions to use mental health services among university students. Results of a pilot randomized controlled trial within the World Health Organization's World Mental Health International College Student Initiative. Int J Methods Psychiatr Res 2019 Jun 20;28(2):e1754 [FREE Full text] [CrossRef] [Medline]
    19. Han J, Batterham P, Calear A, Wu Y, Xue J, van Spijker BA. Development and pilot evaluation of an online psychoeducational program for suicide prevention among university students: a randomised controlled trial. Internet Interv 2018 Jun;12:111-120 [FREE Full text] [CrossRef] [Medline]
    20. King CA, Eisenberg D, Zheng K, Czyz E, Kramer A, Horwitz A, et al. Online suicide risk screening and intervention with college students: a pilot randomized controlled trial. J Consult Clin Psychol 2015 Jun;83(3):630-636 [FREE Full text] [CrossRef] [Medline]
    21. Taylor-Rodgers E, Batterham P. Evaluation of an online psychoeducation intervention to promote mental health help seeking attitudes and intentions among young adults: randomised controlled trial. J Affect Disord 2014 Oct;168:65-71 [FREE Full text] [CrossRef] [Medline]
    22. Tuliao A, Mullet N, Hawkins L, Holyoak D, Weerts M, Gudenrath T. Examining the role of a brief online alcohol use risk feedback on accessing information about available treatment resources for alcohol issues. Addict Behav 2019 Sep;96:164-170 [FREE Full text] [CrossRef] [Medline]
    23. Harper Shehadeh M, Heim E, Chowdhary N, Maercker A, Albanese E. Cultural adaptation of minimally guided interventions for common mental disorders: a systematic review and meta-analysis. JMIR Ment Health 2016 Sep 26;3(3):e44 [FREE Full text] [CrossRef] [Medline]
    24. Rickwood D, Deane FP, Wilson CJ, Ciarrochi J. Young people's help-seeking for mental health problems. Aust e JAdvancement Mental Health 2005 Dec;4(3):218-251. [CrossRef]
    25. Bakker D, Kazantzis N, Rickwood D, Rickard N. Mental health smartphone apps: review and evidence-based recommendations for future developments. JMIR Ment Health 2016 Mar 01;3(1):e7 [FREE Full text] [CrossRef] [Medline]
    26. Peters D, Calvo RA, Ryan RM. Designing for motivation, engagement and wellbeing in digital experience. Front Psychol 2018;9:797 [FREE Full text] [CrossRef] [Medline]
    27. Kessler RC, Andrews G, Colpe LJ, Hiripi E, Mroczek DK, Normand SL, et al. Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychol Med 2002 Aug 26;32(6):959-976. [CrossRef] [Medline]
    28. Andrews G, Slade T. Interpreting scores on the Kessler Psychological Distress Scale (K10). Aust N Z J Public Health 2001 Dec;25(6):494-497 [FREE Full text] [CrossRef] [Medline]
    29. Zhou C, Chu J, Wang T. Reliability and validity of 10-item Kessler scale (K10) Chinese version in evaluation of mental health status of Chinese population. Chinese J Clin Psychol 2008;16:627-629 [FREE Full text]
    30. 4817.0.55.001 - information paper: use of the Kessler psychological distress scale in ABS health surveys, Australia, 2007-08. Australian Bureau of Statistics. 2012 Apr 4.   URL: https://www.abs.gov.au/ausstats/abs@.nsf/mf/4817.0.55.001 [accessed 2020-03-10]
    31. Bastien C, Vallières A, Morin C. Validation of the Insomnia Severity Index as an outcome measure for insomnia research. Sleep Med 2001 Jul;2(4):297-307 [FREE Full text] [CrossRef]
    32. Chung K, Kan K, Yeung W. Assessing insomnia in adolescents: comparison of Insomnia Severity Index, Athens Insomnia Scale and Sleep Quality Index. Sleep Med 2011 May;12(5):463-470 [FREE Full text] [CrossRef] [Medline]
    33. Cohen S, Kamarck T, Mermelstein R. A global measure of Perceived Stress. J Health Social Behav 1983 Dec;24(4):385-396. [CrossRef]
    34. Lu W, Bian Q, Wang W, Wu X, Wang Z, Zhao M. Chinese version of the Perceived Stress Scale-10: a psychometric study in Chinese university students. PLoS One 2017;12(12):e0189543 [FREE Full text] [CrossRef] [Medline]
    35. Bush K, Kivlahan DR, McDonell MB, Fihn SD, Bradley KA. The AUDIT alcohol consumption questions (AUDIT-C): an effective brief screening test for problem drinking. Arch Intern Med 1998 Sep 14;158(16):1789-1795. [CrossRef] [Medline]
    36. Campbell CE, Maisto SA. Validity of the AUDIT-C screen for at-risk drinking among students utilizing university primary care. J Am Coll Health 2018 May 22;66(8):774-782 [FREE Full text] [CrossRef] [Medline]
    37. Li Q, Babor TF, Hao W, Chen X. The Chinese translations of Alcohol Use Disorders Identification Test (AUDIT) in China: a systematic review. Alcohol Alcohol 2011;46(4):416-423 [FREE Full text] [CrossRef] [Medline]
    38. Heatherton TF, Kozlowski LT, Frecker RC, Fagerström KO. The Fagerström Test for Nicotine Dependence: a revision of the Fagerström Tolerance Questionnaire. Br J Addict 1991 Sep;86(9):1119-1127. [CrossRef] [Medline]
    39. Huang C, Lin H, Wang H. The psychometric properties of the Chinese version of the Fagerstrom Test for Nicotine Dependence. Addict Behav 2006 Dec;31(12):2324-2327 [FREE Full text] [CrossRef] [Medline]
    40. Zimet GD, Dahlem NW, Zimet SG, Farley GK. The multidimensional scale of perceived social support. J Personality Assessment 1988 Mar;52(1):30-41. [CrossRef]
    41. Zimet G, Powell S, Farley G, Werkman S, Berkoff K. Psychometric characteristics of the Multidimensional Scale of Perceived Social Support. J Pers Assess 1990;55(3-4):610-617. [CrossRef] [Medline]
    42. Guan NC, Seng LH, Hway Ann AY, Hui KO. Factorial validity and reliability of the Malaysian simplified Chinese version of Multidimensional Scale of Perceived Social Support (MSPSS-SCV) among a group of university students. Asia Pac J Public Health 2015 Mar 27;27(2):225-231. [CrossRef] [Medline]
    43. Proudfoot J, Clarke J, Birch M, Whitton AE, Parker G, Manicavasagar V, et al. Impact of a mobile phone and web program on symptom and functional outcomes for people with mild-to-moderate depression, anxiety and stress: a randomised controlled trial. BMC Psychiatry 2013 Nov 18;13:312 [FREE Full text] [CrossRef] [Medline]
    44. Mahoney A, Elders A, Li I, David C, Haskelberg H, Guiney H, et al. A tale of two countries: increased uptake of digital mental health services during the COVID-19 pandemic in Australia and New Zealand. Internet Interv 2021 Sep;25:100439 [FREE Full text] [CrossRef] [Medline]
    45. Fischer E, Farina A. Attitudes toward seeking professional psychological help: a shortened form and considerations for research. J Coll Stud Dev 1995;36:368-374.
    46. Elhai J, Schweinle W, Anderson S. Reliability and validity of the attitudes toward seeking professional psychological help scale-short form. Psychiatry Res 2008 Jun 30;159(3):320-329 [FREE Full text] [CrossRef] [Medline]
    47. Fang K, Pieterse AL, Friedlander M, Cao J. Assessing the psychometric properties of the attitudes toward seeking professional psychological help scale-short form in Mainland China. Int J Adv Counselling 2011 Oct 1;33(4):309-321. [CrossRef]
    48. Wilson C, Deane F, Ciarrochi J, Rickwood D. Measuring help seeking intentions: properties of the general help seeking questionnaire. Canadian J Counsel 2005;39(1):15-28 [FREE Full text] [CrossRef]
    49. Han J, Batterham P, Calear A, Ma J. Seeking professional help for suicidal ideation: a comparison between Chinese and Australian university students. Psychiatry Res 2018 Dec;270:807-814 [FREE Full text] [CrossRef] [Medline]
    50. Parks AC, Williams AL, Tugade MM, Hokes KE, Honomichl RD, Zilca RD. Testing a scalable web and smartphone based intervention to improve depression, anxiety, and resilience: a randomized controlled trial. Int J Wellbeing 2018 Dec 08;8(2):22-67. [CrossRef]
    51. Peek J, Hay K, Hughes P, Kostellar A, Kumar S, Bhikoo Z, et al. Feasibility and acceptability of a smoking cessation smartphone app (My QuitBuddy) in older persons: pilot randomized controlled trial. JMIR Form Res 2021 Apr 14;5(4):e24976 [FREE Full text] [CrossRef] [Medline]
    52. Fleming T, Bavin L, Lucassen M, Stasiak K, Hopkins S, Merry S. Beyond the trial: systematic review of real-world uptake and engagement with digital self-help interventions for depression, low mood, or anxiety. J Med Internet Res 2018 Jun 06;20(6):e199 [FREE Full text] [CrossRef] [Medline]
    53. Lattie EG, Adkins EC, Winquist N, Stiles-Shields C, Wafford QE, Graham AK. Digital mental health interventions for depression, anxiety, and enhancement of psychological well-being among college students: systematic review. J Med Internet Res 2019 Jul 22;21(7):e12869 [FREE Full text] [CrossRef] [Medline]
    54. Bedford L, Dietch J, Taylor D, Boals A, Zayfert C. Computer-guided problem-solving treatment for depression, PTSD, and insomnia symptoms in student veterans: a pilot randomized controlled trial. Behav Ther 2018 Sep;49(5):756-767 [FREE Full text] [CrossRef] [Medline]
    55. Haas A, Koestner B, Rosenberg J, Moore D, Garlow SJ, Sedway J, et al. An interactive web-based method of outreach to college students at risk for suicide. J Am College Health 2008 Jul;57(1):15-22. [CrossRef]
    56. Horgan A, McCarthy G, Sweeney J. An evaluation of an online peer support forum for university students with depressive symptoms. Arch Psychiatr Nurs 2013 Apr;27(2):84-89 [FREE Full text] [CrossRef] [Medline]
    57. Lee RA, Jung ME. Evaluation of an mHealth app (DeStressify) on university students' mental health: pilot trial. JMIR Ment Health 2018 Jan 23;5(1):e2 [FREE Full text] [CrossRef] [Medline]
    58. Pascoe M, Dash S, Klepac Pogrmilovic B, Patten R, Parker A. The engagement of tertiary students with an online mental health intervention during the coronavirus disease 2019 pandemic: a feasibility study. Digit Health 2022;8:20552076221117746 [FREE Full text] [CrossRef] [Medline]
    59. Wasil AR, Taylor ME, Franzen RE, Steinberg JS, DeRubeis RJ. Promoting graduate student mental health during COVID-19: acceptability, feasibility, and perceived utility of an online single-session intervention. Front Psychol 2021 Apr 7;12:569785 [FREE Full text] [CrossRef] [Medline]
    60. Melnyk B, Amaya M, Szalacha L, Hoying J, Taylor T, Bowersox K. Feasibility, acceptability, and preliminary effects of the COPE online cognitive-behavioral skill-building program on mental health outcomes and academic performance in freshmen college students: a randomized controlled pilot study. J Child Adolesc Psychiatr Nurs 2015 Aug;28(3):147-154 [FREE Full text] [CrossRef] [Medline]
    61. Kaczmarek L, Kashdan T, Kleiman E, Baczkowski B, Enko J, Siebers A, et al. Who self-initiates gratitude interventions in daily life? An examination of intentions, curiosity, depressive symptoms, and life satisfaction. Personality Individual Differences 2013 Oct;55(7):805-810 [FREE Full text] [CrossRef]
    62. Eustis E, Hayes-Skelton S, Orsillo S, Roemer L. Surviving and thriving during stress: a randomized clinical trial comparing a brief web-based therapist-assisted acceptance-based behavioral intervention versus waitlist control for college students. Behav Ther 2018 Nov;49(6):889-903 [FREE Full text] [CrossRef] [Medline]
    63. Lintvedt OK, Griffiths KM, Sørensen K, Østvik AR, Wang CE, Eisemann M, et al. Evaluating the effectiveness and efficacy of unguided internet-based self-help intervention for the prevention of depression: a randomized controlled trial. Clin Psychol Psychother 2013 Sep 02;20(1):10-27. [CrossRef] [Medline]
    64. Hintz S, Frazier PA, Meredith L. Evaluating an online stress management intervention for college students. J Couns Psychol 2015 Apr;62(2):137-147. [CrossRef] [Medline]
    65. Levin ME, Pistorello J, Seeley JR, Hayes SC. Feasibility of a prototype web-based acceptance and commitment therapy prevention program for college students. J Am Coll Health 2014;62(1):20-30 [FREE Full text] [CrossRef] [Medline]
    66. Choi I, Sharpe L, Li S, Hunt C. Acceptability of psychological treatment to Chinese- and Caucasian-Australians: internet treatment reduces barriers but face-to-face care is preferred. Soc Psychiatry Psychiatr Epidemiol 2015 Jan;50(1):77-87. [CrossRef] [Medline]

    ‎
    ATSPPH-SF: Attitudes Toward Seeking Professional Psychological Help Scale–Short Form
    AUDIT-C: Alcohol Use Disorders Identification Test–Concise
    GHSQ: General Help-Seeking Questionnaire
    ISI: Insomnia Severity Index
    K10: Kessler Psychological Distress Scale
    MVP: minimum viable product
    PSS: Perceived Stress Scale

    Edited by A Mavragani; submitted 31.01.22; peer-reviewed by E Ennis, Y Amanvermez, AM Duncan; comments to author 02.09.22; revised version received 03.12.22; accepted 19.12.22; published 17.02.23

    Copyright

    ©Isabella Choi, Gemma Mestroni, Caroline Hunt, Nick Glozier. Originally published in JMIR Formative Research (https://formative.jmir.org), 17.02.2023.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.