Published on in Vol 5, No 7 (2021): July

Preprints (earlier versions) of this paper are available at, first published .
User Engagement and Usability of Suicide Prevention Apps: Systematic Search in App Stores and Content Analysis

User Engagement and Usability of Suicide Prevention Apps: Systematic Search in App Stores and Content Analysis

User Engagement and Usability of Suicide Prevention Apps: Systematic Search in App Stores and Content Analysis

Original Paper

1Department of Psychological Sciences, University of Missouri-St. Louis, St Louis, MO, United States

2Minneapolis Veterans Affairs Health Care System, Minneapolis, MN, United States

3School of Theology, Boston University, Boston, MA, United States

4Department of Psychology, Hofstra University, Hempstead, NY, United States

5Rocky Mountain Regional VA Medical Center, Denver, CO, United States

6University of Colorado Anschutz Medical Campus, Denver, CO, United States

7Department of Psychology, Florida State University, Tallahassee, FL, United States

8Department of Health Care Policy, Harvard Medical School, Boston, MA, United States

9Department of Psychology, Harvard University, Cambridge, MA, United States

Corresponding Author:

Chelsey R Wilks, PhD

Department of Psychological Sciences

University of Missouri-St. Louis

1 University Way

St Louis, MO, 63121

United States

Phone: 1 (314) 516 5383


Background: People with suicidal thoughts are more inclined to seek technology-delivered interventions than in-person forms of treatment, making mobile apps for suicide prevention an ideal platform for treatment delivery. This review examines apps designed for suicide prevention, with a specific focus on user engagement.

Objective: This study aims to update the literature and broadly evaluate the landscape of mobile health apps for suicide prevention; examine apps with key features and primary approaches to suicide prevention; and systematically evaluate the engagement, functionality, aesthetics, and information of the apps.

Methods: All apps related to suicidal thoughts and behaviors were identified in the Google Play and iOS app stores and were systematically reviewed for their content and quality. The mobile app rating scale (MARS) was used to evaluate app usability and engagement.

Results: Of the 66 apps identified, 42 (64%) were specifically designed for people with suicidal ideation, and 59 (89%) had at least one best practice feature for suicide risk reduction. The mean overall MARS score of all apps was 3.5 (range 2.1-4.5), with 83% (55/66) of apps having a minimum acceptability score of 3. The total MARS score was not associated with the user app rating (r=−0.001; P=.99) or the number of features (r=0.24; P=.09).

Conclusions: This study identified many usable and engaging apps in app stores designed for suicide prevention. However, there are only limited apps for clinicians. Thus, mobile apps for suicide prevention should be carefully developed and clinically evaluated.

JMIR Form Res 2021;5(7):e27018




Suicide is the second leading cause of death in the United States among people between the ages of 10-34 years. Suicidal thoughts and behaviors are difficult to treat, and only a few treatments with evidence of efficacy are widely disseminated. Unfortunately, treatment engagement among suicidal patients is low, particularly among those experiencing frequent and intense suicidal ideation [1-3]. Fortunately, although some high-risk suicidal individuals avoid face-to-face intervention, they may be inclined to anonymously seek out help through technology [4-6].

One cost-effective and convenient avenue for mental health delivery is through mobile mental health apps (ie, mobile health [mHealth] apps). There has been an increase in the number of mHealth app targeted for mental health problems in general [7,8] and suicide in particular [9,10]. mHealth may be a novel strategy to target suicide among those in high-income countries, with over 95% of US adults reporting that they own a smartphone [11]. In addition, approximately 64% of adolescents reported using apps to manage their mental health symptoms [12].

The number of mental health–related apps available to users has increased dramatically, with recent estimates suggesting that more than 10,000 such apps have been created [13]. Unfortunately, only a few mHealth apps have demonstrated efficacy [14,15]. In addition, 74% of users reported that they stopped using mHealth apps after only 10 uses [16]. Thus, there is a significant deficit in studies investigating the efficacy and engagement levels of mHealth apps, as well as those particularly focusing on suicide prevention. For instance, Larsen et al [10] identified 49 apps specifically designed to prevent or reduce suicide and concluded that although many apps contained some elements of best practices, none of the apps provided evidence supporting their efficacy. Best practices for suicide prevention include strategies with consistent evidence for reducing suicide and have been outlined in detail in previous reviews [17]. For example, means safety, defined as the removal of lethal means, has consistently been identified as an effective suicide prevention intervention [18]. Other best practices include providing access to suicide hotlines, crisis planning, and social support [19,20]. De La Torre et al [9] performed a more comprehensive review and identified 20 apps related to suicide prevention and 6 published scholarly articles describing the features and clinical utility of mobile apps for suicide prevention. However, neither of these reviews critically evaluated the user experience of mHealth apps related to suicide prevention. In this context, user experience comprises usability and engagement. Usability refers to how simple and intuitive it is to access computing technology [21], whereas engagement refers to the degree to which user interest is maintained when interacting with computing technology [22]. User engagement can be evaluated using objective metrics (eg, downloads, popularity, and dwell time) and expert ratings [23].


As mHealth apps have the potential to monitor and mitigate suicidal crises, it is important to assess the features and quality of smartphone apps currently available. Apps that can engage users toward more effective coping behavior in lieu of suicidal acts could have a sweeping public health impact but only if the user is prompted to open the app during critical times. Therefore, the objectives of this study were to (1) update the literature and broadly evaluate the landscape of mHealth apps for suicide prevention, (2) examine the key features and primary approaches to suicide prevention of these apps, and (3) systematically evaluate the engagement, functionality, aesthetics, and information of the apps. The systematic evaluation of usability and engagement was difficult until the development of mobile app rating scale (MARS), a tool for classifying and rating the quality of mHealth apps [24]. This review aims to help users make more informed decisions by assessing the features, usability, and engagement of apps designed to prevent suicide.

App Selection

Apps were initially identified in October 2018 and rereviewed in October 2020 through a systematic search of the US iTunes and Google Play stores. Search terms included suicide, suicidal ideation, suicide ideation, thoughts about suicide, suicidal thinking, ideation, thinking about suicide, self-harm, self-injury, nonsuicidal self-injury, and NSSI. Apps were included if they (1) were smartphone based, (2) used Android or iOS operating systems, (3) were in the English language, (4) had one or more of the aforementioned search terms in the app description, and (5) were available for download in the US app store (iTunes or Google Play). Apps were excluded if they (1) did not primarily target suicidal thoughts, behaviors, or self-injury; required payment for download; or were no longer available or accessible for download (Figure 1). iPhone apps were downloaded and tested using an iPhone 7 and an iPhone 11 in iOS 11, whereas Android apps were downloaded and tested using a simple mobile phone and One Plus 7 Pro Android in version Oreo 8.1 and Oxygen OS 10.3.2, respectively.

Figure 1. Systematic app selection.
View this figure

Data Extraction

The following data about all apps were recorded: app name, platform (ie, Android or iOS), current version, cost, number of installs (Android only), and user ratings (1-5 stars). The intended best practice prevention strategy of each app is noted using the relevant portion of the coding scheme by Larsen et al [10], including means safety (ie, reducing access to lethal means of suicide), support (ie, providing access to social support networks, such as through a message board), access to crisis support or helpline, psychotherapies (cognitive behavior therapy or dialectical behavior therapy), and safety planning. Discrepancies were resolved by discussion among the authors until a consensus was reached. As suicide prevention can encompass numerous strategies from support to immediate crisis intervention, we identified four approaches that each app used as its main prevention strategy: providing psychoeducation, teaching coping skills, documenting a crisis plan, and providing social support. Apps that reportedly targeted suicide risk but did not fall into those four categories were classified as other.

App Quality

All apps were rated by 2 independent reviewers using the MARS. The 23 items in the MARS were identified from a review of existing criteria for rating app quality. Each item was rated on a 5-point scale (ie, 1=inadequate; 2=poor; 3=acceptable; 4=good; and 5=excellent) with descriptors provided for each anchor rating. MARS grouped the items into four categories, namely engagement (5 items), functionality (4 items), aesthetics (3 items), and information quality (7 items), and a subjective quality scale (eg, worth recommending and overall satisfaction; 4 items). The dimension of subjective quality in MARS was excluded from the analysis to ensure objectivity and consistency of the assessment process. Previous studies using MARS have also excluded the subjective quality dimension for this reason [24]. The MARS was scored using a mean for each category and an overall mean score. The MARS demonstrated good internal consistency (α=.90) and interrater reliability (intraclass coefficient=0.79) in previous research [24].

Before the app assessment, the 4 reviewers (CRW, CC, DS, JL) discussed the use of the MARS for apps intended for people with suicidal thoughts and behaviors. Evaluating the quality and user experience of mobile apps can be unreliable [25]. The first author (CRW) has extensive research experience in developing mobile apps, conducting user research with mobile and web-based applications, and human-computer interaction. We based the target audience on the following: patients or consumers, clinicians, teens, and family or friends. As recommended by the developers of the MARS, the reviewers considered all items of the MARS and confirmed that all were applicable to apps for suicide prevention and that no additional app-specific items were required.

After a consensus was reached with regard to MARS, the reviewers independently rated the included apps. Each reviewer interacted with the identified app for several minutes, ensuring that all aspects of functionality were tested and evaluated. When reviewers had questions or concerns related to the apps, these issues were discussed among the authors and a consensus was reached.

Statistical Analyses

Scores were calculated for each MARS item, along with the total mean score. The interrater reliability of the MARS subscales and total quality score was calculated using the intraclass correlation coefficient two-way random-effects model of absolute agreement between single ratings. The mean value for each dimension of MARS was calculated. The difference in app quality between affiliations was analyzed using analysis of variance to examine the moderating effect of developers. Spearman correlations among the four dimensions of MARS, the number of downloads, and average rating were also analyzed. All statistical analyses were performed using SPSS, version 24 (IBM Corporation).

Search Results

A total of 1593 apps (iTunes Apple store, n=245; Google Play Store, n=1348) were initially screened or manually identified. In the screening stage, 1353 apps were excluded as they were either duplicates, not written in the English language, or irrelevant (eg, games, wallpapers, and quotes). Of the 240 apps that were downloaded and tested for eligibility, 174 (72.5%) were excluded as they were no longer available or accessible for download, unrelated to suicide, or required payment for download. The remaining 66 apps were included in the quality assessment and usability evaluation (Figure 1).

Descriptive Characteristics

The characteristics and the mean MARS scores of the 66 included apps are presented in Multimedia Appendix 1. The average user rating of the apps was 3.5 (range 1-5). Although all included apps were free, 2 offered paid upgraded versions at a cost between US $0.99 and US $149.99 for in-app purchases. A total of 35 apps were found in both iOS and Android app stores, whereas 19 were iOS only and 12 were Android only. More than half (37/66, 56%) of the apps included a privacy policy.

The five features considered to be best practices for suicide prevention were examined for each app and are presented in Table 1. None of the apps had all five features, and only 4 apps had four of the five best practices: Prevent Suicide: Dumfries & Galloway, ReMinder Suicide Safety Plan, SafetyNet: Your Suicide Prevention App, and Don’t Panic–depression and panic help. The average number of features across apps was 1.7. None of the features was found in the following 7 apps: SafeUT, R U Suicidal?, Self Harm Recovery, A Teen Suicide Prevention Anime, Seeking the Military Suicide Solution, Elijah, and Suicide Prevention-Ways to Help a Suicidal Friend. The most common of the five features included were access to a crisis line (37/66, 56%), support (33/66, 50%), and a safety plan (22/66, 33%), whereas means safety was the least integrated feature (8/66, 12% of apps). The apps were most often designed for persons experiencing suicidal thoughts (49/66, 74%), followed by friends and family (11/66, 16%), teens (6/66, 9%), and clinicians (1/66, 2%; Multimedia Appendix 1). On the basis of the Android apps alone, the apps with the most downloads were Calm Harm, Talk Life, and Mood Tools with over 100,000 downloads.

Table 1. Best practice features for suicide prevention of included apps (N=66).
App nameMeans safetySupportCrisis line accessTreatmentSafety plan
Calm in the Storm: Stress Management


MY3 Support Network

TalkLife for Stress & Anxiety


First Step OR

MoodTools-Depression Aid

HOPE-Broome County Mental Health


Jason Foundation A Friend Asks

Suicide Safety Plan

Safe Students

Got your back

Stanley-Brown Safety

Operation Reach Out


Just in Case for Colleges

Relief Link

Ulster County Speak


be safe suicide safety plan

Every Teen Seen


Be Safe

Calm Harm-manages self harm

R U Suicidal?

Say Something

Anemone Crisis App

Prevent Suicide-Highland

There is Hope


Self Harm Recovery

Suicide? Help? Tayside

Stay Alive

Dutchess County HELPLINE

The LifeLine

DMHSb: Suicide Prevention Info

Is S/O Suicidal?

Did someone you know suicide?

Step Up and Speak Out

Kokua Life


Calm Care

My Shiny Thing

PMCS Combating Suicide

SeeSave/See Something Save Someone

Community Stress First Aid

iHelp Sunshine Coast

MS DMH-Shatter in the Silence

Better Stop Suicide

SCNGc Suicide Prevention

Alaska Careline

Prevent Suicide: Dumfries & Galloway

MYPLAN-your safety plan

ReMinder Suicide Safety Plan
SafetyNet: Your Suicide Prevention App

UnCut App

Don’t Panic–depression and panic help
Emotional Support Helpline Directory

Yellow Ribbon Foundation

A Teen Suicide Prevention Anime

Seeking the Military Suicide Solution


Suicide Prevention-Ways to Help a Suicidal Friend

aFeature present.

bDMHS: Durham Mental Health Services.

cSCNG: South Carolina National Guard.

Usability and App Quality

The mean overall MARS score of all apps was 3.5 (range 2.1-4.5), and 83% (55/66) of apps had a minimum acceptability score of 3.0 (Table 2). In general, the Calm Harm: Manages self harm app had the highest MARS overall score (4.5), followed by INSIST (4.49), MY3 Support Network (4.4), and Talklife for Stress and Anxiety (4.4). Apps with the highest MARS scores and at least three of five best practice features for suicide prevention were Got your back, Stanley-Brown Safety, and Operation Reach Out.

Data on the MARS subscale scores and the overall MARS scores categorized according to the main app approaches are presented in Table 2. The interrater reliability was within the acceptable range for each subscale (0.73-0.94), with the highest interrater reliability for information (0.94) and the lowest for function (0.73). Apps with the primary function of providing users with support had the highest overall MARS (mean 3.7, SD 0.8). Coping skills apps scored the highest in engagement (3.2) and function (4.0). In the aesthetics and information subscales, support apps (3.8 and 4.2, respectively) had the highest scores.

Table 2. Mobile app rating scores according to main approach (N=66).
Main approachCount, n (%)Engagement, mean (SD)Functionality, mean (SD)Aesthetics, mean (SD)Information, mean (SD)Overall, mean (SD)
Crisis plan12 (18)3.13 (0.77)3.96 (0.59)3.38 (0.70)3.64 (1.16)3.53 (0.76)
Support24 (36)2.91 (0.76)3.86 (0.71)3.82 (0.72)4.23 (0.53)3.70 (0.83)
Psychoeducation17 (26)2.56 (0.65)3.80 (0.54)3.34 (0.66)3.70 (1.00)3.35 (0.87)
Coping skill10 (15)3.24 (0.64)4.00 (0.64)3.43 (0.83)3.69 (0.87)3.59 (0.78)
Other3 (5)2.40 (0.35)3.4 (0.88)2.50 (0.50)3.66 (1.44)2.99 (0.96)

Correlation Analysis

The total and subscale MARS scores were all significantly correlated, indicating that app quality was consistent across all areas assessed (eg, apps scoring high on engagement also tended to score high on function, aesthetics, and information). The overall MARS score was neither correlated with user app rating (r=−0.001; P=.99) nor with the number of features included within the app (P=.09; Table 3).

Table 3. Correlations among total mobile app rating scale (MARS) score, four MARS dimension scores, rating, and number of features.
CharacteristicsMARSaRatingNumber of features





Correlation factor0.721.00

P value<.001


Correlation factor0.800.401.00

P value<.001<.001


Correlation factor0.830.600.501.00

P value<.001<.001<.001


Correlation factor0.820.350.630.561.00

P value<.001.002<.001<.001


Number of features0.20−

aMARS: mobile app rating scale.

bNot applicable.


Principal Findings

We examined the user experience, usability, and engagement of mHealth apps designed for suicide prevention. There are three main findings of this study. First, although the majority apps included elements of best practices to reduce suicide risk, none included all these features. Second, most of the reviewed apps were designed for suicidal individuals, rather than for clinicians, friends, and families. Third, the MARS score of the majority of apps was in the acceptable range, with the apps designed for support receiving the highest rating; however, the star ratings of users were not correlated with MARS scores, suggesting that star ratings may be indicative of another construct rather than app quality.

Since the app review by Larsen in 2016 [10], nearly twice as many apps designed to reduce suicide have been introduced in app stores. The majority of the apps (59/66, 89%) reviewed in this study have at least one best practice element for suicide prevention. The most common feature across the apps was the inclusion of a crisis line access (37/66, 56%); however, only 12% (8/66) of apps included a means safety feature. Mean safety is considered one of the most potent suicide prevention strategies [26]; thus, it is surprising that it was the least integrated feature. However, integrating content into mobile devices requires intentional design considerations, such as interaction or navigation, which can influence how users learn and engage with the material [27]. In addition, including content on means safety may have been difficult to design and implement. We found no association among the number of best practice elements, MARS scores, or app ratings, indicating that app quality is not solely driven by content, but rather how the content functions and is designed. Notably, an app that implements one aspect well, such as developing a safety plan, may be a better app than one that tries to integrate several features. The apps that yielded the highest MARS scores had a narrow scope. However, this review did not clinically evaluate or evaluate the available research on any of these apps; therefore, it is unclear whether these apps are effective at reducing suicidal crises.

The majority of the apps (49/66, 74%) were specifically designed for suicidal individuals, only 15% (10/66) of which were designed for friends or family and 2% (1/66) for clinicians. This highlights a potential deficit in apps that are designed to treat, manage, or cope with individuals at risk of suicide. A major obstacle in overall suicide prevention is the lack of willingness in treating suicidal individuals among mental health providers [28]. In a previous study, a technology-delivered suicide risk assessment and management tool was associated with reduced fear and increased self-efficacy among clinicians treating suicidal individuals [29]. Given that suicidal people tend to avoid face-to-face treatment [1-3], one avenue to potentially reduce suicide risk could be through a suicide prevention app for friends and family. Although these apps exist in app stores, it is unclear whether they are effective or widely used. Overall, more research is needed to develop and evaluate suicide prevention apps for individuals who work or live with suicidal patients.

Most mobile apps in this review were at least moderately usable and engaging. Although 17% (11/66) of apps yielded unacceptable scores, the average MARS scores were in the acceptable range, indicating that the apps were generally usable and engaging. However, it is unclear whether the apps designed to reduce suicide are reaching the appropriate audience or designed according to what suicidal users need or want. The user context or environment may be a significant driver of the determination of engagement. A suicide app designed to help clinicians assess and manage suicide will likely need to be highly functional, but not necessarily fun to use. In contrast, in apps designed to help users reduce suicidal crises, ongoing app engagement may not be a goal, as app developers hope that suicidal crises will eventually be reduced. This makes the iterative design of suicide-related apps challenging because repeated use may not be an ideal outcome. Traditionally, app developers can use objective measures, such as time spent on app and daily uses, as outcomes to fine-tune and optimize content delivery; however, these metrics may indicate different factors. An engaging suicide app may be one that is immediately accessible to users during key moments, and previous research on user engagement with mHealth apps indicates that immediate access to resources is an integral aspect that can keep users engaged [30]. As 73% of users tend to stop using a mental health app after 10 uses [16], it is important to understand what factors are associated with discontinued use, such as poor user experience versus no longer in crisis. In addition, it may not be profitable to develop a tool to reduce suicide, which may partially explain the relatively large number of nonusable apps designed for suicide. Because developing, publishing, and maintaining mobile apps is time-consuming and expensive, a different funding structure may be required to produce high-quality mobile apps for high-risk users.

In general, there is a lack of research on consumer apps for suicide prevention. Melia et al [31] identified only 7 mobile mental health apps with published outcomes in randomized clinical trials. Although there are apps available to consumers with research support, such as Tec Tec [32] and the Virtual Hope Box [33], these apps were not identified during our search procedure, highlighting a deficit in search term strategy when these apps were released to the public. We were only able to find these apps when we searched for them by name but not when we used our search terms. The function of this review is user-centered, as is the case when users search for apps for suicide or self-harm. Researchers who design and develop apps may benefit from the increased marketing of their apps. In addition, users may not be willing to scroll past the first page of results to search for an app, highlighting the importance of the search term strategy and how apps are weighted in the app store search algorithm. Given that star ratings may not accurately reflect app quality, app store algorithms may benefit from another strategy to move quality apps up the list.


Although this paper is the first review to specifically examine the user experience of mobile apps specifically designed for suicide, there are some limitations to the study worth discussing. First, as we only searched on app stores and systematic literature search was not performed, web-based apps that are not featured in app stores were, therefore, not included. Owing to the ease of development and maintenance of web-based apps compared with native apps [25], it is important to include these apps in future research. Second, we only reviewed the user statistics at one time point, and variables such as the number of downloads and user ratings vary over time. Third, as noted previously, some notable and research-supported apps were missing from this study. We believe this illustrates the lab to marketplace gap, which is prevalent in all aspects of mental health research but especially in research on dissemination and implementation [34]. Finally, we opted not to evaluate apps that required payment upfront, which limited the scope of our review.


Although this study identified many usable and engaging apps designed for suicide prevention in app stores, there are several opportunities for mobile app development and enhancement. In particular, there is a lack of apps designed to assist clinicians in treating suicidal patients. In addition, there is a need for more clinical evaluation of suicide prevention apps found in app stores. In general, mobile apps for suicide prevention should be carefully developed and clinically evaluated.

Conflicts of Interest

CRW receives consulting fees from Mindstrong Health, Click Therapeutics, and Behavioral Tech. In the past 3 years, RCK was a consultant for Datastat Inc, Holmusk, RallyPoint Networks Inc, Sage Pharmaceuticals, and Takeda. He has stock options in Mirah, PYM, and Roga Sciences. MKN receives text book royalties from Macmillan and Pearson publishers and has been a paid consultant in the past year for Microsoft and for a legal case regarding a death by suicide. He is an unpaid scientific advisor for TalkLife and Empatica. All other authors deny conflicts of interest.

Multimedia Appendix 1

Characteristics and mean mobile app rating scale scores of mobile apps (N=66).

DOCX File , 25 KB


  1. Carlton PA, Deane FP. Impact of attitudes and suicidal ideation on adolescents' intentions to seek professional psychological help. J Adolesc 2000 Mar;23(1):35-45. [CrossRef] [Medline]
  2. Pretorius C, Chambers D, Coyle D. Young people's online help-seeking and mental health difficulties: systematic narrative review. J Med Internet Res 2019 Nov 19;21(11):e13873 [FREE Full text] [CrossRef] [Medline]
  3. Yakunina ES, Rogers JR, Waehler CA, Werth JL. College students' intentions to seek help for suicidal ideation: accounting for the help-negation effect. Suicide Life Threat Behav 2010 Oct;40(5):438-450. [CrossRef] [Medline]
  4. Seward A, Harris KM. Offline versus online suicide-related help seeking: changing domains, changing paradigms. J Clin Psychol 2016 Jun;72(6):606-620. [CrossRef] [Medline]
  5. Wilks CR, Coyle TN, Krek M, Lungu A, Andriani K. Suicide ideation and acceptability toward online help-seeking. Suicide Life Threat Behav 2018 Aug;48(4):379-385. [CrossRef] [Medline]
  6. Wilks CR, Yee Ang S, Wang X, Arunagiri V, Ward-Ciesielski EF. Exploring preference to avoid or seek help in person and online among college students with suicidal ideation. J Soc Clin Psychol 2019 Dec;38(10):811-835. [CrossRef]
  7. Firth J, Torous J, Nicholas J, Carney R, Pratap A, Rosenbaum S, et al. The efficacy of smartphone-based mental health interventions for depressive symptoms: a meta-analysis of randomized controlled trials. World Psychiatry 2017 Oct;16(3):287-298 [FREE Full text] [CrossRef] [Medline]
  8. Firth J, Torous J, Nicholas J, Carney R, Rosenbaum S, Sarris J. Can smartphone mental health interventions reduce symptoms of anxiety? A meta-analysis of randomized controlled trials. J Affect Disord 2017 Aug 15;218:15-22 [FREE Full text] [CrossRef] [Medline]
  9. de la Torre I, Castillo G, Arambarri J, López-Coronado M, Franco MA. Mobile apps for suicide prevention: review of virtual stores and literature. JMIR Mhealth Uhealth 2017 Oct 10;5(10):e130 [FREE Full text] [CrossRef] [Medline]
  10. Larsen ME, Nicholas J, Christensen H. A systematic assessment of smartphone tools for suicide prevention. PLoS One 2016;11(4):e0152285 [FREE Full text] [CrossRef] [Medline]
  11. Anderson, Jiang J. Pew Research. 2018.   URL: [accessed 2020-10-20]
  12. Rideout V, Fox S. Digital Health Practices, Social Media Use,mental Well-being Among Teensyoung Adults in the US. Digital Commons. 2018.   URL: [accessed 2020-10-20]
  13. Torous J, Roberts LW. Needed innovation in digital health and smartphone applications for mental health: transparency and trust. JAMA Psychiatry 2017 May 1;74(5):437-438. [CrossRef] [Medline]
  14. Grist R, Porter J, Stallard P. Mental health mobile apps for preadolescents and adolescents: a systematic review. J Med Internet Res 2017 May 25;19(5):e176 [FREE Full text] [CrossRef] [Medline]
  15. Wang K, Varma DS, Prosperi M. A systematic review of the effectiveness of mobile apps for monitoring and management of mental health symptoms or disorders. J Psychiatr Res 2018 Dec;107:73-78. [CrossRef] [Medline]
  16. Lovett L. Mental Health Apps Helpful but Poor Design, Privacy Concerns Contribute to Low Engagement. Mobile Health News. 2018.   URL: https:/​/www.​​content/​mental-health-apps-helpful-poor-design-privacy-concerns-contribute-low-engagement [accessed 2020-10-20]
  17. Scott A, Guo B. For Which Strategies of Suicide Prevention Is There Evidence of Effectiveness. Den World Health Organ. 2012.   URL: [accessed 2020-10-20]
  18. Jin HM, Khazem LR, Anestis MD. Recent advances in means safety as a suicide prevention strategy. Curr Psychiatry Rep 2016 Oct;18(10):96. [CrossRef] [Medline]
  19. Bernert RA, Hom MA, Roberts LW. A review of multidisciplinary clinical practice guidelines in suicide prevention: toward an emerging standard in suicide risk assessment and management, training and practice. Acad Psychiatry 2014 Oct;38(5):585-592 [FREE Full text] [CrossRef] [Medline]
  20. van der Feltz-Cornelis CM, Sarchiapone M, Postuvan V, Volker D, Roskar S, Grum AT, et al. Best practice elements of multilevel suicide prevention strategies: a review of systematic reviews. Crisis 2011;32(6):319-333 [FREE Full text] [CrossRef] [Medline]
  21. Nielsen J. Usability 101: Introduction to Usability. Jakob Nielsen's Alertbox.: introduction to usability. Jakob Nielsen’s Alertbox. Recovered 25/03/2015 useit. com/alertbox/20030825. html; 2003.   URL: [accessed 2021-02-20]
  22. Peters C, Castellano G, De Freitas S. An Exploration of User Engagement in HCI. In: Proceedings of the Affective-aware Virtual Agents and Social Robots. 2009 Presented at: AFFINE'09; April 8-9, 2009; Boston, UK p. 1   URL: [CrossRef]
  23. Lehmann J, Lalmas M, Yom-Tov E, Dupret G. Models of user engagement. In: Masthoff J, Mobasher B, Desmarais MC, Nkambou R, editors. User Modeling, Adaptation, and Personalization. Berlin, Heidelberg: Springer; 2012:164.
  24. Stoyanov SR, Hides L, Kavanagh DJ, Zelenko O, Tjondronegoro D, Mani M. Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR Mhealth Uhealth 2015;3(1):e27 [FREE Full text] [CrossRef] [Medline]
  25. Powell AC, Torous J, Chan S, Raynor GS, Shwarts E, Shanahan M, et al. Interrater reliability of mhealth app rating measures: analysis of top depression and smoking cessation apps. JMIR Mhealth Uhealth 2016;4(1):e15 [FREE Full text] [CrossRef] [Medline]
  26. Milner A, Witt K, Maheen H, LaMontagne AD. Access to means of suicide, occupation and the risk of suicide: a national study over 12 years of coronial data. BMC Psychiatry 2017 Apr 4;17(1):125 [FREE Full text] [CrossRef] [Medline]
  27. Huang K. Challenges in Human-computer Interaction Design for Mobile Devices. In: ECS'09. 2009 Presented at: Proceedings of the World Congress on Engineering and Computer Science; October 24-26, 2009; San Francisco, USA p. 1   URL: https:/​/www.​​paper/​Challenges-in-Human-Computer-Interaction-Design-for-Huang/​abc6c941abe277b1b0c9ea3c5484f2487189a39e
  28. Langreth R, Ruiz R. The Forgotten Patients. Forbes. 2010.   URL: https:/​/www.​​forbes/​2010/​0913/​health-mental-health-industry-suicide-forgotten-patients.​html?sh=97772d4209b9 [accessed 2020-10-20]
  29. Harned MS, Lungu A, Wilks CR, Linehan MM. Evaluating a multimedia tool for suicide risk assessment and management: the Linehan suicide safety net. J Clin Psychol 2017 Mar;73(3):308-318. [CrossRef] [Medline]
  30. Wong HW, Lo B, Shi J, Hollenberg E, Abi-Jaoude A, Johnson A, et al. Postsecondary student engagement with a mental health app and online platform (thought spot): qualitative study of user experience. JMIR Ment Health 2021 Apr 2;8(4):e23447 [FREE Full text] [CrossRef] [Medline]
  31. Melia R, Francis K, Duggan J, Bogue J, O'Sullivan M, Chambers D, et al. Mobile health technology interventions for suicide prevention: protocol for a systematic review and meta-analysis. JMIR Res Protoc 2018 Jan 26;7(1):e28 [FREE Full text] [CrossRef] [Medline]
  32. Franklin JC, Fox KR, Franklin CR, Kleiman EM, Ribeiro JD, Jaroszewski AC, et al. A brief mobile app reduces nonsuicidal and suicidal self-injury: evidence from three randomized controlled trials. J Consult Clin Psychol 2016 Jun;84(6):544-557. [CrossRef] [Medline]
  33. Bush NE, Smolenski DJ, Denneson LM, Williams HB, Thomas EK, Dobscha SK. A virtual hope box: randomized controlled trial of a smartphone app for emotional regulation and coping with distress. Psychiatr Serv 2017 Apr 1;68(4):330-336. [CrossRef] [Medline]
  34. Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Adm Policy Ment Health 2009 Jan;36(1):24-34 [FREE Full text] [CrossRef] [Medline]

MARS: mobile app rating scale
mHealth: mobile health

Edited by G Eysenbach; submitted 07.01.21; peer-reviewed by C Smith, B Loo Gee; comments to author 18.02.21; revised version received 15.04.21; accepted 29.04.21; published 14.07.21


©Chelsey R Wilks, Carol Chu, DongGun Sim, Josh Lovell, Peter Gutierrez, Thomas Joiner, Ronald C Kessler, Matthew K Nock. Originally published in JMIR Formative Research (, 14.07.2021.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.