This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.
People with suicidal thoughts are more inclined to seek technology-delivered interventions than in-person forms of treatment, making mobile apps for suicide prevention an ideal platform for treatment delivery. This review examines apps designed for suicide prevention, with a specific focus on user engagement.
This study aims to update the literature and broadly evaluate the landscape of mobile health apps for suicide prevention; examine apps with key features and primary approaches to suicide prevention; and systematically evaluate the engagement, functionality, aesthetics, and information of the apps.
All apps related to suicidal thoughts and behaviors were identified in the Google Play and iOS app stores and were systematically reviewed for their content and quality. The mobile app rating scale (MARS) was used to evaluate app usability and engagement.
Of the 66 apps identified, 42 (64%) were specifically designed for people with suicidal ideation, and 59 (89%) had at least one best practice feature for suicide risk reduction. The mean overall MARS score of all apps was 3.5 (range 2.1-4.5), with 83% (55/66) of apps having a minimum acceptability score of 3. The total MARS score was not associated with the user app rating (
This study identified many usable and engaging apps in app stores designed for suicide prevention. However, there are only limited apps for clinicians. Thus, mobile apps for suicide prevention should be carefully developed and clinically evaluated.
Suicide is the second leading cause of death in the United States among people between the ages of 10-34 years. Suicidal thoughts and behaviors are difficult to treat, and only a few treatments with evidence of efficacy are widely disseminated. Unfortunately, treatment engagement among suicidal patients is low, particularly among those experiencing frequent and intense suicidal ideation [
One cost-effective and convenient avenue for mental health delivery is through mobile mental health apps (ie, mobile health [mHealth] apps). There has been an increase in the number of mHealth app targeted for mental health problems in general [
The number of mental health–related apps available to users has increased dramatically, with recent estimates suggesting that more than 10,000 such apps have been created [
As mHealth apps have the potential to monitor and mitigate suicidal crises, it is important to assess the features and quality of smartphone apps currently available. Apps that can engage users toward more effective coping behavior in lieu of suicidal acts could have a sweeping public health impact but only if the user is prompted to open the app during critical times. Therefore, the objectives of this study were to (1) update the literature and broadly evaluate the landscape of mHealth apps for suicide prevention, (2) examine the key features and primary approaches to suicide prevention of these apps, and (3) systematically evaluate the engagement, functionality, aesthetics, and information of the apps. The systematic evaluation of usability and engagement was difficult until the development of mobile app rating scale (MARS), a tool for classifying and rating the quality of mHealth apps [
Apps were initially identified in October 2018 and rereviewed in October 2020 through a systematic search of the US iTunes and Google Play stores. Search terms included
Systematic app selection.
The following data about all apps were recorded: app name, platform (ie, Android or iOS), current version, cost, number of installs (Android only), and user ratings (1-5 stars). The intended best practice prevention strategy of each app is noted using the relevant portion of the coding scheme by Larsen et al [
All apps were rated by 2 independent reviewers using the MARS. The 23 items in the MARS were identified from a review of existing criteria for rating app quality. Each item was rated on a 5-point scale (ie, 1=inadequate; 2=poor; 3=acceptable; 4=good; and 5=excellent) with descriptors provided for each anchor rating. MARS grouped the items into four categories, namely engagement (5 items), functionality (4 items), aesthetics (3 items), and information quality (7 items), and a subjective quality scale (eg, worth recommending and overall satisfaction; 4 items). The dimension of subjective quality in MARS was excluded from the analysis to ensure objectivity and consistency of the assessment process. Previous studies using MARS have also excluded the subjective quality dimension for this reason [
Before the app assessment, the 4 reviewers (CRW, CC, DS, JL) discussed the use of the MARS for apps intended for people with suicidal thoughts and behaviors. Evaluating the quality and user experience of mobile apps can be unreliable [
After a consensus was reached with regard to MARS, the reviewers independently rated the included apps. Each reviewer interacted with the identified app for several minutes, ensuring that all aspects of functionality were tested and evaluated. When reviewers had questions or concerns related to the apps, these issues were discussed among the authors and a consensus was reached.
Scores were calculated for each MARS item, along with the total mean score. The interrater reliability of the MARS subscales and total quality score was calculated using the intraclass correlation coefficient two-way random-effects model of absolute agreement between single ratings. The mean value for each dimension of MARS was calculated. The difference in app quality between affiliations was analyzed using analysis of variance to examine the moderating effect of developers. Spearman correlations among the four dimensions of MARS, the number of downloads, and average rating were also analyzed. All statistical analyses were performed using SPSS, version 24 (IBM Corporation).
A total of 1593 apps (iTunes Apple store, n
The characteristics and the mean MARS scores of the 66 included apps are presented in
The five features considered to be best practices for suicide prevention were examined for each app and are presented in
Best practice features for suicide prevention of included apps (N=66).
App name | Means safety | Support | Crisis line access | Treatment | Safety plan |
Calm in the Storm: Stress Management |
|
|
|
|
✓a |
INSIST |
|
✓ |
|
|
|
MY3 Support Network |
|
|
✓ |
|
✓ |
TalkLife for Stress & Anxiety |
|
✓ |
|
|
|
SafeUT |
|
|
|
|
|
First Step OR |
|
✓ | ✓ |
|
|
MoodTools-Depression Aid |
|
|
|
✓ | ✓ |
HOPE-Broome County Mental Health |
|
✓ | ✓ |
|
|
STOPP app |
|
✓ |
|
✓ |
|
Jason Foundation A Friend Asks |
|
✓ | ✓ |
|
|
Suicide Safety Plan | ✓ |
|
|
|
✓ |
Safe Students |
|
✓ | ✓ |
|
|
Got your back |
|
|
✓ | ✓ | ✓ |
Stanley-Brown Safety | ✓ |
|
✓ |
|
✓ |
Operation Reach Out | ✓ | ✓ |
|
|
✓ |
trustTalk247 |
|
✓ | ✓ |
|
|
Just in Case for Colleges |
|
✓ | ✓ |
|
|
Relief Link |
|
|
✓ |
|
|
Ulster County Speak |
|
✓ | ✓ |
|
|
Friend2Friend |
|
✓ |
|
|
|
be safe suicide safety plan |
|
|
✓ | ✓ | ✓ |
Every Teen Seen |
|
✓ |
|
|
|
distract |
|
|
|
|
✓ |
Be Safe |
|
|
✓ |
|
✓ |
Calm Harm-manages self harm |
|
|
|
✓ |
|
R U Suicidal? |
|
|
|
|
|
Say Something |
|
✓ | ✓ |
|
|
Anemone Crisis App |
|
|
✓ | ✓ | ✓ |
Prevent Suicide-Highland |
|
✓ | ✓ |
|
✓ |
There is Hope |
|
✓ | ✓ |
|
✓ |
A.L.E.R.T. |
|
|
✓ |
|
✓ |
Self Harm Recovery |
|
|
|
|
|
Suicide? Help? Tayside |
|
|
|
|
✓ |
Stay Alive |
|
|
|
|
✓ |
Dutchess County HELPLINE |
|
✓ | ✓ |
|
|
The LifeLine |
|
|
✓ |
|
✓ |
DMHSb: Suicide Prevention Info |
|
✓ |
|
|
|
Is S/O Suicidal? |
|
✓ |
|
|
|
Did someone you know suicide? |
|
✓ |
|
|
|
Step Up and Speak Out |
|
✓ | ✓ |
|
|
Kokua Life |
|
✓ | ✓ |
|
✓ |
MSE&SUICIDE ASSESSr |
|
✓ |
|
|
|
Calm Care |
|
|
✓ | ✓ |
|
My Shiny Thing |
|
|
|
✓ |
|
PMCS Combating Suicide |
|
|
✓ |
|
|
SeeSave/See Something Save Someone |
|
✓ | ✓ |
|
|
Community Stress First Aid |
|
✓ |
|
|
|
iHelp Sunshine Coast | ✓ | ✓ | ✓ |
|
|
MS DMH-Shatter in the Silence |
|
|
✓ |
|
|
Better Stop Suicide | ✓ |
|
|
|
✓ |
SCNGc Suicide Prevention |
|
✓ |
|
|
|
Alaska Careline |
|
✓ | ✓ |
|
|
Prevent Suicide: Dumfries & Galloway |
|
✓ | ✓ | ✓ | ✓ |
TheHopeLine |
|
✓ | ✓ |
|
|
MYPLAN-your safety plan |
|
|
✓ |
|
✓ |
ReMinder Suicide Safety Plan | ✓ | ✓ | ✓ | ✓ |
|
SafetyNet: Your Suicide Prevention App | ✓ | ✓ | ✓ | ✓ |
|
TUFMINDS |
|
|
✓ | ✓ |
|
UnCut App |
|
✓ |
|
|
|
Don’t Panic–depression and panic help | ✓ |
|
✓ | ✓ | ✓ |
Emotional Support Helpline Directory |
|
|
✓ |
|
|
Yellow Ribbon Foundation |
|
|
✓ |
|
|
A Teen Suicide Prevention Anime |
|
|
|
|
|
Seeking the Military Suicide Solution |
|
|
|
|
|
Elijah |
|
|
|
|
|
Suicide Prevention-Ways to Help a Suicidal Friend |
|
|
|
|
|
aFeature present.
bDMHS: Durham Mental Health Services.
cSCNG: South Carolina National Guard.
The mean overall MARS score of all apps was 3.5 (range 2.1-4.5), and 83% (55/66) of apps had a minimum acceptability score of 3.0 (
Data on the MARS subscale scores and the overall MARS scores categorized according to the main app approaches are presented in
Mobile app rating scores according to main approach (N=66).
Main approach | Count, n (%) | Engagement, mean (SD) | Functionality, mean (SD) | Aesthetics, mean (SD) | Information, mean (SD) | Overall, mean (SD) |
Crisis plan | 12 (18) | 3.13 (0.77) | 3.96 (0.59) | 3.38 (0.70) | 3.64 (1.16) | 3.53 (0.76) |
Support | 24 (36) | 2.91 (0.76) | 3.86 (0.71) | 3.82 (0.72) | 4.23 (0.53) | 3.70 (0.83) |
Psychoeducation | 17 (26) | 2.56 (0.65) | 3.80 (0.54) | 3.34 (0.66) | 3.70 (1.00) | 3.35 (0.87) |
Coping skill | 10 (15) | 3.24 (0.64) | 4.00 (0.64) | 3.43 (0.83) | 3.69 (0.87) | 3.59 (0.78) |
Other | 3 (5) | 2.40 (0.35) | 3.4 (0.88) | 2.50 (0.50) | 3.66 (1.44) | 2.99 (0.96) |
The total and subscale MARS scores were all significantly correlated, indicating that app quality was consistent across all areas assessed (eg, apps scoring high on engagement also tended to score high on function, aesthetics, and information). The overall MARS score was neither correlated with user app rating (
Correlations among total mobile app rating scale (MARS) score, four MARS dimension scores, rating, and number of features.
Characteristics | MARSa | Rating | Number of features | |||||||||||||||
|
Total | Engagement | Function | Aesthetics | Information |
|
|
|||||||||||
|
||||||||||||||||||
|
Total | 1.00 | —b | — | — | — | — | — | ||||||||||
|
|
|||||||||||||||||
|
|
Correlation factor | 0.72 | 1.00 | — | — | — | — | — | |||||||||
|
|
<.001 | — | — | — | — | — | — | ||||||||||
|
|
|||||||||||||||||
|
|
Correlation factor | 0.80 | 0.40 | 1.00 | — | — | — | — | |||||||||
|
|
<.001 | <.001 | — | — | — | — | — | ||||||||||
|
|
|||||||||||||||||
|
|
Correlation factor | 0.83 | 0.60 | 0.50 | 1.00 | — | — | — | |||||||||
|
|
<.001 | <.001 | <.001 | — | — | — | — | ||||||||||
|
|
|||||||||||||||||
|
|
Correlation factor | 0.82 | 0.35 | 0.63 | 0.56 | 1.00 | — | — | |||||||||
|
|
<.001 | .002 | <.001 | <.001 | — | — | — | ||||||||||
|
Rating | −0.01 | −0.01 | −0.04 | −0.01 | 0.05 | 1.00 | — | ||||||||||
|
Number of features | 0.20 | −0.02 | 0.22 | 0.09 | 0.32c | 0.24 | 1.00 |
aMARS: mobile app rating scale.
bNot applicable.
c
We examined the user experience, usability, and engagement of mHealth apps designed for suicide prevention. There are three main findings of this study. First, although the majority apps included elements of best practices to reduce suicide risk, none included all these features. Second, most of the reviewed apps were designed for suicidal individuals, rather than for clinicians, friends, and families. Third, the MARS score of the majority of apps was in the
Since the app review by Larsen in 2016 [
The majority of the apps (49/66, 74%) were specifically designed for suicidal individuals, only 15% (10/66) of which were designed for friends or family and 2% (1/66) for clinicians. This highlights a potential deficit in apps that are designed to treat, manage, or cope with individuals at risk of suicide. A major obstacle in overall suicide prevention is the lack of willingness in treating suicidal individuals among mental health providers [
Most mobile apps in this review were at least moderately usable and engaging. Although 17% (11/66) of apps yielded unacceptable scores, the average MARS scores were in the acceptable range, indicating that the apps were generally usable and engaging. However, it is unclear whether the apps designed to reduce suicide are reaching the appropriate audience or designed according to what suicidal users need or want. The user context or environment may be a significant driver of the determination of engagement. A suicide app designed to help clinicians assess and manage suicide will likely need to be highly functional, but not necessarily fun to use. In contrast, in apps designed to help users reduce suicidal crises, ongoing app engagement may not be a goal, as app developers hope that suicidal crises will eventually be reduced. This makes the iterative design of suicide-related apps challenging because repeated use may not be an ideal outcome. Traditionally, app developers can use objective measures, such as
In general, there is a lack of research on consumer apps for suicide prevention. Melia et al [
Although this paper is the first review to specifically examine the user experience of mobile apps specifically designed for suicide, there are some limitations to the study worth discussing. First, as we only searched on app stores and systematic literature search was not performed, web-based apps that are not featured in app stores were, therefore, not included. Owing to the ease of development and maintenance of web-based apps compared with native apps [
Although this study identified many usable and engaging apps designed for suicide prevention in app stores, there are several opportunities for mobile app development and enhancement. In particular, there is a lack of apps designed to assist clinicians in treating suicidal patients. In addition, there is a need for more clinical evaluation of suicide prevention apps found in app stores. In general, mobile apps for suicide prevention should be carefully developed and clinically evaluated.
Characteristics and mean mobile app rating scale scores of mobile apps (N=66).
mobile app rating scale
mobile health
CRW receives consulting fees from Mindstrong Health, Click Therapeutics, and Behavioral Tech. In the past 3 years, RCK was a consultant for Datastat Inc, Holmusk, RallyPoint Networks Inc, Sage Pharmaceuticals, and Takeda. He has stock options in Mirah, PYM, and Roga Sciences. MKN receives text book royalties from Macmillan and Pearson publishers and has been a paid consultant in the past year for Microsoft and for a legal case regarding a death by suicide. He is an unpaid scientific advisor for TalkLife and Empatica. All other authors deny conflicts of interest.