Introduction
Limited Accessibility of Existing Mental Health Treatments
Existing mental health treatments have long been inaccessible due to well-established structural (eg, cost, transportation, and time) and individual (eg, stigma and distrust of providers) barriers [1-3]. Furthermore, people experiencing suicidal thoughts or engaging in self-harm often fear the negative consequences of disclosing these experiences (eg, involuntary hospitalization) [4,5], creating additional barriers to treatment among those who may need it most. This is particularly true for teenagers whose caregivers often serve as gatekeepers for their own mental health care [6-8] and who often fear the involvement of a caregiver without their consent [5].
To fill the gap between the need for support and access to it, many teenagers seek and receive mental health support on the web [9,10]. Specifically, a majority (61%-85%) of teens and young adults have sought information or help for their mental health on the web [10-12]. Teenagers report using web-based mental health resources as a way to obtain free, easily accessible, or anonymous support [12,13], and web-based communities via social platforms (eg, forums and discussion boards) offer a space to share one’s experiences with others who have faced similar challenges [14-17]. Youth facing greater psychological distress are also more likely to search for and discuss topics related to their mental health on the web [15]. Thus, these web-based communities in social platforms represent a unique, underexplored avenue for reaching teenagers in need of mental health support.
Offering Mental Health Support via Web-Based Social Platforms
Initial efforts suggest that it is possible to provide mental health supports that are integrated into existing web-based social platforms. Koko, a nonprofit, web-based mental health platform, provides on-demand peer support [18,19], crisis triage [20], and self-guided interventions on topics such as body image, self-harm, and stress management. People can access Koko services on various messenger apps (eg, Telegram, Facebook Messenger, and Kik) or via direct message channels of certain social networking platforms (eg, Tumblr); in recent studies, a portion of Koko users were directly referred from a social media integration [20]. However, no previous evaluation of Koko’s services has focused entirely on investigating the acceptability and effectiveness of embedding mental health supports within a social networking platform.
Single-Session Interventions as Embedded Web-Based Support
Single-session interventions (SSIs)—brief, targeted interventions designed to be completed within 1 sitting or clinical interaction—provide an especially promising intervention format for reaching young people on the web [21]. Multiple large-scale studies have indicated that SSIs can create meaningful changes for key mental health outcomes. In a recent randomized controlled trial of adolescents (N=2452), a 20- to 30-minute web-based behavioral activation SSI (Project Action Brings Change [ABC]) outperformed a supportive therapy control program at reducing postintervention hopelessness and depressive symptoms 3 months later [22]. Another trial (N=565) found Project Stop Adolescent Violence Everywhere (SAVE), a web-based 30-minute SSI targeting self-injurious thoughts and behaviors in teenagers, improved postintervention self-hatred, and desires to stop future self-harm, relative to an active control [23]. Taken together, substantive evidence suggests the efficacy of web-based SSIs in improving mental health outcomes among young people.
Despite the great potential of SSIs to increase access to mental health supports, little large-scale research has evaluated their acceptability and effectiveness when disseminated in web-based settings that young people already routinely access. One study evaluated the acceptability and preliminary utility of a free, open-access platform offering 3 web-based SSIs for youth [24]. After viewing paid advertisements on a social media platform (ie, Instagram), nearly 700 youths clicked on the advertisement and viewed the web-based platform within 6 months. The youths rated all 3 SSIs as acceptable, and analyses indicated significant pre- to postintervention reductions in hopelessness and self-hate among 187 SSI completers. However, to complete an SSI program, the youth had to be (1) aware of the open-access platform and (2) motivated to visit the platform on a separate, unfamiliar website and finish a 30-minute activity. Reducing friction (ie, points of difficulty) between individuals and their technology during key help-seeking thresholds may be particularly important for encouraging greater access to mental health supports on the web [25]. Rather than relying on individuals’ ability and motivation to seek external mental health resources in moments of heightened distress (eg, standalone web-based resources, crisis hotlines, and text lines), SSIs could provide accessible, in-the-moment anonymous supports for young people who are already embedded in popular social platforms.
Considerations for Intervention Design
SSIs designed for these web-based, embedded contexts should be streamlined to improve engagement while retaining their therapeutic value. Once disseminated in the real world, digital interventions often encounter issues with user engagement (ie, low uptake and low completion) [26-28]. Substantially reducing the time and effort required to complete web-based SSIs (eg, condensing content and reducing the number of writing prompts) may improve user engagement, particularly for SSIs delivered as in-the-moment supports at times of elevated distress. Determining which elements of SSIs to condense versus preserve presents a particular design challenge for the creation of SSIs embedded on the web.
Several intervention design principles, drawn from basic research in social psychology, education, and marketing [29], have been theorized to support the use of web-based SSIs for mental health: (1) using neuroscientific evidence to normalize youths’ experiences and boost message credibility; (2) centering youths’ expertise in their own lived experiences; (3) asking youths to share what they have learned with others, using their own words; and (4) providing testimonials from others experiencing similar and relevant challenges [21]. These core principles have been featured in much of the existing research evaluating web-based SSIs [22-24]. However, limited research to date has evaluated whether very brief, streamlined SSIs (ie, SSIs much shorter than 30 minutes) that retain these core principles may still provide some benefit as quick, free mental health supports embedded on the web.
This Study
As a majority of existing mental health treatments remain inaccessible, many young people seek mental health support via social platforms on the web. SSIs are uniquely well positioned for integration within web-based social platforms, providing free, brief, and anonymous mental health support options at scale. However, little research has formally evaluated the acceptability and effectiveness of very brief SSIs designed for this context. This study adapted 3 web-based SSIs (5-8 minutes) that were offered as Koko minicourses on Tumblr, a microblogging and social networking website with 135 million active users monthly [30]. The aims of this study were 2-fold: (1) to evaluate the acceptability of SSIs in this context via user data and acceptability ratings and (2) to assess short-term effectiveness of all 3 SSIs in improving key mental health outcomes (ie, hopelessness, self-hate, desire to discontinue self-harm) from before to after the SSI. Given the unique barriers to mental health treatment access faced by teenagers, the analyses were restricted to teenagers (ages 13-19 years) where possible. The results may inform continued efforts to increase access to free, anonymous, and timely mental health support options.
Methods
Ethical Considerations
For this study, we collected anonymous data exclusively from individuals on Tumblr—all of whom were introduced to the service by either (1) clicking on a featured advertisement from Tumblr (eg, “take control by taking this mood-boosting minicourse”) or (2) direct referral from the platform. As all data were part of a completely anonymous program evaluation, this study was deemed as nonhuman subjects research in consultation with the institutional review board at Stony Brook University. In addition, Koko’s privacy policy and terms of service acknowledge that anonymized data may be shared for research purposes.
Recruitment and Procedure
The direct referral pathway for each of the 3 SSIs was similar. Users who searched for mental health topics on Tumblr were shown an in-app overlay with links to various resources, such as The National Suicide Prevention Lifeline. A set of over 1300 keywords and their derivations were used to detect terms such as “self-harm” or “depression” as well as slang and obfuscations, such as “sewer-slide” and “'s3lf h@rm.” In addition to links to crisis lines, users were also sent a direct message from Koko through the Tumblr direct message channel. Specifically, they were sent the following automated message from a chatbot called “Kokobot”:
Hi! I’m Kokobot [wave emoji]. I’m working with Tumblr to connect people who are interested in mental health topics. Type “hi” to get started...
Next, users were onboarded to the service and asked to describe a recent negative situation that they have been facing, along with any associated negative thoughts. From there, a set of text-based classifiers for mental health [20,31] categorized the user’s posts and directed them to one of several resources, including peer support, crisis lines, and SSIs. Users could also access any of these services from a main menu at any point while using the platform. Users whose text descriptions were flagged by a crisis model were asked to specify whether their current struggles were related to suicidal thoughts, abuse, eating disorders, or self-harm. If a user disclosed that they were struggling with self-harm, they were shown crisis lines from around the world as well as a link to Project SAVE.
The 3 SSIs were initially introduced as Koko minicourses at 3 separate times: March 2021 (Project ABC), June 2021 (REFRAME), and July 2021 (Project SAVE). Across all 3 SSIs, the data for this study were collected through February 2022. Preintervention data were collected immediately before beginning each SSI (ie, each Koko minicourse), and postintervention data were collected immediately following the completion of a program.
SSI Programs
Project ABC Single-Session Intervention
Project ABC minicourse was a briefer 5- to 8-minute version of the original 20- to 30-minute Project ABC SSI evaluated in earlier randomized trial research [22]. As with the full-length program, the abbreviated Project ABC SSI used principles of behavioral activation, encouraging individuals to “take action” and engage in pleasurable behaviors that align with personal values to boost mood and build self-efficacy. Similar to the original SSI, the brief minicourse involved the following components: (1) psychoeducation, describing how taking values-based actions can boost mood over time; (2) values assessment, where individuals identify a top value for them (eg, academics, friendships, hobbies, family, or staying active); (3) “action plan” exercise, where individuals develop a personalized plan for engaging in meaningful activities of their choice; (4) roadblock exercise designed to identify real-life obstacles and how to address them; and (5) writing prompts, where individuals can share what they have learned with other Koko users. This shortened version of the Project ABC SSI contained condensed intervention content, fewer writing prompts, and greater emphasis on multiple-choice, interactive options, relative to the original 30-minute program.
Project SAVE Single-Session Intervention
Before disseminating as a Koko minicourse, an abbreviated (8-minute) version of the Project SAVE SSI was adapted from an original 30-minute program [32]. Project SAVE was designed to reduce the use of self-harm behaviors to cope with high emotional distress, especially in the context of self-hatred or desires to punish oneself [33]. Similar to the original SSI, the 8-minute version of Project SAVE included evidence-based techniques that are common in cognitive behavior therapies (eg, psychoeducation and secondary distress tolerance skills). Specifically, Project SAVE included four key components: (1) information about how changing actions (eg, reducing self-harm behavior, being kinder to yourself) can change feelings and thoughts for the better; (2) statistics and testimonials from other teenagers with lived experiences of coping with self-hatred or self-harm; (3) alternative coping strategies to use in place of self-harm; and (4) offering advice to others using Koko based on what they learned. Compared with the original, the shortened Project SAVE SSI contained condensed material, fewer writing exercises, and more multiple-choice, interactive options.
The REFRAME Single-Session Intervention
The REFRAME SSI (5 minutes) teaches cognitive reappraisal, an emotion regulatory strategy that involves modifying one’s interpretation of stressful situations [34]. A recent multicountry study with 21,644 individuals found that a brief web-based training on cognitive reappraisal reduced negative emotions and increased positive emotions about COVID-19–related stressors [35]. The REFRAME SSI on Koko included similar components, but it did not specifically target COVID-19–related stressors and it did not distinguish between multiple reappraisal strategies (eg, reconstrual vs repurposing). It included (1) a brief introduction to cognitive reappraisal; (2) a simplified description of its neural correlates; (3) practice examples with vignettes taken from the Koko peer support platform; and (4) a set of brief prompts to help users engage positive reappraisals in their own lives (eg, “This could get better because...”; “This isn’t 100% my fault because...”). Users were also taught how to practice this skill on the Koko peer support platform and that helping others reappraise may confer positive psychological outcomes [36,37].
Measures
Demographics
Demographic information (age, gender, and race and ethnicity) was collected as part of the preintervention measures for individuals completing Project SAVE.
Hopelessness
The Beck Hopelessness Scale-4 is a brief and reliable measure used to assess hopelessness in young people [38,39]. At pre- and postintervention time points for the Project ABC and REFRAME SSIs, responders indicated their agreement “right now, in the present moment” with each of 4 statements (eg, “I feel that my future is hopeless and things will not improve”) on a 4-point Likert scale (1=“absolutely disagree”; 4=“absolutely agree”). Hopelessness scores for each person (4-item average) ranged from 1 to 4, with higher values indicating greater hopelessness. Internal consistency was Cronbach α=.84 and Cronbach α=.89 for pre- and post-Project ABC time points, and α=.81 and Cronbach α=.87 for pre- and post-REFRAME SSI time points, respectively.
Self-hate
The Self-Hate Scale is a reliable 7-item measure used to assess self-hatred in young people [24,40]. At pre- and postintervention time points for the Project SAVE SSI, respondents indicated how true each statement (eg, “I hate myself”) is for them, “right now, in the present moment,” on a 7-point Likert scale (1=“not at all true for me”; 4=“somewhat true for me”; 7=“true for me”). The self-hate scores for each person (7-item average) ranged from 1 to 7, with higher values indicating greater self-hate. Internal consistency was Cronbach α=.90 and Cronbach α=.94 for pre- and post-Project SAVE SSI time points, respectively.
Desire to Discontinue Self-harm
Desire to discontinue self-harm behavior was indexed using a single item adapted from the Self-Injurious Thoughts and Behaviors Interview–Revised [23,41]. Individuals were asked, “How much do you want to stop purposefully hurting yourself without wanting to die?” and told to rate their answer on a 5-point Likert scale (1=“I don’t want to stop at all”; 5=“I definitely want to stop”). A sixth option (eg, “I have stopped engaging in these behaviors”) was available for individuals not currently engaging in self-harm behaviors. Data for these individuals were not included in the analyses evaluating pre- to post-SSI changes in this outcome.
SSI Feedback
After completing all 3 SSIs, users were prompted to provide a quantitative “star rating” of the program, from 1 to 5 stars, where higher star values indicate higher ratings or more positive feedback. In addition, after the intervention, all individuals were asked if they would like to provide qualitative feedback via an optional writing prompt (eg, “Do you have any feedback for us?”).
Statistical Analysis
Power
Previous web-based SSI research suggests small to large within-group effect sizes for key outcomes (including hopelessness and self-hatred) using more naturalistic study designs (ie, not randomized controlled trials) [24]. Thus, we estimated our statistical power to detect a small within-group effect (Cohen d=0.20) using a paired 2-tailed t test in the smallest of the 3 samples (ie, REFRAME, n=768 pairs).
Data Exclusion
We assessed the total number of views, starts, and completions for each SSI, as well as item-level drop-off data within each SSI, to describe broad usage patterns without excluding any data. Demographic and outcome data were only recorded and made available once individuals advanced through to the end of a program and clicked “submit.” All pre-post analyses, star ratings, and qualitative data were therefore conducted and reported within program “completers” (ie, individuals who completed an SSI). In addition, where demographic data were available (Project SAVE SSI data), we restricted our analysis to teenagers (ie, excluding individuals who reported ages outside of 13-19 years). As Project SAVE was designed for teenagers engaging in self-harm, only individuals who self-disclosed recently engaging in self-harm (via the Koko onboarding pathways described earlier in this section) were directed to complete this program.
Usage Patterns and SSI Feedback
For each SSI, we evaluated usage patterns and feedback, including the number of people who viewed (ie, opened the first page of the program), started (ie, advanced past the first page), completed (ie, advanced through the entire program and beyond the questions), and provided “star ratings” (ie, quantitative ranking of 1-5 stars, with higher stars reflecting a higher rating) for each SSI. In addition, we calculated the program completion rates (percentage completed out of those who started) and average star ratings for each SSI. To illustrate the types of qualitative feedback each SSI received, we extracted specific examples of positive and negative feedback (see the Results section).
Aggregate-level data were available for views and user dropout, for every page within each SSI, among the entire sample (ie, without focusing solely on program completers). We reported these results by plotting the percentage dropout from the number of views for each page within each SSI.
Evaluating Pre-Post Changes
We evaluated pre- to postintervention changes for 4 outcomes: hopelessness (via two separate tests, 1 for Project ABC and 1 for REFRAME) as well as self-hate and desire to discontinue self-harm (Project SAVE only). After checking appropriate assumptions (ie, verifying that pre-post difference scores for each outcome were approximately normally distributed) [42], we performed 2-tailed, paired t tests across all outcomes for all programs. We calculated Cohen dz for paired t tests using t values, sample sizes, and the Measure of the Effect package in R (version 4.0.0) [43] to evaluate the magnitude of within-group changes for each outcome. For outcomes where difference scores were not normally distributed, we completed an additional sensitivity analysis using the nonparametric Wilcoxon signed-rank test to evaluate within-group changes from before to after the intervention [44]. We applied a false discovery rate correction to all 4 P values to reduce the likelihood of false positives [45]. Thus, we interpreted the results for each outcome as significant if the false discovery rate–corrected P values were <.05. All analysis code and deidentified data for this study have been made publicly available [46].
Discussion
Principal Findings
Within 12 months, 3 very brief (5-8 minutes) web-based SSIs were viewed >18,800 times and completed >6100 times. Offered as Koko minicourses and embedded within a popular social platform, all 3 SSIs received high-quality ratings (average ratings >4 out of 5 stars). Of the participants who started an SSI, between 25% and 57% of them completed one. Among those who completed Project ABC and REFRAME minicourses, individuals reported a decrease in hopelessness from before to after the SSI. For those who completed Project SAVE, individuals reported decreased self-hatred and increased desire to stop future self-harm behavior from before to after the intervention. This study represents a real-world evaluation of the acceptability and short-term utility of web-based SSIs as very brief, anonymous, “in-the-moment” mental health supports that can be integrated within major social platforms such as Tumblr.
Comparison With Prior Work
Consistent with existing research on web-based SSIs [24], all 3 interventions were (1) rated as acceptable by users and (2) associated with improvements across primary outcomes. Furthermore, despite reducing intervention length by nearly 75%, effect sizes for these 5- to 8-minute SSIs were remarkably similar to effect sizes for SSIs designed to last 30 minutes (Cohen dz =0.71 vs 0.81 (hopelessness) and Cohen dz =0.61 vs 0.67 (self-hate) for full and abbreviated SSIs, respectively) [24]. These results support a growing body of evidence that suggests even extremely brief digital interventions may improve clinically relevant outcomes [20,47].
One possible reason for the relative similarity of postintervention effect sizes observed in this study (5-8–minute SSIs) versus within-group effects in earlier randomized trials (30-minute SSIs) [22,23] might be the shared design features and principles across the “brief” and “very brief” versions of these interventions. Given that these core principles have been featured in all versions of these SSIs to date, it is possible that these common principles are more important than the exact length of the SSI. Future research should formally evaluate this possibility in a head-to-head trial by comparing SSIs of various lengths.
Our results may also indicate the real-world utility of these very brief web-based SSIs. Naturalistic study designs similar to that used in this study are crucial for evaluating acceptability and utility among web-based SSIs, as randomized trials overestimate user engagement levels for unguided, digital mental health interventions—often producing usage rates that are 4 times greater than rates observed in real-world settings [27]. Once these interventions are disseminated outside of randomized trials, a small minority of individuals (0.5%-28.6%) complete all content [28]. Therefore, relative to real-world completion rates of other digital self-help interventions for mental health, completion rates for the 3 SSIs tested in this study (25%-57%) were extraordinarily high.
Low real-world completion rates for self-help digital interventions, especially relative to guided digital supports [26,48,49], are unsurprising; unguided supports require substantial emotional and mental bandwidth (eg, energy, sustained attention, and motivation) to find and engage with an unguided mental health support tool at the very moments when these capacities may be at their lowest (eg, when experiencing elevated distress). Our study sought to minimize the bandwidth necessary to engage with 3 single-session, unguided supports by (1) dramatically reducing their length and (2) bringing these interventions to web-based spaces where people already are. Future research should explicitly assess whether these strategies improve completion rates across other unguided digital mental health supports.
Embedding SSIs within a popular social platform (Tumblr) also likely increased the visibility and uptake of these SSIs. For example, after 6 months of recruitment using paid advertising and leveraging substantial media coverage, 1 study found that 700 youths viewed an open-access, web-based mental health platform featuring 3 SSIs [24]. The 3 SSIs in this study received a combined ≥18,800 views within 12 months—nearly 6240 of which were views of interventions that were never featured in a Tumblr-based advertisement (Project SAVE and REFRAME). These numbers suggest considerable interest in accessible, anonymous, “in-the-moment” support options, as well as the potential to sustainably offer these supports at high scale and relatively low cost.
Notably, existing research identifies safety as a primary ethical concern for researchers and stakeholders interested in using digital mental health tools among youth [50]; many digital resources lack evidence or clinical validation and others may not provide sufficient protection for sensitive user data. By using SSIs that have previously been evaluated in rigorous randomized trial research [22,23] and by collecting anonymous pre- and post-SSI user data, in this study, we aimed to mitigate these common concerns. Thus, calls for clearer guidelines for safety and quality in mental health apps [51] should also be accompanied by clearer guidelines, both for academic- and industry-based program developers and researchers, on how to responsibly and ethically disseminate mental health resources via partnerships with social networking platforms.
Strengths and Limitations
This study has several strengths. Although randomized trial research overestimates user engagement for digital interventions [27], it constitutes much of the existing research on web-based SSIs to date [22,23,52]. Therefore, the naturalistic design of this study provides a valuable, more accurate estimate of the potential for web-based SSIs to offer mental health supports at scale. Koko’s partnership with Tumblr also made it possible to offer anonymous, “in-the-moment” support for people seeking mental health–related content within the platform, rather than requiring individuals to search for external resources. Finally, we conducted a fine-grained analysis of usage data (ie, dropout) going item by item for each SSI. It was especially valuable to recognize high levels of user dropout relative to page views (3%-13%) on pages requesting written responses. Further studies should seek to understand patterns in user dropout and how they may inform digital intervention design. In addition, future evaluations may also want to expand upon this study by conducting smaller-scale usability testing [53] and research explicitly designed for the in-depth analysis of rich qualitative user feedback [54].
In addition to the aforementioned strengths, this study has some limitations. First, demographic information was not available for all individuals included in this study. Demographic information that was available from 1 of the 3 SSIs indicated substantial missing data (several users reported skipping demographic items to ensure they could not be identified). Black, Hispanic, and Asian youths consistently access mental health treatment at lower rates than their non-Hispanic White peers [55,56], yet up to 90% of youths in mental health clinical trials are White [57,58]. Digital interventions provide one possible avenue toward reducing disparities in access to mental health resources among racial and ethnic minority populations [59,60]; a vast majority of adolescents in the United States have internet access via either smartphones or desktops or laptops (88% and 95%, respectively) [61]. While there are disparities in broadband access by socioeconomic status, geographic region, educational attainment, and race and ethnicity, some evidence suggests that these gaps in access may be decreasing [62]. Should SSIs hope to provide truly accessible and equitable mental health supports, they must be accompanied with consistent evaluations of who has access to them, with researchers actively working to prevent further (or exacerbated) inequity via a new treatment modality [60].
In addition, as is often the case in web-based mental health support research [24,63,64], cisgender boys were underrepresented in this sample. Given unique barriers to seeking mental health treatments faced by cisgender boys and young men (eg, internalized masculine gender norms, such as “toughness”; elevated mental health stigma) [65,66], future mixed methods research may want to evaluate whether boys view SSIs as a less stigmatizing or more approachable mental health support option.
Finally, given that this study represents an unmasked evaluation of within-group intervention effects in a completers-only sample, our ability to draw causal inferences was limited. For example, those who completed an SSI may have provided more positive star ratings for the programs than those who exited before finishing the program. However, 2 of the 3 SSIs featured in this study have demonstrated positive effects on identical outcomes in large-scale, triple-masked, and randomized research [22,23]. SSIs have shown considerable promise in multiple studies and study designs.
Conclusions
Very brief SSIs (5-8 minutes) can be embedded within web-based social platforms as anonymous, in-the-moment mental health supports capable of reaching many individuals within months. Among those who complete these SSIs, individuals generally rate them as acceptable, and pre-post evaluations suggest that they may be helpful in reducing hopelessness and self-hate as well as in increasing the desire to stop self-harm. These pre-post findings, combined with results of earlier randomized trial research, suggest that SSIs delivered in this context may be a sustainable approach for providing the much-needed mental health resources, particularly for teenagers who may not have access to other mental health supports. Considering the substantial unmet need for mental health care among teenagers in the United States [67], SSIs may provide a valuable and complementary source of mental health support among a broader ecosystem of mental health treatment options (eg, school- and community-based programs, mental health screening, and gatekeeper training), all of which are necessary for reducing mental illness at scale.