Evidence-based technology-delivered interventions (TDIs), including mobile apps that promote wellness and mental health (mobile TDIs; mTDIs), have demonstrated great promise for improving emotional well-being and reducing psychological distress across various population samples, including college students [1,2]. Mindfulness apps in particular have been effective in reducing depression, anxiety, and stress and improving psychological well-being and life satisfaction [3]. Importantly, mindfulness mTDIs also promote the development of related skills, with participants reporting significant improvements in areas such as awareness, acceptance, nonjudgmental attitude, and focus on the present moment [4,5]. Thus, these tools offer an effective and accessible way to address the large treatment gap that exists between individuals who need mental health treatment and those who are receiving it, a discrepancy that is especially pronounced for college students with depression [6].

Beyond their clinical utility, mTDIs have broad appeal, especially among college students, who seem to prefer the convenience, immediacy, and perceived confidentiality [7] paired with the added benefit of 24-hour access, including availability after in-person clinic hours [1]. Although there are hundreds of mTDIs designed to address mental health problems, and these numbers are growing rapidly, many of these apps are not evidence-based [8,9]. Furthermore, clinical science grapples with translating the benefits often seen when evaluating mTDIs through research trials [10] into best practices for promoting real-world applications, often marked by low uptake and engagement [11]. Interventions might work well for a subset of people who engage with them but not for those who fall short of engagement or adherence standards [4]. Indeed, users cannot benefit from the behavioral changes facilitated by mTDIs if they do not engage with such programs and learn the necessary skills. To harness the potential benefits of mTDIs, it is an urgent priority to identify strategies and supports for fostering sustained engagement and integration into daily life so that users can ultimately learn and implement skills for well-being.

Although mTDIs are typically designed to be self-guided, they can be integrated with human support features to enhance user engagement and thus benefits. This study is part of a larger randomized controlled trial (RCT) that established the efficacy—specifically psychosocial benefits—of a mindfulness-based mTDI for college students experiencing depressive symptoms. Building on these findings, this study examines whether 2 human support enhancements—a face-to-face orientation and placement in small peer supportive accountability groups—(1) boosted engagement with the mindfulness-based mTDI, (2) cultivated increased practice integration into daily life, and (3) enhanced users’ self-reported skill learning.

Although mTDIs have great potential to deliver effective treatments to a broad population, such programs are marked by low rates of both uptake and sustained user engagement, particularly for self-guided treatments that involve lower levels of structure and prescriptive guidance [12,13]. Such digital approaches require users to be self-motivated to initiate and sustain their use independently [14]. Furthermore, the mental health symptoms driving users to seek out mTDIs, such as depressive symptoms (eg, reduced interest, energy, and concentration), may themselves interfere with mTDI engagement [15]. Unpacking patterns of user engagement along with studying intervention design strategies to boost and maintain engagement are critical steps for advancing the efficacy of evidence-based mTDI interventions.

Engagement has been defined as “a state of energy investment involving physical, affective, and cognitive energies directed toward a focal stimulus or task” [16,17]. For the purposes of this study, engagement is measured behaviorally via the number of use minutes and completed sessions recorded by the mTDI software. While there is ongoing debate regarding how to best define and measure engagement [18], tabulating app use data (including duration and frequency) remains one of the most common methods [19]. User data from mindfulness-based mTDIs generally show engagement rates that start high but gradually reduce over time, with precipitous drops within the first week of uptake [11]. Of concern, engagement is particularly low during self-guided follow-up study periods [20,21]; for example, Flett et al [4] found that college students used a mindfulness-based mTDI almost daily in the primary 10 days of the study, but less than half had any use in the subsequent “30-day extended use period”. This sentiment is reflected in conversations with college student app users as well: despite almost 75% reporting some benefit from mTDIs, the same proportion engaged with them weekly or less frequently [7]. Such research highlights the discrepancy between interest in, and engagement with, mTDIs.

While emerging research in the area of digital mental health frequently recommends the integration of human support to enhance mTDI engagement, there remain many questions related to what type of support is needed; for example, there is great variability in the provider (eg, peers, unlicensed professionals, and licensed clinicians), the modality (eg, in-person contact, video visits, telephone calls, and text-based messaging), and the frequency of support provided. Further complicating this issue is that few studies provide a detailed description of the human support that was delivered or examine the role of such support in their study outcomes, making it difficult to draw larger conclusions across studies [22]. This study aims to address this gap by specifically examining the impact of 2 human support features—participation in a preintervention orientation session and placement in a peer supportive accountability group—on mTDI engagement, practice integration beyond the app, and skill learning.

Supportive accountability is a theoretical model that seeks to explain how human support can improve engagement with mTDIs [14]. The model posits that a user is likely to engage more with an mTDI when they are accountable to another person. Accountability includes knowing that actions or inactions with the intervention will be observed, and behavioral choices will have to be justified to someone. Ideally, the person a user is being accountable to will be viewed as legitimate and with beneficial knowledge to offer, and therefore the accountability process is viewed as supportive and noncoercive. A systematic review of 208 studies examining user engagement with mTDIs did not explicitly test the supportive accountability model, but it found that guided or supported interventions had higher engagement than self-guided interventions and that being able to connect with others through the mTDI increased engagement [15].

The supportive accountability theory has been tested in a number of formats, including telephone coaching from clinicians and peers as well as through digital formats [14,23]. Coaching delivered by peers is as effective at improving engagement as coaching delivered by licensed clinicians [24]. Peer-delivered support typically has been offered through embedded message boards that include components of visible goal setting and completion as well as methods to provide support and communication to increase goal completion (eg, “likes” and comments). Several studies across different populations have demonstrated increased mTDI use by participants with access to peer support features compared to completely self-guided participants [25-27]. Even in trials with no differences in engagement between groups or no self-guided comparison group, participants tend to rate peer supportive accountability as positive and helpful [24,28-30]. A systematic review of 24 peer-supported digital mental health interventions concluded that using peer support to deliver or supplement mTDIs is feasible, acceptable, and effective at improving engagement and psychological functioning [31]. Ultimately, incorporating supportive accountability features into mTDIs for depression has the potential to improve both engagement and mental health outcomes for a wide array of users.

Research has less commonly explored the ways in which one-time support interactions, such as an orientation to a TDI, might enhance engagement. Psychotherapy research has explored the benefits of engagement sessions, typically using motivational interviewing (MI) tools, to enhance engagement with traditional mental health services [32,33]. More recently, MI has been incorporated into TDIs as a potential engagement strategy. Such techniques allow users to interact with programs in a more intentional manner by setting expectations and goals for program use, encouraging the use of problem-solving and other skills, and normalizing challenges [34]. More specifically, using MI to support TDI users as they prepare for, and enact, behavioral changes is particularly encouraged to support program engagement [34]. Brief motivational interventions, such as through SMS text messages or telephone calls, are viewed positively by TDI users, promote program engagement, and even enhance initial program outcomes [35-38]. Similarly, Linardon and Fuller-Tyszkiewicz [13] found increased engagement for trials that had at least 1 opportunity for contact with the researchers (either a telephone interview or an in-person interview before enrolling) compared to trials where participants could enroll in a study on the web without contacting a researcher.

Some research has targeted motivation before users engage with the program, such as through a synchronous (eg, face-to-face or telephone) orientation session with research staff or through a module within the TDI. While some studies seem to incorporate such techniques into their methodological design [29,39], few have provided details about the content of these sessions or explicitly investigated their impact on engagement or other outcomes. Encouragingly, researchers have begun to recognize that such techniques may serve as an active component of treatment that can be investigated in and of itself; for example, Bur et al [40] conducted an RCT of a web-based self-help program for depression, exploring the effects on user engagement of different types of support, including elements with human contact (personal guidance and a preintervention diagnostic interview) and those without human contact (automated reminders and a preintervention MI module). At the end of the intervention, participants who received human support in the form of individualized guidance demonstrated both greater reductions in depressive symptoms and greater treatment adherence compared to the other support conditions [40]. Beyond building motivation for engagement, these preprogram touchpoints can help users navigate the technology of a new app as well as the large amounts of content that many TDIs provide, both of which can be overwhelming, particularly when users are also contending with mental health challenges such as depression [8]. Overall, more research is needed to better understand the role that preprogram interventions (eg, introductory, orientation, or engagement sessions) may have and their effects on subsequent program engagement and associated benefits.

As with skills-based face-to-face treatments, the goal of mTDIs is to help users learn and practice skills so that they become integrated into their daily lives over time. Notably, the completion of out-of-session exercises is uniquely linked to symptom improvement and treatment outcome across disorders [41]. In a similar vein, Schlosser et al [42] distinguished between TDI active use (eg, tasks or skill practices completed) and passive use (eg, time on the app that was not spent meaningfully engaging with other users or task completion) and found that only active use was linked to symptom improvement. This signals the critical importance of assessing not only objective engagement counts (eg, TDI use minutes and completed sessions) but also broader elements such as practice integration and skill learning.

In a sample of college students with elevated depression, we examined the benefits of two mTDI enhancements—(1) a face-to-face orientation with or without (2) placement in a peer supportive accountability group—on three sets of outcomes: (1) mTDI engagement, (2) practice integration, and (3) skill learning. For the first set of outcomes—mTDI engagement—we hypothesized that those who were randomly allocated to receive the enhancements would demonstrate greater mTDI engagement—in terms of (1) mindfulness minutes and (2) total sessions completed—compared to those who were simply given access to the app without either of these enhancements. We examined whether participants in the peer supportive accountability group experienced incremental benefits in engagement, beyond those of the orientation. In an exploratory fashion, we examined whether there were differences among the 3 intervention groups on 2 other sets of self-report outcomes: For the second set of outcomes, practice integration, we specifically examined (3) “everyday mindfulness” and the likelihood of engaging in future mindfulness practices, whether (4) using the app, (5) engaging in mindfulness exercises on one’s own, or (6) generally being more mindful in daily life. For the third set of outcomes, skill learning, we specifically examined (7) learning about mindfulness as a concept and practice, (8) learning mindfulness skills, and (9) being aware of one’s thoughts and feelings.

This RCT (registered on OSF before the start of analyses) was approved by the Loyola University Chicago Institutional Review Board (project number 2330), and all participants provided informed consent. Minimal identifying data were collected, and identifiers were removed from the dataset for analysis. Confidential (ie, identifying) data were stored on a university-supported, password-protected network drive and in locked filing cabinets to which only the research team had access. Participants were compensated monetarily (up to US $70) or with participant pool course credits (up to 10 credits) for the completion of study sessions.

Over 8 successive semesters, we recruited undergraduate students from a midsized Midwestern university using a multipronged voluntary response sampling strategy, including the psychology participant pool; cross-campus listserve emails; and flyers seeking students who self-identified as “down, sad, or distressed.” Recruitment occurred at the beginning of each semester between 2017 and 2021. Eligible participants endorsed clinically significant levels of depressive symptoms as indicated by a score of ≥10 on a brief web-based screening using the Patient Health Questionnaire-8 (PHQ-8), which omits the suicidality item [43,44]. Individuals were excluded if they were engaged in psychotherapy at the start of the study, had regular practice of mindfulness or consistent use of the Headspace app within the past 6 months, or reported that they were unwilling to join the peer supportive accountability group if randomly allocated to this condition. As the broader project included electroencephalography assessments, participants were also excluded if they had a history of neurological conditions or head trauma (eg, concussions or seizures). Our sample size (10-27 students each semester) was determined by the number of participants who met study criteria each semester, paired with a consideration of resource constraints (ie, time, space, and funding) [45].

In examining group effects, we used an intent-to-treat strategy, including all participants who were randomly allocated to 1 of the 3 intervention groups and for whom we had relevant data. For example, of the 120 participants with valid engagement data, a small subset (10/120, 8.3%; all in the App group) did not engage with the app; following an intent-to-treat approach, these participants were included in analyses. Furthermore, all App+OPS participants with valid data were included in the analyses, regardless of their level of engagement in the web-based forum or attendance at group meetings. All statistical analyses were conducted using SPSS software (version 28.0; IBM Corp) [49]. We report observed effect sizes in the results. Missing data were handled via case-wise deletion for each specific analysis.

To examine overall differences among the 3 groups across 3 time points (midpoint, postintervention, and 1-mo follow-up), we conducted group×time ANOVAs and interpreted the group effects. For outcomes with an overall effect of group (human support condition), we examined pairwise contrasts to identify any differential impact of the orientation alone versus the orientation plus peer supportive accountability.

Accounting for the effect of running multiple tests on type I error, we calculated a Bonferroni correction by dividing .05 by the total number of individual ANOVAs and consider both the standard significance threshold (P<.05) as well as the Bonferroni-adjusted significance threshold (P<.0055) in evaluating statistical significance. We also report effect sizes [50]. Specifically, we examined partial eta-squared values using the standards proposed by Cohen [51] for small (η_p²≥0.01), medium (η_p²≥0.06), and large (η_p²≥0.14) effects. We evaluated the distribution (skewness) of outcome variables at each time point, and we also ran Kolmogorov-Smirnov tests for each variable (Multimedia Appendix 2).

Participant recruitment yielded 123 undergraduate students (age: mean 19.1, SD 1.5 y), with an average PHQ-8 screening score of 14.1 (SD 3.4; range 10-22), randomly allocated to use the app as follows: 30.1% (37/123) App participants, 30.9% (38/123) App+O participants, and 39% (48/123) App+OPS participants (refer to Figure 1 for the CONSORT [Consolidated Standards of Reporting Trials] flow diagram; the results are presented according to the CONSORT-EHEALTH [Consolidated Standards of Reporting Trials of Electronic and Mobile Health Applications and Online Telehealth] checklist [Multimedia Appendix 1]). Of note, of the 37 participants in the App group, 51% (19/37) first completed a waitlist period, and these participants did not differ from the other 49% (18/37) of App participants who activated their Headspace access code at the same time as the rest of their cohort or from App+O or App+OPS participants on baseline depression (F_3,120=0.30; P=.83). Table 1 presents baseline demographic characteristics for the full sample and for participants randomly allocated to the 3 study groups, reflecting a higher proportion of women attending college and endorsing depression, but demonstrating no baseline group differences (refer to footnote a below Table 1).

A few details shown in the CONSORT flow diagram are worth elaborating on. Three participants (n=1 in the App group and n=2 in the App+OPS group) did not have valid app-recorded engagement data due to data recording errors or access code malfunctions; therefore, they were excluded from analyses examining objective mTDI engagement. In addition, there were some missing data from surveys, as noted in the CONSORT flow diagram. Notably, participants who completed surveys at all 3 time points (82/123, 66.7%) did not differ from those who completed surveys at only 1 or 2 time points (41/123, 33.3%) in gender (χ²₂=0.1; P=.96), sexual identity (χ²₄=2.9; P=.58), racial and ethnic identity (χ²₅=2.9; P=.71), or baseline depression scores (t₁₂₀=0.81; P=.42) but were more likely to be younger in age (t₁₂₁=2.77; P=.006) and a first-year student (χ²₄=10.5; P=.03).

The following subsections present the results for the impact of 2 human support enhancements (a face-to-face orientation with or without placement in a peer supportive accountability group) on the 3 categories of outcomes: (1) engagement with the mTDI (minutes and sessions), (2) practice integration (eg, everyday mindfulness practice beyond the mTDI), and (3) skill learning (eg, self-reported awareness, learning, and skills).

This study demonstrated that empirically driven human support enhancements can increase (1) engagement with an mTDI; (2) the integration of intervention practice into daily life; and (3) learning, including perceived intervention-based awareness and skills. Specifically, this study found that users randomly allocated to attend a 90-minute, face-to-face orientation session to the mTDI—whether with or without additional random allocation to a peer supportive accountability group—demonstrated significantly higher (1) intervention engagement, including total minutes of mindfulness content and total number of sessions overall; (2) integration of intervention practice into everyday life, both currently and anticipated in the future; and (3) learning about mindfulness and related skills and an awareness of thoughts and feelings. Among participants randomly allocated to attend an orientation, those who were further randomly allocated to peer supportive accountability groups exhibited higher mean scores on app use across the trial assessment points, but this was not a significant difference. Furthermore, participants randomly allocated to either of the human support–enhanced conditions generally demonstrated significant benefits for the integration of skills and perceived learning compared to those randomly allocated to use the app alone, but there were no significant differences between these 2 human support–enhanced groups. These findings highlight the varied benefits of including human support in otherwise self-guided digital mental health interventions, but they leave open important questions about the connections among human support, engagement, and broader benefits for daily-life integration and learning.

This study used a rigorous RCT design with intent-to-treat analyses to examine multiple indicators of mTDI engagement and benefits, including objectively measured app use and self-reported practice integration and skill learning, through a 1-month follow-up period; yet, there are important limitations to address as well. First, although an RCT is the gold standard for establishing general intervention effects, other research is important for evaluating diverse intervention benefits (eg, dose-response effects, clinical utility, and social validity) as well as revealing processes and mechanisms of change. Future research should continue to explore various designs with a broader array of outcomes and more diverse samples to improve generalizability. Second, it is important to note that the onset of the COVID-19 pandemic occurred during the sixth semester of data collection, necessitating changes (eg, remote data collection, orientation, and App+OPS meetings) impacting cohorts 6 to 8. Third, although mTDIs provide methodological benefits by tracking in-app engagement and skills practice, data recording errors can occur; for example, 2.4% (3/123) of participants were excluded from mTDI engagement analyses due to glitches with their app-recorded use data. It is possible that other data-tracking errors could have occurred without the researchers’ knowledge. Similarly, it is possible that participants could be inattentive, distracted, or asleep during some of the practices recorded by the app. Fourth, although the broad recruitment strategies and voluntary response sampling method have benefits for generalizability, they also yielded a sample consisting largely of women (reflecting both the higher proportion of women in college and their greater likelihood of meeting criteria for depression), which limits generalizability in other ways. Relatedly, not all participants completed the surveys assessing self-reported practice integration and skill learning, further limiting the generalizability of the findings.

Fifth, it is important to note that the App condition included participants who activated their Headspace access code at the same time as the rest of the cohort as well as a subset of participants who accessed their code after a 3-month waitlist period. Although analyses did not reveal any significant differences in mental health symptoms between App participants who could access Headspace immediately and those who waited 3 months, it is possible that combining these 2 groups may have affected the results of the App condition, particularly because prior research suggests that intervention engagement can be negatively impacted by time spent waiting for services [64].

Sixth, and related, the 3 randomly allocated groups were nested such that participants in all 3 groups received the base intervention (App), 2 of the nested groups attended an orientation (App+O and App+OPS), and 1 of these nested groups included additional peer supportive accountability features (App+OPS). Analyses tested for the incremental benefits of the 2 human support enhancements; yet, future research might benefit from examining the benefits of an app+peer supportive accountability condition without an orientation session.

Finally, this study’s implementation of peer supportive accountability had some limitations that may have reduced the potential impact of the group on mTDI engagement and the integration of skills. The web-based forum included a “leaderboard” feature that displayed each member’s use of the app from the current week and the trial period overall. This may have promoted external motivation, which is typically not as effective as intrinsic motivation for sustained engagement [15]. This type of feature also may function differently for individuals experiencing depression compared to those with other mental health symptoms. Given that self-criticism and negative self-concept are common among individuals with depression [65,66], having progress metrics shared with the larger group may have been perceived as discouraging rather than motivating. In addition, the use of Facebook for the online forum—selected for its social connection and privacy features—may have made engagement with the peer group more difficult because Facebook is becoming increasingly less popular among young adult participants [67]. Future studies should explore alternative platforms for web-based interaction among participants, ideally integrating all peer support features into the mTDI. Furthermore, future research should explore whether the benefits of human support, such as those demonstrated in this study, might also be harnessed with automated support, whether preprogrammed or assisted by interactive artificial intelligence. Beyond the more readily automatized elements such as providing information and reminders, research on the benefits of human support tends to emphasize more complex elements such as personalized prompts, interactive coaching, and professional guidance [2,22,34,53,54,68]. Notably, recent advances in artificial intelligence and machine learning might allow for the transfer of some human support elements and benefits to automated programming [2]. Importantly, these developments raise crucial ethical and societal concerns that must be addressed [69].

This study’s findings have important implications that move beyond foundational research on whether to use mindfulness-based mobile apps in clinical intervention work by investigating how best to maximize their benefits, such as through brief, face-to-face orientations and through small-group peer support networks. These results can be applied more broadly to mTDIs and also are likely to extend to other user populations; for example, the peer supportive accountability model implemented here—with the notable request for more face-to-face contact by the participants themselves—might be similarly relevant for mTDI users who share a common identity or life circumstance, such as high school students, employees, veterans, parents, or patients [25,26,28].

Boosting mTDIs with human support features has numerous applications in various settings and with diverse people who can serve to support engagement and thus enhance benefits; for example, many health care systems allow for providers within different departments (eg, behavioral health, primary care, and specialty clinics) to refer patients to mTDIs as a means of addressing lower-severity symptoms, supplementing preexisting mental health services, or bridging the patient to other levels of care, such as while they are on a waitlist [70]. Similarly, many employers now offer mental health mTDIs as part of their health insurance package [71], and there is emerging research on integrating mental health care into routine curricular and extracurricular settings on college campuses [72-74]. While mTDIs are increasingly incorporated into nontraditional settings, the typical approach is simply to provide individuals with access to the mTDI, leaving users to download and engage with the program on their own. The findings from this study suggest that evidence-based mTDIs can be even more effective and have a more powerful reach and impact if supplemented with human support; for example, professionals, paraprofessionals (without a mental health degree), or peers can serve as coaches to support engagement with evidence-based mTDIs through routine settings such as primary care clinics [75], workplaces [53], and college student academic services [76]. This study also suggests that an initial, one-time group introduction to the mTDI conducted by a knowledgeable source can produce a myriad of benefits and represents a cost-effective and scalable option for larger institutions; for example, hospitals and clinics, schools and community agencies, and corporate and nonprofit organizations can pair their offerings of mTDIs with regularly occurring orientation sessions that assist new users in accessing the program, demonstrate the app and its features, answer questions, and elicit motivation. This would allow many potential users to join at once, thus improving efficiency, and might also boost social connection and a sense of community—which in turn would promote mTDI use and well-being more broadly.

While mTDIs have great promise for increasing access to well-being interventions, relatively low rates of user uptake and engagement are current barriers to achieving the potential reach and impact of mTDIs. The results of this study illustrate that human support—particularly an initial face-to-face orientation session that provides psychoeducation, builds familiarity with the mTDI, and promotes motivation and connection—boosts mTDI engagement, enhances the integration of intervention practice into everyday life, and increases skill learning. Future work is needed to determine the active ingredients of the orientation (eg, credibility, personalized guidance, motivational enhancement, and proactive problem-solving) that might drive increased levels of engagement and the associated positive intervention benefits. Identifying optimal intervention dosage levels for evidence-based mTDIs also remains critical to fully harnessing the potential of these interventions and their role in improving societal health and well-being.