Mental illness affects hundreds of millions of people worldwide, resulting in decreased quality of life, family and community disruption, increased health care costs, and a significant economic burden for employers [1,2]. Employee performance, rates of illness, absenteeism, and staff turnover are all affected by employees’ mental health status. In the United Kingdom, workplace mental health problems result in an estimated 70 million lost workdays and a total cost of up to £45 billion (US $61 billion) each year for businesses [3]. This is compounded by an estimated global treatment gap of >50% for people with mental health disorders [4,5].

There is growing interest in web and smartphone apps as a means of increasing the reach of mental health and well-being interventions [6,7]. Digital platforms can offer a broad range of content within a standardized environment that is interactive and dynamic while also being widely accessible, cost-efficient, and nonstigmatizing. With fewer access barriers, digital platforms also have the potential to offer a preventative solution to common mental health problems by facilitating sustained, proactive engagement [8,9]. Such platforms can vary widely in their means of delivery (web vs mobile app), the core therapeutic approach they use (with cognitive behavioral therapy [CBT], mindfulness meditation [MM], and positive psychology being common), and the duration and format of their content.

There is now convincing evidence for the effectiveness of digital interventions when delivered in health and community settings [10], as well as emerging evidence that they may be effective when delivered in occupational settings [11,12]. Previous meta-analyses have found small positive effects on psychological well-being (Hedges g=0.37) and work effectiveness (Hedges g=0.25) [11] and small to moderate effects on common mental health outcomes, such as stress (Hedges g=0.54), anxiety (Hedges g=0.34), and symptoms of depression (Hedges g=0.30) [12]. However, the current evidence base is limited by considerable heterogeneity across studies and an insufficient number of high-quality trials. Moreover, only a fraction of for-profit mental health apps (MHapps) are supported by empirical evidence [13], with added concerns that such platforms are frequently characterized by low adherence [14-16]. Together, these suggest the need for further research.

In this study, we conduct an external pilot randomized controlled trial (RCT) as part of the initial testing of Unmind—a novel digital mental health platform for the workplace. Unmind provides employees with tools to help them track, maintain, and improve their mental health and well-being. It features a broad range of content that draws on multiple evidence-based approaches such as CBT [17], MM [18], behavioral activation [19], acceptance and commitment therapy (ACT [20]), and positive psychology [21]. Central to the platform are individual learning and development courses (known as Series) designed to address specific topics of mental health and well-being. Series are short, standalone interventions, typically ranging between 5 and 7 sessions, each of approximately 10 minutes in duration, and can feature a mix of audio and video content, infographics, and interactions with a chatbot.

Consistent with recent guidelines on pilot trials [22,23], the primary aim of this study is to evaluate the feasibility of the study methods, and 3 separate Unmind Series that address the topics of stress, anxiety, and resilience, respectively, in preparation for a future definitive RCT. We chose to evaluate content relating to stress and anxiety as these are highly prevalent in the workplace [3] and have been extensively studied in previous evaluations of MHapps [24-26], allowing for a comparison of the current findings to previous evidence. In addition, we chose to evaluate content relating to resilience, as evidence suggests that it plays an important role in the prevention of mental health problems [27] and thus may be integral to the effectiveness of a preventative platform. A secondary aim is to establish the preliminary efficacy of each intervention with respect to self-report measures of stress, anxiety, symptoms of depression, and resilience, including establishing between-group effect sizes and 95% CIs (for each intervention compared with the control group). Although depression was not a specific target of any of the study interventions, we chose to include it as an outcome as it is highly comorbid with stress and anxiety [28,29] and a common problem in workplace settings [3]. Finally, we also aim to report on the combined effects of all interventions compared with the control group.

As the Unmind app comprises an extensive library of standalone interventions, it is important that each component of the app is evaluated. We chose to include 3 intervention arms in this study as this is more efficient than performing sequential 2-arm trials and increases the proportion of participants randomized to an intervention arm [30]. In addition, if a definitive RCT is warranted, an aim might be to evaluate whether each intervention arm has a greater effect on the specific outcome targeted by that intervention relative to the other intervention arms. Thus, this study uses a parallel, multi-arm, external pilot RCT design and recruited UK-based, community-dwelling, working adult participants who are randomly allocated to 1 of 3 intervention arms or to a no-intervention control group. We chose to implement a no-intervention control as (1) participants were not selected on the basis of poor mental health or seeking help for a problem, (2) recent evidence suggests that wait-list groups may introduce nocebo effects in psychotherapy trials [31], and (3) participants received monetary compensation for taking part.

The feasibility of each intervention arm is assessed via objective and self-reported outcomes capturing recruitment, retention, intervention uptake and adherence, acceptability, transferability, relevance, and negative effects. The preliminary efficacy of each intervention arm is assessed via self-report outcome measures delivered before (time point 0 [t0]) and after the interventions (time point 1 [t1]; 2 weeks after t0) and at the 1-month follow-up (time point 2 [t2]). The results of this study are intended to inform whether a definitive RCT to evaluate the efficacy of each intervention arm is warranted and provide estimates of the parameters required for its design and implementation.

This study was a parallel, multi-arm, external pilot RCT with pre- (t0) and postintervention (t1; 2 weeks after t0) assessments and a 1-month follow-up (t2). Participants were randomly allocated to 1 of 3 brief, self-guided psychological interventions (Series) featured on the Unmind platform or to a no-intervention control group in a 1:1:1:1 allocation ratio. Participants were working adults recruited from the Prolific web-based recruitment platform [34], and the entire study was conducted on the web between January and March 2021. Of note, the study commenced several weeks after the start of a third national UK lockdown (in response to the SARS-CoV-2 pandemic), and t2 data were collected after the commencement of a phased easing of lockdown restrictions. The trial was preregistered at ISRCTN 80309011, and a full study protocol was preregistered at Open Science Framework in December 2020.

The trial received ethical approval from the University of Sussex sciences and technology research ethics committee (ER/KC226/2).

Participants were recruited via the Prolific web-based recruitment platform, which has been empirically tested across attributes such as participant response rates and data quality [35]. Inclusion criteria were (1) aged at least 18 years, (2) currently residing in the United Kingdom, (3) self-identifying as being in full- or part-time employment, (4) having an active account on Prolific, (5) having access to an internet connection via a smartphone or desktop device, and (6) being fluent in English. Prolific indicated that there were 50,978 eligible individuals at the time of conducting the study.

Prolific implements a prescreening system that allows researchers to screen for eligibility without implementing a screening questionnaire. Prolific also supports the recruitment of study samples representative of the national UK population with respect to age, sex, and ethnicity based on guidelines from the UK Office of National Statistics. This study drew upon this feature to maximize the generalizability of the findings.

All study assessments were hosted on the Qualtrics survey platform [36], and participants were required to provide informed consent via a form built into each study assessment alongside a digital information sheet. All participants who completed the t0 assessment were invited to complete the t1 and t2 assessments via the Prolific recruitment platform. Participants were offered a £7 (US $9.50) incentive for completing each of the 3 study assessments (baseline, postintervention, and 1-month follow-up), delivered via Prolific. Participants randomized to one of the intervention arms received reminder messages on days 5 and 10 of the intervention (delivered via Prolific’s anonymous inbox system), encouraging them to complete all intervention sessions. However, participant reimbursement was not contingent on intervention adherence.

Randomization occurred at the end of t0 and was implemented via the Qualtrics randomizer feature, which uses block randomization to ensure balanced groups. It was not possible to blind the participants to group assignments. After randomization, participants assigned to one of the intervention arms were sent a message via Prolific’s anonymous inbox system with instructions on how to access their intervention, including using a unique voucher code to sign up to the Unmind platform. The research team remained blind to group assignment for the duration of data collection but was unblinded during data analysis.

As per formal guidelines [52], preregistered progression criteria were defined as follows: (1) full study recruitment within 1 month; (2) at least 30% intervention completion rates based on a previous meta-analysis of adherence to unguided psychological interventions [53]; (3) at least 75% adherence to protocol instructions (defined as the proportion of participants who refrain from engaging with ≥1 Series session outside of their allocated Series); (4) at least 50% of participants reporting being satisfied or very satisfied with the intervention and rating the quality of the intervention as good or excellent; and (5) the 95% CI on between-group effect sizes for secondary outcomes including at least a small effect (Hedges g=0.2) for ≥1 outcome measures.

Progression criteria were considered for each intervention arm individually.

This study was powered for CIs on the feasibility outcomes. A sample size calculation indicated that approximately 100 participants were required to estimate feasibility outcomes with a margin of error ≤10% (based on a conservative population proportion of 50% for retention and adherence, and a 95% CI). This is consistent with previous guidelines suggesting that 60 to 100 participants per intervention arm are optimal for estimating binary outcomes in pilot RCTs [54]. Therefore, we aimed to recruit 400 participants in total.

The results from all preregistered primary and secondary measures are reported. Minor deviations from the preregistered data analysis plan are reported in Multimedia Appendix 2 [40,41,55].

Participant demographics and other baseline variables are presented in Table 1. The mean age of the participants was 44.6 (SD 14.3) years, 52% (199/383) were female, and 81.2% (311/383) were White, suggesting that the study sample was broadly representative of the general UK population [59]. All participants were employed (part-time: 94/383, 24.5%; self-employed: 49/383, 12.8%) across a broad range of industries and most had not used an MHapp (306/383, 79.9%) or not engaged in talking therapy (352/383, 91.9%) in a 6-month period before taking part in the study. Almost all participants (372/383, 97.1%) either agreed or strongly agreed with the statement, “It’s important for people to look after their mental health and wellbeing.”

Patient-reported outcome scores suggested that participants were, on average, experiencing mild symptoms of depression and anxiety at t0 (mean PHQ-8 6.9, SD 5.2; mean GAD-7 6.5, SD 5.1). The proportion of participants scoring above the cutoff for mild symptoms was 59.3% (227/383) for anxiety (GAD-7≥5) and 59.8% (229/383) for depression (PHQ-8≥5), whereas the proportion scoring above the cutoff for moderate symptoms was 26.9% (101/383) for anxiety (GAD-7≥10) and 28.2% (108/383) for depression (PHQ-8≥10). Self-reported stress levels were approximately consistent with population norms (mean PSS 17.0, SD 7.7 [55]).

Intervention research can be undermined by problems with intervention delivery, acceptability, participant retention, and smaller than anticipated effect sizes. Therefore, guidelines suggest conducting pilot studies to test trial feasibility and estimate important trial parameters before running a definitive trial [61]. This study reports on the feasibility and preliminary efficacy of 3 interventions featured on the Unmind MHapp when delivered to working adults in the United Kingdom. The study methods and interventions were found to be feasible, and all preregistered progression criteria were met. This suggests that a definitive trial is warranted, although several minor proposed protocol amendments are discussed.

Participant retention and intervention adherence were largely consistent with or higher than those in comparable studies. For instance, only 7% (27/383) of participants were lost to attrition at follow-up, which compares favorably with recent meta-analyses reporting average attrition rates of 23% to 48% for MHapp trials [11,16,62]. This may be because of the use of the Prolific recruitment platform, which is associated with high participant response rates [35], and reimbursing participants at each study assessment. It may also suggest that participants, on average, perceived the Unmind platform as helpful and engaging and were thus motivated to complete the study.

Objective engagement data suggested that 67.8% (196/289) of randomized participants (and 196/237, 82.7% of those starting their intervention) completed all intervention sessions, which is similar to or higher than average completion rates ranging between 30% and 65% for other MHapp interventions [11,16,53,63]. These engagement rates are particularly encouraging, considering that participants were not recruited on the basis of seeking help for a mental health problem and were not randomized to an intervention based on scoring poorly on the target outcome at baseline.

Despite these high levels of engagement, the study interventions were brief, comprising approximately 60 to 80 minutes of learning over 7 sessions. In addition, the study used a nonclinical sample, and regression analyses suggested that participants with higher symptom severity at baseline were less likely to complete their allocated intervention. Thus, these findings may not be generalizable to other interventions featured on the Unmind app or to other study populations. Future studies designed to evaluate the use of the Unmind app in subclinical populations are currently being planned. Interestingly, although participants with higher baseline symptoms were less likely to complete their intervention, feedback ratings at postintervention were largely equivalent across the study sample. For example, 87.1% (203/233) of all participants reported being either satisfied or very satisfied with their intervention and rated the quality of their intervention as either good or excellent, and these ratings did not differ for participants with more severe symptoms at baseline.

Although all 3 interventions met progression criteria, participants randomized to the building resilience arm were marginally less likely to start their allocated intervention and complete all intervention sessions after starting. A potential explanation is that participants may have felt less motivated by the theme of resilience as compared with stress and anxiety. In addition, participants in the building resilience arm reported marginally worse mental health scores at baseline, which may have negatively affected engagement. Although participant feedback was largely equivalent across the 3 interventions, there were several marginal (not statistically significant) differences that may be a contributing factor. For example, compared with combatting stress, participants in the building resilience arm were slightly less likely to rate the intervention as good relative to okay or poor. Although it is difficult to draw firm conclusions from these data, it will be important to test whether such differences persist in a definitive trial and the extent to which any changes or improvements to the building resilience intervention are warranted.

The findings from this pilot study revealed several opportunities for minor protocol improvements before running a definitive trial. First, of the 383 participants, 11 (2.9%) reported not fully understanding the study instructions, and 1.7% (5/289) of participants randomized to an intervention engaged with the wrong intervention. This could be addressed by providing participants with detailed video instructions and implementing a brief quiz to ensure that all participants understand how to access their allocated intervention. If feasible, participants could be given access to a modified, study-specific version of the Unmind platform that only includes the interventions being tested. Second, discrepancies between self-reported and objective in-app engagement meant that 48% (45/93) of participants who stopped using the Unmind app did not provide feedback on their reasons for discontinuing their intervention. This could be addressed by restructuring the feedback questionnaire to capture data from all participants, regardless of self-reported engagement. Third, 0.9% (2/233) of participants reported experiencing negative effects while completing their intervention; however, qualitative feedback suggested that both instances referred to frustration with the intervention (not understanding the content or not finding it useful). This question could be modified to capture lasting bad effects (refer to the study by Crawford et al [64]) with more specific clarifying questions to better differentiate harmful effects from frustration and/or lack of intervention enjoyment. Finally, 4% (16/400) of participants were excluded from the analysis as they reported being unemployed at baseline, which was in contrast to their prescreening responses. Although the Prolific platform precludes the use of additional screening questions at baseline, the sample size calculation for a definitive trial could be adjusted to account for this potential discrepancy.

Although the study was not powered for formal hypothesis testing, preliminary efficacy findings suggested that the study interventions were associated with small to moderate between-group improvements in ≥1 mental health outcomes, which were maintained at the 1-month follow-up. This is consistent with findings from meta-analyses of previous MHapp trials [11,12,26]. All 3 interventions resulted in larger improvements in symptoms of depression, anxiety, perceived stress, and resilience than the control group, although smaller effects (Hedges g<0.3) did not reach statistical significance. An appropriately powered definitive trial may be more likely to capture such small effects with greater precision and, where possible, should aim to report on the clinical importance of these findings (eg, with regard to minimal important difference thresholds). These efficacy findings were consistent across both PP and ITT analyses and are particularly promising, given the brevity of the interventions. In addition, the findings were robust across subgroup analyses that only included participants with at least mild problems at baseline. However, 2 further patterns emerged that merit discussion.

First, relative to baseline, participants in the control group reported statistically significant improvements in stress and anxiety at both study time points and improvements in resilience at the 1-month follow-up, despite not having access to any study interventions. Although this may reflect phenomena such as practice effects or regression to the mean, it is worth noting that baseline data were collected several weeks after the commencement of a third national UK lockdown (in response to rising cases of SARS-CoV-2), whereas follow-up data were collected after the start of a phased easing of restrictions. A longitudinal survey conducted in England suggests that symptoms of anxiety and depression tend to rise rapidly during the early stages of a lockdown but decline quickly thereafter [65], which may partly explain the changes in mental health scores reported by the control group. Thus, the present efficacy findings may not be directly generalizable to other contexts. Importantly, the intervention groups reported larger improvements in mental health despite significant changes in the control group, and MHapps such as Unmind may be an effective way of delivering accessible mental health support to workforces working remotely or during times of national crisis.

Second, although all 3 interventions resulted in significant improvements for at least one mental health outcome, the study interventions did not appear to be sensitive or specific to their target outcome. For example, the combatting stress arm resulted in small between-group (intervention vs control) improvements in perceived stress (not statistically significant), as well as significant reductions in symptoms of depression. Similarly, participants in the working with worry arm reported significant improvements across all outcomes, despite the intervention specifically targeting anxiety. A potential explanation for this is that different facets of mental health tend to strongly covary with one another [66], and transdiagnostic research suggests that symptoms of different mental health problems often respond to the same treatments [67]. The present findings lend credence to this, as the study interventions all primarily involve similar CBT-based techniques (identifying and challenging negative thinking patterns, problem solving, and breaking negative cycles of thinking or behaviors). As this study was not powered to detect differences between intervention arms (or to test whether effect sizes for some outcomes were significantly different from others), it is not possible to draw meaningful conclusions from these data. Future studies should aim to test for any such differences so that users of the Unmind app can access interventions that are most likely to benefit their individual problem areas.

This study had several strengths. First, intervention adherence and engagement were objectively captured via Unmind (the intervention platform). This is important, given the recent evidence of substantial discrepancies between self-reported and objective adherence in digital interventions [68]. Second, participant retention was very high, which is extremely important as missing data can reduce statistical power and lead to biased intervention effects. Second, all study outcomes and analyses were prospectively registered, precluding any selective reporting of outcomes or nonpublication of findings [69]. Third, the study recruited a sample representative of the general UK population with respect to age, sex, and ethnicity, and participants were employed across a variety of industries. This is important as Unmind is designed for use across a broad range of demographics, and a lack of diversity within study samples can limit the generalizability of the study findings.

A limitation of this study is the use of a passive no-intervention control group, as opposed to an active control group in which participants engage with activities matched for duration, attention, and interest. Passive controls do not allow true intervention effects to be differentiated from nonspecific placebo effects and may introduce nocebo effects [31]. Although it is useful in practical terms to estimate this combined effect of the Unmind app, active controls are needed to fully understand the mechanisms underlying its effects. In addition, despite being randomly assigned, participants in the control group had slightly higher levels of self-reported resilience at baseline, and it is unknown whether this may have affected between-group differences in resilience scores over time. In addition, as with most web-based trials, participants were not blinded to group allocation. Finally, although the Prolific recruitment platform has several strengths, the participant pool was entirely self-selected, and it remains unknown to what extent the present findings are generalizable to working adults nationwide.

This study assessed the feasibility of conducting a future definitive RCT to evaluate the efficacy of 3 brief interventions featured on the Unmind MHapp. The study methods and interventions were found to be feasible, and all preregistered criteria for progression to a definitive trial were met. Several minor protocol amendments have been suggested. Preliminary efficacy findings indicate that the study interventions may result in improved mental health outcomes when offered to working adults.