This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.
There is growing interest in digital platforms as a means of implementing scalable, accessible, and cost-effective mental health interventions in the workplace. However, little is known about the efficacy of such interventions when delivered to employee groups.
This study aims to evaluate the feasibility and preliminary efficacy of a digital mental health platform for the workplace, which incorporates evidence-based practices such as cognitive behavioral therapy and acceptance and commitment therapy. A total of 3 brief, unguided interventions designed to address stress, anxiety, and resilience, respectively, are evaluated. The primary aim is to determine the feasibility of the study methods and interventions in preparation for a definitive randomized controlled trial.
The study used a fully remote, parallel, multi-arm, external pilot randomized controlled trial, with 3 intervention arms and a no-intervention control group. Participants were working adults representative of the general UK population with respect to age, sex, and ethnicity who were recruited from a web-based participant platform. Primary outcomes included objective and self-report measures of feasibility, acceptability, engagement, transferability, relevance, and negative effects. Secondary outcomes included 4 self-report measures of mental health and well-being, completed at baseline (time point 0 [t0]), postintervention (time point 1 [t1]), and the 1-month follow-up (time point 2 [t2]). Secondary outcomes were analyzed via linear mixed-effects models using intention-to-treat principles. Preregistered criteria for progression to a definitive trial were evaluated.
Data were collected between January and March of 2021. A total of 383 working adult participants meeting trial eligibility were randomized, of whom 356 (93%) were retained at t2. Objective engagement data showed that 67.8% (196/289) of participants randomized to an intervention arm completed their intervention. Overall, 87.1% (203/233) of participants reported being
The study methods were feasible, and all preregistered criteria for progression to a definitive trial were met. Several minor protocol amendments were noted. Preliminary efficacy findings suggest that the study interventions may result in improved mental health outcomes when offered to working adults.
ISRCTN Registry 80309011; http://www.isrctn.com/ISRCTN80309011
Mental illness affects hundreds of millions of people worldwide, resulting in decreased quality of life, family and community disruption, increased health care costs, and a significant economic burden for employers [
There is growing interest in web and smartphone apps as a means of increasing the reach of mental health and well-being interventions [
There is now convincing evidence for the effectiveness of digital interventions when delivered in health and community settings [
In this study, we conduct an external pilot randomized controlled trial (RCT) as part of the initial testing of
Consistent with recent guidelines on pilot trials [
As the
The feasibility of each intervention arm is assessed via objective and self-reported outcomes capturing recruitment, retention, intervention uptake and adherence, acceptability, transferability, relevance, and negative effects. The preliminary efficacy of each intervention arm is assessed via self-report outcome measures delivered before (time point 0 [t0]) and after the interventions (time point 1 [t1]; 2 weeks after t0) and at the 1-month follow-up (time point 2 [t2]). The results of this study are intended to inform whether a definitive RCT to evaluate the efficacy of each intervention arm is warranted and provide estimates of the parameters required for its design and implementation.
The authors followed the CONSORT (Consolidated Standards of Reporting Trials) 2010 guidelines [
This study was a parallel, multi-arm, external pilot RCT with pre- (t0) and postintervention (t1; 2 weeks after t0) assessments and a 1-month follow-up (t2). Participants were randomly allocated to 1 of 3 brief, self-guided psychological interventions (
The trial received ethical approval from the University of Sussex sciences and technology research ethics committee (ER/KC226/2).
Participants were recruited via the Prolific web-based recruitment platform, which has been empirically tested across attributes such as participant response rates and data quality [
Prolific implements a prescreening system that allows researchers to screen for eligibility without implementing a screening questionnaire. Prolific also supports the recruitment of study samples representative of the national UK population with respect to age, sex, and ethnicity based on guidelines from the UK Office of National Statistics. This study drew upon this feature to maximize the generalizability of the findings.
All study assessments were hosted on the Qualtrics survey platform [
Randomization occurred at the end of t0 and was implemented via the Qualtrics
Although
Screenshots of the
The intervention arms in this study comprised 3 individual
This intervention draws upon CBT and ACT techniques and is designed to help users better manage their day-to-day stressors. Over the course of 7 sessions, it provides psychoeducation on stress and its physical manifestations, helps users spot personal triggers, and explores different approaches to coping. It also introduces the idea of acceptance. Users are taught stress management techniques and are encouraged to practice between sessions.
This intervention is underpinned by theoretical models of generalized anxiety disorder (GAD), although it is targeted at users who identify as worriers rather than those meeting any predefined criteria for a diagnosis of GAD. Content spans 7 sessions and covers key elements of CBT, including tolerance of uncertainty, challenging worry beliefs, problem solving, and working with imagery. It also encourages users to apply evidence-based techniques, including relaxation and attentional focus.
This intervention aims to help users apply evidence-based techniques to aid the cultivation of essential qualities of personal resilience, drawing upon CBT and ACT. Over the course of 7 sessions, learning covers topics such as honing strengths, facing challenges, and tolerating discomfort. It also explores aspects such as coping styles and realistic optimism. The
In each
For the purposes of this study, participants were instructed not to engage with other content and features included in the
Participant demographics and other variables were captured at t0, including whether each participant had engaged with therapy or counseling, and a mental health or well-being app within 6 months before taking part in the study. Participants were also asked to rate the extent to which they agreed with the statement, “Do you agree that it’s important for people to look after their mental health and wellbeing?” on a 5-point Likert scale from
Recent guidelines suggest that complex health interventions should be feasible, acceptable, engaging, transferable to other settings, and relevant [
Feasibility: recruitment, intervention uptake, and retention (at t1 and t2)
Acceptability: intervention adherence and completion rates, participant satisfaction, and reasons for discontinuing the intervention
Engagement: average sessions completed and 3 questions adapted from the Mobile App Rating Scale [
Transferability: 1 question adapted from the Mobile App Rating Scale
Relevance: 1 question assessing subjective relevance
Negative effects: 1 question adapted from recent guidelines on assessing negative effects [
Outcomes were measured through a combination of objective data (captured by the
Preregistered secondary outcomes included self-report measures capturing symptoms of common mental health problems.
The Perceived Stress Scale (PSS) is a 10-item scale that asks respondents to rate how often they feel or think that their lives are unpredictable, uncontrollable, and overloaded on a 5-point Likert scale from 0 (never) to 4 (very often) [
The GAD-7 is a 7-item scale used to screen for the presence and severity of an anxiety disorder [
The Patient Health Questionnaire (PHQ)-8 is an 8-item scale derived from the PHQ-9, which screens for the presence and severity of depression [
The Brief Resilience Scale (BRS) is a short, 6-item scale designed to assess people’s ability to bounce back or recover after stressful events [
As per formal guidelines [
Progression criteria were considered for each intervention arm individually.
This study was powered for CIs on the feasibility outcomes. A sample size calculation indicated that approximately 100 participants were required to estimate feasibility outcomes with a margin of error ≤10% (based on a conservative population proportion of 50% for retention and adherence, and a 95% CI). This is consistent with previous guidelines suggesting that 60 to 100 participants per intervention arm are optimal for estimating binary outcomes in pilot RCTs [
The results from all preregistered primary and secondary measures are reported. Minor deviations from the preregistered data analysis plan are reported in
Descriptive statistics were used to report primary outcomes. Categorical data were reported as proportions in each response category, and Fisher exact test of independence was used to compare responses between intervention arms (with
Objective in-app usage data were provided by
We computed the proportion of participants who self-reported reliable deterioration in mental health scores from t0 to t1, and t1 to t2, based on an estimate of the reliable change index for each outcome measure. The reliable change index was computed based on methods provided by Jacobson and Truax [
Secondary outcome measures were analyzed using both intention-to-treat (ITT) and per-protocol (PP) approaches. For the ITT analysis, all participants with complete t0 data were included, regardless of intervention adherence and any deviation from instructions. Participants were excluded from the PP analysis if they failed to complete all 7 intervention sessions, if they started an
Analyses were performed using linear mixed-effects models (LMMs) with restricted information maximum likelihood estimation (via the
Subgroup analyses were performed to examine changes in secondary outcome measures for participants who self-reported having at least mild symptoms at t0 or at least moderately low resilience. Thresholds for subgroup analyses were as follows: a score ≥16 on the PSS, ≥5 on the GAD-7, ≥5 on the PHQ-8, and <3 on the BRS. For simplicity, we report a comparison of Hedges
Finally, multivariate logistic regression was conducted on the intervention group data only to explore whether any baseline variables were predictive of intervention completion (defined as
Participant demographics and other baseline variables are presented in
Participant demographics and baseline variables (N=383).
Variable | Overall | Study arm | ||||
|
|
Control (n=94) | CSa (n=94) | WWb (n=97) | BRc (n=98) | |
Age (years), mean (SD; range) | 44.6 (14.3; 18-75) | 45.6 (14.2; 18-69) | 44.8 (14.3; 19-75) | 43.6 (14.7; 19-72) | 44.7 (14.3; 18-69) | |
|
||||||
|
Female | 199 (52) | 53 (56.4) | 41 (43.6) | 55 (56.7) | 50 (51) |
|
Male | 184 (48) | 41 (43.6) | 53 (56.4) | 42 (43.3) | 48 (49) |
|
||||||
|
Whited | 311 (81.2) | 76 (80.9) | 79 (84) | 74 (76.3) | 82 (83.7) |
|
Blacke | 19 (5) | 4 (4.3) | 2 (2.1) | 7 (7.2) | 6 (6.1) |
|
Mixed or multiple | 12 (3.1) | 2 (2.1) | 3 (3.2) | 5 (5.2) | 2 (2) |
|
Asianf | 35 (9.1) | 11 (11.7) | 9 (9.6) | 9 (9.3) | 6 (6.1) |
|
Otherg | 6 (1.6) | 1 (1.1) | 1 (1.1) | 2 (2.1) | 2 (2) |
|
||||||
|
Full-time | 238 (62.1) | 60 (63.8) | 63 (67) | 55 (56.7) | 60 (61.2) |
|
Part-time | 94 (24.5) | 26 (27.7) | 18 (19.1) | 28 (28.9) | 22 (22.4) |
|
Self-employed | 49 (12.8) | 8 (8.5) | 13 (13.8) | 13 (13.4) | 15 (15.3) |
|
Other | 2 (0.5) | 0 (0) | 0 (0) | 1 (1) | 1 (1) |
|
||||||
|
Agriculture, forestry, or mining | 1 (0.3) | 1 (1.1) | 0 (0) | 0 (0) | 0 (0) |
|
Industrials | 22 (5.7) | 4 (4.3) | 9 (9.6) | 3 (3.1) | 6 (6.1) |
|
Energy or utilities | 7 (1.8) | 2 (2.1) | 1 (1.1) | 2 (2.1) | 2 (2) |
|
Transport or logistics | 17 (4.4) | 2 (2.1) | 5 (5.3) | 6 (6.2) | 4 (4.1) |
|
Media or creative industries | 24 (6.3) | 6 (6.4) | 9 (9.6) | 4 (4.1) | 5 (5.1) |
|
Data or telecommunication | 18 (4.7) | 5 (5.3) | 5 (5.3) | 2 (2.1) | 6 (6.1) |
|
Health care | 40 (10.4) | 13 (13.8) | 6 (6.4) | 12 (12.4) | 9 (9.2) |
|
Education | 63 (16.4) | 17 (18.1) | 17 (18.1) | 12 (12.4) | 17 (17.3) |
|
Life sciences | 4 (1) | 0 (0) | 1 (1.1) | 2 (2.1) | 1 (1) |
|
Retail | 32 (8.4) | 2 (2.1) | 11 (11.7) | 8 (8.2) | 11 (11.2) |
|
Hospitality, leisure, or travel | 22 (5.7) | 8 (8.5) | 7 (7.4) | 3 (3.1) | 4 (4.1) |
|
Public or social service | 30 (7.8) | 8 (8.5) | 6 (6.4) | 7 (7.2) | 9 (9.2) |
|
Finances, insurance, or real estate | 22 (5.7) | 5 (5.3) | 4 (4.3) | 8 (8.2) | 5 (5.1) |
|
Professional services | 25 (6.5) | 7 (7.4) | 4 (4.3) | 8 (8.2) | 6 (6.1) |
|
Other | 56 (14.6) | 14 (14.9) | 9 (9.6) | 20 (20.6) | 13 (13.3) |
|
||||||
|
None | 4 (1) | 0 (0) | 2 (2.1) | 1 (1) | 1 (1) |
|
High school | 138 (36) | 30 (31.9) | 33 (35.1) | 36 (37.1) | 39 (39.8) |
|
Undergraduate degree | 172 (44.9) | 46 (48.9) | 41 (43.6) | 42 (43.3) | 43 (43.9) |
|
Postgraduate degree | 69 (18) | 18 (19.1) | 18 (19.1) | 18 (18.6) | 15 (15.3) |
|
||||||
|
Yes | 74 (19.3) | 15 (16.0) | 17 (18.1) | 23 (23.7) | 19 (19.4) |
|
No | 306 (79.9) | 78 (83.0) | 76 (80.9) | 73 (75.3) | 79 (80.6) |
|
Maybe | 3 (0.8) | 1 (1.1) | 1 (1.1) | 1 (1) | 0 (0) |
|
||||||
|
Yes | 31 (8.1) | 6 (6.4) | 3 (3.2) | 12 (12.4) | 10 (10.2) |
|
No | 352 (91.9) | 88 (93.6) | 91 (96.8) | 85 (87.6) | 88 (89.8) |
|
||||||
|
Strongly disagree | 6 (1.6) | 0 (0) | 2 (2.1) | 1 (1) | 3 (3.1) |
|
Disagree | 1 (0.3) | 1 (1.1) | 0 (0) | 0 (0) | 0 (0) |
|
Neither | 4 (1) | 0 (0) | 1 (1.1) | 1 (1) | 2 (2) |
|
Agree | 75 (19.6) | 19 (20.2) | 25 (26.6) | 15 (15.5) | 16 (16.3) |
|
Strongly agree | 297 (77.5) | 74 (78.7) | 66 (70.2) | 80 (82.5) | 77 (78.6) |
aCS: combatting stress.
bWW: working with worry.
cBR: building resilience.
dWhite British and other British.
eAfrican, Caribbean, and Black British.
fChinese, Indian, Bangladeshi, Pakistani, and other Asian.
gArabian or any other ethnicity.
hMHapp: mental health app.
iMH: mental health.
Patient-reported outcome scores suggested that participants were, on average, experiencing mild symptoms of depression and anxiety at t0 (mean PHQ-8 6.9, SD 5.2; mean GAD-7 6.5, SD 5.1). The proportion of participants scoring above the cutoff for mild symptoms was 59.3% (227/383) for anxiety (GAD-7≥5) and 59.8% (229/383) for depression (PHQ-8≥5), whereas the proportion scoring above the cutoff for moderate symptoms was 26.9% (101/383) for anxiety (GAD-7≥10) and 28.2% (108/383) for depression (PHQ-8≥10). Self-reported stress levels were approximately consistent with population norms (mean PSS 17.0, SD 7.7 [
The study was enrolled in January 2021 within 48 hours of launching the study advert.
CONSORT (Consolidated Standards of Reporting Trials) flow of participants through the study. t0: time point 0; t1: time point 1; t2: time point 2.
A summary of the intervention engagement is shown in
Overall, participants who objectively completed at least one intervention session self-reported spending a mean of 60.1 (median 60, SD 29.1, minimum 10, maximum 210) minutes on the
Intervention adherence and engagement across the 3 intervention arms (N=289).
Variable | Overall | Study arm | |||
|
|
CSa (n=94) | WWb (n=97) | BRc (n=98) | |
|
|||||
|
Value, n | 196 | 74 | 63 | 59 |
|
Percentage of those randomized (95% CI) | 67.8 (62.1-73.2) | 78.7 (69.1-86.5) | 64.9 (54.6-74.4) | 60.2 (49.8-70) |
|
Percentage of those starting intervention (95% CI) | 82.7 (77.3-87.3) | 91.4 (83.0-96.5) | 80.8 (70.3-88.8) | 75.6 (64.6-84.7) |
Days taken to complete intervention, mean (SD; range) | 6.38 (4.10; 0-15) | 6.59 (3.94; 0-15) | 6.86 (3.85; 0-14) | 5.61 (4.51; 0-14) | |
|
|||||
|
Value, n | 41 | 7 | 15 | 19 |
|
Percentage of those randomized (95% CI) | 14.2 (10.4-18.8) | 7.45 (3.1-14.7) | 15.5 (8.9-24.2) | 19.4 (12.1-28.6) |
|
|||||
|
Values, mean (SD) | 3.24 (1.96) | 2.71 (1.80) | 2.80 (1.47) | 3.79 (2.27) |
|
Values, median (range) | 3 (1-6) | 3 (1-5) | 3 (1-5) | 4 (1-6) |
|
|||||
|
Value, n | 47 | 11 | 16 | 20 |
|
Percentage of those randomized (95% CI) | 16.3 (12.2-21) | 11.7 (6-20) | 16.5 (9.7-25.4) | 20.4 (12.9-29.7) |
|
|||||
|
Value, n | 5 | 2 | 3 | 0 |
|
Percentage of those randomized (95% CI) | 1.7 (0.6-4) | 2.1 (0.3-7.5) | 3.1 (0.6-8.8) | 0 (0-3.7) |
|
|||||
|
Value, n | 12 | 4 | 5 | 3 |
|
Percentage of those randomized (95% CI) | 4.2 (2.2-7.1) | 4.3 (1.2-10.5) | 5.2 (1.7-11.6) | 3.1 (0.6-8.7) |
aCS: combatting stress.
bWW: working with worry.
cBR: building resilience.
Feedback ratings from participants scoring above predefined cutoffs for inclusion in subgroup analyses were largely equivalent to the overall sample and are included in
Postintervention feedback ratings from participants who were retained at time point 1 (t1), both overall and for each intervention arm (N=233).
Feedback ratings | Overall n (%) | Study arm n (%) | ||||||
|
|
CSb (n=79) | WWc (n=76) | BRd (n=78) |
|
|||
|
.16 | |||||||
|
Dull, not fun | 1 (0.4) | 0 (0) | 0 (0) | 1 (1.3) |
|
||
|
Mostly boring | 14 (6) | 1 (1.3) | 7 (9.2) | 6 (7.7) |
|
||
|
OK, fun enough | 55 (23.6) | 17 (21.5) | 15 (19.7) | 23 (29.5) |
|
||
|
Moderately interesting and fun | 103 (44.2) | 41 (51.9) | 32 (42.1) | 30 (38.5) |
|
||
|
Highly interesting and fun | 60 (25.8) | 20 (25.3) | 22 (28.9) | 18 (23.1) |
|
||
|
.46 | |||||||
|
Dull, not fun | 3 (1.3) | 0 (0) | 1 (1.3) | 2 (2.6) |
|
||
|
Mostly boring | 11 (4.7) | 2 (2.5) | 5 (6.6) | 4 (5.1) |
|
||
|
OK, fun enough | 46 (19.7) | 13 (16.5) | 13 (17.1) | 20 (25.6) |
|
||
|
Moderately interesting and fun | 94 (40.3) | 35 (44.3) | 28 (36.8) | 31 (39.7) |
|
||
|
Highly interesting and fun | 79 (33.9) | 29 (36.7) | 29 (38.2) | 21 (26.9) |
|
||
|
.59 | |||||||
|
Strongly disagree | 7 (3) | 1 (1.3) | 3 (3.9) | 3 (3.8) |
|
||
|
Disagree | 16 (6.9) | 3 (3.8) | 7 (9.2) | 6 (7.7) |
|
||
|
Neither agree nor disagree | 40 (17.2) | 14 (17.7) | 9 (11.8) | 17 (21.8) |
|
||
|
Agree | 97 (41.6) | 36 (45.6) | 33 (43.4) | 28 (35.9) |
|
||
|
Strongly agree | 73 (31.3) | 25 (31.6) | 24 (31.6) | 24 (30.8) |
|
||
|
.71 | |||||||
|
Very dissatisfied | 3 (1.3) | 0 (0) | 1 (1.3) | 2 (2.6) |
|
||
|
Dissatisfied | 8 (3.4) | 2 (2.5) | 3 (3.9) | 3 (3.8) |
|
||
|
Neither satisfied nor dissatisfied | 19 (8.2) | 4 (5.1) | 7 (9.2) | 8 (10.3) |
|
||
|
Satisfied | 99 (42.5) | 36 (45.6) | 28 (36.8) | 35 (44.9) |
|
||
|
Very satisfied | 104 (44.6) | 37 (46.8) | 37 (48.7) | 30 (38.5) |
|
||
|
.02 | |||||||
|
Poor | 5 (2.1) | 0 (0) | 3 (3.9) | 2 (2.6) |
|
||
|
Okay | 25 (10.7) | 4 (5.1) | 9 (11.8) | 12 (15.4) |
|
||
|
Good | 96 (41.2) | 42 (53.2) | 23 (30.3) | 31 (39.7) |
|
||
|
Excellent | 107 (45.9) | 33 (41.8) | 41 (53.9) | 33 (42.3) |
|
||
|
.13 | |||||||
|
I would not recommend it to anyone | 7 (3) | 1 (1.3) | 2 (2.6) | 4 (5.1) |
|
||
|
There are very few people I would recommend it to | 29 (12.4) | 5 (6.3) | 11 (14.5) | 13 (16.7) |
|
||
|
There are several people whom I would recommend it to | 65 (27.9) | 28 (35.4) | 17 (22.4) | 20 (25.6) |
|
||
|
There are many people I would recommend it to | 74 (31.8) | 20 (25.3) | 29 (38.2) | 25 (32.1) |
|
||
|
Definitely—I would recommend it to everyone | 58 (24.9) | 25 (31.6) | 17 (22.4) | 16 (20.5) |
|
||
|
.96 | |||||||
|
No (limited instructions, confusing, and complicated) | 1 (0.4) | 0 (0) | 0 (0) | 1 (1.3) |
|
||
|
Useable after a lot of time and effort | 1 (0.4) | 0 (0) | 1 (1.3) | 0 (0) |
|
||
|
Usable after some time and effort | 12 (5.2) | 4 (5.1) | 4 (5.3) | 4 (5.1) |
|
||
|
Easy to learn how to use | 75 (32.2) | 26 (32.9) | 22 (28.9) | 27 (34.6) |
|
||
|
Able to use app immediately | 144 (61.8) | 49 (62) | 49 (64.5) | 46 (59) |
|
||
|
.11 | |||||||
|
Yes | 2 (0.9) | 0 (0) | 2 (2.6) | 0 (0) |
|
||
|
No | 231 (99.1) | 79 (100) | 74 (97.4) | 78 (100) |
|
aFisher exact tests comparing ratings across study arms.
bCS: combatting stress.
cWW: working with worry.
dBR: building resilience.
Of the 76 participants in the
Number and percentage of participants per study arm (and overall) that reliably deteriorated from time point 0 (t0) to time point 1 (t1) and t1 to time point 2 (t2) based on reliable change index for each secondary outcome measure.
Outcome | Overall, n (%) | Study arm, n (%) | |||||||||
|
|
Control | CSa | WWb | BRc | ||||||
|
|||||||||||
|
t1 to t0 (n=365) | 15 (4.1) | 4 (4.3) | 8 (8.9) | 1 (1.1) | 2 (2.2) | |||||
|
t2 to t1 (n=349) | 35 (10) | 9 (9.9) | 6 (7.1) | 11 (12.4) | 9 (10.7) | |||||
|
|||||||||||
|
t1 to t0 (n=353) | 18 (5.1) | 4 (4.4) | 7 (8.0) | 1 (1.1) | 6 (6.9) | |||||
|
t2 to t1 (n=340) | 27 (7.9) | 9 (10.2) | 3 (3.6) | 7 (8.4) | 8 (9.8) | |||||
|
|||||||||||
|
t1 to t0 (n=359) | 17 (4.7) | 7 (7.5) | 6 (6.7) | 1 (1.1) | 3 (3.4) | |||||
|
t2 to t1 (n=344) | 13 (3.8) | 3 (3.4) | 2 (2.4) | 4 (4.5) | 4 (4.8) | |||||
|
|||||||||||
|
t1 to t0 (n=356) | 14 (3.9) | 4 (4.4) | 3 (3.4) | 4 (4.4) | 3 (3.5) | |||||
|
t2 to t1 (n=347) | 22 (6.3) | 5 (5.4) | 6 (7.1) | 7 (7.9) | 4 (4.9) |
aCS: combatting stress.
bWW: working with worry.
cBR: building resilience.
dPSS: Perceived Stress Scale.
eGAD-7: Generalized Anxiety Disorder-7.
fPHQ-8: Patient Health Questionnaire-8.
gBRS: Brief Resilience Scale.
The following are based on ITT analyses that include data from all randomized participants (except for those not meeting the eligibility criteria; see
Estimated marginal means (EMMs) from linear mixed-effects models at each of the 3 study time points shown for each intervention arm and each secondary outcome measure.
Outcome | Time point | |||
|
Baseline (t0) | Postintervention (t1) | Follow-up (t2) | |
|
EMM (SE; 95% CI) | EMM (SE; 95% CI) | EMM (SE; 95% CI) | |
|
||||
|
Control | 16.86 (0.76; 15.37-18.35) | 15.19b (0.76; 13.69-16.69) | 14.62c (0.76; 13.12-16.12) |
|
CSd | 16.35 (0.76; 14.86-17.84) | 13.30c (0.77; 11.79-14.80) | 12.59c (0.77; 11.08-14.10) |
|
WWe | 17.14 (0.75; 15.68-18.61) | 12.93c (0.76; 11.44-14.42) | 12.73c (0.76; 11.24-14.22) |
|
BRf | 17.76 (0.74; 16.29-19.22) | 13.49c (0.76; 12.01-14.98) | 13.17c (0.77; 11.66-14.68) |
|
||||
|
Control | 6.38 (0.50; 5.41-7.36) | 5.26b (0.50; 4.28-6.24) | 5.42h (0.50; 4.44-6.40) |
|
CS | 6.20 (0.50; 5.23-7.18) | 4.53c (0.50; 3.54-5.52) | 4.01c (0.51; 3.02-5.00) |
|
WW | 6.57 (0.49; 5.61-7.53) | 4.12c (0.50; 3.15-5.10) | 4.12c (0.50; 3.15-5.10) |
|
BR | 7.02 (0.49; 6.06-7.98) | 4.69c (0.50; 3.72-5.66) | 4.42c (0.51; 3.43-5.41) |
|
||||
|
Control | 6.62 (0.51; 5.61-7.63) | 6.38j (0.52; 5.37-7.39) | 6.06j (0.52; 5.04-7.08) |
|
CS | 6.68 (0.51; 5.67-7.69) | 4.94c (0.52; 3.92-5.96) | 4.56c (0.52; 3.53-5.59) |
|
WW | 6.85 (0.51; 5.85-7.84) | 4.36c (0.51; 3.35-5.36) | 4.56c (0.51; 3.55-5.57) |
|
BR | 7.34 (0.50; 6.35-8.33) | 5.53c (0.51; 4.53-6.54) | 5.33c (0.52; 4.31-6.36) |
|
||||
|
Control | 20.37 (0.52; 19.36-21.38) | 20.90j (0.52; 19.89-21.92) | 21.18h (0.52; 20.17-22.20) |
|
CS | 19.63 (0.52; 18.62-20.64) | 21.03c (0.52; 20.01-22.05) | 21.30c (0.52; 20.27-22.33) |
|
WW | 19.00 (0.51; 18.00-20.00) | 20.70c (0.51; 19.70-21.71) | 20.85c (0.51; 19.84-21.86) |
|
BR | 19.05 (0.50; 18.06-20.04) | 20.52c (0.51; 19.51-21.52) | 21.29c (0.52; 20.27-22.32) |
aPSS: Perceived Stress Scale.
b
c
dCS: combatting stress.
eWW: working with worry.
fBR: building resilience.
gGAD-7: Generalized Anxiety Disorder-7.
h
iPHQ-8: Patient Health Questionnaire-8.
j
kBRS: Brief Resilience Scale.
Participants in the
Contrasts and between-group (intervention versus control) effect size calculations from linear mixed-effects models (LMMs) applied to each secondary outcome for both intention-to-treat (ITT) and subgroup analyses.
Outcome | Estimatea (SE; 95% CI) | Hedges |
||||||||||
|
|
LMMb | Tukey adjustedc | ITT | Subgroup | |||||||
|
||||||||||||
|
|
|||||||||||
|
|
CSh | 1.38 (0.83; −0.57 to 3.33) | .10 | .34 | 0.24 (−0.05 to 0.53) | 0.29 (−0.08 to 0.66) | |||||
|
|
WWi | 2.54 (0.82; 0.93 to 4.16) | .002 | .01 | 0.45 (0.16 to 0.74) | 0.66 (0.28 to 1.04) | |||||
|
|
BRj | 2.59 (0.82; 0.97 to 4.21) | .002 | .009 | 0.45 (0.17 to 0.74) | 0.47 (0.10 to 0.84) | |||||
|
|
Allk | 2.18 (0.67; 0.86 to 3.50) | .001 | .004 | 0.38 (0.18 to 0.58) | 0.47 (0.21 to 0.73) | |||||
|
|
|||||||||||
|
|
CS | 1.52 (0.83; −0.11 to 3.16) | .07 | .26 | 0.27 (−0.02 to 0.56) | 0.41 (0.04 to 0.78) | |||||
|
|
WW | 2.18 (0.83; 0.55 to 3.80) | .009 | .04 | 0.38 (0.09 to 0.67) | 0.64 (0.26 to 1.02) | |||||
|
|
BR | 2.34 (0.84; 0.70 to 3.98) | .005 | .03 | 0.40 (0.12 to 0.69) | 0.53 (0.16 to 0.90) | |||||
|
|
All | 2.02 (0.68; 0.69 to 3.34) | .003 | .008 | 0.35 (0.15 to 0.55) | 0.52 (0.26 to 0.78) | |||||
|
||||||||||||
|
|
|||||||||||
|
|
CS | 0.55 (0.59; −0.61 to 1.71) | .35 | .79 | 0.14 (−0.15 to 0.42) | 0.29 (−0.09 to 0.66) | |||||
|
|
WW | 1.32 (0.59; 0.17 to 2.48) | .02 | .11 | 0.33 (0.04 to 0.61) | 0.46 (0.09 to 0.84) | |||||
|
|
BR | 1.21 (0.59; 0.06 to 2.36) | .04 | .17 | 0.30 (0.01 to 0.58) | 0.41 (0.04 to 0.78) | |||||
|
|
All | 1.03 (0.48; 0.09 to 1.97) | .03 | .08 | 0.25 (0.05 to 0.46) | 0.39 (0.12 to 0.65) | |||||
|
|
|||||||||||
|
|
CS | 1.23 (0.59; 0.06 to 2.40) | .04 | .16 | 0.30 (0.01 to 0.59) | 0.49 (0.11 to 0.88) | |||||
|
|
WW | 1.48 (0.59; 0.33 to 2.64) | .01 | .06 | 0.36 (0.08 to 0.65) | 0.64 (0.25 to 1.02) | |||||
|
|
BR | 1.64 (0.60; 0.47 to 2.81) | .006 | .03 | 0.40 (0.11 to 0.68) | 0.58 (0.20 to 0.96) | |||||
|
|
All | 1.45 (0.48; 0.51 to 2.40) | .003 | .008 | 0.35 (0.15 to 0.56) | 0.57 (0.30 to 0.84) | |||||
|
||||||||||||
|
|
|||||||||||
|
|
CS | 1.51 (0.59; 0.34 to 2.67) | .01 | .06 | 0.37 (0.08 to 0.66) | 0.47 (0.08 to 0.85) | |||||
|
|
WW | 2.25 (0.59; 1.09 to 3.41) | <.001 | .001 | 0.55 (0.26 to 0.84) | 0.70 (0.32 to 1.09) | |||||
|
|
BR | 1.57 (0.59; 0.41 to 2.73) | .008 | .04 | 0.38 (0.10 to 0.67) | 0.46 (0.08 to 0.83) | |||||
|
|
All | 1.78 (0.48; 0.83 to 2.72) | <.001 | <.001 | 0.43 (0.23 to 0.64) | 0.54 (0.28 to 0.81) | |||||
|
|
|||||||||||
|
|
CS | 1.56 (0.60; 0.39 to 2.74) | .009 | .045 | 0.38 (0.09 to 0.67) | 0.57 (0.19 to 0.96) | |||||
|
|
WW | 1.72 (0.59; 0.56 to 2.89) | .004 | .02 | 0.42 (0.13 to 0.71) | 0.62 (0.24 to 1.00) | |||||
|
|
BR | 1.44 (0.60; 0.27 to 2.62) | .02 | .08 | 0.35 (0.06 to 0.63) | 0.46 (0.09 to 0.84) | |||||
|
|
All | 1.58 (0.49; 0.63 to 2.53) | .001 | .003 | 0.38 (0.18 to 0.59) | 0.55 (0.28 to 0.82) | |||||
|
||||||||||||
|
|
|||||||||||
|
|
CS | 0.87 (0.52; −0.15 to 1.90) | .10 | .34 | 0.24 (−0.05 to 0.53) | 0.35 (0.16 to 0.85) | |||||
|
|
WW | 1.18 (0.52; 0.16 to 2.19) | .02 | .11 | 0.33 (0.04 to 0.62) | 0.66 (0.19 to 1.13) | |||||
|
|
BR | 0.94 (0.52; −0.08 to 1.96) | .07 | .27 | 0.26 (−0.02 to 0.55) | 0.44 (−0.06 to 0.93) | |||||
|
|
All | 1.00 (0.42; 0.17 to 1.83) | .02 | .049 | 0.28 (0.08 to 0.48) | 0.50 (0.16 to 0.83) | |||||
|
|
|||||||||||
|
|
CS | 0.86 (0.53; −0.17 to 1.89) | .10 | .36 | 0.24 (−0.05 to 0.53) | 0.18 (−0.32 to 0.68) | |||||
|
|
WW | 1.04 (0.52; 0.01 to 2.06) | .047 | .19 | 0.29 (0.00 to 0.58) | 0.47 (0.00 to 0.94) | |||||
|
|
BR | 1.43 (0.53; 0.40 to 2.47) | .006 | .03 | 0.39 (0.11 to 0.68) | 0.38 (−0.11 to 0.87) | |||||
|
|
All | 1.11 (0.43; 0.27 to 1.94) | .01 | .03 | 0.30 (0.10 to 0.51) | 0.36 (0.02 to 0.69) |
aContrast estimates from LMMs.
b
c
dPSS: Perceived Stress Scale.
eSample size for ITT analyses n=383; sample size for subgroup analyses n=236.
ft1: time point 1.
gt0: time point 0.
hCS: combatting stress.
iWW: working with worry.
jBR: building resilience.
kPooled effect of all intervention arms.
lt2: time point 2.
mGAD-7: Generalized Anxiety Disorder-7.
nSample size for ITT analyses n=383; sample size for subgroup analyses n=227.
oPHQ-8: Patient Health Questionnaire-8.
pSample size for ITT analyses n=383; sample size for subgroup analyses n=229.
qBRS: Brief Resilience Scale.
rSample size for ITT analyses n=383; sample size for subgroup analyses n=142.
Participants in the
Participants in the
A comparison of the overall (pooled) effect of all intervention arms relative to the control group revealed significantly greater improvement for all 4 secondary outcome measures (Hedges
Subgroup analyses were performed for all secondary outcome measures to explore intervention effects for participants self-reporting at least mild symptoms (or moderately low resilience) at t0 (see
Exploratory multiple logistic regression suggested that participants working in health care (b=−2.11, SE 0.86;
In addition, participants with higher PHQ-8 scores at baseline were less likely to complete their intervention (b=−0.17, SE 0.05;
Intervention research can be undermined by problems with intervention delivery, acceptability, participant retention, and smaller than anticipated effect sizes. Therefore, guidelines suggest conducting pilot studies to test trial feasibility and estimate important trial parameters before running a definitive trial [
Participant retention and intervention adherence were largely consistent with or higher than those in comparable studies. For instance, only 7% (27/383) of participants were lost to attrition at follow-up, which compares favorably with recent meta-analyses reporting average attrition rates of 23% to 48% for MHapp trials [
Objective engagement data suggested that 67.8% (196/289) of randomized participants (and 196/237, 82.7% of those starting their intervention) completed all intervention sessions, which is similar to or higher than average completion rates ranging between 30% and 65% for other MHapp interventions [
Despite these high levels of engagement, the study interventions were brief, comprising approximately 60 to 80 minutes of learning over 7 sessions. In addition, the study used a nonclinical sample, and regression analyses suggested that participants with higher symptom severity at baseline were less likely to complete their allocated intervention. Thus, these findings may not be generalizable to other interventions featured on the
Although all 3 interventions met progression criteria, participants randomized to the
The findings from this pilot study revealed several opportunities for minor protocol improvements before running a definitive trial. First, of the 383 participants, 11 (2.9%) reported not fully understanding the study instructions, and 1.7% (5/289) of participants randomized to an intervention engaged with the wrong intervention. This could be addressed by providing participants with detailed video instructions and implementing a brief quiz to ensure that all participants understand how to access their allocated intervention. If feasible, participants could be given access to a modified, study-specific version of the
Although the study was not powered for formal hypothesis testing, preliminary efficacy findings suggested that the study interventions were associated with small to moderate between-group improvements in ≥1 mental health outcomes, which were maintained at the 1-month follow-up. This is consistent with findings from meta-analyses of previous MHapp trials [
First, relative to baseline, participants in the control group reported statistically significant improvements in stress and anxiety at both study time points and improvements in resilience at the 1-month follow-up, despite not having access to any study interventions. Although this may reflect phenomena such as practice effects or regression to the mean, it is worth noting that baseline data were collected several weeks after the commencement of a third national UK lockdown (in response to rising cases of SARS-CoV-2), whereas follow-up data were collected after the start of a phased easing of restrictions. A longitudinal survey conducted in England suggests that symptoms of anxiety and depression tend to rise rapidly during the early stages of a lockdown but decline quickly thereafter [
Second, although all 3 interventions resulted in significant improvements for at least one mental health outcome, the study interventions did not appear to be sensitive or specific to their target outcome. For example, the
This study had several strengths. First, intervention adherence and engagement were objectively captured via
A limitation of this study is the use of a passive no-intervention control group, as opposed to an active control group in which participants engage with activities matched for duration, attention, and interest. Passive controls do not allow true intervention effects to be differentiated from nonspecific placebo effects and may introduce nocebo effects [
This study assessed the feasibility of conducting a future definitive RCT to evaluate the efficacy of 3 brief interventions featured on the
Postintervention feedback questions.
Deviations from the preregistered data analysis plan.
Reliable change index calculations.
Per-protocol analyses.
Postintervention feedback ratings for subgroups.
Participants’ self-reported reasons for not starting or discontinuing their intervention.
Subgroup sample sizes and baseline scores.
CONSORT-eHEALTH checklist (V 1.6.1).
acceptance and commitment therapy
Brief Resilience Scale
cognitive behavioral therapy
Consolidated Standards of Reporting Trials
estimated marginal mean
generalized anxiety disorder
intention-to-treat
linear mixed-effects model
mental health app
mindfulness meditation
Patient Health Questionnaire
per-protocol
Perceived Stress Scale
randomized controlled trial
time point 0
time point 1
time point 2
ME, HB, and KC conceived and designed the study. ME, HB, and RM collected the data. ME analyzed the data and wrote the initial draft of the manuscript. All authors have discussed the results and contributed to the final manuscript.
This study was funded in full by Unmind Ltd, the creator of the intervention evaluated. The authors ME, RM, and HB are employed by and own share options at Unmind Ltd. Author KC is a formal academic collaborator and receives financial compensation to provide scientific and research consulting services to Unmind Ltd.