This paper is in the following e-collection/theme issue:

Formative Evaluation of Digital Health Interventions (2491) Behavior Change (723)

Published on in Vol 4, No 8 (2020): August

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/18586, first published .
A Web-Delivered Acceptance and Commitment Therapy Intervention With Email Reminders to Enhance Subjective Well-Being and Encourage Engagement With Lifestyle Behavior Change in Health Care Staff: Randomized Cluster Feasibility Stud

A Web-Delivered Acceptance and Commitment Therapy Intervention With Email Reminders to Enhance Subjective Well-Being and Encourage Engagement With Lifestyle Behavior Change in Health Care Staff: Randomized Cluster Feasibility Stud

A Web-Delivered Acceptance and Commitment Therapy Intervention With Email Reminders to Enhance Subjective Well-Being and Encourage Engagement With Lifestyle Behavior Change in Health Care Staff: Randomized Cluster Feasibility Stud

Authors of this article:

Menna Brown1 Author Orcid Image ;   Nic Hooper2 Author Orcid Image ;   Phillip James3 Author Orcid Image ;   Darren Scott3 Author Orcid Image ;   Owen Bodger1 Author Orcid Image ;   Ann John1 Author Orcid Image
Article Authors Cited by (5) Tweetations (4) Metrics

    Original Paper

    1Swansea University Medical School, Swansea University, Swansea, United Kingdom

    2Department of Health and Social Sciences, University of the West of England, Bristol, United Kingdom

    3Department of Computer Science, Swansea University, Swansea, United Kingdom

    Corresponding Author:

    Menna Brown, BSc, MSc

    Swansea University Medical School

    Swansea University

    Singleton Park, Data Science Building, Floor 3

    Swansea, SA2 8PP

    United Kingdom

    Phone: 44 1792 606312

    Fax:44 1792 606312

    Email: menna.brown@swansea.ac.uk


    Background: Poor mental health and emotional well-being can negatively impact ability to engage in healthy lifestyle behavior change. Health care staff have higher rates of sickness and absence than other public sector staff, which has implications at both individual and societal levels. Individual efforts to self-manage health and well-being which add to the UK mental health prevention agenda need to be supported.

    Objective: The objective of this study was to establish the feasibility and acceptability of the inclusion of a self-guided, automated, web-based acceptance and commitment therapy intervention in an existing health promotion program, to improve subjective well-being and encourage engagement with lifestyle behavior change.

    Methods: For this 12-week, 4-armed, randomized controlled cluster feasibility study, we recruited participants offline and randomly allocated them to 1 of 3 intervention arms or control (no well-being intervention) using an automated web-based allocation procedure. Eligibility criteria were current health care staff in 1 Welsh health board, age≥18 years, ability to read English, and ability to provide consent. The primary researcher was blinded to cluster allocation. Feasibility outcomes were randomization procedure, acceptance of intervention, and adherence to and engagement with the wider program. We evaluated health and well-being data via self-assessment at 2 time points, registration and postintervention, using the 14-item Warwick-Edinburgh Mental Well-Being Scale, the 4-item Patient Health Questionnaire, and the 7-item Acceptance and Action Questionnaire—Revised.

    Results: Of 124 participants who provided consent and were randomly allocated, 103 completed full registration and engaged with the program. Most participants (76/103) enrolled in at least one health behavior change module, and 43% (41/96) of those randomly allocated to an intervention arm enrolled in the well-being module. Adherence and engagement was low (7/103, 6.8%), but qualitative feedback was positive.

    Conclusions: The procedure and randomization process proved feasible, and the addition of the well-being module proved acceptable to health care staff. However, participant engagement was limited, and no one completed the full 12-week program. User feedback should be used to develop the intervention to address poor engagement. Effectiveness should then be evaluated in a full-scale randomized controlled trial, which would be feasible with additional recruitment.

    Trial Registration: International Standard Randomised Controlled Trial Number (ISRCTN) 50074817; http://www.isrctn.com/ISRCTN50074817

    JMIR Form Res 2020;4(8):e18586

    doi:10.2196/18586

    Keywords

    well-being; mental health; behavior change; acceptance and commitment therapy; web-delivered intervention; gamification; adherence and engagement; internet-based intervention 


    Background

    Poor mental health and emotional well-being underpin many physical diseases and unhealthy lifestyles and can negatively impact an individual’s ability to engage in healthy lifestyle behavior change [1-3]. Recent recognition that positive mental health and emotional well-being are fundamental components of good health has given rise to a strong UK mental health prevention agenda. For example, the Five Year Forward view for Mental Health [4], the Prevention Concordat for Better Mental Health [5] and the National Health Service (NHS) Long Term Plan [6] saw local and national services come together (in 2019) at an unprecedented level in their commitment to addressing the mental health crisis. This development built on earlier publications from the UK Department of Health and Social Care, which outlined the importance of well-being and its role in health outcomes [7-9]. Economic analysis has further supported the case for greater investment in mental well-being [10]. This recent and sustained UK focus on well-being builds on global recognition that mental health and emotional well-being are a fundamental component of good health [11].

    Mental Health: United Kingdom Picture

    Staff sickness and absenteeism in the public sector is high, more so than that of the private sector [12]. Stress is the most commonly cited reason for absenteeism [13], and the associated economic cost is significant at an estimated £105.2 billion a year [14].

    Further to their own mental health and well-being needs, health care staff are well placed to promote positive lifestyle behaviors to others, through effective role modelling [15]. Personal health behaviors are critical in establishing effective and confident health behaviors [16,17]. Frontline staff have daily contact with patients and the public and can exhibit health behaviors to be emulated by others. In the United Kingdom, the Nursing and Midwifery Council identified role modelling as a statutory requirement, stating that nurses must “take every opportunity to encourage health-promoting behaviour through education, role modelling and effective communication” [18]. Likewise, for UK medical professionals, the General Medical Council include clear expectations in Outcomes for Graduates (pg 23) [19]. Research findings have highlighted the need for ongoing support for health care workers to improve their own health and to fully realize their potential as credible role models and healthy-living advocates for the populations they serve [20-24]. Indeed, Public Health England recently launched Every Mind Matters [25], a website to support well-being and physical health. The proliferation of interest in this area serves to highlight the need for formal evaluation of these approaches.

    Web-Based Approach

    Global mental health prevention has incorporated diverse initiatives directed at different levels in society, such as individual, community, and societal level regulations. One key area, which has seen exponential growth, is that of web-delivered interventions. The cost-effective benefits of evidence-based, web-delivered therapies that improve mental health are well established [26-28]; however, poor adherence and engagement remain a significant factor that limits effectiveness [29,30] and application.

    To address poor adherence and engagement, this study used a therapeutic approach associated with positive adherence: acceptance and commitment therapy (ACT) [31,32]. ACT is based on the principle of psychological flexibility, which involves accepting one’s unwanted thoughts and feelings while moving toward personal values. Several successful web-based ACT intervention studies have been reported [33-40], for example, in the treatment of depression among smokers [37] and in the prevention of mental health problems among university students [38]. Earlier systematic reviews and meta-analyses have found web-delivered ACT to be effective for the management of depression [41], and others report its effectiveness in both group and individual settings [42-45].

    Randomized Controlled Feasibility Study

    This study will provide initial insight into the impact, acceptance, and feasibility of a web-delivered, multifaceted workplace lifestyle behavior change program, which incorporates an ACT-based well-being intervention to support staff mental health in the context of a preventive approach.

    Objectives

    The study objectives were to (1) determine whether the randomization procedure was feasible, (2) determine whether the inclusion of an ACT-based well-being intervention, within a web-based lifestyle behavior change program, was acceptable to health care staff, (3) determine whether the well-being intervention positively affected adherence and engagement to the wider program, and (4) explore the impact of additional intervention elements.


    Trial Design

    This was a 4-armed, cluster randomized controlled feasibility trial.

    Trial consent, registration, and assignment to trial arm were automated. The principal researcher (MB) was blinded to cluster allocation throughout the trial; participants were not. A computer code written in Python randomly allocated each cluster to a trial arm using a built-in randomization function. No changes to the program were made during the trial period.

    Trial Arms

    The control group used Champions for Health, which consisted of 5 lifestyle behavior change modules: Quit Smoking, Drink Responsibly, Weight Optimization, Regular Exercise, and Eat Healthily.

    Intervention 1 group used Champions for Health plus the ACT-based well-being module (ACTivate your Well-being).

    Intervention 2 group used Champions for Health and ACTivate your Well-being, plus 5 premade well-being films (see Multimedia Appendix 1).

    Intervention 3 group used Champions for Health and ACTivate your Well-being, plus a static social norm message (eg, “Other users like you have reduced their weight, on average to 75 kg.”).

    Ethics

    The study received ethical approval from the College of the Human and Health Sciences and the College of Medicine Research Ethics Committee, Swansea University, Swansea, UK, and research and development approval from Abertawe Bro Morgannwg University Health Board Joint Study Review Committee 2017 as service evaluation. The trial is registered with the ISRCTN registry (50074817; Multimedia Appendix 2 [46])

    Clusters

    Staff from 1 health board in Wales, UK participated. A health board is an organizational and administrative unit consisting of hospitals, community clinics, and general practices (primary care). There are 7 in Wales.

    We created 4 clusters based on key hospital and community sites within the participating health board: 3 clusters received the intervention and 1 did not (the control). Use of this trial design is common in health care contexts where cluster trials are an important methodology used to compare different ways of encouraging health behavior change [47,48].

    Clusters. We selected this design for pragmatic reasons. The reasons were 3-fold. First, focus group discussions identified that participants who had taken part in earlier releases of the website had discussed its content with colleagues. As such it became evident that if we allocated participants at the individual level, they may discuss and share the content of the intervention with those not allocated to that trial arm. This approach is reported elsewhere [49,50]. Second, the clusters are natural groups of people, determined by their place of employment. Outcomes within naturally occurring clusters may tend to be more correlated than those across clusters; this is because individuals within a health board may have similar practices, arising from organizational culture and shared environment or demographic features that might influence the outcome [48]. Third, allocation by site location may support recruitment. Undertaking randomization after consent and baseline would introduce significant delay, which might have a negative effect on enrollment and engagement.

    The Program: Champions for Health

    Champions for Health, developed by Public Health Wales, comprised 5 lifestyle behavior change modules. Each included text and images and they were based on the health belief model [51], the theory of planned behavior [52], plan, do, study, act [53], and the self-regulatory model [54].

    The Intervention: ACTivate your Well-being

    Following a participatory design process [55-58], NHS staff (n=39), researchers, ACT practitioners, mental health experts, and computer scientists worked together to co-design the website and intervention through a series of exploratory interviews, focus groups, usability sessions, and a pilot evaluation.

    We developed an automated, interactive, 12-week, self-guided intervention called ACTivate your Well-being for predetermined sequential release. Recommended time spent on each week was 20 minutes per day, 3 days a week. In addition to the structured modules, 3 pop-ups were available: Green Space gallery, Sleep, and Relaxation. These could be accessed freely.

    Recruitment and Eligibility

    All staff employed by the health board at the time of the study were eligible to participate. Eligibility criteria were current health care staff in 1 Welsh health board, age≥18 years, ability to read English, and ability to provide consent.

    We recruited participants between January 28 and February 7, 2019 at 4 hospital sites. Electronic invitation and advertisements were displayed on the hospital intranet, electronic, and physical notice boards. Presentations were made at 4 site locations during induction training to a voluntary staff Well-Being Champions scheme.

    Procedure

    Participants registered individually via the website. All participants were required to provide consent using a checkbox process prior to completing a registration form, which asked for username, password, gender, age, location, self-rated health, self-rated work performance, sickness leave in the past 6 months, quantity of 1-week absences in the past 30 days, and self-assessed primary outcome measures. Users could opt to receive a semiautomated weekly email reminder, which included the website link and a motivational quote.

    Once registered, participants accessed the website freely by logging in to their account. At this point, participants found out whether they had access to the well-being intervention. A personalized dashboard enabled participants to enroll in 1 or more modules and track their progress. We incorporated 2 gamification features to support sustained engagement [59]: Rewards and Feedback. Gamification is the use of game elements in non–game contexts [60]. Health points were rewarded for website engagement and converted into trophies at predetermined thresholds, and a bar showed progress toward each trophy. Feedback graphs were generated to show progress (Multimedia Appendix 3). In week 12 participants were reminded, via the home page and weekly email, to complete the time 2 outcome measures and provide feedback. At this point they were also invited to take part in a focus group. We sent 3 email reminders.

    Outcome Measures

    Primary Outcome Measure

    The 14-item Warwick-Edinburgh Mental Well-Being Scale (WEMWBS) is a validated measure of mental well-being in the general population, responsive to change at both the individual and group levels [61-63]. A higher score represents more positive well-being. A score of 43.5 or less is considered a screening threshold for depression [62]. An increase of 2.77 or greater indicates statistically significant improvement [63].

    The 4-item Patient Health Questionnaire (PHQ-4) Anxiety and Depression Scale [64] screened for anxiety and depression. On each subscale, a score of 3 (range 0-6) or greater is considered positive for screening purposes for anxiety and depression [64].

    Process Measures

    The 7-item Acceptance and Action Questionnaire—Revised (AAQ-II) is a validated, 1-factor measure of psychological inflexibility [65]. Higher scores (range 7-49) indicate greater levels of psychological inflexibility [65]. Cutoff points are not published for this measurement tool; however, a score of 17.5 and greater has been identified to indicate significant psychological inflexibility [66].

    Sample Size Considerations

    Feasibility trial designs do not commonly employ formal power calculations [67]. As such we aimed to recruit 100 participants, 25 in each trial arm, to allow comparison across groups and to explore the study objectives.

    Data Analysis

    Randomization Procedure

    Functionality was assessed by the web developer during week 1 to ensure that self-reported location data were used accurately to populate trial arms.

    Acceptability, Adherence, and Engagement

    Descriptive statistics reported website registration and participant characteristics. We conducted statistical analysis using nonparametric methods in IBM SPSS version 26 (IBM Corporation) to explore baseline characteristics of each trial arm. Adherence was measured by completion of outcome measure at baseline and postintervention. Qualitative feedback, collected at the end of the program via a structured feedback form, interview, and focus group discussion, was audio recorded and transcribed verbatim. Data were analyzed using inductive thematic analysis [68].

    Primary Outcome Effect

    The feasibility study was not powered to test statistical significance, but merely to explore the impact and effectiveness of the intervention on lifestyle behavior change and well-being across trial arms.


    Randomization

    The automated randomization procedure proved effective, and participants were allocated as expected.

    Recruitment and Participant Summary

    A total of 124 users consented to participate and were randomly allocated to a trial arm. Of these, 103 users provided baseline data for the primary outcome measures and were analyzed (Figure 1).

    ‎
    Figure 1. Consolidated Standards of Reporting Trials flow diagram. AAQ-II: Acceptance and Action Questionnaire—Revised; PHQ-4: Patient Health Questionnaire-4; WEMWBS: Warwick-Edinburgh Mental Well-Being Scale.
    View this figure

    Users spanned all age brackets (18-65 years old). Most users (91/103, 88.4%) were female and rated their health as “good” to “excellent” (82/101, 81.2%) on a 5-point Likert scale. Over half (63/103, 61.2%) reported no days off work in the past 6 months and, in line with this, the majority (81/103, 78.6%) did not report any 1-week absences in the past month. On average, users rated their general work performance 8 on a scale of 1 to 10, with 10 being highest. All opted to receive email reminders (Table 1).

    Table 1. Participant characteristics (n=103).
    CharacteristicsTrial arm
    Control (Champions for Health) Intervention 1 (Champions plus ACTivate your Well-beinga)Intervention 2 (Champions and ACTivate plus PocketMedic)Intervention 3 (Champions and ACTivate your Well-being plus social norm message)
    Randomized, n7423441
    Analyzed, n721b3441
    Female, n (%)6 (86)19 (90)30 (88)38 (93)
    Age bracket (years), n (%)

    		18-25004 (12)2 (5)

    		26-352 (28)8 (38)10 (29)10 (24)

    		36-453 (43)5 (24)11 (32)12 (29)

    		46-551(14)5 (24)7 (20)14 (32)

    		56-651 (14)3 (14)2 (6)4 (10)
    Self-reported days off work, mean (SD)1.4 (2.1)9.3 (18.4)1.0 (1.5)6.9 (25.3)
    Self-rated general work performance scorec, mean (SD)7.3 (1.6)7.6 (1.7)7.5 (1.3)7.5 (1.7)

    aACTivate your Well-being intervention based on acceptance and commitment therapy.

    b21 users were excluded based on incomplete registration.

    cOn a scale of 1-10, with 10 being the highest.

    Trial Arms

    We detected no significant differences between trial arms, using the Kruskal-Wallis test, at baseline (n=103) for registration week (P=.17), age (P=.51), self-rated health (P=.36), days off work (P=.84), absences of 1-week duration (P=.09), self-rated work performance (P=.94), WEMWBS (P=.24), PHQ-4 (P=.27), AAQ-II (P=.25) or gender (χ23=103; P=.56).

    We observed no difference for enrollment in module: Weight Optimization, χ22=103, P=.38; Regular Exercise, χ22=103, P=.25; Drink Responsibly, χ23=103, P=.88; Eat Healthily, χ23=103, P=.74; Quit Smoking, χ23=103, P=.85). However, module engagement differed across intervention groups (χ22=96, P=.40).

    Enrollment (χ22=96, P=.10) or engagement (as a binary variable) in the well-being intervention did not differ significantly between the 3 intervention groups (χ22=96, P=.79) (Table 2).

    Table 2. Module engagement (no. of participants) per trial arm (n=103).
    ModuleTrial arm
    Control (Champions for Health)Intervention 1 (Champions and ACTivate your Well-beinga)Intervention 2 (Champions and ACTivate your Well-being plus PocketMedic)Intervention 3 (Champions and ACTivate your Well-being plus social norm message)
    Randomized, n7213441
    Enrolled, n (%)

    		Champions for Health5 (71)13 (62)29 (85)27 (66)

    		ACTivate your Well-beingN/Ab13 (62)11 (32)17 (41)
    Engaged, n (%)

    		Champions for Health2 (28)09 (26)9 (22)

    		ACTivate your Well-beingN/A1 (5)3 (9)4 (10)

    aACTivate your Well-being intervention based on acceptance and commitment therapy.

    Outcome Measures

    Primary Outcome Measure

    At baseline, participant well-being scores, measured using WEMWBS, was mean 46.3 (SD 8.8, range 29-68). This was lower than the general population. Only 6 participants completed WEMWBS postintervention, and this group recorded a higher score (mean 53.8, SD 3.1) and higher minimum score (43). When this subgroup was tested, 5 participants showed raised scores, which presented some evidence of an improvement, but this fell below the level of statistical significance (P=.11; Wilcoxon signed rank test).

    Baseline anxiety and depression scores, measured using PHQ-4, were within the normal population range, with means of 1.8 (SD 1.6) and 1.3 (SD 1.5), respectively. A small subset of users met the screening criteria (20/103, 19.4%; 11/103, 10.7%, respectively). The combined PHQ-4 scores (n=7) did not change significantly postintervention (P=.34; Wilcoxon signed rank test).

    Process Measures

    The mean AAQ-II score was 21.7 (SD 9.8; n=22). However, no comparison could be drawn, as only 2 participants completed the postintervention questionnaire.

    Acceptability, Adherence, and Engagement

    Adherence to the study protocol was poor (7/103, 6.8%). However, the majority of participants (76/103, 73.8%) enrolled in at least one module. Almost half (50/103, 48.5%) enrolled in 1 module, 17 (16.5%) enrolled in 2 modules, 7 (6.8%) enrolled in 3 modules, and 2 (1.9%) enrolled in 4 modules. The most popular modules were well-being (41/96, 43%), Regular Exercise (40/103, 38.8%), and Weight Optimization (39/103, 37.9%).

    Of the 9 participants who enrolled in Drink Responsibly, only 3 (33%) engaged. However, health outcomes improved for these active users; 1 reported a reduction in days per week that they consumed alcohol from 4 to 2, with a reduction from 20 drinks per week to 8. Another reduced their consumption from 4 to 3 days per week, with a reduction from 11 to 5 drinks, and the third increased their number of days drinking but their overall alcohol consumption reduced (from 18 to 4). Of the 23 participants (22.3%) who enrolled in Eat Healthily, only 6 (26%) engaged, 1 until week 6. User data indicated poor fruit and vegetable consumption, with few meeting recommended guidelines and some never consuming the recommended 5-a-day portion (5/23). Almost all who enrolled in Weight Optimization (37/39, 95%) engaged and provided an initial weight (mean 75.77, SD 15.80 kg). A small subsample (10/39, 26%) provided a second weight (mean 75, SD 10.89 kg). Of the 40 participants who enrolled in Regular Exercise, 33% (13/40) actively engaged. This module saw the longest sustained engagement of the modules, with activity recorded until week 9. Only 2 (2/103, 1.9%) participants enrolled in Quit Smoking and neither entered nor tracked any data. Well-being had the highest enrollment (41/96, 42%). However, of those who enrolled, only 7 (17%) completed any of the weekly “try-now” activities. This module had the longest sustained engagement overall, with 1 user active until week 10.

    Qualitative Data

    To fully explore acceptability, we collected a range of qualitative data. We facilitated 2 one-to-one interviews (25-minute duration) and 2 focus groups (77- and 73-minute durations, 4 participants each). A total of 15 participants contacted the principal researcher via email and 8 completed the feedback survey.

    Qualitative feedback was positive (Table 3), and it was clear that staff welcomed the inclusion of the well-being intervention; indeed, people who had used the prior releases requested its inclusion. Analysis identified recommendations for future development.

    Table 3. Sample quotes from participants’ feedback and their recommendations.
    ThemeIllustrative quotes and recommendations
    Feedback on website and well-being module“I liked that it would tell you what you should be doing to keep up with the NHSa recommendations and then how it compared that to Wales and the rest of the population. I liked all that information.” [Interview]

    		“I would have liked to set weekly goals.” [Interview]

    		“Easy to use, visually was nice.” [Interview]

    		“The health and fitness aspects were quite helpful, I was enrolled on the modules, health eating, weight management, and regular exercise. They were quite simple and straightforward and the information there was very useful.” [Interview]

    		“I really liked the PocketMedic. I looked at all the films on there.” [Focus group]

    		“I liked some aspects of the ACTb therapy and I found some aspects helpful. Certainly, I’ve suffered from intrusive thoughts and it’s helpful to sort of just accept them rather than fighting against them.” [Interview]
    Nonadherence“To me it was purely entering my weight, which I understand needed to be done...where I sit in my office everyone can see my screen clearly and that was why I was not happy to enter my weight.” [Email]

    		“Couldn’t log in, then other priorities took over firefighting through work. It’s a time factor thing.” [Email]

    		“I stopped doing the one about weight management, as when I started there were scales up in outpatients that I used to use and then they took them away.” [Interview]
    Participant recommendationsProvide an option to set weekly goals and the option to report whether this goal was achieved or not.

    		Provide the option to return to the previous week to enter progress data.

    		Display progress data per activity undertaken; for example, during week 1 you swam for a total of 80 minutes; you did yoga for 60 minutes, or more detailed track-your-progress options to facilitate competitive and personalized elements.

    		Provide personalization of the profile area. Provide the option to edit data displayed on the screen specifically in reference to weight due to lack of privacy in the work setting or the option to hide private details such as weight.

    		Streamline access to well-being exercises and activities.

    		Include additional signposts to alternative sources of help.

    		Incorporate opportunities to connect and interact with others.

    		Improve the layout of AAQ-IIc.

    aNHS: National Health Service.

    bACT: ACTivate your Well-being intervention based on acceptance and commitment therapy.

    cAAQ-II: Acceptance and Action Questionnaire—Revised.


    Principal Findings

    This study explored the feasibility and acceptability of the inclusion of an ACT-based well-being intervention, within an existing web-based, workplace lifestyle behavior change program for health care staff. The cluster design and automated randomization procedure proved feasible. Participants were successfully randomly allocated based on self-reported location, and the principal researcher remained blinded until postintervention. The new multifaceted program also proved acceptable to NHS staff. Recruitment was positive and participation rates compared equally with previous releases; for example, the 2015 campaign administered by Public Health Wales recruited 140 staff from 1 health board.

    The proportion of users (43%) who selected to enroll in the well-being intervention highlighted that initiatives such as these are desirable and that well-being support is required in the workplace. Indeed, the qualitative interviews, focus groups, and feedback were positive, and this is encouraging. Staff who engaged with the program and well-being module enjoyed the resources and reported personal benefit. Feedback also highlighted specific ways to develop the intervention to address poor engagement from the staff perspective, which will be used to support future development. However, the lack of engagement is still a concern. Only 7 participants were active in the try-now elements of the intervention, and engagement was equally poor across the modules. It is worth noting that poor engagement may have occurred as an artifact of what was measured. Specifically, the website recorded activity only for “track your progress:” it was not possible to assess log-on rate, time spent on each page, and general website activity. Therefore, it is possible that users engaged with other program elements. Indeed, the focus group and interview data suggested that the site overall was well used. In line with this, it is also encouraging that most participants enrolled in and engaged with at least one module, the most popular being Regular Exercise and Weight Optimization. Indeed, these were often selected in combination. Quit Smoking was the least popular, with only 2 users, despite local rates of smoking remaining at 22% [69].

    Looking at the global picture, this study found health care staffs’ well-being scores (WEMWBS) to be lower than the general population (data for England). Mean scores for men and women on this measure are 50.1 and 49.6 [61], respectively, compared with 46.3 in our study. In addition to this, a subset of participants had PHQ-4 scores associated with anxiety and depression. These findings add to the global picture, which suggests prevalence rates of common mental disorders, such as anxiety and depression, are high. In the United Kingdom, estimated population incidence rates are 4% to 10% [70]. Elsewhere, similar instances are reported. For example, lifetime disorder rates in Australia are reported to be 45% [71]. The individual and economic cost associated with common mental disorders is significant. Mental health problems constitute the largest single source of world economic burden, with an estimated global cost of £1.6 trillion [72]. In the United Kingdom, the estimated costs of mental health problems are £70 billion to £100 billion each year and account for 4.5% of gross domestic product [73].

    Finally, in this study we were also interested in exploring the impact of the additional intervention elements, PocketMedic and social norm message. There was no significant difference in engagement across the 3 interventions. Qualitative feedback indicated that PocketMedic films were appreciated. Future exploration could collapse the additional elements into one. This is line with earlier findings [74].

    Limitations

    Several limitations must be acknowledged.

    First, the small sample recruited for this feasibility study limited the statistical analysis undertaken. This was particularly relevant to the control arm, as participants were randomly allocated in a 1:3 ratio in favor of the intervention. While we undertook the same recruitment process at each site, only 7 members of staff registered from the location randomized to the control. However, the sample size is not dissimilar to other published studies reporting the effects of similar interventions [36]. In future, cluster size should be considered.

    Second, intervention 1 had a high number of participants excluded from the analysis as a result of incomplete registration data. This further reduced the sample size and equality between trial arms. The registration process should include mandatory fields to avoid this issue in future iterations.

    A third study limitation resulted from low adherence. Few participants completed the postintervention outcome measures. This limited exploration of intervention impact. In response it will be important to explore alternative ways to encourage self-reported completion at postintervention. One option is to track individual nonuse and request feedback within 1 week. This may go some way to improving adherence rates, as reasons for nonuse could be identified and resolved during the study period. Alternatively, the intervention could be shortened to support continued use. Another option might be to redesign engagement data collection points within the website. In this version, engagement monitoring was limited to the track-your-progress or try-now features, both of which are user initiated. No automated data were recorded. Feedback and interview data suggested that participants engaged at many additional time points, which was not captured in our analysis. This should be addressed. Equally, the use of rewards and feedback did not support sustained engagement. Future developments should consider use of additional or different gamification features, for example, avatars or social interaction [75], or guided support and structured feedback, which have been associated with better adherence.

    Fourth, due to an error in the database, which has been amended, we were not able to examine health care worker role in relation to self-reported absenteeism.

    Comparison With Prior Work

    The inclusion of ACTivate your Well-being within the Champions for Health program created a multifaceted program free and easily accessible to a range of health care staff.

    Limited research has explored the role of well-being interventions on lifestyle behavior change programs. To our knowledge, this study is one of the first to explicitly explore the additional benefit of an emotional well-being intervention on lifestyle behavior change. We identified 2 prior studies that incorporated a mental health intervention within a physical health promotion program [76,77]; however, neither explicitly evaluated the additional benefit of a well-being intervention on lifestyle behavior choices and neither used ACT.

    The web-based interface used in this health care setting offered an opportunity to provide tailored support to public sector staff through convenient and accessible means. The inclusion of the emotional well-being intervention, in combination with the modules, is a significant step forward in terms of prevention and early intervention for self-management of positive health behaviors and builds on the UK mental health prevention agenda. The multifaceted program targeted both physical health behaviors and emotional well-being in 1 integrated program. This is the unique feature of this program.

    Conclusion

    This feasibility study was not powered to statistically assess the impact on physical health. A full-scale randomized controlled trial with wider-ranging recruitment methods and additional participant groups would likely support this. We are hopeful the larger-scale trial will answer this question. Study participants who engaged with ACTivate your Well-being and Champions for Health reported positive feedback and made several useful recommendations to take this project forward.

    Acknowledgments

    This research was conducted independently with the support of a research grant from Healthcare Research Wales (SCS-14-11). The funders had no role in study design, data collection and analyses, decision to publish, or preparation of the manuscript.

    Thank you to eHealth Digital Media Ltd, Swansea, UK, who kindly provided the PocketMedic well-being films.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    PocketMedic.

    PDF File (Adobe PDF File), 232 KB
    ‎
    Multimedia Appendix 2

    CONSORT-EHEALTH checklist V1.6.1.

    TXT File , 81 KB
    ‎
    Multimedia Appendix 3

    Champions for Health website images and screenshots.

    PDF File (Adobe PDF File), 681 KB
    1. Mouchacca J, Abbott GR, Ball K. Associations between psychological stress, eating, physical activity, sedentary behaviours and body weight among women: a longitudinal study. BMC Public Health 2013 Sep 11;13:828 [FREE Full text] [CrossRef] [Medline]
    2. McEwen BS. Central effects of stress hormones in health and disease: understanding the protective and damaging effects of stress and stress mediators. Eur J Pharmacol 2008 Apr 07;583(2-3):174-185 [FREE Full text] [CrossRef] [Medline]
    3. Schneiderman N, Ironson G, Siegel SD. Stress and health: psychological, behavioral, and biological determinants. Annu Rev Clin Psychol 2005;1:607-628 [FREE Full text] [CrossRef] [Medline]
    4. The Mental Health Task Force. The Five Year Forward View for Mental Health: A Report From the Independent Mental Health Taskforce to the NHS in England. 2016 Feb 1.   URL: https://www.england.nhs.uk/wp-content/uploads/2016/02/Mental-Health-Taskforce-FYFV-final.pdf [accessed 2020-06-15]
    5. Public Health England. Prevention Concordat for Better Mental Health: Prevention Planning Resource for Local Areas. 2017 Aug.   URL: https:/​/assets.​publishing.service.gov.uk/​government/​uploads/​system/​uploads/​attachment_data/​file/​740587/​Prevention_Concordat_for_Better_Mental_Health_Prevention_planning.​pdf [accessed 2020-06-15]
    6. NHS England, Department of health and Social Care. The NHS Long Term Plan. 2019 Jan 07.   URL: https://www.longtermplan.nhs.uk/wp-content/uploads/2019/08/nhs-long-term-plan-version-1.2.pdf [accessed 2020-06-15]
    7. De Feo D, Barrett J, Edwards J, Hurst M, Green J. Wellbeing: Why it Matters to Health Policy. London, UK: Department of Health; 2014.   URL: https:/​/assets.​publishing.service.gov.uk/​government/​uploads/​system/​uploads/​attachment_data/​file/​277566/​Narrative__January_2014_.​pdf [accessed 2020-06-15]
    8. Department of Health. A Compendium of Factsheets: Wellbeing Across the Life Course. The Relationship Between Wellbeing and Health. 2014 Jan.   URL: https:/​/assets.​publishing.service.gov.uk/​government/​uploads/​system/​uploads/​attachment_data/​file/​295474/​The_relationship_between_wellbeing_and_health.​pdf [accessed 2020-06-15]
    9. Department of Health. A Compendium of Factsheets: Wellbeing Across the Life Course. Wellbeing and Longevity. 2014 Jan.   URL: https:/​/assets.​publishing.service.gov.uk/​government/​uploads/​system/​uploads/​attachment_data/​file/​277588/​Wellbeing_and_Longevity.​pdf [accessed 2020-06-15]
    10. Department of Health and Social Care. No Health Without Mental Health: A Cross-Government Mental Health Outcomes Strategy for People of All Ages. Supporting Document- The Economic Case For Improving Efficiency and Quality in Mental Health. 2011 Feb 02.   URL: https:/​/assets.​publishing.service.gov.uk/​government/​uploads/​system/​uploads/​attachment_data/​file/​215808/​dh_123993.​pdf [accessed 2020-06-15]
    11. Prince M, Patel V, Saxena S, Maj M, Maselko J, Phillips MR, et al. No health without mental health. Lancet 2007 Sep 08;370(9590):859-877. [CrossRef] [Medline]
    12. Office for National Statistics. Sickness absence falls to the lowest rate on record. 2018 Jul 30.   URL: https:/​/www.​ons.gov.uk/​employmentandlabourmarket/​peopleinwork/​employmentandemployeetypes/​articles/​sicknessabsencefallstothelowestratein24years/​2018-07-30 [accessed 2020-06-22]
    13. Carol B, Frost D. Health at Work – An Independent Review of Sickness Absence. 2011 Nov.   URL: https:/​/assets.​publishing.service.gov.uk/​government/​uploads/​system/​uploads/​attachment_data/​file/​181060/​health-at-work.​pdf [accessed 2020-06-15]
    14. Office for National Statistics. UK labor market: July 2016. Estimates of employment, unemployment, economic inactivity and other employment-related statistics for the UK. 2016 Jul 20.   URL: https:/​/www.​ons.gov.uk/​employmentandlabourmarket/​peopleinwork/​employmentandemployeetypes/​bulletins/​uklabourmarket/​july2016 [accessed 2016-09-16]
    15. Boorman S. NHS Health and Well-Being: Final report November 2009. Leeds, UK: The Health and Well-Being Review Team; 2009.   URL: https:/​/webarchive.​nationalarchives.gov.uk/​20130124052412/​http:/​www.dh.gov.uk/​prod_consum_dh/​groups/​dh_digitalassets/​documents/​digitalasset/​dh_108907.​pdf [accessed 2020-06-15]
    16. Nursing and Midwifery Council. Standards for Pre-Registration Nursing Education. London, UK: Nursing and Midwifery Council; 2010 Sep 16.   URL: https:/​/www.​nmc.org.uk/​globalassets/​sitedocuments/​standards/​nmc-standards-for-pre-registration-nursing-education.​pdf [accessed 2020-06-15]
    17. Vickers KS, Kircher KJ, Smith MD, Petersen LR, Rasmussen NH. Health behavior counseling in primary care: provider-reported rate and confidence. Fam Med 2007;39(10):730-735 [FREE Full text] [Medline]
    18. Nursing and Midwifery Council. Standards for Competence for Registered Nurses. London, UK: Nursing and Midwifery Council; 2010.   URL: https:/​/www.​nmc.org.uk/​globalassets/​sitedocuments/​standards/​nmc-standards-for-competence-for-registered-nurses.​pdf [accessed 2020-06-15]
    19. General Medical Council. Outcomes for graduates. 2018 Jun.   URL: https://www.gmc-uk.org/-/media/documents/outcomes-for-graduates-a4-6_pdf-78952372.pdf [accessed 2020-02-21]
    20. Blake H, Patterson J. Paediatric nurses' attitudes towards the promotion of healthy eating. Br J Nurs 2015;24(2):108-112. [CrossRef] [Medline]
    21. Darch J, Baillie L, Gillison F. Nurses as role models in health promotion: a concept analysis. Br J Nurs 2017 Sep 28;26(17):982-988. [CrossRef] [Medline]
    22. Blake H, Harrison C. Health behaviours and attitudes towards being role models. Br J Nurs 2013;22(2):86-94. [CrossRef] [Medline]
    23. Lobelo F, de Quevedo IG. The evidence in support of physicians and health care providers as physical activity role models. Am J Lifestyle Med 2016 Jan;10(1):36-52 [FREE Full text] [CrossRef] [Medline]
    24. Oberg EB, Frank E. Physicians' health practices strongly influence patient health practices. J R Coll Physicians Edinb 2009 Dec;39(4):290-291 [FREE Full text] [CrossRef] [Medline]
    25. UK National Health Service. Every Mind Matters.: Public Health England; 2020.   URL: https://www.nhs.uk/oneyou/every-mind-matters [accessed 2020-06-15]
    26. Amstadter AB, Broman-Fulks J, Zinzow H, Ruggiero KJ, Cercone J. Internet-based interventions for traumatic stress-related mental health problems: a review and suggestion for future research. Clin Psychol Rev 2009 Jul;29(5):410-420 [FREE Full text] [CrossRef] [Medline]
    27. Morrell CJ, Sutcliffe P, Booth A, Stevens J, Scope A, Stevenson M, et al. A systematic review, evidence synthesis and meta-analysis of quantitative and qualitative studies evaluating the clinical effectiveness, the cost-effectiveness, safety and acceptability of interventions to prevent postnatal depression. Health Technol Assess 2016 May;20(37):1-414 [FREE Full text] [CrossRef] [Medline]
    28. Mohr DC, Burns MN, Schueller SM, Clarke G, Klinkman M. Behavioral intervention technologies: evidence review and recommendations for future research in mental health. Gen Hosp Psychiatry 2013 Aug;35(4):332-338 [FREE Full text] [CrossRef] [Medline]
    29. Hilvert-Bruce Z, Rossouw PJ, Wong N, Sunderland M, Andrews G. Adherence as a determinant of effectiveness of internet cognitive behavioural therapy for anxiety and depressive disorders. Behav Res Ther 2012 Aug;50(7-8):463-468. [CrossRef] [Medline]
    30. Donkin L, Christensen H, Naismith SL, Neal B, Hickie IB, Glozier N. A systematic review of the impact of adherence on the effectiveness of e-therapies. J Med Internet Res 2011;13(3):e52 [FREE Full text] [CrossRef] [Medline]
    31. Hayes SC. Acceptance and commitment therapy, relational frame theory, and the third wave of behavioral and cognitive therapies. Behav Ther 2016 Nov;47(6):869-885. [CrossRef] [Medline]
    32. Hayes S, Barnes-Holmes D, Roche B. Relational Frame Theory: A Post-Skinnerian Account of Human Language and Cognition. New York, NY: Springer Science & Business Media; 2001.
    33. Carlbring P, Hägglund M, Luthström A, Dahlin M, Kadowaki A, Vernmark K, et al. Internet-based behavioral activation and acceptance-based treatment for depression: a randomized controlled trial. J Affect Disord 2013 Jun;148(2-3):331-337. [CrossRef] [Medline]
    34. Pots WTM, Fledderus M, Meulenbeek PAM, ten Klooster PM, Schreurs KMG, Bohlmeijer ET. Acceptance and commitment therapy as a web-based intervention for depressive symptoms: randomised controlled trial. Br J Psychiatry 2016 Jan;208(1):69-77. [CrossRef] [Medline]
    35. Lappalainen P, Granlund A, Siltanen S, Ahonen S, Vitikainen M, Tolvanen A, et al. ACT Internet-based vs face-to-face? A randomized controlled trial of two ways to deliver acceptance and commitment therapy for depressive symptoms: an 18-month follow-up. Behav Res Ther 2014 Oct;61:43-54. [CrossRef] [Medline]
    36. Lappalainen P, Langrial S, Oinas-Kukkonen H, Tolvanen A, Lappalainen R. Web-based acceptance and commitment therapy for depressive symptoms with minimal support: a randomized controlled trial. Behav Modif 2015 Aug 6. [CrossRef] [Medline]
    37. Jones HA, Heffner JL, Mercer L, Wyszynski CM, Vilardaga R, Bricker JB. Web-based acceptance and commitment therapy smoking cessation treatment for smokers with depressive symptoms. J Dual Diagn 2015;11(1):56-62 [FREE Full text] [CrossRef] [Medline]
    38. Levin ME, Pistorello J, Seeley JR, Hayes SC. Feasibility of a prototype web-based acceptance and commitment therapy prevention program for college students. J Am Coll Health 2014;62(1):20-30 [FREE Full text] [CrossRef] [Medline]
    39. Levin ME, Hayes SC, Pistorello J, Seeley JR. Web-based self-help for preventing mental health problems in universities: comparing acceptance and commitment training to mental health education. J Clin Psychol 2016 Mar;72(3):207-225. [CrossRef] [Medline]
    40. Trompetter HR, Bohlmeijer ET, Veehof MM, Schreurs KMG. Internet-based guided self-help intervention for chronic pain based on acceptance and commitment therapy: a randomized controlled trial. J Behav Med 2015 Feb;38(1):66-80. [CrossRef] [Medline]
    41. Brown M, Glendenning A, Hoon AE, John A. Effectiveness of web-delivered acceptance and commitment therapy in relation to mental health and well-being: a systematic review and meta-analysis. J Med Internet Res 2016 Aug 24;18(8):e221 [FREE Full text] [CrossRef] [Medline]
    42. Ruiz FJ. Acceptance and commitment therapy versus traditional cognitive behavioral therapy: a systematic review and meta-analysis of current empirical evidence. Int J Psychol Psychol Ther 2012;12(3):333-358 [FREE Full text]
    43. Bohlmeijer ET, Fledderus M, Rokx TAJJ, Pieterse ME. Efficacy of an early intervention based on acceptance and commitment therapy for adults with depressive symptomatology: evaluation in a randomized controlled trial. Behav Res Ther 2011 Jan;49(1):62-67. [CrossRef] [Medline]
    44. Fledderus M, Bohlmeijer ET, Pieterse ME, Schreurs KMG. Acceptance and commitment therapy as guided self-help for psychological distress and positive mental health: a randomized controlled trial. Psychol Med 2012 Mar;42(3):485-495. [CrossRef] [Medline]
    45. Fledderus M, Bohlmeijer ET, Smit F, Westerhof GJ. Mental health promotion as a new goal in public mental health care: a randomized controlled trial of an intervention enhancing psychological flexibility. Am J Public Health 2010 Dec;100(12):2372. [CrossRef] [Medline]
    46. Eysenbach G, CONSORT-EHEALTH Group. CONSORT-EHEALTH: improving and standardizing evaluation reports of Web-based and mobile health interventions. J Med Internet Res 2011;13(4):e126 [FREE Full text] [CrossRef] [Medline]
    47. Eldridge S, Kerry S, Torgerson DJ. Bias in identifying and recruiting participants in cluster randomised trials: what can be done? BMJ 2009 Oct 09;339:b4006. [CrossRef] [Medline]
    48. Osrin D, Azad K, Fernandez A, Manandhar DS, Mwansambo CW, Tripathy P, et al. Ethical challenges in cluster randomized controlled trials: experiences from public health interventions in Africa and Asia. Bull World Health Organ 2009 Oct;87(10):772-779 [FREE Full text] [CrossRef] [Medline]
    49. Peri K, Kerse N, Robinson E, Parsons M, Parsons J, Latham N. Does functionally based activity make a difference to health status and mobility? A randomised controlled trial in residential care facilities (The Promoting Independent Living Study; PILS). Age Ageing 2008 Jan;37(1):57-63. [CrossRef] [Medline]
    50. Sedgwick P. Cluster randomized controlled trials: sample size calculations. BMJ 2013;346:f2839. [CrossRef]
    51. Rosenstock IM, Strecher VJ, Becker MH. Social learning theory and the health belief model. Health Educ Q 1988;15(2):175-183. [Medline]
    52. Godin G, Kok G. The theory of planned behavior: a review of its applications to health-related behaviors. Am J Health Promot 1996;11(2):87-98. [CrossRef] [Medline]
    53. Coury J, Schneider JL, Rivelli JS, Petrik AF, Seibel E, D'Agostini B, et al. Applying the Plan-Do-Study-Act (PDSA) approach to a large pragmatic study involving safety net clinics. BMC Health Serv Res 2017 Jun 19;17(1):411 [FREE Full text] [CrossRef] [Medline]
    54. Leventhal H, Nenyamini Y, Brownlee S. Illness representations: theoretical foundations. In: Petrie KJ, Weinman JA, editors. Perceptions of Health and Illness. Amsterdam, Netherlands: Harwood Academic Publishers; 1997:1-18.
    55. Spinuzzi C. The methodology of participatory design. Appl Theor 2005 May;52(2):163-174 [FREE Full text]
    56. Kinzie MB, Cohn WF, Julian MF, Knaus WA. A user-centered model for web site design: needs assessment, user interface design, and rapid prototyping. J Am Med Inform Assoc 2002;9(4):320-330 [FREE Full text] [Medline]
    57. Rogers Y, Sharp H, Preece J. Interactive Design: Beyond Human Computer Interaction. Third edition. Hoboken, NJ: Wiley; Jun 26, 2011:9780470665763.
    58. Lindsay S, Jackson D, Schofield G, Olivie P. Engaging older people using participatory design. 2012 Presented at: SIGCHI conference on human factors in computing systems; May 5-10, 2012; Austin, TX, USA. [CrossRef]
    59. Brown M, O'Neill N, van Woerden H, Eslambolchilar P, Jones M, John A. Gamification and adherence to web-based mental health interventions: a systematic review. JMIR Ment Health 2016;3(3):e39 [FREE Full text] [CrossRef] [Medline]
    60. Deterding S, Dixon D, Khaled R, Nacke L. From game design elements to gamefulness: defining gamification. New York, NY: ACM; 2011 Presented at: 15th International Academic Mind Trek Conference: Envisioning Future Media Environments 2011; Sep 28-30, 2011; Tampere, Finland p. 9-15. [CrossRef]
    61. Tennant R, Hiller L, Fishwick R, Platt S, Joseph S, Weich S, et al. The Warwick-Edinburgh Mental Well-being Scale (WEMWBS): development and UK validation. Health Qual Life Outcomes 2007;5:63 [FREE Full text] [CrossRef] [Medline]
    62. Maheswaran H, Weich S, Powell J, Stewart-Brown S. Evaluating the responsiveness of the Warwick Edinburgh Mental Well-Being Scale (WEMWBS): group and individual level analysis. Health Qual Life Outcomes 2012 Dec 27;10:156 [FREE Full text] [CrossRef] [Medline]
    63. The Warwick-Edinburgh Mental Well-Being Scale (WEMWBS): development of clinical cut-off scores.   URL: https://pdfs.semanticscholar.org/c32e/59006fbd769c7e595e23e5b20591de0ee018.pdf [accessed 2019-06-24]
    64. Kroenke K, Spitzer RL, Williams JBW, Löwe B. An ultra-brief screening scale for anxiety and depression: the PHQ-4. Psychosomatics 2009;50(6):613-621. [CrossRef] [Medline]
    65. Bond FW, Hayes SC, Baer RA, Carpenter KM, Guenole N, Orcutt HK, et al. Preliminary psychometric properties of the Acceptance and Action Questionnaire-II: a revised measure of psychological inflexibility and experiential avoidance. Behav Ther 2011 Dec;42(4):676-688. [CrossRef] [Medline]
    66. Shari NI, Zainal NZ, Guan NC, Ahmad Sabki Z, Yahaya NA. Psychometric properties of the acceptance and action questionnaire (AAQ II) Malay version in cancer patients. PLoS One 2019;14(2):e0212788 [FREE Full text] [CrossRef] [Medline]
    67. Thabane L, Ma J, Chu R, Cheng J, Ismaila A, Rios LP, et al. A tutorial on pilot studies: the what, why and how. BMC Med Res Methodol 2010;10:1 [FREE Full text] [CrossRef] [Medline]
    68. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006 Jan;3(2):77-101. [CrossRef]
    69. Cosh H, Davies G, Francis I, Griffiths E, May L, Roberts C, et al. Tobacco and Health in Wales. Cardiff, UK: Public Health Wales Observatory, Welsh Government; 2012 Jun.   URL: http:/​/www2.​nphs.wales.nhs.uk:8080/​PubHObservatoryProjDocs.nsf/​85c50756737f79ac80256f2700534ea3/​509486bfd300fdef80257a29003c3c67/​$FILE/​Eng%20Smoking%20Report%20LowRes.​pdf [accessed 2020-06-15]
    70. NHS Digital. National Study of Health and Wellbeing. 2016.   URL: https:/​/digital.​nhs.uk/​data-and-information/​areas-of-interest/​public-health/​national-study-of-health-and-wellbeing [accessed 2020-06-15]
    71. Australian Institute of Health and Welfare. Mental health services in Australia. Canberra, Australia: Australian Institute of Health and Welfare; 2016.   URL: https://www.aihw.gov.au/reports/mental-health-services/mental-health-services-in-australia [accessed 2020-06-15]
    72. McManus S, Meltzer H, Brugha T, Bebbington P, Jenkins R. Adult Psychiatric Morbidity in England 2007: Results of a Household Survey. London, UK: NHS Information Centre for Health and Social Care; 2009.
    73. McCrone P, Dhanasiri S, Patel A, Knapp M, Lawton-Smith S. Paying the Price: The Cost of Mental Health Care in England to 2026. London, UK: Kings Fund; 2008.   URL: https:/​/www.​kingsfund.org.uk/​sites/​default/​files/​Paying-the-Price-the-cost-of-mental-health-care-England-2026-McCrone-Dhanasiri-Patel-Knapp-Lawton-Smith-Kings-Fund-May-2008_0.​pdf [accessed 2020-06-15]
    74. Kelders SM, Bohlmeijer ET, Van Gemert-Pijnen JE. Participants, usage, and use patterns of a web-based intervention for the prevention of depression within a randomized controlled trial. J Med Internet Res 2013;15(8):e172 [FREE Full text] [CrossRef] [Medline]
    75. Doherty G, Coyle D, Sharry J. Engagement with online mental health interventions: an exploratory clinical study of a treatment for depression. New York, NY: Association for Computing Machinery; 2012 Presented at: SIGCHI Conference on Human Factors in Computing Systems; May 2012; Austin, TX, USA p. 1421-1430. [CrossRef]
    76. Cook RF, Billings DW, Hersch RK, Back AS, Hendrickson A. A field test of a web-based workplace health promotion program to improve dietary practices, reduce stress, and increase physical activity: randomized controlled trial. J Med Internet Res 2007;9(2):e17 [FREE Full text] [CrossRef] [Medline]
    77. Cobb NK, Poirier J. Effectiveness of a multimodal online well-being intervention: a randomized controlled trial. Am J Prev Med 2014 Jan;46(1):41-48 [FREE Full text] [CrossRef] [Medline]

    ‎
    AAQ-II: Acceptance and Action Questionnaire—Revised
    ACT: acceptance and commitment therapy
    NHS: National Health Service
    PHQ-4: 4-item Patient Health Questionnaire
    WEMWBS: Warwick-Edinburgh Mental Well-Being Scale

    Edited by J Torous, G Eysenbach; submitted 06.03.20; peer-reviewed by P Lappalainen, J Vermeir; comments to author 08.04.20; revised version received 29.04.20; accepted 26.05.20; published 07.08.20

    Copyright

    ©Menna Brown, Nic Hooper, Phillip James, Darren Scott, Owen Bodger, Ann John. Originally published in JMIR Formative Research (http://formative.jmir.org), 07.08.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on http://formative.jmir.org, as well as this copyright and license information must be included.