This paper is in the following e-collection/theme issue:

Formative Evaluation of Digital Health Interventions (2226) Web-based and Mobile Health Interventions (3128) Depression and Mood Disorders; Suicide Prevention (1290) Methods and Feasibility Studies (369)

Published on in Vol 6, No 4 (2022): April

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/31835, first published .
Computerized Cognitive Behavioral Therapy Intervention for Depression Among Veterans: Acceptability and Feasibility Study

Computerized Cognitive Behavioral Therapy Intervention for Depression Among Veterans: Acceptability and Feasibility Study

Computerized Cognitive Behavioral Therapy Intervention for Depression Among Veterans: Acceptability and Feasibility Study

Authors of this article:

Kelly A Stearns-Yoder1, 2 Author Orcid Image ;   Arthur T Ryan1, 3, 4 Author Orcid Image ;   Alexandra A Smith1 Author Orcid Image ;   Jeri E Forster1, 2 Author Orcid Image ;   Sean M Barnes1, 3 Author Orcid Image ;   Lisa A Brenner1, 2, 3, 5 Author Orcid Image
Article Authors Cited by Tweetations (11) Metrics

    Original Paper

    1Veterans Affairs Rocky Mountain Mental Illness Research Education and Clinical Center, Rocky Mountain Regional Veterans Affairs Medical Center, Aurora, CO, United States

    2Department of Physical Medicine and Rehabilitation, University of Colorado Anschutz School of Medicine, Aurora, CO, United States

    3Department of Psychiatry, University of Colorado Anschutz School of Medicine, Aurora, CO, United States

    4Research and Development Service, Washington DC Veterans Affairs Medical Center, Washington, DC, United States

    5Department of Neurology, University of Colorado Anschutz School of Medicine, Aurora, CO, United States

    *all authors contributed equally

    Corresponding Author:

    Kelly A Stearns-Yoder, MA

    Veterans Affairs Rocky Mountain Mental Illness Research Education and Clinical Center

    Rocky Mountain Regional Veterans Affairs Medical Center

    1700 N Wheeling Street

    Aurora, CO, 80045

    United States

    Phone: 1 7204151717

    Email: kelly.stearns@va.gov


    Background: Computerized cognitive behavioral therapies (cCBTs) have been developed to deliver efficient, evidence-based treatment for depression and other mental health conditions. Beating the Blues (BtB) is one of the most empirically supported cCBTs for depression. The previous trial of BtB with veterans included regular guidance by health care personnel, which increased the complexity and cost of the intervention.

    Objective: This study, conducted by researchers at a Veterans Affairs Medical Center, aims to test the acceptability and feasibility of unguided cCBT for depression among US military veterans.

    Methods: To examine the acceptability of BtB delivered without additional peer or other mental health care provider support, a before-and-after trial was conducted among United States (US) military veterans experiencing mild to moderate depressive symptoms. The feasibility of the study design for a future efficacy trial was also evaluated.

    Results: In total, 49 veterans completed preintervention assessments and received access to BtB, and 29 participants completed all postintervention assessments. The predetermined acceptability criterion for the intervention was met. Although the predetermined feasibility criteria regarding screening eligibility rate, number of BtB modules completed, and completion of a posttreatment assessment were not met, the results were comparable with those of other cCBT studies.

    Conclusions: This is the first study among US military veterans to demonstrate support for the implementation of cCBT for depression without the assistance of a mental health professional or a peer support specialist, suggesting that stand-alone computer-aided interventions may be viable. Ideas for improving feasibility in future trials based on this study are discussed.

    JMIR Form Res 2022;6(4):e31835

    doi:10.2196/31835

    Keywords

    computerized cognitive behavioral therapy; depression; veterans; acceptability; feasibility 


    Background

    Depressive disorders are among the most common mental health conditions associated with high morbidity [1]. Compared with the general population, US military veterans are at an increased risk of depression [2]. Approximately 1 in 3 veterans visiting primary care clinics has some symptoms of depression, 1 in 5 has serious symptoms that indicate the need for further evaluation, and 1 in 9 requires psychotherapy or antidepressant treatment for major depression [3].

    Although depressive disorders are common and undertreated conditions [4-6], there are effective treatments, including cognitive behavioral therapy (CBT) and pharmacotherapy [7]. Research supports the notion that patients generally prefer psychological therapy to medication [8,9]. However, there is a critical need for trained mental health professionals to treat depressive disorders among individuals in the US, including the US veterans [10]. Although CBT training programs are available for providers within the Department of Veterans Affairs (VA), additional factors can limit individuals’ ability to engage in traditional face-to-face therapy, including the resources required to travel to the clinic for repeated appointments, which may not be available at the most convenient time [11]. Alternative methods have been developed to address limitations inherent to traditionally delivered psychotherapy and facilitate efficient, evidence-based, and appropriate treatment [12]. An alternative method for providing evidence-based psychotherapy to a larger population is the use of computer-aided therapies.

    In the past 40 years, efficacious computerized cognitive behavioral therapies (cCBTs) have been developed to treat depression and other mental health conditions. Most cCBT programs can be accessed using the internet (eg, from an individual’s home or a public library) and are highly interactive (eg, audio, video, and animations) [13]. cCBTs have the potential to address some accessibility concerns related to mental health care, as they can be administered at home during times that are most convenient for the patient, thereby reducing barriers to care, including those posed by geography, transportation costs, travel time, and childcare. Because cCBTs generally require less provider time, they can also be cost-effective alternatives to traditional face-to-face psychotherapy.

    Meta-analyses have found that cCBT for depression and anxiety disorders is similarly effective to traditional face-to-face CBT in reducing symptoms of depression and anxiety [14-16]. Posttreatment satisfaction with cCBT in the general population has also been studied [17,18] and is similar to that of face-to-face therapies [19,20]. Patients receiving cCBT reported being as satisfied as those receiving CBT from a clinician [21] and more satisfied than those receiving treatment as usual [22,23]. These findings suggest that cCBT is a viable alternative for treating common mental health symptoms.

    One of the most empirically supported cCBTs for depression is Beating the Blues (BtB), an 8-session, self-administered, and interactive cCBT program (see the Methods section). Although BtB is self-administered, the program offers a provider portal to track patient progress and facilitate health care provider or peer support to encourage treatment engagement. Participation in the BtB program has been associated with reductions in mild to moderate symptoms of depression and anxiety [24-26], and it has been recommended as the gold standard for treating mild to moderate symptoms of depression in primary care contexts [13,16,27]. In randomized controlled trials, BtB has also been established as a cost-effective intervention [16,28]. In addition, BtB has demonstrated effectiveness in addressing depression in older adults [29].

    A recent study has demonstrated the acceptability and feasibility of BtB with peer support for treating mild to moderate symptoms of depression in veterans receiving primary care or outpatient mental health services [30]. However, because the study intervention included an additional component of weekly interactions with a veteran peer support specialist, it is unclear whether the effects observed were associated with the BtB program, the peer support specialist, or the combination.

    Some studies have shown that veterans respond differently to evidence-based psychotherapies, including CBT for depression [31]. In terms of variables associated with treatment response, including psychiatric comorbidity and lack of stable housing, Veterans receiving health care from the VA differ from other US cohorts [32]. For example, comorbid posttraumatic stress disorder, which is more common in veterans than civilians, may reduce the effectiveness of depression treatments [31].

    Objectives

    This study aims to examine the acceptability and feasibility of delivering BtB without additional peer or mental health care provider support among veterans receiving care at a Veterans Affairs Medical Center (VAMC). Acceptability refers to the suitability of an intervention from the perspective of participants [33]. Feasibility refers to the goodness of fit between an intervention and the system in which it is disseminated. Aligned with the guidance provided by Areán and Kraemer [34], our feasibility assessment focused on the ease of implementation of study design elements, which were participant recruitment, enrollment, and retention. Before commencing the study, a set of a priori criteria (see the Acceptability and Feasibility Criteria and Analysis subsection in the Methods section) was established to measure the acceptability and feasibility per best practices in pilot study designs [35].


    Design

    All participants were allocated to receive the BtB intervention following a before-and-after trial design [36]. This study was approved by the Colorado Multiple Institutional Review Board and the local VA Research and Development Committee. A Health Insurance Portability and Accountability Act waiver was granted for the entire study, as all procedures were completed remotely (over the phone with the participant or on the web via a survey link). Data collection occurred between September 2019 and February 2020. After a research team member read the consent information sheet to the potential participants, verbal consent was obtained. Participants could also download a copy of the consent form via the Research Electronic Data Capture (REDCap; Vanderbilt University) [37] platform, a web-based portal for participants to complete self-report study measures. Following verbal consent, the interviewer conducted a telephone eligibility screening. If the participant was eligible for the study, the interviewer ended the telephone conversation and sent a link to a web-based survey of preintervention (baseline) measures to be completed on the REDCap platform. After completing the preintervention measures, participants were provided a link to register for and access the BtB web-based intervention. All the BtB content, including an introductory session and 8 content modules, could be completed autonomously via an internet browser interface without facilitation by a health care provider, peer, or study staff. Although the core content of BtB can be completed in as few as 8 weeks, participants were given access to the program for 12 weeks. After completing 8 modules or at the end of the 12 weeks, participants were emailed a second link to complete postintervention assessments via REDCap. All participants were compensated for completing the pre- and postintervention surveys.

    Participants

    Initially, veteran participants were recruited exclusively via referral from a Primary Care and Mental Health Integration (PC-MHI) psychologist. PC-MHI psychologists are embedded in VA Patient Aligned Care Teams within primary care clinics to provide brief mental health assessments, referrals, and interventions. The study team received 45 referrals from the PC-MHI psychologist, and 29 veterans consented to participate in the study. Of the 29 veterans, 18 (62%) did not complete any BtB modules, 4 (14%) were lost to follow-up, and 7 (24%) completed postintervention surveys. Because the study did not meet its targets for recruitment, enrollment, and data collection, the study team implemented a new recruitment strategy that involved identifying potential participants using electronic medical records (EMRs). This paper does not report data from participants recruited via PC-MHI psychologist referral because they might systematically differ from those recruited using EMRs. In addition, we may use the EMR recruitment method in future clinical trials based on this feasibility and acceptability study.

    For recruitment using EMRs, potential participants were identified from the VA Corporate Data Warehouse. The records were electronically searched for veterans (1) eligible to receive care at a particular VAMC between August 2017 and July 2019 who (2) were administered the Patient Health Questionnaire-9 (PHQ-9), a brief self-report measure of depression and (3) received a total score between 5 and 15 points, which indicates the presence of mild to moderate symptoms of depression [38]. Considering previous studies supporting the efficacy of cCBT in the treatment of depression and the statistical relationship between depression and suicidal behavior among veterans [39], we chose to recruit veterans based on symptoms of depression instead of other symptomatology. Veterans meeting these criteria were sent a letter inviting them to participate in the study. The letter briefly described the study and instructed interested veterans to call a research team member to be screened for eligibility.

    During the telephone screening, potential participants self-reported whether they met the following inclusion criteria: (1) being a US military veteran aged between 18 and 89 years; (2) having a score of 5 to 15 on a verbally administered PHQ-9, which determined whether the veteran currently endorsed mild to moderate depressive symptoms [38]; (3) able to write, read, and speak English; and (4) having reliable access to the internet. The following exclusion criteria were also assessed: (1) inability to complete the assessment sessions or participate in the intervention because of visual or hearing impairment, severe psychiatric symptoms (eg, active psychosis or imminent suicide risk), or severe cognitive impairment; (2) membership in a vulnerable population (eg, pregnant women and prisoners); and (3) self-reported current participation in another mental health intervention study. The telephone-administered PHQ-9 was only used for screening purposes; another PHQ-9 was administered electronically via REDCap along with the other preintervention assessments to collect the data used in the study analyses.

    Measures

    The measures administered during pre- and postintervention are shown in Multimedia Appendix 1; Table S1.

    Client Satisfaction Questionnaire

    The Client Satisfaction Questionnaire (CSQ) [40] is an 8-item questionnaire used to assess participants’ satisfaction with an intervention. Scores range from 8 to 32, with higher scores indicating greater satisfaction. A score of 24 or higher indicates that the average item rating was in the mostly satisfied or better range. The CSQ has good reliability and validity and has been frequently used to evaluate mental health care [41]. The CSQ was administered during the postintervention assessment and was a measure of acceptability.

    Generalized Anxiety Disorder Inventory-7

    The Generalized Anxiety Disorder Inventory-7 (GAD-7) [41] is a 7-item self-report measure that assesses symptoms related to generalized anxiety disorder. Participants respond to items using a 4-point Likert scale ranging from 0 to 3 (not at all to nearly every day, respectively). Participants endorse the items based on how they felt in the last 2 weeks. Higher scores reflect greater symptoms of generalized anxiety. The GAD-7 has demonstrated good reliability and validity [41]. As our trial was specifically designed to examine BtB acceptability and feasibility and not as an efficacy trial (eg, it did not include a control condition), only baseline GAD-7 scores are reported in this paper.

    Internet Evaluation and Utility Questionnaire

    The Internet Evaluation and Utility Questionnaire (IEUQ) [42] is a 13-item measure that examines the participants’ experiences of a web-based intervention. The IEUQ was adapted for this study to refer specifically to BtB. The constructs measured included items on ease of use, convenience, engagement, enjoyment, layout, privacy, and overall satisfaction. The IEUQ was administered to gather further information on the participants’ impressions of the aspects of the intervention that were relevant to its acceptability (eg, convenience). Participants responded to items on a 5-point Likert scale from 0 (not at all) to 4 (very) or rated them as not applicable. Previous research indicates that the IEUQ has adequate reliability [42,43].

    Internet Impact and Effectiveness Questionnaire

    The Internet Impact and Effectiveness Questionnaire (IIEQ) [42] is a 20-item instrument that measures individuals’ perceptions of the effectiveness of a web-based intervention. The perceived impact is measured in terms of helpfulness, knowledge gains, treatment effectiveness for self and others, long-term effectiveness, quality of life, mood, physical activity, family and peer relationships, social activity, school or work attendance and performance, treatment implementation, goal orientation, confidence in the ability to manage conditions, relapse prevention, and service reduction. The perceived or actual effectiveness of the intervention is likely to influence patients’ perceptions of acceptability [44]. This measure was adapted for this study to refer specifically to BtB. Participants responded to items using a 5-point Likert scale ranging from 0 (not at all) to 4 (very). There was also a not applicable option. Adequate psychometric properties have been demonstrated previously [42,43].

    PHQ-9 Scores

    The PHQ-9 is a 9-item measure that assesses symptoms of depression. Participants respond to items using a 4-point Likert scale ranging from 0 to 3 (not at all to nearly every day, respectively), endorsing them based on how they felt in the last 2 weeks. Higher scores reflect greater symptoms of depression. The PHQ-9 has demonstrated good psychometric properties [38]. Similar to the GAD-7, only baseline PHQ-9 scores are reported in this paper.

    Reasons for Termination–Adapted

    The reasons for termination (RFT) scale [45] assesses 10 common reasons why patients terminate therapy and the impact these reasons have on termination. Thus, the RFT scale gathers information about the elements of a therapy that reduce its acceptability. If a participant completed between 1 and 7 modules of the BtB treatment within 12 weeks of the initial log-in to the BtB treatment website (ie, began but did not finish the therapy within the allotted time), this measure was included with the postintervention measures collected via a REDCap survey link.

    Intervention

    BtB is a computerized cognitive behavioral intervention program aimed at reducing depressive and anxiety symptoms [23]. BtB was designed explicitly to implement the standard CBT model of depression and anxiety and include the therapeutic elements that make up traditional CBT psychotherapy (eg, thought records [46]). The web-based intervention consisted of a 15-minute introductory session followed by 8 interactive modules, usually taken weekly. Each weekly module lasted approximately 50 minutes, with homework projects to complete between the modules (eg, problem diaries, thought records, and behavioral experiments).

    Acceptability and Feasibility Criteria and Analysis

    The a priori criterion for assessing the acceptability of the intervention was that ≥70% of the participants had a score of ≥24 on the CSQ. A score of ≥24 was selected as the cutoff because it indicates that the average item rating was in the mostly satisfied or better range. Additional acceptability data were collected from the IIEQ, IEUQ, and RFT. The a priori criteria for assessing the feasibility of the study were (1) ≥70% of potential participants would meet the eligibility criteria assessed during the telephone screening, (2) ≥70% of those found eligible would consent to participate in the study, (3) ≥60% of those who completed the preintervention survey would complete all the intervention modules, and (4) ≥75% of those who registered for BtB and completed at least one module would complete the postintervention survey measures.

    Study Procedures

    Study recruitment, enrollment, and data collection were conducted at multiple time points (Figure 1).

    ‎
    Figure 1. Enrollment flow diagram. PHQ-9: Patient Health Questionnaire-9.
    View this figure

    Baseline Demographics

    Of the 54 veterans recruited via EMRs who consented to participate in the study, 49 (91%) used the emailed link to complete the baseline assessment measures. The sample was predominantly male (43/49, 88%) and White (37/49, 76%). At baseline, most participants reported moderate depression (ie, a PHQ-9 score in the moderate depression range of 10-14) and mild anxiety (ie, a GAD-7 score in the mild anxiety score range of 5-9). Further details regarding demographic characteristics are shown in Table 1.

    Table 1. Baseline demographics (N=49).
    VariableValue
    Age (years), mean (SD); range57.1 (9.9); 35-74
    Self-identified gender, n (%)

    		Male43 (88)

    		Female6 (12)
    Racial background, n (%)

    		White37 (76)

    		Black or African American7 (14)

    		Native American or Alaskan3 (6)

    		Asian1 (2)

    		Other1 (2)
    Ethnicity, n (%)

    		Hispanic or Latinx7 (14)

    		Non-Hispanic or Non-Latinx41 (84)

    		Refused to respond1 (2)
    Highest level of education, n (%)

    		High-school diploma or equivalent4 (8)

    		Some college, no degree18 (37)

    		Associate’s degree7 (14)

    		Bachelor’s degree9 (18)

    		Master’s degree8 (16)

    		Doctoral degree3 (6)
    Marital or relationship status, n (%)

    		Married28 (57)

    		Single10 (20)

    		Cohabiting1 (2)

    		Divorced or separated10 (20)
    Sexual orientation, n (%)

    		Heterosexual49 (100)
    Employment status, n (%)

    		Employed full-time13 (27)

    		Employed part-time4 (8)

    		Unemployed, not currently seeking employment8 (16)

    		Unemployed, seeking employment1 (2)

    		Retired22 (45)

    		Refused to respond1 (2)
    Currently a student, n (%)2 (4)
    Currently homeless, n (%)0 (0)
    Ever homeless, n (%)16 (30)
    Branch of military service, n (%)

    		Army—active duty21 (43)

    		Army reserve5 (10)

    		Army national guard3 (6)

    		Air force—active duty13 (27)

    		Air force reserve7 (14)

    		Air national guard5 (10)

    		Navy—active duty8 (16)

    		Navy reserve3 (6)

    		Marine corps—active duty7 (14)

    		Marine corps reserve2 (4)

    		Coast guard—active duty1 (2)
    Total months of active duty service, mean (SD)78.5 (73.4)
    Total months of reserve service, mean (SD)24.4 (54.9)
    Service era, n (%)

    		Vietnam (August 1964-May 1975)13 (27)

    		Post-Vietnam/Peacetime (May 1975-July 1990)24 (49)

    		Desert-Storm/Desert-Shield (August 1990-Aug 2001)22 (45)

    		OEFa/OIFb/ONDc (September 2001-Present)14 (29)

    		Other3 (6)
    Highest rank at separation or current rank, n (%)

    		Enlisted35 (71)

    		Noncommissioned officer10 (20)

    		Officer4 (8)
    Number of deployments, mean (SD)2.2 (3.3)
    Number of combat tours, mean (SD)0.9 (1.2)
    PHQ-9d at baseline, mean (SD)12.1 (4.7)
    GAD-7e at baseline, mean (SD)9.0 (4.8)

    aOEF: Operation Enduring Freedom.

    bOIF: Operation Iraqi Freedom.

    cOND: Operation New Dawn.

    dPHQ-9: Patient Health Questionnaire-9.

    eGAD-7: Generalized Anxiety Disorder Inventory-7.

    Acceptability

    The CSQ was used to examine whether participants found the BtB intervention acceptable, which was operationalized as a CSQ score of ≥24. Among the 35 participants who completed the CSQ, 26 (74%; 95% CI 57%-88%) received a score of ≥24, which met the a priori criterion of ≥70% (CSQ mean 25.2, SD 4.8).

    Acceptability data were also collected using the IEUQ, IIEQ, and RFT. The responses to the IEUQ are presented in Table 2. On the IEUQ, many participants reported liking BtB and being satisfied with the intervention. Data from the IEUQ suggested that BtB was mostly or very easy to use (23/30, 77%) and generally kept participants’ attention. On the IEUQ, most of the participants (21/30, 70%) found that the internet was a mostly or very good method for delivering CBT. In addition, participants (18/30, 60%) reported they were mostly or very likely to come back to BtB if depression difficulties continued or returned.

    Responses to the IIEQ regarding the perceived effectiveness of BtB are shown in Table 3. On the IIEQ, most participants reported that BtB somewhat or mostly improved depressive symptoms (18/29, 62%), quality of life (15/29, 51%), and mood (15/29, 51%). Participants felt they somewhat or mostly gained more knowledge while using BtB (21/29, 72%), and they were somewhat or mostly prepared to handle depressive symptoms in the future (16/29, 55%). In addition, they were mostly or very likely to recommend BtB to others with similar problems (20/29, 69%).

    Although many participants found the intervention acceptable, some did not complete all the 8 modules of the intervention (Figure 1). The RFT questionnaire was administered to participants who completed between 1 and 7 modules (Table 4). Some participants reported that the barriers to completing all modules included practical problems (3/14, 21%), time constraints (5/14, 36%), and medical reasons (4/14, 29%). None of the participants who completed the RFT reported that they chose to stop the intervention because they were dissatisfied or wanted a different intervention.

    Table 2. Responses to the Internet Evaluation and Utility Questionnaire after the assessment (n=30).
    DomainResponse, n (%)

    		Not at allSlightlySomewhatMostlyVeryNot applicable or no response
    Easy to use0 (0)2 (7)3 (10)9 (30)14 (47)2 (7)
    Convenient to use0 (0)1 (3)3 (10)8 (27)14 (47)4 (13)
    Keeps interest and attention0 (0)1 (3)7 (23)17 (57)3 (10)2 (7)
    Liked BtBa1 (3)2 (7)4 (13)10 (33)10 (33)3 (10)
    Liked how BtB looked0 (0)1 (3)4 (13)9 (31)9 (31)7 (23)
    Worried about privacy17 (57)5 (17)4 (13)2 (7)1 (3)1 (3)
    Satisfied2 (7)2 (7)4 (13)11 (37)9 (30)20 (7)
    Good fit1 (3)2 (7)8 (27)8 (27)9 (30)2 (7)
    Usefulness0 (0)2 (7)3 (10)8 (27)13 (43)4 (13)
    Easy to understand0 (0)0 (0)5 (17)3 (10)19 (63)3 (10)
    Trust the information0 (0)0 (0)2 (7)11 (37)13 (43)4 (13)
    Likely to come back2 (7)1 (3)7 (23)7 (23)11 (37)2 (7)
    Liked delivery via internet1 (3)3 (10)3 (10)4 (13)17 (57)2 (7)

    aBtB: Beating the Blues.

    Table 3. Responses to the Internet Impact and Effectiveness Questionnaire after the assessment (n=29).
    DomainResponsea, n (%)

    		Not at allSlightlySomewhatMostlyVeryNot applicable or no response
    Improved depressive symptoms1 (3)6 (21)10 (34)8 (28)3 (10)1 (3)
    More knowledge0 (0)4 (14)9 (31)12 (41)2 (7)2 (7)
    How well it worked1 (3)5 (17)9 (31)9 (31)5 (17)0 (0)
    How well it can work for others0 (0)2 (7)6 (21)7 (24)6 (21)8 (28)
    Work as long-term cure2 (7)3 (10)11 (38)7 (24)4 (14)2 (7)
    Improved quality of life1 (3)7 (24)12 (41)3 (10)3 (10)3 (10)
    Improved mood1 (3)7 (24)12 (41)3 (10)3 (10)3 (10)
    Improved physical activities5 (17)9 (31)7 (24)5 (17)1 (3)2 (7)
    Improved relationships with family3 (10)9 (31)5 (17)2 (7)4 (14)6 (21)
    Improved relationships with friends, peers, or coworkers1 (3)11 (38)8 (28)3 (10)1 (3)5 (17)
    Improved social life, such as visiting friends and engaging in community activities5 (17)7 (24)8 (28)5 (17)1 (3)3 (10)
    Improved school or work attendance4 (14)5 (17)0 (0)1 (3)2 (7)17 (59)
    Improve school or work performance4 (14)7 (24)0 (0)1 (3)2 (7)15 (52)
    Able to follow through with BtBb recommendations1 (3)7 (24)5 (17)10 (34)5 (17)1 (3)
    Able to reach goals at beginning of BtB3 (10)6 (21)10 (34)7 (24)2 (7)1 (3)
    Help feel more confident to manage depressive symptoms2 (7)4 (14)11 (38)6 (21)4 (14)2 (7)
    Likely to recommend BtB to others with similar problems1 (3)2 (7)4 (14)7 (24)13 (45)2 (7)
    Prepared to handle depressive symptoms in future1 (3)6 (21)7 (24)9 (31)5 (14)2 (7)
    Reduce the number of office visits with a health professional8 (28)7 (24)2 (7)3 (10)3 (10)6 (21)
    Reduce the number of phone calls and emails with a health professional5 (17)3 (10)4 (14)2 (7)4 (14)11 (38)

    aA participant completed the Internet Evaluation and Utility Questionnaire but stopped filling out the postintervention assessments before completing the Internet Impact and Effectiveness Questionnaire.

    bBtB: Beating the Blues.

    Table 4. Responses to the reasons for termination questionnaire at the postintervention assessment (n=14).
    DomainResponse, n (%)

    		YesNoNot applicable or no response
    Practical problems3 (21)10 (71)1 (7)
    Time problems5 (36)8 (57)1 (7)
    Medical reasons4 (29)9 (64)1 (7)
    Problems improved and no longer felt a need for BtBa0 (0)12 (86)2 (14)
    Not improving as much as wanted to2 (14)10 (71)2 (14)
    Dissatisfied with BtB0 (0)11 (79)3 (21)
    Wanted a different intervention0 (0)10 (71)4 (29)
    Pressured or advised by others (eg, friends, spouse, or other people who criticized participation in BtB or said they did not need it)0 (0)13 (93)1 (7)
    Afraid that employer or others would find out about participation in BtB0 (0)13 (93)1 (7)
    Other reasons that led to ending participation6 (43)6 (43)2 (14)

    aBtB: Beating the Blues.

    Feasibility

    Of the 94 veterans identified via EMRs and screened, 59% (55/94) were eligible for the study, which did not meet the a priori feasibility criterion of ≥70%. Of those eligible, 98% (54/55) consented to participate in the study (Figure 1), meeting the a priori feasibility criterion of ≥70%, and 49 (91%) of them used the emailed link to complete the preintervention assessments. A participant did not register with the BtB program after completing the preintervention assessment, leaving 48 participants in the study. Of these, 33% (16/48) completed all 8 BtB modules, which did not meet the a priori feasibility criterion of ≥60%. Among the other registered participants, 19% (9/48) completed 5 to 7 modules, 38% (18/48) completed 1 to 4 modules, and 10% (5/48) did not complete any module after registering for BtB. Participants were not eligible to complete the postintervention survey if they did not register with BtB and complete at least one module. Of the 43 participants who completed at least one BtB module, 67% (29/43) completed all postintervention measures, which did not meet the a priori criterion of ≥75%, and 6 (14%) individuals partially completed the postintervention survey, meaning that 81% (35/43) of participants completed at least part of it. Of note, based on demographics and preintervention measures, participants who completed this survey did not differ significantly from those who did not (Multimedia Appendix 1; Table S2).


    Principal Findings

    The results of this pilot study indicate that the a priori acceptability criterion for the intervention was met. Veterans generally found BtB helpful and easy to use. Most of the veterans who completed the postintervention assessment reported that they were able to complete the tasks associated with the intervention, that they found the intervention helpful, and that they would recommend it to other veterans with similar problems. The observed module completion rate was similar to the rates found with other unguided cCBT interventions [47] and with BtB in non-Veterans [48]. The completion rate was also similar to that found in a previous study of a peer-supported implementation of BtB in veterans [30]. Among the veterans who completed the RFT measure, the most common reasons for treatment discontinuation were related to external factors (eg, time constraints). None of the veterans reported discontinuing because of dissatisfaction with the treatment.

    Concerning feasibility, three of four a priori criteria were not satisfied. We believe that a reason for this outcome is that, in retrospect, the criteria selected before the pilot study were overly conservative. In this study, participant recruitment via EMRs occurred rapidly (within 2 months), which did not allow for larger modifications to aspects of the design (eg, the retention plan) while the study was being conducted. However, in line with the advice of Thabane et al [35], conducting this pilot study has suggested several modifications to the study design that we will implement in future iterations of this research protocol.

    One of the initial feasibility criteria was that ≥70% of veterans referred to the study would meet the secondary screening criteria, which included current mild to moderate depressive symptoms. As depression is a waxing and waning condition [49], it is unsurprising that many veterans’ symptoms improved or worsened between the initial administration of the PHQ-9 recorded in the EMR and when they were screened for this study. Furthermore, undertaking this pilot trial suggested that only modest experimenter and participant efforts were required to conduct the telephone screening to confirm the presence of current mild to moderate symptoms of depression. Given these considerations, we concluded that the 59% eligibility rate among veterans who received a telephone screening was more than sufficient to demonstrate the feasibility of enrolling this population in future larger efficacy trials. Additional feasibility-related modifications of the study protocol to increase the percentage of veterans with eligible PHQ-9 scores at phone screening (eg, requiring a more recent PHQ-9 administration in the EMR) would not be required.

    Another a priori feasibility criterion was that ≥60% of veterans would complete all the 8 modules associated with the treatment. In retrospect, both the percentage of veterans required to complete the modules and the number of modules required by this criterion were overly conservative. Furthermore, we believe that easily implemented changes to the protocol (described below) may further increase the mean number of modules completed. Regarding the number of modules required, a more appropriate number might have been derived from the previous study of BtB in veterans [30], which found that those who completed 5 or more modules of BtB showed statistically significant improvements in symptoms. In our sample, 52% (25/48) of veterans completed 5 or more modules, compared with only 33% (16/48) who completed all 8 modules. Although a higher percentage of veterans met the 5 or more module completion rate criteria, the fact that this 52% completion rate is still less than the 60% rate that we targeted suggests that we should consider modifications to the study protocol to increase the proportion of veterans who complete at least five modules. Of relevance to identifying potentially useful protocol modifications, a previous study found that simple weekly email reminders to complete web-based psychotherapy modules increased the mean number of modules completed by 50% (from 3.7 modules to 5.5 modules) [50]. Researchers have obtained further increases in the efficacy of such reminder emails by adjusting their content (eg, sending emails that inform the user of new content rather than simply reminding them to complete a session) and timing (eg, emailing users after 2 weeks of absence rather than longer periods) [51]. In future iterations of this protocol, we will implement such email reminders to improve the mean number of modules completed. It should be noted that only 40% of veterans completed at least five modules in the previous trial of BtB that used peer support specialists to facilitate the completion of BtB modules [30] compared with 52% (25/48) in our unfacilitated trial, suggesting that the greater investment of resources required to include peer support specialists in the intervention may not necessarily increase the average number of modules completed by participants.

    Regarding the criterion for the proportion of participants who needed to complete the required number of modules, we believe that setting this criterion to ≥60% of participants was also too conservative. A core advantage of BtB (and other cCBT interventions) is that it requires less time and resources to be administered to an individual, both from the perspective of veterans and the health care system. Therefore, whereas traditionally implemented treatments might indeed require a low noncompletion rate to justify their research or clinical use, a low-intensity intervention such as BtB probably may not. To reiterate, we believe that the study protocol should be modified to increase the mean number of modules completed. However, we also conclude that a lower minimum completion rate might have been a more useful feasibility criterion given the low veteran and provider burden associated with BtB.

    The final a priori feasibility criterion that was not met required that ≥75% of veterans complete the postintervention assessment. In reflecting upon our pilot study, we believe that some small alternations to the experimental design could have improved the observed postintervention survey completion rate of 67%. Previous studies on methods to increase participant retention and assessment completion suggest that using one or more of the following strategies might have increased the rates of postintervention survey completion: a telephone follow-up to the assessment invitation; shortened versions of questionnaires where possible; and, reminders to nonrespondents [52]. We also note that the literature on methods to improve participant retention is growing, and efforts to systematically identify efficacious retention strategies are underway [53].

    To our knowledge, this study is the first to demonstrate that a substantial percentage of veterans can complete BtB without the assistance of a mental health professional or peer support specialist [30]. This is important because one of the main virtues of cCBT interventions is the possibility of providing evidence-based CBT interventions to a larger number of individuals [54]. A strength of our study was its broad eligibility and few exclusion criteria, which allowed us to recruit a veteran sample that was relatively representative of the broader population of veterans to whom a BtB intervention might be provided in regular clinical practice (eg, veterans receiving care at a VAMC and reporting mild to moderate symptoms of depression).

    Limitations

    Our study has several important limitations. First, the relatively small number of veteran participants who were women, non-White, or Hispanic did not represent the entire US veteran population [55] and limited the ability to demonstrate the acceptability of BtB in these important populations. However, other studies have generally shown that cCBT interventions, including BtB, have similar acceptability among women and members of minority groups [56]. Second, veterans with more severe depression (scores >15 on the PHQ-9) were excluded from our study, limiting our ability to draw conclusions regarding the feasibility of the design and acceptability of the intervention among veterans with more severe symptoms of depression. However, this exclusion criterion is consistent with the practice of health care systems where cCBT has already been successfully implemented (eg, in the United Kingdom [57]). In these systems, cCBT (and other less resource-intensive interventions) are initially provided to a larger population of individuals with mild to moderate depression symptoms, whereas a smaller population of high-acuity patients receive more resource-intensive interventions, such as in-person psychotherapy [58]. Finally, 17% (8/48) of the participants who registered with BtB did not complete the postassessment surveys. It is possible that their responses to the CSQ would have suggested that they did not find the intervention acceptable.

    Future Research and Conclusions

    These findings suggest that further studies on cCBT interventions for treating depressive symptoms in veterans are warranted. Furthermore, the results suggest that unguided implementation of such interventions (ie, those that do not include regular interactions with a health care provider) may be a viable treatment delivery modality, at least for a significant proportion of veterans. Future research is needed to further develop cCBT interventions for veterans. Specifically, research is needed to establish the relative efficacy of cCBTs compared with standard face-to-face therapy among veterans, as well as with other treatment modalities that improve access to evidence-based psychotherapy for depression (eg, group-therapy delivery formats). Future research could also investigate the differences between veterans who may benefit from cCBTs on their own and those for whom additional support (eg, contact with peer support specialists) could help with successful therapy completion [58]. Finally, future research is required to empirically establish which veterans are most likely to benefit from cCBT interventions and which require other treatment modalities.

    Acknowledgments

    This project was supported in part by the Veterans Health Administration Rocky Mountain Mental Illness Research Center.

    Disclaimer

    The views, opinions and findings contained in this article are those of the author(s) and should not be construed as an official Department of Defense or Veterans Affairs position, policy, or decision unless so designated by other documentation.

    Conflicts of Interest

    LAB reports grants from the Veterans Affairs, Department of Defense, National Institutes of Health, and the State of Colorado, editorial renumeration from Wolters Kluwer, and royalties from the American Psychological Association and Oxford University Press. In addition, she consults with sports leagues via her university affiliation.

    ‎
    Multimedia Appendix 1

    Measures and administration and comparison of participants completing the postintervention survey versus not completing postintervention survey.

    DOCX File , 20 KB
    1. Reddy M. Depression: the disorder and the burden. Indian J Psychol Med 2010 Jan 01;32(1):1-2 [FREE Full text] [CrossRef] [Medline]
    2. Seal KH, Metzler TJ, Gima KS, Bertenthal D, Maguen S, Marmar CR. Trends and risk factors for mental health diagnoses among Iraq and Afghanistan veterans using department of veterans affairs health care, 2002–2008. Am J Public Health 2009 Sep;99(9):1651-1658. [CrossRef]
    3. VA research on depression. U.S. Department of Veterans Affairs.   URL: https://www.research.va.gov/topics/depression.cfm [accessed 2022-03-01]
    4. Kessler RC, Berglund P, Demler O, Jin R, Koretz D, Merikangas KR, National Comorbidity Survey Replication. The epidemiology of major depressive disorder: results from the National Comorbidity Survey Replication (NCS-R). JAMA 2003 Jun 18;289(23):3095-3105. [CrossRef] [Medline]
    5. Kessler R, Wang P. Epidemiology of depression. In: Handbook of Depression. New York, United States: Guilford Press; 2009.
    6. Richards D. Prevalence and clinical course of depression: a review. Clin Psychol Rev 2011 Nov;31(7):1117-1125. [CrossRef] [Medline]
    7. Depression: The Treatment and Management of Depression in Adults (Updated Edition). London, United Kingdom: The British Psychological Society and The Royal College of Psychiatrists; 2016.
    8. Angermeyer MC, Matschinger H. Public attitude towards psychiatric treatment. Acta Psychiatr Scand 1996 Nov;94(5):326-336. [CrossRef] [Medline]
    9. Tylee A. Major depressive disorder (MDD) from the patient's perspective: overcoming barriers to appropriate care. Int J Psychiatry Clin Pract 2001 Jul 12;5(1):37-42. [CrossRef] [Medline]
    10. Studwell K. The mental health needs of veterans, service members and their families. American Psychological Association.   URL: https://www.apa.org/advocacy/military-veterans/mental-health-needs.pdf [accessed 2022-03-01]
    11. Evaluation of the Department of Veterans Affairs Mental Health Services. Washington, DC: The National Academies Press; 2018.
    12. Lovell K, Richards D. Multiple access points and levels of entry (MAPLE): ensuring choice, accessibility and equity for cbt services. Behav Cognit Psychother 2001 Jan 17;28(4):379-391. [CrossRef]
    13. Cavanagh K, Shapiro DA, Van Den Berg S, Swain S, Barkham M, Proudfoot J. The acceptability of computer-aided cognitive behavioural therapy: a pragmatic study. Cogn Behav Ther 2009;38(4):235-246. [CrossRef] [Medline]
    14. Andrews G, Basu A, Cuijpers P, Craske M, McEvoy P, English C, et al. Computer therapy for the anxiety and depression disorders is effective, acceptable and practical health care: an updated meta-analysis. J Anxiety Disord 2018 Apr;55:70-78 [FREE Full text] [CrossRef] [Medline]
    15. Cavanagh K, Seccombe N, Lidbetter N. The implementation of computerized cognitive behavioural therapies in a service user-led, third sector self help clinic. Behav Cognit Psychother 2011 Feb 22;39(4):427-442. [CrossRef]
    16. Learmonth D, Trosh J, Rai S, Sewell J, Cavanagh K. The role of computer-aided psychotherapy within an NHS CBT specialist service. Counsell Psychother Res 2008 Jun;8(2):117-123. [CrossRef]
    17. Williams C, Whitfield G, Hinshelwood R, Pashley A, Campsie L. The impact of a novel computerized CBT CD-Rom (overcoming depression) offered to patients referred to clinical psychology. European Psychiatry 2007 Mar;22:S248. [CrossRef]
    18. Wright JH, Wright AS, Albano AM, Basco MR, Goldsmith LJ, Raffield T, et al. Computer-assisted cognitive therapy for depression: maintaining efficacy while reducing therapist time. Am J Psychiatry 2005 Jun;162(6):1158-1164. [CrossRef] [Medline]
    19. Hemmings A. Counselling in primary care: a randomised controlled trial. Patient Educ Counsel 1997 Nov;32(3):219-230. [CrossRef]
    20. Seligman ME. The effectiveness of psychotherapy: the Consumer Reports study. Am Psychologist 1995 Dec;50(12):965-974. [CrossRef]
    21. Marks IM, Kenwright M, McDonough M, Whittaker M, Mataix-Cols D. Saving clinicians' time by delegating routine aspects of therapy to a computer: a randomized controlled trial in phobia/panic disorder. Psychol Med 2004 Jan 14;34(1):9-17. [CrossRef] [Medline]
    22. King M, Sibbald B, Ward E, Bower P, Lloyd M, Gabbay M, et al. Randomised controlled trial of non-directive counselling, cognitive-behaviour therapy and usual general practitioner care in the management of depression as well as mixed anxiety and depression in primary care. Health Technol Assess 2000;4(19):1-83 [FREE Full text] [Medline]
    23. Proudfoot J, Ryden C, Everitt B, Shapiro DA, Goldberg D, Mann A, et al. Clinical efficacy of computerised cognitive-behavioural therapy for anxiety and depression in primary care: randomised controlled trial. Br J Psychiatry 2004 Jul 01;185(1):46-54. [CrossRef] [Medline]
    24. Ormrod JA, Kennedy L, Scott J, Cavanagh K. Computerised cognitive behavioural therapy in an adult mental health service: a pilot study of outcomes and alliance. Cogn Behav Ther 2010 Sep;39(3):188-192. [CrossRef] [Medline]
    25. van den Berg S, Shapiro DA, Bickerstaffe D, Cavanagh K. Computerized cognitive-behaviour therapy for anxiety and depression: a practical solution to the shortage of trained therapists. J Psychiatr Ment Health Nurs 2004 Oct;11(5):508-513. [CrossRef] [Medline]
    26. Proudfoot J, Goldberg D, Mann A, Everitt B, Marks I, Gray JA. Computerized, interactive, multimedia cognitive-behavioural program for anxiety and depression in general practice. Psychol Med 2003 Feb 14;33(2):217-227. [CrossRef] [Medline]
    27. Computerised cognitive behaviour therapy for depression and anxiety. National Institute for Health and Care Excellence.   URL: https://www.nice.org.uk/guidance/ta97 [accessed 2022-03-01]
    28. McCrone P, Knapp M, Proudfoot J, Ryden C, Cavanagh K, Shapiro DA, et al. Cost-effectiveness of computerised cognitive-behavioural therapy for anxiety and depression in primary care: randomised controlled trial. Br J Psychiatry 2004 Jul 2;185(01):55-62. [CrossRef] [Medline]
    29. McMurchie W, Macleod F, Power K, Laidlaw K, Prentice N. Computerised cognitive behavioural therapy for depression and anxiety with older people: a pilot study to examine patient acceptability and treatment outcome. Int J Geriatr Psychiatry 2013 Nov;28(11):1147-1156. [CrossRef] [Medline]
    30. Pfeiffer PN, Pope B, Houck M, Benn-Burton W, Zivin K, Ganoczy D, et al. Effectiveness of peer-supported computer-based CBT for depression among veterans in primary care. Psychiatr Serv 2020 Mar 01;71(3):256-262. [CrossRef] [Medline]
    31. Hundt NE, Barrera TL, Robinson A, Cully JA. A systematic review of cognitive behavioral therapy for depression in veterans. Military Med 2014 Sep;179(9):942-949. [CrossRef]
    32. Haibach JP, Haibach MA, Hall KS, Masheb RM, Little MA, Shepardson RL, et al. Military and veteran health behavior research and practice: challenges and opportunities. J Behav Med 2017 Feb 27;40(1):175-193. [CrossRef] [Medline]
    33. Feeley N, Cossette S, Côté J, Héon M, Stremler R, Martorella G, et al. The importance of piloting an RCT intervention. Can J Nurs Res 2009 Jun;41(2):85-99. [Medline]
    34. Arean P, Kraemer H. High Quality Psychotherapy Research: From Conception to Piloting to National Trials. Oxford, United Kingdom: Oxford University Press; 2013.
    35. Thabane L, Ma J, Chu R, Cheng J, Ismaila A, Rios LP, et al. A tutorial on pilot studies: the what, why and how. BMC Med Res Methodol 2010 Jan 06;10:1. [CrossRef] [Medline]
    36. Des Jarlais DC, Lyles C, Crepaz N. Improving the reporting quality of nonrandomized evaluations of behavioral and public health interventions: the TREND statement. Am J Public Health 2004 Mar;94(3):361-366. [CrossRef]
    37. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009 Apr;42(2):377-381 [FREE Full text] [CrossRef] [Medline]
    38. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001 Sep;16(9):606-613 [FREE Full text] [CrossRef] [Medline]
    39. Mental health. U.S. Department of Veterans Affairs.   URL: https://www.mentalhealth.va.gov/docs/data- [accessed 2022-03-01]
    40. Attkisson C, Zwick R. The client satisfaction questionnaire. Evaluation Program Planning 1982 Jan;5(3):233-237. [CrossRef]
    41. Spitzer RL, Kroenke K, Williams JB, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med 2006 May 22;166(10):1092-1097. [CrossRef] [Medline]
    42. Ritterband LM, Ardalan K, Thorndike FP, Magee JC, Saylor DK, Cox DJ, et al. Real world use of an internet intervention for pediatric encopresis. J Med Internet Res 2008 Jun 30;10(2):e16 [FREE Full text] [CrossRef] [Medline]
    43. Thorndike F, Saylor D, Bailey E, Gonder-Frederick L, Morin C, Ritterband L. Development and perceived utility and impact of an internet intervention for insomnia. E J Appl Psychol 2008 Dec 23;4(2):32-42 [FREE Full text] [CrossRef] [Medline]
    44. Sekhon M, Cartwright M, Francis JJ. Acceptability of healthcare interventions: an overview of reviews and development of a theoretical framework. BMC Health Serv Res 2017 Jan 26;17(1):88 [FREE Full text] [CrossRef] [Medline]
    45. Todd DM, Deane FP, Bragdon RA. Client and therapist reasons for termination: a conceptualization and preliminary validation. J Clin Psychol 2003 Jan;59(1):133-147. [CrossRef] [Medline]
    46. Proudfoot J, Swain S, Widmer S, Watkins E, Goldberg D, Marks I, et al. The development and beta-test of a computer-therapy program for anxiety and depression: hurdles and lessons. Comput Human Behav 2003 May;19(3):277-289. [CrossRef]
    47. Karyotaki E, Kleiboer A, Smit F, Turner DT, Pastor AM, Andersson G, et al. Predictors of treatment dropout in self-guided web-based interventions for depression: an 'individual patient data' meta-analysis. Psychol Med 2015 Oct;45(13):2717-2726. [CrossRef] [Medline]
    48. Littlewood E, Duarte A, Hewitt C, Knowles S, Palmer S, Walker S, REEACT Team. A randomised controlled trial of computerised cognitive behaviour therapy for the treatment of depression in primary care: the Randomised Evaluation of the Effectiveness and Acceptability of Computerised Therapy (REEACT) trial. Health Technol Assess 2015 Dec;19(101):viii, xxi-viii, 171 [FREE Full text] [CrossRef] [Medline]
    49. Rhebergen D, Beekman AT, Graaf RD, Nolen WA, Spijker J, Hoogendijk WJ, et al. The three-year naturalistic course of major depressive disorder, dysthymic disorder and double depression. J Affect Disord 2009 Jun;115(3):450-459. [CrossRef] [Medline]
    50. Christensen H, Batterham P, Mackinnon A, Griffiths KM, Kalia Hehir K, Kenardy J, et al. Prevention of generalized anxiety disorder using a web intervention, iChill: randomized controlled trial. J Med Internet Res 2014 Sep 02;16(9):e199 [FREE Full text] [CrossRef] [Medline]
    51. Schneider F, de Vries H, Candel M, van de Kar A, van Osch L. Periodic email prompts to re-use an internet-delivered computer-tailored lifestyle program: influence of prompt content and timing. J Med Internet Res 2013 Jan 31;15(1):e23 [FREE Full text] [CrossRef] [Medline]
    52. Elfeky A, Gillies K, Gardner H, Fraser C, Ishaku T, Treweek S. Non-randomised evaluations of strategies to increase participant retention in randomised controlled trials: a systematic review. Syst Rev 2020 Sep 29;9(1):224 [FREE Full text] [CrossRef] [Medline]
    53. Kearney A, Ashford P, Butlin L, Conway T, Cragg WJ, Devane D, et al. Developing an online, searchable database to systematically map and organise current literature on retention research (ORRCA2). Clin Trials 2022 Feb 24;19(1):71-80 [FREE Full text] [CrossRef] [Medline]
    54. Shapiro DA, Cavanagh K, Lomas H. Geographic inequity in the availability of cognitive behavioural therapy in England and Wales. Behav Cognit Psychother 2003;31(2):185-192. [CrossRef]
    55. Vespa J. Those who served: America's veterans from World War II to the war on terror. United States Census Bureau.   URL: https://www.census.gov/library/publications/2020/demo/acs-43.html [accessed 2022-03-01]
    56. Jonassaint CR, Gibbs P, Belnap BH, Karp JF, Abebe KZ, Rollman BL. Engagement and outcomes for a computerised cognitive-behavioural therapy intervention for anxiety and depression in African Americans. BJPsych Open 2017 Jan 02;3(1):1-5 [FREE Full text] [CrossRef] [Medline]
    57. Depression in adults: recognition and management. National Institute for Health and Care Excellence.   URL: https:/​/www.​nice.org.uk/​guidance/​cg90/​resources/​depression-in-adults-recognition-and-management-pdf-975742636741 [accessed 2022-03-01]
    58. Beatty L, Binnion C. A systematic review of predictors of, and reasons for, adherence to online psychological interventions. Int J Behav Med 2016 Dec;23(6):776-794. [CrossRef] [Medline]

    ‎
    BtB: Beating the Blues
    CBT: cognitive behavioral therapy
    cCBT: computerized cognitive behavioral therapies
    CSQ: Client Satisfaction Questionnaire
    EMR: electronic medical record
    GAD-7: Generalized Anxiety Disorder Inventory-7
    IEUQ: Internet Evaluation and Utility Questionnaire
    IIEQ: Internet Impact and Effectiveness Questionnaire
    PC-MHI: Primary Care and Mental Health Integration
    PHQ-9: Patient Health Questionnaire-9
    REDCap: Research Electronic Data Capture
    RFT: reasons for termination
    VA: Veterans Affairs
    VAMC: Veterans Affairs Medical Center

    Edited by A Mavragani; submitted 06.07.21; peer-reviewed by A MacIsaac, A Pencer; comments to author 19.09.21; revised version received 17.12.21; accepted 02.02.22; published 25.04.22

    Copyright

    ©Kelly A Stearns-Yoder, Arthur T Ryan, Alexandra A Smith, Jeri E Forster, Sean M Barnes, Lisa A Brenner. Originally published in JMIR Formative Research (https://formative.jmir.org), 25.04.2022.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.