This paper is in the following e-collection/theme issue:

Formative Evaluation of Digital Health Interventions (2414) Depression and Mood Disorders; Suicide Prevention (1363) Web-based and Mobile Health Interventions (3299) Innovations in Mental Health Systems (390) Digital Mental Health Interventions, e-Mental Health and Cyberpsychology (1515) Methods and New Tools in Mental Health Research (454)

Published on in Vol 6, No 6 (2022): June

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/36521, first published .
Predictors of Disengagement and Symptom Improvement Among Adults With Depression Enrolled in Talkspace, a Technology-Mediated Psychotherapy Platform: Naturalistic Observational Study

Predictors of Disengagement and Symptom Improvement Among Adults With Depression Enrolled in Talkspace, a Technology-Mediated Psychotherapy Platform: Naturalistic Observational Study

Predictors of Disengagement and Symptom Improvement Among Adults With Depression Enrolled in Talkspace, a Technology-Mediated Psychotherapy Platform: Naturalistic Observational Study

Authors of this article:

Doyanne Darnell1 Author Orcid Image ;   Michael D Pullmann1 Author Orcid Image ;   Thomas D Hull2 Author Orcid Image ;   Shiyu Chen1 Author Orcid Image ;   Patricia Areán1 Author Orcid Image
Article Authors Cited by (3) Tweetations (2) Metrics

    Original Paper

    1Department of Psychiatry & Behavioral Sciences, University of Washington, Seattle, WA, United States

    2Talkspace, Seattle, WA, United States

    Corresponding Author:

    Doyanne Darnell, PhD

    Department of Psychiatry & Behavioral Sciences

    University of Washington

    325 Ninth Avenue

    Box 359911

    Seattle, WA, 98104

    United States

    Phone: 1 206 744 9108

    Email: darnelld@uw.edu


    Background: Depression is a common psychiatric condition with an estimated lifetime prevalence for major depression of 16.6% in the US adult population and is effectively treated through psychotherapy. The widespread availability of the internet and personal devices such as smartphones are changing the landscape of delivery of psychotherapy; however, little is known about whether and for whom this type of therapy is beneficial, and whether having synchronous video-based sessions provides additional benefits to clients above and beyond messaging-based therapy.

    Objective: This study examined the outcomes associated with the use of a digital platform (Talkspace) for technology-mediated psychotherapy. We examined the duration of client engagement in therapy and client depression score trajectories over 16 weeks. We explored the association of client characteristics, therapist characteristics, and service plan type with time-to-disengagement and trajectories of change in depression scores.

    Methods: This naturalistic observational study assessed data collected routinely by the platform between January 2016 and January 2018 and examined psychotherapy outcomes among a large representative sample of adult clients with clinically significant depression. Treatment disengagement was defined as a lack of client-initiated communication for more than 4 weeks. Clients completed the Patient Health Questionnaire-8 item (PHQ-8) at intake and every 3 weeks via an in-app survey. Cox regression analysis was used to examine the time until and predictors of disengagement. Changes in depression scores and predictors of change over time were examined using mixed-effects regression.

    Results: The study included 5890 clients and 1271 therapists. Client scores on the PHQ-8 declined over time, with the average client improving from a score of 15 to below the clinical cutoff of 10 by week 6. At the same time point, 37% of clients had disengaged from the therapy. When combined into a final Cox regression model, those who were more likely to disengage were clients aged 18 to 25 years versus those aged ≥50 years (odds ratio [OR] 0.82, 95% CI 0.74-0.9; P<.001), had higher education (OR 1.14, 95% CI 1.06-1.22; P<.001), had been in therapy before (OR 1.09, 95% CI 1.02-1.17; P=.01), and were living with a partner but unmarried versus single (OR 1.14, 95% CI 1.02-1.27; P=.02). Having a therapist with >10 years of experience was related to lower odds of disengagement (OR 0.87, 95% CI 0.8-0.94; P=.01). When combined into a final regression model predicting improvement in depression scores over time, clients showing more improvement were those with an associate’s degree or higher (linear estimate=−0.07, P=.002) and higher intake PHQ-8 scores (estimate=3.73, P<.001). There were no differences based on the plan type.

    Conclusions: Our findings add to the growing literature showing the benefits of technology-mediated psychotherapy over a relatively brief period (16 weeks).

    JMIR Form Res 2022;6(6):e36521

    doi:10.2196/36521

    Keywords

    depression; psychotherapy; disengagement; internet; web-based; technology-mediated psychotherapy 


    Background

    Depression is a common psychiatric condition with an estimated lifetime prevalence of major depression of 16.6% and a previous year prevalence of a major depressive episode of 7.8% to 8.6% among the US adult population [1,2]. Psychotherapy is an effective treatment for depression [3] and is often preferred to medication [4]; however, clients face numerous challenges in obtaining what has traditionally been face-to-face in-person treatment, such as the need to travel to a psychotherapist’s office and potential stigma associated with accessing this care [5]. The widespread availability of the internet and personal devices such as smartphones is changing the landscape of delivery of psychotherapy, allowing consumers to access care more flexibly and without the need to go to a clinic [6]. These opportunities continue to grow in the wake of the COVID-19 pandemic, and the resultant increase in web-based health care is initiated in response to social distancing policies [7].

    Numerous technology platforms are now available that allow clients to flexibly communicate with a psychotherapist via text messages, voice, or videoconferencing software; this is known as technology-mediated psychotherapy [6]. A common distinction made regarding technology-mediated psychotherapy is whether it is conducted in an asynchronous or synchronous fashion. Asynchronous refers to communication sent and received by therapists and clients at different times, such as a therapist sending a client an email with a homework assignment and then sending feedback on the assignment once completed. Synchronous refers to communication sent and received by therapists and clients at the same time, such as telephone conversations or a web-based chat session.

    Prior Work

    Over the past 20 years, numerous clinical trials of technology-mediated synchronous and asynchronous psychotherapy have been conducted, most of which have evaluated structured cognitive-behavioral approaches. Findings from meta-analyses based on trials conducted in English of psychotherapy for depression by videoconference or telephone (synchronous) indicate that these forms of therapy are similarly effective to face-to-face in-person treatment and with similar (~80%), if not better, completion rates [8-10]. These meta-analyses reflect diverse samples recruited from a variety of settings (eg, veterans, primary care, HIV clinics, outpatient psychotherapy clinics, and community samples).

    Fewer studies on the effectiveness of text- or chat-based therapy (whether synchronous or asynchronous) have been conducted, and the findings have been mixed. One clinical trial evaluating asynchronous cognitive behavior therapy (CBT)–oriented psychotherapy for depression among adults in Sweden assessed the benefit of an 8-week email-based therapy, indicating that email therapy was associated with pre-post benefit but no difference in treatment outcomes as compared with a guided 8-week self-help web-based program or waitlist-control at 6 months [11]. The rates of client completion of the intervention were similarly high between the groups (96% completed the email-based therapy and 93% completed the self-help). Another trial using a web-based platform for chat or text-based psychotherapy found that adult primary care patients in the United Kingdom assigned to a web-based CBT-oriented therapy for 16 weeks experienced greater improvement in depression symptoms than a waitlist or usual primary care comparison group over 8 months [12]. Analyses were based on an intent-to-treat sample, of which a large proportion (48%) did not complete the therapy as intended, according to the therapist’s report.

    Few studies on technology-mediated psychotherapy have examined how client demographics moderate outcomes or therapy completion. A secondary exploratory analysis of the United Kingdom chat or text-based CBT trial [12] examined moderators of treatment outcome at the 4-month follow-up and found that depression outcomes were better for patients with higher baseline depression severity [13]. In addition, being separated, divorced, or widowed as opposed to being single or married was associated with greater treatment outcomes for those in the CBT condition; age and education level were not significant moderators. Findings from a meta-analysis of a broader array of technology-mediated psychological interventions targeting various mental health diagnoses suggest that client age may be a relevant moderator of outcome; in particular, adults aged 19 to 39 years may experience greater benefits than older or younger clients [14].

    Little is known about the effects of technology-mediated psychotherapy delivered routinely, for whom this approach may be beneficial, or whether there is differential effectiveness based on asynchronous versus synchronous delivery. In addition, to date, no studies have compared asynchronous and synchronous technology-mediated psychotherapy. This study aims to fill these gaps by studying outcomes associated with the use of a digital platform (Talkspace) that facilitates technology-mediated psychotherapy either asynchronously or synchronously among adult clients experiencing depression symptoms. Specifically, the platform is a desktop computer and mobile smartphone app that allows 24/7 communication via a secure interactive text chat and voice or video messages with a licensed psychotherapist for a fee [15]. Clients may pay for a plan with optional synchronous 1-hour sessions conducted through chat, telephone, or videoconferencing that can be either 1 or 4 times per month. Several observational studies with selected samples of Talkspace users suggest that the use of the asynchronous text-based psychotherapy plan is associated with improvement in depression symptoms over 12 to 16 weeks of treatment; however, none of these studies examined predictors of these outcomes and for whom the therapy may be most beneficial [15-18].

    Objective

    This naturalistic observational study harnesses data collected routinely by the platform and examines psychotherapy outcomes among a large representative sample of clients with clinically significant depression. We examined how long clients remained in therapy before disengaging from the application after their initial enrollment as well as client trajectories of depression scores over 16 weeks after enrollment. In addition, we explored the association between client characteristics, therapist characteristics, and the type of service plan with time-to-disengagement and trajectories of change in depression scores.


    Design and Procedure

    This study included longitudinal and observational research with clients who signed up for Talkspace between January 2016 and January 2018 and their therapists using data collected routinely as part of the service. Clients accessed the service through an internet search and either paid out-of-pocket or submitted costs to a private insurance plan for out-of-network reimbursement. The signup process includes a brief intake with a consultation therapist who helps the client select their desired plan (eg, messaging only, monthly video, or weekly video plans) and records the client’s therapist preferences, presenting complaints, and demographics. This information is used by the service to identify and offer the client a choice among 3 therapists that most closely match the client’s preferences, are licensed in the client’s state of residence, and have a successful history of treating conditions the client is reporting. After therapist selection, the client is offered a baseline symptom assessment and can then message their therapist as often as they like. Therapists respond to client messages within 24 hours, gather information for a diagnosis, explain the frame of the medium, and conduct informed consent procedures, after which therapy proceeds 5 days a week.

    Therapists encounter the platform through internet searches, professional organizations, and peer contacts. All prospective therapists completed an application process to verify the state licensure, training and degree type, professional insurance, background checks, and confirmation of meeting the National Certification of Quality Assurance standards. Following verification, therapists completed an orientation to the platform, received Health Insurance Portability and Accountability Act, privacy, and security training on the proper use of technological media for delivering care, and took on a small number of training cases with supervision for 30 days before beginning their practice on the platform. Engagement and outcome metrics were monitored after the training phase for quality assurance along with a random peer-review process by other therapists on the platform. The service does not prescribe a specific approach to therapy, and while most therapists (61%) report practicing from a cognitive-behavioral orientation, therapists from many traditions are represented on the platform.

    Participants

    The participants in this study were clients of the Talkspace service. Inclusion criteria was age 18 years and older, with clinically significant depression as indicated by having a primary diagnosis of depression or a depression-related disorder, and a score of 10 or greater on the 8-item version of the Patient Health Questionnaire (PHQ-8) administered at intake for the service. In addition, given the requirements of the service, participants were English language literate, had internet access, and were able to use mobile or desktop applications for the service. Exclusion criteria for the use of the service were indications during the intake with the consultation therapist, or at any point when working with the treating therapist, of any schizophrenia spectrum and psychotic disorder, or any diagnosis with psychotic features, or any condition requiring hospitalization, or suicidal thoughts or behavior sufficient to be marked a Yes on any of questions 3 through 6 on the Columbia Suicide Severity Rating Scale Lifetime-Recent Screen [19] that would require a more intensive level of care than can be offered through an outpatient service.

    Measures and Variables

    Client Characteristics and Demographics

    As part of signing up for the Talkspace service, clients provide their age range (19-25, 26-35, 36-49, 50+ years), gender (female, male, gender queer, transgender male, transgender female, gender variant, other), education level (high school, bachelor’s degree or higher), whether they had ever been in therapy before, and their marital status (single, married, living with a partner, divorced, separated, and widowed).

    Depression-Related Diagnosis

    Clients were assigned a psychiatric diagnosis according to the 5th edition of the Diagnostic and Statistical Manual of Mental Disorders by their matched therapist, which may be assessed through a mix of diagnostic interviewing, often by video, and delivery of standardized symptom assessments. Clients were included in the study if they had a primary diagnosis of a depression-related disorder (eg, major depressive disorder, adjustment disorder with depressed mood, dysthymic disorder, or other mood disorders).

    Therapist Characteristics

    Therapists provide information about demographics and professional experiences when they apply to join a service network. Therapists can select female, male, gender queer, transgender male, transgender female, gender variant, or other gender. Therapists indicate years of postlicensure experience as psychotherapists (coded as <5 years, 5-10 years, and >10 years) and areas of expertise (for this study, dichotomously coded as whether therapists indicated expertise in treating depression).

    Psychotherapy Disengagement

    Disengagement from psychotherapy was defined for this study by a lack of client-initiated communication via any means using the Talkspace application (eg, text, voice messages, and videoconferencing sessions) for more than 4 weeks after initial enrollment. These data are drawn from administrative records on exchanges with therapists and are passively collected by the application.

    Depression Symptoms

    Clients were asked to complete the PHQ-8 [20] at intake and every 3 weeks via an in-app survey. There was an allowance for completing these with a 1-week buffer before or after the deadline. The PHQ-8 includes 8 items that assess the severity of depression symptoms according to the Diagnostic Statistical Manual, Fourth Edition, criteria, with the exception of suicidality or preoccupation with death. Items were rated on a scale from 0 (not at all) to 3 (nearly every day) and summed to obtain a total score. Higher scores indicate greater severity, and a cutoff score of 10 or higher indicates clinically significant depression. A 5-point difference in scores indicates a clinically meaningful change in depression symptoms [21].

    Plan Type

    Platform clients could opt to purchase one of three service plans: (1) unlimited text, voice, or video messages; (2) unlimited text, voice, or video messages plus once per month 1-hour videoconferencing session; or (3) unlimited text, voice, or video messages plus 4 times per month 1-hour videoconferencing sessions. How often and to what extent clients and therapists communicate varies and is dependent upon the frequency with which clients send messages to their therapist. Therapists are expected to respond to client communications within 24 hours and generally respond within 12 hours, except for therapist days off or other times mutually agreed upon by the therapist and client.

    Plan of Analysis

    Using R, we examined descriptive statistics for client and therapist characteristics, depression symptoms at intake or baseline, and service plan type that clients were enrolled in. We examined the missingness of study variables at each point of time as described in the description of our results. For the prediction analysis, we collapsed several categorical variables when the sample size within some of the variable categories was low, which made theoretical sense to do so. We collapsed education into high school education versus associate’s degree or higher. As only 30 participants were widowed, we collapsed this group into the missing category. We conducted 2 sets of mixed effects regression analyses to examine, (1) trajectories of disengagement over 52 weeks and predictors of disengagement and (2) trajectories of depression scores over 16 weeks and predictors of change in depression over time.

    Predicting Psychotherapy Disengagement Over 52 Weeks After Intake

    We examined how long clients were in therapy before disengaging over the course of 52 weeks, following their intake with the platform. Furthermore, a duration of 52 weeks was selected based on visual inspection of the data, showing that most participants had stopped communicating with their therapist for more than 4 weeks by that time. Using R, we conducted mixed-effects Cox regression analyses predicting time until disengagement, with random terms for clinician and client to account for nesting of time within client and client within therapist. Model building followed appropriate procedures for our combined confirmatory and exploratory analyses; predictors were first tested individually to conduct a priori confirmatory hypothesis testing, and those that were significant at P<.05 were included in a final, combined exploratory model to examine the overall model structure. Significance was tested using the log-rank test for individual variable models. For categorical variables, simple contrast comparisons were used in which each category was compared against the reference category, as described in the results. To estimate the goodness-of-fit, we computed the proportion of the variance accounted for using Nagelkerke R2 for each model. We also computed the concordance statistic, which provides the fraction of concordant pairs between the model-predicted and actual data and is equal to the receiver operating characteristic curve. A concordance statistic of 0.50 means that the model is no better than random chance, while a statistic of 1.0 would mean there is a perfect concordance between predicted and actual data. Analyses of the relationship between missing data status on demographic and descriptive variables and time until engagement were conducted separately using Cox regression analyses; cases with missing data were excluded from the primary analyses.

    Predicting Depression Symptoms Over 16 Weeks After Intake

    We examined depression symptoms based on PHQ-8 scores over the course of 16 weeks following the client’s service intake. In addition, a duration of 16 weeks was selected based on visual inspection of the data showing that most of the change in scores occurred during the first 16 weeks and missing data were substantial at that point, which is likely primarily owing to high rates of client disengagement from therapy by 16 weeks. Using R, mixed-effects regressions with random terms for therapist and client were computed to adjust standard errors owing to nesting of time within client and client within therapist. The covariance matrix was specified to be unstructured. As described below, the rates of missing data were high for the dependent variable owing to ending treatment or incomplete measures. Missing data were determined to be not missing at random, based on the empirical analyses described below and theoretical reasons. It is highly likely that those who did not complete measures were due to unmeasured reasons, such as treatment attitudes, motivation, and emotional distress or wellness. Therefore, full information maximum likelihood estimation was used for model testing and parameter estimation (to maximize all available data), and variables associated with missing outcome data were included in the final models to statistically adjust for missingness to the greatest extent possible. Findings may only be inferred to the population of clients who completed measures while receiving treatment. Model building followed standard procedures [22]. Model fit was tested by computing the Akaike information criterion, Bayesian information criterion, and 2-Log Likelihood deviance statistics. The models tested linear, quadratic, and cubic time trends for changes in the PHQ-8 scores over time. The best-fitting models included the linear and quadratic time trends. Then, each potential predictor, including client and therapist characteristics, was entered into individual models, and interaction terms for linear and quadratic time were tested using model fit deviance statistics, with each more complex model tested against simpler models, and all available data were used for each computation. All variables and interaction terms from models that were statistically significant at P<.05 were then entered into a final omnibus model. We ran two types of R2 for each model: the conditional R2 provides the total proportion of variance accounted for, including fixed and random terms, while the marginal R2 provides the proportion of variance accounted for by fixed terms only.

    Ethics Approval

    As part of the terms of service agreement, clients agreed that their anonymized data may be used for research purposes. Considering the data were completely anonymized, this study was exempted from institutional review board approval by the institutional review board of the University of Washington.


    Sample Characteristics

    The study included 5890 clients and 1271 therapists (Table 1). The participants were primarily female (4504/5890, 76.5%), 26 to 35 years of age (3061/5890, 52%), had a bachelor’s degree (3706/5890, 62.9%), were single (3122/5890, 52.8%), and had been in therapy before (4004/5890, 68%). The average PHQ-8 score was 15.2 (SD 3.9). The therapists were primarily female (1114/1271, 87.6%), with more than 5 years of experience (1017/1271, 80%), and close to half endorsed having depression-specific expertise (591/1271, 46.5%). Clients predominantly signed up for a messaging-only Talkspace plan (5389/1271, 91.5%). There were no missing data for client disengagement from therapy; however, we observed modest amounts of missing data for other client and therapist variables (ranging from 0.3% to 13.6%, see Table 1). Missing data analyses found that missing data on the following variables were not significantly associated with the length of time until disengagement: client’s gender, client’s first time in therapy, or therapist’s years of experience. Client age and education were significantly associated, such that those who had missing data on these variables were more likely to disengage (age odds ratio [OR] 1.46, 95% CI 1.15-1.89, P=.002; education OR 1.10, 95% CI 1.02-1.19, P=.013).

    Table 1. Client and therapist characteristics.
    VariableValue
    Client: total, n (%)5890 (100)
    Client age (years), n (%)

    		18-251328 (22.5)

    		26-353061 (52)

    		36-491182 (20.1)

    		>50250 (4.2)

    		Missing69 (1.2)
    Client gender, n (%)

    		Female4505 (76.5)

    		Other5 (0.1)

    		Queer9 (0.2)

    		Variant5 (0.1)

    		Male1317 (22.4)

    		Transgender female3 (0.1)

    		Transgender male7 (0.1)

    		Missing39 (0.7)
    Client education, n (%)

    		High school1327 (22.5)

    		Some college14 (0.2)

    		Associate’s degree6 (0.1)

    		Bachelor’s degree or higher3706 (62.9)

    		Master’s degree21 (0.4)

    		Doctoral degree5 (0.1)

    		Professional degree8 (0.1)

    		Missing803 (13.6)
    Client marital status, n (%)

    		Divorced328 (5.6)

    		Living with a partner497 (8.4)

    		Married1663 (28.2)

    		Separated142 (2.4)

    		Single3112 (52.8)

    		Widowed30 (0.5)

    		Missing118 (2.0)
    Client first time in therapy

    		No, n (%)4004 (68)

    		Yes, n (%)1346 (22.9)

    		Missing, n (%)540 (9.2)

    		Baseline PHQ-8a, mean (SD)15.2 (3.9)
    Talkspace plan type

    		Text only, n (%)5389 (91.5)

    		1-month video, n (%)411 (7)

    		4-month video, n (%)90 (1.5)

    		Therapist, total (N)1271 (100)
    Therapist gender, n (%)

    		Male157 (12.4)

    		Female1114 (87.6)
    Therapist years of experience, n (%)

    		<5250 (19.7)

    		5-10551 (43.4)

    		Missing4 (0.3)
    Therapist with depression expertise, n (%)591 (46.5)

    aPHQ-8: Patient Health Questionnaire-8 item.

    Predicting Psychotherapy Disengagement Over 52 Weeks Post Intake

    By week 6, 37% of the clients had disengaged from therapy. Half of the sample disengaged from therapy by week 9 and by week 52, nearly all clients had disengaged (n=5441, 92.4%; Figure 1). Table 2 displays the single-variable predictor models for all the variables that were significantly associated with disengagement. When combined into a final Cox regression model, significant variables included client age, education, whether the client had been in therapy before, marital status, and years of experience as a therapist (Table 3). Compared with those aged 18 to 25 years, those aged 36 to 49 years had 18.9% lower odds of disengaging at any point in time (OR 0.82, 95% CI 0.74-0.9; P<.001; Figure 2) and those aged >50 years had 30.1% lower odds of disengaging (OR 0.70, 95% CI 0.59-0.83; P<.001). Those with higher education had 13.5% greater odds of disengaging than those with lower education (OR 1.14, 95% CI 1.06-1.22; P<.001), and those who were in therapy for the first time had 8.9% greater odds of disengaging (OR 1.09, 95% CI 1.02-1.17; P=.01). Participants who were living with a partner had 13.9% greater odds of disengaging as compared with those who were single (OR 1.14, 95% CI 1.02-1.27; P=.02); there were no significant differences between those who were single and those who were married, separated, or divorced. Finally, clients with therapists who had more than 10 years of experience had 13.2% lower odds of disengaging (OR 0.87, 95% CI 0.8-0.94; P=.01). Although significant in the single-variable predictor model, there were no significant differences in the likelihood of disengagement for clients based on the therapist endorsement of depression-specific expertise in the combined Cox regression model. Plan type, intake PHQ-8 score, client gender, and therapist gender were not included in the final model, as these variables were not significant in the single-variable predictor models. Model fit statistics revealed a very low proportion of variance accounted for and concordance across all models (Tables 3 and 4). The proportion of variance accounted for in the final model was only 1.8%, and the concordance statistic of 0.54 indicated that the model predicted the length of time until disengagement only slightly better than chance.

    ‎
    Figure 1. Time until disengagement.
    View this figure
    Table 2. Significant single-variable predictor Cox regression models testing time until treatment disengagementa.

    		Median (weeks)ORb (95% CI)P valueLog rankP value
    Age (years)33.78<.001

    		18-258RefRef


    		26-3590.93 (0.87-0.99).035


    		36-49100.83 (0.77-0.91)<.001


    		>50110.71 (0.61-0.82)<.001

    Education17.24<.001

    		High school9RefRef


    		Associate’s degree or higher81.15 (1.08-1.22)<.001

    First time in treatment11.37<.001

    		No9RefRef


    		Yes81.12 (1.05-1.19)<.001

    Marital status15.31.004

    		Single9RefRef


    		Living with partner81.18 (1.07-1.3).001


    		Married90.96 (0.91-1.03).22


    		Separated100.98 (0.82-1.17).83


    		Divorced90.95 (0.84-1.06).37

    Therapist experience16.45<.001

    		<5 years8RefRef


    		5-10 years90.93 (0.86-1).07


    		>10 years90.86 (0.8-0.93).002

    Therapist with depression-specific expertise7.97.004

    		Not endorsed8RefRef


    		Endorsed90.92 (0.88-0.98).03

    aClient n=5890, therapist n=1271. Variables that were tested and were not significant included plan type, intake PHQ-8 score, client gender, and therapist gender. Widowed data were collapsed with missing marital status data. Disengagement from treatment was defined as >4 weeks without any client-initiated communication to the therapist, such as texting, video exchanges, images, or audio clips. The modeling included up to 52 weeks of therapy.

    bOR: odds ratio.

    Table 3. Model fit statistics for individual and final combined nested Cox regression model predicting time until treatment disengagementa.

    		Nagelkerke R2Concordance
    Individual models

    		Age0.00650.52

    		Education0.00360.52

    		First time in therapy0.00220.51

    		Marital status0.00280.51

    		Therapist experience0.00300.51

    		Therapist with depression-specific expertise0.00150.51
    Final model0.0180.54

    aDisengagement from treatment was defined as >4 weeks without any client-initiated communication to the therapist, such as texting, video exchanges, images, or audio clips. The modeling included up to 52 weeks of therapy.

    ‎
    Figure 2. Time until disengagement stratified by age group.
    View this figure
    Table 4. Final combined nested Cox regression model predicting time until treatment disengagementa.

    		CoefficientORb (95% CI)P value
    Age (years; ref=18-25)

    		26-35−0.050.96 (0.88-1.03).27

    		36-49−0.200.82 (0.74-0.9)<.001

    		>50−0.360.7 (0.59-0.83)<.001
    Education (ref=high school)

    		Associate’s or bachelor’s degree or higher0.131.14 (1.06-1.22)<.001

    		First time in treatment0.091.09 (1.02-1.17).011
    Marital status (ref=single)

    		Living with a partner0.131.14 (1.02-1.27).019

    		Married0.031.03 (0.95-1.1).48

    		Separated0.0061.01 (0.82-1.23).95

    		Divorced0.041.04 (0.91-1.2).55
    Therapist experience (years; ref=<5)

    		5-10−0.090.92 (0.84-0.99).06

    		>10−0.140.87 (0.8-0.94).01

    		Therapist with depression-specific expertise−0.060.94 (0.89-1).14

    aDisengagement from treatment was defined as >4 weeks without any client-initiated communication to the therapist, such as texting, video exchanges, images, or audio clips. The modeling included up to 52 weeks of therapy. Predictor variables that were significant in simpler models were also included.

    bOR: odds ratio.

    Predicting Depression Symptoms Over 16 Weeks Post Intake

    There were high levels of missingness in the PHQ-8 over the 16 weeks after the intake observation period. Of the 5890 clients, 3447 (58.5%) completed a 3-week survey, 1978 (33.6%) completed a 6-week survey, 1165 (19.8%) completed a 9-week survey, 724 (12.3%) completed a 12-week survey, and 506 (8.6%) completed a 15-week survey. When comparing those with at least one follow-up time point to those without, there were no differences in the rates of missing data on the basis of age, education level, history of therapy, gender, or PHQ-8 score. Those who did not have any follow-up PHQ-8 were more likely to be divorced (8.1% with follow-up data vs 9.6% missing all follow-up) or living with a partner (8.1% vs 9.6%), less likely to be married (30.2% vs 27.1%), more likely to have a video plan (7.9% vs 9.4%), and less likely to have a messaging-only plan (90.6% vs 92.1%).

    The best-fitting model for change over time in the PHQ-8 is shown in Figure 3, illustrating a curvilinear change in PHQ-8 scores during the first 16 weeks of treatment (quadratic -2LL5=−41,432, deviance=399, P<.001), therefore all subsequent models included linear and quadratic change variables. The model-derived average PHQ-8 score at intake was 14.94 (SE 0.08), with a linear slope decreasing an average of −1.25 points (SE 0.03) between the first and second week and flattening by an additional 0.06 points (SE 0.002) for each subsequent week. Client scores on the PHQ-8 declined over time, dropping below the clinical cut-off from a score of 15 to 10 by week 6.

    Table 5 displays models using single variables to predict change over time and found that client education level and age significantly predicted intake PHQ-8 score and linear change over time (education level linear – 2LL2 deviance=25, P<.001; age linear 2LL6 deviance=9, P=.006). Clients’ PHQ-8 score at intake, and whether it was their first time in therapy, significantly predicted linear and quadratic change over time in the PHQ-8 score (first PHQ-8 score linear -2LL2 deviance=1991, P<.001, quadratic -2LL1 deviance=99, P<.001; first time in therapy linear -2LL2 deviance=12, P<.001, quadratic -2LL1 deviance=9, P<.001). Variables that were not significantly associated with PHQ-8 scores over time included plan type, client gender, therapist gender, therapist years of experience, and therapist endorsing depression-specific expertise. Model fit statistics are displayed in Table 6, demonstrating that, unsurprisingly, the model incorporating intake PHQ-8 score accounted for the largest amount of variance in the model.

    The final model explained 51.1% of the variance in PHQ-8 score change. This model included predictor variables that had significant relationships with the intercept and linear or quadratic change in PHQ-8 scores over time, on the basis of single variable modeling (see Table 7). Significant predictors were as follows: those with an associate’s degree or higher improved more quickly based on their sharper linear slope (linear estimate=−0.07, P=.002). Higher intake PHQ-8 scores, displayed as a median split in Figure 4, were associated with a larger PHQ-8 intercept (estimate=3.73, P<.001), faster linear improvement (estimate=−0.50, P<.001) and faster flattening (quadratic estimate=0.03, P<.001). There were no differences in the PHQ-8 intercept on whether it was the client’s first time in therapy; however, first-time clients had faster linear improvement (linear estimate=−0.37, P<.001) and faster flattening (quadratic estimate=0.03, P<.001). Age was not significant after controlling for other variables.

    ‎
    Figure 3. Model estimated change on the Patient Health Questionnaire-8 item (PHQ-8) over 16 weeks with SE shading.
    View this figure
    Table 5. Significant single-variable mixed effects models predicting Patient Health Questionnaire-8 item (PHQ-8) score changea.

    		EstimateSEP value
    Education (ref=high school)

    		Intercept15.660.13<.001

    		Week−1.220.03<.001

    		Week20.060.002<.001

    		Associate’s or bachelor’s degree or higher−0.900.15<.001

    		Associate’s degree or higher × week−0.020.02.34
    Age (ref=18-25 years)

    		Intercept15.260.13<.001

    		Week−1.240.03<.001

    		Week20.060.002<.001

    		26-35−0.490.14<.001

    		36-49−0.140.17.41

    		>50−0.420.29.14

    		26-35 × week−0.010.03.67

    		36-49 × week0.0050.03.87

    		>50 × week0.050.05.27
    Intake PHQ-8 score

    		Intercept15.070.06<.001

    		Week−1.280.02<.001

    		Week20.070.002<.001

    		PHQ-83.750.05<.001

    		PHQ-8 × week−0.490.02<.001

    		PHQ-8 × week20.030.002<.001
    First time in therapy

    		Intercept15.040.09<.001

    		Week−1.170.03<.001

    		Week 20.060.003<.001

    		First time in therapy−0.200.15.17

    		First time in therapy × week−0.300.07<.001

    		First time in therapy × week20.020.005<.001

    aVariables that were tested and were not significant on the basis of model fit statistics included client gender, marital status, plan type, therapist gender, therapist years of experience, and therapist endorsing depression-specific expertise.

    Table 6. Model fit statistics predicting Patient Health Questionnaire-8 item (PHQ-8) score change.

    		AICaBICb−2LLcdfConditional R2Marginal R2
    Individual models

    		Education66,153.266,204.5−33,069.670.2790.180

    		Age66,082.066,162.7−33,030110.2710.179

    		Intake PHQ-8 score62,024.462,083.0−31,004.280.5100.460

    		First time in therapy62,877.062,935.3−31,430.580.2780.182
    Final combined model64,112.864,252.3−32,037.4190.5110.455

    aAIC: Akaike Information Criterion.

    bBIC: Bayesian Information Criterion.

    c−2LL: −2 Log Likelihood.

    Table 7. Final combined mixed effects model predicting Patient Health Questionnaire (PHQ-8) score changea.

    		EstimateSEP value
    Intercept15.360.14<.001
    Week−1.100.04<.001
    Week20.060.002<.001
    Associate’s or bachelor’s degree or higher (ref=high school)−0.180.12.120
    Associate’s degree or higher × week (ref=high school)−0.070.02.002
    Age (years, ref=18-25)

    		26-35−0.160.13.20

    		36-49−0.100.16.52

    		>50−0.410.26.12
    Age (years, ref=18-25) × week

    		26-35−0.030.02.15

    		36-49−0.010.03.76

    		>500.040.04.32
    Intake PHQ-8 score3.730.06<.001
    Intake PHQ-8 score × week−0.500.03<.001
    Intake PHQ-8 score × week20.030.002<.001
    First time in therapy−0.070.13.58
    First time in therapy × week−0.300.06<.001
    First time in therapy × week20.030.005<.001

    aPredictor and predictor × time interaction variables that were significant in simpler models were included. The intake PHQ-8 scores were grand mean-centered.

    ‎
    Figure 4. Median split Patient Health Questionnaire-8 item (PHQ-8) score at intake and change over time with SE shading.
    View this figure

    Principal Findings

    This is the first population-based study of treatment engagement, subsequent outcomes, and predictors of outcomes in the routine delivery of technology-mediated psychotherapy focused on adult clients experiencing clinically significant levels of depression. Our findings indicate that technology-mediated psychotherapy may be helpful in reducing symptoms of depression. On average, client scores on the PHQ-8 declined over time, dropping below the clinical cut-off from a score of 15 to 10 by week 6. At the same time point, 37% of clients had disengaged from the therapy. There were no differences in the length of treatment engagement or speed of improvement based on whether clients opted for a plan that included synchronous video chat (either once or 4 times per month) in addition to the asynchronous messaging plan.

    Findings from our analyses predicting symptom change and disengagement suggest that the degree to which the therapy was helpful did not depend on the client or therapist characteristics included in the study; however, how quickly clients decided to end the therapy did, though this effect was very small. One exception was education level. Completing formal education beyond high school was related to a sharper decrease in PHQ-8 scores and a shorter length of time before disengagement, which may be interpreted as clients with higher education being a group that is particularly likely to benefit quickly from technology-mediated psychotherapy and, therefore, end it sooner. Consistent with this premise, there is some evidence that individuals with more years of formal education are less likely to end therapy prematurely [23]. Clients who were in therapy for the first time were also more likely to improve faster. While this might be associated with less severe depressive symptoms, this result was true even after controlling for baseline depression scores. It may be that those repeatedly seeking treatment have symptoms that are more chronic, even if they are of equal severity to those seeking treatment for the first time.

    The fact that specific age categories and gender were not significantly associated with depressive symptom change is consistent with a study examining moderators of outcomes among primary care patients in the United Kingdom randomized to chat-based CBT therapy versus usual care [13]. However, in contrast to our findings, educational attainment did not moderate the relationship between therapy and outcomes in the primary care sample. This is also consistent with a recent meta-regression study examining moderators of outcomes from randomized trials of face-to-face in-person psychotherapy versus a control group among adults with depressive symptoms, in which the authors found no difference between older and younger adults and between men and women [24]. The authors note that there were insufficient studies available for the analysis with a low risk of bias to examine education level as a moderator.

    Unfortunately, we did not have data on client race and ethnicity; however, in the meta-regression study mentioned [24], studies focusing on diverse samples did not have better or worse effect sizes than the other studies. This is similar to findings from a previous meta-analysis using a similar methodology, looking at randomized controlled trials in which race and ethnicity data were reported [25]. The authors examined whether the proportion of adults identified as a racial or ethnic minority in the sample was associated with treatment effect size across 56 trials and found no relation. Although psychotherapy appears to be effective across racial and ethnic groups [26], access to and barriers to engagement in psychotherapy are characterized by racial or ethnic disparities [27]. For instance, a large meta-analysis of randomized controlled trials of face-to-face in-person psychotherapy for depression among adults found that studies with a higher proportion of participants identifying with a racial or ethnic minority group had higher rates of early therapy termination, as defined by research teams [28].

    In a nationally representative US-based study, people identified as Black or African American, Latinx, and Asian Americans were less likely to engage in mental health outpatient care and indicated financial costs to be a major barrier to accessing services [29]. Other barriers may be related to beliefs that therapy is not likely to be effective, stigma and shame about needing care and being seen accessing it, language barriers, and cultural practices that may not resonate with a Western medical model [27,30-32]. Technology-mediated psychotherapy holds promise for alleviating some of these barriers to psychotherapy, particularly in reducing the need to find a local therapist and attend in-person sessions that may require transportation and additional time to travel to appointments. Although technology-mediated therapy may increase a sense of privacy and confidentiality as clients do not have to present to a clinic or office for care where others may see them, there may be additional barriers to ensuring privacy that have to do with concern over security breaches [33]. Although technology and internet access are expanding rapidly, there are still substantial disparities in access to stable and consistent internet and internet-capable smartphones and computers [34]. Future research can examine the role of race, ethnicity, and the intersection of access to technology-mediated psychotherapy in both outcomes and therapeutic engagement.

    Our findings also indicate that the length of engagement in services is weakly associated with the variables included in this study. More research should examine the determinants of engagement and work to operationalize them using feasible methods so that clinical care providers can target their efforts to prevent premature disengagement. While the association is weak, our findings indicate that being younger, in therapy for the first time, unmarried but living with their partner (as compared with single), and being with a therapist with less experience predicted a shorter length of time to disengagement from the therapy. Previous research indicates that younger clients are more likely to prematurely end therapy [23], and that less experienced therapists tend to have higher premature termination rates [23] and more difficulty engaging and retaining clients in therapy [35]. Marital status did not appear to be related to psychotherapy or mental health service disengagement in previous face-to-face psychotherapy studies [23,28,36]. However, a study of chat-based CBT with primary care patients observed that those patients grouped into a category of separated, divorced, or widowed had greater benefit from the therapy than those who were married or living as married and those who were single. The authors determined that the effect seemed to be largely owing to the separated, divorced, or widowed group remaining more symptomatic when waitlisted and obtaining usual care. Understanding the role of marital status in therapeutic gains is complicated by examination of this variable. Owing to the small sample size, categories that seem conceptually similar on the surface may collapse in analyses but may actually represent very distinct groups empirically [37]. For instance, being unmarried and living with a partner may hold a unique meaning for the millennial generation, which comprises a large proportion of our sample who are known to delay marriage [38], and combining these clients with married clients, as was done in the Button et al [13] study, could obscure findings that would emerge if examined separately, as was done in our analysis. Qualitative research may help elucidate the significance of living with a partner, which may be related to a more rapid disengagement from therapy.

    Additional research is needed to understand why client sociodemographic factors may influence perceptions of therapy and decisions to end the therapy. However, it is important to note that not all therapeutic disengagement outcomes are negative. In fact, people are known to vary in how quickly their symptoms respond to psychotherapy, and there is evidence to suggest that people leave therapy when they feel they have a good enough level of change [39-41]. A previous study of Talkspace clients suggested that this is often the case. Specifically, in a study of clients experiencing depression or anxiety, one-third of those leaving therapy before 12 weeks of enrollment reported reasons for leaving, with 53% indicating satisfaction with reaching therapeutic goals as the reason [18]. Therefore, engagement in technology-mediated psychotherapy may require shorter engagement times than in-person session-based therapy, potentially because of the ability of clients to access therapists as needed, rather than the typical once a week or every other week schedule.

    This study examined disengagement from the initial enrollment in therapy; however, it is possible that clients re-engaged, and there may be observable patterns of disengagement and re-engagement over time among the clientele. In addition, the concept of disengagement does not shed light on the nature of a client’s engagement in therapy. Therapeutic engagement can be defined as all efforts made by clients during the course of treatment (both within and between sessions) toward the achievement of changes (treatment outcomes) [35]. Once a client has disengaged, it is clear that they no longer participate in the therapy; however, if the client is still in therapy, the degree to which they are genuinely engaged in the process with the therapist can vary substantially and is an important variable in treatment outcomes. In an earlier Talkspace study with depressed or anxious clients, greater amounts of communication between clients in therapists based on word counts across text, audio, and video messaging were related to symptom improvement [18]. Research on face-to-face in-person CBT in which the timing of sessions varied suggests that having more frequent sessions in closer proximity to each other versus longer courses of therapy is related to improved outcomes [42]. A benefit of technology-mediated psychotherapy, such as what the Talkspace platform offers, is that clients have more potential to increase the intensity of their therapy and perhaps more quickly benefit from it.

    Interestingly, it may not be necessary for clients to meet with their therapist in a more traditional synchronous manner, such as having a face-to-face web-based video-based session, to receive an adequate degree of therapeutic intensity through technology-mediated therapy. In this study, we did not observe a relationship between the type of plan clients opted into and changes in depression scores over time, although some plan types included 1 or 4 synchronous video sessions per month in addition to text and audio or video messaging. In fact, the predominant type of plan clients opted for when they enrolled in the Talkspace platform was asynchronous messaging only (n=5389, 91.5%), although we do not know whether clients opted out of the other plans owing to cost, preference, or both.

    Limitations and Considerations

    A key consideration for our findings is the nature of clients enrolled in technology-mediated psychotherapy. The most common client characteristics were being of the millennial generation [38], college educated, single, identifying as female, and having experience with therapy. In addition, as the messaging-only plan was by far the most frequently chosen (91.5%), the study reflects clients willing or preferring to engage in primarily text messaging–based therapy, which may or may not be asynchronous. Men are similarly underrepresented in face-to-face in-person therapy [43] and represent a population that may need additional effort to reach and engage in technology-mediated psychotherapy. The population in our study may represent what could be considered early adopters of the innovation of text-based psychotherapy, which is also consistent with the premise that members of the millennial generation are more generally engaged with internet technology and more comfortable using it in myriad ways [44]. In addition, data were collected before the COVID-19 pandemic, when engaging in technology-mediated health care was novel for most people. Considering social distancing policies to control the spread of COVID-19, technology-mediated health care proliferated, and the business of web-based therapy grew tremendously as the general demand for psychotherapy simultaneously increased [7,45]. The postpandemic population may be different from the population included in our study. In fact, a recent study published with postpandemic Talkspace clients suggests that anxiety is a much more common experience and comorbidity among clients since the pandemic started [46].

    Our findings for depressive symptom change do not generalize well past the first 6 weeks of therapy, given that missing data on the PHQ-8 increases substantially after 6 weeks. A known contributing factor to missing data on the PHQ-8 is that clients stop taking the measure after disengaging from the therapy. Our estimates of later time points were based on the trajectory that was observed at the time the clients stopped taking the PHQ-8. Given that it is common for people with greater improvement in earlier phases of therapy to end therapy earlier than those whose symptoms do not remit as quickly [47], it is possible that the overall improvement in depression symptoms is an overestimation of what we would observe if clients continued taking PHQ-8 assessments. There are high rates of missing data, aside from that, which can be attributed to therapy disengagement. Therefore, these results may only be generalized to people who are receiving therapy and who complete the PHQ-8 assessments. In addition, given that we observed that higher intake PHQ-8 scores were predictive of symptom improvement, a likely caveat for our findings is that there is some regression to the mean, a statistical artifact of more extreme-scoring individuals to score more closely to the average on the next assessment. In randomized controlled trials, where regression to the mean is accounted for, it is more common to actually see higher initial depression scores as related to less symptom improvement [48-50]. Like all regression models, our curvilinear time modeling provides an overall average estimate of symptom trajectories and is not able to demonstrate what are likely nonlinear relationships with symptoms and time that would be observed if examined at the individual client level [51]. However, we can say that, on average, we see a trend for clients to do better symptomatically than when they enrolled.

    We conducted a naturalistic study using routinely collected data from the Talkspace platform. Therefore, there were limitations in the amount and nature of the data available. First, we did not account for other conditions that clients may be seeking therapy for, or conditions that may moderate therapeutic outcomes. For instance, the presence of a personality disorder is related to premature termination of therapy [23] and is known to impede symptom improvement [52]. The PHQ-8 does not capture suicidal thinking, so we do not have estimates of suicide risk using the item from the PHQ-9 that asks about thoughts that you would be better off dead, or of hurting yourself, which is known to be predictive of suicidal behavior and a common question asked as part of routine suicide risk screening procedures [53]. In addition, we do not know what type of therapy is provided by licensed clinicians; however, therapy from various theoretical orientations is known to be effective for depression [54]. The availability of textual data allows for an ample exploration of what is happening in routine psychotherapy in this context. Future research can use the rich data made available through text messages and transcripts made from audio or video interactions to examine the degree to which evidence-based interventions and practices are part of the therapy and how this might be related to disengagement, engagement, and symptom outcomes.

    Conclusions

    Technology-mediated psychotherapy is becoming an increasingly popular alternative to in-person psychotherapy. Our findings add to the growing literature showing the benefits of technology-mediated psychotherapy in general and messaging-based therapy in particular. In addition, benefits were observed over a relatively brief period, with a reduction in symptoms observed within 16 weeks of therapy. Some populations may require additional efforts to remain engaged once they enroll, such as in younger adults. Future research is needed to examine how well technology-mediated psychotherapy addresses disparities in access to therapy and opportunities to enhance reach and ensure the effectiveness of this treatment modality across the full spectrum of adults experiencing depression.

    Acknowledgments

    This research was supported by the National Institute of Mental Health of the National Institutes of Health under award numbers K23MH118361, P50MH115837, and R44MH124334. The content is the sole responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

    Authors' Contributions

    DD, MDP, TDH, and PA contributed to the conceptualization, review, and writing of the manuscript. DD, MDP, and SC formulated the data analytic approach. Data analysis was conducted using MDP and SC. DD, MDP, TDH, and PA drafted portions of the manuscript and provided substantive review of manuscript drafts. DD coordinated the manuscript preparation and revisions.

    Conflicts of Interest

    TDH is an employee of the service that provided the data.

    1. Key Substance Use and Mental Health Indicators in the United States: Results from the 2019 National Survey on Drug Use and Health. Rockville, MD: Center for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration; 2020.
    2. Kessler RC, Petukhova M, Sampson NA, Zaslavsky AM, Wittchen H. Twelve-month and lifetime prevalence and lifetime morbid risk of anxiety and mood disorders in the United States. Int J Methods Psychiatr Res 2012 Sep;21(3):169-184 [FREE Full text] [CrossRef] [Medline]
    3. Cuijpers P. Psychotherapies for adult depression: recent developments. Curr Opin Psychiatry 2015 Jan;28(1):24-29. [CrossRef] [Medline]
    4. McHugh RK, Whitton SW, Peckham AD, Welge JA, Otto MW. Patient preference for psychological vs pharmacologic treatment of psychiatric disorders. J Clin Psychiatry 2013 Jun 15;74(06):595-602. [CrossRef]
    5. Mohr DC, Hart SL, Howard I, Julian L, Vella L, Catledge C, et al. Barriers to psychotherapy among depressed and nondepressed primary care patients. Ann Behav Med 2006 Dec;32(3):254-258 [FREE Full text] [CrossRef] [Medline]
    6. Imel ZE, Caperton DD, Tanana M, Atkins DC. Technology-enhanced human interaction in psychotherapy. J Couns Psychol 2017 Jul;64(4):385-393 [FREE Full text] [CrossRef] [Medline]
    7. Steele A. Pandemic, new platforms prompt surge in new therapists. Wall Street Journal. 2020.   URL: https://www.wsj.com/articles/pandemic-new-platforms-prompt-surge-in-new-therapists-11601816400 [accessed 2021-12-28]
    8. Castro A, Gili M, Ricci-Cabello I, Roca M, Gilbody S, Perez-Ara MA, et al. Effectiveness and adherence of telephone-administered psychotherapy for depression: a systematic review and meta-analysis. J Affect Disord 2020 Jan 01;260:514-526. [CrossRef] [Medline]
    9. Osenbach JE, O'Brien KM, Mishkind M, Smolenski DJ. Synchronous telehealth technologies in psychotherapy for depression: a meta-analysis. Depress Anxiety 2013 Nov 06;30(11):1058-1067. [CrossRef] [Medline]
    10. Mohr DC, Vella L, Hart S, Heckman T, Simon G. The effect of telephone-administered psychotherapy on symptoms of depression and attrition: a meta-analysis. Clin Psychol (New York) 2008;15(3):243-253 [FREE Full text] [CrossRef] [Medline]
    11. Vernmark K, Lenndin J, Bjärehed J, Carlsson M, Karlsson J, Oberg J, et al. Internet administered guided self-help versus individualized e-mail therapy: a randomized trial of two versions of CBT for major depression. Behav Res Ther 2010 May;48(5):368-376. [CrossRef] [Medline]
    12. Kessler D, Lewis G, Kaur S, Wiles N, King M, Weich S, et al. Therapist-delivered internet psychotherapy for depression in primary care: a randomised controlled trial. Lancet 2009 Aug;374(9690):628-634. [CrossRef]
    13. Button KS, Wiles NJ, Lewis G, Peters TJ, Kessler D. Factors associated with differential response to online cognitive behavioural therapy. Soc Psychiatry Psychiatr Epidemiol 2012 May 4;47(5):827-833. [CrossRef] [Medline]
    14. Barak A, Hen L, Boniel-Nissim M, Shapira N. A comprehensive review and a meta-analysis of the effectiveness of internet-based psychotherapeutic interventions. J Technol Human Service 2008 Jul 03;26(2-4):109-160. [CrossRef]
    15. Nitzburg GC, Farber BA. Patterns of utilization and a case illustration of an interactive text-based psychotherapy delivery system. J Clin Psychol 2019 Feb 10;75(2):247-259. [CrossRef] [Medline]
    16. Hull TD, Mahan K. A study of asynchronous mobile-enabled SMS text psychotherapy. Telemed J E Health 2017 Mar;23(3):240-247. [CrossRef] [Medline]
    17. DellaCrosse M, Mahan K, Hull TD. The effect of messaging therapy for depression and anxiety on employee productivity. J technol behav sci 2018 Jun 26;4(1):1-5. [CrossRef]
    18. Hull TD, Malgaroli M, Connolly PS, Feuerstein S, Simon NM. Two-way messaging therapy for depression and anxiety: longitudinal response trajectories. BMC Psychiatry 2020 Jun 12;20(1):297 [FREE Full text] [CrossRef] [Medline]
    19. Posner K, Brown GK, Stanley B, Brent DA, Yershova KV, Oquendo MA, et al. The Columbia-Suicide Severity Rating Scale: initial validity and internal consistency findings from three multisite studies with adolescents and adults. Am J Psychiatry 2011 Dec;168(12):1266-1277 [FREE Full text] [CrossRef] [Medline]
    20. Kroenke K, Strine TW, Spitzer RL, Williams JB, Berry JT, Mokdad AH. The PHQ-8 as a measure of current depression in the general population. J Affect Disord 2009 Apr;114(1-3):163-173. [CrossRef] [Medline]
    21. Löwe B, Unützer J, Callahan CM, Perkins AJ, Kroenke K. Monitoring depression treatment outcomes with the patient health questionnaire-9. Med Care 2004 Dec;42(12):1194-1201. [CrossRef] [Medline]
    22. Singer JD, Willett JB. Applied Longitudinal Data Analysis: Modeling Change and Event Occurrence. Oxford, UK: Oxford Scholarship; 2003.
    23. Swift JK, Greenberg RP. Premature discontinuation in adult psychotherapy: a meta-analysis. J Consult Clin Psychol 2012 Aug;80(4):547-559. [CrossRef] [Medline]
    24. Cuijpers P, Karyotaki E, Reijnders M, Huibers MJ. Who benefits from psychotherapies for adult depression? A meta-analytic update of the evidence. Cogn Behav Ther 2018 Mar 18;47(2):91-106. [CrossRef] [Medline]
    25. Ünlü Ince B, Riper H, van 't Hof E, Cuijpers P. The effects of psychotherapy on depression among racial-ethnic minority groups: a metaregression analysis. Psychiatr Serv 2014 May 01;65(5):612-617. [CrossRef] [Medline]
    26. Huey SJ, Tilley JL, Jones EO, Smith CA. The contribution of cultural competence to evidence-based care for ethnically diverse populations. Annu Rev Clin Psychol 2014 Mar 28;10(1):305-338. [CrossRef] [Medline]
    27. Tambling RR, Hynes KC, D’Aniello C. Are barriers to psychotherapy treatment seeking indicators of social determinants of health?: a critical review of the literature. Am J Family Ther 2021 Jan 08:1-16. [CrossRef]
    28. Cooper AA, Conklin LR. Dropout from individual psychotherapy for major depression: a meta-analysis of randomized clinical trials. Clin Psychol Rev 2015 Aug;40:57-65. [CrossRef] [Medline]
    29. Racial/ethnic differences in mental health service use among adults. Substance Abuse and Mental Health Services Administration.   URL: https://www.samhsa.gov/data/report/racialethnic-differences-mental-health-service-use-among-adults [accessed 2021-12-27]
    30. Barrera I, Longoria D. Examining cultural mental health care barriers among Latinos. J Leadership Equity Res 2018;4(1).
    31. Chen J, Rizzo J. Racial and ethnic disparities in use of psychotherapy: evidence from U.S. national survey data. Psychiatr Serv 2010 Apr;61(4):364-372. [CrossRef] [Medline]
    32. Thompson VL, Bazile A, Akbar M. African Americans' perceptions of psychotherapy and psychotherapists. Professional Psychol Res Practice 2004;35(1):19-26. [CrossRef]
    33. Renn BN, Hoeft TJ, Lee HS, Bauer AM, Areán PA. Preference for in-person psychotherapy versus digital psychotherapy options for depression: survey of adults in the U.S. NPJ Digit Med 2019 Feb 11;2(1):6 [FREE Full text] [CrossRef] [Medline]
    34. Marler W. Mobile phones and inequality: findings, trends, and future directions. New Media Soc 2018 Apr 07;20(9):3498-3520. [CrossRef]
    35. Holdsworth E, Bowen E, Brown S, Howat D. Client engagement in psychotherapeutic treatment and associations with client characteristics, therapist characteristics, and treatment factors. Clin Psychol Rev 2014 Jul;34(5):428-450. [CrossRef] [Medline]
    36. O'Brien A, Fahmy R, Singh SP. Disengagement from mental health services. A literature review. Soc Psychiatry Psychiatr Epidemiol 2009 Jul 26;44(7):558-568. [CrossRef] [Medline]
    37. Afifi TO, Cox BJ, Enns MW. Mental health profiles among married, never-married, and separated/divorced mothers in a nationally representative sample. Soc Psychiatry Psychiatr Epidemiol 2006 Feb 9;41(2):122-129. [CrossRef] [Medline]
    38. Millennial life: how young adulthood today compares with prior generations. Pew Research Center.   URL: https:/​/www.​researchgate.net/​publication/​337113826_Millennial_life_How_young_adulthood_today_compares_with_prior_generations [accessed 2021-12-27]
    39. Lee AA, Sripada RK, Hale AC, Ganoczy D, Trivedi RB, Arnow B, et al. Psychotherapy and depressive symptom trajectories among VA patients: comparing dose-effect and good-enough level models. J Consult Clin Psychol 2021 May;89(5):379-392. [CrossRef] [Medline]
    40. Simon GE, Imel ZE, Ludman EJ, Steinfeld BJ. Is dropout after a first psychotherapy visit always a bad outcome? Psychiatr Serv 2012 Jul;63(7):705-707 [FREE Full text] [CrossRef] [Medline]
    41. Barkham M, Connell J, Stiles WB, Miles JN, Margison F, Evans C, et al. Dose-effect relations and responsive regulation of treatment duration: the good enough level. J Consult Clin Psychol 2006;74(1):160-167. [CrossRef]
    42. Bruijniks SJ, Lemmens LH, Hollon SD, Peeters FP, Cuijpers P, Arntz A, et al. The effects of once- versus twice-weekly sessions on psychotherapy outcomes in depressed patients. Br J Psychiatry 2020 Apr 07;216(4):222-230. [CrossRef] [Medline]
    43. Seidler ZE, Rice SM, Ogrodniczuk JS, Oliffe JL, Shaw JM, Dhillon HM. Men, masculinities, depression: implications for mental health services from a Delphi expert consensus study. Professional Psychol Res Pract 2019 Feb;50(1):51-61. [CrossRef]
    44. Vogels E. Millennials stand out for their technology use, but older generations also embrace digital life. Pew Research Center.   URL: https://www.pewresearch.org/fact-tank/2019/09/09/us-generations-technology-use/ [accessed 2021-12-27]
    45. Worsening mental health crisis pressures psychologist workforce COVID-19 Practitioner Survey. American Psychological Association.   URL: https://www.apa.org/pubs/reports/practitioner/covid-19-2021 [accessed 2021-12-27]
    46. Hull TD, Levine J, Bantilan N, Desai AN, Majumder MS. Analyzing digital evidence from a telemental health platform to assess complex psychological responses to the COVID-19 pandemic: content analysis of text messages. JMIR Form Res 2021 Feb 09;5(2):e26190 [FREE Full text] [CrossRef] [Medline]
    47. Niileksela CR, Ghosh A, Janis RA. The dose-effect and good enough level models of change for specific psychological concerns. J Consult Clin Psychol 2021 Mar;89(3):200-213. [CrossRef] [Medline]
    48. Elkin I, Gibbons RD, Shea MT, Sotsky SM, Watkins JT, Pilkonis PA, et al. Initial severity and differential treatment outcome in the National Institute of Mental Health Treatment of Depression Collaborative Research Program. J Consult Clin Psychol 1995;63(5):841-847. [CrossRef]
    49. Delgadillo J, Moreea O, Lutz W. Different people respond differently to therapy: a demonstration using patient profiling and risk stratification. Behav Res Ther 2016 Apr;79:15-22. [CrossRef] [Medline]
    50. Katon W, Unützer J, Russo J. Major depression: the importance of clinical characteristics and treatment response to prognosis. Depress Anxiety 2010 Jan;27(1):19-26. [CrossRef] [Medline]
    51. Helmich MA, Wichers M, Olthof M, Strunk G, Aas B, Aichhorn W, et al. Sudden gains in day-to-day change: revealing nonlinear patterns of individual improvement in depression. J Consult Clin Psychol 2020 Feb;88(2):119-127. [CrossRef] [Medline]
    52. Newton-Howes G, Tyrer P, Johnson T. Personality disorder and the outcome of depression: meta-analysis of published studies. Br J Psychiatry 2006 Jan 02;188(1):13-20. [CrossRef] [Medline]
    53. Simon GE, Rutter CM, Peterson D, Oliver M, Whiteside U, Operskalski B, et al. Does response on the PHQ-9 Depression Questionnaire predict subsequent suicide attempt or suicide death? Psychiatr Serv 2013 Dec 01;64(12):1195-1202 [FREE Full text] [CrossRef] [Medline]
    54. Cuijpers P, van Straten A, Andersson G, van Oppen P. Psychotherapy for depression in adults: a meta-analysis of comparative outcome studies. J Consult Clin Psychol 2008 Dec;76(6):909-922. [CrossRef] [Medline]

    ‎
    CBT: cognitive behavior therapy
    OR: odds ratio
    PHQ: Patient Health Questionnaire

    Edited by A Mavragani; submitted 16.01.22; peer-reviewed by S Rostam Niakan Kalhori , A Tannoubi; comments to author 19.04.22; revised version received 06.05.22; accepted 22.05.22; published 22.06.22

    Copyright

    ©Doyanne Darnell, Michael D Pullmann, Thomas D Hull, Shiyu Chen, Patricia Areán. Originally published in JMIR Formative Research (https://formative.jmir.org), 22.06.2022.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.