This paper is in the following e-collection/theme issue:

Formative Evaluation of Digital Health Interventions (2491) Web-based and Mobile Health Interventions (3357) Digital Mental Health Interventions, e-Mental Health and Cyberpsychology (1565) Depression and Mood Disorders; Suicide Prevention (1392) Mental Health Issues in Adolescence (457)

Published on in Vol 7 (2023)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/52804, first published .
Exploring the Number of Web-Based Behavioral Health Coaching Sessions Associated With Symptom Improvement in Youth: Observational Retrospective Analysis

Exploring the Number of Web-Based Behavioral Health Coaching Sessions Associated With Symptom Improvement in Youth: Observational Retrospective Analysis

Exploring the Number of Web-Based Behavioral Health Coaching Sessions Associated With Symptom Improvement in Youth: Observational Retrospective Analysis

Authors of this article:

Darian Lawrence-Sidebottom1 Author Orcid Image ;   Landry Goodgame Huffman1 Author Orcid Image ;   Aislinn Beam1 Author Orcid Image ;   Rachael Guerra1 Author Orcid Image ;   Amit Parikh1 Author Orcid Image ;   Monika Roots1 Author Orcid Image ;   Jennifer Huberty1, 2 Author Orcid Image
Article Authors Cited by (1) Tweetations (2) Metrics

    Original Paper

    1Bend Health, Inc, Beaverton, OR, United States

    2FitMinded, Inc LLC, Phoenix, AZ, United States

    *these authors contributed equally

    Corresponding Author:

    Landry Goodgame Huffman, PhD

    Bend Health, Inc

    9450 SW Gemini Drive #58509

    Beaverton, OR, 97008

    United States

    Phone: 1 7708619131

    Email: landry.huffman@bendhealth.com


    Background: Rates of anxiety and depression have been increasing among children and adolescents for the past decade; however, many young people do not receive adequate mental health care. Digital mental health interventions (DMHIs) that include web-based behavioral health coaching are widely accessible and can confer significant improvements in youth anxiety and depressive symptoms. However, more research is necessary to determine the number of web-based coaching sessions that confer clinically significant improvements in anxiety and depressive symptoms in youth.

    Objective: This study uses data from a pediatric DMHI to explore the number of web-based coaching sessions required to confer symptom improvements among children and adolescents with moderate or moderately severe symptoms of anxiety and depression.

    Methods: We used retrospective data from a pediatric DMHI that offered web-based behavioral health coaching in tandem with self-guided access to asynchronous chat with practitioners, digital mental health resources, and web-based mental health symptom assessments. Children and adolescents who engaged in 3 or more sessions of exclusive behavioral health coaching for moderate to moderately severe symptoms of anxiety (n=66) and depression (n=59) were included in the analyses. Analyses explored whether participants showed reliable change (a decrease in symptom scores that exceeds a clinically established threshold) and stable reliable change (at least 2 successive assessments of reliable change). Kaplan-Meier survival analyses were performed to determine the median number of coaching sessions when the first reliable change and stable reliable change occurred for anxiety and depressive symptoms.

    Results: Reliable change in anxiety symptoms was observed after a median of 2 (95% CI 2-3) sessions, and stable reliable change in anxiety symptoms was observed after a median of 6 (95% CI 5-8) sessions. A reliable change in depressive symptoms was observed after a median of 2 (95% CI 1-3) sessions, and a stable reliable change in depressive symptoms was observed after a median of 6 (95% CI 5-7) sessions. Children improved 1-2 sessions earlier than adolescents.

    Conclusions: Findings from this study will inform caregivers and youth seeking mental health care by characterizing the typical time frame in which current participants show improvements in symptoms. Moreover, by suggesting that meaningful symptom improvement can occur within a relatively short time frame, these results bolster the growing body of research that indicates web-based behavioral health coaching is an effective form of mental health care for young people.

    JMIR Form Res 2023;7:e52804

    doi:10.2196/52804

    Keywords

    adolescents; anxiety; children; depression; digital mental health intervention; reliable change 


    Between 2016 and 2019, rates of anxiety and depression in youth increased dramatically, with 5.8 million children and adolescents diagnosed with anxiety and 2.7 million with depression [1]. However, 1 in 5 young people do not receive adequate mental health treatment [1], highlighting a crucial need for greater accessibility in pediatric mental health services. Digital mental health interventions (DMHIs) have emerged as scalable and accessible alternatives to traditional forms of mental health care [2,3]. DMHIs, which encompass a range of mental health services administered through electronic modalities such as digital video chat, smartphone apps, and chat rooms [4], can confer comparable therapeutic benefits while also presenting fewer barriers to care [5]. Behavioral health coaching, which provides nonclinical and goal-focused mental health care to those with mild to moderate symptoms [6], is often more accessible and affordable than traditional therapy [7,8]. Moreover, DMHIs that use web-based behavioral health coaching can confer significant decreases in adult anxiety and depressive symptoms, as well as greater adherence, when compared with purely self-guided digital interventions [9-13]. Preliminary evidence also suggests that web-based behavioral health coaching is linked to decreases in anxiety and depressive symptoms for children and adolescents participating in a pediatric DMHI [14].

    Although it is well known that greater engagement (eg, more sessions and longer participation) in DMHIs that include web-based coaching is linked to increased therapeutic benefits [14-17], more research is necessary to identify the number of web-based coaching sessions that confer clinically significant improvements in anxiety and depressive symptoms in youth. Understanding these nuances in care will not only enable comparisons between DMHIs and traditional in-person interventions but also help caregivers gauge the time and commitment necessary to see meaningful changes in their children [18]. Therefore, this study aims to provide a preliminary exploration of the number of web-based coaching sessions required to confer symptom improvements among children and adolescents with moderate or moderately severe symptoms of anxiety and depression.


    Participants

    Bend Health Inc members (aged between 6 and 17 years) participating in behavioral health coaching only (no therapy) for at least 3 months between January and August 2023 were eligible for inclusion in the study (N=392).

    Treatment

    As described previously [14,17], Bend Health Inc is a pediatric DMHI in which child and adolescent members engage in the following services: digital behavioral video conferencing (synchronous) sessions with mental health providers, a web-based learning resource center, web-based asynchronous messaging with mental health providers (caregiver login), and mental health symptom assessments. Most members are referred by their primary care provider, or they access Bend through insurance, employer benefits, or direct-to-consumer pathways. Each member is assigned a behavioral care manager, who is responsible for overseeing a member’s progress and integrating their care between mental health and medical providers. While members may attend coaching or therapy sessions, only members in exclusive coaching were included in this study. Bend coaches are certified behavioral coaches or masters-level mental health providers trained in standard coaching techniques. During the 30-minute coaching sessions, coaches guide members and their families through structured care programs, which are intended to provide evidence-based behavior-change tools to target a particular mental health symptom. Coaches use tools based on cognitive behavioral therapy, parent management training, behavioral activation, mindfulness-based cognitive therapy, mindfulness-based stress reduction, and motivational interviewing. Members also have access to the content of their care program in a learning resource center, where they can practice and refresh their skills in an asynchronous format. Furthermore, caregivers may message their child’s mental health providers through a secure web-based portal. Some members also receive care from a psychiatric provider, depending on insurance benefits, primary care provider referral, and mental health symptom acuity.

    Measures

    At enrollment with Bend Health Inc, caregivers report their child or adolescent’s date of birth, sex at birth, gender, and race or ethnicity, as described elsewhere [14]. Mental health screening questions are also completed by caregivers of children and adolescents at enrollment. These questions are from the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5), cross-cutting measures for Level 1 caregiver-report (children) and self-report (adolescent) [19], which ask about the frequency of symptoms in the past 2 weeks. Then, full validated assessments are completed to thoroughly assess symptoms flagged by the screening questions using established scoring criteria.

    For children, the full caregiver-reported assessments for anxiety and depressive symptoms are the PROMIS (Patient Reported Outcomes Measurement Information System) measures [20,21], which are 10-items and 11-items, respectively. For adolescents, the full self-reported anxiety symptom assessment is the Generalized Anxiety Disorder 7-item (GAD-7) measure [22], and the full depressive symptom assessment is the Patient Health Questionnaire 9-item adolescent (PHQ-9A) version [23]. The question about suicide was removed from the PHQ-9A in this study, so it was 8 items in length. Assessments (screeners and full assessments) are repeated approximately monthly during care. Further details on the demographic information collected at enrollment, as well as mental health screener questions and scoring criteria, can be found in Multimedia Appendix 1.

    Statistical Analysis

    To ensure that study participants had similar care characteristics, the inclusion criteria listed in Textbox 1 were used:

    Textbox 1. Inclusion criteria for the study participants.

    Inclusion criteria

    • 2 completed assessments (baseline and during coaching; n=24 excluded)
    • 1.5-2.5 sessions per month (n=101 excluded)
    • 1-3 weeks between-sessions (n=77 excluded)
    • 3 or more sessions attended (n=37 excluded)

    Scores from each assessment were aggregated, and then, using standard methods, PROMIS assessment scores were converted to T-scores [10,11], and PHQ-9A scores were adjusted to account for the removed item [23]. Aggregate GAD-7 scores were not converted. Only the first assessment between sessions was retained, and baseline symptom severity was assessed before coaching using established criteria [20-23]. Final analyses were performed using data from members with anxiety or depressive symptoms of moderate or moderately severe severity at baseline. Member characteristics and care statistics (duration in coaching, sessions per month, and between-session duration) were reported for the anxiety and depressive symptoms groups.

    The reliable change criterion (RCC) was calculated for each validated assessment using published values for measure validity and data from all Bend Health Inc members at enrollment in 2023 for measure variance (Table S1 in Multimedia Appendix 1 contains calculated values) [24]. Then, scores from each assessment during coaching were compared against the corresponding RCC. A participant showed reliable change if (1) their scores decreased from baseline by a value greater than the RCC or (2) they screened out of the full assessment. A participant had stable reliable change when they completed 2 successive assessments with reliable change. Kaplan-Meier survival analyses were performed to determine the median number of coaching sessions when the first reliable change and stable reliable change occurred for anxiety and depressive symptoms [25]. For members with no change, the total number of coaching sessions attended was used in the survival analyses. Follow-up group-wise survival analyses were repeated on a children-only subsample (aged between 6 and 12 years) and an adolescent-only subsample (aged between 13 and 17 years) to assess for differences in age and assessment type.

    Ethical Considerations

    All study protocols were reviewed and approved by the Biomedical Research Alliance of New York (Study 23-12-034-1374). Informed consent was obtained at enrollment for participation with Bend Health Inc services after an overview of the services, data use, and privacy policy. Caregivers consented on behalf of child members (aged between 1 and 12 years), and adolescent members (aged between 13 and 17 years) consented on their own behalf. All human participant data were deidentified before analysis. Study participants received no compensation for their inclusion in the study.


    Member Characteristics and Participation in Care

    There were 66 members in the anxiety symptoms group and 59 members in the depressive symptoms group. Comprehensive member demographics are reported in Table 1. Members completed between 2 and 7 assessments (Table 2; the anxiety symptoms group completed a median of 4 [IQR 3-5] assessments, and the depressive symptoms group completed a median of 4 [IQR 3-4.5] assessments). Members participated in coaching for 3.00-7.23 months, attending 3-17 sessions (Table 3). Members in the anxiety symptoms group were in coaching for 4.32 (SD 1.07) months, and they attended a median of 7 (IQR 5.25-10.0) coaching sessions. Members in the depressive symptoms group were in coaching for 4.34 (SD 1.09) months, and they attended a median of 7 (IQR 6-9) coaching sessions. The anxiety symptoms group had 1.97 (SD 0.30) sessions per month (2.19, SD 0.37 weeks between sessions). The depressive symptoms group had 1.99 (SD 0.29) sessions per month (2.19, SD 0.37 weeks between sessions).

    Table 1. Demographic characteristics of members in the analyses for anxiety symptoms and elevated depressive symptoms.
    DemographicAnxiety symptoms (n=66)Depressive symptoms (n=59)
    Age (years), mean (SD)11.0 (3.2)12.4 (3.1)
    Child, n (%)43 (65)26 (44)
    Adolescent, n (%)23 (35)33 (56)
    Sex, n (%)

    		Female34 (52)33 (56)

    		Male31 (47)25 (42)

    		Other1 (2)1 (2)
    Gender conformity, n (%)

    		Conforming63 (96)57 (97)

    		Nonconforming3 (4)2 (3)
    Ethnicity, n (%)

    		Asian4 (6)2 (3)

    		Black or African American1 (2)0 (0)

    		Hispanic or Latino2 (3)4 (7)

    		White30 (46)33 (56)

    		Other or multiracial29 (44)20 (34)
    Table 2. Members that completed assessments 1-6 by symptom group.
    AssessmentAnxiety symptoms (n=66), n (%)Depressive symptoms (n=59), n (%)
    166 (100)59 (100)
    266 (100)59 (100)
    361 (92)56 (95)
    441 (62)37 (63)
    521 (32)15 (25)
    66 (9)2 (3)
    Table 3. Members that attended each coaching session by symptom group.
    Coaching sessionAnxiety symptoms (n=66), n (%)Depressive symptoms (n=59), n (%)
    366 (100)59 (100)
    465 (98)55 (93)
    556 (85)52 (88)
    649 (74)46 (78)
    738 (58)33 (56)
    827 (41)25 (42)
    920 (30)19 (32)
    1019 (29)14 (24)
    1112 (18)7 (12)
    128 (12)3 (5)
    134 (6)2 (3)
    142 (3)1 (2)
    151 (2)0 (0)

    Anxiety Symptoms: Time to Improvement

    For anxiety symptoms over coaching with the DMHI, a total of 96% (63/66) of members had reliable improvements, and 71% (47/66) of members exhibited stable reliable improvements (Table S2 in Multimedia Appendix 1). Reliable change in anxiety symptoms was observed after a median of 2 (95% CI 2-3) sessions, and stable reliable change in anxiety symptoms was observed after a median of 6 (95% CI 5-8) sessions (Figure 1). For children, reliable change was observed after a median of 2 (95% CI 2-3) sessions, and stable reliable change was observed after a median of 6 (95% CI 4-8) sessions. For adolescents, reliable change was observed after a median of 3 (95% CI 2-7) sessions, and stable reliable change was observed after a median of 6 (95% CI 5-not available) sessions.

    ‎
    Figure 1. Survival plots for anxiety symptoms over coaching sessions. The solid black line indicates proportion with no reliable change (survival). The dotted lines indicate the CI. The blue line on the y-axis indicates a proportion of 0.5. A: Reliable change. B: Stable reliable change.

    Depressive Symptoms: Time to Improvement

    For depressive symptoms over coaching with the DMHI, a total of 98% (58/59) of members had reliable improvements, and 71% (42/59) of members exhibited stable reliable improvements (Table S3 in Multimedia Appendix 1). Reliable change in depressive symptoms was observed after a median of 2 (95% CI 1-3) sessions, and stable reliable change in depressive symptoms was observed after a median of 6 (95% CI 5-7) sessions (Figure 2). For children, reliable change was observed after a median of 1.5 (95% CI 1-2) sessions, and stable reliable change was observed after a median of 5 (95% CI 4-11) sessions. For adolescents, reliable change was observed after a median of 3 (95% CI 1-4) sessions, and stable reliable change was observed after a median of 6 (95% CI 5-10) sessions.

    ‎
    Figure 2. Survival plots for depressive symptoms over coaching sessions. The solid black line indicates proportion with no reliable change (survival). The dotted lines indicate the CI. The blue line on the y-axis indicates a proportion of 0.5. A: Reliable change. B: Stable reliable change.

    Principal Findings

    The purpose of this study was to provide a preliminary exploration of the number of web-based coaching sessions associated with clinically significant reductions in symptoms among children and adolescents with moderate to moderately severe symptoms of anxiety and depression. For both anxiety and depressive symptoms, most members showed reliable symptom improvements after 2 coaching sessions and stable improvements after 6 coaching sessions. Children improved 1-2 sessions earlier than adolescents. This is the first study to suggest that reliable and stable changes in anxiety and depressive symptoms can be observed for most participants in a pediatric DMHI after just six 30-minute web-based coaching sessions.

    Although preliminary, these findings provide valuable information to those seeking pediatric mental health treatment for anxiety and depressive symptoms. There are many barriers to accessing mental health care, with affordability being one of them [26]. If caregivers are able to anticipate the number of sessions that will likely lead to improvements in their children’s symptoms, they may be more willing and able to make the financial commitment to their children’s treatment. Additionally, when caregivers and youth have a better understanding of the approximate treatment length, they may be more likely to continue attending their coaching appointments when they encounter factors commonly associated with early dropout from mental health treatment (eg, lack of stability, lack of motivation, symptom severity, or parental stress) [27-29]. Indeed, several studies of traditional therapeutic modalities have found that clients typically attend the number of sessions they expect to attend, suggesting that a realistic expectation for participation will maximize families’ engagement in their children’s mental health care [29-31].

    Comparison With Previous Work

    Adolescents had improvements in symptoms about 1-2 more sessions later than children. While the participants in this study all had moderate or moderately severe symptom severity at baseline, other studies comparing child and adolescent responses to clinical behavioral interventions have found that adolescents typically exhibit more severe symptoms and a less favorable treatment response than children [32,33]. Older youth with anxiety or depressive symptoms may also experience increased distress as their symptoms interact with socioemotional and environmental changes characteristic of adolescence, such as increased autonomy and greater stress at school. Alternatively, this marginal age-based difference may reflect biases in caregiver versus self-report or differences in the sensitivities of the measures used to assess symptoms in children and adolescents. Although further study is necessary to replicate our results, these preliminary findings suggest the advantage of tailoring pediatric DMHI programs to age.

    Limitations

    Our findings are limited by several factors. Given that the reliable change criteria used for the measures in this study were relatively small, further study is necessary to confirm whether 6 coaching sessions consistently confer substantive improvements in mental health symptoms. However, our results are consistent with a systematic review, which found that low-intensity in-person psychotherapy yielded optimal results between 4 and 6 sessions [34]. This study also did not include a nonactive control group; therefore, we could not assess whether coaching alone produced improvements in mental health symptoms. Future studies would be bolstered by the inclusion of a control group to address this limitation. Moreover, we were unable to quantify changes in symptoms for members who screened out of taking the full assessments at later time points. We addressed this potential bias by also assessing stable reliable change as a measure of symptom improvement, thus ensuring our results were not driven by those with screened-out assessments. Finally, we assessed symptom severity approximately every 30 days, rather than after every coaching session. Given that members completed approximately 2 coaching sessions per month, symptom improvements after the coaching sessions between assessments were not reflected in the present results. Therefore, future studies would benefit from improved measurement and timing of mental health symptom assessments.

    Conclusions

    This study provides compelling preliminary evidence that participation in 2-6 web-based coaching sessions is associated with reliable improvements among youth with moderate or moderately severe anxiety and depressive symptoms. Findings from this study inform caregivers and youth seeking mental health care by characterizing the typical time frame in which current participants show improvements in symptoms. Moreover, by suggesting that meaningful symptom improvement can occur within a relatively short time frame, these results bolster the growing body of research that indicates web-based behavioral health coaching is an effective form of mental health care for young people.

    Acknowledgments

    LGH and DLS contributed to conceptualization, methodology, formal analysis, writing the original draft, manuscript reviewing and editing, and visualization of this study. JH contributed to the conceptualization, writing of the original draft, manuscript reviewing and editing, and supervision of this study. MR reviewed and edited the paper and acquired funds. AP and RG reviewed and edited the paper. This work was funded by Bend Health, Inc.

    Data Availability

    The data sets analyzed during this study are not publicly available, as this would violate Bend Health Inc’s privacy policy, but they are available from the corresponding author on reasonable request.

    Conflicts of Interest

    All authors are employed by Bend Health Inc, which delivered the treatment used in this retrospective study. However, authors’ employment status and salary are not dependent upon the results of their research.

    Multimedia Appendix 1

    Supplementary material and tables.

    DOCX File , 22 KB
    1. Lebrun-Harris LA, Ghandour RM, Kogan MD, Warren MD. Five-year trends in US children's health and well-being, 2016-2020. JAMA Pediatr. 2022;176(7):e220056. [FREE Full text] [CrossRef] [Medline]
    2. Naslund JA, Aschbrenner KA, Araya R, Marsch LA, Unützer J, Patel V, et al. Digital technology for treating and preventing mental disorders in low-income and middle-income countries: a narrative review of the literature. Lancet Psychiatry. Jun 2017;4(6):486-500. [FREE Full text] [CrossRef] [Medline]
    3. Schueller SM, Hunter JF, Figueroa C, Aguilera A. Use of digital mental health for marginalized and underserved populations. Curr Treat Options Psych. Jul 5, 2019;6(3):243-255. [CrossRef]
    4. Lehtimaki S, Martic J, Wahl B, Foster KT, Schwalbe N. Evidence on digital mental health interventions for adolescents and young people: systematic overview. JMIR Ment Health. Apr 29, 2021;8(4):e25847. [FREE Full text] [CrossRef] [Medline]
    5. Khanna MS, Carper M. Digital mental health interventions for child and adolescent anxiety. Cognitive and Behavioral Practice. Feb 2022;29(1):60-68. [CrossRef]
    6. Jones RJ, Woods SA, Guillaume YRF. The effectiveness of workplace coaching: a meta‐analysis of learning and performance outcomes from coaching. J Occupat & Organ Psyc. Apr 15, 2015;89(2):249-277. [CrossRef]
    7. Kivelä K, Elo S, Kyngäs H, Kääriäinen M. The effects of health coaching on adult patients with chronic diseases: a systematic review. Patient Educ Couns. Nov 2014;97(2):147-157. [CrossRef] [Medline]
    8. Rosenberg BM, Kodish T, Cohen ZD, Gong-Guy E, Craske MG. A novel peer-to-peer coaching program to support digital mental health: design and implementation. JMIR Ment Health. Jan 26, 2022;9(1):e32430. [FREE Full text] [CrossRef] [Medline]
    9. Loo T, Hunt J, Grodberg D, Bravata D. Clinical and psychosocial outcomes associated with a tele-behavioral health platform for families: retrospective study. JMIR Form Res. Mar 17, 2023;7:e43600. [FREE Full text] [CrossRef] [Medline]
    10. Lungu A, Boone MS, Chen SY, Chen CE, Walser RD. Effectiveness of a cognitive behavioral coaching program delivered via video in real world settings. Telemed J E Health. Jan 2021;27(1):47-54. [FREE Full text] [CrossRef] [Medline]
    11. Mohr DC, Duffecy J, Ho J, Kwasny M, Cai X, Burns MN, et al. A randomized controlled trial evaluating a manualized TeleCoaching protocol for improving adherence to a web-based intervention for the treatment of depression. PLoS One. 2013;8(8):e70086. [FREE Full text] [CrossRef] [Medline]
    12. Hippman C, Mah JWT, MacFadden M. Virtual delivery of parent coaching interventions in early childhood mental health: a scoping review. Child Psychiatry Hum Dev. Sep 23, 2023 [FREE Full text] [CrossRef] [Medline]
    13. Wu MS, Chen SY, Wickham RE, O'Neil-Hart S, Chen C, Lungu A. Outcomes of a blended care coaching program for clients presenting with moderate levels of anxiety and depression: pragmatic retrospective study. JMIR Ment Health. Oct 21, 2021;8(10):e32100. [FREE Full text] [CrossRef] [Medline]
    14. Huffman LG, Lawrence-Sidebottom D, Huberty J, Roots M, Roots K, Parikh A, et al. Using digital measurement-based care for the treatment of anxiety and depression in children and adolescents: observational retrospective analysis of Bend Health data. JMIR Pediatr Parent. Apr 20, 2023;6:e46154. [FREE Full text] [CrossRef] [Medline]
    15. Lappalainen P, Lappalainen R, Keinonen K, Kaipainen K, Puolakanaho A, Muotka J, et al. In the shadow of COVID-19: a randomized controlled online ACT trial promoting adolescent psychological flexibility and self-compassion. J Contextual Behav Sci. Jan 2023;27:34-44. [FREE Full text] [CrossRef] [Medline]
    16. Jeannotte AM, Hutchinson DM, Kellerman GR. The time to change for mental health and wellbeing via virtual professional coaching: longitudinal observational study. J Med Internet Res. May 13, 2021;23(7):e27774. [FREE Full text] [CrossRef] [Medline]
    17. Lawrence-Sidebottom D, Huffman LG, Huberty J, Beatty C, Roots M, Roots K, et al. Using digital measurement-based care to address symptoms of inattention, hyperactivity, and opposition in youth: retrospective analysis of Bend Health. JMIR Form Res. Apr 26, 2023;7:e46578. [FREE Full text] [CrossRef] [Medline]
    18. Strauss J, Zhang J, Jarrett ML, Patterson B, Van AM. 18 Apps for mental health. Ment Health Digit World. 2022:395-433. [FREE Full text] [CrossRef]
    19. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders: DSM-5. Washington, D.C. American Psychiatric Association; 2013.
    20. Irwin DE, Gross HE, Stucky BD, Thissen D, DeWitt EM, Lai JS, et al. Development of six PROMIS pediatrics proxy-report item banks. Health Qual Life Outcomes. 2012;10:22. [FREE Full text] [CrossRef] [Medline]
    21. Sherlock P, Blackwell CK, Kallen MA, Lai JS, Cella D, Krogh-Jespersen S, et al. Measuring PROMIS® emotional distress in early childhood. J Pediatr Psychol. 2022;47(5):547-558. [FREE Full text] [CrossRef] [Medline]
    22. Spitzer RL, Kroenke K, Williams JBW, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006;166(10):1092-1097. [FREE Full text] [CrossRef] [Medline]
    23. Johnson JG, Harris ES, Spitzer RL, Williams JBW. The Patient Health Questionnaire for Adolescents: validation of an instrument for the assessment of mental disorders among adolescent primary care patients. J Adolesc Health. 2002;30(3):196-204. [CrossRef] [Medline]
    24. Jacobson NS, Truax P. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. In: Kazdin AE, editor. Methodological Issues & Strategies in Clinical Research. Washington, D.C. American Psychological Association; 1992;631-648.
    25. Kaplan EL, Meier P. Nonparametric estimation from incomplete observations. J Am Stat Assoc. 1958;53(282):457-481. [CrossRef]
    26. Coombs NC, Meriwether WE, Caringi J, Newcomer SR. Barriers to healthcare access among U.S. adults with mental health challenges: a population-based study. SSM Popul Health. 2021;15:100847. [FREE Full text] [CrossRef] [Medline]
    27. de Haan AM, Boon AE, de Jong JTVM, Hoeve M, Vermeiren RRJM. A meta-analytic review on treatment dropout in child and adolescent outpatient mental health care. Clin Psychol Rev. 2013;33(5):698-711. [CrossRef] [Medline]
    28. O'Keeffe S, Martin P, Target M, Midgley N. 'I just stopped going': a mixed methods investigation into types of therapy dropout in adolescents with depression. Front Psychol. 2019;10:75. [FREE Full text] [CrossRef] [Medline]
    29. Beck NC, Lamberti J, Gamache M, Lake EA, Fraps CL, McReynolds WT, et al. Situational factors and behavioral self-predictions in the identification of clients at high risk to drop out of psychotherapy. J Clin Psychol. 1987;43(5):511-520. [CrossRef] [Medline]
    30. Jenkins SJ, Fuqua DR, Blum CR. Factors related to duration of counseling in a university counseling center. Psychol Rep. 1986;58(2):467-472. [CrossRef] [Medline]
    31. Pekarik G, Wierzbicki M. The relationship between clients' expected and actual treatment duration. Psychother Theory Res Pract Train. 1986;23(4):532-534. [CrossRef]
    32. Wehmeier PM, Schacht A, Escobar R, Savill N, Harpin V. Differences between children and adolescents in treatment response to atomoxetine and the correlation between health-related quality of life and attention deficit/hyperactivity disorder core symptoms: meta-analysis of five atomoxetine trials. Child Adolesc Psychiatry Ment Health. 2010;4:30. [FREE Full text] [CrossRef] [Medline]
    33. Southam-Gerow MA, Kendall PC, Weersing VR. Examining outcome variability: correlates of treatment response in a child and adolescent anxiety clinic. J Clin Child Psychol. 2001;30(3):422-436. [CrossRef] [Medline]
    34. Robinson L, Delgadillo J, Kellett S. The dose-response effect in routinely delivered psychological therapies: a systematic review. Psychother Res. 2020;30(1):79-96. [CrossRef] [Medline]

    ‎
    DMHI: digital mental health intervention
    DSM-5: Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition
    GAD-7: Generalized Anxiety Disorder 7-item
    PHQ-9A: Patient Health Questionnaire 9-item adolescent
    PROMIS: Patient Reported Outcomes Measurement Information System
    RCC: reliable change criterion

    Edited by A Mavragani; submitted 15.09.23; peer-reviewed by J Ohan; comments to author 06.11.23; revised version received 07.11.23; accepted 27.11.23; published 18.12.23.

    Copyright

    ©Darian Lawrence-Sidebottom, Landry Goodgame Huffman, Aislinn Beam, Rachael Guerra, Amit Parikh, Monika Roots, Jennifer Huberty. Originally published in JMIR Formative Research (https://formative.jmir.org), 18.12.2023.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.