This paper is in the following e-collection/theme issue:

Formative Evaluation of Digital Health Interventions (2226) e-Mental Health and Cyberpsychology (1411) Mental Health Issues in Adolescence (386) Web-based and Mobile Health Interventions (3128) Users' and Patients' Needs for Mental Health Services (382)

Published on in Vol 7 (2023)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/47917, first published .
Predicting Youth and Young Adult Treatment Engagement in a Transdiagnostic Remote Intensive Outpatient Program: Latent Profile Analysis

Predicting Youth and Young Adult Treatment Engagement in a Transdiagnostic Remote Intensive Outpatient Program: Latent Profile Analysis

Predicting Youth and Young Adult Treatment Engagement in a Transdiagnostic Remote Intensive Outpatient Program: Latent Profile Analysis

Authors of this article:

Kate Gliske1 Author Orcid Image ;   Katie R Berry1 Author Orcid Image ;   Jaime Ballard2 Author Orcid Image ;   Clare Schmidt1 Author Orcid Image ;   Elizabeth Kroll1 Author Orcid Image ;   Jonathan Kohlmeier1 Author Orcid Image ;   Michael Killian3 Author Orcid Image ;   Caroline Fenkel1 Author Orcid Image
Article Authors Cited by Tweetations (2) Metrics

    Original Paper

    1Charlie Health Inc, Bozeman, MT, United States

    2Center For Applied Research and Educational Improvement, University of Minnesota, St. Paul, MN, United States

    3College of Social Work, Florida State University, Tallahassee, FL, United States

    Corresponding Author:

    Kate Gliske, PhD

    Charlie Health Inc

    233 E Main St

    Ste 233

    Bozeman, MT, 59715-5045

    United States

    Phone: 1 9523341411

    Email: kgliske@umn.edu


    Background: The youth mental health crisis in the United States continues to worsen, and research has shown poor mental health treatment engagement. Despite the need for personalized engagement strategies, there is a lack of research involving youth. Due to complex youth developmental milestones, there is a need to better understand clinical presentation and factors associated with treatment engagement to effectively identify and tailor beneficial treatments.

    Objective: This quality improvement investigation sought to identify subgroups of clients attending a remote intensive outpatient program (IOP) based on clinical acuity data at intake, to determine the factors associated with engagement outcomes for clients who present in complex developmental periods and with cooccurring conditions. The identification of these subgroups was used to inform programmatic decisions within this remote IOP system.

    Methods: Data were collected as part of ongoing quality improvement initiatives at a remote IOP for youth and young adults. Participants included clients (N=2924) discharged between July 2021 and February 2023. A latent profile analysis was conducted using 5 indicators of clinical acuity at treatment entry, and the resulting profiles were assessed for associations with demographic factors and treatment engagement outcomes.

    Results: Among the 2924 participants, 4 profiles of clinical acuity were identified: a low-acuity profile (n=943, 32.25%), characterized by minimal anxiety, depression, and self-harm, and 3 high-acuity profiles defined by moderately severe depression and anxiety but differentiated by rates of self-harm (high acuity+low self-harm: n=1452, 49.66%; high acuity+moderate self-harm: n=203, 6.94%; high acuity+high self-harm: n=326, 11.15%). Age, gender, transgender identity, and sexual orientation were significantly associated with profile membership. Clients identified as sexually and gender-marginalized populations were more likely to be classified into high-acuity profiles than into the low-acuity profile (eg, for clients who identified as transgender, high acuity+low self-harm: odds ratio [OR] 2.07, 95% CI 1.35-3.18; P<.001; high acuity+moderate self-harm: OR 2.85, 95% CI 1.66-4.90; P<.001; high acuity+high self-harm: OR 3.67, 95% CI 2.45-5.51; P<.001). Race was unrelated to the profile membership. Profile membership was significantly associated with treatment engagement: youth and young adults in the low-acuity and high-acuity+low–self-harm profiles attended an average of 4 fewer treatment sessions compared with youth in the high-acuity+moderate–self-harm and high-acuity+high–self-harm profiles (ꭓ23=27.6, P<.001). Individuals in the high-acuity+low–self-harm profile completed treatment at a significantly lower rate relative to the other 2 high-acuity profiles (ꭓ23=13.4, P=.004). Finally, those in the high-acuity+high–self-harm profile were significantly less likely to disengage early relative to youth in all other profiles (ꭓ23=71.12, P<.001).

    Conclusions: This investigation represents a novel application for identifying subgroups of adolescents and young adults based on clinical acuity data at intake to identify patterns in treatment engagement outcomes. Identifying subgroups that differentially engage in treatment is a critical first step toward targeting engagement strategies for complex populations.

    JMIR Form Res 2023;7:e47917

    doi:10.2196/47917

    Keywords

    youth; young adult; virtual; mental health; intensive outpatient treatment; latent profile analysis; personalized treatment 


    Background

    Youth mental health crises continue to persist and grow across the United States [1,2]. Nearly 20% of youth and young adults have a diagnosed mental health condition [2], whereas 20% have seriously considered suicide in the past year. Marginalized groups such as Indigenous; female; and lesbian, gay, bisexual, transgender, queer (or questioning), asexual (or allied), intersex (LGBTQIA+) youth report even higher rates of suicidality [3].

    Psychological interventions can effectively address these mental health concerns, leading to reduced symptoms and improved functioning, productivity, and life span [4,5]. The outcomes are particularly strong with early intervention during adolescence and young adulthood [6,7]. However, many youth and young adults (23%-63%) drop out of treatment before completing the recommended dose [8], leading to a smaller reduction in symptoms and a lower likelihood of maintaining improvement [9]. The purpose of this quality improvement study was to use person-centered analyses to identify client subgroups based on the initial symptom severity of youth and young adult clients of a nationwide telehealth intensive outpatient program (IOP) and to explore differences in treatment engagement between groups.

    Treatment Engagement

    Treatment engagement is associated with better treatment outcomes [10,11]. However, mental health treatment dropout rates are high, particularly among younger clients [12]. Across treatment settings and diagnoses, dropout rates generally range between one-fourth and over half of all clients [8]. Although some patients end treatment because they have seen improvement and feel ready to leave early, others drop out because of changes in motivation or perceptions of usefulness [13] or a lack of stability in their lives [14].

    Clients who drop out early are less likely to have significant improvements in depression or anxiety [10,11] or to maintain improvement [9] and have smaller reductions in trauma symptoms. Similar patterns were observed in the number of sessions attended; for example, in one study of outpatient depression treatments, each additional session attended was associated with a 20% increased likelihood of clinically significant improvement [11]. Therefore, it is critical that providers identify clients at risk of dropout and implement strategies to increase engagement.

    Variable-centered research on individual predictors of dropout among youth and young adults has indicated few reliable pretreatment predictors [15]. Machine learning research on dropout among adults has improved predictive power [12]; however, this research has not been conducted with youth. Patient stratification through group analysis may be ideally positioned to identify youth and young adult patient profiles at a higher risk of dropout and disengagement.

    Patient Stratification and Mental Health Care Engagement

    Across health care, extensive efforts are underway to develop methods to predict individual responses to treatments and select the ideal treatment and engagement strategies for the individual, such as decision support algorithms [16]. Within mental health care for adults, these methods are in development, and several have been tested in clinical trials [17,18]. Patient stratification analyses, such as latent class analysis, can identify subgroups of clients at the highest risk of poor mental health treatment engagement [19] or clinical outcomes [20]. Identifying high-risk subgroups can guide the choice of effective treatment strategies [21]; for example, midweek support calls can increase treatment engagement among clients with depression, but this intervention would be costly to implement for all clients [22]. Patient stratification could identify those at the greatest risk for dropout and indicate targeted engagement interventions. This latent class analysis approach allows for the simultaneous prediction of multiple outcomes and may prove to be the optimal method for treatment selection decisions under real-world circumstances [23].

    One landmark study used a large data set of adult clients receiving stepped-care treatment for depression and anxiety disorders to identify subgroups and explore whether there were differences in dropout between subgroups, as well as differences in recovery and reliable change [19]. The authors identified and replicated subgroups that robustly distinguished profiles that were most likely to drop out and differences in dropout by class between 2 different interventions. This method of developing profiles has the advantage of seamlessly integrating into current clinical care, where client intake information can be used to identify profile alignment and guide effective treatment and engagement planning.

    Lack of Research on Youth

    Despite the potential advantages of tailored treatment and engagement strategies, few studies have specifically assessed the profiles and differential outcomes among youth and young adults [24]. As treatment dropout is higher among youth and young adults, there is a greater need for strategies that help them remain in treatment long enough to benefit [24,25]. Although several engagement strategies have demonstrated effectiveness, researchers have not consistently identified who could benefit the most. A foundational meta-analysis found 3 significant dropout predictors among youth: contact with deviant peers, externalizing problems, and ethnic minority group status [8]. In addition to these factors, research has found few consistent predictors of dropout among youth, although older age and male sex have been reported in some studies [15,25,26].

    This challenge in identifying consistent dropout predictors for youth may be partly due to the complex developmental periods spanning early adolescence to young adulthood. Youth experience a barrage of psychosocial and neurobiological changes during this time while simultaneously experiencing increasing environmental and psychosocial pressures [27,28]. The interactions between environmental factors, genetic influences, and the changing brain create a complicated network of influencers that drives the manifestation of mental health disorders [29,30]. Symptoms of mental health disorders often manifest in early adolescence or young adulthood; 74% of people with mental health disorders at the age of 26 years also have a mental health diagnosis before the age of 18 years [29]. Due to the complexity of this population, research suggests the need for developmentally appropriate mental health treatment and engagement strategies that are distinct from that of adults and responsive to unique needs across the stages of development [29]. There is a pressing need to better understand the mechanisms of action that drive successful mental health treatment and engagement of youth and to be able to identify clients at most risk who could benefit from support.

    This Study

    This quality improvement study used latent profile analysis (LPA) to identify client subgroups based on the initial symptom severity of youth and young adult clients and sought to identify differences in treatment engagement between profiles. This approach provides a data-driven model for identifying subgroups to personalize treatment and engagement strategies [31].

    Drawing from a nationwide remote telehealth IOP, this study provides a practical opportunity to assess subgroups and outcomes in a transdiagnostic IOP among youth and young adults with multiple cooccurring conditions. This addresses a research-practice gap wherein youth mental health providers commonly treat youth with many different diagnoses [32] and at different developmental periods, yet research and best practices for youth mental health treatment have largely focused on homogeneous populations and analyzing single variable predictors [33-35]. The aim of this quality improvement analysis was to (1) identify the clinical acuity profiles of youth and young adults entering web-based intensive outpatient treatment, (2) examine the association between profile membership and demographic characteristics, and (3) test for differences in clinical profiles on measures of treatment engagement.


    Setting

    The data set used in this investigation was collected as part of routine outcome monitoring undertaken at a national provider of mental health services for youth and young adults. Charlie Health delivers remote-only IOP to individuals aged 11 to 30 years with complex mental health disorders. Client outcomes are routinely collected using validated measures to meet stakeholder and accreditation requirements for ongoing program evaluation to identify opportunities for service quality improvement and to drive increases in treatment responsiveness. This study is part of an organization-wide initiative to identify effective ways to personalize the components of mental health treatment for youth with mental health needs that necessitate IOP treatment.

    The client population of Charlie Health typically presents with high clinical acuity, a high prevalence of cooccurring mental and behavioral health issues that impair daily functioning, and significant histories of trauma. Clients often require more intensive services than those that can be delivered in a community-based care setting, or they come from regions lacking mental health resources. Because Charlie Health is in network with both public and private insurers, there is a diverse representation of socioeconomic backgrounds. In addition, clients have widely variable experiences of treatment history before admission to Charlie Health (ie, clients may be stepping down from a higher level of care, stepping up from a lower level of care, or initiating treatment for the first time).

    Weekly IOP at Charlie Health involves 9 hours of web-based group sessions paired with optional 1-hour individual or 1-hour family sessions as determined by the needs and willingness of the clients or caregivers to participate. Group sessions are 3 hours long, broken into 50-minute sessions with 10-minute breaks in between. These sessions include experiential therapy (eg, art, music, and journaling); general therapeutic processing; and evidence-based skill-building interventions (eg, dialectical behavioral therapy and cognitive behavioral therapy). Because of the size and reach of Charlie Health and the accessibility of web-based programming, clients can be assigned to group IOP tracks based on shared identities (eg, gender, age, sexual orientation, and race); presenting issues (eg, depression, anxiety, and substance use); and appropriate treatment (eg, trauma-focused cognitive behavioral therapy). To complement the individual, family, and group sessions of the IOP, Charlie Health offers comprehensive family support programs to provide psychoeducational and support groups to caregivers and loved ones. For clients who require psychopharmacological support, a team of psychiatrists and psychiatric mental health nurse practitioners is on staff to conduct evaluations and prescribe medication as needed. In addition, clients are supported by a team of Care Experience Specialists who offer care coordination outside individual or group therapy sessions. Individual and group sessions are offered a variety of times throughout the day to meet the needs of a range of schedules and to offer a higher degree of accessibility. The average length of stay in the program is 10 to 12 weeks. The target length of stay is individually determined by the clinical staff and varies by client acuity and need.

    Participants

    The sample for this investigation included Charlie Health clients who were discharged between July 2021 and February 2023 (N=2924) and who completed both an intake and discharge survey. The sample was selected to include multiple demographic factors associated with profile membership in the analyses. Some demographics were asked on the intake survey (race and age), and some were asked on the discharge survey (gender, sexual orientation, and transgender status). The sample included clients who were discharged following the successful completion of treatment as well as those discharged due to disengagement, insurance denial, or referral to a higher or lower level of care.

    Ethics Approval

    This program evaluation research was reviewed and approved by the Florida State University Institutional Review Board that deemed this type of investigation “nonhuman subjects research,” given its primary purpose of program evaluation and quality improvement (STUDY00003364).

    Data Collection Procedures

    As part of Charlie Health’s routine data collection procedures, data were collected from clients during the first and last IOP sessions. Clients logged into the web-based group room and were then sent to a web-based survey room, where a Charlie Health staff member provided them with a closed survey Qualtrics link to complete either an intake or discharge survey. For any client who missed their last IOP session, the discharge survey link was emailed and texted to them with a small incentive for completion. All client data were downloaded and deidentified before analyses. All clinical symptom measures in this study were collected during intake survey. The measures were presented in the same order for each client.

    Measures

    Demographics

    Demographic data were collected by self-report on both the intake (race and age) and discharge surveys (gender, sexual orientation, and transgender status) and consisted of questions regarding race (African American or Black, Asian, Indigenous people around the world, Middle Eastern or North African, White, and other); gender (male, female, nonbinary, genderqueer, nonconforming, gender fluid, and gender neutral individuals); and sexual orientation (straight, asexual or gray sexuality, bisexual, pansexual, gay, lesbian, queer, and questioning). Clients could multiselect across each question to capture intersectionalities. Clients were also asked whether they identified themselves as transgender (yes or no).

    Use of High-Acuity Service

    To determine the use of high-acuity service before intake, clients were asked to self-report whether they were admitted to the emergency room (ER) or a higher level of care facility (ie, inpatient and residential treatment) in the 30 days before their first group therapy session at Charlie Health. If a client answered yes to either, they were asked for the reason for their admission (suicidal thoughts, self-harm, suicide attempt, substance use, eating disorder, and other).

    World Health Organization-5 Well-Being Index

    The World Health Organization-5 Well-Being Index (WHO-5) is a self-reported measure of well-being that consists of 5 questions on a scale of 0 (At no time) to 5 (All of the time). The sum score, ranging from 0 to 25, is multiplied by 4 to give a final score out of 100, with 0 being the worst outcome for well-being and 100 being the best outcome for well-being. A cutoff score of ≤50 is used to assign a screening diagnosis of depression [36].

    Generalized Anxiety Disorder-7

    The Generalized Anxiety Disorder-7 (GAD-7) is a self-reported measure of generalized anxiety symptoms that consists of 7 questions on a scale of 0 (not at all) to 3 (nearly every day). The total score, ranging from 0 to 21, is calculated. Scores of 5, 10, and 15 represent cutoff points for mild, moderate, and severe anxiety, respectively [37].

    Patient Health Questionnaire Modified for Adolescents

    The Patient Health Questionnaire modified for adolescents (PHQ-A) is a 9-item, self-reported measure of depressive symptoms that consists of questions on a scale of 0 (not at all) to 3 (nearly every day). The total score, ranging from 0 to 27, is calculated. PHQ-A scores of 5, 10, 15, and 20 represent mild, moderate, moderately severe, and severe depression, respectively [38].

    Self-Harm

    To determine the acuity of self-harm, clients were asked to self-report how many days over the last month before intake that they self-injured and were given a sliding scale from 0 to 30 days.

    Daily Functioning

    To assess how often mental health affected their daily functioning, clients were asked to self-report whether they were enrolled in school or if they were working. If so, they were then asked how many days in the last 7 days their mental health interfered with their ability to attend school or work and were given the option of reporting 0 to 7 days.

    Treatment Engagement Outcomes

    Three treatment engagement outcomes were targeted in the analysis: the total number of groups attended, treatment completion, and disengagement from treatment before completing treatment goals. For the total number of groups attended, each 3-hour block of group therapy was considered one group therapy session. The sum of all group therapy sessions attended was created for each client. Treatment completion was defined as a client completing their treatment goals and being routinely discharged. Disengagement was defined as a client discharging from treatment against clinical recommendations or without completing treatment goals.

    Statistical Methods

    Overview

    We used LPA to uncover homogeneous subgroups of youth and young adults within the larger heterogeneous treatment population who reported similar patterns of responses to a set of observed variables or indicators. In an LPA, means and SDs from continuous indicators are used to estimate the posterior probabilities of belonging to a particular profile [39]. We included a number of variables that were conceptually supported to differentiate clinical acuity in this population. We selected clinical and functional indicator variables that were modeled to arrive at the most parsimonious solution with the best model fit criteria; the model with the 5 indicators is described in subsequent sections.

    Identify Clinical Acuity Profiles of Youth and Young Adults Entering Virtual IOP

    Data preparation was conducted using SPSS Statistics (version 29.0; IBM Corp), and LPA class enumeration and multinomial regressions were conducted in Mplus [40]. The solutions were estimated during the profile enumeration process to determine the optimal number of profiles. Each subsequent profile structure (k) was evaluated and compared with the previous model (k − 1) to determine whether it provided a better relative and absolute fit according to several indicators. These included the log likelihood values; Akaike information criterion [41], Bayesian information criterion (BIC) [42], and a BIC criterion adjusted for sample size, where lower values indicate better model fit; and the Lo-Mendell-Rubin test [43] and Bootstrap Likelihood Ratio test [44], where a failure to reject the null indicates that the current model with k profiles does not substantially improve model fit compared with a prior model with k − 1 profiles. In addition, an entropy score was calculated for each model, where scores closer to 1.0 indicate better profile prediction [45]. Finally, we assessed within-profile homogeneity and between-profile separation using the average posterior class probability, where values >0.70 indicate adequate separation, and the odds of correct classification, where values >5 indicate adequate classification [46].

    Examine the Association Between Profile Membership and Demographic Characteristics

    A modified 3-step multinomial logistic regression analysis (R3STEP multinomial regression) [47,48] was used to assess the demographic and clinical factors associated with latent profile membership. The DU3STEP approach [48] for continuous distal outcomes and the DCAT approach [49] for categorical distal outcomes were used to evaluate the relationship between clinical acuity profiles and various proxies for treatment engagement. The three distal outcome models tested the relationship between membership in each profile at treatment intake and (1) length of stay, (2) treatment completion, and (3) disengagement.

    LPA uses full information maximum likelihood with robust SEs by computing the parameter estimates using all available information, which is considered the best practice for data missing at random, as it avoids listwise deletion [50].

    Treatment Engagement Outcomes Related to Profile Membership

    Chi-squared analysis was used to compare profiles and assess the differences in treatment engagement outcomes. Separate analyses were performed for the 3 variables of the total number of sessions attended, treatment completion, and early disengagement.


    Sample Demographics

    The sample included 2924 clients. Participant demographics are displayed in Table 1.

    Table 1. Demographics of full sample and the 4 classes.
    VariableFull sample (N=2924)Low-acuity profile (n=943)High-acuity+low–self-harm profile (n=1452)High-acuity+moderate–self-harm profile (n=203)High-acuity+high–self-harm profile (n=326)
    Age (years), mean (SD)17.55 (4.62)16.99 (4.26)18.58 (4.99)16.54 (3.73)15.51 (3.01)
    Age group (years), n (%)

    		Youth (<18)1807 (61.8)658 (69.8)738 (50.8)141 (69.5)270 (82.8)

    		Young adult (18-24)759 (26)204 (21.6)471 (32.4)47 (23.2)37 (11.3)

    		Adult (≥25)300 (10.3)68 (7.2)215 (14.8)9 (4.4)8 (2.5)

    		Missing data58 (2)13 (1.4)28 (1.9)6 (3)11 (3.4)
    Gender, n (%)

    		Female1397 (47.8)434 (46)729 (50.2)93 (45.8)141 (43.3)

    		Male250 (8.5)344 (36.5)301 (20.7)39 (19.2)76 (23.3)

    		Nonbinary84 (2.9)54 (5.7)129 (8.9)26 (12.8)41 (12.6)

    		Nonconforming150 (5.1)18 (1.9)45 (3.1)4 (2)17 (5.2)

    		Gender fluid61 (2.1)26 (2.8)73 (5)21 (10.3)30 (9.2)

    		Gender questioning25 (0.9)11 (1.2)34 (2.3)3 (1.5)13 (4)

    		Gender neutral54 (1.8)8 (0.8)12 (0.8)1 (0.5)4 (1.2)

    		Multiple33 (1.1)9 (1)40 (2.8)5 (2.5)0 (0)

    		Other110 (3.8)10 (1.1)18 (1.2)4 (2)1 (0.3)

    		Missing data250 (8.5)29 (3.1)71 (4.9)7 (3.4)3 (0.9)
    Race, n (%)

    		Asian38 (1.3)9 (1)22 (1.5)5 (2)3 (0.9)

    		Black119 (4.1)45 (4.8)62 (4.3)8 (3.9)4 (1.2)

    		Indigenous people around the world26 (0.9)9 (1)9 (0.6)5 (2.5)3 (0.9)

    		Middle Eastern or North African6 (0.2)1 (0.1)3 (0.2)0 (0)2 (0.6)

    		White874 (29.9)277 (29.4)525 (36.2)55 (27.1)17 (5.2)

    		Mixed race94 (3.2)26 (2.8)59 (4.1)8 (3.9)1 (0.3)

    		Other79 (2.7)31 (3.3)44 (3)2 (1)2 (0.6)

    		Missing data1688 (57.5)545 (57.8)728 (50.1)121 (59.6)294 (90.2)
    Ethnicity, n (%)

    		Hispanic or Latino247 (8.4)85 (9)137 (9.4)15 (7.4)10 (3.1)

    		Not Hispanic or Latino761 (26)234 (24.8)462 (31.8)48 (23.6)17 (5.2)

    		Other66 (2.3)30 (3.2)30 (2.1)4 (2)2 (0.6)

    		Missing data1850 (63.3)594 (63)823 (56.7)136 (67)297 (91.1)
    Sexual orientation, n (%)

    		Asexual or gray sexual118 (4)27 (2.9) 56 (3.9)12 (5.9)23 (7.1)

    		Bisexual621 (21.2)173 (18.3)322 (22.2)50 (24.6)76 (23.3)

    		Pansexual38 (13.1)82 (8.7)191 (13.2)33 (16.3)76 (23.3)

    		Gay79 (2.7)22 (2.3)35 (2.4)6 (3)16 (4.9)

    		Heterosexual or straight992 (33.9)430 (45.6)452 (31.1)49 (24.2)61 (18.7)

    		Lesbian138 (4.7)41 (4.3)74 (5.1)9 (4.4)14 (4.3)

    		Queer137 (4.7)23 (2.4)73 (5)13 (6.4)28 (8.6)

    		Questioning135 (4.6)40 (4.2)66 (4.5)8 (3.9)21 (6.4)

    		Other158 (5.4)47 (5)91 (6.2)14 (6.9)6 (1.8)

    		Missing data164 (5.6)58 (6.2)92 (6.3)9 (4.4)5 (1.5)
    Recent mental health treatment history, n (%)

    		Higher level of care use 30 days before admission883 (30.2)319 (33.8)349 (24)78 (38.4)137 (42)

    		ERa use 30 days before admission835 (28.6)226 (24)388 (26.7)77 (37.9)144 (44.2)
    Indicators, mean (SD)

    		Depressive symptoms14.32 (7.22)6.56 (4.20)17.95 (4.67)18.21 (6.26)18.47 (5.93)

    		Anxiety symptoms12.14 (6.13)5.30 (3.34)15.57 (3.79)15.21 (5.21)15.43 (4.67)

    		Days of self-harm5.80 (10.12)0.52 (1.68)1.12 (1.90)13.89 (3.69)29.18 (1.89)

    		Symptom interference with daily functioning2.78 (2.59)1.92 (2.37)3.06 (2.62)3.49 (2.45)5.00 (2.18)

    		Psychological well-being32.99 (22.72)55.69 (21.81)22.57 (13.15)20.43 (14.58)20.80 (14.58)
    Engagement outcomes

    		Total number of IOPb group sessions attended, mean (SD)25.06 (13.22)23.98 (12.91)24.82 (13.36)27.44 (13.93)27.83 (12.50)

    		IOP treatment completion, n (%)1804 (61.7)580 (61.5)876 (60.3)134 (66)214 (65.6)

    		Early disengagement from IOP, n (%)644 (22)228 (24.2)344 (23.7)40 (19.7)32 (9.8)

    aER: emergency room.

    bIOP: intensive outpatient program.

    Aim 1: Identify Clinical Acuity Profiles of Youth and Young Adults Entering Web-Based IOP

    Tables 2 and 3 display the fit statistics and classification diagnostics from the class enumeration process for 1 to 5 latent profile solutions. As shown in Table 2, there were diminishing returns in the Akaike information criterion, BIC, and sample-size adjusted BIC fit statistics. The nonsignificant Lo-Mendell-Rubin value for the 5-profile solution (P=.06) indicates that the 4-profile model should be accepted. The 4-profile model was also the most conceptually parsimonious, as the 5-profile solution displayed an artificial division of 1 profile into 2 new profiles. These results collectively indicate a 4-profile solution as the best fit (Figure 1). The profile interpretation is outlined in the text below.

    Table 2. Results of the latent class enumeration and measures of absolute and relative fit of latent classes among youth and young adults in remote intensive outpatient program (N=2924).
    ProfilesLog likelihoodAICaBICbSample-size adjusted BICEntropyLMRc, P valueBLRTd, P value
    1−28,816.76557,653.53057,713.33757,681.563N/AeN/AN/A
    2−27,202.15054,436.30154,531.99254,481.1540.888<.001<.001
    3−26,290.08552,624.17052,755.74652,685.8440.812<.001<.001
    4−25,483.18151,022.36351,189.82351,100.8560.831<.001<.001
    5−25,197.58650,463.17150,666.51550,558.4850.834.06<.001

    aAIC: Akaike information criterion.

    bBIC: Bayesian information criterion.

    cLMR: Lo-Mendell-Rubin.

    dBLRT: Bootstrap Likelihood Ratio test.

    eN/A: not applicable.

    Table 3. Model classification diagnostics of the 4-class solution (relative entropy, Ek=0.84) among youth and young adults in remote intensive outpatient program (N=2924).
    Class kpkamcaPkbAvePPkcOCCkd
    Class 10.329140.329140.92324.43
    Class 20.126620.126620.9996890.75
    Class 30.466050.466050.8788.25
    Class 40.07820.07820.959275.72

    apk: model estimated proportion for class k.

    bmcaPk: modal class assignment proportion for class k.

    cAvePPk: average posterior probability for class k.

    dOCCk: odds of correct classification.

    ‎
    Figure 1. The 4-profile solution of baseline clinical acuity (N=2924). GAD-7: Generalized Anxiety Disorder-7; PHQ-A: Patient Health Questionnaire modified for adolescents; WHO-5: World Health Organization-5 Well-Being Index.
    Low Acuity

    Profile 1 (943/2924, 32.25%) consisted of individuals who were admitted to treatment reporting mild levels of depression (PHQ-A: mean 6.56, SD 4.20) and anxiety (GAD-7: mean 5.30, SD 3.34) and minimal self-harm (0.52, SD 1.68 days of last 30 days). Youth and young adults in this profile reported lesser, yet still meaningful, interference from their mental health symptoms in their daily functioning (1.92, SD 2.37 out of 7 days in the last week). They also endorsed psychological well-being (WHO-5: mean 55.69, SD 21.81) above the cutoff used to detect a screening diagnosis of depression (ie, ≤50) [38].

    High Acuity+Low Self-Harm

    Profile 2 (1452/2924, 49.66%) included individuals who endorsed significantly higher levels of depression and anxiety and lower levels of psychological well-being (WHO-5: mean 22.57, SD 13.15) at treatment entry relative to the low-acuity profile. These youth reported moderately severe depression (PHQ-A: mean 17.95, SD 4.67) and severe anxiety (GAD-7: mean 15.57, SD 3.79). In contrast to their mental health symptom severity, these youth endorsed minimal self-harm (1.12, SD 1.90 days on average out of the last 30 days) in the last month and reported that their mental health symptoms interfered with their daily functioning for 3.06 (SD 2.62) days on average out of the last 7 days.

    High Acuity+Moderate Self-Harm

    Profile 3 (203/2924, 6.94%) reported similar levels of depression and anxiety to profiles 2 and 4, with youth who entered the treatment reporting moderately severe depression (PHQ-A: mean 18.21, SD 6.26) and severe anxiety (GAD-7: mean 15.21, SD 5.21). In contrast to profiles 1 and 2, these youth endorsed self-harm about half of the days in the last month (13.89, SD 3.69 days on average of the last 30 days) and reported that their mental health symptoms interfered with their daily functioning about half the days in the last week (3.49, SD 20.43 days on average of the last 7 days). Finally, similar to profile 2, they reported low levels of psychological well-being (WHO-5: mean 20.43, SD 14.58).

    High Acuity+High Self-Harm

    Profile 4 (326/2924, 11.15%) showed the most consistently poor clinical and functional outcomes across all the profiles. Similar to the individuals in profiles 2 and 3, they entered treatment reporting moderately severe depression (PHQ-A: mean 18.47, SD 5.93) and severe anxiety (GAD-7: mean 15.43, SD 4.67). These youth reported engaging in self-harm nearly every day in the last month (29.18, SD 1.89 days on average) and reported the highest levels of interference with their daily functioning (5, SD 2.18 days on average out of the last 7 days). Correspondingly, they reported the lowest levels of psychological well-being (WHO-5: mean 20.80, SD 14.58).

    Aim 2: Examine the Association Between Profile Membership and Demographic Characteristics

    Overview

    Next, we used multinomial logistic regression to assess various factors associated with latent profile membership, including both demographic history and mental health treatment characteristics, to better understand the structure of each profile (see Table 4 for model parameters of fully adjusted results, using the low-acuity profile as the reference group).

    Table 4. Odds of latent class membership across demographic and mental health treatment history characteristics at intensive outpatient program admission.

    		Low-acuity profile vs high-acuity+low–self-harm profileLow-acuity profile vs high-acuity+moderate–self-harm profileLow-acuity profile vs high-acuity+high–self-harm profile

    		B (SE)ORa (95% CI)B (SE)OR (95% CI)B (SE)OR (95% CI)
    Higher level of care use in last 30 days−0.68b (0.12)0.51 (0.40-0.64)0.42b (0.18)1.53 (1.08-2.16)0.57b (0.15)1.77 (1.34-2.35)
    ERc use in last 30 days0.14 (0.12)1.15 (0.91-1.45)0.73b (0.17)2.06 (1.48-2.88)0.97b (0.14)2.63 (2.00-3.46)
    Age0.10b (0.01)1.11 (1.08-1.13)−0.02 0.02 (0.02)0.99 (0.95-1.02)−0.10b (0.02)0.91 (0.87-0.94)
    Gender

    		Female0.25 (0.15)1.29 (0.96-1.71)−0.11 (0.21)0.90 (0.59-1.37)−0.29 (0.15)0.75 (0.56-1.01)

    		Male−0.79b (0.17)0.46 (0.33-0.63)−0.78d (0.25)0.46 (0.28-0.75)−0.72b (0.17)0.49 (0.35-0.68)

    		Nonbinary0.73b (0.20)2.08 (1.39-3.10)1.19b (0.26)3.28 (1.99-5.41)1.34b (0.19)3.82 (2.62-5.58)

    		Transgender0.73d (0.22)2.07 (1.35-3.18)1.05b (0.28)2.85 (1.66-4.90)1.30b (0.21)3.67 (2.45-5.51)
    Sexual orientation

    		Bisexual0.15 (0.17)1.16 (0.83-1.62)0.53e (0.24)1.70 (1.07-2.69)0.98b (0.18)2.68 (1.88-3.82)

    		Pansexual0.26 (0.23)1.30 (0.83-2.03)0.66e (0.30)1.93 (1.07-3.48)1.70b (0.21)5.48 (3.61-8.34)

    		Heterosexual−0.76b (0.13)0.47 (0.36-0.60)−0.74d (0.21)0.48 (0.31-0.73)−0.65b (0.17)0.52 (0.38-0.72)
    Race

    		Asian−0.03 (0.12)0.97 (0.77-1.24)0.13 (0.17)1.13 (0.73-2.01)0.32 (0.28)1.37 (0.79-2.39)

    		Black−0.08 (0.37)0.92 (0.45-1.88)0.38 (0.52)1.46 (0.53-4.04)0.94 (0.85)2.57 (0.49-13.59)

    		Indigenous people around the world−0.04 (0.18)0.96 (0.67-1.37)0.19 (0.26)1.21 (0.73-2.01)0.47 (0.43)1.60 (0.70-3.87)

    		Middle Eastern or North African−0.02 (0.10)0.98 (0.82-1.17)0.09 (0.13)1.10 (0.85-1.42)0.24 (0.21)1.27 (0.84-1.92)

    		White−0.01 (0.06)0.99 (0.88-1.11)0.06 (0.09)1.07 (0.90-1.26)0.16 (0.14)1.17 (0.89-1.55)

    aOR: odds ratio.

    bP<.001.

    cER: emergency room.

    dP<.01.

    eP<.05.

    ER Visit 30 Days Before IOP Admission

    Relative to clients in the low-acuity profile, individuals in the high-acuity+moderate–self-harm (odds ratio [OR] 2.06, 95% CI 1.48-2.88; P<.001) and high-acuity+high–self-harm (OR 2.63, 95% CI 2.00-3.46; P<.001) profiles had significantly higher odds of visiting an ER in the 30 days before their admission, whereas there was no significant difference in ER visit between the high-acuity+low–self-harm profile and the low-acuity profile (OR 1.15, 95% CI 0.91-1.45; P=.25).

    Higher Level of Care Stay Within 30 Days of IOP Admission

    Clients in the high-acuity+low–self-harm profile had significantly lower odds of reporting a higher level of care stay in the 30 days before admission than the clients in the low-acuity profile (OR 0.51, 95% CI 0.40-0.64; P<.001), whereas clients in the high-acuity+moderate–self-harm (OR 1.53, 95% CI 1.08-2.16; P<.001) and high-acuity+high–self-harm (OR 1.77, 95% CI 1.34-2.35; P<.001) profiles had significantly higher odds of reporting a higher level of care admission in the month preceding admission to IOP.

    Age

    Age was significantly related to the likelihood of profile memberships, with increased age of the individual associated with a lower likelihood of membership in the high-acuity+high–self-harm group compared with the low-acuity profile group (OR 0.91, 95% CI 0.87-0.94; P<.001). Increased age was significantly related to classification in the high-acuity+low–self-harm profile relative to the low-acuity profile (OR 1.11, 95% CI 1.08-1.13; P<.001). Age was not significantly associated with membership in the high-acuity+moderate–self-harm profile relative to the low-acuity profile (OR 0.99, 95% CI, 0.95-1.02; P=.45).

    Gender

    In terms of gender, identifying as a female individual was not related to profile membership. A significantly smaller proportion of male individuals were classified in all high-acuity profiles relative to the low-acuity profile (high acuity+low self-harm: OR 0.46, 95% CI 0.33-0.63; P<.001; high acuity+moderate self-harm: OR 0.46, 95% CI 0.28-0.75; P=.002; high acuity+high self-harm: OR 0.49, 95% CI 0.35-0.68; P<.001). Alternatively, a significantly larger proportion of clients who were identified as nonbinary were classified as having high-acuity profiles relative to those with low-acuity profiles (high acuity+low self-harm: OR 2.08, 95% CI 1.39-3.10; P<.001; high acuity+moderate self-harm: OR 3.28, 95% CI 1.99-5.41; P<.001; high acuity+high self-harm: OR 3.82, 95% CI 2.62-5.58; P<.001).

    Transgender Status

    Transgender identification was significantly associated with profile membership in all the high-acuity profiles relative to the low-acuity profile (high acuity+low self-harm: OR 2.07, 95% CI 1.35-3.18; P<.001; high acuity+moderate self-harm: OR 2.85, 95% CI 1.66-4.90; P<.001; high acuity+high self-harm: OR 3.67, 95% CI 2.45-5.51; P<.001).

    Sexual Orientation

    Clients who identified as heterosexual were less likely to be classified in the high-acuity profiles than in the low-acuity profile (high acuity+low self-harm: OR 0.47, 95% CI 0.36-0.60; P<.001; high acuity+moderate self-harm: OR 0.48, 95% CI 0.31-0.73; P=.001; high acuity+high self-harm: OR 0.52, 95% CI 0.38-0.72; P<.001). Identification as bisexual or pansexual was not related to profile membership in the high-acuity+low–self-harm profile relative to the low-acuity profile. However, a significantly greater proportion of clients who identified as bisexual or pansexual were classified into the high-acuity+moderate–self-harm (bisexual: OR 1.70, 95% CI 1.07-2.69; P=.02; pansexual: OR 1.93, 95% CI 1.07-3.48; P=.03) and high-acuity+high–self-harm profiles (bisexual: OR 2.68, 95% CI 1.88-3.82; P<.001; pansexual: OR 5.48, 95% CI 3.61-8.34; P<.001).

    Race

    Race was unrelated to profile membership across the 5 racial categories and 4 profiles.

    Aim 3: Treatment Engagement Outcomes Related to Profile Membership

    Total Number of Group Sessions Attended

    Youth and young adults in the low-acuity (23.98 sessions) and high-acuity+low–self-harm (24.82 sessions) profiles attended significantly fewer treatment sessions on average relative to youth in the high-acuity+moderate–self-harm (27.44 sessions) and high-acuity+high–self-harm (27.83 sessions) profiles (ꭓ23=27.6, P<.001)

    Treatment Completion

    Youth and young adults in the high-acuity+low–self-harm profile (876/1452, 60.33%) completed treatment at a significantly lower rate compared with those in the high-acuity+moderate–self-harm profile (134/203, 66%) and those in the high-acuity+high–self-harm profile (214/326, 65.6%; ꭓ23=13.4, P=.004). Individuals in the low-acuity profile (580/943, 61.5%) did not differ significantly in their rates of treatment completion from any of the high-acuity profiles.

    Disengagement From Treatment Before Completing Treatment Goals

    Youth and young adults in the high-acuity+high–self-harm profile (32/326, 9.8%) were significantly less likely to disengage early from treatment than youth within the low-acuity (228/943, 24.2%), high-acuity+low–self-harm (344/1452, 23.69%), and the high-acuity+moderate–self-harm (40/203, 19.7%) profiles (ꭓ23=71.1, P<.001).


    Principal Findings

    This quality improvement investigation sought to use person-centered subgroup analyses to identify hidden clusters of youth and young adult clients admitted to a remote IOP who were differentially engaged in treatment based on shared clinical severity indicators. Four profiles were identified, with separation most evident in reported depression and anxiety symptoms (low acuity vs high acuity) and in the number of days spent engaging in self-harm the month before intake (low, moderate, or high).

    The low-acuity profile was notable for relatively milder depressive, anxiety, and self-harm symptoms. This reflects similarities to the low-acuity class in stratified analysis with adults, with low depression and anxiety, a long duration, and a low likelihood of chronic symptom outcomes [21]. Although this might indicate a group ill-suited to the considerable time demands of IOP, this profile was twice as likely to report having a higher level of care stay in the month before admission to IOP relative to the high-acuity+low–self-harm profile. It is possible that these clients were previously stabilized in a residential treatment setting and used a remote IOP for their step-down level of care to prevent readmission. As a result, these clients may have required fewer sessions to reach their treatment goals. This is consistent with the finding that youth in this profile attended the fewest sessions but had no substantial differences in treatment completion relative to the other profiles.

    Although previous analyses have identified subgroups based on the levels of mental health acuity at intake [19,21], this is the first study to our knowledge to include self-harm. Self-harm is a distinguishing feature among classes; for example, the low-acuity profile and high-acuity+low–self-harm profile were not significantly different for any outcome. Although these 2 profiles reported significantly different levels of depression and anxiety at intake, they both reported very little self-harm (0.52 and 1.12 days on average in the last month). This could suggest that these profiles are more similar than they originally appeared based on their PHQ-A and GAD-7 scores alone, perhaps indicating that the presence of maladaptive coping techniques (such as self-harm) serves as a better clinical indicator of acuity to personalize support in programming.

    One of the key variables that differentiated clinical acuity was the number of days in the month before intake that the client self-harmed. It was unexpected that the profile representing clients who reported daily self-harm tied for the highest rates of treatment completion (66%) and had the lowest levels of early disengagement (9.8%) compared with any of the other profiles analyzed. High-acuity clients are typically the most challenging to engage and retain in treatment programs. The tailored curriculum approach used in Charlie Health considers these factors and proactively assigns clients to a type of treatment that is expected to work best for their needs. This finding suggests that the clients classified into the high-acuity+high–self-harm profile maintain high engagement with the program.

    Second, in line with previous research [51-53], clients identified as LGBTQIA+ were significantly more likely to be classified into one of the high-acuity profiles relative to the low-acuity profile. Specifically, clients who were identified as nonbinary (OR 3.82, 95% CI 2.62-5.58), transgender (OR 3.67, 95% CI 2.45-5.51), bisexual (OR 2.68, 95% CI 1.88-3.82), and pansexual (OR 5.48, 95% CI 3.61-8.34) had the highest odds of being classified as having a high-acuity+high–self-harm profile, indicating significantly higher acuity for these populations at intake relative to their cis-gender, heterosexual peers. Programs that fail to consider the unique needs of LGBTQIA+ clients may overlook important factors that correlate with symptom severity. In contrast to previous research, ethnic minority group status, gender, and age were not more likely to influence the classification into high-acuity profiles 3 or profiles with higher dropout rates [15,26,54].

    Practice Implications

    Treatment providers have been working for decades to unlock the key to personalizing treatment engagement strategies within large care systems. Attention to treatment engagement and those who drop out of treatment is also critical for effective research and evaluation, as clients who drop out often have distinct feature outcomes from those who remain [55]. This analysis was part of an organization-wide initiative to better understand what types of youth and young adult clients enter an IOP level of care and what risk factors for dropout and disengagement may be identified in this setting. Instead of subjectively determining these criteria based on anecdotal evidence alone, using identifying markers that are universally screened has allowed this treatment provider to begin to target specific supports and curriculum adjustments based on a constellation of attributes.

    In practice, many treatment providers find it difficult to personalize treatment and engagement strategies according to the unique needs of each client, even though each distinct program is likely not equally efficacious for all. This process is further complicated when working with youth, as there is increased heterogeneity due to the varying levels of development in each age group. The value of this type of investigation lies not in the replication of these particular profiles, which may or may not translate to other populations or treatment settings, but in the systematic approach to identifying subgroups of clients based on key clinical indicators that can be applied anywhere. This inquiry details one such way to harness the outcome variables that treatment providers are already collecting into actionable targets for personalization that are unique to a specific setting and treatment population.

    Within this care setting and population, the findings suggest immediate implications. The high-acuity+low–self-harm group is less likely to complete treatment, suggesting that youth with this profile are candidates for additional engagement strategies. The high-acuity+high–self-harm group is less likely to disengage early, suggesting that current engagement strategies for this group are effective, and greater attention on engagement strategies can be focused on the other groups. Furthermore, the finding that the low-acuity profile attended fewer sessions but had no differences in treatment completion based on treatment goals suggests that they need fewer sessions to achieve their treatment goals. This finding can guide general expectations regarding the length of stay for clients with different acuity levels.

    Limitations

    Several considerations should be considered when determining the generalizability of the findings of this investigation. Given the heterogeneity of youth and young adult treatment populations, the 4 distinct profiles of clinical acuity at intake may not replicate fully across other treatment providers. These analyses highlight one set of indicators that could be used to understand acuity at intake, but they are by no means exhaustive. Other mental health care providers should endeavor to use their own client data in similar person-centered analyses to uncover solutions that are personalized to their given treatment population. Second, although this evaluation details the identification of discrete profiles, it does not address the application of these profiles to adjust treatment journeys accordingly. Future investigations should build on these findings by helping fine-tune the appropriate levels of support and deployment of further personalized curricula. Even within a well-defined level of care such as IOP, there exists a gradient of ancillary support services. This provides an opportunity to assess the dosage of care and its impact more precisely across different subgroups of patients entering treatment with different clinical and relational needs. Clinical outcome data were not yet available, so this study could not assess the profile association with clinical outcomes; such analysis in the future could guide treatment strategies as well as engagement strategies.

    In addition to these broad areas for future research, this study has several methodological limitations. All data are self-reported and so may be subject to client bias. There are limited demographic variables available, and potentially key variables for engagement, such as life stress, are not captured. In addition, the indicators have different recall periods, leading to measurement inconsistencies.

    Conclusions

    This quality improvement study identified subgroups of adolescents and young adults based on clinical acuity data at intake to identify trends in treatment engagement outcomes. Identifying barriers and protective factors against disengagement is crucial in providers’ ongoing efforts to ensure that an adequate dose of mental health treatment is delivered. The profiles identified in this study showcase further personalization of treatment and engagement strategies across a complex population and represent a novel application of distilling salient tailoring variables around which to build additional resources and support.

    Data Availability

    Owing to the quality improvement nature of this study, participants did not agree to their data being publicly shared; therefore, supporting data are not available.

    Authors' Contributions

    KG and KRB contributed to conceptualization, writing (original draft), formal analysis, and writing (review and editing). JB and CS contributed to writing (original draft) and writing (review and editing). EK was responsible for formal analysis, whereas MK was responsible for methodology. EK, MK, JK, and CF contributed to writing (review and editing).

    Conflicts of Interest

    CF is the cofounder and chief clinical officer of Charlie Health. KG, EK, and JK are employees of Charlie Health. JB and CS are contractors in Charlie Health. MK and KRB report consulting fees from Charlie Health.

    1. Rao U, Hammen CL, Poland RE. Longitudinal course of adolescent depression: neuroendocrine and psychosocial predictors. J Am Acad Child Adolesc Psychiatry. Feb 2010;49(2):141-151. [FREE Full text] [CrossRef]
    2. Data and statistics on children’s mental health. Centers for Disease Control and Prevention. URL: https://www.cdc.gov/childrensmentalhealth/data.html [accessed 2023-04-15]
    3. Jones SE, Ethier KA, Hertz M, DeGue S, Le VD, Thornton J, et al. Mental health, suicidality, and connectedness among high school students during the COVID-19 pandemic - adolescent behaviors and experiences survey, United States, January-June 2021. MMWR Suppl. Apr 01, 2022;71(3):16-21. [FREE Full text] [CrossRef] [Medline]
    4. Cuijpers P, Karyotaki E, Eckshtain D, Ng MY, Corteselli KA, Noma H, et al. Psychotherapy for depression across different age groups: a systematic review and meta-analysis. JAMA Psychiatry. Jul 01, 2020;77(7):694-702. [FREE Full text] [CrossRef] [Medline]
    5. Newby JM, McKinnon A, Kuyken W, Gilbody S, Dalgleish T. Systematic review and meta-analysis of transdiagnostic psychological treatments for anxiety and depressive disorders in adulthood. Clin Psychol Rev. Aug 2015;40:91-110. [FREE Full text] [CrossRef] [Medline]
    6. McGorry PD, Mei C. Early intervention in youth mental health: progress and future directions. Evid Based Ment Health. Nov 2018;21(4):182-184. [CrossRef] [Medline]
    7. Correll CU, Galling B, Pawar A, Krivko A, Bonetto C, Ruggeri M, et al. Comparison of early intervention services vs treatment as usual for early-phase psychosis: a systematic review, meta-analysis, and meta-regression. JAMA Psychiatry. Jun 01, 2018;75(6):555-565. [FREE Full text] [CrossRef] [Medline]
    8. de Haan AM, Boon AE, de Jong JT, Hoeve M, Vermeiren RR. A meta-analytic review on treatment dropout in child and adolescent outpatient mental health care. Clin Psychol Rev. Jul 2013;33(5):698-711. [CrossRef] [Medline]
    9. Steinberg AM, Layne CM, Briggs EC, Liang LJ, Brymer MJ, Belin TR, et al. Benefits of treatment completion over premature termination: findings from the national child traumatic stress network. Psychiatry. 2019;82(2):113-127. [FREE Full text] [CrossRef] [Medline]
    10. Delgadillo J, McMillan D, Lucock M, Leach C, Ali S, Gilbody S. Early changes, attrition, and dose-response in low intensity psychological interventions. Br J Clin Psychol. Mar 2014;53(1):114-130. [CrossRef] [Medline]
    11. Gonzalez Salas Duhne P, Delgadillo J, Lutz W. Predicting early dropout in online versus face-to-face guided self-help: a machine learning approach. Behav Res Ther. Dec 2022;159:104200. [FREE Full text] [CrossRef] [Medline]
    12. Bennemann B, Schwartz B, Giesemann J, Lutz W. Predicting patients who will drop out of out-patient psychotherapy using machine learning algorithms. Br J Psychiatry. Feb 18, 2022;220(4):1-10. [CrossRef] [Medline]
    13. Lawler K, Earley C, Timulak L, Enrique A, Richards D. Dropout from an internet-delivered cognitive behavioral therapy intervention for adults with depression and anxiety: qualitative study. JMIR Form Res. Nov 12, 2021;5(11):e26221. [FREE Full text] [CrossRef] [Medline]
    14. O'Keeffe S, Martin P, Target M, Midgley N. 'I just stopped going': a mixed methods investigation into types of therapy dropout in adolescents with depression. Front Psychol. Feb 5, 2019;10:75. [FREE Full text] [CrossRef] [Medline]
    15. O'Keeffe S, Martin P, Goodyer IM, Wilkinson P, Consortium I, Midgley N. Predicting dropout in adolescents receiving therapy for depression. Psychother Res. Sep 30, 2018;28(5):708-721. [CrossRef] [Medline]
    16. Sheehan J, Sherman KA. Computerised decision aids: a systematic review of their effectiveness in facilitating high-quality decision-making in various health-related contexts. Patient Educ Couns. Jul 2012;88(1):69-86. [CrossRef] [Medline]
    17. Botella C, Moragrega I, Baños R, García-Palacios A. Online predictive tools for intervention in mental illness: the OPTIMI project. Stud Health Technol Inform. 2011;163:86-92. [Medline]
    18. Bowles KH, Hanlon A, Holland D, Potashnik SL, Topaz M. Impact of discharge planning decision support on time to readmission among older adult medical patients. Prof Case Manag. Jan 2014;19(1):29-38. [FREE Full text] [CrossRef] [Medline]
    19. Saunders R, Cape J, Fearon P, Pilling S. Predicting treatment outcome in psychological treatment services by identifying latent profiles of patients. J Affect Disord. Jun 2016;197:107-115. [FREE Full text] [CrossRef] [Medline]
    20. Uckelstam CJ, Philips B, Holmqvist R, Falkenström F. Prediction of treatment outcome in psychotherapy by patient initial symptom distress profiles. J Couns Psychol. Nov 2019;66(6):736-746. [CrossRef] [Medline]
    21. Saunders R, Cohen ZD, Ambler G, DeRubeis RJ, Wiles N, Kessler D, et al. A patient stratification approach to identifying the likelihood of continued chronic depression and relapse following treatment for depression. J Pers Med. Dec 04, 2021;11(12):1295. [FREE Full text] [CrossRef] [Medline]
    22. Pihlaja S, Lahti J, Lipsanen JO, Ritola V, Gummerus EM, Stenberg JH, et al. Scheduled telephone support for internet cognitive behavioral therapy for depression in patients at risk for dropout: pragmatic randomized controlled trial. J Med Internet Res. Jul 23, 2020;22(7):e15732. [FREE Full text] [CrossRef] [Medline]
    23. Saunders R, Buckman JE, Pilling S. Latent variable mixture modelling and individual treatment prediction. Behav Res Ther. Jan 2020;124:103505. [FREE Full text] [CrossRef] [Medline]
    24. Skehan B, Davis M. Aligning mental health treatments with the developmental stage and needs of late adolescents and young adults. Child Adolesc Psychiatr Clin N Am. Apr 2017;26(2):177-190. [CrossRef] [Medline]
    25. Fusar-Poli P. Integrated mental health services for the developmental period (0 to 25 years): a critical review of the evidence. Front Psychiatry. Jun 7, 2019;10:355. [FREE Full text] [CrossRef] [Medline]
    26. Clarke C, Oh J, Khan H, LoParo D, Lamis DA. Referral patterns and demographic factors predict treatment dropout in suicidal youth. Suicide Life Threat Behav. Jun 2021;51(3):616-623. [CrossRef] [Medline]
    27. Tylee A, Haller DM, Graham T, Churchill R, Sanci LA. Youth-friendly primary-care services: how are we doing and what more needs to be done? Lancet. May 05, 2007;369(9572):1565-1573. [CrossRef] [Medline]
    28. Patton GC, Sawyer SM, Santelli JS, Ross DA, Afifi R, Allen NB, et al. Our future: a Lancet commission on adolescent health and wellbeing. Lancet. Jun 11, 2016;387(10036):2423-2478. [FREE Full text] [CrossRef] [Medline]
    29. Jones PB. Adult mental health disorders and their age at onset. Br J Psychiatry Suppl. Jan 2013;54:s5-10. [CrossRef] [Medline]
    30. Paus T, Keshavan M, Giedd JN. Why do many psychiatric disorders emerge during adolescence? Nat Rev Neurosci. Dec 2008;9(12):947-957. [FREE Full text] [CrossRef] [Medline]
    31. Lutz W, Schwartz B, Delgadillo J. Measurement-based and data-informed psychological therapy. Annu Rev Clin Psychol. May 09, 2022;18(1):71-98. [CrossRef] [Medline]
    32. Merikangas KR, Avenevoli S, Costello EJ, Koretz D, Kessler RC. National comorbidity survey replication adolescent supplement (NCS-A): I. Background and measures. J Am Acad Child Adolesc Psychiatry. Apr 2009;48(4):367-379. [FREE Full text] [CrossRef] [Medline]
    33. Selph SS, McDonagh MS. Depression in children and adolescents: evaluation and treatment. Am Fam Physician. Nov 15, 2019;100(10):609-617. [FREE Full text] [Medline]
    34. Sun M, Rith-Najarian LR, Williamson TJ, Chorpita BF. Treatment features associated with youth cognitive behavioral therapy follow-up effects for internalizing disorders: a meta-analysis. J Clin Child Adolesc Psychol. 2019;48(sup1):S269-S283. [FREE Full text] [CrossRef] [Medline]
    35. Döpfner M, Katzmann J, Hanisch C, Fegert JM, Kölch M, Ritschel A, et al. ADOPT Consortium. Affective dysregulation in childhood - optimizing prevention and treatment: protocol of three randomized controlled trials in the ADOPT study. BMC Psychiatry. Sep 02, 2019;19(1):264. [FREE Full text] [CrossRef] [Medline]
    36. Topp CW, Østergaard SD, Søndergaard S, Bech P. The WHO-5 Well-Being Index: a systematic review of the literature. Psychother Psychosom. 2015;84(3):167-176. [FREE Full text] [CrossRef] [Medline]
    37. Spitzer RL, Kroenke K, Williams JB, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. May 22, 2006;166(10):1092-1097. [CrossRef] [Medline]
    38. Johnson JG, Harris ES, Spitzer RL, Williams JB. The patient health questionnaire for adolescents: validation of an instrument for the assessment of mental disorders among adolescent primary care patients. J Adolesc Health. Mar 2002;30(3):196-204. [CrossRef] [Medline]
    39. Muthen B. Latent variable mixture modeling. In: Marcoulides GA, Schumacker RE, editors. New Developments and Techniques in Structural Equation Modeling. London, UK. Psychology Press; 2001;21-54.
    40. Muthén LK, Muthén BO. Mplus user's guide. 8th edition. Muthén & Muthén. 2017. URL: https://www.statmodel.com/download/usersguide/MplusUserGuideVer_8.pdf [accessed 2023-03-15]
    41. Akaike H. Factor analysis and AIC. Psychometrika. Sep 1987;52(3):317-332. [FREE Full text] [CrossRef]
    42. Schwarz G. Estimating the dimension of a model. Ann Stat. Mar 1978;6(2):461-464. [FREE Full text] [CrossRef]
    43. Lo Y, Mendell NR, Rubin DB. Testing the number of components in a normal mixture. Biometrika. Oct 01, 2001;88(3):767-778. [FREE Full text] [CrossRef]
    44. McLachlan G, Peel D. Finite Mixture Models. Hoboken, NJ. John Wiley & Sons; 2000.
    45. Celeux G, Soromenho G. An entropy criterion for assessing the number of clusters in a mixture model. J Classif. Sep 1996;13(2):195-212. [FREE Full text] [CrossRef]
    46. Nagin DS. Group-Based Modeling of Development. London, UK. Harvard University Press; 2005.
    47. Vermunt JK. Latent class modeling with covariates: two improved three-step approaches. Polit Anal. Jan 04, 2017;18(4):450-469. [FREE Full text] [CrossRef]
    48. Asparouhov T, Muthén B. Auxiliary variables in mixture modeling: three-step approaches using Mplus. Struct Equ Modeling. Jun 09, 2014;21(3):329-341. [CrossRef]
    49. Lanza ST, Tan X, Bray BC. Latent class analysis with distal outcomes: a flexible model-based approach. Struct Equ Modeling. Jan 2013;20(1):1-26. [FREE Full text] [CrossRef] [Medline]
    50. Berlin KS, Williams NA, Parra GR. An introduction to latent variable mixture modeling (part 1): overview and cross-sectional latent class and latent profile analyses. J Pediatr Psychol. Mar 2014;39(2):174-187. [CrossRef] [Medline]
    51. Hatchel T, Ingram KM, Mintz S, Hartley C, Valido A, Espelage DL, et al. Predictors of suicidal ideation and attempts among LGBTQ adolescents: the roles of help-seeking beliefs, peer victimization, depressive symptoms, and drug use. J Child Fam Stud. Jan 22, 2019;28(9):2443-2455. [CrossRef]
    52. Jadva V, Guasp A, Bradlow J, Bower-Brown S, Foley S. Predictors of self-harm and suicide in LGBT youth: the role of gender, socio-economic status, bullying and school experience. J Public Health (Oxf). Mar 14, 2023;45(1):102-108. [FREE Full text] [CrossRef] [Medline]
    53. Kahle L. Are sexual minorities more at risk? Bullying victimization among lesbian, gay, bisexual, and questioning youth. J Interpers Violence. Nov 09, 2020;35(21-22):4960-4978. [CrossRef] [Medline]
    54. de Haan AM, Boon AE, de Jong JT, Vermeiren RR. A review of mental health treatment dropout by ethnic minority youth. Transcult Psychiatry. Feb 2018;55(1):3-30. [CrossRef] [Medline]
    55. Karin E, Crane MF, Dear BF, Nielssen O, Heller GZ, Kayrouz R, et al. Predictors, outcomes, and statistical solutions of missing cases in web-based psychotherapy: methodological replication and elaboration study. JMIR Ment Health. Feb 05, 2021;8(2):e22700. [FREE Full text] [CrossRef] [Medline]

    ‎
    BIC: Bayesian information criterion
    ER: emergency room
    GAD-7: Generalized Anxiety Disorder-7
    IOP: intensive outpatient program
    LGBTQIA+: lesbian, gay, bisexual, transgender, queer (or questioning), asexual (or allied), intersex
    LPA: latent profile analysis
    OR: odds ratio
    PHQ-A: Patient Health Questionnaire modified for adolescents
    WHO-5: World Health Organization-5 Well-Being Index

    Edited by T Leung; submitted 05.04.23; peer-reviewed by R Saunders; comments to author 04.05.23; revised version received 20.06.23; accepted 31.07.23; published 07.09.23.

    Copyright

    ©Kate Gliske, Katie R Berry, Jaime Ballard, Clare Schmidt, Elizabeth Kroll, Jonathan Kohlmeier, Michael Killian, Caroline Fenkel. Originally published in JMIR Formative Research (https://formative.jmir.org), 07.09.2023.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.