This paper is in the following e-collection/theme issue:

Formative Evaluation of Digital Health Interventions (2496) Web-based and Mobile Health Interventions (3358) Engagement with and Adherence to Digital Health Interventions, Law of Attrition (505) Digital Mental Health Interventions, e-Mental Health and Cyberpsychology (1568) Mobile Health in Psychiatry (438) Psychotic Disorders (163) Ecological Momentary Assessment (EMA) (322) Patient Engagement and Empowerment (33) Secondary Use of Clinical Data for Research and Surveillance (376)

Published on in Vol 8 (2024)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/57109, first published .
Engagement and Acceptability of Acceptance and Commitment Therapy in Daily Life in Early Psychosis: Secondary Findings From a Multicenter Randomized Controlled Trial

Engagement and Acceptability of Acceptance and Commitment Therapy in Daily Life in Early Psychosis: Secondary Findings From a Multicenter Randomized Controlled Trial

Engagement and Acceptability of Acceptance and Commitment Therapy in Daily Life in Early Psychosis: Secondary Findings From a Multicenter Randomized Controlled Trial

Authors of this article:

Evelyne van Aubel1 Author Orcid Image ;   Thomas Vaessen1, 2 Author Orcid Image ;   Lotte Uyttebroek1 Author Orcid Image ;   Henrietta Steinhart1, 3 Author Orcid Image ;   Annelie Beijer-Klippel1, 3, 4 Author Orcid Image ;   Tim Batink4 Author Orcid Image ;   Ruud van Winkel5 Author Orcid Image ;   Lieuwe de Haan6 Author Orcid Image ;   Mark van der Gaag7, 8 Author Orcid Image ;   Thérèse van Amelsvoort3 Author Orcid Image ;   Machteld Marcelis3, 9 Author Orcid Image ;   Frederike Schirmbeck6 Author Orcid Image ;   Ulrich Reininghaus10, 11 Author Orcid Image ;   Inez Myin-Germeys1 Author Orcid Image
Article Authors Cited by Tweetations Metrics

    Original Paper

    1Center for Contextual Psychiatry, Psychiatry Research Group, Department of Neurosciences, KU Leuven, Leuven, Belgium

    2Department of Psychology, Health & Technology, Faculty of Behavioural, Management and Social Sciences (BMS), University of Twente, Enschede, Netherlands

    3Department of Psychiatry and Neuropsychology, School for Mental Health and Neuroscience, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, Netherlands

    4Department of Lifespan Psychology, Faculty of Psychology, Open University, Heerlen, Netherlands

    5Center for Clinical Psychiatry, Psychiatry Research Group, Department of Neurosciences, KU Leuven, Leuven, Belgium

    6Department of Psychiatry, Amsterdam UMC, University of Amsterdam, Amsterdam, Netherlands

    7Department of Clinical Psychology, VU Amsterdam, Amsterdam, Netherlands

    8Parnassia Psychiatric Institute, The Hague, Netherlands

    9Institute for Mental Health Care Eindhoven (GGzE), Eindhoven, Netherlands

    10Department of Public Mental Health, Central Institute of Mental Health, Medical Faculty Mannheim, University of Heidelberg, Mannheim, Germany

    11ESRC Centre for Society and Mental Health and Centre for Epidemiology and Public Health, Health Service and Population Research Department, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, London, United Kingdom

    Corresponding Author:

    Lotte Uyttebroek, MSc

    Center for Contextual Psychiatry, Psychiatry Research Group

    Department of Neurosciences

    KU Leuven

    Herestraat 49

    ON5B bus 1029

    Leuven, 3000

    Belgium

    Phone: 32 16 37 31 74

    Email: lotte.uyttebroek@kuleuven.be


    Background: Acceptance and commitment therapy (ACT) is promising in the treatment of early psychosis. Augmenting face-to-face ACT with mobile health ecological momentary interventions may increase its treatment effects and empower clients to take treatment into their own hands.

    Objective: This study aimed to investigate and predict treatment engagement with and acceptability of acceptance and commitment therapy in daily life (ACT-DL), a novel ecological momentary intervention for people with an ultrahigh risk state and a first episode of psychosis.

    Methods: In the multicenter randomized controlled trial, 148 individuals with ultrahigh risk or first-episode psychosis aged 15-65 years were randomized to treatment as usual only (control) or to ACT-DL combined with treatment as usual (experimental), consisting of 8 face-to-face sessions augmented with an ACT-based smartphone app, delivering ACT skills and techniques in daily life. For individuals in the intervention arm, we collected data on treatment engagement with and acceptability of ACT-DL during and after the intervention. Predictors of treatment engagement and acceptability included baseline demographic, clinical, and functional outcomes.

    Results: Participants who received ACT-DL in addition to treatment as usual (n=71) completed a mean of 6 (SD 3) sessions, with 59% (n=42) of participants completing all sessions. App engagement data (n=58) shows that, on a weekly basis, participants used the app 13 times and were compliant with 6 of 24 (25%) notifications. Distribution plots of debriefing scores (n=46) show that 85%-96% of participants reported usefulness on all acceptability items to at least some extent (scores ≥2; 1=no usefulness) and that 91% (n=42) of participants reported perceived burden by number and length of notifications (scores ≥2; 1=no burden). Multiple linear regression models were fitted to predict treatment engagement and acceptability. Ethnic minority backgrounds predicted lower notification response compliance (B=–4.37; P=.01), yet higher app usefulness (B=1.25; P=.049). Negative (B=–0.26; P=.01) and affective (B=0.14; P=.04) symptom severity predicted lower and higher ACT training usefulness, respectively. Being female (B=–1.03; P=.005) predicted lower usefulness of the ACT metaphor images on the app.

    Conclusions: Our results corroborate good treatment engagement with and acceptability of ACT-DL in early psychosis. We provide recommendations for future intervention optimization.

    Trial Registration: OMON NL46439.068.13; https://onderzoekmetmensen.nl/en/trial/24803

    JMIR Form Res 2024;8:e57109

    doi:10.2196/57109

    Keywords

    acceptance and commitment therapy; ACT; first episode of psychosis; FEP; ultrahigh risk for psychosis; UHR; ecological momentary intervention; EMI; mobile health; mHealth; blended care; mobile phone 


    Individuals with an ultrahigh risk (UHR) [1,2] state for psychosis are at increased risk of transitioning to a first episode of psychosis (FEP), with meta-analyses showing transition rates of 29% two years after presentation to mental health services, and up to 36% after 3 years [3,4]. These rates suggest that the UHR status temporally and phenomenologically precedes an FEP [5], which are now both conceptualized as the “early stages of psychosis” [1,2,5-7]. At a phenomenological level, psychotic experiences are an important source of distress in UHR individuals [8-10]. Moreover, UHR individuals who do not transition to psychosis nor remit show a reduction in functioning similar to FEP individuals [11-13]. As for the latter, while sustained periods of symptom remission are common after a FEP [14], persisting psychotic symptoms are associated with significant levels of distress [15,16], and for a majority of FEP individuals, impaired long-term functioning, reduced quality of life, social exclusion in work and relationships, and increased mortality have been reported [14,17]. These results highlight the importance of early interventions to prevent transition to more severe stages, as well as to alleviate psychotic distress and improve functioning.

    Currently available psychological and pharmacological interventions can effectively reduce the transition from UHR to FEP, with cognitive behavioral therapy (CBT) showing the most robust effects [18]. However, these meta-analyses did not find significant improvements in psychotic distress, functioning, quality of life, or other affective symptoms [18,19]. One therapy that may successfully target these outcomes is acceptance and commitment therapy (ACT) [20]. ACT is a third-wave behavioral therapy that, unlike traditional CBT, focuses on changing the relationship between an individual and their thoughts and experiences, rather than reappraising them [20]. From the perspective of ACT, individuals tend to avoid, suppress, or control unwanted psychological experiences, a process known as experiential avoidance [20,21]. Previous studies showed that experiential avoidance was a mediator between the experience of daily life hassles and delusional distress [22], and that it was associated with distress related to auditory hallucinations [23], as well as with higher depression, anxiety, and stress [24] in individuals with established psychotic disorder. ACT, therefore, aims to teach individuals alternative acceptance skills to replace experiential avoidance tendencies, reconnect to what they value in life, and set goals to translate these values into committed action, a process known as psychological flexibility [20]. While acceptance may target psychotic distress, components of commitment can potentially improve reward-related motivational deficits, making ACT a promising intervention for early psychosis.

    Studies have indicated that ACT is a feasible and accepted approach in both inpatient and outpatient samples with established psychosis [25-28], with promising results of ACT in comparison to treatment as usual on various clinical and functional outcomes [25-27,29-32]. However, meta-analyses report inconsistent effects of ACT for psychosis on symptoms or clinical outcomes [33,34]. The mixed evidence of the current ACT for psychosis literature may stem from methodological challenges, for example, the heterogeneity due to different intervention protocols and outcomes measured [33,34]. However, the mixed evidence could also reflect a gap between the therapy room and the real world, where patients struggle to apply the skills and techniques learned in therapy in their day-to-day lives due to motivational or functional deficits commonly experienced in early psychosis [11].

    One way to facilitate the therapy to real-world transfer is to provide ACT within a blended care approach, combining face-to-face therapy sessions with an ecological momentary intervention (EMI) [35,36]. EMIs deliver real-time psychological interventions in daily life using digital technology [37], and as such, they enable patients to access interventions that are tailored to what they need in a given moment and context. More importantly, by providing real-world and real-time psychological interventions, EMIs aim to produce changes in underlying mechanisms that may ultimately lead to sustainable changes under real-world conditions [35]. In psychosis, the use of EMIs to deliver treatment has been shown feasible and acceptable with high compliance and satisfaction rates [38-45], indicating a potential avenue to expand ACT therapy for psychosis beyond traditional therapy settings.

    Given the potential of blended care interventions for psychosis, we have developed the ACT in daily life (ACT-DL) intervention, that deploys the use of the ACT-DL EMI in addition to face-to-face sessions with a trained ACT-therapist [46]. The ACT-DL EMI takes the form of a smartphone app that allows patients to practice ACT skills in between therapy sessions, at times when they most need it. Furthermore, the app prompts individuals multiple times a day to fill in short experience sampling method (ESM) [47-49] questionnaires on current affect, context, and behavior with the aim of increasing emotional awareness, followed by an ACT exercise or visual cue of an ACT metaphor. A pilot study has tested ACT-DL and found very good completion rates, use of exercises, and positive user experiences, in a heterogeneous clinical sample of patients with mental disorders [50]. Furthermore, group-based ACT-DL in emerging adults with subclinical symptoms of depression and psychosis was feasible and led to a significant reduction in clinically rated depression relative to an active control condition [51]. In a subsequent multicentered randomized controlled trial known as the INTERACT trial [52], individual ACT-DL was tested in 148 individuals with early psychosis, yielding promising results on efficacy outcomes [53]. However, treatment efficacy must be interpreted alongside treatment engagement and acceptability.

    Effective treatment uptake and engagement are important prerequisites for therapy to be successful and may depend on whether the therapy is acceptable to and inclusive of all individuals taking part in the therapy [54-56]. Engaging individuals in treatment for early psychosis has proven challenging, with a meta-analysis reporting disengagement rates between 1% and 41% [57]. Moreover, meta-analyses have identified several predictors of disengagement, such as substance use, poor medication adherence, symptom severity, and minority status, although evidence on employment status, age, and gender is mixed [57]. Understanding treatment engagement in both face-to-face therapy and EMIs is crucial for interpreting treatment outcomes, exploring the role of blended care in treatment engagement, as well as to identifying key predictors to better target individuals who are more likely to disengage, ultimately improving treatment efficacy and delivery.

    In this study, we aimed to investigate among participants in the intervention arm: (1) treatment engagement with both the face-to-face sessions and the EMI part of ACT-DL; (2) acceptability of the ACT-DL intervention as a whole and the metaphors and exercises in the app; and (3) whether demographic, clinical, or functional characteristics predict treatment engagement and acceptability. We hypothesize that (1) participants show good engagement with ACT-DL, (2) participants will perceive ACT-DL as acceptable, and (3) different characteristics, such as demographical and clinical characteristics, predict treatment engagement and acceptability. We believe that our results will induce useful intervention optimization recommendations, which may ultimately lead to improved treatment engagement, acceptability, and efficacy, as well as a higher potential for clinical implementation.


    Study Design and Participants

    In the multicenter INTERACT randomized controlled trial (OMON NL46439.068.13), individuals with UHR or FEP were randomly allocated (1:1) to ACT-DL in addition to treatment-as-usual (TAU) as the experimental condition or a control condition of TAU only, which included routine mental health care. The aim of the INTERACT trial was to test the efficacy of ACT-DL on reducing psychotic distress, psychotic experiences, psychopathology, and social functioning, as well as to evaluate treatment acceptability, adherence, and fidelity [52]. Based on power calculations, we aimed to recruit 150 participants in secondary mental health services in 5 regions in the Netherlands and Flanders (Belgium): Amsterdam, The Hague, Maastricht/Eindhoven, Flemish-Brabant, and East/West-Flanders. Between June 2015 and December 2018, individuals receiving care from these secondary mental health services were informed about the study by their treating clinician, and if interested, were approached by a member of the research team who provided further information. A full eligibility assessment was conducted by the researcher after informed consent was obtained (see Ethical Considerations section). Inclusion criteria were (1) aged 15-65 years, (2) UHR (without prior use of antipsychotic medication for psychotic symptoms) or FEP (onset within last 3 years) as assessed by the Comprehensive Assessment of At Risk Mental States [1] and the Nottingham Onset Schedule [58], (3) good command of the Dutch language, and (4) ability to provide written informed consent. Exclusion criteria were (1) primary diagnosis of alcohol or substance abuse as established with the Mini International Neuropsychiatric Interview [59], and (2) severe endocrine, cardiovascular, or brain disease. This secondary analysis only focuses on individuals randomly allocated to the intervention arm (ACT-DL + TAU) of the INTERACT trial. The study protocol has been published elsewhere [52].

    ACT-DL Intervention

    The manualized ACT-DL intervention consisted of 8 ACT training sessions administered face-to-face by a clinician (psychologist) trained in ACT, each for around 45-60 minutes, and an ACT-based EMI, over an 8-week intervention period (Multimedia Appendix 1). The intervention included a session for psychoeducation, followed by 6 ACT sessions aimed to enhance participants’ psychological flexibility by training them in a new ACT component each week (ie, creative hopelessness, acceptance, defusion, self and mindfulness, values, and committed action), which were integrated and reviewed in the last session. The ACT-based EMI was administered through a smartphone-based app (ie, the PsyMate app [60]) to allow participants to apply the skills that they trained in therapy into their daily lives for at least 3 consecutive days per week following (from session 2) each face-to-face session. On each of these days, participants received notifications on the app at 8 semirandom moments, asking them to complete a brief ESM questionnaire on their current mood, psychotic experiences, and activities, with the goal of increasing emotional awareness. Participants were then offered, with a 50:50 ratio, either an ACT exercise or metaphor training them in the ACT component covered in the face-to-face session. After participants were trained in each ACT component separately, the EMI was extended to cover the full range of components in order to train participants to adopt ACT skills and techniques flexibly depending on the context. Participants could initiate an on-demand ACT exercise whenever they were struggling with difficult thoughts or emotions. They were also asked to initiate and complete a morning and evening questionnaire every day. In addition to the app, they could also do ACT exercises in a paper workbook. Participants had no longer access to the ACT-DL EMI after the completion of the intervention period. The ACT-DL intervention procedure is described elsewhere [46].

    Ethical Considerations

    This study was conducted in accordance with ethical principles for research involving human participants. The INTERACT study was approved by the Medical Ethics Review Committees at Maastricht University Medical Center in the Netherlands (NL46439.068.13) and the University Clinic Leuven in Belgium (B322201629214). Study participants were informed about the study procedures in person or by phone (including secondary analyses of collected data) and were given time to consider participation. Written informed consent was obtained from each participant prior to assessment and randomization and could be withdrawn by participants at any time. Participants were then deidentified and were allocated a pseudonymized study ID. Safety was monitored throughout the study period, as detailed in the study protocol [52]. Participants were reimbursed according to their attendance at outcome assessment appointments, with increasing amounts for later time points (up to 145 euros in gift vouchers, approximately US $167 based on the exchange rate at the time of the study, in 2020), as well as for additional travel expenses. There was no reimbursement provided for the therapy sessions or the app use during the intervention period.

    Measures

    Treatment Engagement

    We based treatment engagement numbers of the face-to-face sessions on information from treatment integrity questionnaires that were sent to the research department at the end of each treatment combined with email contacts between an independent researcher and the trained clinician. Treatment engagement with the app was based on app use data for each participant, including a completed number of ESM questionnaires, (on-demand) exercises, metaphors, and morning and evening questionnaires. Participants could fill in a maximum of 24 ESM questionnaires per intervention week, followed each by an ACT exercise or visual cue of an ACT metaphor. The number of self-initiated on-demand exercises was unlimited. Morning and evening questionnaires were available each morning and evening between therapy sessions, resulting in a theoretical maximum of 7 questionnaires each to self-initiate between sessions. Therapy sessions were in theory weekly. In reality, in some cases, more than 7 days could pass between two subsequent sessions. In those instances, participants used the app more than a week in between therapy sessions, resulting in a higher number of morning and evening questionnaires.

    Acceptability

    Intervention acceptability was assessed with a bespoke 9-item debriefing questionnaire that participants filled in postintervention. Higher scores indicated higher acceptability, except for 2 items on the burden of the app in terms of both the number and length of items within ESM questionnaires, for which higher scores reflected a higher level of burden. For the exploratory analysis, we created three subscales: acceptability of the (1) ACT training and (2) PsyMate ACT-DL app, and (3) notification burden. Second, momentary acceptability of the app metaphors was assessed with the item “How useful is this metaphor for you right now?” and daily acceptability of the app exercises with the item “How useful were the exercises for you today?” All items were measured on Likert scales with scores ranging 1-7 (1=not at all; 4=average; and 7=very much).

    Prediction of Treatment Engagement and Acceptability

    Demographic characteristics included age, gender (0 male; 1 female), ethnicity (0 nonminority background; 1 minority background), and educational achievement (0 no higher education; 1 higher education). Participants who themselves or whose parents (at least one) were not born in Belgium or the Netherlands were defined as having a minority background. Higher education was defined as having obtained a bachelor’s or master’s degree. Psychotropic medication use (0 no use; 1 use) was assessed with a study-specific questionnaire on current medication use and included among others use of antipsychotics (FEP only), antidepressants, and anxiolytics. Premorbid baseline IQ was assessed with the Dutch Adult Reading Test [61,62], of which age- and gender-corrected IQ scores were used (range: 55-145). Baseline symptom severity was assessed with the Brief Psychiatric Rating Scale (previous 2 weeks) [63] affect (range 5:35), activation (range: 7-49), negative (range: 6-42), and positive symptom (range: 6-42) subscale scores. Baseline functioning was measured with the Dutch version of the Social and Occupational Functioning Scale (previous 2 weeks) [64]. Interrater reliability analysis for the total INTERACT sample demonstrated sufficient agreement with a score of 0.67 for the Social and Occupational Functioning Scale and scores ranging between 0.81 and 0.95 for the Brief Psychiatric Rating Scale subscales [53].

    Statistical Analysis

    As for the treatment engagement with the face-to-face sessions, the frequency distribution and the mean number of sessions attended were calculated. As for treatment engagement with the app, the weekly sample mean number of various interactions with the app was calculated. An overall number of ACT-DL app interactions included completed morning, evening, and ESM questionnaires, as well as completed on-demand exercises. Weekly missingness due to therapy dropout (ie, the patient did not show up for the session), therapist-related causes (ie, the therapist forgot to log into the PsyMate or to send the data after the participant used it), or technical issues (eg, the participant did not receive any beeps) was excluded. Missingness due to participant-related causes (ie, the participant was not motivated or too ill to use the app) was recoded as zero. This analysis approach was chosen given that only the latter type of missingness could be assumed to be related to nonengagement with the app.

    The sample mean of acceptability scores of the debriefing questionnaire items and of the usefulness scores of the app metaphors (ie, reported after a notification) and exercises (ie, reported in the evening) was calculated. Likert plots were designed to inspect the frequency distribution of acceptability scores of these same elements.

    Multiple linear regression models were fitted to investigate whether premorbid IQ, as well as various sociodemographic (ie, age, gender, ethnicity, minority background, and educational achievement), clinical (ie, symptom severity, UHR or FEP status, and psychotropic medication use) and functional characteristics, predicted the number of sessions attended, the weekly mean number of completed ESM questionnaires, completed on-demand exercises, the scores on the debriefing questionnaire subscales, and finally the person-mean app metaphor and exercise usefulness scores. All statistical analyses were conducted in Stata (version 14; StataCorp) [65].


    Sample Characteristics

    Of the 148 participants in the total INTERACT trial, 71 (48%) participants were randomized in the experimental condition (ACT-DL+TAU). Participants had a mean age of 26 (SD 6) years, with slightly more women (n=42, 59%) than men in the sample (Table 1).

    Table 1. Baseline sample characteristics (n=71).
    MeasureParticipants
    Demographics

    		Age (years)


    		Mean (SD)26 (6)


    		Range16-47

    		Sex (female), n (%)42 (59)

    		Education (high), n (%)27 (38)

    		Minority background, n (%)26 (37)
    Clinical characteristics

    		Early psychosis status (FEPa), n (%)36 (51)

    		Psychotropic medication, n (%)44 (62)

    		DARTbIQ


    		Mean (SD)95.10 (12.34)


    		Range61-127

    		BPRSc(positive symptoms)


    		Mean (SD)9.34 (3.32)


    		Range6-18

    		BPRS (negative symptoms)


    		Mean (SD)8.45 (2.68)


    		Range6-17

    		BPRS (affective symptoms)


    		Mean (SD)12.55 (4.44)


    		Range5-23

    		BPRS (activation symptoms)


    		Mean (SD)9.13 (2.19)


    		Range7-17

    		SOFASd


    		Mean (SD)43.92 (10.35)


    		Range21-80

    aFEP: first episode of psychosis.

    bDART: Dutch Adult Reading Test.

    cBPRS: Brief Psychiatric Rating Scale.

    dSOFAS: Social and Occupational Functioning Scale.

    Treatment Engagement

    As to treatment engagement (Figure S1 in Multimedia Appendix 2) with the sessions, 42 of 71 (59%) participants completed all 8 face-to-face ACT-DL sessions, including one psychoeducation session (Tables 2 and 3). In contrast, of the 71 participants, 9 (13%) participants did not attend any face-to-face therapy session, with another 20 (28%) participants dropping out after 1 to 7 sessions. On average, participants completed 6 (SD 3) of 8 sessions. App engagement data were available for 58 participants, with all data missing for 13 participants: 12 because of dropout before the start of the first ACT session (ie, session 2) and one because of technical issues. Data for all 7 ACT-DL app study weeks were available for 18 participants, with data missing for some weeks for 40 participants due to various reasons. On a weekly basis, participants had on average 13 (SD 8.7) ACT-DL app interactions, including 6 of 24 (SD 5.0) ESM questionnaires (8 notifications per day the first 3 days after therapy), indicating response compliance to the notifications of 25% (Figure S2 in Multimedia Appendix 2).

    Table 2. Treatment engagement based on the completion of the face-to-face sessions.
    SessionsValue (n=71), n (%)
    09 (13)
    13 (4)
    23 (4)
    34 (6)
    43 (4)
    51 (1)
    61 (1)
    75 (7)
    842 (59)
    Table 3. Treatment engagement based on completion of the appa.
    InteractionsMean (SD)Range
    Total12.7 (8.7)0-38
    Programmed

    		Morning questionnaires2.1 (2.0)0-8

    		Evening questionnaires2.0 (1.9)0-8

    		ESMb questionnaires5.6 (5.0)0-16

    		Exercises2.9 (2.6)0-8

    		Metaphors2.4 (2.3)0-7
    On-demandc

    		Exercises “yes”3.0 (3.3)0-14

    		Exercises “no, later”1.3 (1.3)0-7

    aThere were n=3 participants who did not have any data due to a lack of motivation to use the app.

    bESM: experience sampling method.

    cParticipants initiating an on-demand exercise were asked “Do you want to do an ACT exercise right now?” upon which they then either clicked “Yes” or “No, later.”

    Acceptability

    From the 46 participants who completed the debriefing questionnaire (Figure 1), almost all indicated that the training in general (n=44, 96%; mean 4.96, SD 1.53), as well as the face-to-face ACT sessions (n=43, 93%; mean 5.11, SD 1.55) and the homework exercises (n=41, 89%; mean 4.54, SD 1.70) were to some extent useful (score≥2). Furthermore, almost all participants felt that the ACT-based EMI in general had been useful (n=41, 89%; mean 4.02, SD 1.77) to some extent (score≥2) and helped to apply the ACT exercises in their daily lives (n=42, 91%; mean 4.87, SD 1.82) and to increase emotional awareness (n= 39, 85%; mean 4.02, SD 2.01). Almost all participants indicated that both the number of notifications a day (n=42, 91%; mean 4.78, SD 1.74) and the number of items within a notification (n=42, 91%; mean 4.67, SD 1.70) were to some extent (score≥2) burdensome. Among the participants, 9% (n=4) said not to be burdened at all (score 1).

    ‎
    Figure 1. Acceptability and burden of the ACT-DL intervention. ACT: acceptance and commitment therapy; ACT-DL: acceptance and commitment therapy in daily life.

    As for the ACT exercises, almost all participants acknowledged having used the ACT-based EMI (n=41, 89%) or the workbook (n=39, 85%) at least once, and 48% (n=22) reported having done at least one exercise autonomously (ie, without the app or workbook). From the participants, 98% (n=45) of participants indicated that they at least to some extent (score≥2) intended to use these exercises in the future (mean 4.70, SD 1.70). While the majority of individuals used either all 3 methods (n=18, 39%; Figure S3 in Multimedia Appendix 2) or both the app and the workbook (n=18, 39%) to practice the exercises, a smaller percentage of individuals used either the app alone (n=5, 11%) or the workbook in combination with autonomous training (n=3, 7%). Only 1 (2%) participant reported to have performed the exercises autonomously, and another participant did not indicate any method.

    When turning to findings on the acceptability of the ACT-based EMI (n=58), 98% (n=57) of the participants indicated that the metaphors (mean 3.96, SD 1.34) were at least to some extent useful to them (score≥2). As for the average usefulness of the exercises, 96% (n=56) of the participants rated the exercises (mean 4.05, SD 1.13) as useful to them to at least some extent (score≥2).

    Predictors of Treatment Engagement and Acceptability

    In the multiple linear regression models (Table 4 and Multimedia Appendix 3) with treatment engagement as outcome variables, ethnic minority background significantly predicted lower response compliance to the ESM questionnaire notifications (B=–4.37; P=.01).

    Table 4. Predicting treatment engagement with the ACT-DLa sessions, notifications, and on-demand exercises based on demographic and clinical characteristics.

    		Sessionsb (n=69)Notificationsc (n=56)On-demandd (n=56)

    		Be (SE)βfP valueB (SE)βP valueB (SE)βP value
    FEPg–0.74 (1.40)–0.12.60–2.69 (2.27)–0.27.241.13 (1.57)0.17.48
    Age–0.06 (0.07)–0.11.440.22 (0.14)0.22.12–0.03 (0.10)–0.05.75
    Female0.95 (0.84)0.15.26–0.37 (1.48)–0.03.801.08 (1.02)0.15.30
    Minority–0.14 (0.93)–0.02.88–4.37 (1.66)–0.40.01–0.11 (1.15)–0.02.92
    Education1.47 (1.05)0.23.17–2.17 (1.77)–0.21.23–0.94 (1.22)–0.14.45
    Psychotropic medication0.80 (1.22)0.13.511.45 (1.95)0.13.46–0.61 (1.35)–0.09.65
    DARTh–0.01 (0.04)–0.05.75–0.05 (0.07)–0.12.460.07 (0.05)0.26.15
    BPRSi positive0.08 (0.14)0.09.540.32 (0.23)0.22.16–0.07 (0.16)–0.07.65
    BPRS negative0.01 (0.16)0.00.980.10 (0.29)0.05.73–0.25 (0.20)–0.20.22
    BPRS affective–0.01 (0.11)–0.01.95–0.16 (0.19)–0.15.41–0.06 (0.13)–0.09.63
    BPRS activation–0.25 (0.20)–0.18.200.20 (0.32)0.09.54–0.10 (0.22)–0.07.64
    SOFASj0.03 (0.05)0.10.510.08 (0.07)0.18.27–0.06 (0.05)–0.20.25

    aACT-DL: acceptance and commitment therapy in daily life.

    bNumber of sessions attended.

    cNumber of ESM notifications filled in.

    dNumber of on-demand exercises to which individuals said yes.

    eB: unstandardized coefficients.

    fβ: standardized coefficients.

    gFEP: first episode of psychosis.

    hDART: Dutch Adult Reading Test [61,62].

    iBPRS: Brief Psychiatric Rating Scale [63].

    jSOFAS: Social and Occupational Functioning Assessment Scale [64].

    In contrast, in the models with treatment acceptability outcome variables (Table 5), ethnic minority backgrounds predicted higher perceived app usefulness (B=1.25; P=.049). Moreover, being female predicted a lower perceived in-the-moment usefulness of the visual cues of the metaphors (B=–1.14; P=.005). While affective symptom severity predicted higher perceived usefulness of the ACT training (B=0.14; P=.04), negative symptom severity predicted lower perceived usefulness of this subscale (B=–0.26; P=.01). No other predictors reached significance.

    Table 5. Predicting acceptability of the ACTa training and the PsyMate app based on demographic and clinical characteristics.

    		ACT trainingb (n=44)PsyMate generalc (n=44)PsyMate burdend (n=44)App metaphorse (n=50)App excercisesf (n=50)

    		Bg (SE)βhP valueB (SE)βP valueB (SE)βP valueB (SE)βP valueB (SE)βP value
    FEPi1.50 (0.81)0.50.080.09 (0.85)0.03.921.03 (0.76)0.36.180.20 (0.61)0.07.750.56 (0.52)0.24.29
    Age0.03 (0.04)0.10.530.03 (0.05)0.10.54–0.04 (0.04)–0.16.320.02 (0.03)0.07.620.04 (0.03)0.16.29
    Female0.08 (0.49)0.03.870.37 (0.51)0.11.480.79 (0.46)0.25.10–1.14 (0.38)–0.40.005–0.34 (0.34)–0.14.32
    Minority0.53 (0.59)0.16.381.25 (0.61)0.36.0490.53 (0.55)0.17.340.60 (0.45)0.20.19–0.07 (0.39)–0.03.86
    Education–0.67 (0.63)–0.22.30–0.20 (0.65)–0.06.760.27 (0.58)0.09.65–0.63 (0.48)–0.23.20–0.46 (0.40)–0.20.27
    Psychotropic medication–1.03 (0.67)–0.32.14–0.67 (0.70)–0.20.34–0.70 (0.62)–0.23.27–0.49 (0.51)–0.17.35–0.41 (0.45)–0.17.37
    DARTj–0.02 (0.02)–0.18.34–0.02 (0.02)–0.17.350.02 (0.02)0.13.480.02 (0.02)0.16.32–0.01 (0.02)–0.11.50
    BPRSk positive0.00 (0.08)–0.01.950.01 (0.08)0.02.93–0.04 (0.07)–0.11.54–0.03 (0.06)–0.08.59–0.02 (0.05)–0.05.77
    BPRS negative–0.26 (0.10)–0.49.01–0.19 (0.10)–0.32.07–0.05 (0.09)–0.10.56–0.01 (0.08)–0.02.890.04 (0.07)0.09.60
    BPRS affective0.14 (0.07)0.44.040.02 (0.07)0.05.800.12 (0.06)0.39.06–0.09 (0.05)–0.31.08–0.09 (0.04)–0.35.06
    BPRS activation–0.10 (0.11)–0.15.36–0.06 (0.12)–0.09.60–0.13 (0.10)–0.19.24–0.06 (0.08)–0.11.440.02 (0.07)0.04.80
    SOFASl0.02 (0.03)0.16.420.00 (0.03)–0.01.940.04 (0.03)0.26.160.03 (0.02)0.22.170.00 (0.02)0.01.94

    aACT: acceptance and commitment therapy.

    bItems: Was the training useful to you? Were the face-to-face sessions useful to you? Were the homework exercises useful to you?

    cItems: Was the app useful to you? Did the app help you to implement the ACT exercises into your daily life? Did the app help you to become more aware of how you feel?

    dItems: the app was burdensome in terms of the length of a beep. The app was burdensome in terms of number of beeps a day.

    e(on the app) How useful is this metaphor for you right now?

    f(on the app) How useful were the exercises today?

    gB: unstandardized coefficients.

    hβ: standardized coefficients.

    iFEP: first episode of psychosis.

    jDART: Dutch Adult Reading Test [61,62].

    kBPRS: Brief Psychiatric Rating Scale [63].

    lSOFAS: Social and Occupational Functioning Assessment Scale [64].


    Principal Findings

    This is the first study to investigate treatment engagement with and acceptability of ACT-DL in an early psychosis sample. We found good treatment engagement when taking into account the attendance to face-to-face sessions and weekly interactions with the app in absolute numbers, yet the proportion of individuals who attended all face-to-face sessions, as well as the proportion of completed ESM questionnaires was low in comparison to reports from previous studies. Our acceptability data showed a positive view of all elements of the intervention including the face-to-face sessions and the ACT-based EMI, suggesting that ACT-DL helped participants apply ACT exercises and increase emotional awareness in daily life, despite a perceived burden by number and length of notifications. Individual differences in demographic and baseline clinical characteristics predicted treatment engagement and acceptability. That is, while ethnic minority status predicted lower notification response compliance, it predicted higher app acceptability. Furthermore, being female predicted lower perceived app metaphor usefulness. Negative symptoms predicted lower ACT training acceptability, while we found the opposite for affective symptoms.

    Treatment Engagement and Acceptability

    The proportion of individuals completing all face-to-face sessions is similar to that found in a previous CBT [66] for psychosis study where treatment disengagement was also defined as no show before or during therapy. However, it is lower than that found in other studies investigating the efficacy of (blended) ACT for psychosis [25-27,31], in a meta-analysis of psychosocial interventions [67] for psychosis, and in a blended care intervention in a psychosis sample [45], where completion rates ranged between 76% and 100%. It is of note that only one of those studies [27] had a comparable treatment format with 8 individual ACT sessions offered. In contrast, the other studies mentioned did include individuals at the later stages of psychosis, had a smaller sample size, and offered only up to 4 sessions [25,26,31,45], limiting the comparability of these findings. Nevertheless, one potential reason for the discrepancy in completion rates may be that we included individuals in the early stages of psychosis, and previous studies have shown high psychotherapy dropout in FEP [66,68-70] individuals and high service disengagement in UHR individuals [71-73].

    Second, the ACT-DL intervention manual and EMI were standardized and generic and did not allow for much flexibility and personalization of the treatment, which may have hampered adequate treatment engagement for some individuals. In this respect, a recent meta-analysis showed that in comparison to generic CBT for psychosis, personalized and targeted CBT for psychosis was more effective in alleviating distress related to auditory hallucinations [74]. In that same vein, a recent pilot trial [45] used ESM questionnaire data as input for subsequent functional analysis of voice-hearing and voice-related coping in the face-to-face sessions and used the EMI to provide personalized coping reminders in the daily lives of participants, with excellent treatment engagement and acceptability. As such, a fruitful alteration to the ACT-DL protocol used in this study could be to shift toward a more personalized and formulation-based intervention approach where a functional diagnostic analysis of the patient’s difficulties and not manually steer the content and the order of the sessions and the app. At the same time, the potential of the EMI may further improve by advanced personalization of the ACT-DL app itself, which we will hint at below.

    Treatment engagement with the ACT-DL app may be approached from two different angles. That is, when looking at the level of engagement with the ACT-DL app as the number of ACT exercises or metaphors performed or viewed, our results suggest that the app helped participants to engage in their therapeutic trajectory on average 9 times a week (6 exercises or metaphors followed after a prompt in addition to 3 on-demand exercises). This number is comparable to other ACT-based EMI [50,51,75] or EMI for psychosis studies [41], where participants performed on average 4 to 12 prompted or self-initiated exercises a week. However, do note that direct comparisons with other EMIs for psychosis are difficult to make due to different definitions of response compliance [42,44]. In contrast, the mean response compliance to the ESM questionnaires was clearly lower than that found in other blended interventions where monitoring of affect, symptoms, and their context was (part of) the intervention and where the number of prompts was comparable [45,51,76]. The lower response compliance could reflect the perceived burden of the notification schedule, as was also indicated in the debriefing questionnaire. Notification burden may be attributable to various reasons: the notification schedule nor the content of the items was personalized, and there was no feedback provided after ESM monitoring. In this respect, the optimal number of notifications may not be the same for every patient and a person-tailored beep schedule may prove an effective addition to our intervention. An additional alteration would be to improve the therapy to real-world transfer by providing personalized feedback to patients based on the questionnaire data on affect, context, activities, and ACT skills, which may then function as a starting point for the next ACT face-to-face session. These alterations could potentially increase response compliance to the ESM questionnaires, and more broadly, treatment engagement with the ACT-DL app in general.

    Despite the perceived notification burden, our results indicate the acceptability of the ACT-DL intervention with positive views on all of its elements, including the face-to-face sessions, which is in line with previous findings on studies investigating the acceptability of ACT [25-27], EMIs [41,42,76-78], blended care [45] for psychosis, and ACT in a blended care format [46,50,79]. Our results showed that the app helped participants to apply the ACT exercises and to increase emotional awareness in their daily lives, suggesting that ACT-DL does indeed improve the therapy to real-world transfer. Second, almost all individuals indicated to have used the app in combination with the workbook to do the exercises, encouraging the continued use of a blended care approach in early psychosis, instead of offering the intervention as a stand-alone EMI.

    Predicting Treatment Engagement and Acceptability

    Acceptability scores showed substantial variability, which was partly predicted by negative and affective symptom severity, ethnic minority (higher app usefulness), and gender (being female predicted lower usefulness of the visual metaphor). Negative symptoms predicted lower training usefulness, whereas affective symptom severity predicted higher training usefulness.

    In this study, ethnic minority status was a significant predictor of lower compliance to the ACT-based EMI and negative symptom severity predicted lower acceptability of the ACT training. These results are in line with a recent review that linked negative symptom severity with decreased treatment engagement with digital interventions [80], and with other studies showing that ethnic minority status and negative symptom severity are predictors of higher dropout and may hamper treatment effects in regular psychotherapies, as well as in EMIs for psychosis [44,68,81-87]. These results align with a study that assessed engagement with a CBT-based intervention for psychosis, the Actissist app, that found that White ethnicity was associated with higher levels of engagement [88]. Interestingly though, we found that ethnic minority background was related to higher perceived usefulness of the ACT-DL app. One explanation here could be that there was fewer acceptability data available than there was for treatment engagement with the app, potentially inflating acceptability results. Another explanation could be that individuals with a minority background, who may experience more stigma around mental health disorders [89], view the app as a low-barrier form of care, indicating that engagement with and acceptability of the treatment may not necessarily be related and should be considered as two independent factors in intervention evaluation and optimization.

    In any case, understanding how we can culturally adapt our intervention in dialogue with ethnic minority individuals is especially important in early psychosis, given that having an ethnic minority background is a known risk factor for the disorder [90]. In this respect, one study looked into specific challenges to delivering ACT to consumers of mental health services from underserved and underrepresented backgrounds and provided recommendations on how to address these challenges [91]. At the same time, it has been shown that cultural adaptation of interventions is feasible and that it can enhance patient engagement and outcomes [92]. As such, tailoring the intervention to individuals of various backgrounds will be an important addition to increasing the inclusiveness of our intervention.

    Furthermore, the intervention may have been more demanding for participants with more negative symptoms. The intervention format was not specifically adapted for psychosis nor negative symptoms in psychosis specifically. It is thus possible that individuals with more pronounced negative symptoms need more sessions, and that their therapy should focus primarily on the commitment skills within ACT. Although speculative, these adaptations may improve acceptability for individuals with more negative symptoms in particular. A recent meta-analysis revealed varying effects of negative symptoms, indicating that both lower (no perceived need) and higher levels of symptoms are associated with disengagement [57]. These findings may suggest a window of opportunity for effectively targeting these individuals. Therefore, personalization and tailoring of intervention components, such as questionnaire items, exercises, and metaphors to patients’ personal needs, preferences, and symptoms, could enhance the impact of these interventions. Further understanding of characteristics that affect treatment engagement, as well as acceptability, is crucial to optimize and personalize current interventions, which in turn, may improve treatment efficacy.

    Limitations

    Some limitations need to be considered. First, we did not assess motivation, nor satisfaction (eg, with the System Usability Scale [93]) with technology (ie, the smartphone app). As to the first, it is possible that some individuals are more open than others to adopt new technologies, which may be an important predictor of treatment engagement in itself [56,94]. As to the latter, this information could have informed us better about the user experience of our app. Second, it is also possible that, due to a lack of debriefing questionnaire data for 25 participants who dropped out before postintervention, acceptability ratings were inflated. Third, we had missing user data due to technical or practical difficulties during the intervention. Future studies investigating blended care interventions should closely monitor the functionality, stability, and fidelity of the intervention while providing extensive training on its use to therapists [95]. Finally, we did not include participants from the control condition, so we cannot conclude whether engagement with the face-to-face sessions was greater in the blended care format.

    Conclusions

    The ACT-DL intervention showed promise despite low compliance to the ESM beeps, with participants attending an encouraging number of face-to-face sessions and weekly interactions, suggesting effective real-world application of ACT techniques and improved emotional awareness. These findings support the value of a blended approach for early psychosis and highlight the importance of personalizing interventions based on symptom severity and demographic factors. Future efforts should focus on collaboration with individuals with lived experiences to refine and optimize the intervention for better clinical implementation.

    Acknowledgments

    This work was funded by an ERC Consolidator grant (ERC-2012-StG, project 309767- INTERACT) and an FWO Odysseus Grant (G0F8416N) awarded to IM-G, as well as an NWO VENI Grant (451-13-022) and a DFG Heisenberg Professorship (389624707) awarded to UR. The authors thank all research coordinators (Silke Apers, Nele Volbragt, Wendy Beuken), research assistants (Dieuwke Siegmann, Davinia Verhoeven, Anna de Zwart, Iris de Wit, Lore Depraetere, Tessa Biesemans, and Lotte Hendriks), and data managers (Martien Wampers and Jolien Bynens) past and present who were involved in the INTERACT trial. We also like to thank all individuals who participated in the study and were essential for its successful completion.

    Data Availability

    The dataset generated during this study are available from the corresponding authors upon a reasonable request through the Data curation for OPen Science (DROPS) system using the REDCap (Research Electronic Data Capture) electronic data capture tool [96,97], hosted by the Center for Contextual Psychiatry, KU Leuven. Interested researchers can submit an abstract, which will be reviewed by the research team to avoid overlap with ongoing research projects. After approval, a variable access request is made, and researchers are required to preregister their analysis plan. A data manager will then provide the dataset including the variables for the proposed analysis, accompanied by a time-and date-stamped receipt of data access.

    Authors' Contributions

    EA was responsible for conceptualization, methodology, software, formal analysis, investigation, data curation, and writing, editing, visualizing, and review of the original draft. TV contributed to the conceptualization, methodology, writing review, editing, and provided supervision to EA. LU participated in writing review and editing, while HS, AB-K, and TB assisted with investigation, writing, review, and editing. TB also provided resources. LH, MG, TA, and MM contributed resources, reviewed and edited the manuscript, and secured funding for the project. RW was involved in writing, review, editing, supervision, and funding acquisition. UR and IM-G both contributed to conceptualization, writing, review, editing, and supervision, along with project administration and securing funding for the study.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    ACT-DL (acceptance and commitment therapy in daily life) intervention components.

    DOCX File , 19 KB
    Multimedia Appendix 2

    Engagement with ACT-DL.

    DOCX File , 78 KB
    Multimedia Appendix 3

    Zero-order correlations and p-values between treatment engagement and acceptability outcomes and predictors of interest.

    DOCX File , 31 KB
    1. Yung AR, Yuen HP, McGorry PD, Phillips LJ, Kelly D, Dell'Olio M, et al. Mapping the onset of psychosis: the comprehensive assessment of at-risk mental states. Aust N Z J Psychiatry. 2005;39(11-12):964-971. [CrossRef] [Medline]
    2. Fusar-Poli P, Borgwardt S, Bechdolf A, Addington J, Riecher-Rössler A, Schultze-Lutter F, et al. The psychosis high-risk state: a comprehensive state-of-the-art review. JAMA Psychiatry. 2013;70(1):107-120. [CrossRef] [Medline]
    3. Fusar-Poli P, Bonoldi I, Yung AR, Borgwardt S, Kempton MJ, Valmaggia L, et al. Predicting psychosis: meta-analysis of transition outcomes in individuals at high clinical risk. Arch Gen Psychiatry. 2012;69(3):220-229. [CrossRef] [Medline]
    4. Kempton MJ, Bonoldi I, Valmaggia L, McGuire P, Fusar-Poli P. Speed of psychosis progression in people at ultra-high clinical risk: a complementary meta-analysis. JAMA Psychiatry. 2015;72(6):622-623. [CrossRef] [Medline]
    5. Linscott RJ, van Os J. An updated and conservative systematic review and meta-analysis of epidemiological evidence on psychotic experiences in children and adults: on the pathway from proneness to persistence to dimensional expression across mental disorders. Psychol Med. 2012;43(6):1133-1149. [CrossRef]
    6. Van Os J, Linscott RJ. Introduction: the extended psychosis phenotype—relationship with schizophrenia and with ultrahigh risk status for psychosis. Schizophr Bull. 2012;38(2):227-230. [CrossRef] [Medline]
    7. Van Os J, Reininghaus U. Psychosis as a transdiagnostic and extended phenotype in the general population. World Psychiatry. 2016;15(2):118-124. [CrossRef] [Medline]
    8. Rekhi G, Rapisarda A, Lee J. Impact of distress related to attenuated psychotic symptoms in individuals at ultra high risk of psychosis: findings from the longitudinal youth at risk study. Early Intervention Psychiatry. 2019;13(1):73-78. [CrossRef] [Medline]
    9. Rapado-Castro M, McGorry PD, Yung A, Calvo A, Nelson B. Sources of clinical distress in young people at ultra high risk of psychosis. Schizophr Res. 2015;165(1):15-21. [CrossRef] [Medline]
    10. van der Steen Y, Gimpel-Drees J, Lataster T, Viechtbauer W, Simons CJP, Lardinois M, et al. Clinical high risk for psychosis: the association between momentary stress, affective and psychotic symptoms. Acta Psychiatr Scand. 2017;136(1):63-73. [CrossRef] [Medline]
    11. Schlosser DA, Jacobson S, Chen Q, Sugar CA, Niendam TA, Li G, et al. Recovery from an at-risk state: clinical and functional outcomes of putatively prodromal youth who do not develop psychosis. Schizophr Bull. 2012;38(6):1225-1233. [CrossRef] [Medline]
    12. Beck K, Studerus E, Andreou C, Egloff L, Leanza L, Simon AE, et al. Clinical and functional ultra-long-term outcome of patients with a clinical high risk (CHR) for psychosis. Eur Psychiatry. 2019;62:30-37. [CrossRef] [Medline]
    13. de Pablo GS, Soardo L, Cabras A, Pereira J, Kaur S, Besana F, et al. Clinical outcomes in individuals at clinical high risk of psychosis who do not transition to psychosis: a meta-analysis. Epidemiol Psychiatr Sci. 2022;31:e9. [CrossRef] [Medline]
    14. Morgan C, Lappin J, Heslin M, Donoghue K, Lomas B, Reininghaus U, et al. Reappraising the long-term course and outcome of psychotic disorders: the AESOP-10 study. Psychol Med. 2014;44(13):2713-2726. [CrossRef] [Medline]
    15. Birchwood M. Pathways to emotional dysfunction in first-episode psychosis. Br J Psychiatry. 2003;182:373-375. [Medline]
    16. Steel C, Garety PA, Freeman D, Craig E, Kuipers E, Bebbington P, et al. The multidimensional measurement of the positive symptoms of psychosis. Int J Methods Psychiatr Res. 2007;16(2):88-96. [CrossRef] [Medline]
    17. Reininghaus U, Dutta R, Dazzan P, Doody GA, Fearon P, Lappin J, et al. Mortality in schizophrenia and other psychoses: a 10-year follow-up of the ӔSOP first-episode cohort. Schizophr Bull. 2015;41(3):664-673. [CrossRef] [Medline]
    18. Mei C, van der Gaag M, Nelson B, Smit F, Yuen HP, Berger M, et al. Preventive interventions for individuals at ultra high risk for psychosis: an updated and extended meta-analysis. Clin Psychol Rev. 2021;86:102005. [CrossRef] [Medline]
    19. Zheng Y, Xu T, Zhu Y, Li C, Wang J, Livingstone S, et al. Cognitive behavioral therapy for prodromal stage of psychosis-outcomes for transition, functioning, distress, and quality of life: a systematic review and meta-analysis. Schizophr Bull. 2022;48(1):8-19. [CrossRef] [Medline]
    20. Hayes SC, Luoma JB, Bond FW, Masuda A, Lillis J. Acceptance and commitment therapy: model, processes and outcomes. Behav Res Ther. 2006;44(1):1-25. [CrossRef] [Medline]
    21. Bond FW, Hayes SC, Baer RA, Carpenter KM, Guenole N, Orcutt HK, et al. Preliminary psychometric properties of the acceptance and action questionnaire-II: a revised measure of psychological inflexibility and experiential avoidance. Behav Ther. 2011;42(4):676-688. [CrossRef] [Medline]
    22. Goldstone E, Farhall J, Ong B. Life hassles, experiential avoidance and distressing delusional experiences. Behav Res Ther. 2011;49(4):260-266. [CrossRef] [Medline]
    23. Varese F, Morrison AP, Beck R, Heffernan S, Law H, Bentall RP. Experiential avoidance and appraisals of voices as predictors of voice-related distress. Br J Clin Psychol. 2016;55(3):320-331. [CrossRef] [Medline]
    24. Perry Y, Henry JD, Grisham JR. The habitual use of emotion regulation strategies in schizophrenia. Br J Clin Psychol. 2011;50(2):217-222. [CrossRef] [Medline]
    25. White R, Gumley A, McTaggart J, Rattrie L, McConville D, Cleare S, et al. A feasibility study of acceptance and commitment therapy for emotional dysfunction following psychosis. Behav Res Ther. 2011;49(12):901-907. [CrossRef] [Medline]
    26. Johns LC, Oliver JE, Khondoker M, Byrne M, Jolley S, Wykes T, et al. The feasibility and acceptability of a brief acceptance and commitment therapy (ACT) group intervention for people with psychosis: the 'ACT for life' study. J Behav Ther Exp Psychiatry. 2016;50:257-263. [CrossRef] [Medline]
    27. Shawyer F, Farhall J, Thomas N, Hayes SC, Gallop R, Copolov D, et al. Acceptance and commitment therapy for psychosis: randomised controlled trial. Br J Psychiatry. 2017;210(2):140-148. [CrossRef] [Medline]
    28. Bouws J, Henrard A, de Koning M, Schirmbeck F, van Ghesel Grothe S, van Aubel E, et al. Acceptance and commitment therapy for individuals at risk for psychosis or with a first psychotic episode: a qualitative study on patients' perspectives. Early Intervention Psychiatry. 2024;18(2):122-131. [CrossRef] [Medline]
    29. Bach P, Hayes SC. The use of acceptance and commitment therapy to prevent the rehospitalization of psychotic patients: a randomized controlled trial. J Consult Clin Psychol. 2002;70(5):1129-1139. [CrossRef] [Medline]
    30. Bach P, Hayes SC, Gallop R. Long-term effects of brief acceptance and commitment therapy for psychosis. Behav Modif. 2012;36(2):165-181. [CrossRef] [Medline]
    31. Gaudiano BA, Herbert JD. Acute treatment of inpatients with psychotic symptoms using acceptance and commitment therapy: pilot results. Behav Res Ther. 2006;44(3):415-437. [CrossRef] [Medline]
    32. Tyrberg MJ, Carlbring P, Lundgren T. Brief acceptance and commitment therapy for psychotic inpatients: a randomized controlled feasibility trial in Sweden. Nordic Psychol. 2016;69(2):110-125. [CrossRef]
    33. Jansen JE, Gleeson J, Bendall S, Rice S, Alvarez-Jimenez M. Acceptance- and mindfulness-based interventions for persons with psychosis: a systematic review and meta-analysis. Schizophr Res. 2020;215:25-37. [CrossRef] [Medline]
    34. Brown E, Shrestha M, Gray R. The safety and efficacy of acceptance and commitment therapy against psychotic symptomatology: a systematic review and meta-analysis. Braz J Psychiatry. 2021;43(3):324-336. [CrossRef] [Medline]
    35. Reininghaus U, Depp CA, Myin-Germeys I. Ecological interventionist causal models in psychosis: targeting psychological mechanisms in daily life. Schizophr Bull. 2016;42(2):264-269. [CrossRef] [Medline]
    36. Steinhart H, Myin-Germeys I, Reininghaus U. The development of ecological momentary interventions. In: Palmier-Claus JE, Haddock G, Reininghaus U, editors. Experience Sampling in Mental Health Research. United Kingdom. Routledge; 2019:81-106.
    37. Heron KE, Smyth JM. Ecological momentary interventions: incorporating mobile technology into psychosocial and health behaviour treatments. Br J Health Psychol. 2010;15(1):1-39. [CrossRef] [Medline]
    38. Myin-Germeys I, Klippel A, Steinhart H, Reininghaus U. Ecological momentary interventions in psychiatry. Curr Opin Psychiatry. 2016;29(4):258-263. [CrossRef] [Medline]
    39. Bonet L, Izquierdo C, Escartí MJ, Sancho JV, Arce D, Blanquer I, et al. Use of mobile technologies in patients with psychosis: a systematic review. Revista de Psiquiatría y Salud Mental (English Edition). 2017;10(3):168-178. [CrossRef]
    40. Bell IH, Lim MH, Rossell SL, Thomas N. Ecological momentary assessment and intervention in the treatment of psychotic disorders: a systematic review. Psychiatr Serv. 2017;68(11):1172-1181. [CrossRef] [Medline]
    41. Schlosser D, Campellone T, Kim D, Truong B, Vergani S, Ward C, et al. Feasibility of PRIME: a cognitive neuroscience-informed mobile app intervention to enhance motivated behavior and improve quality of life in recent onset schizophrenia. JMIR Res Protoc. 2016;5(2):e77. [CrossRef] [Medline]
    42. Bucci S, Barrowclough C, Ainsworth J, Machin M, Morris R, Berry K, et al. Actissist: proof-of-concept trial of a theory-driven digital intervention for psychosis. Schizophr Bull. 2018;44(5):1070-1080. [CrossRef] [Medline]
    43. Ben-Zeev D, Buck B, Meller S, Hudenko WJ, Hallgren KA. Augmenting evidence-based care with a texting mobile interventionist: a pilot randomized controlled trial. Psychiatr Serv. 2020;71(12):1218-1224. [CrossRef] [Medline]
    44. Granholm E, Ben-Zeev D, Link PC, Bradshaw KR, Holden JL. Mobile assessment and treatment for schizophrenia (MATS): a pilot trial of an interactive text-messaging intervention for medication adherence, socialization, and auditory hallucinations. Schizophr Bull. 2012;38(3):414-425. [CrossRef] [Medline]
    45. Bell IH, Rossell SL, Farhall J, Hayward M, Lim MH, Fielding-Smith SF, et al. Pilot randomised controlled trial of a brief coping-focused intervention for hearing voices blended with smartphone-based ecological momentary assessment and intervention (SAVVy): feasibility, acceptability and preliminary clinical outcomes. Schizophr Res. 2020;216:479-487. [CrossRef] [Medline]
    46. Vaessen T, Steinhart H, Batink T, Klippel A, Van Nierop M, Reininghaus U, et al. ACT in daily life in early psychosis: an ecological momentary intervention approach. Psychosis. 2019;11(2):93-104. [CrossRef]
    47. Myin-Germeys I, Kasanova Z, Vaessen T, Vachon H, Kirtley O, Viechtbauer W, et al. Experience sampling methodology in mental health research: new insights and technical developments. World Psychiatry. 2018;17(2):123-132. [CrossRef] [Medline]
    48. Csikszentmihalyi M, Larson R. Validity and reliability of the experience-sampling method. J Nerv Ment Dis. 1987;175(9):526-536. [CrossRef] [Medline]
    49. Myin-Germeys I, Oorschot M, Collip D, Lataster J, Delespaul P, van Os J. Experience sampling research in psychopathology: opening the black box of daily life. Psychol Med. 2009;39(9):1533-1547. [CrossRef] [Medline]
    50. Batink T, Bakker J, Vaessen T, Kasanova Z, Collip D, van Os J, et al. Acceptance and commitment therapy in daily life training: a feasibility study of an mHealth intervention. JMIR Mhealth Uhealth. 2016;4(3):e103. [CrossRef] [Medline]
    51. Van Aubel E, Bakker JM, Batink T, Michielse S, Goossens L, Lange I, et al. Blended care in the treatment of subthreshold symptoms of depression and psychosis in emerging adults: a randomised controlled trial of acceptance and commitment therapy in daily-life (ACT-DL). Behav Res Ther. 2020;128:103592. [CrossRef] [Medline]
    52. Reininghaus U, Klippel A, Steinhart H, Vaessen T, van Nierop M, Viechtbauer W, et al. Efficacy of acceptance and commitment therapy in daily life (ACT-DL) in early psychosis: study protocol for a multi-centre randomized controlled trial. Trials. 2019;20(1):769. [CrossRef] [Medline]
    53. Myin-Germeys I, van Aubel E, Vaessen T, Steinhart H, Klippel A, Lafit G, et al. Efficacy of acceptance and commitment therapy in daily life in early psychosis: results from the multi-center INTERACT randomized controlled trial. Psychother Psychosom. 2022;91(6):411-423. [CrossRef] [Medline]
    54. Hardy A, Wojdecka A, West J, Matthews E, Golby C, Ward T, et al. How inclusive, user-centered design research can improve psychological therapies for psychosis: development of SlowMo. JMIR Ment Health. 2018;5(4):e11222. [CrossRef] [Medline]
    55. Peiris D, Miranda JJ, Mohr DC. Going beyond killer apps: building a better mHealth evidence base. BMJ Glob Health. 2018;3(1):e000676. [FREE Full text] [CrossRef] [Medline]
    56. Simblett S, Greer B, Matcham F, Curtis H, Polhemus A, Ferrão J, et al. Barriers to and facilitators of engagement with remote measurement technology for managing health: systematic review and content analysis of findings. J Med Internet Res. 2018;20(7):e10480. [CrossRef] [Medline]
    57. Robson E, Greenwood K. Rates and predictors of disengagement and strength of engagement for people with a first episode of psychosis using early intervention services: a systematic review of predictors and meta-analysis of disengagement rates. Schizophr Bull Open. 2022;3(1):sgac012. [CrossRef] [Medline]
    58. Singh SP, Cooper JE, Fisher HL, Tarrant CJ, Lloyd T, Banjo J, et al. Determining the chronology and components of psychosis onset: the Nottingham onset schedule (NOS). Schizophr Res. 2005;80(1):117-130. [CrossRef] [Medline]
    59. Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, et al. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry. 1998;59(20):22-23. [Medline]
    60. PsyMate™. URL: https://www.psymate.eu/ [accessed 2024-10-24]
    61. Schmand B, Bakker D, Saan R, Louman J. The Dutch reading test for adults: a measure of premorbid intelligence level. Tijdschr Gerontol Geriatr. 1991;22(1):15-19. [Medline]
    62. Nelson HE. National Adult Reading Test (NART): For the Assessment of Premorbid Intelligence in Patients with Dementia: Test Manual. Windsor. NFER-Nelson; 1982.
    63. Ventura J, Lukoff D, Nuechterlein KH. Brief psychiatric rating scale (BPRS) expanded version (4.0): scales, anchor points and administration manual. Int J Methods Psychiatr Res. 1993;3:227-244.
    64. Goldman HH, Skodol AE, Lave TR. Revising axis V for DSM-IV: a review of measures of social functioning. Am J Psychiatry. 1992;149(9):1148-1156. [CrossRef] [Medline]
    65. StataCorp. Stata Statistical Software: Release 14. College Station, TX. StataCorp LLC; 2015.
    66. Richardson T, Dasyam B, Courtney H, White L, Tedbury J, Butt J, et al. Predictors of disengagement from cognitive behavioural therapy for psychosis in a national health service setting: a retrospective evaluation. Br J Clin Psychol. 2019;58(4):440-451. [CrossRef] [Medline]
    67. Villeneuve K, Potvin S, Lesage A, Nicole L. Meta-analysis of rates of drop-out from psychosocial treatment among persons with schizophrenia spectrum disorder. Schizophr Res. 2010;121(1-3):266-270. [CrossRef] [Medline]
    68. Fanning F, Foley S, Lawlor E, McWilliams S, Jackson D, Renwick L, et al. Group cognitive behavioural therapy for first episode psychosis: who's referred, who attends and who completes it? Early Interv Psychiatry. 2012;6(4):432-441. [CrossRef] [Medline]
    69. Doyle R, Turner N, Fanning F, Brennan D, Renwick L, Lawlor E, et al. First-episode psychosis and disengagement from treatment: a systematic review. Psychiatr Serv. 2014;65(5):603-611. [CrossRef] [Medline]
    70. Jolley S, Garety P, Peters E, Fornells-Ambrojo M, Onwumere J, Harris V, et al. Opportunities and challenges in improving access to psychological therapies for people with severe mental illness (IAPT-SMI): evaluating the first operational year of the South London and Maudsley (SLaM) demonstration site for psychosis. Behav Res Ther. 2015;64:24-30. [CrossRef] [Medline]
    71. Hengartner MP, Heekeren K, Dvorsky D, Walitza S, Rössler W, Theodoridou A. Checking the predictive accuracy of basic symptoms against ultra high-risk criteria and testing of a multivariable prediction model: evidence from a prospective three-year observational study of persons at clinical high-risk for psychosis. Eur Psychiatry. 2017;45:27-35. [CrossRef] [Medline]
    72. Leanza L, Studerus E, Mackintosh AJ, Beck K, Seiler L, Andreou C, et al. Predictors of study drop-out and service disengagement in patients at clinical high risk for psychosis. Soc Psychiatry Psychiatr Epidemiol. 2020;55(5):539-548. [CrossRef] [Medline]
    73. Stowkowy J, Liu L, Cadenhead KS, Tsuang MT, Cannon TD, Cornblatt BA, et al. Exploration of clinical high-risk dropouts. Schizophr Res. 2018;195:579-580. [CrossRef] [Medline]
    74. Hazell CM, Greenwood K, Fielding-Smith S, Rammou A, Bogen-Johnston L, Berry C, et al. Understanding the barriers to accessing symptom-specific cognitive behavior therapy (CBT) for distressing voices: reflecting on and extending the lessons learnt from the CBT for psychosis literature. Front Psychol. 2018;9:727. [CrossRef] [Medline]
    75. Ivanova E, Lindner P, Ly KH, Dahlin M, Vernmark K, Andersson G, et al. Guided and unguided acceptance and commitment therapy for social anxiety disorder and/or panic disorder provided via the internet and a smartphone application: a randomized controlled trial. J Anxiety Disord. 2016;44:27-35. [CrossRef] [Medline]
    76. Kramer I, Simons CJP, Hartmann JA, Menne-Lothmann C, Viechtbauer W, Peeters F, et al. A therapeutic application of the experience sampling method in the treatment of depression: a randomized controlled trial. World Psychiatry. 2014;13(1):68-77. [CrossRef] [Medline]
    77. Ben-Zeev D, Brenner CJ, Begale M, Duffecy J, Mohr DC, Mueser KT. Feasibility, acceptability, and preliminary efficacy of a smartphone intervention for schizophrenia. Schizophr Bull. 2014;40(6):1244-1253. [CrossRef] [Medline]
    78. Hanssen E, Balvert S, Oorschot M, Borkelmans K, van Os J, Delespaul P, et al. An ecological momentary intervention incorporating personalised feedback to improve symptoms and social functioning in schizophrenia spectrum disorders. Psychiatry Res. 2020;284:112695. [CrossRef] [Medline]
    79. Levin ME, Haeger J, Pierce B, Cruz RA. Evaluating an adjunctive mobile app to enhance psychological flexibility in acceptance and commitment therapy. Behav Modif. 2017;41(6):846-867. [CrossRef] [Medline]
    80. Arnold C, Farhall J, Villagonzalo KA, Sharma K, Thomas N. Engagement with online psychosocial interventions for psychosis: a review and synthesis of relevant factors. Internet Interventions. 2021;25:100411. [CrossRef] [Medline]
    81. Lincoln TM, Rief W, Westermann S, Ziegler M, Kesting M, Heibach E, et al. Who stays, who benefits? Predicting dropout and change in cognitive behaviour therapy for psychosis. Psychiatry Res. 2014;216(2):198-205. [CrossRef] [Medline]
    82. Garety P, Waller H, Emsley R, Jolley S, Kuipers E, Bebbington P, et al. Cognitive mechanisms of change in delusions: an experimental investigation targeting reasoning to effect change in paranoia. Schizophr Bull. 2015;41(2):400-410. [CrossRef] [Medline]
    83. Zimmermann D, Rubel J, Page AC, Lutz W. Therapist effects on and predictors of non-consensual dropout in psychotherapy. Clin Psychol Psychother. 2017;24(2):312-321. [CrossRef] [Medline]
    84. Swift JK, Greenberg RP. Premature discontinuation in adult psychotherapy: a meta-analysis. J Consult Clin Psychol. 2012;80(4):547-559. [CrossRef] [Medline]
    85. Wierzbicki M, Pekarik G. A meta-analysis of psychotherapy dropout. Prof Psychol Res Pract. 1993;24(2):190-195. [CrossRef]
    86. Owen J, Imel Z, Adelson J, Rodolfa E. 'No-show': therapist racial/ethnic disparities in client unilateral termination. J Couns Psychol. 2012;59(2):314-320. [CrossRef] [Medline]
    87. Ruglass LM, Pedersen A, Cheref S, Hu MC, Hien DA. Racial differences in adherence and response to combined treatment for full and subthreshold post-traumatic stress disorder and alcohol use disorders: a secondary analysis. J Ethn Subst Abuse. 2016;15(4):434-448. [CrossRef] [Medline]
    88. Hassan L, Eisner E, Berry K, Emsley R, Ainsworth J, Lewis S, et al. User engagement in a randomised controlled trial for a digital health intervention for early psychosis (Actissist 2.0 trial). Psychiatry Res. 2023;329:115536. [FREE Full text] [CrossRef] [Medline]
    89. Wood L, Byrne R, Enache G, Lewis S, Díaz MF, Morrison AP. Understanding the stigma of psychosis in ethnic minority groups: a qualitative exploration. Stigma Health. 2022;7(1):54-61. [CrossRef]
    90. Morgan C, Charalambides M, Hutchinson G, Murray RM. Migration, ethnicity, and psychosis: toward a sociodevelopmental model. Schizophr Bull. 2010;36(4):655-664. [CrossRef] [Medline]
    91. Sobczak LTR, West LM. Clinical considerations in using mindfulness- and acceptance-based approaches with diverse populations: addressing challenges in service delivery in diverse community settings. Cognit Behav Pract. 2013;20(1):13-22. [CrossRef]
    92. Rathod S, Phiri P, Harris S, Underwood C, Thagadur M, Padmanabi U, et al. Cognitive behaviour therapy for psychosis can be adapted for minority ethnic groups: a randomised controlled trial. Schizophr Res. 2013;143(2-3):319-236. [CrossRef] [Medline]
    93. Tullis T, Albert B. Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics. London. Elsevier Inc; 2013.
    94. Michie S, Yardley L, West R, Patrick K, Greaves F. Developing and evaluating digital interventions to promote behavior change in health and health care: recommendations resulting from an international workshop. J Med Internet Res. 2017;19(6):e232. [FREE Full text] [CrossRef] [Medline]
    95. WHO. Monitoring and Evaluating Digital Health Interventions: A Practical Guide to Conducting Research and Assessment. USA. World Health Organization; 2016.
    96. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377-381. [FREE Full text] [CrossRef] [Medline]
    97. Harris PA, Taylor R, Minor BL, Elliott V, Fernandez M, O'Neal L, et al. The REDCap consortium: building an international community of software platform partners. J Biomed Inform. 2019;95:103208. [FREE Full text] [CrossRef] [Medline]

    ‎
    ACT: acceptance and commitment therapy
    ACT-DL: acceptance and commitment therapy in daily life
    CBT: cognitive behavioral therapy
    EMI: ecological momentary intervention
    ESM: experience sampling method
    FEP: first episode of psychosis
    TAU: treatment as usual
    UHR: ultrahigh risk

    Edited by A Mavragani; submitted 10.02.24; peer-reviewed by H Li; comments to author 21.06.24; revised version received 16.08.24; accepted 25.09.24; published 21.11.24.

    Copyright

    ©Evelyne van Aubel, Thomas Vaessen, Lotte Uyttebroek, Henrietta Steinhart, Annelie Beijer-Klippel, Tim Batink, Ruud van Winkel, Lieuwe de Haan, Mark van der Gaag, Thérèse van Amelsvoort, Machteld Marcelis, Frederike Schirmbeck, Ulrich Reininghaus, Inez Myin-Germeys. Originally published in JMIR Formative Research (https://formative.jmir.org), 21.11.2024.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.