This paper is in the following e-collection/theme issue:

Formative Evaluation of Digital Health Interventions (2226) Web-based and Mobile Health Interventions (3128) Usability of Apps and User Perceptions of mHealth (902)

Published on in Vol 8 (2024)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/38803, first published .
Use Intention and User Expectations of Human-Supported and Self-Help eHealth Interventions: Internet-Based Randomized Controlled Trial

Use Intention and User Expectations of Human-Supported and Self-Help eHealth Interventions: Internet-Based Randomized Controlled Trial

Use Intention and User Expectations of Human-Supported and Self-Help eHealth Interventions: Internet-Based Randomized Controlled Trial

Authors of this article:

Talia R Cohen Rodrigues1 Author Orcid Image ;   Thomas Reijnders1 Author Orcid Image ;   Linda D Breeman1 Author Orcid Image ;   Veronica R Janssen1, 2 Author Orcid Image ;   Roderik A Kraaijenhagen3, 4 Author Orcid Image ;   Douwe E Atsma2 Author Orcid Image ;   Andrea WM Evers1, 5, 6 Author Orcid Image
Article Authors Cited by Tweetations (2) Metrics

    Original Paper

    1Health, Medical, and Neuropsychology Unit, Leiden University, Leiden, Netherlands

    2Department of Cardiology, Leiden University Medical Center, Leiden, Netherlands

    3NDDO Institute for Prevention and Early Diagnostics (NIPED), Amsterdam, Netherlands

    4Vital10, Amsterdam, Netherlands

    5Department of Psychiatry, Leiden University Medical Center, Leiden, Netherlands

    6Medical Delta, Leiden University, Technical University of Delft, Erasmus University Rotterdam, Leiden, Delft, Rotterdam, Netherlands

    ‡the BENEFIT consortium

    Corresponding Author:

    Talia R Cohen Rodrigues, MSc

    Health, Medical, and Neuropsychology Unit

    Leiden University

    Wassenaarseweg 52

    Leiden, 2333 AK

    Netherlands

    Phone: 31 71 527 3627

    Email: t.r.cohen.rodrigues@fsw.leidenuniv.nl


    Background: Self-help eHealth interventions provide automated support to change health behaviors without any further human assistance. The main advantage of self-help eHealth interventions is that they have the potential to lower the workload of health care professionals. However, one disadvantage is that they generally have a lower uptake. Possibly, the absence of a relationship with a health care professional (referred to as the working alliance) could lead to negative expectations that hinder the uptake of self-help interventions. The Unified Theory of Acceptance and Use of Technology (UTAUT) identifies which expectations predict use intention. As there has been no previous research exploring how expectations affect the adoption of both self-help and human-supported eHealth interventions, this study is the first to investigate the impact of expectations on the uptake of both kinds of eHealth interventions.

    Objective: This study investigated the intention to use a self-help eHealth intervention compared to a human-supported eHealth intervention and the expectations that moderate this relationship.

    Methods: A total of 146 participants were randomly assigned to 1 of 2 conditions (human-supported or self-help eHealth interventions). Participants evaluated screenshots of a human-supported or self-help app–based stress intervention. We measured intention to use the intervention-expected working alliance and the UTAUT constructs: performance expectancy, effort expectancy, and social influence.

    Results: Use intention did not differ significantly between the 2 conditions (t142=–1.133; P=.26). Performance expectancy (F1,140=69.269; P<.001), effort expectancy (F1,140=3.961; P=.049), social influence (F1,140=90.025; P<.001), and expected working alliance (F1,140=26.435; P<.001) were positively related to use intention regardless of condition. The interaction analysis showed that performance expectancy (F1,140=4.363; P=.04) and effort expectancy (F1,140=4.102; P=.045) more strongly influenced use intention in the self-help condition compared to the human-supported condition.

    Conclusions: As we found no difference in use intention, our results suggest that we could expect an equal uptake of self-help eHealth interventions and human-supported ones. However, attention should be paid to people who have doubts about the intervention’s helpfulness or ease of use. For those people, providing additional human support would be beneficial to ensure uptake. Screening user expectations could help health care professionals optimize self-help eHealth intervention uptake in practice.

    Trial Registration: OSF Registries osf.io/n47cz; https://osf.io/n47cz

    JMIR Form Res 2024;8:e38803

    doi:10.2196/38803

    Keywords

    eHealth; human support; Unified Theory of Acceptance and Use of Technology; use intention; UTAUT; working alliance 


    eHealth provides the opportunity to provide remote or automated health care support through digital tools [1]. eHealth is becoming increasingly relevant, for example, because of the physical restrictions during the recent COVID-19 outbreak [2]. During this pandemic, the demand for health care support increased too. Especially vulnerable groups experienced increased mental health difficulties [3,4], which require professional support. However, health care professionals already have a high workload and pressure [5] and, in some cases, even experience an additional workload from using eHealth [6]. Self-help eHealth interventions might provide a potential solution to these problems. Self-help eHealth interventions are defined as interventions in which automated support instead of human assistance is provided [1]. As this means that no human professionals are involved, self-help eHealth interventions are easier and cheaper to widely implement [1].

    Despite these advantages, self-help interventions generally deal with low levels of adherence [7-10] and low uptake [11,12]. People generally show a higher intention to start with lifestyle changes using an intervention with additional human assistance compared to a self-help intervention [13]. While there has been extensive research on the factors contributing to nonadherence, there is a notable gap in our understanding when it comes to expectations that influence whether individuals will choose to use an intervention before starting. This information is important, as a growing number of eHealth tools are being developed and proven to be effective but hardly used [14,15]. Therefore, the aim of this study is to investigate whether there is a difference in use intentions between self-help and human-supported eHealth interventions and if user expectations influence the intention to use the intervention. If we know what expectations drive people’s intention to either use self-help or human-supported eHealth interventions, we could predict and even influence their actual uptake [16].

    A possible explanation for the low use intention of self-help interventions could be the lack of a relationship with a health care professional [17]. This so-called working alliance, the degree to which a health care professional and patient is involved in a useful and collaborative working relationship [18], is an important predictor of intervention adherence and effectiveness [19,20]. People are more engaged with the intervention and motivated to work on their goals when they feel supported. This effect is not exclusive to face-to-face settings; it is also evident when internet-based human assistance is involved in the use of eHealth interventions [21,22]. It is even shown to be present in self-help eHealth interventions with automated support, using, for example, a human avatar [23-25]. Thus, people can form relationships not only with other people but also with technology [26]. Therefore, we predict that people’s expectations toward a potential future working alliance when using an eHealth intervention will influence their intention to use that intervention.

    Other important expectations that may influence the use intention of human-supported and self-help eHealth interventions can be found within the Unified Theory of Acceptance and Use of Technology (UTAUT) [16]. According to this model, 3 different types of expectations explain people’s intention to start with an eHealth intervention. These UTAUT expectations are (1) performance expectancy: the extent to which someone expects that the eHealth intervention will be helpful in reaching their goals; (2) effort expectancy: the extent to which someone expects that the eHealth intervention will be easy to use; and (3) social influence: the extent to which someone expects that important others believe one should use the eHealth intervention [16]. Although the UTAUT model has been used to explain people’s intention to use eHealth in general [27,28], to our knowledge, no studies have used this model to investigate differences in people’s intention to use either human-supported or self-help eHealth interventions.

    In this study, we aim to investigate (1) whether there is a difference in use intention between human-supported and self-help eHealth interventions, (2) whether the expected working alliance predicts the use intention of human-supported and self-help eHealth interventions, and (3) what UTAUT constructs predict the use intention of human-supported and self-help eHealth interventions.


    Design and Sample

    In an experiment, people were presented with a sham stress management app. In this app, people would either be supported by a human coach or by an automated coach. We decided to use a student sample, as they experience high levels of stress and could therefore benefit from an eHealth stress intervention [29], especially given their increased need for support during the COVID-19 pandemic [3,4]. They were asked to evaluate the screenshots of the app and measure their use intention, the 3 UTAUT constructs (performance expectancy, effort expectancy, and social influence), and their expected working alliance. We used a randomized between-participants design with 2 experimental conditions (human-supported or self-help eHealth interventions). Healthy participants aged 18 years or older, who had a sufficient level of grasp in English, were recruited on the campus of Leiden University with internet-based and offline flyers. Power calculations [30] identified a minimum sample size of 119 to detect a medium effect (f2=0.15) with an α of .05, based on a linear multiple regression with 3 predictors.

    Procedure and Manipulation

    Interested participants could open the internet-based questionnaire and would be offered the internet-based consent form. After reading and agreeing to the informed consent, participants were automatically randomized into 1 of 2 experimental conditions (human-supported or self-help eHealth interventions). In both conditions, participants were instructed to evaluate a nonexistent stress management app for students called “Bye Bye Stress.” They were asked to carefully assess the screenshots of the app and give feedback to help the researchers make the app fit the needs of students. Although the design of the app and the content of the intervention were identical in both conditions, the conditions differed in the type of support that would be offered in the app. In the human-supported condition, the description of the app explained how a human coach would support the participants and provide them feedback. The screenshots of the app showed a picture of a human coach and messages with a human tone of voice (Figure 1). In the self-help condition, the description of the app explained how participants would receive automated feedback. In the screenshots, there was no picture of a human being, and the messages had a neutral tone of voice (Figure 1). All screenshots used in both conditions can be found in Multimedia Appendix 1. After this, participants were asked to complete the questionnaire.

    ‎
    Figure 1. Example screenshot of the app for human-supported (left) and self-help conditions (right).

    Measures

    Use Intention

    The behavioral intention subscale of the UTAUT questionnaire [16] was used to assess use intention. The subscale consists of 3 items (eg, “I would intend to use ‘Bye Bye Stress’ in the next 6 months.”) measured on a Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). A higher score indicates a higher intention to use the app. The scale showed a high internal consistency (Cronbach α=.953).

    Expected Working Alliance

    The expected working alliance was measured with an adjusted version of the Working Alliance Inventory–Short Revised form (WAI-SR) [31], which consists of 12 items measured on a 5-point Likert-type scale ranging from 1 (seldom) to 5 (always). Questions were adjusted to fit the context of the study by using the words “coach,” “lifestyle,” and “intervention” and being written in the future tense (eg, “The coach and I will collaborate on setting lifestyle goals.”). A higher score indicates a stronger expected working alliance. The adjusted version had a high internal consistency (Cronbach α=.917).

    Performance Expectancy, Effort Expectancy, and Social Influence

    The constructs predicting behavioral intention according to the UTAUT model—performance expectancy, effort expectancy, and social influence—were measured with the corresponding UTAUT subscales [16]. Each subscale consisted of 4 items (eg, “I find ‘Bye Bye Stress’ useful.”), measured with a Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). A higher score indicates a higher expectation of the app’s efficacy in helping the participant, a higher expectation toward the ease of use of the app, and a higher expectation that important others will approve the use of the app. The performance expectancy, effort expectancy, and social influence subscales all had sufficient internal consistency (Cronbach α of .764, .730, .792, respectively).

    Manipulation Check

    To assess whether participants carefully read the information and whether the manipulation had worked, they were asked to complete a manipulation check question (“During the intervention, I would be supported by...” followed by several options, such as “doctor” or “chatbot”).

    Analyses

    To test whether there was a difference in use intention between conditions, we ran a 2-tailed independent-sample t test with use intention as the dependent variable and condition (human-supported vs self-help eHealth interventions) as the independent variable. To test whether the association between condition and use intention differed for different levels of the working alliance, we conducted a univariate general linear model (GLM) analysis with interactions. We added use intention as the dependent variable, condition as a fixed factor, and expected working alliance as a covariate. We analyzed both the main effects of condition and expected working alliance, as well as their interaction effect on use intention. To further investigate the interaction patterns found in the data, we conducted a simple slopes analysis. To formulate the simple slope equations for both the human-supported condition and the self-help condition, the intercept and the slope were obtained from the parameter estimates of the GLM analysis testing the association between expected working alliance and use intention.

    To test whether the association between condition and use intention differed for different levels of the UTAUT constructs of performance expectancy, effort expectancy, and social influence, we conducted 3 univariate GLM analyses with interactions. We added use intention as dependent variable, condition as fixed factor, and each of the UTAUT constructs (performance expectancy, effort expectancy, or social influence) as a covariate in 3 separate analyses. We analyzed both the main effects of condition and the UTAUT construct, as well as their interaction effect on use intention. To further investigate the interaction patterns found in the data, we again conducted 3 simple slopes analyses: the intercept and the slope were obtained for both conditions from the parameter estimates of the GLM analyses testing the association between the UTAUT construct and use intention.

    Statistical analyses were conducted with SPSS (version 26; IBM Corp) with a significance level set at P≤.05.

    Ethical Considerations

    The study was approved by the Psychology Research Ethics Committee of Leiden University (CEP19-1125/557). Furthermore, the study was preregistered through the Center for Open Science [32]. Before the start of the study, participants were asked to sign an informed consent form. After completing all the questionnaires, they were debriefed and provided with a few examples of real internet-based stress management interventions in case they needed one. As compensation, participants received course credits.


    Demographic Characteristics

    A total of 146 students participated in our study and completed the questionnaire. Their mean age was 21.8 (SD 4.51) years, 103 (70.5%) were female, and 104 (71.2%) were of Dutch nationality (Table 1). There were no significant differences in demographic characteristics between the 2 groups.

    Table 1. Baseline demographic characteristics.
    VariableTotal sample (n=146)Human-supported condition (n=73)Self-help condition (n=73)
    Age (years), mean (SD)21.8 (4.5)22.0 (4.6)21.6 (4.4)
    Female, n (%)103 (70.5)47 (66.2)56 (76.7)
    Nationality, n (%)

    		Dutch104 (71.2)49 (67.1)55 (75.3)

    		European (non-Dutch)37 (25.3)20 (27.4)17 (23.3)

    		Other5 (3.4)4 (5.5)1 (1.4)

    Use Intention Per Condition

    We found no significant difference in use intention between the human-supported condition and self-help condition (t142=–1.133; P=.26; Table 2). Furthermore, we found no differences between the 2 conditions in any of the other constructs (Table 2).

    Table 2. Mean scores and SDs of use intention and its predictors.
    Variable (scoring range)Human-supported condition (n=73), mean (SD)Self-help condition (n=73), mean (SD)P value
    Use intention (3-15)7.5 (3.6)8.2 (3.6).26
    Expected working alliance (12-60)42.3 (8.2)40.3 (8.7).16
    Performance expectancy (4-20)13.8 (2.9)14.0 (2.5).69
    Effort expectancy (4-20)16.9 (2.4)16.8 (2.3).94
    Social influence (4-20)12.5 (2.9)12.9 (3.1).66

    Working Alliance and Use Intention

    The GLM showed no significant association between condition and expected working alliance (F1,140=0.051; P=.82; η2=0). However, we did find a significant positive association between expected working alliance and use intention (F1,140=26.435; P<.001; η2=0.159). We found no significant interaction effect of condition and expected working alliance on use intention (F1,140=0.367; P=.55; η2=0.003; Figure 2).

    ‎
    Figure 2. Simple slopes of the effects of expected working alliance, performance expectancy, effort expectancy, and social influence on use intention.

    UTAUT Constructs and Use Intention

    The GLM showed no significant association between condition and performance expectancy (F1,140=3.34; P=.07; η2=0.024). We did, however, find a significant positive association between performance expectancy and use intention (F1,140=69.269; P<.001; η2=0.331) and a significant interaction effect of condition and performance expectancy on use intention (F1,140=4.363; P=.04; η2=0.030). An increase in performance expectancy was related to a greater increase in use intention in the self-help condition compared to the human-supported condition (Figure 2).

    We also found no significant association between condition and effort expectancy (F1,140=3.4086; P=.07; η2=0.024). However, again, we did find a significant positive association between effort expectancy and use intention (F1,140=3.961; P=.049; η2=0.028) and a significant interaction effect of condition and effort expectancy on use intention (F1,140=4.102; P=.045; η2=0.028). An increase in effort expectancy was related to a greater increase in use intention in the self-help condition but not in the human-supported condition (Figure 2).

    Again, we found no significant association between condition and social influence (F1,140=0.003; P=.96; η2=0). We did find a significant positive association between social influence and use intention (F1,140=90.025; P<.001; η2=0.391) but this time we found no significant interaction effect of condition and social influence on use intention (F1,140=0.020; P=.89; η2=0; Figure 2).


    Overview

    In our study, we asked university students to evaluate a sham stress management app. We aimed to investigate whether there is a difference in use intention for self-help eHealth interventions compared to human-supported ones and what user expectations may influence this. We found that people were as likely to start using a self-help eHealth intervention as an eHealth intervention with human support. More than with human-supported interventions, the perception that the intervention might be ineffective or difficult to use limits the intention to start using self-help interventions. See Figure 3 for an overview of the findings.

    ‎
    Figure 3. Overview of study findings.

    Although previous studies show a relatively low uptake and use intention of self-help eHealth interventions [11-13], we did not find differences in use intentions between the self-help and human-supported interventions. Possibly, the health beliefs, perceptions, and skills of our student sample might have played a role in this [33]. Not only do perceptions about the effectiveness or ease of use of an eHealth tool affect the start of an intervention but also perceptions about the risks of getting health-related problems and actually performing the health-promoting behavior [34]. Furthermore, a younger age and higher educational level are related to a higher intention to start eHealth interventions in general [13]. Our sample might therefore have been more open to using eHealth interventions and were less influenced by the presence, or lack thereof, of human support. Future research could focus on investigating the role of age and educational level on use intentions of self-help and human-supported eHealth interventions. Another explanation for the differences in findings between our and previous studies [11,12] could be the use of different outcome measures. Although the UTAUT model predicts that use intention can predict actual use, studies do show that people have difficulties translating their intentions into actual behavior [35]. The objective measure of uptake might therefore have led to different results compared to the more subjective measure of use intention we used, which would be interesting to additionally take into account. Finally, the study that did find a difference in use intention between self-help and human-supported interventions focused on interventions for mental health, such as depression [13]. It would be interesting to test if the need for social support during eHealth interventions depends on the goal of the intervention (eg, psychological vs lifestyle improvements).

    Interestingly, we found that an expected working alliance has an equally strong effect on the intention to use either a human-supported or self-help intervention. This result is in line with previous studies showing a positive effect of working alliance on intervention effectiveness and adherence, both within human-supported [21,22] and self-help eHealth interventions with automated support [23-25]. Our findings show that working alliance is not important only during an intervention but even before the intervention has started in the form of expectations. The similar effect of the expected working alliance in both conditions suggests that people not only are able to actually have relationships with technology [26] but also seem to expect building one with the technology they are about to interact with. These results would also mean that improving the expected working alliance before the start of an intervention (eg, by designing a digital character that would welcome the user) would be a way to possibly increase the uptake of self-help eHealth interventions.

    Finally, we found that performance and effort expectancy had a stronger effect on the use intention of self-help interventions compared to human-supported interventions. Not only the UTAUT model but also models such as the Health Belief model show that perceived benefits and perceived barriers affect whether people start with a health-promoting behavior, such as stress management [33]. What is new, though, is that the perceived effectiveness and ease of use of the intervention have a more pronounced impact on intention to use an intervention for interventions with an absence of human support compared to interventions where human support is available. This suggests that the perception that the intervention might be ineffective or difficult to use diminishes the intention to start using a self-help intervention but not the intention to start using a human-supported intervention. Meta-analyses show that the mere presence of a human being (even a nonprofessional) is a key ingredient in intervention effectiveness and the prevention of dropout [36-38]. Just the option of having someone available to provide procedural support (related to performance expectancy) or technical support (related to effort expectancy) seems to be enough for people to be motivated to start something new. The presence of a human coach could act as a buffer against negative expectations, which would make it easier for these people to adhere to the intervention [39]. Possibly, the mere presence of social support in the human-supported intervention could compensate for a lack of self-efficacy (the extent to which one believes in his or her own capabilities [40]) that people may feel when using a new intervention [41,42]. This could lower the perceived barriers and increase willingness to start using the intervention [33]. Exploring this further is crucial in a clinical context because individuals with limited social support tend to experience reduced adherence to health interventions and demonstrate less favorable intervention outcomes [39,43]. Even despite the relatively high use intention of self-help eHealth interventions, these results indicate that it is important to take the user’s needs and wishes into account when deciding on the level of human support to provide during an intervention.

    Self-help eHealth interventions will become more and more important in health care practice. To ensure uptake of new eHealth interventions, professionals could screen the user’s expectations toward the intervention’s helpfulness and ease of use beforehand (Table 3). If the user’s expectations turn out to be low, it would be useful to incorporate some level of human support into the eHealth intervention to prevent people from dropping out even before the start of the intervention. Additionally, designers of self-help eHealth interventions could pay extra attention toward its perceived helpfulness and ease of use. Preventing negative user expectations toward the intervention’s performance or effort expectancy could help increase the uptake of self-help eHealth interventions.

    Table 3. Items of the Unified Theory of Acceptance and Use of Technology subscales: performance expectancy (PE) and effort expectancy (EE).
    ItemStatement
    PE1I find [name eHealth technology] useful.
    PE2Using [name eHealth technology] enables me to [target behavior].
    PE3Using [name eHealth technology] will [target behavior].
    PE4If I use [name eHealth technology] I will know how to [target behavior].
    EE1My interaction with [name eHealth technology] is clear and understandable.
    EE2It would be easy for me to develop the skills needed to use [name eHealth technology].
    EE3I think [name eHealth technology] would be easy to use.
    EE4It would be easy to learn how to operate [name eHealth technology].

    Strengths and Limitations

    Our study was not without limitations. For example, although the screenshots of the app were adjusted to the experiences and interests of our sample, it is plausible that the topic of stress management was not equally relevant for all students, which could also have affected use intentions. For future studies, it would be better to tailor the goal of the eHealth intervention (eg, decreasing stress or improving physical activity) to the actual interests of the individual participants to investigate if and how this affects a participant’s use intention. Second, we used a university student population to test our hypotheses. People with a younger age and higher educational level have a more favorable attitude toward eHealth interventions in general [13]. To be able to generalize our findings, future research should investigate whether the same effects are found in other populations. It would be interesting to replicate this study with a target population who would benefit the most from eHealth interventions, for example, older patients with a chronic disease, to see if their expectations toward either human or automated support have similar effects on their intention to start with such interventions.

    Conclusions

    In our study, we investigated what expectations drive the intention to start using self-help and human-supported eHealth interventions. The results suggest that expectations toward the intervention’s helpfulness and ease of use are especially relevant regarding the use of self-help interventions. This means that people who have doubts about the intervention’s usefulness or usability would benefit the most from additional human support. The question, however, remains whether such expectations are also relevant for actual uptake. Our study provides a basis to further investigate user expectations within a clinical sample, which will provide health care practitioners with the tools to influence the uptake of eHealth interventions.

    Acknowledgments

    This work was supported by the Netherlands Cardiovascular Research Initiative, an initiative with the support of the Dutch Heart Foundation, CVON2016-12 BENEFIT, ZonMw (the Netherlands Organization for Health Research and Development), and the members of the BENEFIT consortium. Furthermore, we would like to thank Yalou Schoot (master student of psychology, Leiden University) for her assistance in the practical implementation of the study procedure and for data acquisition.

    Authors' Contributions

    TRCR and TR contributed to study design. TRCR was responsible for data acquisition. TRCR, TR, and AWME were involved in data analysis and interpretation and drafting the manuscript. TRCR, TR, LDB, VRJ, RAK, DEA, and AWME contributed to manuscript revision. All authors gave final approval and agreed to be accountable for all aspects of the work ensuring integrity and accuracy.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Screenshots shown in human-supported and self-help conditions.

    DOCX File , 2647 KB
    Multimedia Appendix 2

    CONSORT-eHEALTH checklist (V 1.6.1).

    PDF File (Adobe PDF File), 1095 KB
    1. Barak A, Klein B, Proudfoot JG. Defining internet-supported therapeutic interventions. Ann Behav Med. 2009;38(1):4-17. [https://academic.oup.com/abm/article/38/1/4/4569461?login=false] [CrossRef] [Medline]
    2. Bokolo AJ. Application of telemedicine and eHealth technology for clinical services in response to COVID-19 pandemic. Health Technol (Berl). 2021;11(2):359-366. [https://europepmc.org/abstract/MED/33469474] [CrossRef] [Medline]
    3. Browning MHEM, Larson LR, Sharaievska I, Rigolon A, McAnirlin O, Mullenbach L, et al. Psychological impacts from COVID-19 among university students: risk factors across seven states in the United States. PLoS One. 2021;16(1):e0245327. [https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0245327] [CrossRef] [Medline]
    4. Husky MM, Kovess-Masfety V, Swendsen JD. Stress and anxiety among university students in France during COVID-19 mandatory confinement. Compr Psychiatry. 2020;102:152191. [https://linkinghub.elsevier.com/retrieve/pii/S0010-440X(20)30033-X] [CrossRef] [Medline]
    5. Simionato GK, Simpson S. Personal risk factors associated with burnout among psychotherapists: a systematic review of the literature. J Clin Psychol. 2018;74(9):1431-1456. [CrossRef] [Medline]
    6. Bellicha A, Macé S, Oppert JM. Prescribing of electronic activity monitors in cardiometabolic diseases: qualitative interview-based study. J Med Internet Res. 2017;19(9):e328. [https://www.jmir.org/2017/9/e328/] [CrossRef] [Medline]
    7. Kelders SM, Kok RN, Ossebaard HC, Van Gemert-Pijnen JEWC. Persuasive system design does matter: a systematic review of adherence to web-based interventions. J Med Internet Res. 2012;14(6):e152. [https://www.jmir.org/2012/6/e152/] [CrossRef] [Medline]
    8. Kelders SM, Van Gemert-Pijnen JEWC, Werkman A, Nijland N, Seydel ER. Effectiveness of a web-based intervention aimed at healthy dietary and physical activity behavior: a randomized controlled trial about users and usage. J Med Internet Res. 2011;13(2):e32. [https://www.jmir.org/2011/2/e32/] [CrossRef] [Medline]
    9. Murray E, White IR, Varagunam M, Godfrey C, Khadjesari Z, McCambridge J. Attrition revisited: adherence and retention in a web-based alcohol trial. J Med Internet Res. 2013;15(8):e162. [https://www.jmir.org/2013/8/e162/] [CrossRef] [Medline]
    10. Wangberg SC, Bergmo TS, Johnsen JK. Adherence in internet-based interventions. Patient Prefer Adherence. 2008;2:57-65. [https://europepmc.org/abstract/MED/19920945] [CrossRef] [Medline]
    11. Lillevoll KR, Vangberg HCB, Griffiths KM, Waterloo K, Eisemann MR. Uptake and adherence of a self-directed internet-based mental health intervention with tailored e-mail reminders in senior high schools in Norway. BMC Psychiatry. 2014;14:14. [https://bmcpsychiatry.biomedcentral.com/articles/10.1186/1471-244X-14-14] [CrossRef] [Medline]
    12. Lin J, Faust B, Ebert DD, Krämer L, Baumeister H. A web-based acceptance-facilitating intervention for identifying patients' acceptance, uptake, and adherence of internet- and mobile-based pain interventions: randomized controlled trial. J Med Internet Res. 2018;20(8):e244. [https://www.jmir.org/2018/8/e244/] [CrossRef] [Medline]
    13. Apolinário-Hagen J, Kemper J, Stürmer C. Public acceptability of e-mental health treatment services for psychological problems: a scoping review. JMIR Ment Health. 2017;4(2):e10. [https://mental.jmir.org/2017/2/e10/] [CrossRef] [Medline]
    14. Ross J, Stevenson F, Lau R, Murray E. Factors that influence the implementation of e-health: a systematic review of systematic reviews (an update). Implement Sci. 2016;11(1):146. [https://implementationscience.biomedcentral.com/articles/10.1186/s13012-016-0510-7] [CrossRef] [Medline]
    15. van Gemert-Pijnen JEWC, Nijland N, van Limburg M, Ossebaard HC, Kelders SM, Eysenbach G, et al. A holistic framework to improve the uptake and impact of eHealth technologies. J Med Internet Res. 2011;13(4):e111. [https://www.jmir.org/2011/4/e111/] [CrossRef] [Medline]
    16. Venkatesh V, Morris MG, Davis GB, Davis FD. User acceptance of information technology: toward a unified view. MIS Quarterly. 2003;27(3):425-478. [CrossRef]
    17. Brandt CJ, Søgaard GI, Clemensen J, Søndergaard J, Nielsen JB. Determinants of successful eHealth coaching for consumer lifestyle changes: qualitative interview study among health care professionals. J Med Internet Res. 2018;20(7):e237. [https://www.jmir.org/2018/7/e237/] [CrossRef] [Medline]
    18. Hatcher RL, Barends AW. How a return to theory could help alliance research. Psychotherapy (Chic). 2006;43(3):292-299. [CrossRef] [Medline]
    19. Goldberg SB, Davis JM, Hoyt WT. The role of therapeutic alliance in mindfulness interventions: therapeutic alliance in mindfulness training for smokers. J Clin Psychol. 2013;69(9):936-950. [https://europepmc.org/abstract/MED/23775222] [CrossRef] [Medline]
    20. Martin DJ, Garske JP, Davis MK. Relation of the therapeutic alliance with outcome and other variables: a meta-analytic review. J Consult Clin Psychol. 2000;68(3):438-450. [CrossRef] [Medline]
    21. Flückiger C, Del Re AC, Wampold BE, Horvath AO. The alliance in adult psychotherapy: a meta-analytic synthesis. Psychotherapy (Chic). 2018;55(4):316-340. [https://www.zora.uzh.ch/id/eprint/157613/] [CrossRef] [Medline]
    22. Sucala M, Schnur JB, Constantino MJ, Miller SJ, Brackman EH, Montgomery GH. The therapeutic relationship in e-therapy for mental health: a systematic review. J Med Internet Res. 2012;14(4):e110. [https://www.jmir.org/2012/4/e110/] [CrossRef] [Medline]
    23. Bickmore TW, Mitchell SE, Jack BW, Paasche-Orlow MK, Pfeifer LM, Odonnell J. Response to a relational agent by hospital patients with depressive symptoms. Interact Comput. 2010;22(4):289-298. [https://europepmc.org/abstract/MED/20628581] [CrossRef] [Medline]
    24. Clarke J, Proudfoot J, Whitton A, Birch MR, Boyd M, Parker G, et al. Therapeutic alliance with a fully automated mobile phone and web-based intervention: secondary analysis of a randomized controlled trial. JMIR Ment Health. 2016;3(1):e10. [https://mental.jmir.org/2016/1/e10/] [CrossRef] [Medline]
    25. Hauser-Ulrich S, Künzli H, Meier-Peterhans D, Kowatsch T. A smartphone-based health care chatbot to promote self-management of chronic pain (SELMA): pilot randomized controlled trial. JMIR Mhealth Uhealth. 2020;8(4):e15806. [https://mhealth.jmir.org/2020/4/e15806/] [CrossRef] [Medline]
    26. Nass C, Moon Y. Machines and mindlessness: social responses to computers. J Soc Issues. 2002;56(1):81-103. [CrossRef]
    27. Alharbi F. The use of digital healthcare platforms during the COVID-19 pandemic: the consumer perspective. Acta Inform Med. 2021;29(1):51-58. [https://europepmc.org/abstract/MED/34012214] [CrossRef] [Medline]
    28. Duarte P, Pinho JC. A mixed methods UTAUT2-based approach to assess mobile health adoption. J Bus Res. 2019;102:140-150. [CrossRef]
    29. Stallman HM. Psychological distress in university students: a comparison with general population data. Aust Psychol. 2010;45(4):249-257. [CrossRef]
    30. Faul F, Erdfelder E, Lang AG, Buchner A. G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods. 2007;39(2):175-191. [https://link.springer.com/article/10.3758/BF03193146] [CrossRef] [Medline]
    31. Hatcher RL, Gillaspy JA. Development and validation of a revised short version of the working alliance inventory. Psychother Res. 2006;16(1):12-25. [CrossRef]
    32. Rodrigues TC, Reijnders T. Intended use of human-supported and self-help eHealth interventions. OSF Registries. 2021. URL: https://osf.io/n47cz [accessed 2023-12-29]
    33. van der Waal NE, de Wit J, Bol N, Ebbers W, Hooft L, Metting E, et al. Predictors of contact tracing app adoption: integrating the UTAUT, HBM and contextual factors. Technol Soc. 2022;71:102101. [https://www.sciencedirect.com/science/article/pii/S0160791X22002421?via%3Dihub] [CrossRef]
    34. Rosenstock IM. The health belief model and preventive health behavior. Health Educ Monogr. 1974;2(4):354-386. [CrossRef]
    35. Webb TL, Sheeran P. Does changing behavioral intentions engender behavior change? a meta-analysis of the experimental evidence. Psychol Bull. 2006;132(2):249-268. [CrossRef] [Medline]
    36. Etzelmueller A, Vis C, Karyotaki E, Baumeister H, Titov N, Berking M, et al. Effects of internet-based cognitive behavioral therapy in routine care for adults in treatment for depression and anxiety: systematic review and meta-analysis. J Med Internet Res. 2020;22(8):e18100. [https://www.jmir.org/2020/8/e18100/] [CrossRef] [Medline]
    37. Karyotaki E, Ebert DD, Donkin L, Riper H, Twisk J, Burger S, et al. Do guided internet-based interventions result in clinically relevant changes for patients with depression? an individual participant data meta-analysis. Clin Psychol Rev. 2018;63:80-92. [CrossRef] [Medline]
    38. Richards D, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev. 2012;32(4):329-342. [CrossRef] [Medline]
    39. Miller T, Dimatteo MR. Importance of family/social support and impact on adherence to diabetic therapy. Diabetes Metab Syndr Obes. 2013;6:421-426. [https://europepmc.org/abstract/MED/24232691] [CrossRef] [Medline]
    40. Bandura A, Freeman WH, Lightsey R. Self-efficacy: the exercise of control. J Cogn Psychother. 1999;13(2):158-166. [CrossRef]
    41. Fernández BR, Montenegro EM, Knoll N, Schwarzer R. Self-efficacy, action control, and social support explain physical activity changes among Costa Rican older adults. J Phys Act Health. 2014;11(8):1573-1578. [CrossRef] [Medline]
    42. Zhou G, Gan Y, Hamilton K, Schwarzer R. The role of social support and self-efficacy for planning fruit and vegetable intake. J Nutr Educ Behav. 2017;49(2):100-106.e1. [CrossRef] [Medline]
    43. Lindfors O, Ojanen S, Jääskeläinen T, Knekt P. Social support as a predictor of the outcome of depressive and anxiety disorder in short-term and long-term psychotherapy. Psychiatry Res. 2014;216(1):44-51. [CrossRef] [Medline]

    ‎
    GLM: general linear model
    UTAUT: Unified Theory of Acceptance and Use of Technology
    WAI-SR: Working Alliance Inventory–Short Revised form

    Edited by A Mavragani; submitted 16.04.22; peer-reviewed by WP Brinkman, K Roystonn, L Guo; comments to author 12.09.23; revised version received 28.10.23; accepted 20.11.23; published 15.02.24

    Copyright

    ©Talia R Cohen Rodrigues, Thomas Reijnders, Linda D Breeman, Veronica R Janssen, Roderik A Kraaijenhagen, Douwe E Atsma, Andrea WM Evers. Originally published in JMIR Formative Research (https://formative.jmir.org), 15.02.2024.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.