In an experiment, people were presented with a sham stress management app. In this app, people would either be supported by a human coach or by an automated coach. We decided to use a student sample, as they experience high levels of stress and could therefore benefit from an eHealth stress intervention [29], especially given their increased need for support during the COVID-19 pandemic [3,4]. They were asked to evaluate the screenshots of the app and measure their use intention, the 3 UTAUT constructs (performance expectancy, effort expectancy, and social influence), and their expected working alliance. We used a randomized between-participants design with 2 experimental conditions (human-supported or self-help eHealth interventions). Healthy participants aged 18 years or older, who had a sufficient level of grasp in English, were recruited on the campus of Leiden University with internet-based and offline flyers. Power calculations [30] identified a minimum sample size of 119 to detect a medium effect (f²=0.15) with an α of .05, based on a linear multiple regression with 3 predictors.

Interested participants could open the internet-based questionnaire and would be offered the internet-based consent form. After reading and agreeing to the informed consent, participants were automatically randomized into 1 of 2 experimental conditions (human-supported or self-help eHealth interventions). In both conditions, participants were instructed to evaluate a nonexistent stress management app for students called “Bye Bye Stress.” They were asked to carefully assess the screenshots of the app and give feedback to help the researchers make the app fit the needs of students. Although the design of the app and the content of the intervention were identical in both conditions, the conditions differed in the type of support that would be offered in the app. In the human-supported condition, the description of the app explained how a human coach would support the participants and provide them feedback. The screenshots of the app showed a picture of a human coach and messages with a human tone of voice (Figure 1). In the self-help condition, the description of the app explained how participants would receive automated feedback. In the screenshots, there was no picture of a human being, and the messages had a neutral tone of voice (Figure 1). All screenshots used in both conditions can be found in Multimedia Appendix 1. After this, participants were asked to complete the questionnaire.

To test whether there was a difference in use intention between conditions, we ran a 2-tailed independent-sample t test with use intention as the dependent variable and condition (human-supported vs self-help eHealth interventions) as the independent variable. To test whether the association between condition and use intention differed for different levels of the working alliance, we conducted a univariate general linear model (GLM) analysis with interactions. We added use intention as the dependent variable, condition as a fixed factor, and expected working alliance as a covariate. We analyzed both the main effects of condition and expected working alliance, as well as their interaction effect on use intention. To further investigate the interaction patterns found in the data, we conducted a simple slopes analysis. To formulate the simple slope equations for both the human-supported condition and the self-help condition, the intercept and the slope were obtained from the parameter estimates of the GLM analysis testing the association between expected working alliance and use intention.

To test whether the association between condition and use intention differed for different levels of the UTAUT constructs of performance expectancy, effort expectancy, and social influence, we conducted 3 univariate GLM analyses with interactions. We added use intention as dependent variable, condition as fixed factor, and each of the UTAUT constructs (performance expectancy, effort expectancy, or social influence) as a covariate in 3 separate analyses. We analyzed both the main effects of condition and the UTAUT construct, as well as their interaction effect on use intention. To further investigate the interaction patterns found in the data, we again conducted 3 simple slopes analyses: the intercept and the slope were obtained for both conditions from the parameter estimates of the GLM analyses testing the association between the UTAUT construct and use intention.

Statistical analyses were conducted with SPSS (version 26; IBM Corp) with a significance level set at P≤.05.

The study was approved by the Psychology Research Ethics Committee of Leiden University (CEP19-1125/557). Furthermore, the study was preregistered through the Center for Open Science [32]. Before the start of the study, participants were asked to sign an informed consent form. After completing all the questionnaires, they were debriefed and provided with a few examples of real internet-based stress management interventions in case they needed one. As compensation, participants received course credits.

A total of 146 students participated in our study and completed the questionnaire. Their mean age was 21.8 (SD 4.51) years, 103 (70.5%) were female, and 104 (71.2%) were of Dutch nationality (Table 1). There were no significant differences in demographic characteristics between the 2 groups.

We found no significant difference in use intention between the human-supported condition and self-help condition (t₁₄₂=–1.133; P=.26; Table 2). Furthermore, we found no differences between the 2 conditions in any of the other constructs (Table 2).

The GLM showed no significant association between condition and expected working alliance (F_1,140=0.051; P=.82; η²=0). However, we did find a significant positive association between expected working alliance and use intention (F_1,140=26.435; P<.001; η²=0.159). We found no significant interaction effect of condition and expected working alliance on use intention (F_1,140=0.367; P=.55; η²=0.003; Figure 2).

The GLM showed no significant association between condition and performance expectancy (F_1,140=3.34; P=.07; η²=0.024). We did, however, find a significant positive association between performance expectancy and use intention (F_1,140=69.269; P<.001; η²=0.331) and a significant interaction effect of condition and performance expectancy on use intention (F_1,140=4.363; P=.04; η²=0.030). An increase in performance expectancy was related to a greater increase in use intention in the self-help condition compared to the human-supported condition (Figure 2).

We also found no significant association between condition and effort expectancy (F_1,140=3.4086; P=.07; η²=0.024). However, again, we did find a significant positive association between effort expectancy and use intention (F_1,140=3.961; P=.049; η²=0.028) and a significant interaction effect of condition and effort expectancy on use intention (F_1,140=4.102; P=.045; η²=0.028). An increase in effort expectancy was related to a greater increase in use intention in the self-help condition but not in the human-supported condition (Figure 2).

Again, we found no significant association between condition and social influence (F_1,140=0.003; P=.96; η²=0). We did find a significant positive association between social influence and use intention (F_1,140=90.025; P<.001; η²=0.391) but this time we found no significant interaction effect of condition and social influence on use intention (F_1,140=0.020; P=.89; η²=0; Figure 2).

In our study, we asked university students to evaluate a sham stress management app. We aimed to investigate whether there is a difference in use intention for self-help eHealth interventions compared to human-supported ones and what user expectations may influence this. We found that people were as likely to start using a self-help eHealth intervention as an eHealth intervention with human support. More than with human-supported interventions, the perception that the intervention might be ineffective or difficult to use limits the intention to start using self-help interventions. See Figure 3 for an overview of the findings.

Although previous studies show a relatively low uptake and use intention of self-help eHealth interventions [11-13], we did not find differences in use intentions between the self-help and human-supported interventions. Possibly, the health beliefs, perceptions, and skills of our student sample might have played a role in this [33]. Not only do perceptions about the effectiveness or ease of use of an eHealth tool affect the start of an intervention but also perceptions about the risks of getting health-related problems and actually performing the health-promoting behavior [34]. Furthermore, a younger age and higher educational level are related to a higher intention to start eHealth interventions in general [13]. Our sample might therefore have been more open to using eHealth interventions and were less influenced by the presence, or lack thereof, of human support. Future research could focus on investigating the role of age and educational level on use intentions of self-help and human-supported eHealth interventions. Another explanation for the differences in findings between our and previous studies [11,12] could be the use of different outcome measures. Although the UTAUT model predicts that use intention can predict actual use, studies do show that people have difficulties translating their intentions into actual behavior [35]. The objective measure of uptake might therefore have led to different results compared to the more subjective measure of use intention we used, which would be interesting to additionally take into account. Finally, the study that did find a difference in use intention between self-help and human-supported interventions focused on interventions for mental health, such as depression [13]. It would be interesting to test if the need for social support during eHealth interventions depends on the goal of the intervention (eg, psychological vs lifestyle improvements).

Interestingly, we found that an expected working alliance has an equally strong effect on the intention to use either a human-supported or self-help intervention. This result is in line with previous studies showing a positive effect of working alliance on intervention effectiveness and adherence, both within human-supported [21,22] and self-help eHealth interventions with automated support [23-25]. Our findings show that working alliance is not important only during an intervention but even before the intervention has started in the form of expectations. The similar effect of the expected working alliance in both conditions suggests that people not only are able to actually have relationships with technology [26] but also seem to expect building one with the technology they are about to interact with. These results would also mean that improving the expected working alliance before the start of an intervention (eg, by designing a digital character that would welcome the user) would be a way to possibly increase the uptake of self-help eHealth interventions.

Finally, we found that performance and effort expectancy had a stronger effect on the use intention of self-help interventions compared to human-supported interventions. Not only the UTAUT model but also models such as the Health Belief model show that perceived benefits and perceived barriers affect whether people start with a health-promoting behavior, such as stress management [33]. What is new, though, is that the perceived effectiveness and ease of use of the intervention have a more pronounced impact on intention to use an intervention for interventions with an absence of human support compared to interventions where human support is available. This suggests that the perception that the intervention might be ineffective or difficult to use diminishes the intention to start using a self-help intervention but not the intention to start using a human-supported intervention. Meta-analyses show that the mere presence of a human being (even a nonprofessional) is a key ingredient in intervention effectiveness and the prevention of dropout [36-38]. Just the option of having someone available to provide procedural support (related to performance expectancy) or technical support (related to effort expectancy) seems to be enough for people to be motivated to start something new. The presence of a human coach could act as a buffer against negative expectations, which would make it easier for these people to adhere to the intervention [39]. Possibly, the mere presence of social support in the human-supported intervention could compensate for a lack of self-efficacy (the extent to which one believes in his or her own capabilities [40]) that people may feel when using a new intervention [41,42]. This could lower the perceived barriers and increase willingness to start using the intervention [33]. Exploring this further is crucial in a clinical context because individuals with limited social support tend to experience reduced adherence to health interventions and demonstrate less favorable intervention outcomes [39,43]. Even despite the relatively high use intention of self-help eHealth interventions, these results indicate that it is important to take the user’s needs and wishes into account when deciding on the level of human support to provide during an intervention.

Self-help eHealth interventions will become more and more important in health care practice. To ensure uptake of new eHealth interventions, professionals could screen the user’s expectations toward the intervention’s helpfulness and ease of use beforehand (Table 3). If the user’s expectations turn out to be low, it would be useful to incorporate some level of human support into the eHealth intervention to prevent people from dropping out even before the start of the intervention. Additionally, designers of self-help eHealth interventions could pay extra attention toward its perceived helpfulness and ease of use. Preventing negative user expectations toward the intervention’s performance or effort expectancy could help increase the uptake of self-help eHealth interventions.

Our study was not without limitations. For example, although the screenshots of the app were adjusted to the experiences and interests of our sample, it is plausible that the topic of stress management was not equally relevant for all students, which could also have affected use intentions. For future studies, it would be better to tailor the goal of the eHealth intervention (eg, decreasing stress or improving physical activity) to the actual interests of the individual participants to investigate if and how this affects a participant’s use intention. Second, we used a university student population to test our hypotheses. People with a younger age and higher educational level have a more favorable attitude toward eHealth interventions in general [13]. To be able to generalize our findings, future research should investigate whether the same effects are found in other populations. It would be interesting to replicate this study with a target population who would benefit the most from eHealth interventions, for example, older patients with a chronic disease, to see if their expectations toward either human or automated support have similar effects on their intention to start with such interventions.

In our study, we investigated what expectations drive the intention to start using self-help and human-supported eHealth interventions. The results suggest that expectations toward the intervention’s helpfulness and ease of use are especially relevant regarding the use of self-help interventions. This means that people who have doubts about the intervention’s usefulness or usability would benefit the most from additional human support. The question, however, remains whether such expectations are also relevant for actual uptake. Our study provides a basis to further investigate user expectations within a clinical sample, which will provide health care practitioners with the tools to influence the uptake of eHealth interventions.