Participant Adherence and Contact Behavior in a Guided Internet Intervention for Depressive Symptoms: Exploratory Study

Introduction

Research on internet interventions has snowballed in the last 2 decades. Most studies investigated a guided self-help approach, in which a web-based self-help program is presented with therapist guidance, a minimal but regular therapist contact often via email [1]. Intensive research has shown that these interventions are effective in various clinical problems [2]. Furthermore, concerning depression, it has been shown that internet interventions effectively reduce depressive symptoms and that guided interventions tend to be more efficacious than unguided interventions [3-5]. Thereby, it seems that participants show a larger symptom improvement when they engage intensively with the program and demonstrate high adherence (usually measured by the number of clicks, completed modules, or time spent in the program) [3,6-10].

Apart from adherence to web-based programs, easily measurable aspects of guidance could also serve as indicators for the likelihood of participants’ symptom improvement. Indeed, there is anecdotal evidence that the number of messages written by participants may provide clues (personal communication from a study by Berger and colleagues [11]). For example, some participants had written many messages to the therapists and sought more contact. These participants tended to improve less than participants who wrote fewer messages to their therapist.

In this exploratory paper, we wanted to explore whether participants who differ in their adherence and contact behavior also differ in the extent of change in depressive symptoms. For this purpose, we used a completer sample of guided participants from a previous study [3]. We divided the sample based on median splits of adherence and the number of messages they sent to therapists. We then examined the 4 groups for their average rate of change in depressive symptoms. Our results may inspire future research to examine participants’ contact behavior toward therapists more closely.

Methods

Participants

For the current analyses, we used data from guided participants who filled in questionnaires at either posttreatment or follow-up (n=113) from a randomized full factorial trial [3]. The original study recruited 317 participants between February 2020 and February 2021 with mild to moderate depressive symptoms from Switzerland, Germany, and Austria through depression-related websites, radio interviews, self-help groups, Facebook groups, Google advertisements, and the website of the University of Bern (Bern, Switzerland). Interested individuals registered on our study website HERMES [12]. Inclusion criteria were (1) being at least 18 years of age, (2) indicating mild to moderate depressive symptoms on the Patient Health Questionnaire-9 (PHQ-9 score between 5 and 14), (3) providing written informed consent, (4) having access to the internet and an email account, and (5) providing an emergency contact. Exclusion criteria were (1) reporting a present or past psychotic or bipolar disorder or (2) indicating increased suicidal tendencies on the Suicidal Behavior Questionnaire-Revised (SBQ-R; score >7). Of note, participants taking medication or seeing a psychotherapist could take part in the study. The participants were not compensated for taking part in the study. Please see our previous publications for more details about the study design, randomization procedure, power considerations, the self-help program, treatment conditions, and study outcome measures [3,13].

Ethical Considerations

The ethics committee of the canton of Bern (Kantonale Ethikkommission Bern) approved the HERMES study on January 20, 2020 (2019‐01795). The study was preregistered at ClinicalTrial.gov (NCT04318236). The participants could only take part in the study if they provided informed consent (Multimedia Appendix 1). They were informed that taking part in the study was at all times voluntary and that they could opt out of the study at any time without providing a reason. The informed consent covered both primary and secondary research questions and analyses. The participant data were anonymized and replaced with a code. Data could not be tracked back to an individual except with a list that included the names of the individuals and their respective codes. This list was securely locked and only accessible to the authors of this paper. The participants were not compensated for taking part in the study.

Statistical Analyses

We used assessments at pretreatment, 8 weeks after pretreatment (posttreatment), and 16 weeks after pretreatment (follow-up). We focused on guided participants who completed the Patient Health Questionnaire-9 (PHQ-9) assessment posttreatment or follow-up. The PHQ-9 is a validated 9-item questionnaire to assess depressive symptoms [14]. We defined “adherence” as the extent to which participants used the self-help program. Following a suggestion by Donkin and colleagues [6], we calculated a composite score by averaging the z scores of the following indicators: number of clicks, number of topics worked on, number of completed exercises, and time spent on the program. We calculated adherence for the time from baseline to posttreatment. “Participants’ messages” were defined as the number of all messages that guided participants sent to their therapist within the self-help program from pre- to posttreatment.

For the analyses, we divided the completer sample into 4 groups based on median splits of 2 variables: adherence and participants’ messages. Therefore, the participants were categorized as showing low versus high adherence and sending few versus many messages to the therapist. We used chi-square (χ²) tests to evaluate group differences in participant characteristics at baseline for categorical data and Kruskal-Wallis χ² tests for nonnormally distributed continuous data. Furthermore, we used Kruskal-Wallis χ² tests to calculate the main effects of the 2 variables: adherence and participants’ messages. To compare the 4 groups concerning the PHQ-9 score changes from pre- to posttreatment and pretreatment to follow-up, we used Kruskal-Wallis tests and post hoc Dunn tests for pairwise comparisons.

Results

Baseline Evaluation

Of the original sample, 113 of 150 (75.3%) guided participants provided PHQ-9 data posttreatment or follow-up. The baseline characteristics of the participants and their respective tests for differences are displayed in Table 1. No pretreatment differences were observed between the groups in terms of primary outcome or participant characteristics (Ps>.05), with one exception (concurrent psychotherapy: χ²₃=11.19; P=.01).

Group Differences in PHQ-9 Change From Pre- to Posttreatment

The Kruskal-Wallis chi-square test indicated no main effect for participants’ messages (H₁=0.18, P=.67). However, there was a significant main effect for adherence (H₁=5.10, P=.02). The Kruskal Wallis chi-square test revealed a significant effect of the 4 groups on PHQ-9 change from pre- to posttreatment (H₃=8.26, P=.04). Post hoc pairwise comparisons revealed a significant difference between groups 3 and 4 (Z=–2.84, P=.002), with larger symptom improvement for participants with high adherence. The mean PHQ-9 change for each of the 4 groups is shown in Table 2 (posttreatment).

Group Differences in PHQ-9 Change From Pretreatment to Follow-Up

Similar to the results from pre- to posttreatment, there was no main effect for participants’ messages from pretreatment to follow-up (H₁=0.15, P=.69). Again, there was a significant main effect for adherence (H₁=7.18, P=.007). The Kruskal-Wallis chi-square test revealed a significant effect of the 4 groups on PHQ-9 change from pretreatment to follow-up (H₃=8.90, P=.03). Post hoc pairwise comparisons revealed a significant difference between the groups 3 and 2 (Z=–2.47, P=.007) and group 3 and 4 (Z=–2.62, P=.004), with larger symptom improvement for participants with high adherence. All other group comparisons did not yield significant differences. The mean PHQ-9 change for each of the 4 groups is shown in Table 3 (follow-up).

Differences in PHQ-9 Change for Concurrent Psychotherapy

The baseline evaluation showed that the number of participants in concurrent psychotherapy was unequally distributed across the 4 groups. Participants who wrote many messages to their guiding therapist were more likely to see a psychotherapist outside the study (group 3=52% and group 4=35%) than participants who wrote fewer messages to their guiding therapist (group 1=12.5% and group 2=26.5%). Based on this result, we further explored whether there were indicative differences in symptom changes within the four groups regarding whether participants were in concurrent psychotherapy. In most cases, the differences were negligible (range 0.06‐1.18 points on PHQ-9). However, there was one notable exception, that is, the difference in group 3 from pretreatment to follow-up. Within this group, participants in concurrent psychotherapy showed an improvement of 1.78 points on the PHQ-9, while those not in concurrent psychotherapy experienced a deterioration of 2.22 points.

Discussion

Principal Findings

This exploratory paper investigated whether adherence and contact behavior with the therapist are associated with the change in depressive symptoms. As noted in a previous study, high-adherent participants in our sample benefited more from the self-help program than low-adherent participants [3]. This was true for the improvement from pre- to posttreatment and from pretreatment to follow-up. For the difference in contact behavior, that is, participants who had written few or many messages, there was no overall difference.

However, looking more closely at the 4 groups, it was found that low-adherent participants writing many messages (group 3) showed a significantly smaller symptom improvement than group 4 from pre- to posttreatment. Furthermore, group 3 did not show a symptom improvement from pretreatment to follow-up and was significantly inferior compared with groups 2 and 4. These results suggest that low-adherent participants who write many messages may primarily seek contact with a therapist instead of trying to help themselves with a self-help program. It would not be surprising if some participants need more interpersonal support than is provided by guidance, be it because they do not have enough personal responsibility or resources to work on a program or because their motivation to change is too low still. For these participants, guided self-help interventions may not be a sufficient treatment. Thus, treatment providers might react to low-adherent and contact-seeking participants, for example, by providing extra support with telephone or face-to-face contact. Alternatively, they might refer them to other forms of treatment (eg, face-to-face psychotherapy). Apart from a need for contact, another reason for seeking much contact might be that these participants have a higher need for self-reflection and self-expression. If that were the case, treatment providers might use additional interventions to address that need, such as expressive writing tasks. To prevent participants not getting the contact they wish for, it would be possible to clarify more precisely what the idea of guidance is and how much contact they can expect. Some participants may decide against guided self-help and look for another treatment that suits their needs better.

The question of whether participants seek increased contact because they are not benefiting from the therapy, or conversely, whether increased contact hinders participants from benefiting remains unresolved and requires further investigation in future studies. Nevertheless, the behavior of group 3 may potentially serve as an indicator for treatment providers to refer participants to other treatments (eg, face-to-face psychotherapy).

Participants writing many messages seemed to be in concurrent psychotherapy more often than participants writing few, that is, 52% and 35% (groups 3 and 4), and 12.5% and 26.5% (groups 1 and 2), respectively. Interestingly, it was also found that in group 3, there was a striking difference in long-term symptom change depending on whether the participants saw a psychotherapist outside the study. Those with additional psychotherapy improved, whereas those without additional psychotherapy deteriorated. Similar to the result of the previous section, this result could indicate that guided self-help is less suitable for some participants because they need more contact or support than they receive through guidance. The participants without additional psychotherapy probably could not fully satisfy their interpersonal needs during the study, whereas participants with additional psychotherapy were probably able to fulfill them.

Our exploratory study suggests that low-adherent participants writing many messages could belong to a subgroup that does not benefit from a self-help program. These participants might need more support and contact, which is not satisfied by weekly guidance via email. Since this is an exploratory study, our findings and assumptions should be treated with caution and rather be used to systematically investigate the topic in future research.

Strengths and Limitations

To the best of our knowledge, this is the first study to examine the interplay between program adherence and therapist contact regarding symptom improvement. It provides initial evidence that low-adherent participants seeking much contact may not benefit from guided self-help. Important limitations are the relatively small sample size and the exploratory nature of the study. Therefore, the results should be cautiously interpreted and primarily be used to build hypotheses for future studies. Furthermore, since it was exploratory, we did not correct for type I error in multiple comparisons in this study. Another limitation of this study is that we did not investigate the qualitative content of the messages regarding outcome. This might be an interesting subject for future research. There is some evidence that content of messages relates to outcome and module completion [15].

Conclusions

This exploratory study suggests that low-adherent participants who write many messages during a guided self-help program may not benefit as much from the intervention. These individuals could have a higher need for interpersonal support, which is not met through minimal therapist guidance alone. Future research should investigate whether these participants would benefit more from alternative treatments, such as face-to-face psychotherapy, or additional interventions to address their specific needs. While our findings are preliminary, they may indicate the importance of considering participant behavior and adherence when tailoring treatment approaches.