The delivery of psychological therapies via the internet has received a good deal of research attention in recent years. Multiple randomized controlled trials (RCTs) have shown that CBT-based internet interventions can be effective in treating depression and anxiety disorders when compared with no treatment [1-3], with the effects being larger when patients’ online work is guided and supported by a clinician or coach [4-6]. Some guided internet treatments have also shown results comparable to face-to-face treatment in RCTs, despite requiring much less clinician time [1,7-9]. These findings suggest that internet therapies have substantial potential for expanding access to effective psychological treatments, which is critical given the undertreatment of common mental health problems such as social anxiety in almost all countries [10]. However, this potential will only be realized if the results that are obtained by the developers of these internet interventions in tightly controlled RCTs can be maintained when those interventions are made available in other settings, providing treatment routinely and with guidance from clinicians who were not involved in developing the program. Whether this is possible will depend critically on the extent to which the field can develop efficient and effective ways of training clinicians to support and guide patients going through these interventions. Large-scale dissemination of therapist-guided interventions will only be achievable if effective therapist training methods are in place.

Promisingly, several studies have attempted to disseminate internet therapies beyond the teams that have developed the interventions and have reported promising results. These include the translation and trialing of Swedish internet-based Cognitive Behavioral Therapy interventions in Romania [11] and Norway [12], a Swiss intervention implemented in China [13], and a Spanish intervention trialed in the Netherlands [14]. However, none of these studies have provided sufficient detail about how the therapists or coaches in the dissemination sites were trained and how the effectiveness of the training was evaluated. There is a clear need for such descriptions if the field is to reliably succeed with dissemination. This study aimed to address this by describing in detail an example of how clinicians in Hong Kong were trained to deliver an effective internet therapy for social anxiety disorder (SAD) that was developed in the United Kingdom and how this training was evaluated.

The internet-based Cognitive Therapy for social anxiety disorder (iCT-SAD) program is based on Clark and Wells’ face-to-face Cognitive Therapy for SAD (CT-SAD). CT-SAD has strong empirical support for the treatment of SAD, showing efficacy superior to that of various alternative treatments in RCTs [15-18] and a network meta-analysis [19]. As a result, the UK National Institute for Health and Care Excellence recommends CT-SAD as a first-line treatment for adults with SAD [20]. The iCT-SAD program implements all the procedures of face-to-face CT-SAD online, including special features such as video feedback of social performance and behavioral experiments that allow patients to test out their feared concerns. It includes multiple brief video clips illustrating key assignments that patients are encouraged to do to challenge their fearful beliefs, as modeling has been shown to be one of the most effective ways of reducing anxiety [21]. The treatment is delivered through a series of online modules, which include core modules given to all patients, and a range of optional modules targeting different specific concerns (eg, blushing, feeling boring, worrying in advance). These allow the therapist to tailor the treatment to patients’ individual needs. As most of the therapy content is delivered by the program, iCT-SAD greatly reduces the amount of therapist time required per patient compared with face-to-face CT-SAD [22]. The aim of iCT-SAD is that most of the key learning that occurs in the treatment can be achieved by the patient’s self-study on the internet, facilitated by therapists who are themselves skilled in delivering the face-to-face treatment. This online facilitation draws strongly on the clinical knowledge expected of an experienced clinician but asks them to apply this in a very different way. The program introduces the key concepts that would normally be introduced by the therapist and guides much of the patient’s learning. Therefore, the therapist must carefully monitor patients’ progress within the program, looking closely at what the patient has and has not already learnt, and then make clinically informed suggestions to help them deepen and extend their learning. For example, they may support the patient to plan new behavioral experiments or direct them to other sections of the program that are most appropriate for their particular concerns. Therefore, it may be particularly critical to pay close attention to how therapists are trained to deliver an internet intervention of this sort, given that the therapist’s role, and the application of their clinical knowledge, is different compared with face-to-face interventions.

As this study involved dissemination to a different culture as well as to a different team, it also had to consider issues about the extent of cultural adaptation of the treatment that would be required. One approach to this problem would be to assume that if the treatment is delivered in a different culture it has to be adapted in ways beyond linguistic translation alone and that this must be done before any evaluation of the treatment in the new context. An alternative approach is to first pilot the intervention in its unadapted form to benchmark it against results from studies in its country of origin and, at the same time, to obtain feedback from clients and therapists about what adaptations they think might be needed. This latter approach, which aims to identify the minimum number of adaptations required, was taken for this study.

This study therefore aimed to describe and evaluate a program of therapist training for iCT-SAD. As this treatment was developed in the United Kingdom and this study was conducted in Hong Kong with local therapists, an additional objective was to examine the feasibility, acceptability, and initial efficacy of iCT-SAD in this cultural context, based on data from a pilot case series. We aimed to identify whether significant cultural adaptations would be required for further dissemination in this setting.

The training consisted of 3 phases, outlined below:

The training was evaluated using the following 4 methods:

Mean CTCS-SP scores across the 15 face-to-face CT skill areas were 3.9, 3.9, and 4.0 across the 3 therapists. These scores indicated that all 3 therapists achieved a level of competence in the delivery of the face-to-face treatment that would be expected of a high-intensity IAPT therapist (mean score >3).

Therapists’ self-reported knowledge and skills ratings across the 3 timepoints are shown in Figure 1. This figure shows that at baseline, where therapists had been trained in the face-to-face CT protocol but not iCT, self-reported iCT knowledge and skills ratings were low. These mean scores increased following the iCT training workshops in Phase 2 and further increased following completion of iCT practice cases under supervision in Phase 3.

The 3 therapists’ scores on the different components of the skills test are shown in Table 1. As this is a newly developed instrument, there are no established competency cutoffs. However, as the test was designed in a similar manner to the CTCS-SP, with each item examining competence in a distinct skill, the CTCS-SP competency threshold of 50% is thought to be broadly applicable to the skills test and is used here to aid interpretation.

In this skills test, the 3 therapists achieved overall scores of 71%, 60%, and 64%, indicating good proficiency in the skills assessed. Benchmarking against the CTCS-SP, these scores therefore indicate a level of competency well above a 50% minimum standard. It should be noted that this assessment was completed following Phase 2 of the training, before the therapists’ pilot cases; this means they had further opportunities to practice and consolidate their skills in Phase 3.

All patients showed excellent adherence. Each patient completed all of the core treatment modules. Overall, patients were issued with a mean of 13.7 modules (SD 1.4) and completed a mean of 12.8 modules (SD 1.7). They spent an average of 40.2 hours (SD 26.4) using the website in the 14-week treatment period. As the program includes an automatic logout feature following periods of inactivity, the activity recorded on the site is thought to be a fair reflection of the time spent actively using the program.

Mean scores on all outcome measures are shown in Table 2 and effect sizes in Table 3. On the primary patient outcome measure, the LSAS, the mean decrease across the weekly treatment period was 53.8 points (SD 39.5). The within-group effect size (Cohen d) was 2.06 (95% CI 0.66-3.46), indicating an effect of medium to very large magnitude in the direction of reduced social anxiety. These figures are at least as good as data from the developers of the treatment [22], where the mean pre-post LSAS decrease was 40.2 points and the effect size was 1.64. Effect size point estimates for the present secondary outcome measures were also large, ranging from 0.92 to 1.89, with the lower limit of the CIs falling in the small-to-medium range for most, though not all, measures. Although highly tentative because of the sample size, these findings suggest the treatment was effective in reducing social anxiety, negative social cognitions, safety behaviors, worry, and functional impairment, and increasing social participation, but did not show a reliable effect on depressed mood or social satisfaction.

Changes in scores from posttreatment to 3-month follow-up were analyzed to review the maintenance of treatment gains over this period. Patients retained access to the program during this 3-month phase, though therapist contact was reduced to just one brief phone call at monthly intervals. As shown in Table 3, the effect size point estimates for most measures ranged between −0.20 and 0.20, meaning the threshold for a small effect in either direction was not reached. This suggested that the improvements from treatment had been maintained over this period. There was some evidence of a small effect in the direction of further improvement on the LSAS, SPS, and social participation measures during this period. On the SBQ, SAQ, and PHQ, there was some evidence of a small effect indicating a deterioration. Although this may indicate a lessening of some secondary treatment effects during the booster phase, these results may have been heavily influenced by the scores of 1 patient who did not respond to treatment and showed a decline on secondary outcome measures over the booster phase. Again, the CIs for these effect sizes were extremely wide because of sample size, so they can only provide a preliminary indication of the maintenance of treatment gains.

Overall, 5 of the 6 patients showed an improvement on the LSAS greater than 31% at the posttreatment assessment and were thus classified as treatment responders. All 5 patients were also classified as remitted from their social anxiety, showing a drop of at least 12 points and falling below the clinical cutoff at posttreatment. Individual LSAS scores over time are shown in Figure 2. Using the IAPT criteria based on social anxiety (SPIN) and depression (PHQ) scores, 4 patients showed reliable improvement and 3 were classified as reliably recovered from SAD.

The 1 patient who was not classified as a responder to treatment within the weekly treatment phase experienced difficulties with depressed mood, which may have impacted their motivation to log in and work on the treatment website and to complete behavioral experiments. When compared with some other patients, they spent relatively little time on the website (12.5 hours total) and completed fewer behavioral experiments. Across the patient cohort, the amount of time spent on the website was positively correlated with percentage change on the LSAS (r=.584). It is also noted that the 2 patients with relatively lower baseline LSAS scores, because of more situation-specific forms of social anxiety, finished treatment with scores higher than some other patients whose baseline scores were more severe. The correlation between baseline LSAS score and percentage change on the LSAS was r=.443.

All 6 patients completed the anonymous feedback survey at the end of the booster period. Given the small sample size, the frequencies of responses are reported. Overall, 5 of the patients felt that the program was relatively easy to understand in English, though 1 experienced some difficulties expressing their ideas and feelings and felt this could be a barrier for the therapy (respondent 4). Patients reported various therapist behaviors that they found helpful, including sending a written summary following phone calls, helping to generate new perspectives, identifying details of modules they may have missed, advising on behavioral experiments to try, and helping them to review their week. In terms of the treatment as a whole, respondents reported liking the phone calls (described as “intensive but helpful” by respondent 1); the practical nature of the contents; the effort, help, and professionalism of the therapist; the inclusion of both general and specific content; and the way the questionnaires allowed them to review their emotional state. In addition, 4 patients did not report any dislikes about the treatment. For security reasons, iCT-SAD uses 2-factor authentication at log-in, and 1 participant disliked having to enter an authentication code each time (respondent 1). One participant felt it was sometimes hard to motivate themselves to finish the tasks and expressed a concern that they may not have understood their own anxiety very well (respondent 4). All patients rated the program as easy to use on a desktop or laptop computer, but 2 reported difficulty when using a tablet or mobile phone. In sum, responses indicated a good level of treatment satisfaction and acceptability, with some but relatively few technical or practical difficulties.

Therapists’ feedback indicated that they did not think any major cultural adaptation was required to the program content or implementation procedures for the Hong Kong context. However, the therapists did identify certain aspects of the iCT-SAD program that may prove difficult to understand when patients are working in a nonnative language. This included some video and audio material where groups of people are chatting informally, perhaps talking at the same time or in incomplete sentences. In addition, one of the exercises used in the program to practice externally focused attention asks patients to watch a video of someone reading a story to them while they alternate their attention between focusing on themselves and on the story content.

This study has provided an example of a procedure for training therapists to deliver a guided internet intervention, including details of the content covered, and how progress was evaluated. The results indicated that the training was effective in improving therapists’ knowledge and skills in relation to iCT-SAD. The 3 therapists achieved a good level of competence when objectively measured using the skills test following Phase 2 of training. They were then able to consolidate these skills in their work with pilot cases, as demonstrated by the positive outcomes achieved by this cohort. These results suggested that the clinical outcomes for patients in Hong Kong undertaking iCT-SAD with local therapists were comparable to those seen in a UK study from the developers of the treatment [22].

The 6 patients showed good adherence to the iCT-SAD program, completing treatment modules, phone calls with the therapist, behavioral experiments, and weekly questionnaire measures. All patients completed the full weekly and booster phases of treatment, indicating a good level of treatment acceptability and feasibility in this setting and sufficiently reliable functioning of the website. The therapists successfully implemented the treatment in line with the protocol and no significant deviations from this were required. These findings are of course preliminary given the present sample size and an RCT would be required to establish the feasibility and efficacy of the program in Hong Kong more robustly.

This study took the approach of piloting the intervention in its unadapted form, meaning that the results could be benchmarked against the UK data and feedback could be elicited from both patients and therapists. Promisingly, the mean pre-post change on the LSAS and associated effect size point estimate (Cohen d=2.06) were large, suggesting that the high level of efficacy observed in the United Kingdom appears to have been maintained within a different cultural setting. Patient and therapist feedback did not identify any major cultural or linguistic barriers; the suggestions made to assist understanding were modest. We have since created some additional Chinese resources for the components of the program mentioned along with English-subtitled versions of the site videos. Overall, the findings in this study suggest that the approach of starting with the unadapted intervention was justified given the clinical outcomes obtained and that the adaptations suggested were quite modest and smaller than might have been expected at the outset of the project.

Limitations of this study include the small number of therapists trained and the fact that the self-report assessment and skills test were developed specifically for this study and are therefore not yet externally validated. As mentioned above, the small number of patients in this pilot study can only tentatively indicate treatment efficacy and linguistic or cultural considerations, and larger controlled studies are therefore required. The training program implemented here was intensive and was not designed to be scalable at this stage. Future work should consider the best methods to provide such training in less time and to larger groups of therapists. In developing training programs for internet interventions that are based closely on a face-to-face treatment, our view is that familiarity with the face-to-face protocol is an important preliminary step and that treating a small number of training cases online under clinical supervision is essential for applying and consolidating skills. Online therapist training methods are likely to be the most scalable approach for the future, but preliminary work such as the approach taken here can help to develop a clearer understanding of what training should cover and how. The aim should be to determine what therapists need from training, making it easier to design online materials for that purpose. Our research team has recently started to develop online therapist training materials for iCT-SAD, which are available on our resources website [30]. Further research could also usefully explore whether less-experienced therapists can be trained to achieve similar outcomes.

Conclusions

This study has described and evaluated a program of therapist training in iCT-SAD, which appeared effective at increasing therapists’ knowledge and skills and resulted in positive clinical outcomes among a pilot case series of 6 patients in Hong Kong, with initial evidence that these were sustained at 3-month follow-up. The treatment appeared feasible and acceptable in this setting. An RCT of the iCT-SAD treatment in Hong Kong is now in progress. Given that therapist training is critical to the successful dissemination of internet interventions, further research to develop, describe, and evaluate therapist training procedures is recommended.