Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Monday, March 11, 2019 at 4:00 PM to 4:30 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 15.05.19 in Vol 3, No 2 (2019): Apr-Jun

Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/13446, first published Jan 20, 2019.

This paper is in the following e-collection/theme issue:

    Original Paper

    Internet-Based Cognitive Therapy for Social Anxiety Disorder in Hong Kong: Therapist Training and Dissemination Case Series

    1Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom

    2Oxford University Hospitals National Health Service Foundation Trust, Oxford, United Kingdom

    3Oxford Health National Health Service Foundation Trust, Oxford, United Kingdom

    4New Life Psychiatric Rehabilitation Association, Hong Kong, China (Hong Kong)

    5New Territories East Cluster, Hospital Authority, Hong Kong, China (Hong Kong)

    6Kowloon Central Cluster, Hospital Authority, Hong Kong, China (Hong Kong)

    7Kings College London, London, United Kingdom

    8The Chinese University of Hong Kong, Hong Kong, China (Hong Kong)

    Corresponding Author:

    Graham R Thew, DClinPsy, DPhil

    Department of Experimental Psychology

    University of Oxford

    The Old Rectory

    Paradise Square

    Oxford, OX1 1TW

    United Kingdom

    Phone: 44 01865 618600

    Email:


    ABSTRACT

    Background: Guided internet-based psychological interventions show substantial promise for expanding access to evidence-based mental health care. However, this can only be achieved if results of tightly controlled studies from the treatment developers can also be achieved in other independent settings. This dissemination depends critically on developing efficient and effective ways to train professionals to deliver these interventions. Unfortunately, descriptions of therapist training and its evaluation are often limited or absent within dissemination studies.

    Objective: This study aimed to describe and evaluate a program of therapist training to deliver internet-based Cognitive Therapy for social anxiety disorder (iCT-SAD). As this treatment was developed in the United Kingdom and this study was conducted in Hong Kong with local therapists, an additional objective was to examine the feasibility, acceptability, and initial efficacy of iCT-SAD in this cultural context, based on data from a pilot case series.

    Methods: Training in iCT-SAD was provided to 3 therapists and included practice of the face-to-face format of therapy under clinical supervision, training workshops, and treating 6 patients with the iCT-SAD program. Training progress was evaluated using standardized and self-report measures and by reviewing patient outcomes. In addition, feedback from patients and therapists was sought regarding the feasibility and acceptability of the program.

    Results: The training program was effective at increasing therapists’ iCT-SAD knowledge and skills, resulting in levels of competence expected of a specialist Cognitive Behavioral Therapy practitioner. The 6 patients treated by the trainees all completed their treatment and achieved a mean pre- to posttreatment change of 53.8 points (SD 39.5) on the primary patient outcome measure, the Liebowitz Social Anxiety Scale. The within-group effect size (Cohen d) was 2.06 (95% CI 0.66-3.46). There was evidence to suggest that the patients’ clinical outcomes were sustained at 3-month follow-up. These clinical results are comparable to those achieved by UK patients treated by the developers of the internet program. Patient and therapist feedback did not identify any major cultural barriers to implementing iCT-SAD in Hong Kong; some modest language suggestions were made to assist understanding.

    Conclusions: The therapist training implemented here facilitated the successful dissemination of an effective UK-developed internet intervention to Hong Kong. The treatment appeared feasible and acceptable in this setting and showed highly promising initial efficacy. A randomized controlled trial is now required to examine this more robustly. As therapist training is critical to the successful dissemination of internet interventions, further research to develop, describe, and evaluate therapist training procedures is recommended.

    JMIR Form Res 2019;3(2):e13446

    doi:10.2196/13446

    KEYWORDS



    Introduction

    Background

    The delivery of psychological therapies via the internet has received a good deal of research attention in recent years. Multiple randomized controlled trials (RCTs) have shown that CBT-based internet interventions can be effective in treating depression and anxiety disorders when compared with no treatment [1-3], with the effects being larger when patients’ online work is guided and supported by a clinician or coach [4-6]. Some guided internet treatments have also shown results comparable to face-to-face treatment in RCTs, despite requiring much less clinician time [1,7-9]. These findings suggest that internet therapies have substantial potential for expanding access to effective psychological treatments, which is critical given the undertreatment of common mental health problems such as social anxiety in almost all countries [10]. However, this potential will only be realized if the results that are obtained by the developers of these internet interventions in tightly controlled RCTs can be maintained when those interventions are made available in other settings, providing treatment routinely and with guidance from clinicians who were not involved in developing the program. Whether this is possible will depend critically on the extent to which the field can develop efficient and effective ways of training clinicians to support and guide patients going through these interventions. Large-scale dissemination of therapist-guided interventions will only be achievable if effective therapist training methods are in place.

    Promisingly, several studies have attempted to disseminate internet therapies beyond the teams that have developed the interventions and have reported promising results. These include the translation and trialing of Swedish internet-based Cognitive Behavioral Therapy interventions in Romania [11] and Norway [12], a Swiss intervention implemented in China [13], and a Spanish intervention trialed in the Netherlands [14]. However, none of these studies have provided sufficient detail about how the therapists or coaches in the dissemination sites were trained and how the effectiveness of the training was evaluated. There is a clear need for such descriptions if the field is to reliably succeed with dissemination. This study aimed to address this by describing in detail an example of how clinicians in Hong Kong were trained to deliver an effective internet therapy for social anxiety disorder (SAD) that was developed in the United Kingdom and how this training was evaluated.

    The internet-based Cognitive Therapy for social anxiety disorder (iCT-SAD) program is based on Clark and Wells’ face-to-face Cognitive Therapy for SAD (CT-SAD). CT-SAD has strong empirical support for the treatment of SAD, showing efficacy superior to that of various alternative treatments in RCTs [15-18] and a network meta-analysis [19]. As a result, the UK National Institute for Health and Care Excellence recommends CT-SAD as a first-line treatment for adults with SAD [20]. The iCT-SAD program implements all the procedures of face-to-face CT-SAD online, including special features such as video feedback of social performance and behavioral experiments that allow patients to test out their feared concerns. It includes multiple brief video clips illustrating key assignments that patients are encouraged to do to challenge their fearful beliefs, as modeling has been shown to be one of the most effective ways of reducing anxiety [21]. The treatment is delivered through a series of online modules, which include core modules given to all patients, and a range of optional modules targeting different specific concerns (eg, blushing, feeling boring, worrying in advance). These allow the therapist to tailor the treatment to patients’ individual needs. As most of the therapy content is delivered by the program, iCT-SAD greatly reduces the amount of therapist time required per patient compared with face-to-face CT-SAD [22]. The aim of iCT-SAD is that most of the key learning that occurs in the treatment can be achieved by the patient’s self-study on the internet, facilitated by therapists who are themselves skilled in delivering the face-to-face treatment. This online facilitation draws strongly on the clinical knowledge expected of an experienced clinician but asks them to apply this in a very different way. The program introduces the key concepts that would normally be introduced by the therapist and guides much of the patient’s learning. Therefore, the therapist must carefully monitor patients’ progress within the program, looking closely at what the patient has and has not already learnt, and then make clinically informed suggestions to help them deepen and extend their learning. For example, they may support the patient to plan new behavioral experiments or direct them to other sections of the program that are most appropriate for their particular concerns. Therefore, it may be particularly critical to pay close attention to how therapists are trained to deliver an internet intervention of this sort, given that the therapist’s role, and the application of their clinical knowledge, is different compared with face-to-face interventions.

    As this study involved dissemination to a different culture as well as to a different team, it also had to consider issues about the extent of cultural adaptation of the treatment that would be required. One approach to this problem would be to assume that if the treatment is delivered in a different culture it has to be adapted in ways beyond linguistic translation alone and that this must be done before any evaluation of the treatment in the new context. An alternative approach is to first pilot the intervention in its unadapted form to benchmark it against results from studies in its country of origin and, at the same time, to obtain feedback from clients and therapists about what adaptations they think might be needed. This latter approach, which aims to identify the minimum number of adaptations required, was taken for this study.

    Objectives

    This study therefore aimed to describe and evaluate a program of therapist training for iCT-SAD. As this treatment was developed in the United Kingdom and this study was conducted in Hong Kong with local therapists, an additional objective was to examine the feasibility, acceptability, and initial efficacy of iCT-SAD in this cultural context, based on data from a pilot case series. We aimed to identify whether significant cultural adaptations would be required for further dissemination in this setting.


    Methods

    Therapist Training Procedure

    The training consisted of 3 phases, outlined below:

    Phase 1: Initial Face-To-Face Cases Under Supervision

    As iCT-SAD is a direct adaptation of the face-to-face Cognitive Therapy (CT) protocol, it is important that therapists are familiar with CT before learning internet-based Cognitive Therapy (iCT). This permits a greater understanding of the key points the online modules are aiming to convey.

    Training for this study therefore started with 3 therapists from Hong Kong undertaking a phase of face-to-face CT-SAD. The therapists had been qualified as clinical psychologists for an average of 10.3 years (7, 9, and 15 years). All spoke English as an additional language. Although they were experienced in delivering cognitive-behavioral interventions for anxiety, including SAD, and 2 of them had previously attended a CT-SAD workshop, none of them had used the full CT protocol previously. The therapists were sent the most recent version of the treatment manual [23], which explains the CT-SAD protocol as applied to adults as well as how treatment is adapted for adolescents. The 3 therapists then implemented the treatment in Cantonese with a total of 5 cases between January and May 2017, with weekly group supervision from the first author via Skype, which focused on learning and implementing the treatment protocol.

    Phase 2: Internet-Based Cognitive Therapy for Social Anxiety Disorder Training

    The second phase consisted of training in the iCT-SAD program and treatment protocol. The iCT-SAD training was developed and delivered by the authors (GT, DMC, and JW) and was informed by the results of interviews with 3 iCT-SAD therapists from the United Kingdom who were involved in the development of the treatment as well as the first author’s own experience of learning and delivering iCT-SAD. Training was delivered in an intensive format, totaling 7.5 days over a 2-week period. Each therapist was registered with an account on the iCT-SAD site, together with a test patient account to allow them to view the site content from the patient’s perspective and to practice using the site within the training sessions. The training was divided into the following sections:

    Site Navigation and Functionality

    Learning to navigate the site and use the site functions, such as releasing treatment modules, sending messages, and using the resource library.

    Structure and Timings

    Learning how the iCT-SAD treatment protocol is implemented over the treatment period, for example, when each core module is released and when to schedule phone calls.

    Module Content

    The majority of the training focused on reviewing the content and general aims of each module to help the therapists know how to review and discuss modules with patients.

    Therapist Communication

    Effective use of the iCT-SAD communication methods (messaging, short message service (SMS), and phone calls) was demonstrated and practiced by the therapists using role play.

    Implementing Key Techniques

    As some of the key therapy techniques from CT, such as developing an individualized model, and the self-focused attention and safety behaviors experiment are implemented slightly differently online, time was allocated to review and practice these specifically.

    Behavioral Experiments

    The training included experiential practice of planning and completing behavioral experiments in real social situations. The aim was to support the therapists to generate ideas for appropriate experiments and to provide them with first-hand experience to draw on when discussing experiments in iCT-SAD.

    The Role of the Therapist in Internet-Based Cognitive Therapy for Social Anxiety Disorder

    Training focused on understanding the role of the therapist in iCT-SAD as a guide and facilitator, how they apply their clinical expertise, and how this differs from face-to-face work.

    Troubleshooting

    Common problems, such as patients not logging into the site, low motivation, or avoidance of behavioral experiments, were discussed and possible solutions reviewed.

    Phase 3: Internet-Based Cognitive Therapy for Social Anxiety Disorder Pilot Cases Under Supervision

    Phase 3 consisted of piloting the iCT-SAD treatment with a small number of cases under weekly group clinical supervision from the first author via Skype. Each therapist treated 2 clients from a local clinical service in Hong Kong using the program. Supervision focused on reinforcing learning from Phase 2, adherence to the treatment protocol, and further development of skills. The therapist training and a subsequent clinical trial were approved by the Joint Chinese University of Hong Kong—New Territories East Cluster Clinical Research Ethics Committee (Ref: 2016.611-T) and by the University of Oxford Tropical Research Ethics Committee (Ref: 531-17).

    Training Evaluation

    The training was evaluated using the following 4 methods:

    Face-to-Face Cognitive Therapy for Social Anxiety Disorder: Cognitive Therapy Competence Scale for Social Phobia

    The Cognitive Therapy Competence Scale for Social Phobia (CTCS-SP) [24] (also available in an unpublished manuscript by Clark et al) is a disorder-specific adaptation of the general Cognitive Therapy Scale—Revised (CTS-R) [25], allowing the assessor to rate skills and techniques specific to the face-to-face CT-SAD treatment protocol. This competence measure has been shown to be a strong predictor of patient outcomes [26]. The CTCS-SP contains 15 skill areas, each rated on a 0 to 6 scale, with higher scores indicating greater competence. A mean score of 3 (ie, 50%) is considered the threshold to demonstrate competence in delivering the treatment. During Phase 1 (face-to-face CT training), 1 complete session from each therapist was rated by the first author using a video of the session (in Cantonese) together with an English transcript of the conversation. Sessions were chosen by each therapist, but could not include the first or last sessions, and were required to include at least one behavioral experiment. Feedback from these assessments was provided in a subsequent supervision session.

    Internet-Based Cognitive Therapy for Social Anxiety Disorder Self-Evaluation Assessment

    This self-report assessment was developed specifically for this study. It asks respondents to rate their own knowledge and skills in relation to 34 internet therapy components and activities, including those applicable to most online interventions (eg, “Developing and maintaining client engagement with the online programme”) and those more specific to iCT-SAD (eg, “Suggesting strategies to overcome problems clients may experience with video feedback”). An initial item pool was generated through interviews with 3 experienced UK iCT-SAD trial therapists and their supervisor, which was then reviewed and refined in consultation with these experts to ensure face validity. For each item, respondents provide 1 rating for knowledge (ie, how much they know about the item) and 1 for skills (ie, how well they think they can currently implement it). Ratings are given on 0 to 8 Likert scales, where 0, 2, 4, 6, and 8 represent no, limited, some, good, and extensive knowledge or skills, respectively. Mean scores for each subscale are calculated. The full assessment is provided in Multimedia Appendix 1. The therapists completed the assessment following each phase of training.

    Internet-Based Cognitive Therapy for Social Anxiety Disorder Skills Test

    To obtain a more objective assessment of what the therapists had learned following Phase 2, a standardized skills test was developed and completed by the therapists at this timepoint. The test consisted of brief information about a series of fictional patients undertaking iCT-SAD. For each, an extract of some completed work on the site is shown and the respondent is asked questions about what the patient has written and how they would proceed with the treatment. For example, questions might ask how they would help a patient to improve their individualized cognitive model or to write a response to a message where the patient reports a drop in mood (The test is provided in Multimedia Appendix 2). Items were generated following the same iterative process of consultation with iCT-SAD experts as used for the self-evaluation assessment. Respondents’ answers are scored in relation to set of exemplar responses generated by iCT-SAD experts. Exemplar responses were divided into distinct components, with a point awarded for each one mentioned by the respondent. The maximum score was 55.

    Internet-Based Cognitive Therapy for Social Anxiety Disorder Patient Outcomes

    The final method to evaluate the training effectiveness was to examine the outcomes of the pilot patient cohort being treated by the therapists using the program. This study also provided an initial test of the iCT-SAD program in Hong Kong, examining its feasibility, acceptability, and technical functioning, while also offering a tentative indication of efficacy and cultural applicability.

    Internet-Based Cognitive Therapy for Social Anxiety Disorder Pilot Case Series

    Participants

    A cohort of 6 patients (3 female; mean age 31.3 years, age range 18 to 49 years) was recruited from the New Life Psychiatric Rehabilitation Association, Hong Kong. The cohort represented a consecutive series of referrals for probable social anxiety. All the patients met the Diagnostic and Statistical Manual 4th Edition (DSM-IV) criteria for SAD, which was confirmed at assessment using the Anxiety Disorders Interview Schedule for DSM-IV [27]. Within the cohort, 4 patients showed generalized social anxiety, whereas 2 experienced anxiety in a more restricted range of situations. The mean duration of social anxiety was 7.5 years (SD 8.8). Comorbidities were assessed using the Structured Clinical Interview for DSM-5 [28]. SAD was the primary diagnosis for all 6 patients, but 1 met the criteria for comorbid Generalized Anxiety Disorder (GAD), 1 for comorbid major depressive disorder, and 1 for comorbid GAD, other specified depressive disorder, panic disorder, agoraphobia, specific phobia, and separation anxiety. The remaining 3 only met the criteria for SAD. Assessments were conducted by a trainee clinical psychologist under supervision. All patients were Chinese residents of Hong Kong, with non-native but sufficient proficiency in English to undertake the treatment in its original language. All participants were in full-time employment or study. Regarding marital status, 2 participants were married or cohabiting and 4 were single.

    Procedure

    The iCT-SAD program, which was written in English, was implemented as described by Stott et al [22]. Therapists communicated with their patients using brief secure messages, telephone calls, and occasional webcam chats. Therapists’ messages were written in English, and the telephone calls were conducted in both English and Cantonese as required. The treatment lasted 14 weeks, followed by a 3-month booster period, during which participants retained access to the program. At the end of treatment, participants were invited to complete a brief online survey about their experience of treatment and their suggestions for how it could be improved. Questions examined the ease of understanding the treatment in English, helpfulness of therapists’ behaviors, overall likes and dislikes, and general ease of use. The respondents were not asked to give their name and completed the survey outside of the treatment website, meaning the therapists could not see their responses. The therapists were also invited to share their feedback and suggestions during a focus group discussion within clinical supervision. This centered on the cultural applicability of the treatment and the need for any adaptation in future.

    Measures

    The primary patient outcome measure was the self-report version of the Liebowitz Social Anxiety Scale (LSAS) [29]. Secondary patient outcome measures included other measures of SAD to facilitate comparison with other studies, process measures, and assessments of general mood and functioning:

    Social Anxiety:

    • Social Phobia Weekly Summary Scale [30]
    • Social Phobia Inventory (SPIN) [30,31]
    • Fear of Negative Evaluation Scale [30,32].
    • Social Phobia Scale (SPS) and Social Interaction Anxiety Scale [30,33].

    Social anxiety process measures:

    • Social Cognitions Questionnaire [30]
    • Social Behavior Questionnaire (SBQ) [30]
    • Social Attitudes Questionnaire (SAQ) [30]
    • Social Participation and Social Satisfaction scales [34]

    General Mood and Functioning Measures:

    • Patient Health Questionnaire (PHQ)—9-item version [35]
    • Generalized Anxiety Disorder Questionnaire (GAD)—7-item version [36]
    • Work and Social Adjustment Scale [37]
    Analysis of Clinical Data

    Mean scores on each measure at pretreatment, posttreatment, and 3-month follow-up were examined. Given the small sample size, these were evaluated using effect size estimates and 95% CIs rather than significance tests. In addition, the reliability of each patient’s change in scores was examined using the response and remission criteria outlined below. Effect sizes (Cohen d) were calculated using the pooled SD as the denominator, calculated as SQRT((SD2initial+SD2post)/2) [38]. Interpretation followed the rules of thumb outlined by Cohen [39], with effect sizes of 0.2, 0.5, and 0.8 indicating small, medium, and large effects, respectively. CIs for Cohen d were calculated using the Hedges and Olkin formula [40].

    Response and Remission Criteria

    For classifying patients as treatment responders and/or remitted social anxiety, we used the criteria described by Stott et al’s [22] original iCT-SAD case series, which use both the reliable change formulae described by Jacobson and Truax [41] and normative data from Fresco et al [42]. Response to treatment was defined as an improvement greater than 31% on the LSAS between pretreatment and posttreatment [43]. Remission was defined as a drop of at least 12 points on the LSAS between pretreatment and posttreatment combined with a posttreatment score at or below the clinical threshold of 38 points.

    We also examined reliable improvement and reliable recovery from social anxiety using the English Improving Access to Psychological Therapies (IAPT) [44] outcome criteria, which consider change on both anxiety disorder–specific (SPIN) and depressed mood (PHQ) measures [45]. Reliable improvement was defined as a decrease of 10 or more points on the SPIN and/or 6 or more points on the PHQ and no reliable deterioration (increases of 10 or more, or 6 or more, respectively) on either measure. Reliable recovery was defined as reliable improvement combined with scores on both measures below the clinical threshold, that is, a SPIN score of 18 or below and a PHQ score of 9 or below.


    Results

    Face-to-Face Cognitive Therapy for Social Anxiety Disorder: Cognitive Therapy Competence Scale for Social Phobia

    Mean CTCS-SP scores across the 15 face-to-face CT skill areas were 3.9, 3.9, and 4.0 across the 3 therapists. These scores indicated that all 3 therapists achieved a level of competence in the delivery of the face-to-face treatment that would be expected of a high-intensity IAPT therapist (mean score >3).

    Internet-Based Cognitive Therapy for Social Anxiety Disorder Self-Evaluation Assessment

    Therapists’ self-reported knowledge and skills ratings across the 3 timepoints are shown in Figure 1. This figure shows that at baseline, where therapists had been trained in the face-to-face CT protocol but not iCT, self-reported iCT knowledge and skills ratings were low. These mean scores increased following the iCT training workshops in Phase 2 and further increased following completion of iCT practice cases under supervision in Phase 3.

    Figure 1. Therapists’ self-reported knowledge (top panel) and skills (bottom panel) ratings for internet-based Cognitive Therapy for social anxiety disorder following each phase of training.
    View this figure

    Internet-Based Cognitive Therapy for Social Anxiety Disorder Skills Test

    The 3 therapists’ scores on the different components of the skills test are shown in Table 1. As this is a newly developed instrument, there are no established competency cutoffs. However, as the test was designed in a similar manner to the CTCS-SP, with each item examining competence in a distinct skill, the CTCS-SP competency threshold of 50% is thought to be broadly applicable to the skills test and is used here to aid interpretation.

    Table 1. Therapists’ scores on each component of the skills test.
    View this table

    In this skills test, the 3 therapists achieved overall scores of 71%, 60%, and 64%, indicating good proficiency in the skills assessed. Benchmarking against the CTCS-SP, these scores therefore indicate a level of competency well above a 50% minimum standard. It should be noted that this assessment was completed following Phase 2 of the training, before the therapists’ pilot cases; this means they had further opportunities to practice and consolidate their skills in Phase 3.

    Patient Adherence

    All patients showed excellent adherence. Each patient completed all of the core treatment modules. Overall, patients were issued with a mean of 13.7 modules (SD 1.4) and completed a mean of 12.8 modules (SD 1.7). They spent an average of 40.2 hours (SD 26.4) using the website in the 14-week treatment period. As the program includes an automatic logout feature following periods of inactivity, the activity recorded on the site is thought to be a fair reflection of the time spent actively using the program.

    Internet-Based Cognitive Therapy for Social Anxiety Disorder Patient Outcomes

    Mean scores on all outcome measures are shown in Table 2 and effect sizes in Table 3. On the primary patient outcome measure, the LSAS, the mean decrease across the weekly treatment period was 53.8 points (SD 39.5). The within-group effect size (Cohen d) was 2.06 (95% CI 0.66-3.46), indicating an effect of medium to very large magnitude in the direction of reduced social anxiety. These figures are at least as good as data from the developers of the treatment [22], where the mean pre-post LSAS decrease was 40.2 points and the effect size was 1.64. Effect size point estimates for the present secondary outcome measures were also large, ranging from 0.92 to 1.89, with the lower limit of the CIs falling in the small-to-medium range for most, though not all, measures. Although highly tentative because of the sample size, these findings suggest the treatment was effective in reducing social anxiety, negative social cognitions, safety behaviors, worry, and functional impairment, and increasing social participation, but did not show a reliable effect on depressed mood or social satisfaction.

    Table 2. Means and SDs at pretreatment, posttreatment, and 3-month follow-up (N=6).
    View this table
    Table 3. Within-group effect sizes between pretreatment, posttreatment, and 3-month follow-up (N=6).
    View this table

    Changes in scores from posttreatment to 3-month follow-up were analyzed to review the maintenance of treatment gains over this period. Patients retained access to the program during this 3-month phase, though therapist contact was reduced to just one brief phone call at monthly intervals. As shown in Table 3, the effect size point estimates for most measures ranged between −0.20 and 0.20, meaning the threshold for a small effect in either direction was not reached. This suggested that the improvements from treatment had been maintained over this period. There was some evidence of a small effect in the direction of further improvement on the LSAS, SPS, and social participation measures during this period. On the SBQ, SAQ, and PHQ, there was some evidence of a small effect indicating a deterioration. Although this may indicate a lessening of some secondary treatment effects during the booster phase, these results may have been heavily influenced by the scores of 1 patient who did not respond to treatment and showed a decline on secondary outcome measures over the booster phase. Again, the CIs for these effect sizes were extremely wide because of sample size, so they can only provide a preliminary indication of the maintenance of treatment gains.

    Overall, 5 of the 6 patients showed an improvement on the LSAS greater than 31% at the posttreatment assessment and were thus classified as treatment responders. All 5 patients were also classified as remitted from their social anxiety, showing a drop of at least 12 points and falling below the clinical cutoff at posttreatment. Individual LSAS scores over time are shown in Figure 2. Using the IAPT criteria based on social anxiety (SPIN) and depression (PHQ) scores, 4 patients showed reliable improvement and 3 were classified as reliably recovered from SAD.

    Figure 2. Weekly scores on the Liebowitz Social Anxiety Scale (self-report) across treatment for the 6 pilot patients. Week 15 represents the posttreatment assessment. The dotted line represents the clinical cutoff score of 38 points.
    View this figure

    The 1 patient who was not classified as a responder to treatment within the weekly treatment phase experienced difficulties with depressed mood, which may have impacted their motivation to log in and work on the treatment website and to complete behavioral experiments. When compared with some other patients, they spent relatively little time on the website (12.5 hours total) and completed fewer behavioral experiments. Across the patient cohort, the amount of time spent on the website was positively correlated with percentage change on the LSAS (r=.584). It is also noted that the 2 patients with relatively lower baseline LSAS scores, because of more situation-specific forms of social anxiety, finished treatment with scores higher than some other patients whose baseline scores were more severe. The correlation between baseline LSAS score and percentage change on the LSAS was r=.443.

    Patient and Therapist Feedback

    All 6 patients completed the anonymous feedback survey at the end of the booster period. Given the small sample size, the frequencies of responses are reported. Overall, 5 of the patients felt that the program was relatively easy to understand in English, though 1 experienced some difficulties expressing their ideas and feelings and felt this could be a barrier for the therapy (respondent 4). Patients reported various therapist behaviors that they found helpful, including sending a written summary following phone calls, helping to generate new perspectives, identifying details of modules they may have missed, advising on behavioral experiments to try, and helping them to review their week. In terms of the treatment as a whole, respondents reported liking the phone calls (described as “intensive but helpful” by respondent 1); the practical nature of the contents; the effort, help, and professionalism of the therapist; the inclusion of both general and specific content; and the way the questionnaires allowed them to review their emotional state. In addition, 4 patients did not report any dislikes about the treatment. For security reasons, iCT-SAD uses 2-factor authentication at log-in, and 1 participant disliked having to enter an authentication code each time (respondent 1). One participant felt it was sometimes hard to motivate themselves to finish the tasks and expressed a concern that they may not have understood their own anxiety very well (respondent 4). All patients rated the program as easy to use on a desktop or laptop computer, but 2 reported difficulty when using a tablet or mobile phone. In sum, responses indicated a good level of treatment satisfaction and acceptability, with some but relatively few technical or practical difficulties.

    Therapists’ feedback indicated that they did not think any major cultural adaptation was required to the program content or implementation procedures for the Hong Kong context. However, the therapists did identify certain aspects of the iCT-SAD program that may prove difficult to understand when patients are working in a nonnative language. This included some video and audio material where groups of people are chatting informally, perhaps talking at the same time or in incomplete sentences. In addition, one of the exercises used in the program to practice externally focused attention asks patients to watch a video of someone reading a story to them while they alternate their attention between focusing on themselves and on the story content.


    Discussion

    Principal Findings

    This study has provided an example of a procedure for training therapists to deliver a guided internet intervention, including details of the content covered, and how progress was evaluated. The results indicated that the training was effective in improving therapists’ knowledge and skills in relation to iCT-SAD. The 3 therapists achieved a good level of competence when objectively measured using the skills test following Phase 2 of training. They were then able to consolidate these skills in their work with pilot cases, as demonstrated by the positive outcomes achieved by this cohort. These results suggested that the clinical outcomes for patients in Hong Kong undertaking iCT-SAD with local therapists were comparable to those seen in a UK study from the developers of the treatment [22].

    The 6 patients showed good adherence to the iCT-SAD program, completing treatment modules, phone calls with the therapist, behavioral experiments, and weekly questionnaire measures. All patients completed the full weekly and booster phases of treatment, indicating a good level of treatment acceptability and feasibility in this setting and sufficiently reliable functioning of the website. The therapists successfully implemented the treatment in line with the protocol and no significant deviations from this were required. These findings are of course preliminary given the present sample size and an RCT would be required to establish the feasibility and efficacy of the program in Hong Kong more robustly.

    This study took the approach of piloting the intervention in its unadapted form, meaning that the results could be benchmarked against the UK data and feedback could be elicited from both patients and therapists. Promisingly, the mean pre-post change on the LSAS and associated effect size point estimate (Cohen d=2.06) were large, suggesting that the high level of efficacy observed in the United Kingdom appears to have been maintained within a different cultural setting. Patient and therapist feedback did not identify any major cultural or linguistic barriers; the suggestions made to assist understanding were modest. We have since created some additional Chinese resources for the components of the program mentioned along with English-subtitled versions of the site videos. Overall, the findings in this study suggest that the approach of starting with the unadapted intervention was justified given the clinical outcomes obtained and that the adaptations suggested were quite modest and smaller than might have been expected at the outset of the project.

    Limitations

    Limitations of this study include the small number of therapists trained and the fact that the self-report assessment and skills test were developed specifically for this study and are therefore not yet externally validated. As mentioned above, the small number of patients in this pilot study can only tentatively indicate treatment efficacy and linguistic or cultural considerations, and larger controlled studies are therefore required. The training program implemented here was intensive and was not designed to be scalable at this stage. Future work should consider the best methods to provide such training in less time and to larger groups of therapists. In developing training programs for internet interventions that are based closely on a face-to-face treatment, our view is that familiarity with the face-to-face protocol is an important preliminary step and that treating a small number of training cases online under clinical supervision is essential for applying and consolidating skills. Online therapist training methods are likely to be the most scalable approach for the future, but preliminary work such as the approach taken here can help to develop a clearer understanding of what training should cover and how. The aim should be to determine what therapists need from training, making it easier to design online materials for that purpose. Our research team has recently started to develop online therapist training materials for iCT-SAD, which are available on our resources website [30]. Further research could also usefully explore whether less-experienced therapists can be trained to achieve similar outcomes.

    Conclusions

    This study has described and evaluated a program of therapist training in iCT-SAD, which appeared effective at increasing therapists’ knowledge and skills and resulted in positive clinical outcomes among a pilot case series of 6 patients in Hong Kong, with initial evidence that these were sustained at 3-month follow-up. The treatment appeared feasible and acceptable in this setting. An RCT of the iCT-SAD treatment in Hong Kong is now in progress. Given that therapist training is critical to the successful dissemination of internet interventions, further research to develop, describe, and evaluate therapist training procedures is recommended.

    Acknowledgments

    The authors wish to thank the case series participants. GRT is supported by the Wellcome Trust (102176), the National Institute for Health Research (NIHR) Oxford Biomedical Research Centre, and the NIHR Oxford Health Biomedical Research Centre. JW, EWP, and DC are supported by the Wellcome Trust (069777). The views expressed are those of the authors and not necessarily those of the National Health Service (NHS), the National Institute for Health Research (NIHR), or the Department of Health.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Internet-based Cognitive Therapy for social anxiety disorder self-evaluation assessment.

    DOCX File, 24KB

    Multimedia Appendix 2

    Internet-based Cognitive Therapy for social anxiety disorder skills test.

    DOCX File, 1MB

    References

    1. Andrews G, Basu A, Cuijpers P, Craske MG, McEvoy P, English CL, et al. Computer therapy for the anxiety and depression disorders is effective, acceptable and practical health care: an updated meta-analysis. J Anxiety Disord 2018 Apr;55:70-78 [FREE Full text] [CrossRef] [Medline]
    2. Cuijpers P, Marks IM, van Straten A, Cavanagh K, Gega L, Andersson G. Computer-aided psychotherapy for anxiety disorders: a meta-analytic review. Cogn Behav Ther 2009;38(2):66-82. [CrossRef] [Medline]
    3. Mayo-Wilson E, Montgomery P. Media-delivered cognitive behavioural therapy and behavioural therapy (self-help) for anxiety disorders in adults. Cochrane Database Syst Rev 2013;9:CD005330. [CrossRef] [Medline]
    4. Andersson G, Cuijpers P. Internet-based and other computerized psychological treatments for adult depression: a meta-analysis. Cogn Behav Ther 2009;38(4):196-205. [CrossRef] [Medline]
    5. Baumeister H, Reichler L, Munzinger M, Lin J. The impact of guidance on internet-based mental health interventions — a systematic review. Internet Interv 2014 Oct;1(4):205-215. [CrossRef]
    6. Spek V, Cuijpers P, Nyklícek I, Riper H, Keyzer J, Pop V. Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol Med 2007 Mar;37(3):319-328. [CrossRef] [Medline]
    7. Andersson G, Cuijpers P, Carlbring P, Riper H, Hedman E. Guided internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: a systematic review and meta-analysis. World Psychiatry 2014 Oct;13(3):288-295 [FREE Full text] [CrossRef] [Medline]
    8. Carlbring P, Andersson G, Cuijpers P, Riper H, Hedman-Lagerlöf E. Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: an updated systematic review and meta-analysis. Cogn Behav Ther 2018 Jan;47(1):1-18. [CrossRef] [Medline]
    9. Cuijpers P, Donker T, van Straten A, Li J, Andersson G. Is guided self-help as effective as face-to-face psychotherapy for depression and anxiety disorders? A systematic review and meta-analysis of comparative outcome studies. Psychol Med 2010 Dec;40(12):1943-1957. [CrossRef] [Medline]
    10. Stein D, Lim C, Roest A, de Jonge P, Aguilar-Gaxiola S, Al-Hamzawi A, WHO World Mental Health Survey Collaborators. The cross-national epidemiology of social anxiety disorder: data from the World Mental Health Survey Initiative. BMC Med 2017 Dec 31;15(1):143 [FREE Full text] [Medline]
    11. Tulbure BT, Szentagotai A, David O, Ștefan S, Månsson KN, David D, et al. Internet-delivered cognitive-behavioral therapy for social anxiety disorder in Romania: a randomized controlled trial. PLoS One 2015;10(5):e0123997 [FREE Full text] [CrossRef] [Medline]
    12. Jakobsen H, Andersson G, Havik OE, Nordgreen T. Guided internet-based cognitive behavioral therapy for mild and moderate depression: a benchmarking study. Internet Interv 2017 Mar;7:1-8 [FREE Full text] [CrossRef] [Medline]
    13. Kishimoto T, Krieger T, Berger T, Qian M, Chen H, Yang Y. Internet-based cognitive behavioral therapy for social anxiety with and without guidance compared to a wait list in China: a propensity score study. Psychother Psychosom 2016;85(5):317-319. [CrossRef] [Medline]
    14. Gallego MJ, Emmelkamp PM, van der Kooij M, Mees H. The effects of a Dutch version of an internet-based treatment program for fear of public speaking: a controlled study. Int J Clin Health Psychol 2011;11(3):459-472 [FREE Full text]
    15. Clark DM, Ehlers A, Hackmann A, McManus F, Fennell M, Grey N, et al. Cognitive therapy versus exposure and applied relaxation in social phobia: a randomized controlled trial. J Consult Clin Psychol 2006 Jun;74(3):568-578. [CrossRef] [Medline]
    16. Clark DM, Ehlers A, McManus F, Hackmann A, Fennell M, Campbell H, et al. Cognitive therapy versus fluoxetine in generalized social phobia: a randomized placebo-controlled trial. J Consult Clin Psychol 2003 Dec;71(6):1058-1067. [CrossRef] [Medline]
    17. Mörtberg E, Clark DM, Sundin O, Aberg Wistedt A. Intensive group cognitive treatment and individual cognitive therapy vs. treatment as usual in social phobia: a randomized controlled trial. Acta Psychiatr Scand 2007 Feb;115(2):142-154 [FREE Full text] [CrossRef] [Medline]
    18. Stangier U, Schramm E, Heidenreich T, Berger M, Clark DM. Cognitive therapy vs interpersonal psychotherapy in social anxiety disorder: a randomized controlled trial. Arch Gen Psychiatry 2011 Jul;68(7):692-700 [FREE Full text] [CrossRef] [Medline]
    19. Mayo-Wilson E, Dias S, Mavranezouli I, Kew K, Clark DM, Ades AE, et al. Psychological and pharmacological interventions for social anxiety disorder in adults: a systematic review and network meta-analysis. Lancet Psychiatry 2014 Oct;1(5):368-376 [FREE Full text] [CrossRef] [Medline]
    20. NICE. Social anxiety disorder: The NICE guideline on recognition, assessment and treatment. Cambridge, UK: RCPsych Publications; 2013.
    21. Bandura A, Blahard EB, Ritter B. Relative efficacy of desensitization and modeling approaches for inducing behavioral, affective, and attitudinal changes. J Pers Soc Psychol 1969 Nov;13(3):173-199. [Medline]
    22. Stott R, Wild J, Grey N, Liness S, Warnock-Parkes E, Commins S, et al. Internet-delivered cognitive therapy for social anxiety disorder: a development pilot series. Behav Cogn Psychother 2013 Jul;41(4):383-397. [CrossRef] [Medline]
    23. Leigh E, Clark DM. Cognitive therapy for social anxiety disorder in adolescents: a development case series. Behav Cogn Psychother 2016 Jan;44(1):1-17 [FREE Full text] [CrossRef] [Medline]
    24. von Consbruch CK, Clark DM, Stangier U. Assessing therapeutic competence in cognitive therapy for social phobia: psychometric properties of the cognitive therapy competence scale for social phobia (CTCS-SP). Behav Cogn Psychother 2012 Mar;40(2):149-161. [CrossRef] [Medline]
    25. Blackburn IM, James IA, Milne DL, Baker C, Standart S, Garland A, et al. The revised cognitive therapy scale (CTS-R): psychometric properties. Behav Cogn Psychother 2001 Oct 23;29(04):431-446 [FREE Full text] [CrossRef]
    26. Ginzburg DM, Bohn C, Höfling V, Weck F, Clark DM, Stangier U. Treatment specific competence predicts outcome in cognitive therapy for social anxiety disorder. Behav Res Ther 2012 Dec;50(12):747-752 [FREE Full text] [CrossRef] [Medline]
    27. Brown TA, Barlow DH, DiNardo PA. Anxiety Disorders Interview Schedule for DSM-IV (ADIS-IV): Client Interview Schedule. New York: Oxford University Press; 1994.
    28. First MB, Williams JB, Karg RS, Spitzer RL. Structured Clinical Interview for DSM-5. Arlington, VA: American Psychiatric Association; 2015.
    29. Baker SL, Heinrichs N, Kim HJ, Hofmann SG. The Liebowitz social anxiety scale as a self-report instrument: a preliminary psychometric analysis. Behav Res Ther 2002;40(6):701-715. [CrossRef] [Medline]
    30. Oxford Centre for Anxiety Disorders and Trauma. 2019. OXCADAT Resources: Resources for cognitive therapy for PTSD, social anxiety disorder and panic disorder   URL: https://oxcadatresources.com/ [accessed 2019-04-22] [WebCite Cache]
    31. Connor KM, Davidson JR, Churchill LE, Sherwood A, Foa E, Weisler RH. Psychometric properties of the Social Phobia Inventory (SPIN). New self-rating scale. Br J Psychiatry 2000 Apr;176:379-386 [FREE Full text] [Medline]
    32. Watson D, Friend R. Measurement of social-evaluative anxiety. J Consult Clin Psychol 1969 Aug;33(4):448-457. [Medline]
    33. Mattick RP, Clarke JC. Development and validation of measures of social phobia scrutiny fear and social interaction anxiety. Behav Res Ther 1998 Apr;36(4):455-470. [Medline]
    34. Alden LE, Taylor CT. Relational treatment strategies increase social approach behaviors in patients with Generalized Social Anxiety Disorder. J Anxiety Disord 2011;25(3):309-318 [FREE Full text] [CrossRef] [Medline]
    35. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001 Sep;16(9):606-613 [FREE Full text] [Medline]
    36. Spitzer RL, Kroenke K, Williams JB, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med 2006 May 22;166(10):1092-1097. [CrossRef] [Medline]
    37. Mundt JC, Marks IM, Shear MK, Greist JH. The Work and Social Adjustment Scale: a simple measure of impairment in functioning. Br J Psychiatry 2002 May;180:461-464 [FREE Full text] [Medline]
    38. Van Etten ML, Taylor S. Comparative efficacy of treatments for post-traumatic stress disorder: a meta-analysis. Clin Psychol Psychother 1998;5(3):126-144 [FREE Full text] [CrossRef]
    39. Cohen J. Statistical Power Analysis For The Behavioral Sciences (2nd Edition). Hillsdale, NJ: Lawrence Erlbaum Associates; 1988.
    40. Lee D. Alternatives to P value: confidence interval and effect size. Korean J Anesthesiol 2016 Dec;69(6):555-562 [FREE Full text] [CrossRef] [Medline]
    41. Jacobson NS, Truax P. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol 1991 Feb;59(1):12-19. [Medline]
    42. Fresco DM, Coles ME, Heimberg RG, Liebowitz MR, Hami S, Stein MB, et al. The Liebowitz Social Anxiety Scale: a comparison of the psychometric properties of self-report and clinician-administered formats. Psychol Med 2001;31(6):1025-1035 [FREE Full text] [CrossRef] [Medline]
    43. Bandelow B, Baldwin DS, Dolberg OT, Andersen HF, Stein DJ. What is the threshold for symptomatic response and remission for major depressive disorder, panic disorder, social anxiety disorder, and generalized anxiety disorder? J Clin Psychiatry 2006;67(09):1428-1434. [CrossRef] [Medline]
    44. Clark DM. Realizing the mass public benefit of evidence-based psychological therapies: the IAPT program. Annu Rev Clin Psychol 2018;14:159-183 [FREE Full text] [CrossRef] [Medline]
    45. National Collaborating Centre for Mental Health. 2018. The Improving Access to Psychological Therapies Manual   URL: https://www.england.nhs.uk/publication/the-improving-access-to-psychological-therapies-manual/ [accessed 2019-04-22] [WebCite Cache]


    Abbreviations

    CT: Cognitive Therapy
    CTCS-SP: Cognitive Therapy Competence Scale for Social Phobia
    CT-SAD: Cognitive Therapy for social anxiety disorder
    GAD: Generalized Anxiety Disorder
    IAPT: Improving Access to Psychological Therapies
    iCT: internet-based Cognitive Therapy
    iCT-SAD: internet-based Cognitive Therapy for social anxiety disorder
    LSAS: Liebowitz Social Anxiety Scale
    PHQ: Patient Health Questionnaire
    RCT: randomized controlled trial
    SAD: social anxiety disorder
    SAQ: Social Attitudes Questionnaire
    SBQ: Social Behavior Questionnaire
    SPIN: Social Phobia Inventory
    SPS: Social Phobia Scale


    Edited by G Eysenbach; submitted 20.01.19; peer-reviewed by A Bergin, J Högström; comments to author 06.03.19; revised version received 20.03.19; accepted 23.03.19; published 15.05.19

    ©Graham R Thew, Candice LYM Powell, Amy PL Kwok, Mandy H Lissillour Chan, Jennifer Wild, Emma Warnock-Parkes, Patrick WL Leung, David M Clark. Originally published in JMIR Formative Research (http://formative.jmir.org), 15.05.2019.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on http://formative.jmir.org, as well as this copyright and license information must be included.