Original Paper
Abstract
Background: Internet-delivered cognitive behavioral therapy (ICBT) is an effective and convenient means of offering cognitive behavioral therapy to the general population. To increase access to ICBT among Canadian public safety personnel (PSP)—a group that experiences elevated rates of mental health concerns and barriers to mental health care—a clinical research unit called PSPNET has tailored ICBT to PSP, primarily through offering case stories and PSP-specific examples within an ICBT program. PSPNET’s first and most frequently used ICBT program, called the PSP Wellbeing Course, has been found to reduce symptoms of mental disorders (eg, anxiety, depression, and posttraumatic stress) among PSP. Little research, however, has investigated clients’ perceptions of the case stories in this course.
Objective: This study was designed to expand the literature on the use and evaluation of case stories in ICBT among PSP. Specifically, this study investigated (1) PSP’s perceptions of the case stories using the theoretical model provided by Shaffer and Zikmund-Fisher and (2) PSP feedback on the case stories in the PSP Wellbeing Course.
Methods: This study included 41 clients who completed the PSP Wellbeing Course. Of these, 27 clients completed a bespoke questionnaire called the Stories Questionnaire, 10 of whom also participated in a semistructured interview.
Results: Findings show that perceptions of the case stories in the PSP Wellbeing Course were largely positive and that the case stories were generally successful in achieving the 5 purposes of case stories (ie, informing, comforting, modeling, engaging, and persuading) proposed by Shaffer and Zikmund-Fisher. Client feedback also identified 3 tangible areas for story improvement: characters, content, and delivery. Each area highlights the need for and potential benefits of story development. Not all PSP engaged with the case stories, though, so results must be interpreted with caution.
Conclusions: Overall, this study adds to the growing body of research supporting the use of case stories in internet-delivered interventions among PSP.
Trial Registration: ClinicalTrials.gov NCT04127032; https://www.clinicaltrials.gov/ct2/show/NCT04127032
doi:10.2196/64454
Keywords
Introduction
Background
The term “public safety personnel” (PSP) refers to individuals whose work ensures the safety and security of citizens (eg, border security officers, correctional workers, firefighters, Indigenous emergency managers, operational intelligence personnel, paramedics, police, public safety communicators, and search and rescue personnel [
]). Due to high-stress work environments and exposures to potentially psychologically traumatic events (PPTE), PSP are at high risk of mental health challenges [ , ], with 44.5% screening positive for at least 1 mental disorder [ ]. PSP also often experience barriers to receiving treatment such as workplace stigma toward mental health, the desire to self-manage symptoms of mental disorders, limited time available to attend treatment, limited accessibility of mental health providers in rural areas, and treatment costs [ , ].Internet-delivered cognitive behavioral therapy (ICBT) is an effective psychological treatment for various mental disorders that includes evidence-based cognitive behavioral therapy (CBT) techniques in a digital format [
]. An advantage of ICBT is that clients can access it privately at a time and place that is most convenient for them. ICBT can be disorder specific [ , ] or transdiagnostic (ie, designed to treat symptoms of several disorders) and can be delivered in a self- or therapist-guided format (eg, phone and email [ ]). Clients are more likely to choose transdiagnostic over disorder-specific ICBT despite comparable treatment satisfaction and outcomes [ , ]. Overall, ICBT is similarly effective to face-to-face CBT in managing mental health challenges such as anxiety, depression, and posttraumatic stress [ ].A clinical research unit called PSPNET was launched in 2019 with funding from the Government of Canada to develop, implement, and evaluate ICBT tailored to the needs of Canadian PSP [
]. PSPNET adapted a previously established ICBT course that has excellent treatment outcomes in Australia [ ] and Canada [ , ] to address common mental health concerns of PSP. The course was titled the PSP Wellbeing Course and, like the established ICBT course upon which it was based, is therapist guided and transdiagnostic. Mental health outcomes of the PSP Wellbeing Course demonstrate moderate to large, statistically significant declines in PSP’s anxiety, depression, and posttraumatic stress symptoms from before to after treatment [ , ]. The PSP Wellbeing Course was tailored via interviews and focus groups to better understand PSP’s preferences in ICBT [ ] and to develop fictional client narratives called case stories. Of note, PSP Wellbeing Course findings [ ] parallel other ICBT program findings, with many clients reporting the case stories as helpful but needing improvement [ - ].Case stories are illustrative examples that are commonly used in health care services to provide anecdotes, analogies, and metaphors to facilitate learning and enhance the client experience. Case stories can be sourced from real or fictitious user profiles [
]. Fictitious user profiles, also known as personas, are designed to represent current or prospective clients from specific groups with detailed characteristics (eg, name, photo, symptoms, and diagnosis). Information used to develop such personas can be sourced from clients [ ] or experts (eg, clinician experiences working with PSP [ ]). Shaffer and Zikmund-Fisher [ ] identified five main purposes of case stories in health care: (1) to inform clients of basic knowledge and lived experiences, (2) to provide comfort to clients by normalizing their experiences, (3) to model targeted behaviors, (4) to persuade clients to use targeted behaviors, and (5) to enhance client engagement. This taxonomy offers a valuable guideline for designing, implementing, and evaluating case stories [ ].Client perceptions of case stories in health care services [
, , ] and in ICBT programs are generally positive [ - ], but clients often suggest stories could be improved. Preliminary findings show that PSP are also generally satisfied with case stories in ICBT, citing them as a valuable component that portrays authentic and relatable experiences [ ] while providing some feedback to improve content and delivery. Content feedback from PSP has centered on improving story authenticity [ , , ] and relatability [ ]. For example, some clients request the use of real, rather than fictitious, case stories [ ]. Other feedback includes increasing the complexity of stories [ , ] and adding more examples on the application of treatment principles [ ]. Delivery feedback has centered on improving story accessibility (eg, adding audio or video narratives [ , ]).The extant research provides preliminary support for the inclusion of case stories in ICBT tailored for PSP, but some gaps in knowledge still exist, highlighting the need for further story evaluation. Improving health care services often relies on client perceptions and feedback [
, ]. Client perceptions can identify treatment satisfaction [ ] and sustain good practices (eg, reinforce good practices among clinicians [ ]), while client feedback can be used to identify problems, inform areas for improvement, evaluate improvements, and assess progress toward organizational goals [ ]. To date, story feedback in ICBT has largely been gained through open-ended, web-based questions asking clients to provide brief written feedback about what they liked and disliked about ICBT generally [ - , , ]. In-depth client perceptions of and feedback on case stories in ICBT among PSP have not yet been obtained.Research Questions and Hypotheses
This study was designed to expand the literature on the use and evaluation of case stories in ICBT among PSP to help identify what is effective and ineffective about case stories and how limitations can be addressed. Specifically, this study investigated (1) PSP’s perceptions of the case stories using the theoretical model provided by Shaffer and Zikmund-Fisher [
] and (2) PSP’s feedback on the case stories in the PSP Wellbeing Course. The research team hypothesized that (1) PSP’s perceptions of the case stories would be generally positive, (2) PSP’s perceptions would align with the purposes of stories identified by Shaffer and Zikmund-Fisher [ ], and (3) PSP would provide tangible feedback on ways to improve case stories in the PSP Wellbeing Course.Methods
Ethical Considerations
This mixed methods study was approved by the Research Ethics Board at the University of Regina (#2019-157) and was carried out within the context of a longitudinal, single-group, open trial registered on ClinicalTrials.gov (NCT04127032). Clients provided informed consent after being made aware of the study details and the potential benefits and risks of participation. Participants were provided with access to the PSP Wellbeing Course but were not offered other incentives to encourage participation. Client data were stored on a secure server and were deidentified prior to analyses.
Study Design
Prospective clients signed up for a PSPNET account; provided informed consent to participate; and completed eligibility screening questionnaires that collected demographic, clinical, and occupational information. This was followed by a telephone interview, after which eligible clients were enrolled in an ICBT course appropriate to their symptoms and preferences. This study describes outcomes only for clients enrolled in the therapist-guided PSP Wellbeing Course (English version). At 6 weeks after enrollment, clients were invited to schedule a semistructured interview for around the 8-week postenrollment time point. At 8 weeks after enrollment, we administered the Stories Questionnaire, a bespoke questionnaire designed to assess perceptions of and feedback on the case stories (described in detail below).
Participants
Web-based screening took place between August 2022 and March 2023 and resulted in a sample of 36 PSP (
). Eligibility criteria required that all clients who enroll in the English version of the therapist-guided PSP Wellbeing Course to (1) be 18 years or older; (2) be current or former PSP; (3) have internet access; (4) provide local emergency medical contact if presenting with severe symptoms; and (5) live in New Brunswick, Nova Scotia, Prince Edward Island, Québec, or Saskatchewan. Individuals were excluded from the PSP Wellbeing Course, and correspondingly, this study (and referred to other services as appropriate) based on the following criteria: (1) reporting high risk of suicide; (2) reporting a suicide attempt or hospitalization for high suicide risk within the past year; (3) reporting primary problems of psychosis, mania, or severe alcohol or drug use; (4) being in receipt of psychological treatment from other health care services more than twice a month; (5) not being in Canada during the treatment period; and (6) no longer being interested in service (eg, seeking other services, no longer requiring services, and being unable to devote time). Furthermore, this study only included clients who accessed at least the first 4 of 5 lessons in the English version of the PSP Wellbeing Course to ensure adequate exposure to the case stories to provide feedback.Measures
Participant Information Questionnaire
During web-based screening, prospective clients completed a Participant Information Questionnaire. This questionnaire solicited demographic data including age, gender, ethnicity, marital status, number of children, province, and education. This questionnaire also solicited occupational data (eg, PSP sector and years of experience).
Stories Questionnaire
The Stories Questionnaire was a bespoke questionnaire administered at 8 weeks after enrollment. It included branch logic and was composed of 4-13 items, depending on clients’ responses, assessing clients’ perceptions of and feedback on the case stories in the PSP Wellbeing Course. If clients reported reviewing at least 1 of the case stories, 9 items appeared asking for their level of agreement with the provided statements on a 5-point Likert-type scale ranging from 1 (strongly disagree) to 5 (strongly agree). Clients were asked whether the case stories represented relatable and authentic PSP experiences. Five items assessed the degree to which clients perceived the stories as fulfilling the 5 purposes of stories proposed in Shaffer and Zikmund-Fisher’s [
] taxonomy (ie, inform, comfort, model, persuade, and engage). Subsequently, clients completed 3 open-ended items inquiring about their likes, dislikes, and suggestions for story improvement. Cronbach α for Likert-type items was excellent (α=0.94). The Stories Questionnaire is shown in .Semistructured Interview
At 6 weeks after enrollment, a PSPNET team member invited clients to schedule a semistructured interview via email. Semistructured interviews were only conducted with the first 10 clients interested, lasted 15-60 minutes, and were scheduled for around 8 weeks after enrollment. A research assistant (JG) conducted the interviews by phone, and clients provided verbal consent to the interview and recording of the interview. Interview questions assessed clients’ perceptions of and feedback on the case stories. Clients were specifically asked questions relating to (1) perceptions of the case stories’ authenticity and relatability, (2) perceptions of the degree to which the case stories adhered to the taxonomy proposed by Shaffer and Zikmund-Fisher [
] (ie, inform, comfort, model, persuade, and engage), (3) overall perceptions of the helpfulness or unhelpfulness of the case stories, and (4) feedback for improving the case stories. Given the semistructured nature of the interview, the research assistant (JG) also asked follow-up questions where appropriate to gather more insight into story perceptions including more detailed feedback for improving the stories.The PSP Wellbeing Course
The PSP Wellbeing Course is an 8-week transdiagnostic ICBT program composed of 5 lesson slideshows featuring instructive text, diagrams, and case stories that exemplify CBT coping strategies. The first lesson introduces clients to the cognitive behavioral treatment model and how to identify symptoms of mental disorders. The second lesson provides information about maladaptive thoughts and how to challenge them. The third lesson focuses on managing physical symptoms of under and overarousal. The fourth lesson teaches the implementation of graded exposure to manage behavioral symptoms. The fifth lesson concludes with information on preventing symptom relapse and sustaining treatment outcomes [
]. Throughout the course, clients also receive optional therapist support via secure email or phone up to twice a week for up to 16 weeks. Therapists provide encouragement, clarification on course material, and support to help clients integrate CBT skills into their daily lives.Each lesson offers a supplementary video summary and downloadable PDF resources (eg, frequently asked questions, do-it-yourself guides, and additional case stories not included in the lesson slides). Additional resources cover various topics relating to mental health and situational issues (eg, sleep problems, anger, and grief) and are accessible at any time throughout the course. The lesson slideshows include text narratives of 2 fictional characters (ie, Chris [
] and Mike) who discuss their experiences working through the PSP Wellbeing Course and how they implemented the CBT skills outlined in each lesson. Supplementary materials include additional case stories featuring six other fictional characters who discuss their experiences, impressions of the course, successes, and challenges faced while working through course material ( ). Clients in the PSP Wellbeing Course are given access to all course materials for up to 1 year.Name (age [years] and gender) | PSPa sector | Primary symptoms | Significant life events | Core thoughts | Core behaviors |
Chrisb (40s, woman) | Police | Depression; trouble sleeping | Traumatic work events | Failure; burden to family; lazy | Social isolation; avoidance; alcohol use |
Mikeb (50s, man) | Not specified | Anxiety; panic attacks; depression | N/Ac | Excessive worry; self-conscious; unable to cope | Unproductive at work; avoidance |
Deborah (57, woman) | Public Safety Communications | Anxiety; depression; panic attacks | Cancer survivor; children are moving out; stressful work calls | Excessive worry; family is not safe; distrust | Overeating; overprotective of children; pulling out hair |
Greg (43, man) | Police | Chronic pain; depression; trouble sleeping | Motor vehicle accident at work; death of a colleague | Guilt; burden to family and colleagues | Social isolation; overuse of pain medication; alcohol and drug use |
Jeff (28, man) | Corrections | Anger; trouble sleeping | Fighting with girlfriend; overwhelmed at work; surgery recovery | Unable to cope; burden to family; distrust | Aggression; alcohol use; social isolation |
Lisa (55, woman) | Police | PTSDd | Assaulted at work | Shame; self-blame; guilt; hopelessness; burden to colleagues | Avoidance; hypervigilance; alcohol use |
Mark (28, man) | Fire | Anger; panic attacks; social anxiety | Expecting first baby; training; work schedule changes | Self-doubt; poor social skills | Social isolation; angry outbursts; overuse of prescription drugs |
Nicole (32, woman) | Paramedic | Depression; trouble sleeping | Caring for ill mother; overwhelmed at work | Inadequacy; letting family down; unable to care for self | Alcohol and tobacco use; unhealthy eating; social isolation |
aPSP: public safety personnel.
bStory included in the core content of each lesson.
cNot applicable.
dPTSD: posttraumatic stress disorder.
Data Analyses
Quantitative data were analyzed using SPSS (version 28; IBM Corp) to provide descriptive statistics from all eligible clients (n=36) via the Patient Information Questionnaire and Stories Questionnaire. Qualitative data were then analyzed using NVivo 20 (QSR International) analysis software [
] using a reflexive approach to open-ended responses via the Stories Questionnaire (n=25) and the semistructured interviews (n=10). All client data were deidentified prior to analyses, and semistructured interviews were transcribed using a professional transcription service. The reflexive analysis was conducted by identifying, analyzing, and reporting coherent and meaningful patterns in the data [ ]. Overarching topics were categorized as main themes and evaluated for any underlying topics using deductive and inductive approaches. The initial codebook was created by JG, revised by the primary coder (JABP), and reviewed by CAL. Upon completion of coding, each theme was assessed for semantic meaning and given an appropriate name [ ].Results
Client Flow and Demographics
shows the screening, enrollment, and treatment process. The mean client age was 41.4 (SD 11.3) years. Most clients resided in Saskatchewan (n=24, 67%) and self-identified as White (n=32, 89%), women (n=17, 47%), married (n=22, 61%), and parents (n=56, 56%). Clients also self-identified most frequently as police (n=17, 47%), with others self-identifying as paramedics (n=4, 11%), firefighters (n=6=, 17%), correctional workers (n=4, 11%), or other PSP sectors (n=5, 14%; ). Overall, most clients who started the PSP Wellbeing Course self-reported that they reviewed at least one of the case stories (n=25, 69%).
All clients (N=36) | Completed the Stories Questionnaire (n=32) | Did not complete the Stories Questionnaire (n=4) | ||||
Reviewed at least 1 story (n=25) | Did not review any stories (n=7) | |||||
Age (years), mean (SD) | 42.9 (11.1) | 44.7 (10.1) | 40.3 (15.5) | 36.6 (4.7) | ||
Gender, n (%) | ||||||
Woman | 17 (47) | 13 (52) | 1 (14) | 3 (75) | ||
Man | 19 (53) | 12 (48) | 6 (86) | 1 (25) | ||
Ethnicity, n (%) | ||||||
White | 32 (89) | 22 (88) | 6 (86) | 4 (100) | ||
Other | 4 (11) | 3 (12) | 1 (14) | 0 (0) | ||
Married or common law, n (%) | ||||||
Yes | 22 (61) | 17 (68) | 3 (43) | 2 (50) | ||
No | 14 (39) | 8 (32) | 4 (57) | 2 (50) | ||
Children, n (%) | ||||||
Yes | 20 (56) | 16 (64) | 3 (43) | 1 (25) | ||
No | 16 (44) | 9 (36) | 4 (57) | 3 (75) | ||
Home province, n (%) | ||||||
Maritimes | 11 (31) | 9 (36) | 2 (29) | 0 (0) | ||
Saskatchewan | 24 (67) | 15 (60) | 5 (71) | 4 (100) | ||
Other | 1 (3) | 1 (4) | 0 (0) | 0 (0) | ||
PSPasector, n (%) | ||||||
Corrections | 4 (11) | 1 (4) | 2 (29) | 1 (25) | ||
Fire | 6 (17) | 3 (12) | 3 (43) | 0 (0) | ||
Paramedics | 4 (11) | 3 (12) | 0 (0) | 1 (25) | ||
Police | 17 (47) | 13 (52) | 2 (29) | 2 (50) | ||
Other | 5 (14) | 5 (20) | 0 (0) | 0 (0) | ||
Years in PSP field, mean (SD) | 13.8 (10.7) | 14.0 (9.3) | 17.4 (15.5) | 5.8 (4.9) | ||
Highest level of education, n (%) | ||||||
High school diploma or less | 2 (6) | 1 (4) | 1 (14) | 0 (0) | ||
Some college or university | 11 (31) | 7 (28) | 4 (57) | 0 (0) | ||
College or university degree | 23 (64) | 17 (68) | 2 (29) | 4 (100) |
aPSP: public safety personnel.
Story Perceptions
Story perceptions were evaluated using quantitative data from the Stories Questionnaire for clients who self-reported viewing at least 1 of the case stories (25/36, 69%). Descriptive statistics showed of the 25 clients, most agreed or strongly agreed that “I could relate to at least one story” (n=15, 60%) and that “the stories show a bias-free perspective about what it is like to be a PSP” (n=21, 84%). Consistent with expectations, story perceptions in the PSP Wellbeing Course were generally positive with most clients endorsing the case stories as fulfilling the taxonomy proposed by Shaffer and Zikmund-Fisher [
]. Results showed that most clients agreed or strongly agreed that “I find that the stories are a trustworthy source of information” (n=16, 64%), “at least one story increased my knowledge about my mental health” (n=18, 72%), “reading the stories made me realize I am not alone with my mental health experiences” (n=18, 72%), “at least one story gave me ideas about how to use the skills to improve my well-being” (n=18, 72%), and “at least one story motivated me to use the skills” (n=16, 64%). Most clients also agreed or strongly agreed that “at least one story helped me better understand the lesson content” (n=14, 56%) and “at least one story made me want to continue on with the course” (n=14, 56%; ).Score, mean (SD) | Strongly disagree (1), n (%) | Disagree (2), n (%) | Neutral (3), n (%) | Agree (4), n (%) | Strongly agree (5), n (%) | |
I could relate to at least one story | 3.8 (1.2) | 1 (4) | 3 (12) | 6 (24) | 5 (20) | 10 (40) |
I find that the stories are trustworthy source of information | 3.8 (1.0) | 0 (0) | 3 (12) | 6 (24) | 8 (32) | 8 (32) |
The stories show a bias-free perspective about what it is like to be a PSPa | 4.0 (0.9) | 0 (0) | 1 (4) | 6 (24) | 9 (36) | 12 (36) |
Reading the stories made me realize I am not alone with my mental health experience | 4.2 (0.8) | 0 (0) | 0 (0) | 7 (28) | 5 (20) | 13 (52) |
At least one story gave me ideas about how to use the skills to improve my well-being | 3.9 (0.8) | 0 (0) | 1 (4) | 6 (24) | 12 (48) | 6 (24) |
At least one story motivated me to use the skills | 3.8 (1.1) | 1 (4) | 2 (8) | 6 (24) | 8 (32) | 8 (32) |
At least one story increased my knowledge about my mental health | 3.8 (0.9) | 0 (0) | 1 (4) | 9 (36) | 9 (36) | 6 (24) |
At least one story helped me to better understand the lesson content | 3.6 (1.3) | 2 (8) | 3 (12) | 6 (24) | 7 (28) | 7 (28) |
At least one story made me want to continue on with the course | 3.5 (1.4) | 3 (12) | 3 (12) | 5 (20) | 6 (24) | 8 (32) |
aPSP: public safety personnel.
Story Feedback
Story feedback was evaluated using qualitative data from the Stories Questionnaire and interview to gain insight on ways to improve the case stories in the PSP Wellbeing Course. Clients recommended improvements to characters, content, and delivery (
). Character feedback focused on demographics and photos. Clients most frequently requested improving character demographics by incorporating more ethnicities (eg, Indigenous PSP), sectors (ie, coroner service personnel, office worker), and employment status (eg, retired PSP). Some clients also recommended using real PSP photos instead of stock photos ( ).Themes | Example quotes | Count, n (%) | |||
Characters | 6 (22) | ||||
Demographics | 4 (15) | ||||
Employment status | “This is really, I...I think for a person that is still in uniform.” | 1 (4) | |||
Ethnicity | “More racial/cultural diversity.” | 2 (7) | |||
Sector | “Add the story of a PSPa who works in an office, supporting field PSP and who is not a call center employee.” | 2 (7) | |||
Photos | “The pictures are obviously stock photos which adds to the fictitious feel of the story.” | 2 (7) | |||
Content | 5 (19) | ||||
Topics | 4 (15) | ||||
Failure | “That failure thing was overwhelming within the stories.” | 1 (4) | |||
Family | “I would have gotten more out of listening to the more fulsome story, like, you know, this is...this is the impact it’s had. You know, whether it’s the daughter or the son, or the husband, or the wife or somebody who...who, you know, saw their loved ones and the helpless feeling they have.” | 1 (4) | |||
Grief | “I came into this course with my own selfish goal. And that was this grief. If there is something here that...that could help me. And the only thing that was on here was...it was in the...of those files.” | 1 (4) | |||
Positivity | “We need some stories where people somehow turn that energy...that negative energy...and make it positive energy.” | 1 (4) | |||
PPTEb | “More background on some of the situations that the PSP faced prior to feeling this way, could be helpful.” | 1 (4) | |||
Relationships | “How to deal with relationships, uh, would probably have been really helpful.” | 1 (4) | |||
Vocabulary | “The word fear isn’t in there. And that was a word that I was using. Scared to death. You know? Like, scared and fear. And I know those words.” | 2 (7) | |||
Delivery | 9 (33) | ||||
Formatting | 2 (7) | ||||
Labels | “Label who the person is not just by the name but their age and occupations makes it easier for some people to know who is who.” | 1 (4) | |||
Navigational links | “I’d actually put a reference in there, and this is how I solved it, or this is how I dealt with it. A little click link and it’s in the material on the same module.” | 1 (4) | |||
Media | “I think it would be a different...a different reaction even if that, you know, if you had a person sitting in front of you that told you the story. Or, um, or even, like, a short video.” | 1 (4) | |||
Realism | “I didn’t get the impression that I was sitting down listening to somebody tell their story.” | 6 (22) | |||
Quantity | “Have more options. I think if there were even more stories, there would be even more things for people to relate to and learn from.” | 2 (7) |
aPSP: public safety personnel.
bPPTE: potentially psychologically traumatic event.
Content feedback focused on modifying topics and vocabulary. Specifically, six topics were highlighted for improvement: (1) failure, (2) family, (3) grief, (4) positive self-talk, (5) PPTE, and (6) relationships. One client requested reducing the emphasis on failure in the case stories while other clients suggested adding more content on the impact of PSP mental health challenges on family members and how to navigate these relationships. Other topics that clients suggested for inclusion are grief, positive self-talk in the workplace, and PPTEs that precede characters’ mental health challenges. Clients further recommended tailoring story vocabulary to include terms more familiar to PSP such, as “fear” and “scared” in anxiety-related content, as well as the phrase “caring for those who care” among the general content (
).Delivery feedback focused on improving story formatting, media, realism, and quantity. Clients requested modifying story formatting by adding labels to improve recognition of characters throughout the course and inserting links to easily navigate from the case stories to relevant course material. One client advocated for the addition of video content in the case stories. Clients most frequently requested improving realism, citing a fictitious feel to the case stories (
).Discussion
Principal Findings
Overview
There is a growing body of research demonstrating the value of ICBT as an effective and convenient means of treatment for various mental disorders among the general population [
, ]. To increase access to ICBT among Canadian PSP, PSPNET developed the PSP Wellbeing Course. Findings show that the PSP Wellbeing Course can reduce symptoms of anxiety, depression, and posttraumatic stress [ , ]. Previously identified areas for improvement include the case stories [ ]. This study expands on previous research by showing that PSP’s perceptions of the case stories in the PSP Wellbeing Course were generally positive and that PSP perceived the case stories as adhering to the taxonomy proposed by Shaffer and Zikmund-Fisher [ ]. Nevertheless, findings identified 3 tangible areas for story improvement: characters, content, and delivery.Characters
Most clients who accessed the PSP Wellbeing Course reviewed at least 1 of the case stories. This finding is consistent with previous research showing that many—but not all—clients engage with case stories embedded in internet-delivered interventions [
, ]. Story engagement begins by creating a connection between the client and the character via authentic [ ] and relatable experiences [ ]. This study found that most clients perceived the case stories as genuine and identifiable representations of PSP. Still, consistent with previous research [ , ], some clients questioned character demographics, citing a lack of diversity. The legitimacy of the characters was also questioned as some clients indicated that the photos appeared artificial. Improving character representation is important as it could enhance story engagement [ , ]. These findings highlight the need for and potential benefits of character development in the case stories in the PSP Wellbeing Course and other internet-delivered interventions.Characters are typically designed to represent groups of target clientele that share common features [
]. Common features can yield high recognition among clients [ ], but they can also lead to systematic errors such as gender and ethnic profiling [ ]. Soliciting client feedback in development can help avoid such errors [ ]. Clients in this study requested increasing character diversity with respect to employment status, ethnicity, and sector. Diversifying characters can increase representation [ ], which may, in turn, lead to improved engagement. Clients in this study also requested the use of real PSP photos. Character photos support story engagement and empathy by creating a connection between photos and content [ ]. The case stories in the PSP Wellbeing Course currently use stock photos. Stock photos are considered a convenient and cost-effective solution for case stories, but their use may hinder character authenticity and relatability [ , ]. In lieu, researchers recommend using consistent quality, head-and-shoulder photos [ ] of real people in real situations [ , ]. While increasing character diversity may pose new challenges for selecting appropriate photos, we foresee the benefits of real photos outweighing these potential challenges. Still, real photos may not always be feasible in all contexts due to privacy concerns and regulations (eg, codes of conduct among provincial colleges of psychology in Canada) forbidding the use of client testimonials [ ].Content
The PSP Wellbeing Course was successful in achieving the five purposes of case stories proposed by Shaffer and Zikmund-Fisher [
]. To recall, case stories (1) provide information on basic knowledge and lived experiences, (2) provide comfort by normalizing experiences, (3) model targeted behaviors, (4) motivate the use of targeted behaviors, and (5) enhance engagement by increasing understanding and use of course material. Findings showed that clients most often endorsed the case stories as a source of comfort that modeled targeted behaviors, whereas clients least often endorsed the case stories as enhancing their understanding and engagement with the course material. These discrepancies were minimal and were not endorsed by all clients. Consistent with previous research [ ], clients recommended expanding the depth and variety of topics discussed in the case stories but the nature of the topics was highly diverse with each client having a different recommendation. It would be impractical to add new stories to address every topic recommended by clients, and a topic recommended by a single past client would not necessarily be perceived as relevant or important by future clients; therefore, efforts to address a greater range of topics should likely begin with the topics that are recommended most frequently by multiple clients. Vocabulary can play a critical role in communicating about such topics [ , ]. Consistent with previous research [ ], clients also recommended prioritizing vocabulary familiar to the clientele. Research suggests avoiding jargon and unexplained acronyms, which can result in increased treatment withdrawals [ ]. Therefore, improving topics and vocabulary in case stories may indirectly improve client comfort, motivation, persuasion, and engagement.Delivery
Clients highlighted four areas for improving story delivery: formatting, media, realism, and quantity. Formatting recommendations focused on increasing the accessibility of stories. Accessibility ensures that all current and prospective clients have the same opportunities to succeed regardless of physical or mental disabilities [
]. Accessibility can also increase acceptance and engagement in internet-delivered interventions [ ]. To improve story accessibility in the PSP Wellbeing Course, clients suggested adding character labels to facilitate recognition throughout the course, and navigational links to relevant course material. To further promote accessibility, case stories could incorporate media content as not all clients support text-only narratives in case stories [ , , ]. Benefits of audiovisual narratives include decreased cognitive load [ ], as well as increased engagement [ , ] and persuasion [ , ]. Shaffer and Zikmund-Fisher [ ] explain that audiovisual narratives require less cognitive resources compared to text narratives. Decreasing cognitive load means that clients have more cognitive resources available to engage in internet-delivered interventions [ ]. Audiovisual narratives can also be more persuasive in changing client attitudes or behaviors compared to text narratives [ , , ]. For example, clients report higher empathy for audiovisual characters [ ]. Incorporating audiovisual narratives into case stories in the PSP Wellbeing Course may then be cost-effective when seeking to improve treatment engagement, persuasion, and accessibility.Perceived realism refers to the degree to which characters and content resemble real people and situations [
]. Consistent with previous research [ , ], some clients recommended improving the perceived realism of case stories in the PSP Wellbeing Course. Improving perceived realism can impact clients’ identification with the characters and content [ ], and in turn, client engagement [ ] and persuasion of attitudes [ ]. Actionable steps may include co-design via PSP working groups and adding a new survey question that asks clients to share elements of their own stories to help inform future case stories.Overall, consistent with previous research [
], clients recommended adding more case stories to the PSP Wellbeing Course. While increasing the number of case stories may increase character representation [ ], it may hinder character adherence [ ]. Nevertheless, the ideal quantity of case stories in internet-delivered interventions is unclear. Casts typically vary between 3 and 12 characters [ ]. The PSP Wellbeing Course maintains this norm with 8 characters. Adding more case stories may help address character diversity and other delivery issues, but such issues could also be addressed by revising existing characters and stories instead.Limitations and Future Directions
This study had several limitations that can further inform future research directions. First, the sample size was relatively small. Second, this study may be at risk of sampling bias, as clients with more favorable perspectives on the PSP Wellbeing Course or PSPNET may have been more receptive to PSPNET’s request for them to complete questionnaires and interviews than clients with less favorable perspectives, possibly leading to an overrepresentation of favorable perceptions of the stories in our sample. Relatedly, our exclusion of clients who had completed fewer than 4 of the 5 lessons of the course from this study may have resulted in the systematic exclusion of clients with unfavorable perspectives on the course as well. Further, most clients self-identified as White, police, or residing in Saskatchewan, and results may have limited generalizability to other groups of PSP. Third, some clients may have provided feedback without reviewing all case stories, resulting in recommendations already addressed in the case stories they did not review. Overall, results should be interpreted with caution. Future research should consider investigating whether improving the case stories in the PSP Wellbeing Course as described in this study impacts client perceptions, client engagement with treatment, and mental health outcomes. Future research could help identify perceptions of and feedback on case stories in other PSPNET courses. Finally, future research could explore whether artificial intelligence could be useful for personalizing case stories and its impact on treatment outcomes [
].Conclusions
This study adds to the growing body of research on the use of case stories in internet-delivered interventions. First, clients’ perceptions of the case stories in the PSP Wellbeing Course were largely positive and adhered to the taxonomy proposed by Shaffer and Zikmund-Fisher [
]. Second, client feedback provides tangible ways to improve the case stories in the PSP Wellbeing Course. Client feedback specifically identified 3 areas for story improvement: characters, content, and delivery. Each area highlights the need for and potential benefits of further story development. These findings should be interpreted with caution due to the small sample size and possible concerns with sampling and response bias. Overall, this study adds to the growing body of research supporting the use of case stories in internet-delivered interventions among PSP.Acknowledgments
The authors would like to acknowledge the contributions of the eCentre Clinic at Macquarie University, Australia, who developed the original version of the Wellbeing Course; and the Online Therapy Unit for their support in setting up PSPNET. The authors would also like to acknowledge all working groups, whose contributions are integral to the development of this course; all public safety personnel who participated in this study; and all PSPNET staff for their support with this project. This manuscript is based on a published honors thesis defended by JG [
]. PSPNET is part of the Canadian Institute for Public Safety Research and Treatment, which is funded by the Canadian Government. The funder was not involved in the study design or the collection, analysis, or interpretation of the data.Conflicts of Interest
None declared.
Stories questionnaire.
PDF File (Adobe PDF File), 53 KBCase story sample.
PDF File (Adobe PDF File), 104 KBReferences
- Heber A, Testa V, Groll D, Ritchie K, Tam-Seto L, Mulligan A, et al. Glossary of terms: a shared understanding of the common terms used to describe psychological trauma, version 3.0. Health Promot Chronic Dis Prev Can. 2023;43(10-11):S1-S999. [FREE Full text] [CrossRef] [Medline]
- Carleton RN, Afifi TO, Turner S, Taillieu T, Duranceau S, LeBouthillier DM, et al. Mental disorder symptoms among public safety personnel in Canada. Can J Psychiatry. 2018;63(1):54-64. [FREE Full text] [CrossRef] [Medline]
- Carleton RN, Afifi TO, Taillieu T, Turner S, Krakauer R, Anderson GS, et al. Exposures to potentially traumatic events among public safety personnel in Canada. Can J Behav Sci. 2019;51(1):37-52. [CrossRef]
- McCall HC, Helgadottir FD, Menzies RG, Hadjistavropoulos HD, Chen FS. Evaluating a web-based social anxiety intervention among community users: analysis of real-world data. J Med Internet Res. 2019;21(1):e11566. [FREE Full text] [CrossRef] [Medline]
- McCall HC, Landry CA, Ogunade A, Carleton RN, Hadjistavropoulos HD. Why do public safety personnel seek tailored internet-delivered cognitive behavioural therapy, an observational study of treatment-seekers. Int J Environ Res Public Health. 2021;18(22):11972. [FREE Full text] [CrossRef] [Medline]
- Andersson G, Carlbring P, Titov N, Lindefors N. Internet interventions for adults with anxiety and mood disorders: a narrative umbrella review of recent meta-analyses. Can J Psychiatry. 2019;64(7):465-470. [FREE Full text] [CrossRef] [Medline]
- Păsărelu CR, Andersson G, Nordgren LB, Dobrean A. Internet-delivered transdiagnostic and tailored cognitive behavioral therapy for anxiety and depression: a systematic review and meta-analysis of randomized controlled trials. Cogn Behav Ther. 2017;46(1):1-28. [CrossRef] [Medline]
- Andersson G, Cuijpers P, Carlbring P, Riper H, Hedman E. Guided internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: a systematic review and meta-analysis. World Psychiatry. 2014;13(3):288-295. [FREE Full text] [CrossRef] [Medline]
- McCall HC, Dear BF, Landry CA, Beahm JD, Gregory J, Titov N, et al. Internet-delivered cognitive behavioural therapy for symptoms of PTSD among public safety personnel: initial outcomes of an open cohort preference trial of transdiagnostic and disorder-specific therapy. Internet Interventions. 2023;33:100656. [FREE Full text] [CrossRef] [Medline]
- Titov N, Dear BF, Staples LG, Bennett-Levy J, Klein B, Rapee RM, et al. MindSpot clinic: an accessible, efficient, and effective online treatment service for anxiety and depression. Psychiatr Serv. 2015;66(10):1043-1050. [CrossRef] [Medline]
- Supporting Canada's public safety personnel: an action plan on post-traumatic stress injuries. Public Safety Canada. 2019. URL: https://www.publicsafety.gc.ca/cnt/rsrcs/pblctns/2019-ctn-pln-ptsi/index-en.aspx [accessed 2024-09-25]
- Guliani H, Witt J, Peynenburg V, Wilhelms A, Nugent M, Dear BF, et al. Cost-effectiveness of varying degrees and models of therapist-assisted transdiagnostic internet-delivered cognitive behaviour therapy: evidence from a randomized controlled trial. Internet Interventions. 2022;29:100567. [FREE Full text] [CrossRef] [Medline]
- Hadjistavropoulos HD, Peynenburg V, Thiessen DL, Nugent M, Karin E, Staples L, et al. Utilization, patient characteristics, and longitudinal improvements among patients from a provincially funded transdiagnostic internet-delivered cognitive behavioural therapy program: observational study of trends over 6 years. Can J Psychiatry. 2022;67(3):192-206. [FREE Full text] [CrossRef] [Medline]
- Hadjistavropoulos HD, McCall HC, Dear BF, Beahm JD, Carleton RN, Titov N. Outcomes of transdiagnostic internet-delivered cognitive behavioural therapy tailored to public safety personnel: a longitudinal observational study. J Anxiety Disord. 2024;104:102861. [FREE Full text] [CrossRef] [Medline]
- Hadjistavropoulos HD, McCall HC, Thiessen DL, Huang Z, Carleton RN, Dear BF, et al. Initial outcomes of transdiagnostic internet-delivered cognitive behavioral therapy tailored to public safety personnel: longitudinal observational study. J Med Internet Res. 2021;23(5):e27610. [FREE Full text] [CrossRef] [Medline]
- McCall HC, Beahm JD, Fournier AK, Burnett JL, Carleton RN, Hadjistavropoulos HD. Stakeholder perspectives on internet-delivered cognitive behavioural therapy for public safety personnel: a qualitative analysis. Can J Behav Sci. 2021;53(3):232-242. [CrossRef]
- Hadjistavropoulos HD, Reiser SJ, Beahm JD, McCall HC, Dena I, Phillips AR, et al. Internet-delivered cognitive behavioral therapy tailored to spouses and significant others of public safety personnel: formative evaluation study. JMIR Form Res. 2023;7:e51088. [FREE Full text] [CrossRef] [Medline]
- Mehta S, Hadjistavropoulos HD, Earis D, Titov N, Dear BF. Patient perspectives of internet-delivered cognitive behavior therapy for psychosocial issues post spinal cord injury. Rehabil Psychol. 2019;64(3):351-359. [CrossRef] [Medline]
- Hadjistavropoulos HD, Faller YN, Klatt A, Nugent MN, Dear BF, Titov N. Patient perspectives on strengths and challenges of therapist-assisted internet-delivered cognitive behaviour therapy: using the patient voice to improve care. Community Mental Health J. 2018;54(7):944-950. [FREE Full text] [CrossRef] [Medline]
- Hadjistavropoulos HD, Chadwick C, Beck CD, Edmonds M, Sundström C, Edwards W, et al. Improving internet-delivered cognitive behaviour therapy for alcohol misuse: patient perspectives following program completion. Internet Interventions. 2021;26:100474. [FREE Full text] [CrossRef] [Medline]
- Shaffer VA, Zikmund-Fisher BJ. All stories are not alike: a purpose-, content-, and valence-based taxonomy of patient narratives in decision aids. Med Decis Making. 2013;33(1):4-13. [CrossRef] [Medline]
- Shen F, Sheer VC, Li R. Impact of narratives on persuasion in health communication: a meta-analysis. J Advert. 2015;44(2):105-113. [CrossRef]
- Bartels SL, Taygar AS, Johnsson SI, Petersson S, Flink I, Boersma K, et al. Using personas in the development of eHealth interventions for chronic pain: a scoping review and narrative synthesis. Internet Interventions. 2023;32:100619. [FREE Full text] [CrossRef] [Medline]
- Drewniak D, Glässel A, Hodel M, Biller-Andorno N. Risks and benefits of web-based patient narratives: systematic review. J Med Internet Res. 2020;22(3):e15772. [FREE Full text] [CrossRef] [Medline]
- Grob R, Schlesinger M, Barre LR, Bardach N, Lagu T, Shaller D, et al. What words convey: the potential for patient narratives to inform quality improvement. Milbank Q. 2019;97(1):176-227. [FREE Full text] [CrossRef] [Medline]
- Beahm JD, Landry CA, McCall HC, Carleton RN, Hadjistavropoulos HD. Understanding and addressing occupational stressors in internet-delivered therapy for public safety personnel: a qualitative analysis. Int J Environ Res Public Health. 2022;19(8):4744. [FREE Full text] [CrossRef] [Medline]
- Beahm JD, McCall HC, Carleton RN, Titov N, Dear B, Hadjistavropoulos HD. Insights into internet-delivered cognitive behavioural therapy for public safety personnel: exploration of client experiences during and after treatment. Internet Interventions. 2021;26:100481. [FREE Full text] [CrossRef] [Medline]
- Randolph G, Esporas M, Provost L, Massie S, Bundy DG. Model for improvement—part two: measurement and feedback for quality improvement efforts. Pediatr Clin North Am. 2009;56(4):779-798. [CrossRef] [Medline]
- Cherry JM, Freund L, Duff WM. Students' perceptions of information programs in Canada. J Educ Lib Inf Sci. 2013;54(3):174-190. [FREE Full text]
- NVivo. Lumivero. URL: https://lumivero.com/products/nvivo/ [accessed 2024-09-25]
- Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77-101. [CrossRef]
- Lewis C, Roberts NP, Simon N, Bethell A, Bisson JI. Internet-delivered cognitive behavioural therapy for post-traumatic stress disorder: systematic review and meta-analysis. Acta Psychiatr Scand. 2019;140(6):508-521. [CrossRef] [Medline]
- Hadjistavropoulos HD, Hill TG, Philipp-Muller AE, Dear B, Titov N. Perceptions of client stories in internet-delivered cognitive behaviour therapy: a mixed-methods evaluation. Internet Interventions. 2023;34:100692. [FREE Full text] [CrossRef] [Medline]
- Shaffer VA, Owens J, Zikmund-Fisher BJ. The effect of patient narratives on information search in a web-based breast cancer decision aid: an eye-tracking study. J Med Internet Res. 2013;15(12):e273. [FREE Full text] [CrossRef] [Medline]
- Salminen J, Santos JM, Kwak H, An J, Jung S, Jansen BJ. Persona perception scale: development and exploratory validation of an instrument for evaluating individuals’ perceptions of personas. Int J Hum-Comput Stud. 2020;141:102437. [CrossRef]
- Sapkota RP, Valli E, Wilhelms A, Adlam K, Bourgeault L, Heron V, et al. Patient-oriented research to improve internet-delivered cognitive behavioural therapy for people of diverse ethnocultural groups in routine practice. Healthcare (Basel). 2023;11(15):2135. [FREE Full text] [CrossRef] [Medline]
- Nielsen L. Personas: User Focused Design. Redmond, MA. Springer London; 2019.
- Salminen J, Nielsen L, Jung SG, An J, Kwak H, Jansen BJ. “Is More Better?”: impact of multiple photos on perception of persona profiles. 2018. Presented at: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems; April 21, 2018; Montréal, QC. [CrossRef]
- Marsden N, Haag M. Stereotypes and politics: reflections on personas. 2016. Presented at: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems; May 7, 2016:7-12; San Jose, CA. [CrossRef]
- Floyd IR, Jones CM, Twidale MB. Resolving incommensurable debates: a preliminary identification of persona kinds, attributes, and characteristics. Artifact. 2008;2(1):12-26. [CrossRef]
- Salminen J, Chhirang K, Jung SG, Thirumuruganathan S, Guan KW, Jansen BJ. Big data, small personas: how algorithms shape the demographic representation of data-driven user segments. Big Data. 2022;10(4):313-336. [CrossRef] [Medline]
- Mulder S, Yaar Z. The user is always right: a practical guide to creating and using personas for the web. O'Reilly. 2006. URL: https://learning.oreilly.com/library/view/the-user-is/0321449924/pt02.html [accessed 2024-02-12]
- Lacroix E, Dobson KS, von Ranson KM. Scope and ethics of psychologists’ use of client testimonials on professional websites. Can Psychol. 2018;59(3):232-238. [CrossRef]
- Henka A, Zimmerman G. Personas based accessibility testing: towards user-centered accessibility evaluation. 2014. Presented at: Proceedings at the HCI International 2014; June 22, 2014:22-27; Heraklion, Greece. [CrossRef]
- Riggs EE, Knobloch-Westerwick S. Beyond the text: testing narrative persuasion mechanisms with audio messages. Mass Commun Soc. 2022;27(1):148-172. [CrossRef]
- Braverman J. Testimonials versus informational persuasive messages. Commun Res. 2008;35(5):666-694. [CrossRef]
- Rubin AM. Television use by children and adolescents. Human Commun Res. 1979;5(2):109-120. [CrossRef]
- Chock TM. Is it seeing or believing? Exposure, perceived realism, and emerging adults' perceptions of their own and others' attitudes about relationships. Media Psychol. 2011;14(4):355-386. [CrossRef]
- Cohen SS, Parra LC. Memorable audiovisual narratives synchronize sensory and supramodal neural responses. eNeuro. 2016;3(6):1-11. [FREE Full text] [CrossRef] [Medline]
- Cho H, Shen L, Wilson K. Perceived realism. Commun Res. 2012;41(6):828-851. [CrossRef]
- Salminen J, Jung SG, Nielsen L, Jansen B. Creating more personas improves representation of demographically diverse populations: implications towards interactive persona systems. 2022. Presented at: Proceedings of the 2022 Nordic Human-Computer Interaction Conference; October 12, 2022; Aarhus, Denmark. [CrossRef]
- Goodwin K. Designing for the digital age: how to create human-centered products and services. O'Reilly. 2009. URL: https://learning.oreilly.com/library/view/designing-for-the/9780470229101/ch11.html#definition_and_uses [accessed 2024-02-05]
- Nielsen L, Hansen KS, Stage J, Billestrup J. A template for design personas: analysis of 47 persona descriptions from Danish industries and organizations. Int J Sociotechnol Knowl Dev. 2015;7(1):45-61. [CrossRef]
- Carlbring P, Andersson G, Cuijpers P, Riper H, Hedman-Lagerlöf E. Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: an updated systematic review and meta-analysis. Cogn Behav Ther. 2018;47(1):1-18. [CrossRef] [Medline]
- Gregory J. Perceptions of case stories in internet-delivered cognitive behavioural therapy for public safety personnel: a mixed methods study [Thesis]. University of Regina. 2023. URL: https://instrepo-prod7.cc.uregina.ca/server/api/core/bitstreams/c14ee832-99d1-4c65-ac0a-b0a758fd4ca3/content [accessed 2023-09-06]
Abbreviations
CBT: cognitive behavioral therapy |
ICBT: internet-based cognitive behavioral therapy |
PPTE: potentially psychologically traumatic events |
PSP: public safety personnel |
Edited by A Mavragani; submitted 18.07.24; peer-reviewed by M Bisby; comments to author 22.08.24; revised version received 04.09.24; accepted 20.09.24; published 25.10.24.
Copyright©Jill AB Price, Julia Gregory, Hugh C McCall, Caeleigh A Landry, Janine D Beahm, Heather D Hadjistavropoulos. Originally published in JMIR Formative Research (https://formative.jmir.org), 25.10.2024.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.