This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on http://formative.jmir.org, as well as this copyright and license information must be included.
The majority of people with spinal cord injury (SCI) in the United States are not meeting the recommended guidelines for regular physical activity. Behavior change techniques (eg, goal setting and action planning) that are framed within the principles of the social cognitive theory (self-efficacy and self-regulation) have the potential to enhance physical activity behavior.
The aim of the study was to develop and test the usability of an electronic learning (e-learning) program for improving social cognitive factors related to physical activity behavior among people with SCI.
The program was created through an iterative process of development and refinement, using a modification of a similar methodology used to develop evidence-informed guidelines in health promotion for people with disabilities (Guidelines, Recommendations, and Adaptations Including Disability; GRAIDs framework). The study included 4 phases: (1) initial product creation, (2) national survey, (3) expert review, and (4) usability testing. Usability testing included both quantitative and qualitative data collection and analyses.
The review of the program by an expert panel (n=5) and the results from a national survey (n=142) led to several refinements. Usability testing demonstrated that the program could be completed in a timely manner (<30 min). Participants reported 5 themes: (1) the program improves social cognitions related to physical activity participation; (2) reflection of physical activity behavior; (3) positive perceptions of the quality of the program; (4) positive perceptions of the program operation and effectiveness; and (5) recommendations for improvement. Each item was incorporated into a revised program version 1.0.
This study incorporated an evidence-based framework for developing a brief 30-min e-learning program for increasing the physical activity behavior among people with SCI. The Exercise Strategies Through Optimized Relevant Interactive E-learning Storytelling (e-STORIES) program could be completed in a timely manner and was reported by participants as valuable and useful for enhancing intent-to-perform physical activity in individuals with SCI. The program has the potential to be applied in a variety of settings, but feasibility testing is required before implementing in a larger trial.
Regular physical activity participation is a core component of the postrehabilitative care among people with spinal cord injury (SCI). Over 30 years of research has indicated that people with SCI can increase one or more aspects of their physical and mental health through participation in physical activity [
Individuals with SCI are often overwhelmed by a multitude of barriers [
Interventions aimed at increasing LTPA in people with neurological disabilities have been found to be more effective when framed within a theory of behavior change [
One method that may be effective for communicating physical activity information is through story-based communications [
Preliminary data suggest that storytelling might be an effective strategy for conveying physical activity information to people with SCI [
E-learning technologies, combined with health behavior theory and narrative style, have the potential to provide educational opportunities to teach people how to overcome barriers to LTPA. E-learning authoring software (eg, Articulate 360) can be used to deliver easily accessible, interactive, and personalized educational programs. With the guidance of a narrative, individuals can be the narrators of their own e-learning storyline that adapts to their responses from programmed questions and queries. E-learning narratives can be combined with behavior change activities and engaging, visually attractive content (eg, text, pictures, audio, animation, and video) to modify SCT determinants, enhancing the likelihood of an individual engaging in LTPA. Although there are long-term behavioral coaching programs that can increase engagement in LTPA [
The aim of this study was to develop and test the usability of a brief e-learning program that embedded SCI narratives in a story-based approach, with the goal of improving SCT constructs that are related to LTPA. Program development included immersive involvement by people with SCI and input from an expert panel of researchers. This paper describes the development and usability of the program e-STORIES (Exercise Strategies Through Optimized Relevant Interactive E-learning Storytelling).
The e-STORIES development process was adapted from the Guidelines, Recommendations, and Adaptations Including Disability (GRAIDs) framework [
Research and development process for spinal cord injury Exercise Strategies Through Optimized Relevant Interactive E-learning Storytelling program.
Initial development of the computer-based program involved 4 elements: literature review, storyboard creation, theory and intervention alignment, and creation of the infrastructure for the program.
A systematic review of peer-reviewed publications was conducted using the search terms: “spinal cord injury” AND “physical activity” AND “behavior change” AND “self-management.” The study inclusion criteria included studies that were (1) peer-reviewed, (2) conducted within the past 2 years (2016-2018), (3) classified as a review paper, and (4) indexed in PubMed, Cumulative Index to Nursing and Allied Health Literature, or Scopus. The search for review papers returned a total of 10 articles and no duplicates. After screening these articles for eligibility criteria there was only 1 article remaining. Other articles were removed owing to not being a systematic review or not targeting SCI population.
The single systematic review of physical activity self-management interventions for people with SCI was used to identify behavioral LTPA interventions and strategies to aid in content development for the e-STORIES program [
The first stage of development was to create a storyboard that outlined the flow of the program, content suggestions, and scripts, as well as naming the project and designing a logo. The storyboard displayed the flow of the e-learning intervention and whether the information would be delivered by audio, text, pictures, animation, video, or some combinations of these media. The storyboard also helped with the construction of the pathways for tailored survey responses (ie, the flow of the program would adapt, based on a participant’s responses, to items related with confidence and self-regulatory strategies such as goal setting or planning).
SCT is based on the interplay between the behavioral factors, personal factors, and environmental factors [
Storyline 360, developed by Articulate Global Inc, is an e-learning authoring tool that was used to create the e-STORIES program. This software allowed the program to be interactive and instructional through multimedia that included digital movies, pictures, sound files, animation, graphics, and other electronic files. After a story board was created, the basic outline of the program was developed in Articulate’s Storyline 360. The outline included (1) program introduction, (2) assessment of theory constructs, (3) LTPA behavior change activities, and (4) printed copy of the participant’s results.
The second phase of the study included a survey of individuals with SCI that was delivered on the Web using Qualtrics. The survey aimed to collect descriptive information on the physical activity readiness and identification with SCI narratives related to LTPA, to inform and revise the program. Participants were asked to indicate which of the following best described their current level of physical activity: (a) Yes, I have been doing LTPA regularly for more than 6 months; (b) Yes, I have been doing LTPA regularly for less than 6 months; (c) No, but I intend to start doing LTPA regularly in the next 30 days; (d) No, but I intend to start doing LTPA regularly in the next 6 months; or (e) No, and I do NOT intend to start doing LTPA regularly in the next 6 months [
A description of each of the three SCI LTPA narrative styles was developed from the literature [
People with SCI were recruited through one of 12 SCI-related support groups or organization pages on a social media platform. The eligibility criteria included participants who (1) were aged 18 years or above, (2) had a diagnosis of SCI, (3) had the ability to use hands and arms to exercise, and (4) had the ability to converse in English and complete the survey forms.
Using a measure of physical activity readiness, participants were categorized as physical activity nonintenders, intenders, or actors [
An expert panel reviewed each component of the e-learning program for quality and accuracy using Articulate’s 360 Review, which allowed them to review the program online and add comments. The panel included an exercise psychologist, an exercise physiologist, a rehabilitation scientist, an information specialist, and a clinician who had several decades of experience in exercise research in SCI. After completing the review, each expert provided a summary of suggested revisions. This feedback led to several revisions, including textual, audio, and video changes and adding features to better guide participants. This phase included program
The usability testing incorporated a nested mixed-methods design (quantitative+qualitative), where the qualitative component was embedded within a primarily quantitative study design. In other words, the usability study included a qualitative interview within a design that emphasized the core fundamentals of usability testing. Quantitative and qualitative aspects were treated with equal value. On the basis of the best practice recommendations for usability testing [
Belief systems in the mixed-methods studies are critical as they influence all aspects of a study, including the design, analyses, and even the presentation of the results [
This study aimed to enroll 12 inactive people with SCI to satisfy the best practice recommendations for usability testing [
Participants were briefed about the study and provided the written consent forms. After obtaining the consent, participants’ demographics and clinical characteristics were recorded. Participants were then instructed to complete 16 usability tasks which involved navigating through and completing the entirety of the program and responding to the interactive questions. A research assistant took written notes while observing the participants perform the usability tasks and recorded the time taken by the participants to complete the entire e-learning program (all tasks). This was followed by a one-on-one interview conducted in a private and comfortable setting within the research laboratory. Examples of the interview questions included: “Describe to me some positive experiences of the program,” “Describe to me some negative experiences or issues you experienced with the program,” and “Tell me your overall thoughts about the program.” The interview was recorded by an audio device, which was later transcribed for qualitative analysis.
Usability was defined in terms of effectiveness (ie, the ease at which individuals can use the product), efficiency (ie, the speed at which an individual can accurately complete a task), usefulness (ie, the extent a product can enable the users to achieve their goals and the willingness to use the product), and satisfaction (ie, the users’ perceptions and opinions of the product). Usefulness and satisfaction were explored through qualitative means, whereas effectiveness and efficiency were examined through quantitative metrics.
Effectiveness was measured as the number of slides on which participants experienced an issue or problem, divided by the total slides that were completed without issues, which resulted in a single percentage value. The research team set an a priori benchmark of acceptable effectiveness at 90% [
Relative efficiency was measured by the time required to complete the entire module, which included the 16 tasks. The research team established a benchmark of acceptable efficiency at 30 min, which was 8 min longer than it took the lead researcher to complete the module. The program was developed to be brief so that it could be used during clinic visits (eg, waiting periods) or at a baseline visit for someone starting an exercise intervention.
Researchers assessed the usefulness and satisfaction through participants’ qualitative feedback from the face-to-face interviews. Each interview included open-ended questions that sought to gain insight into the participants’ overall perceptions of the module, their likes and dislikes regarding module features and content, whether they would use the module if it was provided to them in the clinic, and whether they perceived the information as valuable. One member of the research team conducted the interviews. The interviewer had 2 years of experience with qualitative research that involved usability testing of exercise technology.
Participant characteristics and quantitative usability data were descriptively reported. Qualitative data were analyzed using latent deductive thematic analysis (ie, a thematic analysis process was used in which data were analyzed with the intent of developing a surface level description of the 4 core areas of usability) [
A total of 154 people with SCI contributed to the development of the program. The national survey was completed by 142 participants, and 12 people enrolled in and completed the usability testing.
Participant characteristics for the national survey are shown in
Results of national survey (N=142).
Characteristics | Values | ||
Age (years), mean (SD) | 42 (14) | ||
Years since injury (years), mean (SD) | 12 (12) | ||
Male | 71 | ||
Female | 55 | ||
Paraplegia | 91 | ||
Tetraplegia | 51 | ||
Cure | 87 | ||
Quest | 43 | ||
Chaos | 12 | ||
Actor | 115 | ||
Intender | 22 | ||
Nonintender | 5 |
aData from 16 participants are unreported.
The following summarizes the revisions completed after the expert review: a statement about pain and exercising as a person with SCI was changed; some videos were re-recorded to improve quality; volume was increased on several audio clips; minor text edits were made; option was added in narratives for participants to use their own words or phrase; and examples of goals and next steps were provided.
Usability testing of program
Participant demographics of usability study (N=12).
Characteristics | Values | ||
Age (years), mean (SD) | 53 (11) | ||
Years since injury (years), mean (SD) | 23 (13) | ||
Male | 8 | ||
Female | 4 | ||
Paraplegia | 12 | ||
Tetraplegia | 0 |
The mean effectiveness score was 86% (SD 10%; 95% CI 80%-91%). This value was slightly lower than our a priori benchmark of 90%, indicating that minor changes were required to reduce operation issues.
The mean time to complete the program was 28 min 20 seconds (SD 6 min 45 seconds; 95% CI 25 mins 43 seconds to 33 min 3 seconds), which was lower than our a priori benchmark of 30 min.
The analysts identified 5 themes regarding the usability of the program: (1) the program promoted reflection/introspection of physical activity behavior; (2) the program improved social cognitions related to physical activity participation; (3) the participants had positive perceptions of the quality of the program; (4) the participants had positive perceptions of program operation and effectiveness; and (5) participant recommendations for improvement. These themes and their subthemes and supporting quotes are shown in
Participants reported that the program promoted valuable thoughts of self-reflection regarding physical activity behavior. The program provided relevant and encouraging information to establish reasonable outcome expectations of LTPA and to progress from a focus on the injury to resuming normal daily activities as reported by participant 1: “It definitely gets you from being focused on the actual injury to getting back into life, getting that groove back into life.”
Participants reported that the program improves social cognitions related to physical activity participation such as self-regulation. The program increased the participants’ confidence to plan and schedule PA through behavior change techniques that included goal setting, demonstration of the behavior, and how to plan physical activity around barriers that prevent participation. Participants further noted that the content provided meaningful and relevant information, with an uplifting and motivating tone:
It gets you to thinking. It gets you out of the doom and gloom process, and into the hopefulness, the looking forward to...And that's why I see it as being a major help in that area.
Participants reported high levels of satisfaction with the program design, including the animations and general flow of the slides. Participants further described the program as self-explanatory, intuitive, easy to operate, and engaging. However, the participants also had recommendations for improving the usability of the program. Participants noted that minor changes in the form of visual cues or instructional prompts and a back button could assist navigation and clarity in areas of the program that caused confusion.
Qualitative results of usability study.
Themes and subthemes | Supporting quotes | |
The program provided relevant and encouraging information that fostered thoughts of hope | "It gets you to thinking. It gets you out of the doom and gloom process, and into the hopefulness, the looking forward to...And that's why I see it as being a major help in that area."[Participant 002] | |
The program provided valuable suggestions to progress from a focus on the injury to resuming normal daily activities | “It definitely gets you from being focused on the actual injury to getting back into life, getting that groove back into life.” [Participant 001] | |
The program enhanced participants’ intentions to plan and schedule physical activity. | “It was very valuable because, like I said, I'm going to apply this to my everyday life from now on, and I'm going to start setting more planned goals and put them into motion.” [Participant 009] | |
Goal setting and vicarious experiences were memorable techniques that were learned from the program | “I think the positive is kinda relaying how you would go about obtaining a goal, like the time aspect of it. Set it 10 minutes per day, 30 days. And, the stories about the spinal cord injuries. Just seeing somebody else overcome the journey you've already been on.” [Participant 005] | |
The content provided information that was meaningful and relevant to the individual | “It's really a new day about spinal cord injury, and it show a lot about peer support when you're a spinal cord injury. It shows a lot about activities, it shows about people who've had trauma in their life and who have overcome the trauma, and they're living a very high functional life.” [Participant 002] | |
Participants were satisfied with the program design | “That was awesome and I liked the way he told his story about what happened to him. Some of the drawings and the way they did, I thought that was pretty good. The animation, the way they did that. I thought that was good.” [Participant 013] | |
Perceived as a valuable tool to enhance physical activity behavior for people with acute spinal cord injury | “Just that, I think, it would be beneficial for anyone with a spinal cord injury to participate in it or see it, especially somebody newly injured.” [Participant 008] | |
The content had a tone that was uplifting and motivating | “Very uplifting. I loved the manner in which the information was presented. It was very positive. It was motivating.” [Participant 008] | |
The program was self-explanatory and intuitive | “It kind of explained what it wanted you to do. You just clicked on it and it told you.” [Participant 011] | |
The program was easy to operate | “I guess with the mouse, with the clicking...The mouse helped me navigate through it easier and stuff, instead of using the pad.” [Participant 001] | |
The program was engaging | “It's quite interactive, and well thought out I will say. It's easy to navigate and keeps you into it. So, it's very much needed for a lot of people to get back into feeling the need for this type of program.” [Participant 009] | |
Improve visual cues/prompts to assist navigation | “The navigation was not bad, but it’s just...do I click on the box or do I move it? So that was a little confusing.” [Participant 001] | |
Enhance the definition of the different types of exercise intensities and the action prompts | “The only negative I had was the calculation of the mild to moderate when it changed it from. I entered 30 and it went to 90. It's just a little confusing...Yeah. A different color on the words mild, moderate, heavy. Maybe put them in bold. A different font. Just something to make it stand out so that it's emphasized...Potentially lead with: “You're going to be asked about three different levels of exercise,“ so that you know ahead of time that you'll be asked the same thing three times just differentiating mild, moderate, and heavy.” [Participant 008] | |
Provide the ability to go back to a previous screen | “Well not being able to go back to the previous screen if I messed up. I even tried to click next one time and it didn't work either. So that was a negative.” [Participant 006] |
The final program version included a total of
Screenshots of spinal cord Exercise Strategies Through Optimized Relevant Interactive E-learning Storytelling injury program.
This project incorporated an evidence-based framework for developing and testing a storied e-learning program to increase physical activity among people with SCI. Usability testing demonstrated that people could independently complete the program in a timely manner and that the program was perceived as valuable and useful for improving confidence to engage in physical activity and associated self-regulating behaviors among people with SCI. Usability testing identified minor operation issues that were rectified in the final version of the e-STORIES program.
A strength of this project was the high value placed on stakeholder feedback. Development of the program included input from over 150 participants with SCI. Such feedback informed critical development decisions, including the decision to not tailor the presentation of the SCI narratives based on the participant’s physical activity readiness and to halt further usability testing. Given that usability testing can be defined in several ways, the determination of an endpoint for usability testing is often a difficult task [
Statistical analysis of the results from the national survey (phase 2) did not support tailoring the presentation of SCI narratives based on the participant’s physical activity readiness. Findings demonstrated no significant differences among the SCI narratives and the physical activity readiness. However, only 8.3% (12/142) of the people identified with the chaos narrative, which likely affected the analyses. This finding is consistent with previous research which reported that less people with SCI identified with the chaos narrative when compared with the cure and quest narratives in LTPA studies [
Although usability results suggested that the e-STORIES program could potentially improve self-efficacy and self-regulatory skills to perform physical activity, further research is required to explore whether the program can be successfully implemented in pragmatic, real-world settings. One such setting is within the waiting area of a rehabilitation hospital for SCI patients, where the patient could discuss the e-STORIES results with their physician, as people with SCI perceive doctors as a trusted and credible source of PA information [
This project had a few limitations. Although the sample size for usability testing met 1 minimum recommendation, the sample limited our understanding of usability within various subgroups of people with SCI. Only people with paraplegia enrolled in the study, and these individuals were, on average, middle-aged. This limitation supports the need to test the program with larger heterogeneous samples of people with SCI, particularly individuals who are in the acute or subacute stage postinjury, a period when the program could be highly valuable as noted by several study participants. Another limitation was the low number of individuals in the chaos narrative, reducing the power to detect differences in readiness among the narratives. Finally, although only 3 narratives were presented in the program (cure, quest, and chaos), it is important to note that there have been several other narratives concerning LTPA among people with SCI in the literature [
e-STORIES is a brief, online program developed through an iterative process that was informed by the life experiences of more than 150 people with SCI, including the creator of the program (JW). The usability study demonstrated that people with SCI perceived the e-STORIES program as valuable, useful, effective, and of high quality. However, feasibility testing is required before the implementation of the program in a larger trial.
Video of Exercise Strategies Through Optimized Relevant Interactive E-learning Storytelling program.
electronic learning
Exercise Strategies Through Optimized Relevant Interactive E-learning Storytelling
Guidelines, Recommendations, Adaptations Including Disability
leisure-time physical activity
spinal cord injury
social cognitive theory
This project was funded by a postdoctoral fellowship (JW) through the Craig H Neilsen Foundation and conducted at Lakeshore Foundation in Birmingham, Alabama. KMG holds the Reichwald Family University of British Columbia Southern Medical Program Chair in Preventive Medicine.
JW, BL, JR, and KMG contributed to the design of the study and the development of the program. JH completed all recruitment and data collection for the usability study. JW, BL, and HW analyzed the study results. JW led the program development and the manuscript development. KMG and JR oversaw the implementation of the project. All authors contributed to the formulation of this manuscript.
None declared.