This paper is in the following e-collection/theme issue:

Formative Evaluation of Digital Health Interventions (2226) Web-based and Mobile Health Interventions (3128) Behavior Change (655) Formative Studies and eHealth/mHealth Development (516) Methods and Feasibility Studies (369)

Published on in Vol 7 (2023)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/43557, first published .
A Digital Health Intervention for Concussion: Development and Clinical Feasibility Study

A Digital Health Intervention for Concussion: Development and Clinical Feasibility Study

A Digital Health Intervention for Concussion: Development and Clinical Feasibility Study

Authors of this article:

Christine d'Offay1, 2 Author Orcid Image ;   Xin Yi Ng3 Author Orcid Image ;   Laura Alexander4 Author Orcid Image ;   Alison Grant5 Author Orcid Image ;   Julia Grahamslaw5 Author Orcid Image ;   Claudia Pagliari1 Author Orcid Image ;   Matthew J Reed5, 6 Author Orcid Image ;   Alan Carson4 Author Orcid Image ;   David C Gillespie4 Author Orcid Image ;   Aimun A B Jamjoom2, 4, 7 Author Orcid Image
Article Authors Cited by (3) Tweetations (8) Metrics

    Original Paper

    1Centre for Population Health Sciences, Usher Institute, The University of Edinburgh, Edinburgh, United Kingdom

    2HeadOn Health Ltd, Edinburgh, United Kingdom

    3University of Edinburgh Medical School, The University of Edinburgh, Edinburgh, United Kingdom

    4Centre for Clinical Brain Sciences, The University of Edinburgh, Edinburgh, United Kingdom

    5The Emergency Medicine Research Group Edinburgh (EMERGE), Royal Infirmary of Edinburgh, Edinburgh, United Kingdom

    6Acute Care Edinburgh, Usher Institute, The University of Edinburgh, Edinburgh, United Kingdom

    7Department of Clinical Neuroscience, Edinburgh Royal Infirmary, Edinburgh, United Kingdom

    Corresponding Author:

    Aimun A B Jamjoom, PhD

    Centre for Clinical Brain Sciences

    The University of Edinburgh

    Chancellors Building

    Edinburgh, EH16 4SA

    United Kingdom

    Phone: 44 131 650 1000

    Email: v1ajamjo@ed.ac.uk


    Background: Concussion is a common condition that can lead to a constellation of symptoms that affect quality of life, social integration, and return to work. There are several evidence-based behavioral and psychological interventions that have been found to improve postconcussion symptom burden. However, these are not routinely delivered, and individuals receive limited support during their concussion recovery.

    Objective: This study aimed to develop and test the feasibility of a digital health intervention using a systematic evidence-, theory-, and person-based approach.

    Methods: This was a mixed methodology study involving a scoping review (n=21), behavioral analysis, and logic model to inform the intervention design and content. During development, the intervention was optimized with feedback from individuals who had experienced concussions (n=12) and health care professionals (n=11). The intervention was then offered to patients presenting to the emergency department with a concussion (n=50). Participants used the intervention freely and input symptom data as part of the program. A number of outcome measures were obtained, including participant engagement with the intervention, postconcussion symptom burden, and attitudes toward the intervention. A selection of participants (n=15) took part in in-depth qualitative interviews to understand their attitudes toward the intervention and how to improve it.

    Results: Engagement with the intervention functionality was 90% (45/50) for the symptom diary, 62% (31/50) for sleep time setting, 56% (28/50) for the alcohol tracker, 48% (24/50) for exercise day setting, 34% (17/50) for the thought diary, and 32% (16/50) for the goal setter. Metrics indicated high levels of early engagement that trailed off throughout the course of the intervention, with an average daily completion rate of the symptom diary of 28.23% (494/1750). A quarter of the study participants (13/50, 26%) were classified as high engagers who interacted with all the functionalities within the intervention. Quantitative and qualitative feedback indicated a high level of usability and positive perception of the intervention. Daily symptom diaries (n=494) demonstrated a wide variation in individual participant symptom burden but a decline in average burden over time. For participants with Rivermead scores on completion of HeadOn, there was a strong positive correlation (r=0.86; P<.001) between their average daily HeadOn symptom diary score and their end-of-program Rivermead score. Insights from the interviews were then fed back into development to optimize the intervention and facilitate engagement.

    Conclusions: Using this systematic approach, we developed a digital health intervention for individuals who have experienced a concussion that is designed to facilitate positive behavior change. Symptom data input as part of the intervention provided insights into postconcussion symptom burden and recovery trajectories.

    Trial Registration: ClinicalTrials.gov NCT05069948; https://clinicaltrials.gov/ct2/show/NCT05069948

    JMIR Form Res 2023;7:e43557

    doi:10.2196/43557

    Keywords

    concussion; digital intervention; behavior change; feasibility study 


    Concussion is a common condition that can occur in a range of circumstances, including falls, assaults, road traffic collisions, and playing sports [1]. Estimates place the incidence of concussion at 600 per 100,000, and it is a major cause of presentation to the emergency department (ED), with approximately 2 million visits in the United States per year [1,2]. Individuals can experience a constellation of postconcussion symptoms, including physical (headaches and dizziness), cognitive (difficulty concentrating and memory problems), and emotional (depression and anxiety) symptoms as well as sleep disturbances [3]. Although these symptoms typically improve over time, a considerable proportion of individuals remain with persistent symptoms up to a year after their injury [4,5]. Postconcussion symptom burden is associated with poorer health-related quality of life [6], reduced community integration [7], work absenteeism [8], and increased use of health care resources [9]. Despite the serious public health concern that concussion poses, individuals are typically offered limited support and follow-up after their injury [10,11]. This is a concern as several behavioral and psychological interventions have been found to improve postconcussion symptom burden, including early educational material [12], cognitive behavioral therapy (CBT) [13], and aerobic exercise [14]. Importantly, intervening early in the natural history following concussion appears to play a role in improving outcomes [12,15]. Digital health interventions are increasingly being used to manage a range of medical conditions [16,17]. They are scalable and can be cost-effectively delivered early in the clinical course of the disease. A review of digital solutions for concussion found limited options and a lacking evidence-base [18]. To address this problem, we developed HeadOn—a digital health intervention that uses behavior change techniques to encourage positive behaviors to facilitate postconcussion recovery. The intervention was developed using a systematic evidence-, theory-, and person-based approach and the Medical Research Council (MRC) guidance on the development of complex interventions [19]. In this paper, we describe the development of the intervention and subsequent clinical feasibility study examining the acceptability and use of the intervention by a cohort of participants presenting to an ED with a concussion.


    Scoping Review

    A literature search was conducted in February 2021 to identify relevant studies. The electronic databases used were MEDLINE (via Ovid), Scopus, PsycINFO (via Ovid), and the Cochrane Library. The following terms and their derivatives were used in the electronic search: “concussion,” “post-concussion syndrome,” “mild traumatic brain injury,” “cognitive behavioral therapy,” “exercise therapy,” “self-management,” “self-care,” and “health education” (Multimedia Appendix 1 [20-40]). The inclusion criteria were as follows: (1) peer-reviewed English-language articles published from the year 2000 onward, (2) qualitative research (interviews, surveys, and focus groups), and (3) studies examining behavioral (exercise and physical rehabilitation, rest, and self-management) and psychological (health education, CBT, counseling, psychotherapy, and cognitive rehabilitation) interventions for concussion (including postconcussion syndrome). After removing duplicates, the titles and abstracts of papers that met the inclusion criteria were screened by 2 reviewers (XN and AABJ), and full-text article reviews were carried out on eligible papers. The bibliographies of relevant articles found during screening were also searched manually. The included articles were reviewed, and data were extracted and organized into a predefined table of potential barriers to and facilitators of the use of interventions. These were mapped onto the following domains: intervention engagement, information, symptom monitoring, lifestyle and behavior, thoughts and emotions, design, and technical aspects. The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) scoping review checklist is detailed in Multimedia Appendix 1.

    Behavioral Analysis

    The behavioral analysis involved linking the barriers identified in the scoping review to target behaviors (intervention engagement, symptom monitoring, healthy lifestyle habits, addressing negative thoughts, and goal setting). The Behavior Change Wheel (BCW) [41] and taxonomy of behavior change techniques [42] were used as the theoretical framework to systematically address each barrier. The BCW was used to identify the target construct and intervention functions, followed by the taxonomy to identify the appropriate behavior change technique to address the barrier. This ensured a systematic and theoretically driven approach to addressing barriers to behavior change.

    Logic Model

    A logic model was developed based on the outcome of the scoping review and behavioral analysis. A logic model is a graphic representation of the shared relationships among the activities, outputs, and outcomes of a program. For HeadOn, the model included five parts: (1) the clinical problem that HeadOn aims to address; (2) the intervention targets, which are the behaviors that HeadOn aims to promote; (3) the intervention ingredients, which are the program components of HeadOn; (4) examined mechanisms, which are the metrics to measure the impact of HeadOn; and (5) clinical outcomes to examine the efficacy of HeadOn.

    Optimization Study 1: Concept and Design Review

    Once intervention planning was complete, the HeadOn intervention components were defined, and the user interface was designed. Feedback was sought at this stage with face-to-face questionnaire-based surveys conducted with National Health Service patients presenting to the ED at the Royal Infirmary of Edinburgh and with a range of health care professionals who are involved in the management of concussions. The intervention concept was explained to the patients and health care professionals, and they were shown figures of the proposed design. The interviewer (LA) went through a structured questionnaire (Multimedia Appendix 1). Each interview lasted approximately 15 minutes.

    Optimization Study 2: Prototype Assessment

    Having completed a prototype of HeadOn, a series of “think-aloud” interviews were conducted with volunteers who had experienced a concussion. Volunteers were recruited from concussion support groups on a social media site and via advertising through a brain injury charity and were invited to participate in the interview, which was conducted by videoconference. During the interviews, participants were given access to the intervention and were asked a series of questions (conducted by CD) about the usability and initial perception of the intervention. Participant comments were recorded, and areas of feedback were implemented.

    Mixed Methods Clinical Feasibility Study

    A prospective mixed methods study was conducted in which participants who had experienced a concussion were given access to HeadOn. This study included both quantitative and qualitative components. The main quantitative study examined participant engagement with HeadOn and a range of clinical outcomes. Participants could also volunteer to take part in a qualitative substudy involving an in-depth interview about their experience with HeadOn. The study ran for a predefined 6-month period between November 2021 and April 2022. Patients presenting to the Edinburgh Royal Infirmary and St John’s Hospital EDs with a concussion were invited to participate. Inclusion criteria were patients aged ≥16 years presenting with a concussion, which was defined according to the American Congress of Rehabilitation Medicine—a traumatically induced disruption of brain function presenting as any alteration of mental status, loss of consciousness, or posttraumatic amnesia [43]. Participants were required to have a Glasgow Coma Scale score of 13 to 15 on initial presentation to the ED and would need to be able to register with HeadOn within 14 days of their concussion. Exclusion criteria were patients requiring surgical management of their cranial injury, substantial other associated injuries requiring hospitalization (spinal injury; fractures; and abdominal, cardiothoracic, or vascular injuries), lack of capacity to provide consent, non-English speakers, and patients in police custody or in prison. To take part in the study, participants gave written informed consent through an electronic consent system. Consent was obtained either while the patient was in the ED or after discharge up to 14 days after their injury. Following recruitment, study participants were taken through the HeadOn registration process, which included an introductory video and the completion of the Rivermead Post-Concussion Symptom Questionnaire [44] and Patient Health Questionnaire-9 (PHQ-9) [45]. In addition, a series of researcher-led anonymized data points was collected, including demographics, date of concussion, and neurological and imaging findings. Participants were then given open access to HeadOn, which ran over a 5-week period. At the completion of HeadOn, participants were invited to complete the Rivermead Post-Concussion Symptom Questionnaire and PHQ-9 again. Participants were also invited to complete the mHealth App Usability Questionnaire (MAUQ) to provide feedback on the usability of HeadOn [46]. As HeadOn did not provide functionality to communicate with a health care provider, the final 4 questions of the MAUQ that focused on this were excluded. There was no control group in this feasibility study. Flow diagrams of the study design and recruitment can be found in Figures S1 and S2 in Multimedia Appendix 1. The study protocol was published on ClinicalTrials.gov (NCT05069948) on October 6, 2021. The CONSORT (Consolidated Standards of Reporting Trials) 2010 checklist of information to include when reporting a pilot or feasibility trial is reported in Multimedia Appendix 1 [47].

    Qualitative Interviews

    As part of the consent process for the mixed methodology study, participants could opt in to take part in the qualitative interviews. Study participants who consented to take part in the qualitative interviews (44/50, 88%) were contacted by email 3 weeks after registration with HeadOn to arrange an interview date. Participants who did not initially respond were contacted once more to organize a qualitative interview. Those who did not respond after 2 attempts were sent a short electronic survey about their attitudes toward HeadOn (no responses were gathered through this method). Interviews were conducted by 1 interviewer (CD—female Master of Science by Research student) via telephone and recorded using an encrypted recording device. The interviews were conducted directly between CD and the interviewee without anyone else participating in the conversation. Average interview time was between 15 and 45 minutes. CD had previous experience in conducting qualitative interviews and had completed additional training in conducting and analyzing qualitative research. The interview schedule can be found in Multimedia Appendix 1. Owing to CD’s close involvement in the design and development of the intervention, careful attention was paid to designing the interview guide to reduce the risk of bias. In addition, it was made clear that CD was interested in the participants’ experience of using (or not using) the intervention, of which they were the experts. The recordings were transcribed by CD within 24 hours. Transcribed interviews were then coded and analyzed using NVivo (version 12; QSR International). Deductive thematic analysis was used to identify barriers to and facilitators of using HeadOn, which were subsequently mapped onto the same domains that were used in the scoping review: patient engagement, information, symptom tracking, lifestyle and behavior, thoughts and emotions, design, and technical aspects. To check for consistency, a second researcher (AABJ) reviewed a random selection (3/15, 20%) of the transcripts and conducted thematic analysis. No significant differences were found between the analyses. The interviews were conducted until data saturation was achieved. Data saturation indicates that, based on the data that have been collected and analyzed so far, no further data collection and analysis is necessary to gain new insights. Our stopping criterion for data saturation was the emergence of no new themes in 3 consecutive interviews having conducted a minimum of 8 interviews. To ensure that no new themes emerged, the first 8 interviews were analyzed and coded. Following this, each new interview was coded and analyzed sequentially until 3 consecutive interviews found no new themes. Data saturation was achieved at 15 interviews and was agreed upon by 2 researchers (CD and AABJ). The COREQ (Consolidated Criteria for Reporting Qualitative Research) guidelines [48] can be found in Multimedia Appendix 1.

    Statistical Analysis

    Engagement with functionality was defined as the inputting of data into a given functionality as part of the HeadOn program. A high engager was defined as a participant who engaged with all the functionalities of HeadOn during the 5-week program. A nonengager was defined as a participant who did not engage with any HeadOn functionality after initial registration. A “clinically significant” change in PHQ-9 score was defined as a change of ≥5 points from baseline [45]. The normality of continuous data was checked using the D’Agostino and Pearson tests. The unpaired Student 2-tailed t test was used for examining parametric data. The Fisher exact test or chi-square test was used to analyze categorical data. For correlation, the Pearson correlation coefficient was calculated. Prism (version 9.3.1; GraphPad Software) was used for the statistical analysis.

    Ethics Approval

    The methods were performed in accordance with relevant guidelines and regulations and approved by the North West - Preston Research Ethics Committee (21/NW/0211) on September 14, 2021. Informed consent was provided by study participants for the clinical feasibility study. Participants did not receive any compensation as part of the study. The data captured as part of the study were kept in accordance with the University of Edinburgh and National Health Service Lothian privacy and data protection requirements. The databases used in this study met industry-standard data protection protocols. Researcher-captured data were anonymous, and the exported data from HeadOn were deidentified and encrypted before analysis.


    Development of the Intervention

    Rapid Scoping Review

    A scoping review was conducted to better understand the factors that prevent or facilitate patient engagement with concussion interventions. A total of 21 studies were included in the rapid scoping review (Figure 1). These studies examined a range of interventions for concussions. The characteristics of the study participants or of the intervention that appeared to affect its success were mapped onto a table of facilitators and barriers across 7 domains (intervention engagement, information, symptom tracking, lifestyle and behavior, thoughts and emotions, design, and technical aspects; Table S1 in Multimedia appendix 1). These findings were used to guide the behavioral analysis and logic model.

    ‎
    Figure 1. The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram for the scoping review.
    Behavioral Analysis

    A behavioral analysis was conducted to address several key barriers identified in the scoping review (Table S2 in Multimedia Appendix 1). The BCW [41] was the theoretical framework used for the development of the intervention. HeadOn aimed to overcome the barriers to the target behaviors (intervention engagement, symptom monitoring, healthy lifestyle habits, addressing negative thoughts, and goal setting) by using 20 behavior change techniques [42] that targeted all 6 sources of behavior (social opportunity, physical opportunity, reflective motivation, automatic motivation, physical capability, and psychological capability) and 8 intervention functions (education, training, persuasion, incentivization, enablement, environmental restructuring, restriction, and modeling). For example, forgetfulness because of the cognitive disturbance following a concussion was one of the barriers identified during the scoping review. By providing the participants with information on how to set up a routine around accessing HeadOn, both psychological capability and automatic motivation were targeted. Physical opportunity was targeted with the use of automated emails that prompted the participants to log in.

    Logic Model

    A logic model was developed to map out the mechanisms underpinning the intervention (Figure 2). This consisted of 5 parts. First, we identified the problem regarding postconcussion symptoms having a negative impact on participants’ functional outcomes. Second, we defined the intervention targets, which covered 2 areas: positive health behaviors (such as symptom monitoring, physical activity, and good sleep hygiene) and improved mental health (through CBT and relaxation techniques). Third, we laid out the intervention ingredients, which included the following: health information, self-monitoring, goal setting, engagement enhancement, and CBT and mindfulness techniques. Fourth, we defined the mechanisms that we would need to examine to determine the impact of the intervention. This included the participants’ engagement with and their attitudes toward the intervention and any associated behavior change. Finally, we laid out the outcome measures to determine the efficacy and effectiveness of the intervention. The primary outcome was an improvement in functional outcomes following concussion, and secondary outcomes included reduction in postconcussion symptoms and improvement in mental health and sleep.

    ‎
    Figure 2. HeadOn logic model. CBT: cognitive behavioral therapy; mHealth: mobile health.
    Design of the Digital Health Intervention

    The scoping review, behavioral analysis, and logic model were fed into the design of the HeadOn intervention components. HeadOn was designed to include 5 stages that run sequentially, with each stage lasting 7 days (total duration of 5 weeks). The five stages of HeadOn are (1) Understanding your symptoms, (2) Sleep after a concussion, (3) Lifestyle and exercise, (4) Your thoughts, and (5) Getting back to baseline (Table 1). The sequence of these stages was chosen for a number of reasons. First, we chose to examine postconcussion symptoms and sleep early as this provided the user with early knowledge of what to expect with their recovery and the type of symptoms that they may encounter. Second, we chose to have the lifestyle and exercise stage in the third week as major sporting body regulations commonly recommend that physical activity can start 2 weeks after a concussion [49]. At the start of HeadOn, an introductory video explains what HeadOn is and how to use it. At the start of each stage, an introductory audio explains the stage (accessible on the HeadOn home page), outlining important knowledge for the user and the task for that week. For each stage, the user is invited to complete a task using a piece of HeadOn functionality. All these tasks are one-off tasks except for the completion of the Symptom Diary, which the user is invited to complete every day to allow them to track their recovery. Data input into the Symptom Diary can be tracked visually through the HeadOn Progress Tracker.

    Table 1. HeadOn intervention components.
    Intervention componentDescriptionTask
    OverviewHeadOn is a digital health intervention designed to support patients who have experienced a concussion. It contains 5 stages, each of which lasts 7 days. Each stage is introduced with an audio clip providing the patient with information about the stage and associated tasks.Watch introductory video (one-off task)
    IntroductionAn introductory video explains what HeadOn is and how to use it. The patient is then invited to complete a series of questionnaires (Rivermead Post-Concussion Symptom Questionnaire and PHQ-9a).Complete Rivermead questionnaire and PHQ-9 (one-off task)
    Understanding your symptomsThe first week of HeadOn focuses on providing patients with information about postconcussion symptoms and techniques for managing them. This includes access to health information (including symptom-specific information and also frequently asked questions). The patient is also invited to complete a daily symptom diary and is able to view their data in the Progress Tracker area.Complete symptom diary (recurring daily task)
    Sleep after a concussionThe second week of HeadOn focuses on sleep disturbance after a concussion. The audio introduction provides the patient with important information on good sleep hygiene. The patient is also invited to set a wake-up time and bedtime for the week. They are notified by HeadOn when this time comes.Set wake-up and sleep times (one-off task)
    Lifestyle and exerciseThe third week of HeadOn focuses on 2 areas: physical activity and examining alcohol consumption. All patients are invited to set 3 days of the week to perform noncontact physical activity. If the patient indicates that they had consumed alcohol at the time of the concussion, they are invited to set 3 alcohol-free days of the week and use an alcohol tracker to monitor the number of alcoholic units that they drink throughout the week.Set exercise days (one-off task); set alcohol-free days and use alcohol tracker (one-off tasks)
    Your thoughtsThe fourth week of HeadOn focuses on examining the patient’s thoughts regarding their concussion. The audio introduction discusses CBTb concepts regarding the role of thoughts on behavior and emotion. The patient is invited to use a thought diary to explore their thoughts regarding their concussion.Complete the thought monitor (one-off task)
    Getting back to baselineThe final week of HeadOn focuses on supporting the patient to return to their preinjury function. During this week, the patient is invited to set a goal to complete by the end of the program. They are encouraged to use the SMARTc approach to set and complete the goal. The patient uses the HeadOn goal-setter function, which provides reminders throughout the week.Set goal using HeadOn goal setter (one-off task)
    CompletionOn completion of the HeadOn program, the patient is invited to retake the same series of questionnaires (Rivermead Post-Concussion Symptom Questionnaire and PHQ-9).Complete Rivermead questionnaire and PHQ-9 (one-off task)

    aPHQ-9: Patient Health Questionnaire-9.

    bCBT: cognitive behavioral therapy.

    cSMART: Specific, Measurable, Achievable, Relevant, and Time-Bound.

    Optimization Study 1: Concept and Design Review

    We sought stakeholder feedback by conducting a series of semistructured interviews in which the HeadOn concept (including proposed intervention components and functionality) and designs were discussed. A total of 19 interviews were conducted with 8 (42%) individuals who had presented to a local ED with a concussion and 11 (58%) health care professionals (including specialists in neuropsychology, neurosurgery, neurology, sports medicine, general practice, and physiotherapy). Across the interviewed cohort, the HeadOn concept was viewed favorably by 74% (14/19; 6/8, 75% of individuals with a concussion and 8/11, 73% of health care professionals) of the respondents, considering it a “very good-to-excellent” concept (Table S3 in Multimedia Appendix 1). Of the proposed functionalities, those focused on symptom monitoring and management (symptom diary, progress tracker, and symptom support) were viewed most favorably. The progress tracker was viewed favorably by 79% (15/19) of the respondents (6/8, 75% of individuals with a concussion and 9/11, 82% of health care professionals), the symptom support area was viewed favorably by 74% (14/19) of the respondents (5/8, 62% of individuals with a concussion and 9/11, 82% of health care professionals), and the Symptom Diary was viewed favorably by 53% (10/19) of the respondents (5/8, 62% of individuals with a concussion and 5/11, 45% of health care professionals). Over half (11/19, 58%; 4/8, 50% of individuals with a concussion and 7/11, 64% of health care professionals) of the interview respondents thought that the HeadOn design was “very good-to-excellent.” A respondent commented on the importance of using colors wisely, suggesting that warm colors were easier on the eye. Of the 8 participants who had experienced a concussion, 7 (88%) said that they would have been interested in using HeadOn if it had been available at the time. Potential barriers to the use of HeadOn that were highlighted included the need for screen time, which can be difficult for participants who have had a concussion. Having to pay to use HeadOn and a lack of evidence of its efficacy were also identified as factors that could reduce uptake and engagement.

    Optimization Study 2: Prototype Assessment

    Following the development of the intervention prototype, user testing was then conducted through “think-aloud” interviews. Think-aloud interviews provided information to understand participants’ thoughts as they interacted with HeadOn by having them think aloud while they used the intervention. A total of 4 think-aloud interviews were conducted with stakeholders. The interviews were conducted on the web using videoconference, and participants were recruited through social media support forums for concussion and via advertising through a brain injury charity. During the interviews, participants were given access to the intervention, asked a series of questions about their initial perception of the intervention, and given a series of tasks to perform. Overall, perceptions of HeadOn were positive regarding its structure and layout. Some elements of the intervention were described as difficult to locate on the application, such as a progress tracker. To address this, an explainer video was included at the start of the registration process providing information on the layout of the intervention. In conjunction, a reminder for the user to review their progress within the task section of HeadOn that linked through to the progress tracker was included. Another piece of feedback was that it was not obvious to the user that they had to listen to the audio clip at the start of each week. We addressed this in 2 ways: through the introduction of an explainer video and by redesigning the home page to ensure that the audio clip was a central feature and visually obvious to the user. One user felt strongly that using smiley faces for the Likert scale in the symptom diary was inappropriately childish, so these were changed to a numerical scale (0-4). Another user noted the lack of “back buttons” in the application, which made navigation difficult. These were also added to improve the user experience.

    Mixed Methods Clinical Feasibility Study

    Study Overview

    A total of 50 participants presenting to the ED with a concussion were recruited for the study (Table 2). The cohort had an average age of 41.9 (SD 16) years, and 54% (27/50) were female. The most common mechanism of injury was a fall (26/50, 52%), and 42% (21/50) of the participants had consumed alcohol around the time of injury. Loss of consciousness at the time of the concussion was reported by 50% (25/50) of the participants. A total of 28 participants had a brain computer tomography scan, of whom 9 (32%) were found to have a radiological abnormality. These findings included 33% (3/9) cerebral contusions, 33% (3/9) skull fractures, 22% (2/9) traumatic subarachnoid hemorrhages, and 11% (1/9) extradural hematomas. Of the 50 participants, 5 (10%) were admitted to the hospital for a median of 1 (range 1-6) day. At registration with HeadOn, the average Rivermead Post-Concussion Questionnaire score was 31 (SD 13), and the average PHQ-9 score was 13 (SD 7). There were no reported harms during the feasibility study.

    Table 2. Participant characteristics in the clinical feasibility study (N=50).
    CharacteristicsValues
    Age (years), mean (SD; range)42 (16; 18-73)
    Sex, n (%)

    		Female27 (54)

    		Male23 (46)
    Employment status, n (%)

    		Full-time employment22 (44)

    		Unemployed18 (36)

    		Unknown7 (14)

    		Part-time employment3 (6)
    Alcohol consumption, n (%)

    		No29 (58)

    		Yes21 (42)
    Mechanism, n (%)

    		Fall26 (52)

    		Assault11 (22)

    		Sports-related10 (20)

    		Road traffic collision2 (4)

    		Other1 (2)
    Loss of consciousness, n (%)

    		Yes25 (50)

    		No14 (28)

    		Unknown11 (22)
    Presentation GCSa, n (%)

    		1542 (84)

    		146 (12)

    		131 (2)

    		Unknown1 (2)
    Rivermead Post-Concussion Symptom Questionnaire score, mean (SD; range)31 (13; 0-50)
    PHQ-9b score, mean (SD; range)13 (7; 0-25)

    aGCS: Glasgow Coma Score.

    bPHQ-9: Patient Health Questionnaire-9.

    Participant Intervention Engagement and Quantitative Feedback

    Participant engagement was quantified by examining the use of the 6 core functionality features of HeadOn. Across the 50 participants, engagement with the functionalities was 90% (45/50) for the symptom diary, 62% (31/50) for sleep time setting, 56% (28/50) for the alcohol tracker, 48% (24/50) for exercise day setting, 34% (17/50) for the thought diary, and 32% (16/50) for the goal setter. Of the 50 study participants, 13 (26%) could be classified as high engagers who used every function within HeadOn. Conversely, 8% (4/50) were nonengagers who did not use any of the functionalities after registering. High engagers did not differ significantly from other participants with regard to their age, sex, or Rivermead and PHQ-9 scores at registration. Study participants were invited to complete the symptom diary every day for the 5 weeks. Completion of the symptom diary started high on the first day of HeadOn (43/50, 86%) but then diminished rapidly over the course of the 5 weeks (Figure 3A). Throughout the 5 weeks, a total of 494 symptoms diaries were completed. The average daily completion rate of the symptom diary was 28.23% (494/1750). Upon completion of HeadOn, participants were invited to complete the MAUQ [46]. The questionnaire quantifies respondents’ perceptions of the usability of a mobile health app using a range from 1 to 7. A score of 7 indicates a high degree of usability. A total of 58% (29/50) of the respondents provided feedback on HeadOn using the MAUQ. From these responses, HeadOn had an average score of 6.1 (SD 1.3), indicating a high degree of usability. The MAUQ is composed of 3 domains for which HeadOn obtained the following average scores: 6.2 (SD 1.2) for Ease of Use and Satisfaction, 6.0 (SD 1.4) for System Information Arrangement, and 6.2 (SD 1.1) for Usefulness (Table S4 in Multimedia Appendix 1).

    ‎
    Figure 3. (A) The percentage of study participants completing the symptom diary. (B) Distribution of the symptom burden based on 494 symptom diary responses. (C) Temporal change in symptom burden over the course of HeadOn.
    Qualitative Interviews

    A total of 15 participants took part in the qualitative interviews. The participants had a mean age of 47 years, and 67% (10/15) were female. A comparison of the interviewed and noninterviewed cohorts revealed a statistically significantly higher percentage of high engagers in the interviewed group (P=.01; Table 3). The interviews were transcribed and then coded according to themes. Overall, there was a high level of satisfaction, which was consistent with the quantitative feedback from the MAUQ. Participants referred to HeadOn as “easy to use” and “straightforward.” Facilitators of and barriers to using HeadOn were identified and mapped onto the HeadOn intervention characteristics (Table 4). A major facilitator of engagement with HeadOn cited by interviewees was setting up a routine to access HeadOn by logging in at the same time each day alongside another daily activity (brushing teeth, taking medication, or going to bed), saving the app to their home page, or saving HeadOn emails to their inbox. For many participants, a high symptom burden following their injury negatively affected their physical and psychological capabilities. As a result, participants reported struggling with registration and the digital format of the intervention. In addition, some participants were advised to reduce their use of devices and screens. However, all participants were able to use the intervention successfully after the first week. Several participants who reported this difficulty at the beginning had assistance from their families or support network. Participants reported benefitting from the multiple options of either audio or text files to receive the information. Table S5 in Multimedia Appendix 1 includes the barriers and facilitators mapped onto the Capability, Opportunity, and Motivation–Behavior framework from the BCW linked to the individual interview responses. When examining how to improve HeadOn, most participants stated that they would have benefitted from an increased number of prompts and cues, such as emails, SMS text messages, and notifications. The inability to remove units from the alcohol tracker was problematic as several interviewees added accidental units that they wished to deduct. Several interviewees disliked the language used in the thought monitor. The feedback included observations that the thought monitor put too much focus on the concussion incident rather than on recovery and provided a negatively biased range of emotions (eg, anger, disgust, and anxiety) to pick from. Table S6 in Multimedia Appendix 1 contains the key qualitative interview feedback (including selected quotations) and associated changes made to HeadOn.

    Table 3. Comparison of interviewed and noninterviewed cohorts in the feasibility study (N=50).
    CharacteristicsInterviewed (n=15)Noninterviewed (n=35)P value
    Age (years), mean (SD)47.2 (15.7)39.7 (16.3).14
    Sex, n (%).35

    		Female10 (67)17 (49)

    		Male5 (33)18 (51)
    Employment status, n (%).29

    		Full-time employment7 (47)15 (43)

    		Unemployed7 (47)11 (31)

    		Unknown0 (0)7 (20)

    		Part-time employment1 (7)2 (6)
    Alcohol consumption, n (%).21

    		No11 (73)18 (51)

    		Yes4 (27)17 (49)
    Mechanism, n (%).30

    		Fall5 (33)21 (60)

    		Assault4 (27)7 (20)

    		Sports-related4 (27)6 (17)

    		Road traffic collision1 (7)1 (3)

    		Other1 (7)0 (0)
    Engagement, n (%).01

    		High engager8 (53)5 (14)

    		Non–high engager7 (47)30 (86)
    Rivermead Post-Concussion Symptom Questionnaire score, mean (SD)31 (13)31 (14).85
    PHQ-9a score, mean (SD)14 (9)13 (7).69

    aPHQ-9: Patient Health Questionnaire-9.

    Table 4. Barriers to and facilitators of using HeadOn based on qualitative interviews.
    Intervention characteristicFacilitatorsBarriers
    Intervention engagement
    • Family support
      • ‎
    • Weekly email reminders
      • ‎
    • Logging in via emails
      • ‎
    • Building a routine
      • ‎
    • Setting own reminder
      • ‎
    • Easy to use
      • ‎
    • Symptom burden
      • ‎
    • Motivation to recover and belief intervention was supporting recovery
      • ‎
    • Credible source
      • ‎
    • Days in a row
      • ‎
    • Flexibility of intervention being digital and self-directed
      • ‎
    • COVID-19 pandemic improved digital literacy
      • ‎
    • Being part of academic study
      • ‎
    • Saved to home screen on device
      • ‎
    • Forgetfulness and memory
      • ‎
    • High symptom burden after concussion
      • ‎
    • Difficulty with screens and devices or digital
      • ‎
    • Low symptom burden
      • ‎
    • Difficulty registering
      • ‎
    • Work or other commitments
      • ‎
    • Not enough prompts to log in (notifications, emails, SMS text messages, and calendar reminders)
      • ‎
    • Not a native app and log-in process
      • ‎
    Information
    • Credible source
      • ‎
    • Audio or transcript available
      • ‎
    • Difficulty with screens
      • ‎
    • Difficulty concentrating
      • ‎
    Symptom monitoring
    • Days in a row
      • ‎
    • Ability to review progress
      • ‎
    • Repetitive
      • ‎
    • Main symptom not included
      • ‎
    • Progress review limited to 7 days
      • ‎
    Exercise and lifestyleN/Aa
    • Alcohol tracker lacking ability to go back or take off units
      • ‎
    Thought monitorN/A
    • Language used did not resonate with participants or they did not feel it suited their situation
      • ‎
    • Memory loss of injury
      • ‎
    • Repetitive task
      • ‎
    • Emotions not an issue
      • ‎
    Technical and design
    • Easy to navigate
      • ‎
    • Simple design
      • ‎
    • Instructions
      • ‎
    • Audio or text files
      • ‎
    • Registration
      • ‎
    • Difficulty logging in
      • ‎
    • Difficulty with screens
      • ‎

    aN/A: not applicable.

    Participant Outcomes on Completion of the Intervention

    Upon completion of HeadOn, a series of participant outcomes were collected. In the cohort of 50 participants, a functional outcome using the Glasgow Outcome Scale Extended (GOSE) was collected for 58% (29/50). Of these 29 participants, 17 (59%) had a GOSE score of 8, indicating a complete functional recovery upon completion of HeadOn. A total of 44% (22/50) of the participants took the Rivermead Post-Concussion Symptom Questionnaire and PHQ-9 at completion. The average Rivermead Post-Concussion Symptom Questionnaire and PHQ-9 scores were 16 (SD 13) and 8 (SD 7), respectively. Among the 22 participants with pre- and postintervention scores, there was a statistically significant improvement in the Rivermead score upon completion of HeadOn (preintervention score mean 31, SD 2.4; postintervention score mean 16, SD 2.7; P<.001), but this was not the case for the PHQ-9 score (preintervention score: mean 11.5, SD 7.5; postintervention score: mean 8, SD 7.0; P=.12). Saying that, of these 22 participants, 8 (36%) had a “clinically significant” change of ≥5 points between the start and finish of HeadOn. Of these 22 participants, 7 (32%) had a clinically significant improvement in their PHQ-9 scores, and 1 (5%) had a worsening score. Participants with a complete functional recovery (GOSE score of 8) had statistically significantly lower Rivermead (mean 7, SD 6 vs mean 25, SD 14; P=.008) and PHQ-9 (mean 4, SD 3 vs mean 13, SD 8; P=.02) scores compared with those who did not have a complete recovery at the end of HeadOn. Of the 29 participants who responded, 10 (34%) represented to a health care professional because of their postconcussion symptoms. Of the 25 participants who were known to be in employment before the concussion, 10 (40%) had returned to work by the end of HeadOn.

    Postconcussion Symptom Burden and Temporal Profile

    A total of 494 symptom diaries were completed by 90% (45/50) of the study participants during the 5-week HeadOn program. On average, 11 entries were completed per participant, with a range of 1 to 34. The symptom diary allowed participants to rate the severity of 8 postconcussion symptoms from 0 (none) to 4 (severe), giving each symptom diary a cumulative range from 0 to 32. Across the 494 completed diaries, the average postconcussion symptom burden was 12 (SD 8). Fatigue, sleep disturbance, and difficulty concentrating were the top 3 symptoms based on overall severity (Figure 3B). As detailed previously, engagement with the symptom diary diminished over time, with a peak of 43 entries on day 1. The average number of entries per day was 14 (SD 7). Daily symptom burdens were highly variable, but the average score diminished over the 5-week period from a peak of 14 (SD 7) in the first week to 7 (SD 5) in the final week (Figure 3C). For the 22 participants with Rivermead scores on completion of HeadOn, there was a strong positive correlation (r=0.86; P<.001) between their average daily HeadOn symptom diary score and their end-of-program Rivermead score.


    Principal Findings

    Concussions are a major public health issue [3]. Although historically viewed as benign, there is a growing understanding that individuals experience a constellation of symptoms and can be left with substantially impaired function [5,50]. This has led to calls for a more systematic and targeted approach to follow up on and support individuals who have sustained a concussion [3]. In reality, many individuals who have experienced a concussion have limited information or follow-up after they are discharged from an ED [10,11]. To address this problem, we developed HeadOn—a digital health intervention specifically designed to support individuals in their concussion recovery. We aimed to incorporate evidence-based educational, behavioral, and psychological interventions that have been found to improve outcomes in concussion [51]. This included early educational material, which Ponsford et al [12] found to significantly reduce postconcussion symptom burden in a randomized controlled trial. We also incorporated elements of CBT, which several trials have demonstrated to have a positive impact on postconcussion symptom burden and recovery. Mittenberg et al [13] examined the effects of an early single session of CBT and found that it significantly reduced the duration of postconcussion symptoms and led to fewer symptoms at the 3-month follow-up. Potter et al [52] examined the delivery of CBT to patients with persistent postconcussion symptoms and found that CBT led to a significant reduction in symptom burden. In addition, we included the introduction of exercise as a recent systematic review found that symptom-limited aerobic exercise has a significant beneficial effect on symptomatic recovery compared with controls [14]. Finally, we also incorporated interventions that have been found to be effective in other diseases and, theoretically, should be effective for concussion, including symptom monitoring [53], alcohol reduction [54], and breathing exercises [55].

    HeadOn was developed using a systematic evidence-, theory-, and person-based approach based on the MRC guidance on the development of complex interventions [19]. This approach has been used by others to develop digital health interventions. Bradbury et al [56] developed a digital intervention for cancer survivors. We used a similar approach to that of Bradbury et al [56] but structured the development process to align with the latest MRC guidance, which was recently published [19]. This framework is divided into 4 phases: development of the intervention, feasibility, evaluation, and implementation. In this paper, we describe the first 2 phases—development and feasibility testing. As part of the development process, we synthesized the literature examining what the facilitators of and barriers to using concussion-related interventions were. This information was then used to identify potential barriers to behavior change to address during the behavioral analysis. Some of the key barriers identified during the scoping review were forgetfulness, lack of time, skepticism regarding the ability to fully recover, and lack of motivation. We also identified several contradictory barriers and facilitators, such as minimal in-person contact being both a barrier to and a facilitator of using concussion interventions. This highlights the range of different attitudes in individuals who have experienced a concussion given their demographic breadth and differing recovery trajectories. Importantly, it demonstrates the need to develop a flexible intervention that can fit different use patterns. The BCW was then used as the theoretical model for intervention development [41]. This allowed for mapping of the behavioral analysis onto the BCW and the taxonomy of behavior change techniques [42]. The logic model then provided a framework to conceptualize how HeadOn would work and affect patient outcomes. We then used a person-based approach and conducted optimization studies to understand patient needs and views, which were incorporated into the design to maximize engagement [57]. As recommended in the MRC complex intervention framework, refinement of the intervention should happen after each stage. Therefore, a major component of our mixed methods feasibility study was to obtain feedback from patients who used HeadOn. Quantitative feedback using the MAUQ was broadly very positive, indicating high levels of usability and satisfaction among responders [46]. The MAUQ has been recently introduced and so has not yet gained widespread uptake; however, HeadOn has similar levels of usability compared with other digital interventions in the literature, including those targeting childhood feeding and pediatric burns [58,59]. The qualitative interviews also highlighted some important areas for refinement, including increasing the number of notifications, adding a remove unit function to the alcohol tracker, and altering the language used in the thought monitor. Along with these areas for improvement, difficulty looking at the screen was also mentioned by several interviewees. Concussion guidance is to limit screen time early after an injury. A clinical trial looking at this question found that patients who abstained from screen time for the first 48 hours after their injury had a shorter recovery than those permitted to engage in screen time [60]. Although HeadOn is delivered mostly through a screen (mobile or desktop), there is the option of consuming some of the content through audio. Coupled with this, in the trial, the screen time–abstinent group had 130 minutes of screen time in the first 3 days after their injury. HeadOn can be delivered with limited screen time (5-10 minutes per day), which could easily be achieved within the 130 minutes of the intervention group quoted in the screen time RCT.

    Engagement with digital interventions is an important area and, therefore, we aimed to examine how patients engaged with HeadOn during the feasibility study. We found that the initial high levels of engagement trailed off rapidly during the first week. The driving force behind engagement after this was a group of high engagers who constituted approximately one-quarter of the cohort (13/50, 26%). Chien et al [61] examined the issue of user engagement in a cohort of 54,604 users of an internet-delivered mental health intervention. In total, 5 categories of engagers were identified: low engagers (36.5%), late engagers (21.4%), high engagers with rapid disengagement (25.5%), high engagers with moderate decrease (6.0%), and highest engagers (10.6%). Even though Chien et al [61] used a more nuanced definition of engagement, the finding of higher engagers, consisting of approximately one-third of the cohort, broadly fits with the findings of this study. One of the key aspects of patient engagement with HeadOn was the completion of the symptom diary. During the feasibility study, a total of 494 diaries were completed. These data provided interesting insights into postconcussion symptom burden and recovery trajectories. Our study found that physical symptoms such as headache and dizziness were some of the least problematic symptoms. This is in contrast to a prospective study by McMahon et al [5], who found physical symptoms to be the most common in a cohort of 348 patients. The same study found a gradual decline in the Rivermead Post-Concussion Symptom Questionnaire score over time between 3 and 6 months. Although we did not follow up on patients for as long as 6 months, we found a gradual decline in their average daily symptom burden over the 5-week duration of HeadOn. Despite the natural history of postconcussion symptoms to decline over time, there is a substantial percentage of patients who are left with persistent postconcussion symptoms. Cnossen et al [4] found that, in a cohort of 591 patients, 41% had persistent postconcussion symptoms at 6 months after their injury. The authors found that the main predictors of developing persistent symptoms included female gender, postconcussion symptoms at 2 weeks, and posttraumatic stress at 2 weeks. One of the key difficulties in implementing predictive models such as this one into routine care is that patients who experience concussion are not commonly followed up on [10,11]. HeadOn addresses this problem by collecting symptom data digitally and could be cost-effectively scaled. These data could then be used to stratify patients and deliver personalized support.

    Study Limitations

    This study has several limitations. This includes a low number of participants in the optimization studies during development compared with other authors [56]. We attempted to address this by designing the feasibility study with person-based methodology and conducting qualitative interviews to gain feedback on HeadOn. These interviews, though highly useful, were also prone to bias because of the substantially higher proportion of high engagers. We attempted to reach out to patients who engaged less with the intervention but were unable to receive feedback from them. During recruitment for the feasibility study, the demographic data and the reasons why patients declined participation were not recorded. Therefore, we were unable to determine whether there was recruitment bias in the feasibility study, in particular whether this bias may have contributed to the digital health divide [62]. Finally, our definition of engagement was based on the metric of the patient inputting data into HeadOn. We would have missed patients who accessed the application without inputting data. Some authors argue that these types of basic engagement metrics are too blunt and advocate for the use of measures of cognitive investment to obtain a true impression of engagement with digital interventions [63].

    Conclusions

    In this paper, we describe the development of a digital health intervention for concussion using a rigorous evidence-, theory-, and person-based approach. Behavioral theory was used to optimize the intervention to encourage positive behavior change during recovery from concussion. Using the person-based approach, the intervention was optimized through multiple rounds of feedback, which led to participants reporting high levels of usability during the feasibility study. Symptom data input as part of the intervention provided interesting insights into postconcussion symptom burden and, in the future, will provide a means to better target patients at risk of persistent symptoms who require more support. This work lays a robust foundation that supports the progression to the evaluation and implementation phases of the MRC complex intervention development framework.

    Acknowledgments

    This research was supported by the National Institute for Health and Care Research Brain Injury MedTech Co-operative based at Cambridge University Hospitals National Health Service (NHS) Foundation Trust and the University of Cambridge. The views expressed are those of the authors and not necessarily those of the NHS, the National Institute for Health and Care Research, or the Department of Health and Social Care. MJR and DCG are supported by NHS Research Scotland Career Researcher Clinician awards. The authors would like to extend their thanks to all the study participants who were generous with their time and who provided their feedback and thoughts on how to improve HeadOn. Special thanks to the Headway brain injury charity for helping promote the study on their website. The authors would also like to extend thanks to Mr Paul Brennan for his support in setting up and delivering the clinical feasibility study and to the Emergency Medicine Research Group Edinburgh research nurses for clinical feasibility study recruitment.

    Data Availability

    The clinical feasibility study data are freely available from the University of Edinburgh Datashare [64].

    Conflicts of Interest

    CD, AABJ, and DCG are shareholders in HeadOn Health Ltd, which has an exclusive license for the HeadOn intellectual property from the University of Edinburgh. AC is a paid editor for JNNP, an unpaid president of FNDsociety and gave testimony in Court on a range of neuropsychiatric topics including TBI.

    ‎
    Multimedia Appendix 1

    Supplementary material.

    PDF File (Adobe PDF File), 954 KB
    1. Taylor CA, Bell JM, Breiding MJ, Xu L. Traumatic brain injury-related emergency department visits, hospitalizations, and deaths - United States, 2007 and 2013. MMWR Surveill Summ 2017 Mar 17;66(9):1-16 [FREE Full text] [CrossRef] [Medline]
    2. Cassidy JD, Carroll LJ, Peloso PM, Borg J, von Holst H, Holm L, WHO Collaborating Centre Task Force on Mild Traumatic Brain Injury. Incidence, risk factors and prevention of mild traumatic brain injury: results of the WHO Collaborating Centre Task Force on Mild Traumatic Brain Injury. J Rehabil Med 2004 Feb(43 Suppl):28-60 [FREE Full text] [CrossRef] [Medline]
    3. Levin HS, Diaz-Arrastia RR. Diagnosis, prognosis, and clinical management of mild traumatic brain injury. Lancet Neurol 2015 May;14(5):506-517. [CrossRef] [Medline]
    4. Cnossen MC, van der Naalt J, Spikman JM, Nieboer D, Yue JK, Winkler EA, et al. Prediction of persistent post-concussion symptoms after mild traumatic brain injury. J Neurotrauma 2018 Nov 15;35(22):2691-2698. [CrossRef] [Medline]
    5. McMahon P, Hricik A, Yue JK, Puccio AM, Inoue T, Lingsma HF, TRACK-TBI Investigators. Symptomatology and functional outcome in mild traumatic brain injury: results from the prospective TRACK-TBI study. J Neurotrauma 2014 Jan 01;31(1):26-33 [FREE Full text] [CrossRef] [Medline]
    6. Emanuelson I, Andersson Holmkvist E, Björklund R, Stålhammar D. Quality of life and post-concussion symptoms in adults after mild traumatic brain injury: a population-based study in western Sweden. Acta Neurol Scand 2003 Nov;108(5):332-338. [CrossRef] [Medline]
    7. Stålnacke BM. Community integration, social support and life satisfaction in relation to symptoms 3 years after mild traumatic brain injury. Brain Inj 2007 Aug;21(9):933-942. [CrossRef] [Medline]
    8. Graff HJ, Siersma V, Møller A, Kragstrup J, Andersen LL, Egerod I, et al. Labour market attachment after mild traumatic brain injury: nationwide cohort study with 5-year register follow-up in Denmark. BMJ Open 2019 Apr 11;9(4):e026104 [FREE Full text] [CrossRef] [Medline]
    9. Hartvigsen J, Boyle E, Cassidy JD, Carroll LJ. Mild traumatic brain injury after motor vehicle collisions: what are the symptoms and who treats them? A population-based 1-year inception cohort study. Arch Phys Med Rehabil 2014 Mar;95(3 Suppl):S286-S294. [CrossRef] [Medline]
    10. Seabury SA, Gaudette É, Goldman DP, Markowitz AJ, Brooks J, McCrea MA, TRACK-TBI Investigators, et al. Assessment of follow-up care after emergency department presentation for mild traumatic brain injury and concussion: results from the TRACK-TBI study. JAMA Netw Open 2018 May 18;1(1):e180210 [FREE Full text] [CrossRef] [Medline]
    11. Foks KA, Cnossen MC, Dippel DW, Maas AI, Menon D, van der Naalt J, et al. Management of mild traumatic brain injury at the emergency department and hospital admission in Europe: a survey of 71 neurotrauma centers participating in the CENTER-TBI study. J Neurotrauma 2017 Sep 01;34(17):2529-2535. [CrossRef] [Medline]
    12. Ponsford J, Willmott C, Rothwell A, Cameron P, Ayton G, Nelms R, et al. Impact of early intervention on outcome after mild traumatic brain injury in children. Pediatrics 2001 Dec;108(6):1297-1303. [CrossRef] [Medline]
    13. Mittenberg W, Tremont G, Zielinski RE, Fichera S, Rayls KR. Cognitive-behavioral prevention of postconcussion syndrome. Arch Clin Neuropsychol 1996;11(2):139-145. [Medline]
    14. Langevin P, Frémont P, Fait P, Dubé MO, Bertrand-Charette M, Roy JS. Aerobic exercise for sport-related concussion: a systematic review and meta-analysis. Med Sci Sports Exerc 2020 Dec;52(12):2491-2499. [CrossRef] [Medline]
    15. Kontos AP, Jorgensen-Wagers K, Trbovich AM, Ernst N, Emami K, Gillie B, et al. Association of time since injury to the first clinic visit with recovery following concussion. JAMA Neurol 2020 Apr 01;77(4):435-440 [FREE Full text] [CrossRef] [Medline]
    16. Espie CA, Kyle SD, Williams C, Ong JC, Douglas NJ, Hames P, et al. A randomized, placebo-controlled trial of online cognitive behavioral therapy for chronic insomnia disorder delivered via an automated media-rich web application. Sleep 2012 Jun 01;35(6):769-781 [FREE Full text] [CrossRef] [Medline]
    17. Ebert DD, Zarski AC, Christensen H, Stikkelbroek Y, Cuijpers P, Berking M, et al. Internet and computer-based cognitive behavioral therapy for anxiety and depression in youth: a meta-analysis of randomized controlled outcome trials. PLoS One 2015 Mar 18;10(3):e0119895 [FREE Full text] [CrossRef] [Medline]
    18. Christopher E, Alsaffarini KW, Jamjoom AA. Mobile health for traumatic brain injury: a systematic review of the literature and mobile application market. Cureus 2019 Jul 10;11(7):e5120 [FREE Full text] [CrossRef] [Medline]
    19. Skivington K, Matthews L, Simpson SA, Craig P, Baird J, Blazeby JM, et al. A new framework for developing and evaluating complex interventions: update of Medical Research Council guidance. BMJ 2021 Sep 30;374:n2061 [FREE Full text] [CrossRef] [Medline]
    20. Ahmed OH, Schneiders AG, McCrory PR, Sullivan SJ. Sport concussion management using Facebook: a feasibility study of an innovative adjunct "iCon". J Athl Train 2017 Apr;52(4):339-349 [FREE Full text] [CrossRef] [Medline]
    21. Soeker MS, Van Rensburg V, Travill A. Are rehabilitation programmes enabling clients to return to work? Return to work perspectives of individuals with mild to moderate brain injury in South Africa. Work 2012;43(2):171-182. [CrossRef] [Medline]
    22. Thastum MM, Rask CU, Naess-Schmidt ET, Jensen JS, Frederiksen OV, Tuborgh A, et al. Design of an early intervention for persistent post-concussion symptoms in adolescents and young adults: a feasibility study. NeuroRehabilitation 2018;43(2):155-167. [CrossRef] [Medline]
    23. Hunt AW, De Feo L, Macintyre J, Greenspoon D, Dick T, Mah K, et al. Development and feasibility of an evidence-informed self-management education program in pediatric concussion rehabilitation. BMC Health Serv Res 2016 Aug 17;16(1):400 [FREE Full text] [CrossRef] [Medline]
    24. Hunt TN, Harris LL. Psychological impact of cognitive rest following sport-related concussion on low socioeconomic status adolescent patients. J Allied Health 2017;46(4):e81-e83. [Medline]
    25. Snell DL, Martin R, Surgenor LJ, Siegert RJ, Hay-Smith EJ, Melzer TR, et al. Wrestling with uncertainty after mild traumatic brain injury: a mixed methods study. Disabil Rehabil 2020 Jul;42(14):1942-1953. [CrossRef] [Medline]
    26. Patel SJ, Trowbridge CA. Caregivers of youth athletes: developing a theory-based health communication intervention for concussion education. J Appl Behav Res 2017 Apr 24;22(4):e12098 [FREE Full text] [CrossRef]
    27. Brockway JA, St De Lore J, Fann JR, Hart T, Hurst S, Fey-Hinckley S, et al. Telephone-delivered problem-solving training after mild traumatic brain injury: qualitative analysis of service members' perceptions. Rehabil Psychol 2016 Aug;61(3):221-230. [CrossRef] [Medline]
    28. Brunger H, Ogden J, Malia K, Eldred C, Terblanche R, Mistlin A. Adjusting to persistent post-concussive symptoms following mild traumatic brain injury and subsequent psycho-educational intervention: a qualitative analysis in military personnel. Brain Inj 2014;28(1):71-80. [CrossRef] [Medline]
    29. Hunt AW, Laupacis D, Kawaguchi E, Greenspoon D, Reed N. Key ingredients to an active rehabilitation programme post-concussion: perspectives of youth and parents. Brain Inj 2018;32(12):1534-1540. [CrossRef] [Medline]
    30. Yost TL, Taylor AG. Qigong as a novel intervention for service members with mild traumatic brain injury. Explore (NY) 2013;9(3):142-149. [CrossRef] [Medline]
    31. Renaud MI, Klees C, van Haastregt JC, Catsman-Berrevoets CE, van de Port IG, Lambregts SA, et al. Process evaluation of 'Brains Ahead!': an intervention for children and adolescents with mild traumatic brain injury within a randomized controlled trial. Clin Rehabil 2020 May;34(5):688-697. [CrossRef] [Medline]
    32. Twamley EW, Jak AJ, Delis DC, Bondi MW, Lohr JB. Cognitive Symptom Management and Rehabilitation Therapy (CogSMART) for veterans with traumatic brain injury: pilot randomized controlled trial. J Rehabil Res Dev 2014;51(1):59-70 [FREE Full text] [CrossRef] [Medline]
    33. Daugherty J, Miles I, Sarmiento K, Sansone C, Kroshus E, Bethea B. A description and evaluation of the concussion education application HEADS UP rocket blades. Health Promot Pract 2019 Jan;20(1):22-30 [FREE Full text] [CrossRef] [Medline]
    34. King EG, Kretzmer TS, Vanderploeg RD, Asmussen SB, Clement VL, Belanger HG. Pilot of a novel intervention for postconcussive symptoms in active duty, veterans, and civilians. Rehabil Psychol 2013 Aug;58(3):272-279. [CrossRef] [Medline]
    35. Hoek AE, van den Hamer M, Deelstra CK, van Beeck EF, Dippel DW, Haagsma JA, et al. Attitude of patients, healthcare professionals, and noninjured lay persons towards online video instructions on mild traumatic brain injury: a cross-sectional study. Int J Emerg Med 2017 Dec;10(1):25 [FREE Full text] [CrossRef] [Medline]
    36. Kjeldgaard D, Forchhammer HB, Teasdale TW, Jensen RH. Cognitive behavioural treatment for the chronic post-traumatic headache patient: a randomized controlled trial. J Headache Pain 2014 Dec 02;15(1):81 [FREE Full text] [CrossRef] [Medline]
    37. Little JR, Pavliscsak HH, Cooper MR, Goldstein LA, Fonda SJ. Does mobile care ('mCare') improve quality of life and treatment satisfaction among service members rehabilitating in the community? Results from a 36-wk, randomized controlled trial. Mil Med 2018 Mar 01;183(3-4):e148-e156. [CrossRef] [Medline]
    38. Cole MA, Muir JJ, Gans JJ, Shin LM, D'Esposito M, Harel BT, et al. Simultaneous treatment of neurocognitive and psychiatric symptoms in veterans with post-traumatic stress disorder and history of mild traumatic brain injury: a pilot study of mindfulness-based stress reduction. Mil Med 2015 Sep;180(9):956-963. [CrossRef] [Medline]
    39. Chrisman SP, Whitlock KB, Mendoza JA, Burton MS, Somers E, Hsu A, et al. Pilot randomized controlled trial of an exercise program requiring minimal in-person visits for youth with persistent sport-related concussion. Front Neurol 2019 Jun 17;10:623 [FREE Full text] [CrossRef] [Medline]
    40. Levy CE, Miller DM, Akande CA, Lok B, Marsiske M, Halan S. V-Mart, a virtual reality grocery store: a focus group study of a promising intervention for mild traumatic brain injury and posttraumatic stress disorder. Am J Phys Med Rehabil 2019 Mar;98(3):191-198. [CrossRef] [Medline]
    41. Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci 2011 Apr 23;6:42 [FREE Full text] [CrossRef] [Medline]
    42. Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W, et al. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Ann Behav Med 2013 Aug;46(1):81-95 [FREE Full text] [CrossRef] [Medline]
    43. Kay T, Harrington DE, Adams R, Anderson T, Berrol S, Cicerone K, Mild Traumatic Brain Injury Committee of the Head Injury Interdisciplinary Special Interest Group of the American Congress of Rehabilitation Medicine. Definition of mild traumatic brain injury. J Head Trauma Rehabil 1993;8(3):86-87 [FREE Full text] [CrossRef]
    44. King NS, Crawford S, Wenden FJ, Moss NE, Wade DT. The Rivermead Post Concussion Symptoms Questionnaire: a measure of symptoms commonly experienced after head injury and its reliability. J Neurol 1995 Sep;242(9):587-592. [CrossRef] [Medline]
    45. Kroenke K, Spitzer RL. The PHQ-9: a new depression diagnostic and severity measure. Psychiatr Ann 2002 Sep 01;32(9):509-515. [CrossRef]
    46. Zhou L, Bao J, Setiawan IM, Saptono A, Parmanto B. The mHealth app usability questionnaire (MAUQ): development and validation study. JMIR Mhealth Uhealth 2019 Apr 11;7(4):e11500 [FREE Full text] [CrossRef] [Medline]
    47. Eldridge SM, Chan CL, Campbell MJ, Bond CM, Hopewell S, Thabane L, PAFS consensus group. CONSORT 2010 statement: extension to randomised pilot and feasibility trials. BMJ 2016 Oct 24;355:i5239 [FREE Full text] [CrossRef] [Medline]
    48. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care 2007 Dec;19(6):349-357. [CrossRef] [Medline]
    49. The FA's Concussion Guidelines. The Football Association. 2019 Aug.   URL: https://www.thefa.com/-/media/files/pdf/my-football/the-fa-concussion-guidelines-2019.ashx [accessed 2022-10-12]
    50. Nelson LD, Temkin NR, Dikmen S, Barber J, Giacino JT, Yuh E, the TRACK-TBI Investigators, et al. Recovery after mild traumatic brain injury in patients presenting to us level I trauma centers: a transforming research and clinical knowledge in traumatic brain injury (TRACK-TBI) study. JAMA Neurol 2019 Sep 01;76(9):1049-1059 [FREE Full text] [CrossRef] [Medline]
    51. Arbabi M, Sheldon R, Bahadoran P, Smith JG, Poole N, Agrawal N. Treatment outcomes in mild traumatic brain injury: a systematic review of randomized controlled trials. Brain Inj 2020 Jul 28;34(9):1139-1149. [CrossRef] [Medline]
    52. Potter SD, Brown RG, Fleminger S. Randomised, waiting list controlled trial of cognitive-behavioural therapy for persistent postconcussional symptoms after predominantly mild-moderate traumatic brain injury. J Neurol Neurosurg Psychiatry 2016 Oct;87(10):1075-1083. [CrossRef] [Medline]
    53. Basch E, Deal AM, Kris MG, Scher HI, Hudis CA, Sabbatini P, et al. Symptom monitoring with patient-reported outcomes during routine cancer treatment: a randomized controlled trial. J Clin Oncol 2016 Feb 20;34(6):557-565 [FREE Full text] [CrossRef] [Medline]
    54. Baumgartner C, Schaub MP, Wenger A, Malischnig D, Augsburger M, Lehr D, et al. "Take Care of You" - efficacy of integrated, minimal-guidance, internet-based self-help for reducing co-occurring alcohol misuse and depression symptoms in adults: results of a three-arm randomized controlled trial. Drug Alcohol Depend 2021 Aug 01;225:108806 [FREE Full text] [CrossRef] [Medline]
    55. Ma X, Yue ZQ, Gong ZQ, Zhang H, Duan NY, Shi YT, et al. The effect of diaphragmatic breathing on attention, negative affect and stress in healthy adults. Front Psychol 2017 Jun 6;8:874 [FREE Full text] [CrossRef] [Medline]
    56. Bradbury K, Steele M, Corbett T, Geraghty AW, Krusche A, Heber E, et al. Developing a digital intervention for cancer survivors: an evidence-, theory- and person-based approach. NPJ Digit Med 2019 Sep 2;2:85 [FREE Full text] [CrossRef] [Medline]
    57. Yardley L, Morrison L, Bradbury K, Muller I. The person-based approach to intervention development: application to digital health-related behavior change interventions. J Med Internet Res 2015 Jan 30;17(1):e30 [FREE Full text] [CrossRef] [Medline]
    58. Mobley A, Jake-Schoffman D, Fedele D, Varela E, Zeldman J. Babybyte: usability testing of a mHealth app prototype to improve responsive feeding practices in parents of infants and toddlers. Curr Dev Nutr 2021 Jun;5(Suppl 2):983. [CrossRef]
    59. Lesher A, Gavrilova Y. 111 Optimizing an outpatient mHealth intervention for Dallas with burns: a mixed methods study. J Burn Care Res 2022 Mar 23;43(Suppl 1):S72-S73 [FREE Full text] [CrossRef]
    60. Macnow T, Curran T, Tolliday C, Martin K, McCarthy M, Ayturk D, et al. Effect of screen time on recovery from concussion: a randomized clinical trial. JAMA Pediatr 2021 Nov 01;175(11):1124-1131 [FREE Full text] [CrossRef] [Medline]
    61. Chien I, Enrique A, Palacios J, Regan T, Keegan D, Carter D, et al. A machine learning approach to understanding patterns of engagement with internet-delivered mental health interventions. JAMA Netw Open 2020 Jul 01;3(7):e2010791 [FREE Full text] [CrossRef] [Medline]
    62. Makri A. Bridging the digital divide in health care. Lancet Digital Health 2019 Sep 01;1(5):e204-e205. [CrossRef]
    63. Torous J, Michalak EE, O'Brien HL. Digital health and engagement-looking behind the measures and methods. JAMA Netw Open 2020 Jul 01;3(7):e2010918 [FREE Full text] [CrossRef] [Medline]
    64. Edinburgh DataShare.   URL: https://datashare.ed.ac.uk/handle/10283/4747 [accessed 2023-01-24]

    ‎
    BCW: Behavior Change Wheel
    CBT: cognitive behavioral therapy
    CONSORT: Consolidated Standards of Reporting Trials
    COREQ: Consolidated Criteria for Reporting Qualitative Research
    ED: emergency department
    GOSE: Glasgow Outcome Scale Extended
    MAUQ: mHealth App Usability Questionnaire
    MRC: Medical Research Council
    PHQ-9: Patient Health Questionnaire-9
    PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses

    Edited by T Leung; submitted 15.10.22; peer-reviewed by M Kneavel, S Marceglia; comments to author 23.11.22; revised version received 07.12.22; accepted 31.12.22; published 01.02.23

    Copyright

    ©Christine d'Offay, Xin Yi Ng, Laura Alexander, Alison Grant, Julia Grahamslaw, Claudia Pagliari, Matthew J Reed, Alan Carson, David C Gillespie, Aimun A B Jamjoom. Originally published in JMIR Formative Research (https://formative.jmir.org), 01.02.2023.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.