The usability study consisted of a two-week, community-oriented, prospective single-arm observational pilot study involving adults aged 18 years and older with a physician-diagnosed concussion. Participants were engaged in an active, home-based rehabilitation program accessed through the mobile app. Individuals were recruited through word-of-mouth, flyers, and social media platforms (eg, X, formerly known as Twitter, Facebook) and were provided detailed information regarding study requirements and procedures. Participant enrollment was conducted remotely via Zoom, and individuals were screened by a research assistant to confirm inclusion criteria consisting of a concussion diagnosis by a physician, aged at least 18 years, and access to an iPhone. Furthermore, individuals with coexisting conditions such as musculoskeletal or soft-tissue injuries, vestibular disorders, or neurological conditions that could impede physical activity, were excluded from the study. Twenty-six individuals consented and enrolled in the study. Upon enrollment, participants were mailed an Apple Watch Series 6 to use with their iPhones and access to the Rhea Health Inc. mobile app. Ultimately, 23 participants completed the usability questionnaire after using the mobile app for two weeks.

This study received ethical approval from the Health Sciences Research Ethics Board at the University of Toronto (protocol #39143). To ensure comprehensive understanding and informed decision-making, prospective participants were given an information sheet by a member of the research team. This sheet detailed the study’s eligibility criteria, purpose, methodologies, and any potential risks involved. Before participants gave their consent, their eligibility was confirmed via email. A Zoom videoconference was then conducted to clarify any queries regarding the study and reinforce the voluntary nature of participation and the freedom to withdraw at any point without adverse consequences. Informed consent was secured through an electronic process in which participants reviewed and signed a digital document that outlined the study’s purpose, procedures, and potential risks. This consent form was then submitted using the REDCap (Research Electronic Data Capture) platform, hosted by the University of Toronto.

Data collected via the mobile app and Apple Watch were securely stored on the Amazon Web Services’ DynamoDB database, located in Montreal, Quebec, Canada. This storage solution employed Amazon Web Services’ proprietary encryption and restricted access system to ensure data security. Although participants were required to use their emails and passwords for accessing the mobile app and syncing data from their Apple Watch to Apple Health, personally identifiable information was stored in a separate database table and was not required for data analysis.

Upon the study’s completion, participants were compensated CAD $50 (equivalent to approximately US $37.16) for their participation and were entered into a lottery for a chance to win an Apple Watch Series 6. This gesture not only acknowledged their contribution but also encouraged engagement and retention in the study.

The rehabilitation protocol in this study was delivered through the Rhea mobile app (Rhea Health Inc.) (Figure 1), a rehabilitation platform that devises personalized programs based on an individual’s symptoms and time elapsed since injury. The app integrates symptom data, including 22 items from the Sports Concussion Assessment Tool (SCAT-5) [20], along with seven additional questions related to vision and balance. Participants assessed their symptom severity using a 4-point Likert Scale, ranging from 0 (none) to 3 (severe). Following a staged approach endorsed by consensus statements on concussion management, the Rhea rehabilitation protocol employed a six-stage model that progressively increased in intensity and complexity. These stages extend from Initial Rest (Stage 1) to Return (Stage 6). Participants commenced their rehabilitation at Stage 2, completing three sessions at this level before progressing. One of the Rhea app’s unique features is its dynamic adjustment of the rehabilitation plan that is based on participant feedback, which categorizes symptomology as “same,” “better,” or “worse.” This feedback-driven mechanism ensures that the participants either stay at their current stage, advance to a more challenging stage, or revert to a less intensive stage, as required. The primary objective of this adaptive approach is to fine-tune the intensity and complexity of exercises to avoid exacerbating symptoms, thus promoting a safe and effective journey through the recovery stages. In addition to the six core stages, Rhea’s program provided participants with exercises specifically tailored to neck, vision, and balance rehabilitation. These exercises are designed to address each participant’s unique symptom profile, further personalizing the recovery experience. This approach not only ensures adherence to the rehabilitation protocol but also maximizes the potential for effective recovery tailored to individual needs. To monitor adverse events, participants were instructed to inform a member of the research team of any adverse events (eg, injuries, medical conditions) that were experienced while using the mobile app.

The mHealth App Usability Questionnaire (MAUQ) [21] contains 21 questions rated on a 7-point Likert scale, ranging from strongly agree to strongly disagree, under three categories: ease of use and satisfaction, system information arrangement, and usefulness. Studies have identified the MAUQ to be a reliable and valid tool for assessing mHealth app usability [21]. In this study, we used 16 questions from the MAUQ and omitted the 5 questions from the usefulness category, as they assess aspects related to sharing information with health care providers and clinical utility. Due to the preliminary nature of the research and the use of a research-grade tool that lacked standards and approval for clinical use, we did not ask or expect participants to engage with health care professionals using the tool. Additionally, we included five custom questions to evaluate the app’s critical features related to the delivery of exercise rehabilitation, as the app was based on evidence-based exercise rehab principles. These custom questions were designed to assess user confidence in the app’s recommendations, clarity of exercise instructions, the flow of the exercise content, and the usability of the app’s voice commands. More specifically, given the central role of accurate exercise prescription in rehabilitation, it was important to gauge whether users trusted the app’s recommendations; therefore the question, “I had confidence in the recommendations I received from the app” was included. Since effective rehabilitation relies on the clarity and ease of exercise guidance, we included two questions targeting the flow and clarity of the exercise sessions: “The flow of the exercise sessions on the app is easy to follow” and “I knew exactly which exercises I had to do during an exercise session.” These questions allowed us to determine whether participants found the session progression intuitive and well-structured. We also added a question about voice commands: “The in-app voice commands are clear and understandable.” This was a critical feature for users who might benefit from auditory guidance, particularly those experiencing cognitive fatigue or visual sensitivity due to their concussion. Clear voice commands would support a hands-free interaction and reduce the cognitive load associated with reading instructions on a screen. Lastly, we evaluated the app’s reporting functions through the question, “The exercise report section is easy to follow and understand.” A clear and accessible exercise report is vital for tracking progress and ensuring participants understand their rehabilitation. These custom questions were specifically tailored to capture the nuances of delivering exercise rehabilitation through a digital platform, ensuring that the app was not only usable but also effective in guiding users through their recovery process.

The purpose of the current study was to summarize and report participant feedback following a two-week period of using the mobile app. Therefore, the Likert-scale responses from the MAUQ and additional customized app usability questions were summarized and reported as raw counts and percentages. For additional continuous variables, the median (IQR) were calculated.

Study participant characteristics can be seen in Table 1. Briefly, the sample comprised predominantly of female participants (n=18, 78%), aged 26‐38 years. On average, the participants were nearly three months post-concussion, with 48% (n=11) sustaining injury during sports or physical activity.

Participants entered the study with an average of 14 symptoms and a symptom severity score of 29. On average, they reported feeling at approximately 70% of their usual health, and 43% (n=10) of the participants stated that they were getting better following their injury at first, but later experienced symptom worsening. Please see Table 2 for the complete symptom profiles of the participants.

Participants’ responses to the MAUQ can be seen in Figure 2. Across the 16 MAUQ questions, “agree” or “strongly agree” were the most common responses. Under the “ease of use and satisfaction” category, all 23 participants selected “agree” or “strongly agree” for the question “easy to learn.” Twenty participants (87%) selected “agree” or “strongly agree” for both “easy to use” and “well organized” questions. The question with the least favorable response in this category was “appropriate time involved,” where 4 participants (17%) selected “disagree” or “somewhat disagree.”

Under the category “system information arrangement,” 20 (87%) participants selected “agree” or “strongly agree” for the question “all functions usable,” and 19 (87%) for the question “screen navigation consistent.” The most unfavorably responded question for this category was “ease of recovery following mistake,” where 3 participants (13%) chose either “strongly disagree,” “somewhat disagree,” or “disagree.”

In the “usefulness” category, 20 (87%) participants responded with “agree” or “strongly agree” for “health and well-being usefulness,” and 21 participants (87%) reported similarly for “helped manage health.” Please see Multimedia Appendix 1 for a summary of participant responses to each question of the MAUQ.

The additional questions specific to the app yielded similar responses to the MAUQ, with “agree” or “strongly agree” being the most common response for all five questions (Figure 3); only a single negative response was reported across all questions. The most favorable response was for “voice command clarity” where all but one participant (n=22, 95%) reported either “agree” or “strongly agree”; the other participant selected “somewhat agree.” Please see Multimedia Appendix 2 for a summary of the participant responses for the additional questions.