When designing the first iteration of the health app, we considered our target users’ (individuals aging with a physical disability) characteristics (Table 1). To ensure a high degree of usability, age-related changes and limitations due to disease or injury were taken into consideration, particularly as they related to cognitive overload, dexterity, and sensory function. To reduce cognitive overload, the app was designed to provide written instructions immediately preceding a task. Only one set of instructions was presented per slide, and large graphics depicting the task were also provided. This layout streamlined the app and reduced the need for working memory of the participants. In total, there were 14 slides, taking approximately 10 minutes to complete. Decrements in dexterity are commonly seen in those who have neurological complications [21-23] and age-associated arthritis [24]. To account for this within the app, selection options and buttons were made large, and typed responses were avoided. Sensory-related changes were accommodated by the use of black text written in a 14-point font on a white background [25], and auditory processing deficits were accommodated by the use of leading audio cues with simultaneous vibrations.

The fall risk app, Steady-Wheels, was developed in Android Studio 3.1.2. Upon opening the app, users are presented with a welcome screen that outlines the purpose of the app and provides an overview of the process (Figure 1). Steady-Wheels has two main components: a patient-reported outcome section and a performance test section (Figure 2). The patient-reported outcome component asks the participant to complete a 13-item health history questionnaire (including age, sex, number of falls in the last year, and activities that provoke concerns about falling [9]) (Figure 3). The performance component leads participants through a progressive series of seated postural control tasks (Figure 4). Before testing, participants were provided with written safety instructions. Participants were instructed to engage their wheel locks, and power wheelchair and scooter users were instructed to turn off their devices. All participants were asked to have a handrail or wall nearby in case they lost their balance. To complete the testing, the device guided participants through the completion of three 30-second seated balance tasks in a standardized order that increasingly challenged the participant’s base of support: an eyes-open balance task, an eyes-closed balance task, and a functional stability boundary task (Figure 4). These tests were chosen because they can provide insight into postural control [26] and have been linked to fall risk [27]. Written instructions on how to properly complete the balance tasks were provided before the start of each task. After the participant self-selected the “Let’s Start” option, an audio and vibratory countdown began from 5, leading to the word “start,” which cued the start of the test. The completion of the test was auditorily cued with the word “stop.” Participants were asked to hold the smartphone against the middle of their chest with their dominant hand for the duration of each test. Upon completion of each balance task, users reported if they were able to complete the task by selecting one of the following: “I completed the test,” “I was unable to complete the test,” and “I did NOT attempt to complete the test.” If participants were dissatisfied with their attempt at the task, they could select “I’d like to retry” and make another attempt.

A future goal of this work is to utilize the participants’ demographic and movement data to generate a personalized fall risk score. To better understand users’ preferences for receiving their fall risk score, they were asked to rate different result screen options and provide insight on what made some illustrations better than others (Figure 5).

The Institutional Review Board of the University of Illinois at Urbana-Champaign approved all procedures (20192), and all participants provided informed consent before engaging in research activities. All research procedures were performed in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975.

To be eligible, individuals were required to be ≥18 years old, utilize a wheeled mobility device for their main form of mobility, be able to sit unsupported for 30 seconds, have manual dexterity sufficient to use a smartphone, have hearing and vision that were normal or corrected to normal, and be able to read and speak English. In light of the COVID-19 pandemic, having access to video conferencing software (eg, Zoom, Facetime, or Skype), was an inclusion criterion for the second round of testing.

This study included 2 rounds of 5 different older adult wheelchair users (age 58.5 years, SD 12.6 years; 5 male, 5 female) who were recruited from the community through existing participant pools, sharing of research flyers, and word of mouth (Table 1). The first round of testing was completed in person between November 2020 and February 2021, while the second round was performed remotely between April and May 2021. During sessions, participants completed using the app, identified barriers to usability, and gave their rationale for their preferred results options. Feedback from the first round of 5 participants was used to modify the app. Following modification, the second round of volunteers participated in usability testing. This iterative design process is ideal for identifying use challenges, and having a sample size of 5 individuals per round of testing has been shown to be sufficient for identifying usability problems [28,29]. On average, the first round of testing in iterative design identifies 85% of usability problems, and the second round identifies an additional 13% [30]. This approach has been successful in the development of various health apps [31-33], including 2 recent fall risk apps for older adults [18] and patients with multiple sclerosis [34].

After providing informed consent, each participant was given a smartphone (Samsung Galaxy S6, Samsung) that had the Steady-Wheels app installed. The participants were read an instructional prompt (Multimedia Appendix 1) asking them to speak their thought processes aloud while they independently used the app [35]. After the questions were answered, the researchers began visually and auditorily recording the participants’ interactions with the app and wrote field notes. After they completed using the app, the participants completed a semistructured interview in which they were asked to expand upon their likes and dislikes about the app’s layout and features (eg, graphics and wording) and to provide any suggestions for future iterations of the app. During this time, the participants also ranked the fall risk score results options from most to least favorite (Figure 5).

For the most part, these procedures remained constant for the second round of usability testing. The only difference was that the research supplies were delivered to the participants’ residences and the experimental session was completed over video conferencing software.

Along with feedback from the participant interviews, a smartphone usage questionnaire and the System Usability Scale (SUS) [36] were used to understand the participants’ experiences using smartphone and health apps and to quantify the usability of Steady-Wheels, respectively. While the questionnaire had a total of 6 choice and written response questions, the SUS consists of 10 questions with 5 response options [36,37], ranging from “strongly agree” (5 points) to “strongly disagree” (1 point). After calculation, results from the SUS range from zero (lowest usability) to 100 (highest usability); technology in general has an average score of 60 [37].

A thematic analysis approach was used to conduct the qualitative analysis [38]. Video recordings from the think-aloud activity and interviews were transcribed verbatim. The text was then independently reviewed and assigned codes (eg, instructions, testing duration, and graphics) based on its content using the software MAXQDA (version 12.3.3; Verbi GMBH). Once codes were reviewed and discussed by 2 authors (MF and KH), they were grouped into themes based on the commonality of the data. The same 2 authors (MF and KH) then deliberated on the main themes to ensure they reflected participant insights as accurately as possible. Both authors had prior experience conducting qualitative analyses.

The first round of usability testing yielded 2 themes: ease of use and flexibility of design. Representative participant quotes concerning these themes are reported throughout the following sections. The quotes are accompanied by participant characteristics (eg, sex, and age). System usability scores ranged from 72.5 to 97.5 and averaged 84.5 (SD 11.4), indicating “excellent” usability [39].

The second round of usability testing yielded 2 themes: app layout and clarity of instruction. SUS scores ranged from 87.5 to 97.5 and averaged 91.9 (SD 4.3), indicating “best imaginable” usability [39].

Understanding the usability of a smartphone app provides insight into the quality and overall satisfaction of the user’s experience. An improved experience could lead to greater use of health apps and increased adherence to suggested interventions [40]. Consequently, the purpose of the current study was to determine the usability of a fall risk mobile health app, Steady-Wheels, and identify key insights into technology development for aging adults who use wheeled devices. Initial design considerations were based on age-related changes and physical limitations associated with disability, including motor, sensation, and cognitive impairments. A mixed-method, iterative design and testing process yielded high SUS scores; the app was rated as having “excellent” and “best imaginable” levels of usability. The main themes for each iteration were informed by participant feedback, with the first round of testing yielding 2 main themes (ease of use and flexibility of design) and the second round of testing yielding 2 different main themes (app layout and clarity of instruction). These themes helped identify insights into app development that could promote usability for aging adults who use wheeled devices.

Overall, participants found that the app was straightforward, easy to use, supportive of individual preferences, and allowed for easy recovery from errors. They appreciated the simple, objective fall risk score. App development and modifications came from participant feedback and insights from previously developed apps [18,34,40-42] and an understanding of usability heuristics for interface design, such as the visibility of the system, use of recognition rather than recall, aesthetics, minimalist design, error prevention, and a match between the system and the real world [43].

The visibility of the system and the timeliness and adequacy of feedback and information to users informed the modifications made to the welcome screen and the overall “step-by-step” style of the app. This approach allowed for the recognition of a recently described task rather than recall of prior instruction. While this promoted the ease of use of Steady-Wheels, the primary complaints about health apps made by users are often related to their aesthetics, especially poor or difficult to interpret color coding, graphics, and fonts [42]. Providing a simple color scheme and font with the inclusion of only essential graphics aided this and created minimal distractions within the app. This approach placed a reduced cognitive load on users and helped to reduce the occurrence of errors.

The primary goal of the results scores at the end of testing was to intuitively convey fall risk results to diverse users. Individuals learn and retain content better from visual information (eg, cartoons and graphics) [44] and can interpret its meaning much more easily if the design is recognizable [41] or matches real-world experiences and expectations [43]. Providing users with a results option that mimicked their preexisting knowledge of speedometers followed these concepts, was well received, and promoted curiosity in the users about what could be done to lower their fall risk. Collectively, these design features led to the development of an app with high perceived ease of use, which is associated with greater adoption of technology [45].

Along with the personal adoption of technology, it is also important to gain insights into the likelihood of users recommending this technology to other individuals. The SUS has a strong relationship with the Net Promotor Score, which has become a common metric to understand customer loyalty [46]. Consumers will likely promote a product if it achieves a SUS score of 81 or greater [39]. In the current study, both iterations of testing yielded a SUS score above this threshold. This is particularly noteworthy as technology in general has an average SUS score of 60 [37]. Overall, these findings indicate that Steady-Wheels may not only be adopted on a personal level, but will likely also be recommended to others at an equal or greater rate than other forms of technology.

Although this is the first app designed to measure fall risk in aging adults who use wheeled devices, our initial design was informed by the needs of users and learned experiences from prior attempts to develop fall risk screening apps, both for older adults [18] and for people with multiple sclerosis [34]; all of which have received high scores for usability from their respective users. Throughout the iterative testing of Steady-Wheels, we identified key insights that could further inform the development of mobile health apps for older users of wheeled devices. Depending on their physical ability, some individuals are reliant on the use of a single hand for all activities of daily living. Future development of remote assessment should account for this in the test selection and the test’s method of completion. By failing to consider this, researchers may increase the task’s safety risk, complexity, and error rates. It was common to see participants that had difficulties with dexterity, making large buttons to select and swiping options key to the app’s ease of use. Also critical to ease of use was the guided (step-by-step) navigation of the app with clear and brief instructions accompanied by representative illustrations along the way. These features helped to reduce the risk of errors, but if mistakes were made during testing, simple error recovery (eg, allowing for retests or access to previous slides to adjust choice responses) should be made possible. Lastly, intuitive fall risk reporting was achieved through the presentation of a single number located on a color-coordinated continuum for low, medium, and high fall risk. While participants found the simple reporting of their fall risk score to be useful, they were eager to learn ways to improve their fall risk. Providing follow-up preventative information may increase the app’s usefulness and encourage further engagement with the app and shared content [40]. Personalized messaging is an easy and effective strategy for altering patient behavior [47] and could be a feasible way to share such information.

Due to the COVD-19 pandemic, the second iteration of this study was completed remotely. This successful experience highlighted the potential feasibility of the home use of the app. Participants received a smartphone that they may not have been previously exposed to, but were able to turn it on, locate the previously installed app, and follow the instructions to completion. The validity and reliability of this novel measurement tool will need to be measured and compared to common clinical tests [48,49].

The current investigation has three primary limitations: (1) baseline interviews with target users were not conducted to inform the app’s initial iteration, (2) all participants in the second round of usability testing were required to have access to videoconferencing software (eg, Zoom, Skype, or Facetime), and (3) 9 of the 10 participants had received some form of higher education.

While the current app considered our target users’ characteristics and abilities and the lessons learned during the development of previous fall risk apps, baseline interviews were not performed. Taking a more traditional user-centered approach would likely have highlighted additional thoughts, wants, and needs concerning technology and the app’s design. Identifying these key insights early on would have been a way to better serve the target users and help ensure that the researchers’ time and resources were being used most efficiently.

Although the second round of usability testing helped to provide insights into the app’s feasibility in a home setting, exclusively enrolling individuals that already used videoconferencing software may have created a biased, “technology-friendly” sample. Unfortunately, this was the only possible method of testing during the COVID-19 pandemic. Moving forward, researchers should aim to prioritize in-person data collection sessions when possible.

Despite efforts to recruit through a variety of methods and locations, most participants had received some form of higher education. Higher education may provide individuals with more experience engaging with technology and a better understanding of it, and our participants may have been more likely to understand the fall risk scores as presented. This, too, may have contributed to bias. Future researchers should consider accounting for this effect by enrolling roughly equal proportions of individuals with different educational backgrounds.

Previous literature has demonstrated that falls are common for individuals who use wheeled devices and are detrimental. The development of an objective, remote fall risk assessment tool could allow for accessible fall risk screening. Smartphone technology is a promising way to provide users with this information. Overall, aging adults who use wheeled devices found the mobile health app easy to use with a high level of usability due to characteristics such as guided navigation of the app, large text and radio buttons, clear and brief instructions that were accompanied by representative illustrations, and simple error recovery. Intuitive fall risk reporting was achieved through the presentation of a single number on a continuum of colors indicating low, medium, and high risk. Future apps developed for fall risk reporting for this population should consider leveraging the insights identified here to maximize usability.