This study assessed the usability of CimplyMe, a new mobile technology companion app designed to be used alongside any CZP self-injection device to further support and engage CZP-treated patients throughout their disease journey. The app provides step-by-step guidance through its timeline feature, providing information on the disease and CZP treatment. Guidance is focused on the initial 12 weeks of treatment but continues to support patients throughout their disease journey. Patients can track their pain, energy levels, mood, and activities using the Health Metrics Tracking feature. This is incorporated into the Health Summary that provides patients with an overview of their progress, including treatment adherence tracking, which they can choose to share with health care practitioners and their family. Milestones are included in the app and were designed to help patients acknowledge important moments in their disease journey and motivate patients to continue treatment. Additionally, injection and clinic appointment reminders aim to improve patient outcomes by supporting treatment adherence and disease monitoring.

This study was designed in line with ANSI/AAMI/IEC62366-1:2015 on applying human factors and usability engineering to medical devices [10]. Each participant gave informed consent before attending a single session; all sessions took place in London, United Kingdom. Figure 1 shows the study design. Patients were not given any formal training but had some time to familiarize themselves with the app. They were also able to use an instruction manual if requested, although the app was designed to be used without this document. Patients were asked to complete 11 use scenarios that were designed to simulate the intended use of the app. A moderator asked all patients to behave in the way they thought they would if the scenario was real. During the session, a moderator recorded all observations and patient feedback in a workbook. The sessions were also video recorded.

The primary study objective was to validate that the user interface of CimplyMe is intuitive, easy, and safe to use by intended users so as not to cause injury or harm. The secondary objective was to confirm that the risk of errors was reduced as far as reasonably possible through risk-mitigation methods implemented in the user interface (UI) of the beta version of the app used in this study.

Patients were assessed on their ability to complete 8 critical tasks composed of 14 component tasks and 3 noncritical tasks composed of 8 component tasks (Table 1). All participants were observed and timed while completing each task by study personnel. Each test session was scheduled to last for 105 minutes (1.75 hours); the moderator divided the time allocated between each task (between approximately 3 and 10 minutes depending on the task) to ensure that the participants could at least attempt each task once. Each component task had to be completed correctly; otherwise, a use error was recorded. If a patient struggled to complete the component tasks but avoided a use error, the task was recorded as a “success with difficulty.” If a patient made a use error on their first attempt, they could attempt the task a second time. If a use error was made at the second attempt, it was recorded as a task failure. All patient actions that led to, or resulted from, successes with difficulty or use errors were recorded. Detailed root cause analysis was carried out on all tasks completed with difficulty or recorded as a use error to determine whether the cause was a result of the user interface.

Adult patients (≥18 years) diagnosed with moderate to severe RA were invited to participate in the study. To ensure at least 97% of potential use errors were identified [11], a minimum of 15 intended users were recruited. To verify their RA diagnosis, patients were asked to provide a confirmation letter from their physician. In addition, patients had to have been prescribed biologic treatment at the time of enrollment. Patients must have been using a smart phone and had experience using apps, of any type, on their phone. Patients must also have stated that they would use a medical mobile app to assist with their disease treatment. The study aimed to recruit at least 7 patients with impaired vision, either correctable or not correctable with glasses or contact lenses.

Patients were excluded if they had participated in medical device research for a medical app in the previous 6 months, were app developers, or had stated that they would not use a mobile app to assist with their disease treatment. Patients were also excluded if they experienced <10 minutes of joint stiffness in the morning without treatment.

Of the 30 patients who were screened to take part in the study, 15 patients completed the study. Of the 15 patients that did not take part, 8 were unable to provide a physician’s letter, 4 were not able to take part due to scheduling, and 2 patients self-withdrew. One patient was excluded as they were known to the sponsor. Familiarity with the use of mobile apps was a stated requirement; therefore, there were no screen failures relating to this.

Patient baseline characteristics are shown in Table 2. The mean patient age was 53.5 years (SD 12.0 years), and 66.7% (10/15) patients were female. All patients were using either a prefilled syringe or prefilled pen to administer their current biologic medication.

Throughout the study, 207 first attempts at critical component tasks were completed. Of these, 86.0% (178/207) tasks were successful; 7.7% (16/207) were successes with difficulty; and 6.3% (13/207) were use errors (Table 3). Twelve second attempts at critical tasks were completed in the study; one task was not re-attempted due to time constraints. No use errors were encountered when performing tasks for a second time, although 4 tasks were recorded as successes with difficulty (Table 3).

Reasons for patient difficulties and use errors during first and second attempts at critical tasks are shown in Multimedia Appendix 1. Several (6/13, 46.2%) first attempts classed as use errors were cases of patients not completing the task in the allotted session time. Patients were not given a set amount of time to complete each task; however, the total time of each session was limited to 110 minutes so not all patients were able to complete all tasks within this time frame.

Difficulties at the first attempt were due to various reasons; however, most (10/16, 62.5%) resulted from patients getting confused by the app terms, organization, or process (Multimedia Appendix 1). Additionally, one difficulty was caused by an app fault. There were 4 successes with difficulty during the 12 second attempts at the critical tasks. As with those observed at first attempt, most (3/4, 75.0%) of these difficulties were due to patients getting confused by the app terms or organization (Multimedia Appendix 1).

The root cause analysis of all use errors is shown in Table 4. The UI contributed to 61.5% (8/13) of use errors. Of these, 42.9% (3/8) were attributable to the small font size in the instructions for use (IFU). In 37.5% (3/8) of the use errors, patients did not see or were not prompted to scroll to the necessary button(s). The remaining errors associated with the UI (2/8, 25.0%) were attributed to app complexity or lack of clarity.

A total of 107 first attempts at the 3 noncritical tasks were performed in the study. Of these, 97.2% (104/107) were successful, 2.8% (3/107) were successes with difficulty, and none were use errors (Table 3). Reasons for patient difficulties when completing noncritical tasks are shown in Figure 2.

No failures were observed for any of the attempted tasks; all patients were able to successfully complete both critical and noncritical tasks using the companion app on their first or second attempt without any formal training.

There were several instances where use errors could be attributed to the UI; these can all be resolved by small changes to the app. For example, patients did not realize that they needed to scroll further down the page to find the correct information or buttons in multiple instances. To minimize this problem, the scaling of the app could be optimized to ensure that most buttons are always on the screen. If future feedback suggests patients still have difficulties, a small note or animation could be added to indicate the need to scroll down on first use. Additionally, patients found the font size in the IFU and app too small or did not read the IFU because it was too long. However, the font size on the app is dependent on the default font size settings on the patient’s phone and can be changed by the patient, if necessary. Two patients did not understand the terminology used in the app, or struggled with the complexity of the app itself, resulting in confusion and error in completing tasks and navigating the app. However, this was likely due to the short amount of time patients were given to familiarize themselves with the app. It is anticipated that patient understanding of the app’s terminology and layout would improve with use, as indicated by the reduction in errors when tasks were attempted for a second time in this study.

Mobile technology apps have been developed to provide information and patient education, facilitate lifestyle changes as a strategy to ameliorate disease, provide medication reminders to promote treatment adherence, and record patient metrics for disease monitoring [12-15]. Previous research suggests that mobile technology apps can help patients feel supported and improve patient satisfaction [12,16]. For example, compared to usual care, a mobile health technology–supported disease management model for atrial fibrillation resulted in improvements in patient quality of life [17]. Patient adherence and disease management may also be improved with the support of treatment management apps [16,18,19]. For example, a 2019 systematic literature review of studies investigating mobile apps as a method to improve treatment adherence identified 11 studies of patients with a variety of diseases [18]. Of these, 7 studies reported an increase in treatment adherence with use of a mobile app [18].

One of the key challenges associated with the use of mobile apps is the high user attrition rate, with the level of mobile app engagement dropping off over time [20]. Understanding the features and characteristics most valued by patients may improve app engagement. Users appreciate features that save time over current methods and identify an app as valuable when it is simple and intuitive to use, provides specific instructions to better manage a condition, shares data with designated individuals, and incorporates motivational “push” factors [20,21]. Increasing age is also associated with reduced use of smartphones for internet browsing, social media, and apps [22]. Consequently, mobile app attrition is likely to be increasingly challenging in the population of patients likely to use mobile apps such as CimplyMe; patients with chronic inflammatory diseases are likely to be older and so are likely to be less comfortable using mobile apps. Designing apps with the user in mind and assessing usability in human factors studies will help ensure successful patient engagement.

Human factors studies only simulate use in an everyday environment. Patients were asked to behave in the way they thought they would if the scenario was real; however, this may be difficult to judge accurately. Each session was limited to 105 minutes, and, as a result, patients may have felt under some time pressure to complete each task. Time constraints also restricted patients’ ability to familiarize themselves with the app. Additionally, each task was completed discretely, which may have interfered with their familiarization of the app, owing to the lack of continuity between tasks. Patients were selected based on their willingness to use mobile apps, which may limit the generalizability of the results; however, the app will be provided as an additional support component and so use by patients will be voluntary. Therefore, the study population is reflective of the final population of patients who would likely use this in clinical practice. Finally, the number of patients participating in the study was small; therefore, the results may not represent the app’s usability in a wider patient group or capture all potential use errors and difficulties when using the app.

In a simulated setting, patients were able to use the companion app successfully. No critical task failures were observed; all patients were able to successfully complete both critical and noncritical tasks within two attempts without any formal training. This study demonstrates that the companion app UI is generally intuitive and easy to use among patients familiar with using apps. The app could support CZP patients to self-manage their treatment.