Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 27.07.20 in Vol 4, No 7 (2020): July

This paper is in the following e-collection/theme issue:

    Original Paper

    A Companion App to Support Rheumatology Patients Treated with Certolizumab Pegol: Results From a Usability Study

    1UCB Pharma, Slough, United Kingdom

    2UCB Pharma, Brussels, Belgium

    *all authors contributed equally

    Corresponding Author:

    Barbara Domańska, MSc

    UCB Pharma

    208 Bath Road

    Slough

    United Kingdom

    Phone: 44 1615 197676

    Email: barbara.domanska@UCB.com


    ABSTRACT

    Background: Certolizumab pegol (CZP) is an anti-tumor necrosis factor drug approved for the treatment of multiple moderate to severe chronic inflammatory diseases. In the European Union, CZP is approved for administration by subcutaneous self-injection using a prefilled syringe, prefilled pen, or reusable electromechanical auto-injector (electronic device). CimplyMe is a companion app for use alongside CZP self-injection devices, designed to support CZP-treated patients self-managing their treatment and disease.

    Objective: This study aimed to validate the usability of the companion app by demonstrating that tasks required for use can be performed successfully by intended end users.

    Methods: We recruited 15 patients with moderate to severe rheumatoid arthritis, currently prescribed biologic treatment, and using apps on a smart phone. Patients were assessed on their ability to use the companion app in a setting designed to simulate a location where patients regularly administer biologic treatment. To assess the usability of the key features of the app, 8 critical and 3 noncritical tasks were designed. Patients’ success on each task was recorded through observations or knowledge-based questions. Successes with difficulty and use errors were also recorded. If a patient made a use error at the first attempt, a second attempt was allowed. Second-attempt use errors were recorded as a task failure.

    Results: A total of 207 first attempts at the 14 components of the 8 critical tasks were evaluated (3 patients failed to complete one component); 178 (86.0%) critical tasks were successfully completed at the first attempt. The remaining first attempts comprised 16 (7.7%) successes with difficulty and 13 (6.3%) use errors, which had to be repeated. One critical task was not re-attempted by one patient due to time constraints; however, there were no use errors in the 12 completed second attempts. A total of 107 first attempts at the 3 noncritical tasks were made, all of which (107/107, 100.0%) were completed without use errors.

    Conclusions: In simulated testing, patients were able to successfully use the companion app without formal training. This study suggests the companion app is easy to use and could help patients prescribed CZP better manage their treatment and disease.

    JMIR Form Res 2020;4(7):e17373

    doi:10.2196/17373

    KEYWORDS



    Introduction

    Anti-tumor necrosis factors (TNFs) are established and effective treatments for moderate to severe chronic inflammatory diseases [1]. The use of anti-TNFs alongside conventional disease-modifying antirheumatic drugs has been proven to result in better long-term disease control and reduced functional impairment [1].

    Certolizumab pegol (CZP) is an Fc-free, PEGylated anti-TNF approved for use in 66 countries [2]. CZP is approved to treat adults with moderate to severe rheumatoid arthritis (RA), axial spondyloarthritis (including both radiographic and nonradiographic axial spondyloarthritis), psoriatic arthritis, and plaque psoriasis in European Union (EU) countries and is also indicated for Crohn’s disease in the United States [3,4]. In the EU, CZP treatment is administered via prefilled syringe or prefilled pen [5]. Recently, an electromechanical autoinjection device (electronic device [e-Device]), ava, has also been approved for use [3,5]. All CZP-injection devices were designed with patient input. The e-Device aims to provide patients with customizable features to improve patient satisfaction and overall self-injection experience [5].

    Subcutaneous self-injection is associated with several specific challenges that can lead to reduced treatment adherence. Increasing use of smartphones and tablets provides a unique opportunity to address some of these challenges. For example, mobile phone apps can be designed to support patients and provide a means to overcome challenges such as forgetfulness [6]. Previous studies looking into the effectiveness of mobile phone health apps suggest that if well designed, they may provide an effective tool, empowering patients with long-term chronic conditions to self-manage their own health [7-9]. CimplyMe is a new mobile technology companion app designed to be used alongside any CZP self-injection device to further support and engage CZP-treated patients throughout their disease journey. By providing additional support, the companion app aims to improve patient satisfaction, treatment adherence, and resulting clinical outcomes.

    The aim of this study was to validate the usability of the companion app by demonstrating that critical tasks can be performed successfully by a group of patients diagnosed with RA, representative of the intended end users.


    Methods

    Study Design

    This study assessed the usability of CimplyMe, a new mobile technology companion app designed to be used alongside any CZP self-injection device to further support and engage CZP-treated patients throughout their disease journey. The app provides step-by-step guidance through its timeline feature, providing information on the disease and CZP treatment. Guidance is focused on the initial 12 weeks of treatment but continues to support patients throughout their disease journey. Patients can track their pain, energy levels, mood, and activities using the Health Metrics Tracking feature. This is incorporated into the Health Summary that provides patients with an overview of their progress, including treatment adherence tracking, which they can choose to share with health care practitioners and their family. Milestones are included in the app and were designed to help patients acknowledge important moments in their disease journey and motivate patients to continue treatment. Additionally, injection and clinic appointment reminders aim to improve patient outcomes by supporting treatment adherence and disease monitoring.

    This study was designed in line with ANSI/AAMI/IEC62366-1:2015 on applying human factors and usability engineering to medical devices [10]. Each participant gave informed consent before attending a single session; all sessions took place in London, United Kingdom. Figure 1 shows the study design. Patients were not given any formal training but had some time to familiarize themselves with the app. They were also able to use an instruction manual if requested, although the app was designed to be used without this document. Patients were asked to complete 11 use scenarios that were designed to simulate the intended use of the app. A moderator asked all patients to behave in the way they thought they would if the scenario was real. During the session, a moderator recorded all observations and patient feedback in a workbook. The sessions were also video recorded.

    The primary study objective was to validate that the user interface of CimplyMe is intuitive, easy, and safe to use by intended users so as not to cause injury or harm. The secondary objective was to confirm that the risk of errors was reduced as far as reasonably possible through risk-mitigation methods implemented in the user interface (UI) of the beta version of the app used in this study.

    Figure 1. Study design flowchart.
    View this figure

    Study Procedures and Evaluations

    Patients were assessed on their ability to complete 8 critical tasks composed of 14 component tasks and 3 noncritical tasks composed of 8 component tasks (Table 1). All participants were observed and timed while completing each task by study personnel. Each test session was scheduled to last for 105 minutes (1.75 hours); the moderator divided the time allocated between each task (between approximately 3 and 10 minutes depending on the task) to ensure that the participants could at least attempt each task once. Each component task had to be completed correctly; otherwise, a use error was recorded. If a patient struggled to complete the component tasks but avoided a use error, the task was recorded as a “success with difficulty.” If a patient made a use error on their first attempt, they could attempt the task a second time. If a use error was made at the second attempt, it was recorded as a task failure. All patient actions that led to, or resulted from, successes with difficulty or use errors were recorded. Detailed root cause analysis was carried out on all tasks completed with difficulty or recorded as a use error to determine whether the cause was a result of the user interface.

    Table 1. Critical and noncritical tasks.
    View this table

    Participants

    Adult patients (≥18 years) diagnosed with moderate to severe RA were invited to participate in the study. To ensure at least 97% of potential use errors were identified [11], a minimum of 15 intended users were recruited. To verify their RA diagnosis, patients were asked to provide a confirmation letter from their physician. In addition, patients had to have been prescribed biologic treatment at the time of enrollment. Patients must have been using a smart phone and had experience using apps, of any type, on their phone. Patients must also have stated that they would use a medical mobile app to assist with their disease treatment. The study aimed to recruit at least 7 patients with impaired vision, either correctable or not correctable with glasses or contact lenses.

    Patients were excluded if they had participated in medical device research for a medical app in the previous 6 months, were app developers, or had stated that they would not use a mobile app to assist with their disease treatment. Patients were also excluded if they experienced <10 minutes of joint stiffness in the morning without treatment.


    Results

    Patient Disposition and Baseline Characteristics

    Of the 30 patients who were screened to take part in the study, 15 patients completed the study. Of the 15 patients that did not take part, 8 were unable to provide a physician’s letter, 4 were not able to take part due to scheduling, and 2 patients self-withdrew. One patient was excluded as they were known to the sponsor. Familiarity with the use of mobile apps was a stated requirement; therefore, there were no screen failures relating to this.

    Patient baseline characteristics are shown in Table 2. The mean patient age was 53.5 years (SD 12.0 years), and 66.7% (10/15) patients were female. All patients were using either a prefilled syringe or prefilled pen to administer their current biologic medication.

    Table 2. Patient baseline characteristics.
    View this table

    Critical Task Successes and Use Errors

    Throughout the study, 207 first attempts at critical component tasks were completed. Of these, 86.0% (178/207) tasks were successful; 7.7% (16/207) were successes with difficulty; and 6.3% (13/207) were use errors (Table 3). Twelve second attempts at critical tasks were completed in the study; one task was not re-attempted due to time constraints. No use errors were encountered when performing tasks for a second time, although 4 tasks were recorded as successes with difficulty (Table 3).

    Reasons for patient difficulties and use errors during first and second attempts at critical tasks are shown in Multimedia Appendix 1. Several (6/13, 46.2%) first attempts classed as use errors were cases of patients not completing the task in the allotted session time. Patients were not given a set amount of time to complete each task; however, the total time of each session was limited to 110 minutes so not all patients were able to complete all tasks within this time frame.

    Difficulties at the first attempt were due to various reasons; however, most (10/16, 62.5%) resulted from patients getting confused by the app terms, organization, or process (Multimedia Appendix 1). Additionally, one difficulty was caused by an app fault. There were 4 successes with difficulty during the 12 second attempts at the critical tasks. As with those observed at first attempt, most (3/4, 75.0%) of these difficulties were due to patients getting confused by the app terms or organization (Multimedia Appendix 1).

    Table 3. Critical and noncritical task outcomes.
    View this table

    Root Cause Analysis of Critical Task Errors

    The root cause analysis of all use errors is shown in Table 4. The UI contributed to 61.5% (8/13) of use errors. Of these, 42.9% (3/8) were attributable to the small font size in the instructions for use (IFU). In 37.5% (3/8) of the use errors, patients did not see or were not prompted to scroll to the necessary button(s). The remaining errors associated with the UI (2/8, 25.0%) were attributed to app complexity or lack of clarity.

    Table 4. Root cause analysis of all use errors.
    View this table

    Noncritical Task Successes and Use Errors

    A total of 107 first attempts at the 3 noncritical tasks were performed in the study. Of these, 97.2% (104/107) were successful, 2.8% (3/107) were successes with difficulty, and none were use errors (Table 3). Reasons for patient difficulties when completing noncritical tasks are shown in Figure 2.

    Figure 2. Reasons for noncritical task difficulties and use errors at first attempt. The number of patients who completed noncritical tasks was <15 as some patients did not have time to complete all tasks during the session.
    View this figure

    Discussion

    Principal Findings

    No failures were observed for any of the attempted tasks; all patients were able to successfully complete both critical and noncritical tasks using the companion app on their first or second attempt without any formal training.

    There were several instances where use errors could be attributed to the UI; these can all be resolved by small changes to the app. For example, patients did not realize that they needed to scroll further down the page to find the correct information or buttons in multiple instances. To minimize this problem, the scaling of the app could be optimized to ensure that most buttons are always on the screen. If future feedback suggests patients still have difficulties, a small note or animation could be added to indicate the need to scroll down on first use. Additionally, patients found the font size in the IFU and app too small or did not read the IFU because it was too long. However, the font size on the app is dependent on the default font size settings on the patient’s phone and can be changed by the patient, if necessary. Two patients did not understand the terminology used in the app, or struggled with the complexity of the app itself, resulting in confusion and error in completing tasks and navigating the app. However, this was likely due to the short amount of time patients were given to familiarize themselves with the app. It is anticipated that patient understanding of the app’s terminology and layout would improve with use, as indicated by the reduction in errors when tasks were attempted for a second time in this study.

    Mobile technology apps have been developed to provide information and patient education, facilitate lifestyle changes as a strategy to ameliorate disease, provide medication reminders to promote treatment adherence, and record patient metrics for disease monitoring [12-15]. Previous research suggests that mobile technology apps can help patients feel supported and improve patient satisfaction [12,16]. For example, compared to usual care, a mobile health technology–supported disease management model for atrial fibrillation resulted in improvements in patient quality of life [17]. Patient adherence and disease management may also be improved with the support of treatment management apps [16,18,19]. For example, a 2019 systematic literature review of studies investigating mobile apps as a method to improve treatment adherence identified 11 studies of patients with a variety of diseases [18]. Of these, 7 studies reported an increase in treatment adherence with use of a mobile app [18].

    One of the key challenges associated with the use of mobile apps is the high user attrition rate, with the level of mobile app engagement dropping off over time [20]. Understanding the features and characteristics most valued by patients may improve app engagement. Users appreciate features that save time over current methods and identify an app as valuable when it is simple and intuitive to use, provides specific instructions to better manage a condition, shares data with designated individuals, and incorporates motivational “push” factors [20,21]. Increasing age is also associated with reduced use of smartphones for internet browsing, social media, and apps [22]. Consequently, mobile app attrition is likely to be increasingly challenging in the population of patients likely to use mobile apps such as CimplyMe; patients with chronic inflammatory diseases are likely to be older and so are likely to be less comfortable using mobile apps. Designing apps with the user in mind and assessing usability in human factors studies will help ensure successful patient engagement.

    Study Limitations

    Human factors studies only simulate use in an everyday environment. Patients were asked to behave in the way they thought they would if the scenario was real; however, this may be difficult to judge accurately. Each session was limited to 105 minutes, and, as a result, patients may have felt under some time pressure to complete each task. Time constraints also restricted patients’ ability to familiarize themselves with the app. Additionally, each task was completed discretely, which may have interfered with their familiarization of the app, owing to the lack of continuity between tasks. Patients were selected based on their willingness to use mobile apps, which may limit the generalizability of the results; however, the app will be provided as an additional support component and so use by patients will be voluntary. Therefore, the study population is reflective of the final population of patients who would likely use this in clinical practice. Finally, the number of patients participating in the study was small; therefore, the results may not represent the app’s usability in a wider patient group or capture all potential use errors and difficulties when using the app.

    Conclusions

    In a simulated setting, patients were able to use the companion app successfully. No critical task failures were observed; all patients were able to successfully complete both critical and noncritical tasks within two attempts without any formal training. This study demonstrates that the companion app UI is generally intuitive and easy to use among patients familiar with using apps. The app could support CZP patients to self-manage their treatment.

    Acknowledgments

    The authors thank the patients, the investigators, and their teams who took part in this study. The authors also acknowledge Susanne Wiegratz, UCB Pharma, Monheim am Rhein, Germany, for publication coordination and Oliver Palmer, BSc (Hons), Emma Phillips, PhD, and Simon Foulcer, PhD, from Costello Medical, United Kingdom, for medical writing and editorial assistance based on the authors’ input and direction; their support was funded by UCB Pharma in accordance with Good Publication Practice (GPP3) guidelines (http://www.ismpp.org/gpp3). The development of the CimplyMe app and this study were sponsored by UCB Pharma.

    Authors' Contributions

    BD, SV, and IM contributed substantially to study conception and design as well as the analysis and interpretation of the data; drafted the article or revised it critically for important intellectual content; and approved the final version of the article to be published.

    Conflicts of Interest

    BD, SV, and IM are employees of UCB Pharma.

    Multimedia Appendix 1

    Reasons for critical task difficulties and use errors. Only 12 patients completed critical task 6 (“log an injection date”) due to time constraints.

    PNG File , 267 KB

    References

    1. Udalova I, Monaco C, Nanchahal J, Feldmann M. Anti-TNF Therapy. Microbiol Spectr 2016 Aug;4(4). [CrossRef] [Medline]
    2. Curtis JR, Mariette X, Gaujoux-Viala C, Blauvelt A, Kvien TK, Sandborn WJ, et al. Long-term safety of certolizumab pegol in rheumatoid arthritis, axial spondyloarthritis, psoriatic arthritis, psoriasis and Crohn's disease: a pooled analysis of 11 317 patients across clinical trials. RMD Open 2019 May 31;5(1):e000942 [FREE Full text] [CrossRef] [Medline]
    3. Cimzia Summary of Product Characteristics. European Medicines Agency. 2018.   URL: http:/​/www.​ema.europa.eu/​docs/​en_GB/​document_library/​EPAR_-_Product_Information/​human/​001037/​WC500069763.​pdf [accessed 2020-06-29]
    4. Cimzia Prescribing Information. United Stated Food and Drug Administration. 2018.   URL: https://www.accessdata.fda.gov/drugsatfda_docs/label/2018/125160s283lbl.pdf [accessed 2020-06-29]
    5. Domańska B, Stumpp O, Poon S, Oray S, Mountian I, Pichon C. Using Patient Feedback to Optimize the Design of a Certolizumab Pegol Electromechanical Self-Injection Device: Insights from Human Factors Studies. Adv Ther 2017 Dec 8;35(1):100-115. [CrossRef]
    6. Singh K, Drouin K, Newmark L, Rozenblum R, Lee J, Landman A, et al. Developing a Framework for Evaluating the Patient Engagement, Quality, and Safety of Mobile Health Applications. Issue Brief (Commonw Fund) 2016 Feb;5:1-11. [Medline]
    7. Salimzadeh Z, Damanabi S, Kalankesh L, Ferdousi R. Mobile Applications for Multiple Sclerosis: a Focus on Self-Management. Acta Inform Med 2019 Mar;27(1):12-18 [FREE Full text] [CrossRef] [Medline]
    8. Mahmood A, Kedia S, Wyant DK, Ahn S, Bhuyan SS. Use of mobile health applications for health-promoting behavior among individuals with chronic medical conditions. Digit Health 2019 Oct 10;5:2055207619882181 [FREE Full text] [CrossRef] [Medline]
    9. Lee J, Choi M, Lee SA, Jiang N. Effective behavioral intervention strategies using mobile health applications for chronic disease management: a systematic review. BMC Med Inform Decis Mak 2018 Feb 20;18(1):12 [FREE Full text] [CrossRef] [Medline]
    10. Medical devices Part 1: Application of usability engineering to medical devices. ANSI/AAMI/IEC 62366-1. 2015.   URL: https://my.aami.org/aamiresources/previewfiles/6236601_1503_preview.pdf [accessed 2020-04-20]
    11. Faulkner L. Beyond the five-user assumption: Benefits of increased sample sizes in usability testing. Behavior Research Methods, Instruments, & Computers 2003 Aug;35(3):379-383. [CrossRef]
    12. Cahn A, Akirov A, Raz I. Digital health technology and diabetes management. Journal of Diabetes 2017 Nov 06;10(1):10-17. [CrossRef]
    13. Santo K, Chow CK, Thiagalingam A, Rogers K, Chalmers J, Redfern J. MEDication reminder APPs to improve medication adherence in Coronary Heart Disease (MedApp-CHD) Study: a randomised controlled trial protocol. BMJ Open 2017 Oct 08;7(10):e017540 [FREE Full text] [CrossRef] [Medline]
    14. Tabi K, Randhawa AS, Choi F, Mithani Z, Albers F, Schnieder M, et al. Mobile Apps for Medication Management: Review and Analysis. JMIR Mhealth Uhealth 2019 Sep 11;7(9):e13608 [FREE Full text] [CrossRef] [Medline]
    15. Santo K, Singleton A, Rogers K, Thiagalingam A, Chalmers J, Chow CK, et al. Medication reminder applications to improve adherence in coronary heart disease: a randomised clinical trial. Heart 2018 Aug 27;105(4):323-329. [CrossRef]
    16. Torbjørnsen A, Ribu L, Rønnevig M, Grøttland A, Helseth S. Users' acceptability of a mobile application for persons with type 2 diabetes: a qualitative study. BMC Health Serv Res 2019 Sep 06;19(1):641 [FREE Full text] [CrossRef] [Medline]
    17. Guo Y, Chen Y, Lane DA, Liu L, Wang Y, Lip GY. Mobile Health Technology for Atrial Fibrillation Management Integrating Decision Support, Education, and Patient Involvement: mAF App Trial. The American Journal of Medicine 2017 Dec;130(12):1388-1396.e6. [CrossRef]
    18. Pérez-Jover V, Sala-González M, Guilabert M, Mira JJ. Mobile Apps for Increasing Treatment Adherence: Systematic Review. J Med Internet Res 2019 Jun 18;21(6):e12505. [CrossRef]
    19. Faridi Z, Liberti L, Shuval K, Northrup V, Ali A, Katz DL. Evaluating the impact of mobile telephone technology on type 2 diabetic patients’ self-management: the NICHE pilot study. J Eval Clin Pract 2008 Jun;14(3):465-469. [CrossRef]
    20. Druce KL, Dixon WG, McBeth J. Maximizing Engagement in Mobile Health Studies. Rheumatic Disease Clinics of North America 2019 May;45(2):159-172. [CrossRef]
    21. Mendiola MF, Kalnicki M, Lindenauer S. Valuable Features in Mobile Health Apps for Patients and Consumers: Content Analysis of Apps and User Ratings. JMIR mHealth uHealth 2015 May 13;3(2):e40. [CrossRef]
    22. Kumar D, Hemmige V, Kallen M, Giordano T, Arya M. Mobile phones may not bridge the digital divide: a look at mobile phone literacy in an underserved patient population. Cureus 2019 Feb 20;11(2):e4104 [FREE Full text] [CrossRef] [Medline]


    Abbreviations

    CZP: certolizumab pegol
    e-Device: electronic device
    EU: European Union
    IFU: instructions for use
    RA: rheumatoid arthritis
    TNF: tumor necrosis factor
    UI: user interface


    Edited by G Eysenbach; submitted 10.12.19; peer-reviewed by I Contreras, X Ji, N Heynsbergh; comments to author 10.03.20; revised version received 05.05.20; accepted 03.06.20; published 27.07.20

    ©Barbara Domańska, Stijn Vansant, Irina Mountian. Originally published in JMIR Formative Research (http://formative.jmir.org), 27.07.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on http://formative.jmir.org, as well as this copyright and license information must be included.