A Patient Safety Educational Tool for Patients With Chronic Kidney Disease: Development and Usability Study

Introduction

Chronic kidney disease (CKD) is characterized by progressive loss of kidney function and increases the risk of adverse patient safety events [1]. As kidney function declines, renal clearance is reduced, which can result in elevated and potentially hazardous circulating levels of medications. For example, individuals with CKD and concomitant diabetes who are on antidiabetic medications are at heightened risk of developing hypoglycemia due to the delayed renal clearance of these therapies [2]. Furthermore, as CKD progresses, susceptibility to hemodynamic changes or nephrotoxins such as nonsteroidal anti-inflammatory drugs (NSAIDs) is amplified, significantly increasing the risk of acute kidney injury (AKI) development and accelerated renal function decline [3,4].

Despite these well-established risks, patient awareness of CKD and of its potential safety hazards remains low, which limits the effectiveness of current CKD treatment strategies [5,6]. A high prevalence of limited health literacy among individuals with CKD [7] and poor patient-provider communication about CKD and its risk factors all contribute to low awareness and ineffective patient self-management among individuals with kidney disease [8]. Consequently, behaviors linked with CKD risk and progression, such as NSAID use, are commonly reported by individuals at high risk of adverse outcomes [9]; however, few interventions have been developed to address these challenges and attenuate risks.

We hypothesized that educational tools tailored to audiences with low health literacy can increase disease awareness and improve self-management among individuals with kidney disease. Therefore, we sought to develop a digital educational tool to promote patient awareness of relevant patient safety issues in CKD. Here, we describe the development of this educational tool; our experience usability testing its digital platform among the target patient population; and the results of our assessment of the users’ experience with the educational tool.

Methods

Educational Tool Development

This tablet-based educational tool was developed to support patient awareness of important disease-specific patient safety topics relevant for individuals with CKD. The tool was designed to evaluate participants’ awareness of common safety risks associated with reduced kidney function and to provide education regarding optimal patient-centered behavior to minimize risk. Materials were designed for low health literacy audiences, as it is estimated that the health literacy of approximately 25% of the general CKD population is inadequate [10]. The educational content was developed using Adult Learning Theory and plain language principles [11]. The andragogy theory of adult learning by Malcolm Knowles (Adult Learning Theory) postulates that adult learning is motivated by 6 principles: (1) the need to know, or “why” a person should learn; (2) a foundation or experience that provides the basis for learning activities; (3) a self-concept, or the idea that adults need to be responsible for and take charge of their learning; (4) readiness to learn stemming from perceived relevance of the knowledge to be gained; (5) orientation, or the concept that adults learn best with task-oriented learning that exercises their problem-solving ability; and (6) internal motivation [11]. The educational content of the tool draws on this framework to promote CKD awareness and self-management.

The educational objectives of the tool were based on patient safety topics relevant to individuals with reduced renal function based on prior work examining adverse safety events in CKD [1,12-14]. Educational content was created using previously derived educational materials addressing patient safety in CKD [15]; this content was reviewed and refined by adult educational curriculum experts using a rubric based on Adult Learning Theory [16]. The educational material encompassed four patient safety objectives pertaining to CKD: (1) avoidance of NSAIDs [17,18], (2) hypoglycemia awareness (among individuals with diabetes) [19], (3) temporary cessation of certain medications during acute volume depletion (ie, “sick day protocol”) [5,20,21], and (4) iodinated contrast risk awareness [22-24] (Table 1).

The developed teaching strategies optimized human-computer interaction and content retention. The hypothetical experiences of 2 diverse patients (Mr. Smith and Mrs. Johnson), with Hispanic and African American backgrounds, respectively, were used to provide clinical context (Figure 1). Simple animations accompanied the audio presentations in real time and emphasized key concepts [11,25]. Visual images and audio were chosen in consideration of information clarity, participant involvement, and captions to meet participant comprehension needs [26,27] (see Multimedia Appendices 1 and 2).

‎View this figure

Usability Testing

We recruited 12 patients receiving care from outpatient CKD clinics at Duke University Hospital in fall 2016 and again in early 2018. Individuals who were non–English-speaking, illiterate, physically unable to use a tablet (ie, individuals with limb amputation, debilitating arthritis, or legal blindness), on dialysis, or under the age of 18 years were excluded. If interested, patients were taken to a private room, where written consent was obtained and testing was initiated. The study was approved by the Institutional Review Board of Duke University. All participants were compensated for their time.

Testing Protocol

The usability testing was based on the performance of the users and their ability to answer questions and navigate the educational tool on a tablet device. Participants were given a pair of headphones connected to an Android tablet (Samsung Galaxy Tab A, model SM-T550). Once seated comfortably, participants were asked to complete a series of tasks designed to evaluate their ability to interact with the technical aspect of the safety tool (Multimedia Appendix 3). All participants were asked to complete a minimum of 52 tasks and a maximum of 67 tasks based on their diabetes status. The study facilitator remained in the room with the participant and monitored the ease or difficulty with which tasks were completed as well as participant commentary about the activity. Task completions were categorized as no error, noncritical error, or critical error. If the participant had no difficulty completing the task independently, the task was marked as no error (easily completed). A noncritical error was defined as a task that was completed with an alternative strategy. For example, if a participant chose a wrong answer but made the correction before selecting “check my answers,” the error was categorized as noncritical. A critical error was defined as a task that could not be completed independently by the participant without assistance from study personnel. After completing the tool, participants were asked to evaluate their user experience with the tool.

Statistical Analysis

No formal hypothesis testing was performed in this analysis due to the qualitative study methods of usability testing. Participants’ demographic characteristics and their typical use of the internet and mobile devices were described using counts and percentages. Participant satisfaction using the educational tool and the tablet was recorded. Critical and noncritical errors were evaluated for each subject along with the time to complete each exercise. Errors and completion times were analyzed to recognize any significant differences between participants who regularly use the internet and mobile apps and participants who do not.

Results

Usability Testing

Twelve participants took part in the usability testing of the patient safety educational tool. The majority of participants were aged 45 years and older (11/12, 92%) and were female (7/12, 58%) (Table 2). The races of the participants were diverse. All participants had at least completed high school and owned a mobile device that they used daily. Half the population owned a tablet they used for internet and mobile apps. Most participants used mobile apps (9/12, 75%) and the internet (8/12, 67%).

Figure 2 illustrates user performance by displaying the numbers of critical errors, noncritical errors, and easily completed tasks out of the total assigned tasks. Out of 725 tasks, 15 critical errors (2.1%), 31 noncritical errors (4.3%), and 679 easily completed tasks (93.7%) were recorded. Of the 12 participants, 10 (83%) easily completed at least 90% of their tasks. The median time to complete all tasks was 19 minutes (SD 3.4 minutes, range 11-26 minutes). The median time for internet users to complete all tasks was 18 minutes (range 11-21 minutes); in contrast, participants who did not use the internet completed all tasks with a median time of 21 minutes (range 20-26 minutes). There was 1 outlier who completed all tasks in 11 minutes. The participants who used the internet had fewer errors than those who did not: 4/725 errors versus 42/725 (0.55% versus 5.8%, respectively). The 3 participants who did not use the internet or mobile apps were all aged 65 years or older. Five participants completed the testing without any errors, and 2 others completed it with only 1 error. Participants 4 and 8 had the most difficulty, with 7 and 30 errors, respectively. Both of these participants were over the age of 65, did not own a tablet, did not use the internet, and did not use mobile apps. All subjects completed more than 50% of the exercises without any errors, and 11/12 subjects (92%) completed more than 90% of the tasks without critical errors.

‎View this figure

User Experience Survey

The Likert scale used to evaluate user satisfaction using the tablet application is shown in Table 3. Participants were asked questions about their experience using the app and provided a ranking for the activity from 0-2 or 0-4, with lower numbers indicating higher satisfaction. The median and range for the ranking of the tool were used to determine the results. The median ranking for the overall activity was 0 (range 0-4), which describes the activity as “very easy.” Participants found the tablet to be very easy to use based on the median ranking of 0 (range 0-4). The length of time to complete the activity had a median ranking of 1 (range 0-2), indicating that the length was “just right.” The subjects ranked the patient story examples and audio as 1 (range 0-4), stating they agree that these aspects of the app helped them to understand the information. When participants were asked if they would recommend the activity to others, the median ranking was 1 (range 0-4), indicating that the participants agree they would recommend the app to others. Lastly, the participants were asked how they would rate the activity on a scale of 1-10 (10 being the best); they provided a median rating of 8 (range 0-10).

After completion of the activity, participants were given the opportunity to provide feedback and comments about the tool, such as what changes they would make, what they liked or did not like, and their overall thoughts about the exercise. Most comments reiterated the helpfulness and quality of the material learned using the app. Other comments suggested ways to improve the tool by stating that the exercise should be faster to navigate, while others expressed the need to simplify some medication names. The majority of participants found the educational tool to be a worthwhile experience that helped them learn about adverse safety events (Multimedia Appendix 4).

Discussion

The CKD educational tool was designed to help audiences with low health literacy learn about potential safety hazards associated with kidney disease through an interactive and simple-to-use digital interface. Our findings demonstrate that the majority of participants found the digital tool to be helpful and easy to navigate and would recommend it to others. Most participants were proficient in using the tablet without significant guidance and without errors; those with critical errors were older than 65 years and less familiar with digital technologies in general but still predominantly completed the programming without difficulty. The participants’ ability to complete the tasks did not appear to be related to their degrees of education, which supports our intent to provide an educational tool for populations with low health literacy regardless of educational background. Although some participants did not regularly use the internet or mobile apps, the task completion times in this group did not greatly differ from those of regular internet users. Together, these findings suggest that further testing to evaluate the impact of the CKD educational tool on outcomes is warranted.

CKD places individuals at high risk for developing adverse safety events, and effective tools that attenuate this heightened risk are needed. The principal feature of CKD that can influence patient risk is the reduced level of renal function or glomerular filtration rate (GFR), which is typical of the condition. A low GFR affects the clearance of many drugs, confounds therapeutic interventions, and increases susceptibility to acute kidney injury [28]. Moreover, CKD is frequently associated with several comorbidities, such as diabetes and cardiovascular disease [28]. The clustering of these health conditions leads to increased inpatient and outpatient medical encounters, with potential consequences of polypharmacy and high self-management burden. Compounded by low patient awareness of kidney disease [29] and poor knowledge of contributory health behaviors [5,30], potential adverse outcomes in CKD are abundant.

Our previous work demonstrated high acceptance of digital tools and general ease of use among individuals with CKD, supporting mobile health (mHealth) as a feasible mechanism through which patient-centered programming can be delivered to CKD populations [31-33]. Other studies using mHealth tools in non-CKD populations have demonstrated similar results for provision of tailored educational programming across an array of health conditions, such as obesity, asthma, and diabetes [34-37]. For individuals with CKD, responsiveness to the high degree of limited health literacy in the CKD population [10,38] is fundamental to the development of any educational curriculum, and use of audio and visual support can address differences in learning styles. We developed our app using plain language principles to address this need. Similarly, user experience testing of such digital tools in the CKD population can unearth interface difficulties by users and inform refinements to optimize user understanding and retention. The importance of such assessment is evidenced by one recent study that systematically evaluated all available and updated patient-facing CKD mobile apps. Of the 174 unique applications found, only 38 were related to kidney disease, were patient-facing, and had been updated in the previous 4 years. The quality of app content widely varied, and the authors found high levels of discordance between patient and provider app reviewers regarding value and usability [39]. These findings highlight the need for digital health content that is developed with a patient-centric approach and tested for ease of use in the target population prior to implementation in clinical care and broad dissemination. Similar content should also be made available through alternative platforms, such as on paper or via a website, to address differences in digital readiness and access.

Empowering patients with proper information and education in a manner they can comprehend can improve self-efficacy and self-management and, in turn, improve health outcomes [40]. The patient-actionable tool we developed emphasizes the importance of a patient-centered approach to educating CKD patients about self-care at home. Further, developing digital tools for patient education can bridge the digital gap between younger and older generations. When provided with online tools to manage chronic illnesses, older individuals access the digital content more frequently and sustain its use over longer periods than their younger counterparts [41]. This engagement by older populations is particularly relevant in CKD, as its onset typically occurs later in life. Digital tools can also narrow disparities in accessibility between rural and urban CKD populations, especially with the widespread availability of smartphones and the mobile web as well as the increasing use of telehealth applications for clinical care [42,43]. However, in kidney disease, few tools have been appropriately developed for use in a low health literacy population or have undergone formal usability testing. Fewer still tools are available outside of the research setting, which creates opportunities for such tools to be integrated into clinical care and evaluated in pragmatic studies [44,45].

Our study has limitations that are worthy of mention. Although our sample size is within the recommended range of 5-7 participants for usability testing [46,47], our sample size is small, and we recognize that this limits the inferences was can draw from our findings. Further, our participants were recruited from the Duke University Hospital nephrology clinics and may not be representative of the general CKD population, particularly given that only half our study participants were elderly. We also restricted our usability testing to individuals with CKD, which may not be generalizable to individuals with other kidney-related (eg, acute kidney injury) or non–kidney-related conditions. Finally, the specific patient safety conditions included in our educational curriculum may not be comprehensive, although they do represent many common patient safety events reported and detected in patients with kidney disease [1,48].

In summary, usability testing of a patient-centered digital tool to promote safety among patients with CKD demonstrated general ease of use and acceptability. The impact of the intervention on mitigation of safety events in a CKD population has yet to be determined. However, the information derived from this study strengthens the growing consensus that tailored digital tools can be used effectively by high risk populations, including older persons and persons with low health literacy. Future studies are needed to evaluate the benefit of such tools in clinical care.