This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.
Patient awareness of chronic kidney disease (CKD) is low in part due to suboptimal testing for CKD among those at risk and lack of discussions about kidney disease between patients and clinicians. To bridge these gaps, the National Kidney Foundation developed the Kidney Score Platform, which is a web-based series of tools that includes resources for health care professionals as well as an interactive, dynamic patient-facing component that includes a brief questionnaire about risk factors for kidney disease, individualized assessment of risk for developing CKD, and self-management tools to manage one’s kidney disease.
The aim of this study is to perform usability testing of the patient component of the Kidney Score platform among veterans with and at risk for kidney disease and among clinicians working as primary care providers in Veterans Affairs administration.
Think-aloud exercises were conducted, during which participants (veterans and clinicians) engaged with the platform while verbalizing their thoughts and making their perceptions, reasonings, and decision points explicit. A usability facilitator observed participants’ behaviors and probed selectively to clarify their comprehension of the tool’s instructions, content, and overall functionality. Thematic analysis on the audio-recording transcripts was performed, focusing on positive attributes, negative comments, and areas that required facilitator involvement.
Veterans (N=18) were 78% (14/18) male with a mean age of 58.1 years. Two-thirds (12/18) were of non-White race/ethnicity, 28% (5/18) had laboratory evidence of CKD without a formal diagnosis, and 50% (9/18) carried a diagnosis of hypertension or diabetes. Clinicians (N=19) were 29% (5/17) male, 30% (5/17) of non-White race/ethnicity, and had a mean of 17 (range 4-32) years of experience. Veterans and clinicians easily navigated the online tool and appreciated the personalized results page as well as the inclusion of infographics to deliver key educational messages. Three major themes related to content and communication about risk for CKD emerged from the think-aloud exercises: (1) tension between lay and medical terminology when discussing kidney disease and diagnostic tests, (2) importance of linking general information to concrete self-management actions, and (3) usefulness of the tool as an adjunct to the office visit to prepare for patient-clinician communication. Importantly, these themes were consistent among interviews involving both veterans and clinicians.
Veterans and clinicians both thought that the Kidney Score Platform would successfully promote communication and discussion about kidney disease in primary care settings. Tension between using medical terminology that is used regularly by clinicians versus lay terminology to promote CKD awareness was a key challenge, and knowledge of this can inform the development of future CKD educational materials.
Chronic kidney disease (CKD) is a chronic disease that requires individual participation in health-related behaviors to decrease the risk of progression and associated cardiovascular disease [
Reasons for the low prevalence of CKD awareness among individuals with CKD are varied and include patient, provider, and health system factors [
To bridge the communication gap about kidney disease among patients and health care professionals and increase testing among individuals at risk for CKD, the National Kidney Foundation (NKF) developed the Kidney Score Platform, leveraging the behavior change wheel, a validated framework used to design interventions to incite individual behavior change [
The patient-facing component of the Kidney Score platform includes a brief questionnaire about risk factors for kidney disease that results in a personalized educational results page providing an individualized assessment of risk for developing CKD or self-management tools to manage one’s kidney disease (
Example questions within the CKD risk self-assessment tool. CKD: chronic kidney disease.
Example self-assessment results, linking risk factors to kidney disease risk, providing education about CKD diagnostic tests, and encouraging patients to review diagnostic tests with their primary care clinician to increase awareness of CKD. CKD: chronic kidney disease.
Patient educational materials to promote CKD self-management. This page is available to individuals who document that they are aware of their own kidney disease. CKD: chronic kidney disease.
The Kidney Score Platform was field tested in 2 phases among 20 veterans and 19 clinicians from the VA NY Harbor Healthcare System (VA-NYHHS) and the VA CT Healthcare System at West Haven (VA-CTHS). We used a purposeful sampling approach using the electronic medical record to identify potential veteran participants who were English-speaking, active primary care patients between the ages of 18 and 75 years and who were living with diabetes or hypertension, the 2 most common causes of chronic kidney disease in the United States. Although having kidney disease was not an inclusion or exclusion criterion, we excluded veterans with very advanced kidney disease, including those with an estimated glomerular filtration rate (eGFR) <15 ml/min/1.73m2, those receiving dialysis treatments, and individuals who were kidney transplant recipients. Veterans who were unable to use a tablet or computer device (ie, blind, illiterate, with moderate-to-severe dementia) were also excluded. Potential veteran participants were mailed a flyer about the study and provided a phone number to opt out of the study. A research coordinator subsequently called veterans who did not opt out of the research to explain study goals and procedures, obtain consent, arrange for hard signatures of necessary forms, and schedule the think-aloud interviews.
Clinician participants included physicians and nurse practitioners who were actively engaged in primary care delivery at either Veterans Administration site. Recruitment of clinicians occurred via email by members of the research team. A research coordinator then followed up with eligible clinicians who responded favorably to schedule the think-aloud interviews.
After providing online documentation of informed consent, each participant joined an online platform using a numeric study ID. Two usability facilitators (one expert consultant external to the research team and DST, a nephrologist) introduced themselves and provided a brief overview of the goals of the project. Participants were asked again to acknowledge informed consent that participation was voluntary, could cease at any time, and that the session would be audiotaped but their privacy safeguarded. The study was approved by the institutional review boards at VA-NYHHS (approval #1705) and VA-CTHS (approval #02290). The COVID-19 pandemic required protocol modifications with appropriate institutional review board amendments to allow remote participation of veterans and clinicians in online think-aloud sessions in contrast to the original investigation design, which included face-to-face study interactions. The COVID-19 pandemic also extended the research timeline, creating 2 similar phases of work instead of 1.
Each session began with a warm-up interview exploring participants’ experience with online resources related to health and kidney health. Veterans were asked about experiences using web-based resources to gain information about their own health; clinicians were asked about use of online resources to communicate about chronic diseases, including but not limited to kidney disease. Thereafter, all participants were provided a weblink to the Kidney Score Platform website and were asked to participate in a think-aloud exercise, during which they were asked to engage with the platform while verbalizing their thoughts and making their perceptions, reasonings, and decision points explicit. Think-aloud exercises are increasingly being used to conduct user testing of digital health interventions [
During phase 1 (March 2020 to April 2020), audio recordings of field tests with 10 veterans and 19 clinicians were reviewed. Thematic analysis was performed, with focus on positive attributes, negative comments, and areas that required facilitator involvement. Clinicians were also asked how they might use this tool in clinical practice. Areas of improvement in the layout or design of the tool were quickly addressed by the research team and the NKF, which developed a second iteration of the Kidney Score Platform. Examples of changes included the following: reducing white space in between educational sections, rearranging location of kidney graphics, and replacing a picture of a heart with the word “love” for clarity. The Kidney Score Platform was reviewed by an additional 10 veterans during phase 2 of field testing (February 2021 to March 2021). Results from both phases were used to finalize the tool.
No formal hypothesis testing was performed due to the qualitative study methods of usability testing. Participants’ demographic characteristics are described using counts and percentages. Notes from direct (although online) observations by the usability facilitators and audio recordings of the field tests were reviewed. Thematic analysis was performed (without any special software), with a focus on themes derived from the data regarding positive attributes, negative comments, and areas that required facilitator involvement.
Veterans were 78% (14/18) male with a mean age of 58.1 years (range 27-71 years); see
Characteristics of study participants.
Characteristics | Veterans (N=18) | Clinicians (N=17)a | |||
Male gender, n (%) | 14 (78) | 5 (29) | |||
Age, mean (range) | 58 (27-71) | N/Ab | |||
|
|||||
|
African American | 11 (61) | 3 (18) | ||
|
Hispanic | 1 (6) | 0 | ||
|
Asian | 0 (0) | 2 (12) | ||
|
Non-Hispanic White | 6 (33) | 11 (65) | ||
|
|||||
|
Chronic kidney disease (eGFRc 30-60 ml/min/1.73m2) but no diagnosis | 5 (28) | N/A | ||
|
Diagnosis of diabetes | 9 (50) | N/A | ||
|
Diagnosis of hypertension | 9 (50) | N/A | ||
|
|||||
|
Nurse practitioner | N/A | 2 (12) | ||
|
Physician | N/A | 15 (88) | ||
|
Time since completing training (years), mean (range) | N/A | 17 (4-32) |
aData are missing for 2 clinicians who did not respond to demographic questions.
bN/A: not applicable.
ceGFR: estimated glomerular filtration rate.
All veterans and clinicians successfully navigated to the Kidney Score Platform website after being provided a link to the home page. Thereafter, they easily followed the self-assessment questionnaire portion of the Kidney Score Platform, aided by standard button labels and a simple progress indicator (eg, “3 of 8”) that provided a clear signal about survey length, which was also considered reasonable. The visual design of the personalized results page was appealing to all participants, who also appreciated the inclusion of infographics. Clinicians and veterans both recommended to reduce the overall text copy and include links to additional educational resources on the personalized educational results page.
Three major themes related to content and communication about risk for CKD emerged from the think-aloud exercises: (1) tension between lay and medical terminology when discussing kidney disease and diagnostic tests, (2) importance of linking general information to concrete self-management actions, and (3) usefulness of the tool as an adjunct to the office visit to prepare for patient-clinician communication. Importantly, these themes were consistent among interviews involving both veterans and clinicians.
Clinicians affirmed that phrasing of questions such as “Have you been told” or “Do you take medications for [disease name]” were likely to yield accurate responses during self-assessment rather compared to asking patients whether they have a particular disease. Providers were concerned that patients may not associate (or identify) with heart disease or high blood pressure even though they were actively taking medications to manage these conditions, especially if these chronic conditions were being well managed. However, a question asking about “Have you been told” directly relates to prior conversations about chronic diseases, regardless of management strategy or success.
Aligned with clinician feedback, veterans generally understood the phrasing “Have you been told you have [health condition]” and related questions appeared to garner accurate responses from patients regarding their health risks. Veterans correctly answered yes to being told they had high blood pressure or heart failure even if they were taking medications to manage the underlying condition. However, some individuals struggled to assess whether being at risk for diabetes meant they had prediabetes, in part because they had not heard of the term “prediabetes” before.
So my doctor never really told me I have prediabetes. He told me my A1C.
Nearly all clinicians were concerned about veterans not recognizing the medical terms “heart failure” or “chronic kidney disease.” Clinicians stated that during their communications with patients, they were more likely to describe the underlying disease rather than rely on disease names, for example, noting to patients that “your kidneys are not functioning properly” rather than using the term “chronic kidney disease.” To be consistent with the lay terminology used during clinic conversations, one primary care clinician suggested the following:
I wonder if a question like, have you been told that you have chronic kidney disease, whether it should also say something like, have you been told that you have any problems with your kidneys or if your kidneys don't work completely normally or something that might not be a phrase in medical jargon.
Two questions embedded in the self-assessment tool asked about eGFR and urine albumin-creatinine ratio (uACR). Clinicians doubted that any of their patients would know their eGFR or uACR.
I’m almost sure no one would know their GFR … I think only a very special person would know [their uACR] even more so than GFR.
Most clinicians noted that they rarely used these terms in clinical practice when discussing kidney disease with their patients. Instead, they relied on plain language descriptions (“urine test” for uACR) and simplified conceptual explanations (“kidneys functioning at 60%” to describe eGFR). Although several providers acknowledged that this was an imprecise translation, they thought that it was important to make the information more accessible to patients to promote CKD awareness.
The question that I get most often when I talk about CKD is, “What percent of my kidneys are still functioning?” Because GFR is a hard number to remember … I’ll tell people, “Listen, I think we lost 50% of your kidney function.”
Clinicians had a similar approach when discussing the presence of albuminuria.
When I talk to patients about having protein in their urine, I don't reference the number all that often. Though more so than with eGFR, I'll pull up the trend … But even then, I'll rarely refer to it by name.
As clinicians predicted, hardly any veterans were familiar with eGFR or uACR.
This is like a foreign language to me. I've never heard those words before.
For most patients, the terms themselves were new; all patients answered “I don’t know” to the questions asking about levels of eGFR and uACR (
I may not know what my numbers are, but I do know what the tests are, and I do know that I've had them done before.
Patients used lay terminology to describe eGFR and uACR, consistent with what was described by clinicians. Most veterans said that their primary care clinician would talk more about the significance of the laboratory results as opposed to using specific terms or values.
My doctor just said, “You don’t have protein in your urine.” I don't know the number or whether it was elevated.
I know that they have done urine tests in the past, and I know protein and sugar was in my urine.
Although clinicians appreciated the personalized nature of the individualized results page about risk factors for CKD (
I don't see much here for how to manage the risk factors… There's not much here about next steps.
If I were a patient, I'd like links to more info about kidney-friendly eating and exercising options. I would want to have information about medications to avoid, medications that can be helpful.
Similarly, while most patients understood from the results page that they were at risk for kidney disease, some did not read carefully enough to fully realize the perceived-risk concept between diabetes, heart disease, and kidney disease and that management of the diabetes and heart disease would help mitigate risk of kidney disease. Of those patients who read the results more carefully, some felt empowered by the information, while others reacted with alarm to the risk of CKD. In particular, veterans emphasized the importance of providing actionable education to help motivate individuals to change their personal risks for kidney disease without paralyzing them with the idea that kidney failure requiring dialysis or transplant was inevitable.
It's kind of gloom and doom; if I hit these thresholds, things may not be working properly, or may be approaching failure. So the message I get is: if you see these numbers, you're in deep trouble.
Although most veterans planned on speaking with their clinicians about the questionnaire results, some were seeking more actionable steps that they could take on their own. They yearned for more concrete recommendations that they could adopt.
It suggests exercise, but it’s not telling me the type of exercise to do.
What I would be mostly interested in is what is happening, why is it happening, and what can I do to slow it down? Anything I see that's clickable that touches on those points, I would be interested in clicking on.
When asked whether they would use the tool in clinical practice, most clinicians viewed the tool’s primary value was for educating patients prior to their next appointment. With limited time during office visits, clinicians did not think that they could review the tool in its entirety with patients; however, they viewed the tool as one that patients could use in preparation for an office visit that could be dedicated to a discussion about kidney disease and cardiovascular risk. Clinicians felt that patients would benefit from having time to go through the self-assessment and results on their own time, particularly if there were more actionable next steps identified for them.
If they can sort of generate the results and then bring them in, I think that would be helpful to have a discussion about where they are with their CKD and how we can help sort of reduce their risk of progression.
From my perspective, I wouldn’t use our visit time to go through this. Though maybe patients might find value in going through ahead of time.
Overall, most veterans felt the tool was useful since it made them aware of kidney disease. All participants said that they planned to email their CKD risk results to themselves to prepare for a discussion about kidney health with their clinician at their next visit.
Actually, I like the site better than I thought I would. … those questions were very precise and specific questions. As long as everybody's being honest when they're answering, I think the information that they're going to receive is going to be very useful.
There's some good information here. It sounds like it at least presents [information] to you a way to have this discussion with your doctor, and then see if they can test your blood or test your urine, and do the necessary tests to see where you are.
Although no patients knew their kidney-health lab values, most were intrigued enough that they planned to bring it up with their primary care clinician at their next visit.
It made me more interested in getting a test to see where I’m at.
For many, being asked about their kidney health values and not knowing the answer motivated them to speak to their clinician about CKD.
Well, let me put this way: I'm now well aware now of the significance of the kidneys and about what the issues are here. And I would definitely consider... When I go to the doctor, I would say to him, “Now, listen. You did the blood tests. But how are my kidneys doing? What are the numbers?”
Part of me is kind of mad there, because this a blind spot that me and my doctor, who I feel pretty comfortable with; we have not talked about CKD. I don't know if he didn't want to scare me, or maybe because he's concentrated on the prediabetes. I feel like I'm going to harass him about CKD now.
This is something new, so immediately I was like, just another thing to be concerned about. But then I felt kind of empowered, and like I really do want to get ahead of this thing. I feel like I do want to have a conversation with my primary care physician. CKD makes me feel better than end stage renal failure, so that makes me feel empowered because I'm at risk for chronic kidney disease, which is not end stage renal failure.
The Kidney Score Platform is an online educational tool that was developed to promote communication and discussion about kidney disease among patients and their primary care clinicians. Usability findings demonstrate that this goal was met, with the majority of patients finding the digital tool to be helpful and easy to navigate. Content areas that would benefit from refinement were also clearly identified.
The think-aloud exercises identified 2 key themes pertinent to the development of all educational materials related to kidney disease. First, there is a tension between using medical terminology that is used regularly by clinicians as well as reinforced in laboratory reports and electronic records versus lay terminology to educate patients and promote CKD awareness. This has been an area of debate in the nephrology field for quite some time [
The use of low grade-level vocabulary is an important component of adult education and written education materials [
Prior investigations that assess CKD awareness using lay terms such as “kidney problem” have shown incremental improvement but overall residual low awareness [
Connecting medical and lay terms may also require examples or approaches to enhance understanding. Conceptually, describing medical terminology like eGFR in ml/min/1.73m2 as a percentage of kidney function, as suggested by this investigation, may help patients understand the relationship between medical and lay terms. Using the test percent performance concept for percent of kidney function with 60 or higher being normal may also make conceptual sense to patients, as 60% is approximately the customary level for scholarly examination failure in the United States. In addition, some patients may better conceptualize eGFR and uACR with the images that were explored in this investigation, with heat map colors (from low risk) of green, yellow, orange, and (high risk) red. Study findings are hypothesis generating for future investigations regarding methods to connect medical and lay terminology as well as integration of images to illustrate the relationships.
The second key theme identified the importance of linking education about kidney disease with action-oriented recommendations that can decrease the risk of kidney disease or CKD progression. Clinicians and public health officials may consider awareness of kidney disease an important outcome on its own. However, patients may consider awareness of chronic conditions only as a means to an end—a transitory step that may not lead to improved health unless the education is coupled with actionable risk-reduction tools and motivational interviewing [
Both content and sequence are important elements of CKD educational media design. Workflow for time-constrained primary care clinicians is a major barrier to clear and effective bidirectional communication about kidney health, kidney disease risk, and the interplay with an array of cardiometabolic risk conditions. We designed the study to occur before the veteran-practitioner encounters to address this barrier that in turn contributed to the clinicians recommending that the Kidney Score Platform education be delivered before the visits. Additionally, the platform could also be used after a clinician visit, allowing a modified education tailored to the interventions emphasized in the encounter or the after-visit summary for reinforcement.
The results of this study should be taken in context of its limitations. Although our sample size was larger than the recommended range of 5 to 7 participants for usability testing [
Information derived from this usability study enabled the NKF to strengthen the Kidney Score Platform tool to promote its usefulness as an empowering adjunct to care and provided some key themes that will be applicable to the development of future educational materials.
chronic kidney disease eGFR: estimated glomerular filtration rate
National Kidney Foundation
urine albumin-creatinine ratio
VA NY Harbor Healthcare System
VA CT Healthcare System at West Haven
Lesley Hunter, Gary Bronz, and Anne Cordato provided operational and administrative support. Centralis provided facilitation for the usability studies.
None declared.