This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.
Improvements in the digital capabilities of health systems provide new opportunities for the integration of patient-reported outcome (PRO) solutions in routine care, which can facilitate the delivery of person-centered diabetes care. We undertook this study as part of our development of a new digital PRO diabetes questionnaire and clinical dialog support tool for use by people with diabetes and their health care professionals (HCPs) to improve person-centered diabetes care quality and outcomes.
This study evaluates the feasibility, acceptability, and perceived benefits and impacts of using a digital PRO diabetes tool,
Overall, 12 people with diabetes scheduled for routine medical diabetes visits at the outpatient clinic were recruited. Purposive sampling was used to optimize heterogeneity regarding age, gender, duration, type of diabetes, treatment modality, and disease severity. Participants filled out a PRO diabetes questionnaire 2 to 5 days before their visit. During the visit, HCPs used a digital PRO tool to review PRO data with the person with diabetes for collaborative care planning. Participants completed evaluation forms before and after the visit and were interviewed for 30 to 45 minutes after the visit. HCPs completed the evaluation questionnaires after each visit. All visits were audio-recorded and transcribed for analysis. Data were analyzed using quantitative, qualitative, and mixed methods analyses.
People with diabetes found the PRO diabetes questionnaire to be relevant, acceptable, and feasible to complete from home. People with diabetes and HCPs found the digital PRO tool to be feasible and acceptable for use during the diabetes visit and would like to continue using it. HCPs were able to use the tool in a person-centered manner, as intended. For several people with diabetes, completion of the questionnaire facilitated positive reflection and better preparation for the visit. The use of the PRO tool primarily improved the quality of the dialog by improving the identification and focus on the issues most important to the person with diabetes. People with diabetes did not report any negative aspects of the PRO tool, whereas HCPs highlighted that it was demanding when the person with diabetes had many PRO issues that required attention within the predefined time allocated for a visit.
The Danish PRO diabetes questionnaire and the digital tool,
Successful diabetes care requires a whole-person, collaborative care approach that focuses on an individual’s biological, psychological, and social health, well-being, functioning, values, preferences, and priorities [
Digital patient-reported outcome (PRO) solutions for clinical practice may help improve aspects of the care experience regarding person-centered chronic illness care [
We developed a new digital tool,
This study aims to evaluate the feasibility and acceptability of the PRO diabetes questionnaire and the first viable version of
How do people with diabetes experience the feasibility, acceptability, relevance, comprehension, and adequacy of topic coverage of the PRO questionnaire when used as intended in the context of a routine visit?
How do people with diabetes and HCPs experience the use of
Specifically, do people with diabetes and HCPs experience the intended and hypothesized benefits of the PRO tool in improving patient participation and quality of the dialog?
As part of the formative process, this study additionally aims to facilitate the refinement of research hypotheses and finalize evaluation questionnaires for use in future real-world studies on the implementation and effectiveness of the
This study was a formative, mixed-methods, single-arm, acceptability, feasibility pilot study that evaluated an office-based digital PRO tool intervention at an outpatient diabetes clinic.
The eligibility criteria were age >18 years, diagnosis of type 1 or type 2 diabetes, diabetes duration of at least 1 year, and planned attendance for a routine visit at the diabetes outpatient clinic during the study period. Exclusion criteria were severe mental illness or major cognitive or language difficulties that would prevent the ability to fill out the diabetes questionnaire.
We used purposive sampling and consecutive recruitment to maximize the representation of type and duration of diabetes, age, gender, treatment regimen, and disease severity. Eligible participants were identified by the study nurses from the electronic booking system in the diabetes clinic and invited to take part in the pilot study by telephone. The study was described as a pilot test of a diabetes questionnaire designed to help improve the quality of diabetes visits. All participants signed informed consent before study enrollment.
The study was approved by the local institutional review board and deemed out of scope for the ethical review board because of the absence of a clinical treatment or intervention and limited risk.
The study design and intervention included the following:
Each participant received a secure email with a weblink for mobile or web-based access to complete informed consent, the PRO diabetes questionnaire, and an evaluation questionnaire about the PRO questionnaire. The participant was asked to complete this questionnaire 2-10 days before their scheduled visit. If it was not possible to do it at home, the person with diabetes was encouraged to contact the clinic to arrange on-site completion.
HCPs were able to immediately access PRO results once completed by the person with diabetes but were asked to only access results immediately before the visit to mimic routine diabetes care.
At the visit, HCPs used the
The HCPs and people with diabetes completed the evaluation questionnaires immediately after the visit about their experiences using PRO. HCPs filled out a web-based form describing any issues, errors, or concerns relating to the PRO results and how they were displayed using algorithms.
People with diabetes were interviewed for 30-45 minutes using a semistructured interview guide right after the diabetes visit by a researcher not involved in care for that person with diabetes. The PRO dashboard and questionnaire results were available in the interview to facilitate a detailed discussion with the person with diabetes about any feedback to the individual’s PRO results, as shown on the PRO dashboard. It was emphasized upfront to people with diabetes that their feedback via questionnaires and interviews was kept strictly confidential and would not be shared with the clinical care team.
The aim of the PRO diabetes intervention was to increase the active participation of people with diabetes in their own care and improve the quality of the dialog between people with diabetes and HCPs, and overall care quality by focusing on optimizing value for people with diabetes [
Basic steps of the patient-reported outcome diabetes intervention in clinical practice. PRO: patient-reported outcome.
People with diabetes completed the PRO diabetes questionnaire by phone, tablet, or PC using
This formative study is a part of the participatory development process of the digital PRO tool
Each PRO item or scale score is shown on the
The interface for people with diabetes includes a user-friendly digital interface for questionnaire completion, which was developed and tested with people with diabetes using an iterative participatory process with user-testing to optimize user-friendliness. Only one question is depicted on the screen at a time to facilitate ease of use and lower cognitive burden.
The dashboard provides a one-screen instant overview of PRO results by presenting the results in 9 main themes. On the right side of the screen, the person with diabetes’ own priorities for self-management support and topics to discuss are flagged for use as a starting point for the dialog. By clicking or touching the screen, the HCP can access dialog tips, information resources, local treatment, and referral information relevant for each PRO output.
Screenshot from the digital patient-reported outcome diabetes tool,
A panel of people with diabetes who represented the target group for the PRO intervention were involved as partners in this study to ensure that the perspective of people with diabetes was considered at all research phases [
Clinical charts (HbA1c, cholesterol, blood pressure, and complication data), sociodemographic data, and treatment data (age, gender, duration of diabetes, type of diabetes, medical therapy, and technology use) were collected from all study participants using chart reviews and questionnaires. PRO and evaluation questionnaires were administered to people with diabetes and HCPs using the
An overview of the evaluation questionnaires is shown in
All consultations and semistructured interviews were audio-recorded and transcribed verbatim. Four consultations were observed in person by a clinical diabetes psychologist for HCP supervision purposes to complement informant perspectives and assess any potential risks related to the way psychological issues are identified, addressed, followed up on, and reacted to.
Completed by people with diabetes at home in advance of the visit. Evaluates general health and life situation, well-being, depression, symptom distress, annual check of feet and eyes, daily life with diabetes, worries about diabetes, confidence in managing diabetes, blood sugar regulation and hypoglycemia, medicine experience, access to health care professionals (HCPs), priority areas for self-management support, and preferred topics to focus on for the visit.
Completed by people with diabetes immediately after the PRO questionnaire; evaluates perceptions of (1) relevance, (2) difficulty, (3) comprehension, (4) topic coverage/comprehensiveness, (5) acceptability, and (6) item-specific issues.
Completed by people with diabetes immediately after the PRO questionnaire diabetes visit; evaluates perceived (1) support for autonomy and person-centered communication; (2) PROs impact on dialog, role, and care; (3) potential negative impacts; (4) face validity of scoring algorithms; (5) interest in continued use and advocacy; and (6) suggestions for improvement.
Completed by HCPs immediately after the PRO questionnaire diabetes visit; measures perceived (1) quality of visit; (2) PROs impact on dialog, roles, visit outcome; (3) challenges with the use of PRO; (4) satisfaction and interest in future use; and (5) clinical validity of items and algorithms.
Quantitative data (blood tests, sociodemographic, PRO questionnaire, and Likert scale evaluation questionnaire) and qualitative data (consultation and interview transcripts, free-text evaluation responses, and notes from debriefing meetings) were analyzed in SPSS Statistics 25.0 for Windows (IBM Cooperation) and NVIVO 12.0 (QSR International), respectively. Primary quantitative evaluation data were presented descriptively. The main qualitative analysis used a simple stepwise coding process adapted from thematic analysis and a phenomenological and combined inductive and deductive approach [
The characteristics of the 12 people with diabetes enrolled are shown in
Characteristics of study participants (N=12).
Characteristics | Values | ||
Gender (female), n (%) | 7 (58) | ||
|
|||
|
Type 1 | 8 (67) | |
|
Type 2 | 4 (33) | |
Age (years), median (range) | 56.6 (24-79) | ||
Duration (years), median (range) | 19.5 (2-50) | ||
|
|||
|
Insulin pen | 7 (58) | |
|
Insulin pump | 2 (17) | |
GLP-1a, n (%) | 3 (17) | ||
Tablet, n (%) | 1 (8) | ||
|
|||
|
Mean | 1.58 | |
|
Median (range) | 1 (0-8) | |
|
|||
|
Mean | 1.3 | |
|
Median (range) | 0.5 (0-4) | |
|
|||
|
HbA1cb (mmol/mol) | 85.8 (20.7; 61-113) | |
|
Health (SF-1c; score range 1-5) | 3.0 (0.7; 2-4) | |
|
Well-being (WHO-5d; score range 0-100) | 60.4 (20.7; 20-96) | |
|
Diabetes-specific distress (PROe Diabetes Questionnaire–Negative Impact of Diabetes Scale, three-item distress scale; score range 0-100) | 41.0 (14.0; 8-58) | |
|
Number of PRO topics flagged for action (scored with a |
3.8 (2.7; 0-10) | |
|
Number of PRO topics flagged for action (scored with a |
4.8 (3.0; 1-11) | |
|
Total number of PRO topics flagged for action (yellow and red topics and additional topics selected by people with diabetes) | 14.3 (6; 4-26) |
aGLP-1: glucagon-like peptide-1.
bHbA1c: glycated hemoglobin A1c.
cSF-1: global health item of the Short-Form Health Survey.
dWHO-5: World Health Organization–Five Well-Being Index.
ePRO: patient-reported outcome.
The results from the quantitative evaluation of the PRO questionnaire by people with diabetes are shown in
All the people with diabetes were positive about the relevance of the questions for their diabetes, except 1 person with diabetes who indicated a moderate negative appraisal. All people with diabetes found it easy to complete the entire questionnaire. Of the 12 participants, 6 (50%) indicated no items, and 6 (50%) indicated
Quantitative evaluation of the patient-reported outcome diabetes questionnaire by people with diabetes (N=12)a.
Response options | Value | |||
|
||||
|
Mean score (SD) | 3.5 (0.7) | ||
|
|
|||
|
|
1 | 0 (0) | |
|
|
2 | 1 (8) | |
|
|
|||
|
|
3 | 4 (33) | |
|
|
4 | 7 (67) | |
|
||||
|
Mean score (SD) | 3.6 (0.7) | ||
|
|
|||
|
|
1 | 0 (0) | |
|
|
2 | 0 (0) | |
|
|
|||
|
|
3 | 6 (50) | |
|
|
4 | 6 (50) | |
|
||||
|
No | 9 (75) | ||
|
Yes, one or a few | 3 (25) | ||
|
Yes | 0 (0) | ||
|
||||
|
No | 10 (83) | ||
|
Yes | 2 (17) |
aShows the responses of people with diabetes to the PRO-EVAL-P-1A pilot evaluation questions immediately after completing the PRO questionnaire.
bA score of 1 and 2 reflects a negative appraisal, and 3 and 4 represents a positive appraisal.
The mean single-item scores by item for the primary evaluation questions are shown in
All 12 people with diabetes rated the person-centered autonomy-supportive communication style of the HCPs positively, with mean scores ranging between 4.5 and 4.8, with a score range of 1-5. People with diabetes felt that their HCPs were focused on their priorities, encouraged them to speak, and made them feel comfortable talking about their needs, and all felt they got the care and advice that they had hoped for. In order of decreasing positive rating, people with diabetes expressed high interest in continued use, that PRO should be offered as standard care to all, that PRO helped focus on what was most important to them, that PRO helped focus the conversation on what mattered most to them, that they would like to use PRO in their future care, that they felt better prepared for the visit, and that the PRO dashboard provided a good picture of their current diabetes situation, needs, and priorities. There was only 1 person with diabetes who indicated a moderate degree of being uncomfortable or having a problem related to the use of PRO. During the interview, where answers were debriefed, she explained that she had had a bad day, was feeling very distressed because of diabetes, and had not felt the HCP understood her issues as they were raised. Despite this, she was very positive about the PRO tool and did not attribute the problem to the tool.
Quantitative evaluation of the use of patient-reported outcome (PRO) during the visit by people with diabetes (N=12)a.
Items | Single-item scoreb, mean (SD) | |
|
||
|
Did the HCPc make you feel comfortable talking about the topics that you needed to? | 4.8 (0.6) |
|
Did the HCP give you the treatment, advice, referral, or other assistance that you needed? | 4.6 (0.5) |
|
Did the HCP focus on what is most important to you? | 4.5 (0.9) |
|
Did the HCP encourage you to give input or ask questions? | 4.5 (0.7) |
|
||
|
Interested in using PRO in future care | 4.9 (0.3) |
|
PRO should be part of standard care | 4.8 (0.4) |
|
The |
4.7 (0.3) |
|
HCP was more or less prepared because of use of PRO | 4.4 (0.9) |
|
I felt more or less prepared because of use of PRO | 4.4 (0.9) |
|
PRO dashboard gave a good picture of my situation, needs, and priorities related to my diabetes | 4.3 (1.0) |
|
Any problems or uncomfortable experiences related to the use of |
1.3 (0.6) |
aMean single-item scores of evaluation by people with diabetes of use of PRO after the visit using the PRO-CON-EVAL-P-1A-pilot questionnaire.
bScore range is 1-5, except for the last item, which is 1-3.
cHCP: health care professional.
The main themes identified from the qualitative analysis of the interviews are shown in
People with diabetes expressed that they felt the PRO questionnaire covered all relevant issues, was straightforward to fill out, and facilitated positive self-reflection. In line with the questionnaire evaluations, approximately half of the people with diabetes reported minor issues with understanding one or a few items.
Most people with diabetes expressed that they found it positive to complete the questionnaire at home in advance, as it helped them know what the conversation would be about at the visit. Filling out the questionnaire made the people with diabetes feel reassured that they would remember and get to talk about their priority issues with their HCP. This was important as several people with diabetes expressed frustration that they would often forget to talk about the issues that mattered to them during visits. A family member who participated with a study participant explained the following:
It is a nice thing to fill it out at home, and the HCP is also better prepared for what you want to ask about. I would like to have this every time.
Several people with diabetes had a positive personal experience filling out the questionnaire. One person with diabetes said the following:
When I sat down with the questionnaire, I had some time for it, and I felt really positive and surprised. Because it went straight in and touched on some issues where I had to feel and actually remove the shutters and relate to it. ”How am I doing? If I am totally honest, how is it going with this?“ It is easy to say, ”well it’s going to be fine.“ Let’s just keep going as usual. So, in this way it was really an eye-opener for me.
Only one person with diabetes expressed an uncomfortable situation related to the use of PRO during the visit, as noted earlier, and it was clarified in the interview that the person with diabetes did not attribute the issue to the PRO tool but attributed it to not feeling fully understood by the HCP.
Analysis of semistructured interviews with people with diabetes.
Theme and subcategories | Quotes for illustration | |||
|
||||
|
|
|||
|
|
Easy to do A positive experience No or few items difficult to understand |
“I felt it was easy and I think it was well laid out with 5 response options every time–and I just felt it was simple” | |
|
|
|||
|
|
All questions relevant No key topics missing 360° coverage is good |
“It gets into all the issues, which I think is good because it makes you think about stuff you might not otherwise have” | |
|
||||
|
|
|||
|
|
Self-reflection and self-insight |
“It motivated me hugely to get some things on the table that I have been needing to talk about but closed my eyes to because I tend to just say things are 'kind of fine'?” | |
|
|
Comfort knowing own priorities will be addressed in the visit |
“Many times I feel I forget when I get to the visit, shoot, this or that I forgot to ask about when you sit there–and when you leave you haven’t asked about what you needed. So, this is really super this tool” | |
|
||||
|
|
|||
|
|
PRO screen was easy and intuitive in visit |
“I think it was great to see [DiaProfil screen]–it made sense; red is bad, yellow is less bad and green that is perfect-ish, right?–I think it gave a good picture [of my situation] and it was easy for me to grasp it” | |
|
|
HCP talked about PRO results in a pleasant way |
“I liked the way that she [HCP] went at it right away. She made me actually feel reassured by showing me the screen and just mentioning e.g. there was this box with a mental issue” | |
|
||||
|
|
|||
|
|
HCP is better prepared |
”It felt very nice [that the nurse had seen my PRO answers], because I am thinking if she has read it through she might see some connections between my issues – makes sense.“ | |
|
|
Cover new important topics that matter to me |
“We covered some other topics than normal, because I usually just get the numbers [blood test results] and sometimes gets measured and weighted and then go home again. It got more personal. And I think that was awesome.” | |
|
|
Easier to talk about difficult or sensitive issues |
”It was a bit easier to sit at home and write that I actually would like to get some help to stop smoking than to sit down here–now it’s out in the open without having to say it face to face. It also makes it easier for the HCP I think–you can maybe get to talk about some of the difficult topics that you wouldn’t just sit there and say.“ | |
|
|
Getting better care |
“I get better help by answering these questions” |
aPRO: patient-reported outcome.
bHCP: health care professional.
The main results from the HCP questionnaire evaluation of the visits are shown in
Results of questionnaire evaluation of the use of the patient-reported outcome (PRO) diabetes tool by health care professionals (N=12)a.
Item contentb | Mean single-item score (SD) | |
|
||
|
How would you rate the overall quality of the dialogue with this patient? | 4.25 (1.1) |
|
Did you cover all the topics in the visit that were important to you from a clinical perspective? | 4.4 (0.7) |
|
||
|
How interested are you in using |
9.25 (1.0) |
|
How satisfied or dissatisfied were you overall with use of |
4.6 (0.7) |
|
Was the person with diabetes was more or less prepared for your dialog due to answering the PRO questionnaire? | 3.75 (0.75) |
|
Do you feel that your use of |
3.5 (1.0) |
|
Did you experience that the person with diabetes got to speak more or less during this visit due to use of |
3.6 (0.5) |
|
Did DiaProfil make you aware of clinically relevant issues for this patient you were not aware of before? | 2.67 (1.3) |
|
Did you experience challenges due to the use of |
2.33 (1.4) |
aMean single-item scores of health care professional evaluations of use of patient-reported outcome in the visit (PRO-CON-EVAL-P-1A-pilot).
bScore range is 1-5. Score of 1-2: negative; 3: neutral; and 4-5: positive (except for the question on interest in continued use, which has a score range of 1-10).
Themes and illustrative quotes from the HCP’s evaluation of the visits are shown in
Analysis of open-ended text evaluations by the health care professional after each visit.
Main themes | Quote or case example | |
|
||
|
The person with diabetes was better prepared due to self-reflection in advance | “I tend to ask a lot of questions if the patient does not say so much. In this case the patient had already reflected and prioritized which allowed us to focus on this instead of ‘shooting in the blind’.” |
|
Better able to set agenda in line with patient priorities | “I distributed the available time better, focused on the problems of the patient, listened more” |
|
New insights about which topics are important to the person with diabetes. | New topics identified as important to the person with diabetes included worry about complications, foot wound, erectile dysfunction, sleep, pain, and barriers in life situation to managing diabetes. |
|
||
|
Managing the conversation when there are many flagged PRO topics | “Dialogue would have been better if we had had more time to wrap up the various problem areas identified” |
|
||
|
Red score on pain | A person with diabetes scored red on pain, but it was because of arthritis pain that was already treated and addressed in other care setting. |
|
Red score on low well-being | A person with diabetes scored red on low well-being, but it was because of life issues unrelated to diabetes. |
|
Uncertainties regarding use | Unsure how to handle a discrepancy between a PRO score and what the person with diabetes says in the visit. |
aPRO: patient-reported outcome.
The results of the content analysis of the audio recordings of the visits are summarized in
Many PRO topics prompted by the HCPs were validated as relevant by the person with diabetes and, for the most part, resulted in relevant follow-up dialog and, in some cases, action. However, in line with HCP evaluations, there were several instances where topics raised pertaining to, for example, symptoms unrelated to diabetes were found not to lead to concrete action or follow-up plans. By cross-matching the topics discussed during the visit with the PRO data, we identified some individual errors because of mistakes by people with diabetes during questionnaire completion; however, we could exclude any structural PRO assessment problems and could confirm the clinical validity and utility of the PRO outputs. Observations of a subset of consultations by a clinical diabetes psychologist provided additional reassurance that it was possible for HCPs to incorporate PRO data in a person-centered manner into the dialog. Observations provided input to our future person-centered training for the use of PRO, especially regarding identifying and clarifying previously undetected psychosocial problems.
Results of analysis of audio-recording transcripts of patient-reported outcome (PRO) diabetes consultations (N=12).
Category | Case | Result, n (%) | |
|
|||
|
HCPa used open-ended questions to prompt people with diabetes about a flagged PRO topic or result | “Is this something that you recognize?” [HCP] |
12 (100) |
|
HCP showed and explained the PRO dashboard | “What you answered in the questionnaire is shown on this screen.” [HCP] |
10 (83) |
|
HCP explicitly used a PRO to set visit agenda | “And then there is red which we definitely should talk about...” [HCP] |
8 (67) |
|
All flagged PRO topics were mentioned during the visit. | “You have indicated you feel that diabetes takes up too much of your daily life?” [HCP] | 12 (100) |
|
|||
|
At least one PRO topic was prompted by the HCP, validated as relevant by the person with diabetes, and followed up with actionb | Case example: |
12 (100) |
aHCP: health care professional.
bActions may include follow-up dialog, advise, treatment, education, self-help resources, care planning, and referrals.
Life situation impacting diabetes
Psychological well-being
Daily life with diabetes
Worry about complications
Blood sugar regulation and hypo/hyperglycemia worry
Diet, carbohydrate counting, and smoking
Medicine experience
Symptom distress: Sleep, sexual function, foot, neuropathic pain, gastrointestinal, and cardiovascular
Confidence in access to health care professionals for diabetes
Qualitative and PRO data were compared with questionnaire evaluation data to verify the findings and establish robust overall findings regarding feasibility, acceptability, and perceived benefits. Both questionnaire and qualitative data confirmed that the PRO questionnaire and the
Both questionnaire data and interviews for people with diabetes and visit transcripts supported that HCPs used the PRO dashboard in the intended person-centered way and achieved a more focused and relevant dialog. The HCP questionnaire and open-ended text confirmed general satisfaction with the use of PRO tools.
Some people with diabetes had a high number of flagged PRO topics in this version of the tool, and some PRO-flagged topics were not relevant for the HCP to act on in the specific visit; however, people with diabetes were satisfied with the use and did not report concerns or negative experiences related to this. We used a multi-informant mixed-methods approach to analyze negative outliers in the questionnaire data. As an example, we looked at the only person with diabetes who had rated the relevance of the PRO diabetes questionnaire as somewhat not relevant, whereas the 11 others had rated it relevant. The person with diabetes had had diabetes for >13 years, was resourceful and empowered in relation to diabetes, and had seen the same physician for several years. The person with diabetes’s
People with diabetes were able to fill out the evaluation questionnaires digitally without the need for support after each visit and expressed, in interviews, that the questions were relevant and easy to understand and fill out. We obtained complete evaluation data from both people with diabetes and HCP from all visits. This corroborated our initial finding from user-testing that they were suitable for use in routine care situations. As part of the formative design, the pilot evaluation questionnaires were revised based on our mixed-methods analysis and feedback from people with diabetes for use in a real-world study of how people with diabetes experience use of PRO [
The PRO tool helped improve the quality of the dialog and visit by facilitating the identification and prioritization of topics according to the needs of people with diabetes. In line with the extant PRO literature, the use of PRO helped introduce new talk topics, supported dialog about difficult psychosocial issues previously left unattended, and facilitated a more active role for people with diabetes [
A common concern expressed by HCPs is that PRO questionnaires may raise unrealistic expectations for people with diabetes and that resources are not available to address topics raised by PRO [
Our finding that none of the people with diabetes expressed disappointment or unfulfilled expectations may be partly because the questionnaire had been specifically designed to only include items that were perceived as directly relevant for routine care by both people with diabetes and HCPs. Given the importance of expectation setting in the clinical use of PRO, we attempted to give instructions to people with diabetes about PRO carefully to avoid inadvertently raising unrealistic expectations.
Our combined analysis indicated good fidelity related to the HCP’s use of the basic recommendations for person-centered use pertaining to the use of open questions to verify and clarify priorities of people with diabetes and ensure coverage of all key topics highlighted by them. The use of open questions and active listening was applied in all visits, and we found it to be an essential component that may be particularly important in the care of people with cognitive or language difficulties who may have difficulties completing the PRO questionnaire as intended. The PRO intervention did not include requirements for HCPs to use specific methods or tools for agenda-setting, structuring the visit, collaborative goal-setting, action-planning, and shared decision-making beyond the general recommendations for person-centered use of the tool. Therefore, this study provided a first opportunity for HCPs to try out the PRO tool in routine visits, identify key communication challenges, and begin to develop individual strategies on how to use the tool in an optimal way. The main challenges reported by the HCPs relate to how to structure the time and balance focus on clinical and PRO issues. An HCP expressed that it could be difficult to juggle tasks during the visit when people with diabetes had a high number of PRO topics, and an HCP found it difficult to review the many topics on the screen while maintaining a natural conversation. In contrast, and interestingly, the people with diabetes did not express concerns or problems related to the number of topics being identified and were overwhelmingly positive about the way the HCPs used the tool. We believe this may be partly because of the HCPs putting in an extra effort to ensure that people with diabetes had a good dialog experience despite the high number of flagged issues.
In a few cases, it was noted by HCPs that some flagged PRO topics, especially those related to symptoms, were not related to diabetes or were not relevant to act on in this visit; however, it was still necessary to review the topics as they were flagged on the PRO dashboard. The issue of not being able to act on especially generic PRO issues has also been reported by others [
People with diabetes expressed positive experiences related to the completion of the questionnaire. We found several factors that could potentially explain this. The completion of the questionnaire led to a reflective process that facilitated self- and disease-insight related to diabetes. During the design of the PRO tool, people with type 2 diabetes highlighted the potential for this questionnaire to help people with diabetes understand their situation and options for acting for their own health. On the basis of our results, this effect is likely important for a subgroup and dependent on diabetes duration, empowerment, and whether the person with diabetes is completing the PRO questionnaire for the first time. Filling out the questionnaire at home made people with diabetes feel confident that they would get to talk about their priority issues during the visit. This was very important as people with diabetes reported that they do not normally prepare for visits and often forget to ask about issues that are important to them. Furthermore, this appeared to have a potential impact on expectations and motivation related to participating in the upcoming visit.
The broad and balanced coverage of topics in the PRO diabetes questionnaire represents the result of an extensive iterative national multistakeholder participatory design process to achieve a core questionnaire acceptable and useful for both people with diabetes and HCPs in different care settings [
Our study supports that it is feasible to use a PRO diabetes questionnaire with a very broad range of topics and that the questionnaire may generate both specific benefits related to each PRO construct as well as related to its use as an overview to facilitate a person-centered dialog [
Further research is warranted to investigate the importance of a broad topic coverage to facilitate the potential therapeutic and empowering effect of self-completing the PRO. It is possible that PRO instruments that focus only on one or a few topics may not provide the same support for disease insight, care navigation, and active participation as instruments which provide a comprehensive 360°-review of diabetes issues
Another possible benefit related to the broad topic coverage, when compared with, for example, depression-focused diabetes screeners, is that people with diabetes who may be doing well emotionally are still given an opportunity to express other priorities and issues that affect them in relation to diabetes.
HCPs noted that an important benefit of the PRO tool was related to obtaining a structured overview of each person with diabetes’ overall situation, which was made possible by the comprehensive topic coverage. In a previous study, we found that there is a need to use a broad set of PRO outcome constructs to evaluate outcomes in diabetes as the needs of people with diabetes vary individually and change over time [
Our PRO tool includes 5 items specifically related to depression and diabetes-related distress as it is important to legitimize and prompt dialog on these often insufficiently addressed issues [
This study provides initial insights on how to evaluate the potential public health impact of the PRO tool using the dimensions of RE-AIM (reach, efficacy, adoption, implementation, and maintenance) [
Conceptual working model for hypothesized processes and impacts for the use of the patient-reported outcome diabetes tool in a routine diabetes visit. HCP: health care professional; PRO: patient-reported outcome; PWD: people with diabetes.
The first stage of reflection and improved disease- and self-insight and motivation facilitated by the questionnaire completion by people with diabetes were found to be important to the study participants and suggest the potential for the intervention to facilitate diabetes-related empowerment [
Both people with diabetes and HCPs reported that people with diabetes were more actively engaged as a result of PRO. In this pilot study, HCPs used the PRO tool without detailed protocolized steps for agenda-setting, shared decision-making [
The anticipated beneficial impact of the PRO tool for people with diabetes relates to confidence in diabetes management [
HCPs appreciated gaining a structured insight into the lived experience of diabetes among people with diabetes, which facilitated reflection and allowed them to provide more personalized care. The PRO tool facilitated the introduction of daily life and psychosocial issues to achieve a comprehensive biopsychosocial review of the person with diabetes’ situation. As HCPs achieve self-efficacy for the use of PRO with their patients, there is a potential to experience improved work satisfaction and fulfillment. Given the utility of a wide range of topics, we believe the PRO tool has a particularly high potential for improving the individualization of treatments and use of a wider set of support resources in accordance with what can benefit individuals with diabetes the most, thereby potentially promoting better self-management, outcomes, and use of resources.
The
An important additional aim of the PRO diabetes tool is to improve care by monitoring outcomes that matter to people with diabetes for value-based care [
Our results should be examined while considering that this was a formative pilot study of the first viable version of the digital PRO tool,
The small number of people with diabetes and HCPs limits the basis for generalization; even so, purposive sampling provided a good basis for examining experiences from a group of people with diabetes, which was diverse in terms of age, gender, type of diabetes, duration, treatment, and disease progression.
The 4 HCPs were previously involved in the design of the PRO tool, which limits the generalizability of their experiences. Further research is required to evaluate the adoption and implementation by a diverse group of PRO-naïve HCPs in different health care settings.
Despite these limitations, our use of purposive sampling, multi-informant, and mixed-methods data analysis provided an opportunity to show the robustness of our core findings. Further research is required and planned based on the detailed findings in the study to examine the impacts of, facilitators of, and barriers to effective and integrated standard use of our PRO diabetes tool on a larger scale in different health care settings [
This is the first study to show the feasibility, acceptability, and perceived benefits of using the Danish PRO diabetes questionnaire and the
In conclusion, we found that our newly developed PRO diabetes tool, consisting of the national PRO diabetes questionnaire and the digital PRO tool
health care professional
patient-reported outcome
reach, efficacy, adoption, implementation, and maintenance
Value-Based Health Care and Patient-Reported Outcome in Diabetes
Members of the authors’ user panel consisting of people with type 1 and type 2 diabetes provided invaluable input and codeveloped study materials along the way: Dorthe Hinzman, David Rasmussen, Henning Nielsen, Anni Fynbo, and Tove Brix.
The authors would like to thank the Department of Endocrinology and Steno Diabetes Center North Denmark at Aalborg University Hospital for invaluable support for participatory design, user-testing, study recruitment, and data collection; Karleen Parquette for observing visits and providing feedback regarding the health care professional’s handling of psychosocial issues identified by patient-reported outcome; and Lise Boel and medical students Mikkel Wandahl Mathiesen, Sebastian Lawrence, and Frederik Østergaard Klit for providing support for the transcription of interviews, consultations, and coding. The region of North Denmark provided financial support to make this research possible. The authors would also like to thank Zitelab as their partner in the participatory design of the digital
The pilot evaluation questionnaires and study materials were designed by the lead author (SES) using qualitative research and iterative development with input from people with diabetes and health care professionals. They were used to develop the final PRO-EVAL and PRO-CON-EVAL questionnaires which are now used in real-world studies of patient-reported outcome also in other disease areas. The final evaluation questionnaires can be obtained for research use without license costs by contacting the author.
None declared.