This paper is in the following e-collection/theme issue:

Formative Evaluation of Digital Health Interventions (2491) Decision Support for Health Professionals (1210) Clinical Information and Decision Making (1434) Consumer & Patient Education and Shared-Decision Making (695)

Published on in Vol 6, No 10 (2022): October

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/40018, first published .
Shared Decision-Making for Drug-Drug Interactions: Formative Evaluation of an Anticoagulant Drug Interaction

Shared Decision-Making for Drug-Drug Interactions: Formative Evaluation of an Anticoagulant Drug Interaction

Shared Decision-Making for Drug-Drug Interactions: Formative Evaluation of an Anticoagulant Drug Interaction

Authors of this article:

Ainhoa Gomez Lumbreras1 Author Orcid Image ;   Thomas J Reese2 Author Orcid Image ;   Guilherme Del Fiol3 Author Orcid Image ;   Malinda S Tan1 Author Orcid Image ;   Jorie M Butler3 Author Orcid Image ;   Jason T Hurwitz4 Author Orcid Image ;   Mary Brown5 Author Orcid Image ;   Kensaku Kawamoto3 Author Orcid Image ;   Henrik Thiess6 Author Orcid Image ;   Maria Wright1 Author Orcid Image ;   Daniel C Malone1 Author Orcid Image
Article Authors Cited by (1) Tweetations (9) Metrics

    Original Paper

    1Department of Pharmacotherapy, Skaggs College of Pharmacy, University of Utah, Salt Lake City, UT, United States

    2Department of Biomedical Informatics, Vanderbilt University, Nashville, TN, United States

    3Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, United States

    4Center for Health Outcomes and Pharmacoeconomic Research, University of Arizona, Tucson, AZ, United States

    5University of Arizona, Tucson, AZ, United States

    6University of Heidelberg, Heidelberg, Germany

    *these authors contributed equally

    Corresponding Author:

    Daniel C Malone, RPh, PhD

    Department of Pharmacotherapy

    Skaggs College of Pharmacy

    University of Utah

    30 S 2000 East, Room 4922

    Salt Lake City, UT, 84112

    United States

    Phone: 1 801 581 8054

    Email: dan.malone@utah.edu


    Background: Warnings about drug-drug interactions (DDIs) between warfarin and nonsteroidal anti-inflammatory drugs (NSAIDs) within electronic health records indicate potential harm but fail to account for contextual factors and preferences. We developed a tool called DDInteract to enhance and support shared decision-making (SDM) between patients and physicians when both warfarin and NSAIDs are used concurrently. DDInteract was designed to be integrated into electronic health records using interoperability standards.

    Objective: The purpose of this study was to conduct a formative evaluation of a DDInteract that incorporates patient and product contextual factors to estimate the risk of bleeding.

    Methods: A randomized formative evaluation was conducted to compare DDInteract to usual care (UC) using physician-patient dyads. Using case vignettes, physicians and patients on warfarin participated in simulated virtual clinical encounters where they discussed the use of taking ibuprofen and warfarin concurrently and determined an appropriate therapeutic plan based on the patient’s individualized risk. Dyads were randomized to either DDInteract or UC. Participants completed a postsession interview and survey of the SDM process. This included the 9-item Shared Decision-Making Questionnaire (SDM-Q-9), tool usability and workload National Aeronautics and Space Administration (NASA) Task Load Index, Unified Theory of Acceptance and Use of Technology (UTAUT), Perceived Behavioral Control (PBC) scale, System Usability Scale (SUS), and Decision Conflict Scale (DCS). They also were interviewed after the session to obtain perceptions on DDInteract and UC resources for DDIs.

    Results: Twelve dyad encounters were performed using virtual software. Most (n=11, 91.7%) patients were over 50 years of age, and 9 (75%) had been taking warfarin for more than 2 years (75%). Regarding scores on the SDM-Q-9, participants rated DDInteract higher than UC for questions pertaining to helping patients clarify the decision (P=.03), involving patients in the decision (P=.01), displaying treatment options (P<.001), identifying advantages and disadvantages (P=.01), and facilitating patient understanding (P=.01) and discussion of preferences (P=.01). Five of the 8 UTAUT constructs showed differences between the 2 groups, favoring DDInteract (P<.05). Usability ratings from the SUS were significantly higher (P<.05) for physicians using DDInteract compared to those in the UC group but showed no differences from the patient’s perspective. No differences in patient responses were observed between groups using the DCS. During the session debrief, physicians indicated little concern for the additional time or workload entailed by DDInteract use. Both clinicians and patients indicated that the tool was beneficial in simulated encounters to understand and mitigate the risk of harm from this DDI.

    Conclusions: Overall, DDInteract may improve encounters where there is a risk of bleeding due to a potential drug-drug interaction involving anticoagulants. Participants rated DDInteract as logical and useful for enhancing SDM. They reported that they would be willing to use the tool for an interaction involving warfarin and NSAIDs.

    JMIR Form Res 2022;6(10):e40018

    doi:10.2196/40018

    Keywords

    decision making, shared; decision support systems, clinical; decision making; decision support; user-centered design; patient-centered care; risk management; drug interaction; pharmacotherapy; pharmacy; pharmaceutical; warfarin; unified theory of acceptance and use of technology; UTAUT; NSAID; anti-inflammatory; non-steroidal 


    Approximately 8 million people in the United States received anticoagulants, including warfarin, in 2019 [1]. Warfarin is well known for having a multitude of drug-drug interactions (DDIs), with many avoided through health care provider awareness. However, studies found that up to 24% of people taking warfarin also received a prescription for nonsteroidal anti-inflammatory drugs (NSAIDs), and almost 50% of patients on warfarin have some form of ongoing pain [2,3]. Concomitant use of a NSAID and warfarin increases the risk of bleeding [4,5]. The risk of gastrointestinal (GI) bleeding is nearly 2-fold greater with this combination compared to using warfarin alone [5]. Furthermore, because ibuprofen and naproxen sodium are NSAIDs that are available over the counter (OTC), it is critical that patients understand the risks of taking the 2 medications concurrently. While the use of warfarin has declined since the approval of direct oral anticoagulants in 2013, it is the only anticoagulant indicated for mechanical valve replacement and is used by at least half of the patients taking oral anticoagulants [6-8]. Furthermore, patients’ knowledge of the adverse effects of NSAIDs is incomplete [9,10].

    To improve health outcomes, stakeholders have advocated for the incorporation of shared decision-making (SDM) that encourages physicians to involve patients in selecting therapeutic treatments, among others [11]. SDM improves patient behavior and results in higher adherence to physician guidance, leading to improved health outcomes [12,13]. Currently, there are SDM tools available to facilitate decisions regarding nonvalvular atrial fibrillation anticoagulant treatment [14-18]. However, none of these SDM tools focus on or incorporate anticoagulant DDIs and their associated risks [19].

    Current DDI alerting systems within electronic health records (EHRs) are clinician centric, suffer from high override rates, and do not account for patient and drug attributes that affect risk of harm from the interaction [20]. Furthermore, the primary source of DDI information (online DDI checkers) do not provide any personalized quantification of the risk of harm [21-24]. Accordingly, we developed DDInteract, a tool to facilitate SDM for patients receiving warfarin that calculates evidence-based risk of GI bleeding by considering an individual patient’s risk factors such as age, history of previous GI bleeding, and other medications [25]. We previously reported on the user-centered design and usability assessment of DDInteract. The purpose of this study was to conduct a formative evaluation comparing DDInteract to usual care (UC).


    Study Design

    This manuscript follows SUNDAE (Standards for Universal Reporting of Decision Aid Evaluations) guidelines [26]. We conducted a randomized formative evaluation of DDInteract compared to UC from March 2021 to August 2021. Formative evaluation is a research methodology used to rigorously examine factors that might influence the progress during the implementation of health care innovations [27]. We used a mixed methods approach, as we collected both quantitative and qualitative data.

    Ethics Approval

    This project was approved by University of Utah Institutional Review Board (00127062). All participants signed the inform consent before participating in this study.

    Intervention

    We designed DDInteract to be a free SDM tool that is integrated with the EHR to mitigate risk of harm due to DDIs involving warfarin and NSAIDs [25]. When launched from within an EHR, the tool retrieves risk factors from the patient’s record (eg, history of GI bleeding, age, medications) and calculates the patient’s risk of bleeding. A dynamic risk array displays and updates the risk of GI bleeding in real time based on clinician input, patient history, and medication choices (Figure 1). When an NSAID is selected, the risk array changes to reflect the increase in risk due to the combined effects of both agents on risk of bleeding. The tool also (1) provides information on nonmedication treatments for pain, such as acupuncture and physical therapy; (2) includes strategies to reduce risk of GI bleeding through use of proton pump inhibitors; (3) facilitates the medication prescription process; (4) automates clinical documentation of the SDM discussion; and (5) provides patient education in an “after visit summary.” Further details about DDInteract functionality are described elsewhere, and the tool is available for demonstration purposes on the DDInteract website [28].

    ‎
    Figure 1. DDInteract tool (partial screen).
    View this figure

    Study Settings

    We randomized physician-patient dyads to participate in simulated clinical encounters using either DDInteract or conventional drug information resources (UC) to treat a patient on warfarin who is also seeking relief of pain by using an NSAID. Physicians in both groups (DDInteract or UC) received access to various online DDI resources (eg, Micromedex), while those in the intervention group also had access to DDInteract.

    Recruitment: Inclusion and Exclusion Criteria

    We recruited physicians through electronic communications, presentations to medical groups, and snowball sampling. The inclusion criteria for physicians required experience with treating patients on oral anticoagulants. We recruited patients from the University of Utah Health System, including the Division of Cardiology and the Anticoagulation Service. Patient participants needed to be older than 21 years of age, fluent in English, and currently taking an oral anticoagulant.

    Encounters

    We integrated and deployed DDInteract with the Logica sandbox EHR environment using the Substitutable Medical Applications and Reusable Technologies (SMART) guidelines on Fast Health Interoperability Resources (FHIR) standards [29]. Details on the technical architecture including interoperability approach are available elsewhere [30]. Physician-patient dyads participated in a simulated virtual encounter using an online meeting software and provided consent prior to starting the simulated encounter. We instructed both physicians and patients to conduct themselves as they would in an encounter where an anticoagulated patient wants to use ibuprofen to treat their pain but has concerns about bleeding. Participants permitted us to record the encounters. Physicians and patients received instructions separately via breakout rooms and asked study investigators any questions prior to the simulated visit.

    Physicians randomized to the DDInteract group received a link to a video demonstrating the functionality of DDInteract, while those randomized to the UC group received information on the main topic and of online drug information sources that can screen for drug interactions (see Table S1 in the Multimedia Appendix 1). Physicians in both groups received access to online resources for DDIs to use during the encounter at their discretion. After the simulated encounters, both physicians and patients participated in a semistructured interview with a member of the research team to discuss the encounter and use of DDInteract and online drug information resources. Upon concluding the simulated encounter, participants completed an online survey containing the instruments described in the sequent section.

    Outcomes

    The primary outcome was quality of SDM according to the 9-item Shared Decision-Making Questionnaire (SDM-Q-9) questionnaire assessed by physicians. The physician postsimulation survey included items in 6 groupings: (1) SDM-Q-9 scale [31]; (2) adapted System Usability Scale (SUS) [32]; (3) National Aeronautics and Space Administration (NASA) Task Load Index (NASA-TLX) [33]; (4) Unified Theory of Acceptance and Use of Technology (UTAUT) [34], including the performance expectancy, effort expectancy, attitude toward using technology, social influence, facilitating conditions, self-efficacy, anxiety, and behavior constructs; (5) Perceived Behavioral Control (PBC) scale [35]; and (6) questions pertaining to participants’ demographics and professional experience. The patients’ survey included 6 items adapted from the SUS scale, 11 items adapted from the Decisional Conflict Scale (DCS) [36], and questions about their demographics and anticoagulation treatment.

    Statistical Analysis

    We calculated the mean and standard deviation for each item/construct from the various scales and instruments. For SDM quality, which was the primary outcome, we conducted a 2-group Student t-test assessing differences for DDInteract versus UC as rated by physicians. A similar approach was used to test for differences in secondary measures including the SUS, NASA-TLX, and UTAUT. The PBC scale rated by physicians was also analyzed using the same approach, as well as items on the DCS. The total time of the simulated encounter for both groups was compared using Student t-test. All statistical comparisons used a 2-tailed test with an alpha <.05. Given the relatively small sample size, no adjustments were made for multiple comparisons. Since we could not assume that the populations were normally distributed, we estimated the Mann-Whitney U test for all the outcomes results reported. We also reviewed transcriptions of the post encounter semistructured interviews and highlighted the most insightful comments. All transcriptions were reviewed, and the principal investigators extracted the most relevant and repeated comments. Data were analyzed using Microsoft Excel for Mac 2016 (IBM Corp) and Stata (version 17, basic edition).


    DDInteract Tool and UC Survey Results

    A total of 12 physician-patient dyads completed the formative evaluation. We randomized 6 dyads to DDInteract and 6 to UC. Participant demographics appear in Table 1. The UC group had 1 more female physician and 2 more female patients than the DDInteract group. All patients and 9 (75%) of the physicians reported being Caucasian. There was no difference in physicians’ self-reported experience managing patients with warfarin between the 2 groups (DDInteract 68.8, SD 31.5 vs UC 77.7, SD 20.6, from a 0-100 scale, P=.58).

    The primary outcome of interest (SDM), as measured by the SDM-Q-9, appears in Table 2. Perceptions of SDM differed significantly between DDInteract and UC groups for 6 of the 9 attributes (P<.05) and include: facilitated the discussion on preferences, assisted patient comprehension, presented different options, presented advantages and disadvantages of the different treatment options, involved the patient, and provided clarity on the decision to be made. Not surprisingly, the largest difference between the groups was with respect to presentation of different treatment options (mean 5.7, SD 0.5 vs mean 1.8, SD 1.2, P<.001) and presentation of advantages and disadvantages (mean 5.0, SD 1.1 vs mean 2.8, SD 1.2, P=.008) for DDI and UC, respectively.

    The task load assessed by the NASA-TLX index, scaled from zero (low/easy/successful) to 10 (high/demanding/unsuccessful), was lower for the DDInteract group than the UC group; it was not statistically significant but trended toward significance (mean 2.0, SD 1.2 vs mean 4.2, SD 2.1, P=.08, respectively) (Table 3). Participants indicated that DDInteract took significantly less effort to use than the tools in UC (mean 0.8, SD 0.7 vs mean 7.3, SD 3.7, P=.008, respectively), but none of the other constructs differed significantly between the study groups.

    Patient-reported scores on the adapted SUS scale did not differ between groups, but physician ratings differed significantly, with DDInteract perceived as more logical, efficient, and helpful/effective than UC and SDM perceived as more valuable than using traditional DDI tools in UC (P<.05) (Table 4).

    Physicians’ responses on a Likert scale ranging from 1 (strongly disagree) to 7 (strongly agree) in the DDInteract group differed from those in the UC group on the following UTAUT constructs: performance expectancy (mean 5.6, SD 1.2 vs mean 3.4, SD 1.7, P<.001), effort expectancy (mean 6.8, SD 0.5 vs mean 5.2, SD 1.6, P<.001), attitude toward using technology (mean 5.7, SD 1.1 vs mean 3.9, SD 1.8, P<.001), social influence (mean 5.5, SD 0.8 vs mean 4.1, SD 1.8, P<.001), and anxiety (mean 1.4, SD 0.5 vs mean 2.5, SD 1.5, P<.001), respectively (Table 5). In general, it appears that physicians perceived DDInteract to perform better, require less effort, improve attitudes toward technology, be supported by administration and colleagues, and reduce perceived anxiety more compared to UC.

    Table 6 shows the physicians’ assessment of PBC. Physicians exposed to DDInteract were more likely to indicate using SDM in the clinic compared to physicians randomized to UC (mean 6.8, SD 0.4 vs mean 6.0, SD 0.6, P=.02, respectively). DDInteract physicians were also more likely to perceive conducting SDM without extending the duration of the visit than UC physicians (mean 5.5, SD 0.5 vs mean 3.7, SD 1.5, P=.03). However, the mean duration of the encounters increased 5 minutes on average for the DDInteract, but this increase did not significantly differ between the DDInteract and UC groups (mean 17.6, SD 5.4 minutes vs mean 12.7, SD 4.8 minutes, P=.13). No other PBC items differed significantly between the 2 groups. There were no differences between the groups in patients’ ratings on the DCS (Table S2 in Multimedia Appendix 1).

    The results of the nonparametric tests for the different outcomes (SDM-Q-9, UTAUT, SUS, NASA-TLX, PBC, and DCS) did not differ in terms of statistical significance from the ones presented above (Tables S3-S9 in Multimedia Appendix 1). Qualitative narrative data were collected to gain insight into participants thoughts on SDM and DDI.

    Table 1. Participants’ sociodemographic, professional, and treatment characteristics.
    CharacteristicsPhysiciansPatients

    		DDInteract (N=6), n (%)UCa (N=6), n (%)P valueDDInteract (N=6), n (%)UC (N=6), n (%)P value
    Age.34
    		.39

    		21-291 (16.7)—
    		——

    		30-392 (33.3)1 (16.7)——

    		40-493 (50)2 (33.3)1 (16.7)—

    		50-59—1(16.7)3 (50)2 (33.3)

    		≥60—2 (33.3)2 (33.3)4 (66.7)
    Female sex2 (33.3)3 (50).563 (50)5 (83.3).22
    Race.37
    		>.99

    		American Indian/ Alaska Native——
    		——

    		Asian/Pacific Islander—1 (16.7)——

    		African American————

    		Hispanic1 (16.7)———

    		White/ Caucasian4 (66.7)5 (83.3)6 (100)6 (100)

    		Multiple ethnicity/other1 (16.7)———
    Clinical experience.64
    		—

    		<5 years1 (16.7)1 (16.7)
    		——

    		6-10 years2 (33.3)1 (16.7)——

    		11-15 years2 (33.3)1 (16.7)——

    		≥16 years1 (16.7)3 (50)——
    Level of education—
    		.99

    		College graduate——
    		2 (33.3)3 (50)

    		Attended college——1 (16.7)1 (16.7)

    		High school graduate——1 (16.7)1 (16.7)

    		Completed graduate school——2 (33.3)1 (16.7)
    Time on anticoagulant—
    		.50

    		≤6 months——
    		2 (33.3)1 (16.7)

    		>6 months ≤2 years——00

    		>2 years——4 (66.7)5
    Time in clinical care.08
    		—

    		≤20% week—2 (33.3)
    		——

    		21%-40% week1 (16.7)1 (16.7)——

    		41%-60% week—2 (33.3)——

    		61%-80% week————

    		≥81% week5 (83.3)1 (16.7)——
    Specialty.56
    		—

    		Internal medicine3 (50)3 (50)
    		——

    		Family medicine1 (16.7)2 (33.3)——

    		Geriatric medicine1 (16.7)
    		——

    		Other1 (16.7)1 (16.7)——

    aUC: usual care.

    Table 2. The 9-item Shared Decision-Making Questionnaire (SDM-Q-9).

    		Physicians, mean (SD)P value

    		DDInteractUCa
    Clarity of the SDMb4.5 (0.8)3.3 (0.8).03
    Involve the patient5.5 (0.5)3.2 (1.3).01
    Different option presented5.7 (0.5)1.8 (1.2)<.001
    Presented advantages/disadvantages5.0 (1.1)2.8 (1.2).01
    Assisted patient comprehension4.8 (0.4)2.7 (1.4).01
    Facilitated discussion on preferences5.2 (0.8)3.0 (1.3).01
    Evaluated options5.0 (0.0)4.7 (0.8).36
    Co-selected a treatment option5.2 (0.4)5.0 (0.9).69
    Reach an agreement5.7 (0.5)5.3 (0.8).42

    aUC: usual care.

    bSDM: shared decision-making.

    Table 3. Adapted National Aeronautics and Space Administration Task Load Index (NASA-TLX)a.
    NASAb Task Load Index constructQuestionsPhysicians, mean (SD)P value


    		DDInteractUCc
    Mental demandHow much mental effort was required to decide on the patient’s treatment?3.8 (1.0)4.7 (3.2).57
    Physical effortWas using the DDInteract decision tool easy or demanding?0.8 (0.7)7.3 (3.7).008
    Temporal demandHow much time did it take to investigate the drug interaction during the simulated visit?1.8 (1.6)3.6 (3.5).28
    EffortHow hard did you have to work to make a decision with the patient?2.3 (1.0)3.6 (2.6).30
    PerformanceHow successful do you think you were in making a shared decision with the DDInteract decision tool?1 (0.9)1.7 (1.8).45
    Total average
    		2.0 (1.2)4.2 (2.1).08

    aLikert scale from 0 (low/easy/successful) to 10 (high, demanding, unsuccessful).

    bNASA: National Aeronautics and Space Administration.

    cUC: usual care.

    Table 4. Modified System Usability Scale (SUS).
    ItemsPatients, mean (SD)P valuePhysicians, mean (SD)P value

    		DDInteractUCa
    		DDInteractUC
    It was logical4.7 (0.5)4.8 (0.4).654.8 (0.4)3.7 (1).03
    It was efficient4.5 (0.5)4.4 (0.5).774.7 (0.5)3.5 (1).04
    Helpful/effective in the decision-making process4.7 (0.5)4.4 (0.9).574.5 (0.5)2.8 (1.2).02
    The SDMb using the tool was valuable4.5 (0.5)4.8 (0.4).264.7 (0.5)2.8 (0.7)<.001
    The tool was valuableN/AcN/AN/A4.3 (0.5)2.8 (1.2).02
    Easy to use4.5 (0.5)4.4 (0.9).834.8 (0.4)3.2 (1.5).04
    Enjoyed the experience4.7 (0.5)4.8 (0.4).55N/AN/AN/A
    Learned something from this experience4.8 (0.4)4. 7 (0.8).674.5 (0.5)2.8 (1.7).06

    aUC: usual care.

    bSDM: shared decision-making.

    cN/A: not applicable.

    Table 5. Unified Theory of Acceptance and Use of Technology (UTAUT) responses.

    		Physicians, mean (SD)P value

    		DDInteractUCa
    Performance expectancy5.6 (1.2)3.4 (1.8)<.001
    Effort expectancy6.8 (0.5)5.2 (1.6)<.001
    Attitude toward using technology5.7 (1.1)3.9 (1.8)<.001
    Social influence5.5 (0.9)4.1 (1.8)<.001
    Self-efficacy5.3 (2.1)4.6 (2.2).26
    Facilitating conditions5.1 (1.9)5.4 (1.7).58
    Anxiety1.4 (0.5)2.5 (1.5)<.001
    Behavioral5.5 (1.0)4.7 (2.9).53

    aUC: usual care.

    Table 6. Perceived Behavioral Control (PBC) scale.
    ItemPhysicians, mean (SD)P value

    		DDInteractUCa
    I am convinced that I can share decision‐making in the clinic6.8 (0.4)6 (0.6).02
    I have control over the level of SDMb that is accomplished in the clinic6.2 (0.8)5.7 (0.5).21
    I can perform SDM without extending the duration of the consultation5.5 (0.5)3.7 (1.5).03
    Knowledge about SDM is important in order to apply SDM6.5 (0.5)5.7 (1.4).20
    Communication skills are important for SDM7 (0)6.7 (0.5).17
    Patients are motivated to participate in SDM6 (0.9)5.5 (1.2).44
    In general patients have enough knowledge, intelligence and understanding needed for SDM6 (0.9)5 (1.7).23

    aUC: usual care.

    bSDM: shared decision-making.

    Physician and Patient Perceptions Regarding the DDInteract Tool and Online DDI Resources

    After the virtual encounters, both clinicians and patients discussed the simulation using a semistructured interview format to elicit their impressions of DDInteract and online DDI resources. Below are select quotes from the participants. In general, DDInteract was well received by the physicians, as indicated by the following statements:

    I like it; it helps patients visualize the risk instead of me just talking statistics, they see the risk in the more obvious way. I like the intervention that shows the risk of bleeding, you can increase more risk factors for bleeding, the idea is excellent.
    [Physician #4, male]
    I really like it. Whenever I can, I like to show something visual while taking to the patient. It is very user friendly, simple. It is not overly complicated.
    [Physician #8, female]

    Concerns about extra time involved in using DDInteract were not evident among the participants. One physician stated:

    I am trying to imagine having this conversation without the tool. I don't even think the tool would even make it longer; I think what it does is cut down on having to overly explain things, cut down on the feeling that I have to reemphasize things because it was a visual tool, and the patient is seeing what I am seeing, so they don't ask for repeating. So I probably save some questions too. In all I think it would save some time.
    [Physician #8, female]

    Another physician felt that the tool would encourage more conversations with the patient and help them understand the risks associated with bleeding and anticoagulation treatment:

    I think it was fast to use. I see thrombosis patients, so I do a lot of counseling about warfarin or anticoagulation. I think I tend to assume that people have already been through education, but this would be nice because it will slow me down and help me actually understand what the risk is, which I might just summarize very quickly, but I think this would be really helpful from a patient standpoint to actually, like, seeing is something that explains it a little better.
    [Physician #10, female]

    When patients were asked about the icon array in DDInteract as an approach to display the risk of bleeding, some comments included:

    I think seeing the graphic portrayal of the different risk levels and how to treat the pain was very helpful. I think it was well done…I thought this tool was much more thorough than the information that I've gotten in the normal clinic visit. I did not know the number of bleeds per 100 patients; that has never been discussed with me during my doctor visit.
    [Patient #10, female]
    The icon array definitely makes sense to me.
    [Patient #12, male]
    I learn better by what I see.
    [Patient #9, female]
    I think the chart with different numbers showing different reactions you get by taking ibuprofen kind of spells it out like black and white for me.
    [Patient #8, male]

    Table S10 in Multimedia Appendix 1 provides additional comments made by the physicians and patients.


    Principal Results

    Results of the formative evaluation suggest that there was a positive opinion of DDInteract as a potentially useful tool to help facilitate SDM concerning concomitant therapies that could result in a DDI. Physicians perceived the tool as intuitive, easy to use, and not increasing the amount of time during an encounter. After the simulated encounter, patients exposed to the DDInteract tool commented that they liked the way the information was presented, including the quantitative estimation of the risk and how risk changed according to patient factors in the user interface.

    Physicians that used the DDInteract reported enhanced SDM compared to physicians randomized to UC using a standardized and validated tool (SDM-Q-9) [31,37]. This study demonstrates that DDInteract assisted in facilitating discussion, increasing patient comprehension, providing different treatment options with their respective advantages and disadvantages, and providing clarity around the treatment choices and the final decision. This contrasts with current warnings about potential interactions that are clinician orientated and not designed to be shared with patients.

    Due to the ubiquitous use of NSAID pain relievers, including both prescription and over-the-counter products, patients play an important role in preventing harm from DDIs. Through education, patients can reduce harm by avoiding interacting OTC medications and monitoring signs and symptoms of adverse consequences [38]. One of the most important aspects of DDInteract is the risk array, which dynamically changes when risk factors or medications are selected. Usability studies have demonstrated that this approach is appealing and helpful for displaying the risk of harm [25]. Furthermore, physicians using DDInteract during the encounter received no additional training in SDM but rated SDM aspects higher than physicians randomized to UC.

    Secondary Results

    Results from the SUS index suggest that physicians rated DDInteract as more logical, efficient, valuable, easy to use, and helpful/effective than physicians in the UC who rated online compendia and traditional EHR tools. This finding is not surprising given the lack of patient-specific information provided by both EHR systems and online resources for DDIs. Only 1 attribute of the SUS (“learned something”) did not differ significantly between groups (P=.06). Patient perceptions of DDInteract and UC tools did not differ on the SUS scale, a finding that might be attributable to patients wanting more information about DDIs [39,40].

    DDInteract appeared to be intuitive and easy to use according to participant ratings on the NASA-TLX and UTAUT instruments. These results support those of a previously reported usability study that was conducted when designing the tool [25]. The NASA-TLX has been previously used to evaluate the workload associated with the new Clinical Decision Support Tool similar to DDInteract [41]. Compared to UC, physicians rated DDInteract higher on the UTAUT domains of performance expectancy, effort expectancy, attitude toward the technology, and social influence. Furthermore, DDInteract was associated with lower anxiety than UC resources for DDIs. These findings align with a metaregression of studies that evaluated effective clinical decision support using the UTAUT model and suggested that effort expectancy, facilitating conditions, and performance expectancy had a significant impact on clinician behavior [42]. Numerous studies evaluated medication-related CDS using the UTAUT, including studies examining a comprehensive CDS suite of tools called Sentri7/Quantifi [43], use of a decision aid for psychotic disorders [44], medication management in oncology settings [45], and a tool to help remind HIV positive men to use antiviral therapies when undertaking sexual activities [46]. However, to our knowledge, none have applied the UTAUT model to DDI-specific SDM.

    Patient responses to the DCS did not differ between groups, which is unsurprising. Studies using the DCS in the context of nonvalvular atrial fibrillation (NVAF) found mixed results with patient decision aids [47,48]. NVAF decision aids focused on anticoagulant treatments show lower DCS scores or a decrease in the DCS score after using the decision aid, but these differences were not significantly different when compared with groups not using such aids [18,47,48]. In addition, the small sample size may contribute to the lack of significant differences between the groups with respect to the DCS in our study.

    The time required to use CDS and SDM applications is a common concern among clinicians. Our study found that physicians did not perceive DDInteract to affect the amount of time or effort spent with patients (PBC scale question on performing SDM without extending the time of consultation favored DDInteract, P=.03), though there was a nonsignificant 5-minute difference in the time to conduct the simulated encounter, with DDInteract encounters being longer. During the debrief, 1 physician proposed that DDInteract could actually save time because it visually explains the risk of harm and various treatment options, as compared to having to explain the risks verbally. This disconnect between observed time to use the DDInteract and perceived time to use the tool is probably due to clinicians, who had no previous experience with the tool, having to learn how to use DDInteract during the simulation. Clinicians found the tool easy to use and intuitive, indicating a reduced time to use the tool in future encounters.

    DDInteract is a novel approach to providing information about potential interactions. Currently, online drug compendia and warnings within EHR systems have numerous issues [49]. They do not account for individual patient risk factors or provide quantitative estimates of the risk. Additionally, they lack the ability to explore changes in the risk with therapeutic alternatives and are almost completely text based without any visual display of information. Furthermore, they may contain incorrect information. For example, Drugs.com currently states that the mechanism to control bleeding risk in patients on warfarin and ibuprofen is to closely monitor the international normalized ratio (INR); however, concurrent use of NSAIDs and warfarin interaction is a pharmacodynamic interaction that will not affect the INR [24]. Studies have found patients’ INRs to be within range among individuals experiencing bleeding and taking both medications, and a systematic review did not find an effect on INR for ibuprofen or naproxen [50,51]. A review of 7 Chinese apps that support DDI checking and target health care professionals had tremendous variation in information about DDIs, with over 50% lacking accurate information on DDIs [52]. Although DDInteract is currently limited in scope in terms of the interactions it includes, the approach is generalizable to other interactions, and efforts are underway to expand these interactions.

    Limitations

    This study has several limitations that should be considered when interpreting the results. We recruited fewer participants than our a priori sample size (16 dyads), but we still observed significant differences between the study groups in terms of the primary outcome of SDM quality and several secondary outcomes of interest. COVID-19 protocols and restrictions limited our recruitment of subjects because clinic visits have been in part shifted to telehealth, making it more difficult to contact patients, and because physicians were overburdened with demands due to the pandemic [53,54]. In addition, this study focused on a specific DDI involving warfarin, which is being prescribed less because of alternative anticoagulants that require less frequent monitoring. Several physicians commented that they would have liked to see DDInteract incorporate other anticoagulants and other bleeding risks beyond GI. As stated previously, efforts are underway to broaden the interactions incorporated into the tool. Another limitation is that the clinical scenario required dyads to role-play a scenario that may not occur in clinical practice, where a patient is seeking advice from the physician about treating pain while on an anticoagulant. While this scenario may be infrequent in most practice settings, previous studies have found this to occur in up to 25% of patients receiving warfarin [2,3]. Furthermore, many physicians in the study indicated they were aware of the interaction when researchers explained the scenario prior to the simulated visit. Thus, many practitioners may have behaved differently when provided with DDInteract or other DDI information. Nonetheless, after the simulated visit, many practitioners exposed to DDInteract were enthusiastic about the tool. Therefore, we believe DDInteract may be useful in many practice settings, especially as the tool is expanded to support SDM for other drug interactions, such as other oral anticoagulants.

    Conclusions

    DDInteract is a novel tool designed to facilitate SDM during encounters concerning potentially harmful DDIs. The tool improved SDM, was not overly burdensome according to the NASA-TLX and UTAUT scales, and was generally well received by both physicians and patients. However, further research is needed to evaluate the impact of DDInteract on exposure to harmful DDIs and other important health outcomes.

    Acknowledgments

    This project has been supported by the Agency for Health Care Research and Quality (U18HS027099). The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Health Care Research and Quality. Additionally, we would like to thank project coordinator Maria Wright for her administrative support in all aspects of this study.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Supplementary Tables S1-S10.

    DOCX File , 29 KB
    1. Bleeding risk awareness survey. National Blood Clot Alliance. Philadelphia, PA; 2020 Jun 23.   URL: https://www.stoptheclot.org/news/bleeding-risk-survey/ [accessed 2021-12-14]
    2. Malone D, Hutchins D, Haupert H, Hansten P, Duncan B, Van Bergen RC, et al. Assessment of potential drug-drug interactions with a prescription claims database. Am J Health Syst Pharm 2005 Oct 01;62(19):1983-1991. [CrossRef] [Medline]
    3. Gabilondo M, Loza J, Pereda A, Caballero O, Zamora N, Gorostiza A, et al. Quality of life in patients with nonvalvular atrial fibrillation treated with oral anticoagulants. Hematology 2021 Dec 25;26(1):277-283. [CrossRef] [Medline]
    4. Delaney JA, Opatrny L, Brophy JM, Suissa S. Drug drug interactions between antithrombotic medications and the risk of gastrointestinal bleeding. CMAJ 2007 Aug 14;177(4):347-351 [FREE Full text] [CrossRef] [Medline]
    5. Villa Zapata L, Hansten PD, Panic J, Horn JR, Boyce RD, Gephart S, et al. Risk of bleeding with exposure to warfarin and nonsteroidal anti-inflammatory drugs: a systematic review and meta-analysis. Thromb Haemost 2020 Jul;120(7):1066-1074 [FREE Full text] [CrossRef] [Medline]
    6. Maggioni AP, Dondi L, Andreotti F, Pedrini A, Calabria S, Ronconi G, et al. Four-year trends in oral anticoagulant use and declining rates of ischemic stroke among 194,030 atrial fibrillation patients drawn from a sample of 12 million people. Am Heart J 2020 Feb;220:12-19. [CrossRef] [Medline]
    7. Shah RU, Rupp AB, Mowery D, Zhang M, Stoddard G, Deshmukh V, et al. Changes in oral anticoagulant treatment rates in atrial fibrillation before and after the introduction of direct oral anticoagulants. Neuroepidemiology 2016 Jan 31;47(3-4):201-209. [CrossRef] [Medline]
    8. Kohsaka S, Katada J, Saito K, Jenkins A, Li B, Mardekian J, et al. Safety and effectiveness of non-vitamin K oral anticoagulants versus warfarin in real-world patients with non-valvular atrial fibrillation: a retrospective analysis of contemporary Japanese administrative claims data. Open Heart 2020 Apr 01;7(1):e001232 [FREE Full text] [CrossRef] [Medline]
    9. Mullan J, Weston KM, Bonney A, Burns P, Mullan J, Rudd R. Consumer knowledge about over-the-counter NSAIDs: they don't know what they don't know. Aust N Z J Public Health 2017 Apr;41(2):210-214. [CrossRef] [Medline]
    10. Cryer B, Barnett MA, Wagner J, Wilcox CM. Overuse and misperceptions of nonsteroidal anti-inflammatory drugs in the United States. Am J Med Sci 2016 Nov;352(5):472-480. [CrossRef] [Medline]
    11. Elwyn G, Frosch D, Thomson R, Joseph-Williams N, Lloyd A, Kinnersley P, et al. Shared decision making: a model for clinical practice. J Gen Intern Med 2012 Oct;27(10):1361-1367 [FREE Full text] [CrossRef] [Medline]
    12. Shay LA, Lafata JE. Where is the evidence? A systematic review of shared decision making and patient outcomes. Med Decis Making 2014 Oct 28;35(1):114-131. [CrossRef]
    13. Fiorillo A, Barlati S, Bellomo A, Corrivetti G, Nicolò G, Sampogna G, et al. The role of shared decision-making in improving adherence to pharmacological treatments in patients with schizophrenia: a clinical review. Ann Gen Psychiatry 2020 Aug 05;19(1):43 [FREE Full text] [CrossRef] [Medline]
    14. Loewen P. SPARC - stroke prevention in atrial fibrillation risk tool. Vancouver, BC; 2019 Jun 01.   URL: https://www.sparctool.com/ [accessed 2021-11-16]
    15. O'Neill ES, Grande SW, Sherman A, Elwyn G, Coylewright M. Availability of patient decision aids for stroke prevention in atrial fibrillation: A systematic review. Am Heart J 2017 Sep;191:1-11. [CrossRef] [Medline]
    16. Zeballos-Palacios CL, Hargraves IG, Noseworthy PA, Branda ME, Kunneman M, Burnett B, Shared Decision Making for Atrial Fibrillation (SDM4AFib) Trial Investigators. Developing a conversation aid to support shared decision making: reflections on designing anticoagulation choice. Mayo Clin Proc 2019 Apr;94(4):686-696 [FREE Full text] [CrossRef] [Medline]
    17. Anticoagulation Toolkit. 2019 Dec 03.   URL: http://www.anticoagulationtoolkit.org/ [accessed 2021-11-20]
    18. Hong C, Kim S, Curnew G, Schulman S, Pullenayegum E, Holbrook A. Validation of a patient decision aid for choosing between dabigatran and warfarin for atrial fibrillation. J Popul Ther Clin Pharmacol 2013;20(3):e229-e237. [Medline]
    19. Noseworthy P, Brito J, Kunneman M, Hargraves I, Zeballos-Palacios C, Montori VM, et al. Shared decision-making in atrial fibrillation: navigating complex issues in partnership with the patient. J Interv Card Electrophysiol 2019 Nov;56(2):159-163 [FREE Full text] [CrossRef] [Medline]
    20. Chou E, Boyce R, Balkan B, Subbian V, Romero A, Hansten P, et al. Designing and evaluating contextualized drug-drug interaction algorithms. JAMIA Open 2021 Jan;4(1):ooab023 [FREE Full text] [CrossRef] [Medline]
    21. Medscape Drug Interaction Checker.   URL: https://reference.medscape.com/drug-interactionchecker [accessed 2020-12-08]
    22. Lexicomp drug interactions. Wolters Kluwer.   URL: https:/​/www.​wolterskluwer.com/​en/​solutions/​lexicomp/​resources/​lexicomp-user-academy/​drug-interactions-analysis [accessed 2020-12-08]
    23. RxList Drug Interaction Checker.   URL: https://www.rxlist.com/drug-interaction-checker.htm [accessed 2020-12-09]
    24. Drug interactions checker. Drugs.com.   URL: https://www.drugs.com/drug_interactions.html [accessed 2020-12-08]
    25. Reese TJ, Del Fiol G, Morgan K, Hurwitz JT, Kawamoto K, Gomez-Lumbreras A, et al. A Shared Decision-making Tool for Drug Interactions Between Warfarin and Nonsteroidal Anti-inflammatory Drugs: Design and Usability Study. JMIR Hum Factors 2021 Oct 26;8(4):e28618 [FREE Full text] [CrossRef] [Medline]
    26. Sepucha KR, Abhyankar P, Hoffman AS, Bekker HL, LeBlanc A, Levin CA, et al. Standards for universal reporting of patient decision aid evaluation studies: the development of SUNDAE checklist. BMJ Qual Saf 2018 May;27(5):380-388 [FREE Full text] [CrossRef] [Medline]
    27. Stetler CB, Legro MW, Wallace CM, Bowman C, Guihan M, Hagedorn H, et al. The role of formative evaluation in implementation research and the QUERI experience. J Gen Intern Med 2006 Feb;21(S2):S1-S8. [CrossRef] [Medline]
    28. Drug-Drug Interaction Clinical Decision Support.   URL: https://ddi-cds.org/apps/ [accessed 2020-07-07]
    29. Mandel JC, Kreda DA, Mandl KD, Kohane IS, Ramoni RB. SMART on FHIR: a standards-based, interoperable apps platform for electronic health records. J Am Med Inform Assoc 2016 Feb 17 [FREE Full text] [CrossRef] [Medline]
    30. Thiess H, Del Fiol G, Malone DC, Cornia R, Sibilla M, Rhodes B, et al. Coordinated use of Health Level 7 standards to support clinical decision support: Case study with shared decision making and drug-drug interactions. Int J Med Inform 2022 Mar 21;162:104749. [CrossRef] [Medline]
    31. Scholl I, Kriston L, Dirmaier J, Buchholz A, Härter M. Development and psychometric properties of the Shared Decision Making Questionnaire--physician version (SDM-Q-Doc). Patient Educ Couns 2012 Aug;88(2):284-290. [CrossRef] [Medline]
    32. Brooke J. SUS: A quickdirty usability scale. In: Usability Evaluation in Industry. Oxfordshire, UK: CRC Press; 1996:189-194.
    33. Hart SG, Staveland LE. Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. In: Hancock PA, Meshkati N, editors. Human Mental Workload. Amsterdam, Netherlands: North-Holland; 1988:139-183.
    34. Venkatesh V, Morris MG, Davis GB, Davis FD. User acceptance of information technology: toward a unified view. MIS Quarterly 2003;27(3):425. [CrossRef]
    35. Ajzen I. The theory of planned behavior. Organ Behav Hum Decis Process 1991 Dec;50(2):179-211. [CrossRef]
    36. O'Connor AM. Validation of a decisional conflict scale. Med Decis Making 1995;15(1):25-30. [CrossRef] [Medline]
    37. Doherr H, Christalle E, Kriston L, Härter M, Scholl I. Use of the 9-item Shared Decision Making Questionnaire (SDM-Q-9 and SDM-Q-Doc) in intervention studies-A systematic review. PLoS One 2017 Mar 30;12(3):e0173904 [FREE Full text] [CrossRef] [Medline]
    38. Mutebi A, Warholak TL, Hines LE, Plummer R, Malone DC. Assessing patients' information needs regarding drug-drug interactions. J Am Pharm Assoc 2013;53(1):39-45. [CrossRef] [Medline]
    39. Justad H, Askfors Y, Shemeikka T, Andersson ML, Hammar T. Patients' use and perceptions of a drug-drug interaction database: a survey of Janusmed interactions. Pharmacy (Basel) 2021 Jan 19;9(1):23 [FREE Full text] [CrossRef] [Medline]
    40. Kusch MK, Haefeli WE, Seidling HM. How to meet patients' individual needs for drug information - a scoping review. Patient Prefer Adherence 2018;12:2339-2355 [FREE Full text] [CrossRef] [Medline]
    41. Richardson KM, Fouquet SD, Kerns E, McCulloh RJ. Impact of mobile device-based clinical decision support tool on guideline adherence and mental workload. Acad Pediatr 2019 Sep;19(7):828-834 [FREE Full text] [CrossRef] [Medline]
    42. Liu S, Reese T, Kawamoto K, Del Fiol G, Weir C. A theory-based meta-regression of factors influencing clinical decision support adoption and implementation. J Am Med Inform Assoc 2021 Oct 12;28(11):2514-2522 [FREE Full text] [CrossRef] [Medline]
    43. English D, Ankem K, English K. Acceptance of clinical decision support surveillance technology in the clinical pharmacy. Inform Health Soc Care 2017 Mar 18;42(2):135-152. [CrossRef] [Medline]
    44. Roebroek LO, Bruins J, Delespaul P, Boonstra A, Castelein S. Qualitative analysis of clinicians' perspectives on the use of a computerized decision aid in the treatment of psychotic disorders. BMC Med Inform Decis Mak 2020 Sep 17;20(1):234 [FREE Full text] [CrossRef] [Medline]
    45. Dabliz R, Poon SK, Ritchie A, Burke R, Penm J. Usability evaluation of an integrated electronic medication management system implemented in an oncology setting using the unified theory of the acceptance and use of technology. BMC Med Inform Decis Mak 2021 Jan 06;21(1):4 [FREE Full text] [CrossRef] [Medline]
    46. Ma Y, Zhong X, Lin B, He W. Factors influencing the intention of MSM to use the PrEP intelligent reminder system. RMHP 2021 Nov;Volume 14:4739-4748. [CrossRef]
    47. Loewen PS, Bansback N, Hicklin J, Andrade JG, Kapanen AI, Kwan L, et al. Evaluating the effect of a patient decision aid for atrial fibrillation stroke prevention therapy. Ann Pharmacother 2019 Jul;53(7):665-674. [CrossRef] [Medline]
    48. Kunneman M, Branda ME, Hargraves IG, Sivly AL, Lee AT, Gorr H, Shared Decision Making for Atrial Fibrillation (SDM4AFib) Trial Investigators. Assessment of shared decision-making for stroke prevention in patients with atrial fibrillation: a randomized clinical trial. JAMA Intern Med 2020 Sep 01;180(9):1215-1224 [FREE Full text] [CrossRef] [Medline]
    49. Shariff A, Belagodu Sridhar S, Abdullah Basha NF, Bin Taleth Alshemeil SSH, Ahmed Aljallaf Alzaabi NA. Assessing consistency of drug-drug interaction-related information across various drug information resources. Cureus 2021 Mar 08;13(3):e13766 [FREE Full text] [CrossRef] [Medline]
    50. Knijff-Dutmer EA, Schut GA, van de Laar MA. Concomitant coumarin–NSAID therapy and risk for bleeding. Ann Pharmacother 2016 Jun 28;37(1):12-16. [CrossRef]
    51. Holbrook AM, Pereira JA, Labiris R, McDonald H, Douketis JD, Crowther M, et al. Systematic overview of warfarin and its drug and food interactions. Arch Intern Med 2005 May 23;165(10):1095-1106. [CrossRef] [Medline]
    52. Shen C, Jiang B, Yang Q, Wang C, Lu KZ, Gu M, et al. Mobile apps for drug-drug interaction checks in Chinese app stores: systematic review and content analysis. JMIR Mhealth Uhealth 2021 Jun 15;9(6):e26262 [FREE Full text] [CrossRef] [Medline]
    53. Sathian B, Asim M, Banerjee I, Pizarro AB, Roy B, van Teijlingen ER, et al. Impact of COVID-19 on clinical trials and clinical research: A systematic review. Nepal J Epidemiol 2020 Sep 30;10(3):878-887 [FREE Full text] [CrossRef] [Medline]
    54. Unger JM, Xiao H, LeBlanc M, Hershman DL, Blanke CD. Cancer clinical trial participation at the 1-year anniversary of the outbreak of the COVID-19 pandemic. JAMA Netw Open 2021 Jul 01;4(7):e2118433 [FREE Full text] [CrossRef] [Medline]

    ‎
    DCS: Decisional Conflict Scale
    DDI: drug-drug interaction
    EHR: electronic health record
    FHIR: Fast Health Interoperability Resources
    GI: gastrointestinal
    INR: international normalized ratio
    NASA-TLX: National Aeronautics and Space Administration Task Load Index
    NSAID: nonsteroidal anti-inflammatory drug
    NVAF: nonvalvular atrial fibrillation
    OTC: over the counter
    PBC: Perceived Behavioral Control
    SDM: shared decision-making
    SDM-Q-9: 9-item Shared Decision-Making Questionnaire
    SMART: Substitutable Medical Applications and Reusable Technologies
    SUNDAE: Standards for Universal Reporting of Decision Aid Evaluations
    SUS: System Usability Scale
    UC: usual care
    UTAUT: Unified Theory of Acceptance and Use of Technology

    Edited by A Mavragani; submitted 01.06.22; peer-reviewed by J Walsh, P Dunn, T Zhang; comments to author 15.08.22; revised version received 21.08.22; accepted 22.09.22; published 19.10.22

    Copyright

    ©Ainhoa Gomez Lumbreras, Thomas J Reese, Guilherme Del Fiol, Malinda S Tan, Jorie M Butler, Jason T Hurwitz, Mary Brown, Kensaku Kawamoto, Henrik Thiess, Maria Wright, Daniel C Malone. Originally published in JMIR Formative Research (https://formative.jmir.org), 19.10.2022.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.