Original Paper
Abstract
Background: Despite routine review of medication lists during patient encounters, patients’ medication lists are often incomplete and not reflective of actual medication use. Contributing to this situation is the challenge of reconciling medication information from existing health records, along with external locations (eg, pharmacies, other provider/hospital records, and care facilities) and patient-reported use. Advances in the interoperability and digital collection of information provides a foundation for integration of these once disparate information sources.
Objective: We aim to evaluate the effectiveness of and satisfaction with an electronic health record (EHR)-integrated web-based medication reconciliation application, MedTrue (MT).
Methods: We conducted a cluster-randomized controlled trial of MT in 6 primary care clinics within an integrated health care delivery system. Our primary outcome was medication list accuracy, as determined by a pharmacist-collected best-possible medication history (BPMH). Patient and staff perspectives were evaluated through surveys and semistructured interviews.
Results: Overall, 224 patients were recruited and underwent a BPMH with the pharmacist (n=118 [52.7%] usual care [UC], n=106 [47.3%] MT). For our primary outcome of medication list accuracy, 8 (7.5%) patients in the MT arm and 9 (7.6%) in the UC arm had 0 discrepancies (odds ratio=1.01, 95% CI 0.38-2.72, P=.98). The most common discrepancy identified was patients reporting no longer taking a medication (UC mean 2.48 vs MT mean 2.58, P=.21). Patients found MT easy to use and on average would highly recommend MT (average net promoter score=8/10). Staff found MT beneficial but difficult to implement.
Conclusions: The use of a web-based application integrated into the EHR which combines EHR, patient-reported data, and pharmacy-dispensed data did not improve medication list accuracy among a population of primary care patients compared to UC but was well received by patients. Future studies should address the limitations of the current application and assess whether improved implementation strategies would impact the effectiveness of the application.
doi:10.2196/33488
Keywords
Introduction
Background
Nearly 50% of Americans are on at least 1 prescription medication [
]. Among Medicare patients, these prescriptions are prescribed on average by 7 different physicians [ ]. This does not include nonprescription medications, such as over-the-counter medications, herbal medications, vitamins, and other supplements. The prescribing of medications from multiple providers and the prevalence of nonprescription medication use contribute to inaccurate medication lists. Inaccurate medication lists can lead to adverse consequences, including hospitalization and death [ ]. This well-recognized problem has been the subject of initiatives from a variety of organizations [ - ] and is designated a National Patient Safety Goal by the Joint Commission [ ].Medication reconciliation has been recognized as problematic across a variety of health care settings, including primary care. Studies in primary care have found that the rate of discrepancies is high [
- ]. For example, we conducted a study within our primary care environment and identified that when asked, 369 (89.1%) of 414 patients requested a change to their medication list in the electronic health record (EHR) and, on average, patients noted 2.4 discrepancies on their medication lists [ ]. At the time of this finding, there were few published tools to facilitate medication reconciliation in primary care and those that did had limitations. For example, a tool developed by Schnipper et al [ ] was designed to be used after hospital discharge and compared the discharge medication list to the preadmission list held by the ambulatory care site. This tool was limited in that it was not applicable to all primary care patients and did not facilitate the collection of patient-reported medication use. Conversely, a tool by Lesselroth et al [ ] collected medication information from patients using a kiosk; however, this technology was limited to Veterans Affairs facilities and may have missed medications not prescribed outside of those facilities.To address the limitations of existing tools and improve medication reconciliation within our primary care environment, we developed an EHR-integrated web-based reconciliation application, MedTrue (MT), to facilitate the process of reconciling EHR medication lists with other sources, including patient-reported medications within a primary care environment. We designed this tool to have both staff- and patient-facing interfaces.
Objectives
The goal of this paper is to present the results of an evaluation study aimed at assessing the impact of the application MT on medication list accuracy and patient and staff satisfaction. Using mixed methods, we aimed to evaluate the effectiveness of and satisfaction with MT.
Methods
Study Design
We conducted a mixed methods evaluation of MT, including a pragmatic cluster-randomized controlled trial (cRCT), along with patient and provider surveys and semistructured interviews. This study was approved by the Geisinger Institutional Review Board (2018-0174). Patient use of MT was considered not research as this technology was planned to become part of usual care (UC); however, the evaluative components were considered research.
Setting
The trial was conducted at 6 primary care sites, with an even distribution of rural and urban clinics, within a large integrated health care delivery system. Within our system, there are over 40 primary care sites eligible to participate. The specific sites included were chosen collaboratively with input from primary care and clinic site leadership. Leadership was consulted to ensure alignment with ongoing initiatives and recommended sites based on their capacity to participate in the study (eg, adequate staffing). The included clinics had an average staff size (nurses/physicians/physician assistants) of 10. In the sites, nurses or medical assistants would bring the patient to the room, conduct a medication history and reconciliation, and perform a variety of other tasks to prepare the patient for a visit with their clinician. The clinician would then verify and approve any changes to the patient’s medication list. All sites used the same EHR, EpicCare® version 2018 (Madison, WI, USA).
Population
Patients were eligible to participate if they were 18 years of age or older, able to speak English, and seen at a participating site by a member of the primary care team.
Intervention
MT is a real-time web-based application that integrates information from the EHR, patients, and pharmacy-dispensed data into a common database that is viewable to patients through an online portal and in-clinic tablets and to clinicians through an EHR interface. The application was developed using an iterative design process. First, meetings were held with key stakeholders (eg, informatics, clinic staff, and pharmacy) to determine the necessary functionalities of the tool. Based on these initial requirements, we created a minimally viable prototype. We then iteratively improved upon this prototype by implementing it in 2 sites not involved in the later evaluation. These sites would use the tool and provide feedback to the study team. Additionally, study team members would observe the use of the tool in those sites to gather additional insight into how the tool was performing and what adjustments needed to be made. From the feedback received, we created a list of potential improvements, which was prioritized based on necessity and feasibility. The tool tested in this study is the result of several rounds of improvements.
The patient interface presents the patient with their active medication list that is located in the EHR, as well as pharmacy-dispensed data available as an additional function within the EHR (
). The patient is asked to review their list of medications, remove medications they are not currently taking, and enter additional medications missing from the list, such as medications purchased over the counter or medications they take that belong to a friend or family member. Additionally, patients are asked about their adherence to their medications over the past month using a Likert-type scale (ranging from 0% for “never” to 100% for “always”). This type of scale is similar to other validated scales for adherence [ , ]. Patients with an upcoming visit at 1 of the intervention sites had access to MT through our online patient portal up to 2 weeks prior to their visit. If the patient did not complete MT through the patient portal prior to their visit, they were provided with a tablet computer (iPad®) by patient access representatives upon check-in and were asked to complete MT while waiting for their appointment. Patients were not given any training in the use of the tool, since during prototyping, we found that most patients were able to navigate the tool without assistance. Approximately a third of the patients accessed MT using the online portal, while the remainder used iPads.![](https://asset.jmir.pub/assets/41c7ea18199403835809423f3191dda4.png)
MT was available to staff through a direct link (access tab) within the EHR as a web-based application. During the intake process and normal workflow of rooming a patient, staff (ie, nurses and medical assistants) accessed MT when assessing medication use. The interface was presented in an embedded browser within the EHR. For the staff interface, MT displayed all the medications the patient was presented with as well as those the patient entered (
). Discrepancies (ie, patient-suggested removals and additions or pharmacy-dispensed differences) were displayed alongside the EHR-listed medications that were not flagged as discrepant. For each medication adherence, calculated as the proportion of days covered from pharmacy-dispensed data [ ], patient-reported medication adherence, captured from the patient-completed Likert-type item, was also displayed. Staff were instructed to conduct their medication history normally and use MT to confirm or change medications. All changes made within MT were “pushed” to the EHR upon clicking the “Save to EHR” button in MT. Staff at the intervention sites had access to paper and audiovisual training materials but also attended an in-person meeting where MT was introduced and reviewed by members of the study team. Study team members were available for consultation and guidance throughout the study.![](https://asset.jmir.pub/assets/9af98d6ae24e7d4e9fedea4b836ef632.png)
Usual Care
UC providers accessed the EHR-based medication profile that included the standard EHR list of active and discontinued/cancelled medications, along with a separate tab of medications collected through a feed of pharmacy-dispensed data. Patient-reported discrepancies (additions, changes, etc) were also available if the patient completed a previsit questionnaire through the online patient portal (approximately a third of patients have portal accessibility). Standard medication list patient information collected was similar to the MT intervention but did not include self-reported adherence. Hence, UC clinics also had a degree of patient and pharmacy-dispensed information available to clinicians, along with EHR lists, although this information was not available for all patients and information in the EHR was not available within 1 tab. The EHR system used in both UC and MT clinics was EpicCare® version 2018.
Data Collection
Best-Possible Medication History
Trained pharmacists approached a convenience sample of patients exposed to MT as well as a convenience sample of patients in the UC arm to collect a best-possible medication history (BPMH); feasibility considerations informed our decision to use convenience sampling. All pharmacists were licensed and had experience conducting medication histories. Prior to enrolling patients, pharmacists underwent brief training by study staff reviewing components of a BPMH and a clinical interview guide complete with prompts (
). This guide was based on the literature related to conducting a BPMH [ , , ]. Pharmacists identified patients via a custom web-based dashboard listing patients with a documented launch of MT by a clinician (MT arm only) or arrival to an eligible visit for UC. Patients completing an encounter in UC were assumed to have EHR-based medication reconciliation due to routine workflow completion of medication reconciliation in all primary care encounters. Pharmacists were blinded to clinic site use of MT, as dashboards only supplied names, not information about MT use. Only study personnel and clinic staff were aware of study group assignment. Pharmacists approached eligible patients after their visit and asked whether they would be interested in participating in a study. If a patient was agreeable, the pharmacist brought them to a private room for consent and to conduct the BPMH.Pharmacists documented discrepancies for each medication on the postvisit EHR active medication list and also for any medication not present on the medication list. Each medication was coded with only 1 type of medication discrepancy, which fell into 1 of 4 domains. The first was patient acknowledgement of taking the medication differently than prescribed (eg, allergy medication is prescribed daily, but the patient takes it seasonally). Second was removal of a medication on the medication list. The third domain was a modification to the formulation, strength, dose, frequency, route of medication, or timing of administration. The final discrepancy was adding a medication to a profile that was not on the EHR medication list. Since taking a medication differently than prescribed would not automatically infer an inaccurate medication or description of the medication on the medication profile (eg, the patient reported missing a weekend dose of a medication due to forgetfulness), we did not include this in our composite discrepancy endpoint (see later) but did report on this individually. In addition, after each BPMH, pharmacists provided feedback on their confidence that the list they gathered from the patient was accurate using a 3-point scale (not confident, somewhat confident, and confident).
Following the final BPMH collection, study investigators reviewed the reconciled lists of medications in a blinded manner to ensure consistent coding of discrepancies. Inconsistencies were flagged and recoded for final analysis.
Consent, the BPMH, and pharmacist questions were documented using REDCap (Vanderbilt University, Nashville, TN, USA) [
, ]. The interviews took up to an hour, and patients were compensated US $10 for their time in the form of a gift card.Surveys
All patients exposed to MT were given an opportunity to complete a survey on the usability of and satisfaction with using MT. The survey was presented to the patient following MT completion. Responses were collected within MT.
Staff participating in the intervention arm of the study were also given a survey on the usability of and satisfaction with using MT near the end of the study. Staff were contacted by email and invited to participate in the survey, with up to 2 reminder emails sent at weekly intervals. Surveys were collected using Microsoft Forms (Microsoft Corporation, Redmond, WA, USA).
Interviews
The interviews were guided by a qualitative descriptive approach [
]. Patients in the intervention arm were invited to participate in interviews about their experience using MT through convenience sampling. After a visit where MT was used, patients were approached by a research assistant, who invited them to participate in a brief interview. Consenting patients were interviewed using a semistructured interview guide ( ). The research assistant underwent brief training in qualitative interviewing from 1 of the investigators with experience in qualitative interviewing. The interviews lasted no more than 30 minutes, and patients were compensated US $10 for their time in the form of a gift card.Rooming staff who used MT during the study period were also given an opportunity to voluntarily participate in semistructured interviews to discuss their experience using MT. The interviews were conducted by an investigator with experience in qualitative interviewing using a semistructured interview guide (
). The interviews lasted no more than 30 minutes, and staff were not offered compensation.Primary Outcome
The primary outcome of the study was the accuracy of the medication list. We defined accuracy by counting discrepancies between the medication list gathered through MT or UC and the list gathered by the pharmacist conducting the BPMH. A perfectly accurate medication list would be a list with 0 identified discrepancies.
Secondary Outcomes
We secondarily measured accuracy using a composite numerical assessment of accuracy, where each medication in a list could contribute either 0 or 1 discrepancy, and all discrepancies were totaled per patient. This endpoint included all of the following medication discrepancies: not taking the listed medication, needing the medication to be modified (eg, change in dose or frequency), or having the medication added to their list. Although taking a medication differently was included in our list of discrepancies, this was not included in the composite accuracy list, since this discrepancy would in of itself not necessitate a change in the actual prescription as written, and if the list needed to be modified by the BPMH, 1 of the other listed discrepancies would have been selected over this discrepancy, as per BPMH instructions. We reported individual types of discrepancies identified per group (ie, adding, modifying, removing, or taking a medication differently) on a medication level. To determine the rate of medication discrepancies per patient, we calculated the percentage of the patients’ medications that had discrepancies. Finally, we conducted a sensitivity analysis based on the pharmacists’ confidence in their BPMH, including only those patients where the pharmacist conducting the BPMH was confident in having an accurate medication list. Patient and staff satisfaction with MT, as measured by surveys and qualitative interviews, was also reported.
Sample Size
Assuming a 1% intracluster correlation coefficient and a 19% difference in accuracy rates, our target sample size was 300 patients (n=150 [50%] intervention, n=150 [50%] control; each of the 6 sites was expected to enroll 50 [16.7%] participants) to achieve 80% power. During data collection, we conducted an interim look to verify our sample size assumptions and found that our initial assumptions regarding the proportion of completely accurate medication lists (no discrepancies), which were based upon previous studies [
- ], were overly optimistic (~50% in previous studies vs 4% in our interim analysis). As a result, we were underpowered to detect our primary outcome; however, we were powered to detect a secondary composite outcome based on a count of discrepancies. Later, due to feasibility, namely difficulties in recruitment, the trial was stopped.Randomization
In total, 6 sites were chosen in collaboration with primary care leadership and randomized to the intervention or the control arm using a random number generator in Microsoft Excel (Microsoft Corporation). Prior to recruitment, due to unforeseen challenges, 1 UC site was substituted for a different clinic with similar baseline characteristics.
Blinding
Neither patients nor clinicians were blinded due to the nature of the intervention; however, there were steps taken to blind the pharmacists conducting the BPMH (as described in the Data Collection section). During the analysis, the statistician was blinded as to which arm was intervention and which was control.
Analysis
Qualitative Analysis
Interviews were transcribed verbatim and analyzed thematically [
]. Patient and staff interviews were separately analyzed. The transcripts were independently read by 2 study team members, who collaboratively developed 2 codebooks (1 for patients and 1 for staff). The transcripts were independently coded by 2 study team members using the appropriate codebook. Discrepancies in coding were discussed and resolved by consensus. The codes were then grouped into themes, which were refined based on discussion.Statistical Analysis
To understand the representativeness of our sample, we compared the demographics of patients seen at the clinics during the intervention period to those who received a BPMH. We described continuous variables using means and SDs as well as medians and IQRs. Categorical variables were described using frequencies and percentages. Wilcoxon rank-sum tests were used to compare the differences between 2 groups for skewed continuous outcomes; chi-square tests were used to evaluate the independence of categorical outcomes between MT and UC, where odds ratios (ORs) were presented. A mixed effects model with Poisson regression was performed to model the impact of MT on count data (number of medication discrepancies) after adjusting for age, sex, and Charlson comorbidity index, with the total number of medications as the offset variable. Incidence rate ratios (IRRs) and 95% CIs were estimated from Poisson regression models. Sensitivity analyses were performed with or without considering cluster-level covariates for cRCTs. P<.05 was considered statistically significant. Statistical analyses were performed using RStudio version 1.2.1335 (RStudio, PBC, Boston, MA, USA) [
].Additionally, we conducted sensitivity analyses including only those patients for whom the pharmacist rated that they were confident in the history they gathered from the patients; histories where the pharmacist was either somewhat confident or not confident were excluded.
Results
Patient Characteristics
MT was implemented in 3 clinics from July 10, 2018, through August 1, 2019. Over this time frame, MT was accessed 10,835 times by 7342 distinct patients. The majority of access (6501/10,835, 60%) was by tablet computer. Of the 7342 patients using MT, 4005 (54.55%) removed at least 1 medication (mean 3.6 medications removed), 606 (8.25%) added at least 1 medication (mean 1.6 medications added), and 6061 (82.55%) completed the adherence questions. Staff accessed MT for 8443 distinct patients (staff could access MT even if patients did not), removed at least 1 medication for 3544 (41.98%) patients (mean 2.1 medications removed) and added at least 1 medication for 1643 (19.46%) patients (mean 3.3 medications added).
Clinics allocated to MT and UC had similar baseline characteristics, although patients at MT clinics were slightly younger in age (46 vs 48 years), had fewer African Americans (n=871 [3%] vs n=1255 [5%]), less depression (n=1441 [5%] vs n=1490 [6%]), and took more medications (7.75 vs 6.53 medications per patient); see
.Overall, 224 patients were recruited and underwent a BPMH with the pharmacist (n=106 [47.3%] MT, n=118 [52.7%] UC). The samples of patients who underwent a BPMH were older; were more likely to be female; had depression, dementia, or mild cognitive impairment; were more likely to be married; and had more medications than the baseline clinic population (similar between groups).
Characteristic | MTa (clinic level), N=30,275 | UCb (clinic level), N=26,463 | P value (MT vs UC population) | UC (patient level), N=118 | P value (population vs sample) | MT patient level), N=106 | P value (population vs sample) | P value (MT vs UC sample) | |
Age (years), mean (SD) | 46 (23) | 48 (21) | <.001 | 55 (18) | <.001 | 52 (18) | .01 | .23 | |
Female, n (%) | 15,740 (59.48) | 17,886 (59.08) | .35 | 76 (64.41) | .04 | 71 (66.98) | .10 | .69 | |
Race, n (%) | <.001 | .02 | .54 | .79 | |||||
Caucasian | 29,021 (95.86) | 24,882 (94.03) | 114 (96.6) | 104 (95.3) | |||||
African American | 871 (2.88) | 1255 (4.74) | 3 (2.5) | 3 (2.8) | |||||
Other | 383 (1.27) | 326 (1.23) | 1 (0.8) | 2 (1.9) | |||||
Ethnicity: Hispanic, n (%) | 1243 (4) | 1003 (4) | .06 | 4 (3) | .03 | 2 (2) | .03 | .49 | |
Charlson comorbidity index, mean (SD) | 2.20 (2.79) | 2.37 (2.84) | <.001 | 3.12 (2.92) | <.001 | 2.64 (2.72) | .01 | .25 | |
Disorder, n (%) | |||||||||
MCI or dementia | 350 (1.16) | 372 (1.41) | .01 | 5 (4.2) | .01 | 5 (4.7) | <.001 | .86 | |
Depression | 1441 (4.76) | 1490 (5.63) | <.001 | 23 (19.49) | <.001 | 25 (23.58) | <.001 | .46 | |
Relationship status (married), n (%) | 1682 (5.56) | 1478 (5.59) | .88 | 63 (53.4) | <.001 | 47 (44.3) | <.001 | .18 | |
Number of medications, mean (SD) | 7.75 (5.64) | 6.53 (5.05) | <.001 | 12.38 (7.54) | <.001 | 11.15 (7.29) | <.001 | .16 |
aMT: MedTrue.
bUC: usual care.
Primary Outcome
For our primary outcome of medication list accuracy (defined as 0 discrepancies), we did not detect a statistically significant difference between MT and UC (OR=1.01, 95% CI 0.38-2.70, P=.98). Pharmacists completing the BPMH reported being confident in the medication history for 200 (89.2%) of 224 patients (91 [85.8%] MT, 109 [92.4%] UC). The primary outcome remained the same upon conducting a sensitivity analysis limited to those BPMHs in which the pharmacist was confident (OR=1.08, 95% CI 0.39-3.02, P=.88).
Secondary Outcomes
The composite secondary outcome of total medication discrepancies, defined as the sum of additions, medications the patient was not taking, and medications that were modified summed per patient, was comparable between MT and UC arms (MT median=4, IQR 2-7 vs UC median=3, IQR 2-6; P=.15). The most common discrepancy was medication the patients’ reported not taking, with a median of 2 (IQR 1-3) identified per patient in the UC arm and 1 (IQR 1-3) per patient identified in the MT arm.
The composite outcome was consistent upon sensitivity analysis limited to those patients where a pharmacist was confident in their BPMH (MT median=3, IQR 2-5 vs UC median=4, IQR 2-7; P=.10).
A Poisson regression model indicated that the MT and UC arms were comparable regarding the rate of discrepancies (IRR=1.22, 95% CI 0.97-1.55, P=.09). The rate of discrepancies was significantly affected by age (IRR=1.01, 95% CI 1.00-1.01, P=.03), male sex (IRR=0.81, 95% CI 0.71-0.93, P=.003), and Charlson comorbidity index (IRR=0.95, 95% CI 0.92-0.98, P=.001). Sensitivity analyses with or without considering cluster-level covariates confirmed the robustness of findings of the Poisson regression model (data not shown).
In addition to the composite, we assessed accuracy as the proportion of medications that were discrepant. The proportion of discrepant medications was not significantly affected by MT compared to UC (MT median=40%, IQR 23%-54% vs UC median=34%, IQR 21%-59%; P=.87); this finding was similar on sensitivity analysis (MT median=40%, IQR 22%-55% vs UC median=33%, IQR 21%-56%; P=.66). Further accuracy data can be found in
- .Medication discrepancies | UCa (N=118), frequency (%) | MTb (N=106), frequency (%) | ORc (95% CI) | P valued | |
Primary outcomee | 109 (92.4) | 98 (92.5) | 1.01 (0.38-2.72) | .98 | |
Modification | 72 (61.0) | 55 (51.9) | 0.69 (0.41-1.17) | .17 | |
Addition | 51 (43.2) | 42 (39.6) | 0.86 (0.51-1.47) | .59 | |
Not taking medication | 93 (78.8) | 79 (74.5) | 0.79 (0.42-1.46) | .45 | |
Taking medications differently | 24 (20.3) | 22 (20.8) | 1.03 (0.54-1.96) | .94 |
aUC: usual care.
bMT: MedTrue.
cOR: odds ratio.
dP value based on chi-square test.
ePrimary outcome including not taking + modifying + adding medications.
Medication discrepancies | UCa (N=118), mean (SD) | MTb (N=106), mean (SD) | UC (N=118), median (IQR) | MT (N=106), median (IQR) | P valuec | |
Primary outcomed | 4.84 (3.85) | 4.58 (4.89) | 4 (2-7) | 3 (2-6) | .15 | |
Modification | 1.51 (1.84) | 1.29 (1.96) | 1 (0-2) | 1 (0-2) | .17 | |
Addition | 0.85 (1.33) | 0.72 (1.23) | 0 (0-1) | 0 (0-1) | .43 | |
Not taking medication | 2.48 (2.60) | 2.58 (4.10) | 2 (1-3) | 1 (0-3) | .21 | |
Taking medications differently | 0.33 (0.77) | 0.28 (0.61) | 0 (0-0) | 0 (0-0) | .92 |
aUC: usual care.
bMT: MedTrue.
cP value based on Wilcoxon test.
dPrimary outcome including not taking + modifying + adding medications.
Medication discrepancies | MTa list %, mean (SD) | UCb list %, mean (SD) | MT list %, median (IQR) | UC list %, median (IQR) | P valuec | |
Primary outcomed | 40 (23) | 39 (24) | 40 (23-54) | 34 (21-59) | .87 | |
Modification | 11 (13) | 10 (14) | 8 (0-18) | 6 (0-17) | .36 | |
Addition | 8 (14) | 8 (14) | 0 (0-12) | 0 (0-11) | .68 | |
Not taking medication | 20 (17) | 21 (21) | 18 (6-29) | 16 (1-33) | .82 | |
Taking medications differently | 3 (8) | 4 (9) | 0 | 0 | .63 |
aMT: MedTrue.
bUC: usual care.
cP value based on Wilcoxon test.
dPrimary outcome including not taking + modifying + adding medications.
Usability and Satisfaction
Of the 7342 patients who accessed MT, 1450 (19.75% response rate) completed the survey. Patients were satisfied with MT, giving it an average rating of 8 out of 10 for their likelihood to recommend to a friend or family member (
).Additionally, 1276 (88%) of 1450 patients agreed or strongly agreed that it was easy to use, and 1233 (85%) agreed or strongly agreed that it helped them create a more accurate medication list (
).Patient survey question | Score (1-10) | Mean (SD) | |||||||||
1, n (%) | 2, n (%) | 3, n (%) | 4, n (%) | 5, n (%) | 6, n (%) | 7, n (%) | 8, n (%) | 9, n (%) | 10, n (%) | ||
On a scale of 1 (low) to 10 (high), what is the likelihood you would recommend MTa to a friend or family member? | 41 (2.8) | 15 (1) | 15 (1) | 18 (1.2) | 155 (10.7) | 112 (7.7) | 149 (10.3) | 222 (15.3) | 156 (10.8) | 567 (39.1) | 7.95 (2.29) |
aMT: MedTrue.
Patient survey | Strongly agree, n (%) | Agree, n (%) | Neutral, n (%) | Disagree, n (%) | Strongly disagree, n (%) |
I would use MTa in the future. | 643 (44.3) | 568 (39.2) | 169 (11.7) | 24 (1.7) | 46 (3.2) |
MT helped me create an accurate medication list. | 641 (44.2) | 596 (41.1) | 124 (8.6) | 44 (3.03) | 45 (3.1) |
MT is easy to use. | 724 (49.9) | 559 (38.6) | 103 (7.1) | 17 (1.2) | 47 (3.2) |
MT reminded me to include all my medications, including those my doctor prescribes and those I purchase over the counter. | 721 (49.7) | 592 (40.8) | 77 (5.3) | 21 (1.4) | 39 (2.7) |
aMT: MedTrue.
These findings were echoed in the patients’ qualitative interviews and were consistent across sites. Of the 7342 patients who used MT, 14 (0.2%) participated in interviews (1 spouse was also present for 1 of the interviews). Overall, participants had a positive experience with MT. Most found it easy to use and were able to complete the application quickly. The ability to visually see their list of medications rather than be asked about them was noted as an important feature. Seeing the medication names helped patients recognize the medications because they are used to seeing them in print on their medication packaging. Some patients felt it was more efficient than the normal workflow, whereas others disagreed and felt it was creating unnecessary work for them. The patients also noted that having the indication for the medication would help with recognition and recall. Example quotes from patients illustrating these themes can be found in
.Of the 22 rooming staff members who accessed MT, 18 (82% response rate) completed the survey. Staff were unsatisfied with MT, giving it an average rating of 0 out of 10 (
).All staff members disagreed or strongly disagreed that they were satisfied with MT and noted that it did not fit well in their workflow (
).Interviews provided a more nuanced perspective from staff, with themes and example quotes in
.MT characteristic | Example quote |
Usability |
|
Benefits |
|
Necessity |
|
Suggestions |
|
aMT: MedTrue.
Staff survey question | Score (1-10) | Mean (SD) | |||||||||
1, n (%) | 2, n (%) | 3, n (%) | 4, n (%) | 5, n (%) | 6, n (%) | 7, n (%) | 8, n (%) | 9, n (%) | 10, n (%) | ||
On a scale of 1 (low) to 10 (high), what is the likelihood you would recommend MTa to a friend or family member? | 16 (88) | 1 (6) | —b | — | 1 (6) | — | — | — | — | — | 0.28 (0.96) |
aMT: MedTrue.
bNot applicable.
Staff survey | Strongly agree, n (%) | Agree, n (%) | Neutral, n (%) | Disagree, n (%) | Strongly disagree, n (%) |
I am satisfied with MTa. | —b | — | — | 4 (22) | 14 (78) |
I want to use MT in the future. | — | — | — | 3 (17) | 15 (83) |
I would want to use MT at every visit with every patient. | — | — | — | 3 (17) | 15 (83) |
MT helped me conduct medication reconciliation with my patients. | — | — | — | 6 (33) | 12 (67) |
MT is compatible with my clinic flow. | — | — | — | 3 (17) | 15 (83) |
MT is easy to use. | — | 1 (6) | — | 3 (17) | 14 (78) |
aMT: MedTrue.
bNot applicable.
MT characteristic | Example quote |
Usability |
|
Benefits |
|
Functionality |
|
Patient Concerns |
|
aMT: MedTrue.
Discussion
Principal Results
The use of MT, a web-based application facilitating medication reconciliation, did not impact the accuracy of medication lists. The application was well received by patients; however, implementation challenges limited usability and acceptance by staff. The failure to improve medication list accuracy reflects both implementation challenges as well as the challenge of reliance on a technology-only solution to obtaining accurate medication lists.
Comparison With Prior Work
Our application is 1 of many technologies and applications being developed to obtain and maintain accurate medication lists [
- ]. Many applications are focused on hospitalized patients, and not all applications have been pragmatically evaluated or included a methodology to compare accuracy with the gold-standard BPMH. Lesselroth et al [ , ] have developed a medication reconciliation kiosk where patients can review and edit their medication lists. Similar to MT, this information is imported into the EHR and reviewed by clinicians [ ]. In their evaluation, they found similar rates of discrepancies (29%-39% vs 34%-40%), and the overall number of patients with discrepancies were similar (91%-99% vs 92%-93%) [ , ]. When evaluating the implementation of their application, they also found that time is a factor in using medication reconciliation technology and that, to be taken up, the technology needs to be well integrated into current workflows [ ]. This was also found to be a challenge with our application despite its intent to work within the staff workflow.Strengths
We implemented several design elements to enhance the validity of our evaluation. We randomized clinics to MT or UC to avoid potential contamination that might have occurred at the patient or clinician level. We controlled for possible confounding factors through an adjusted analysis, the results of which mirrored the unadjusted analysis, increasing our confidence in our results. Throughout the trial, we also attempted blinding, wherever possible, to minimize bias. Other strengths include the use of pharmacists to conduct a BPMH and our application itself, which incorporates patient, clinician, and pharmacy-dispensed data to facilitate medication reconciliation.
Limitations
Despite these strengths, a few limitations should be noted. Due to the nature of the intervention, clinicians and, to an extent, patients could not be blinded to the intervention. Fidelity to the use of MT was not always optimal and was variably used among rooming staff. As indicated in the staff survey and interviews, MT was not favorably viewed by the staff and as such they might have failed to use MT consistently, which would reduce observed effectiveness. Technical challenges, such as loading delays and internet connectivity for tablets, also affected the usability of the application in our clinics. Additionally, although the application was implemented within the EHR, it was outside of the existing workflow and required several additional clicks. Finally, we are underpowered to confidently detect the differences we observed in this trial. This was, in part, due to optimistic initial estimates of the effect of MT, as well as recruitment challenges. Our initial estimates of the effect of MT may have been optimistic as by the time the application was implemented, certain features were also available in the EHR, thereby reducing the difference between MT and UC.
Conclusion
Novel approaches to medication reconciliation are needed to ensure safe and effective medication use. The use of technology, such as MT, a web-based application integrating various data sources, is a promising solution to reduce the rate of discrepancies. In our study, MT did not affect the rate of discrepancies, which may have been the result of failure to use the tool consistently and with fidelity due to implementation challenges. Thus, technology solutions may need to be paired with additional implementation strategies, and particular attention should be paid to how these technologies and other innovations are integrated into clinical workflows to facilitate fidelity and maximize effectiveness.
Acknowledgments
We would like to thank all the patients, pharmacists, nurses, and clinicians who participated in this study. Funding was provided by Merck Sharp & Dohme Corp, a subsidiary of Merck & Co, Inc, Kenilworth, NJ, USA.
Conflicts of Interest
None declared.
Editorial notice: This randomized study was not registered. The authors explained that the trial was intended to be a part of a staged roll-out within usual care to improve the quality of treatment received by patients. Randomization was used to assign which sites received the improvements first. The editor granted an exception from ICMJE rules mandating prospective registration of randomized trials, because the risk of bias appears low and the study was considered formative. However, readers are advised to carefully assess the validity of any potential explicit or implicit claims related to primary outcomes or effectiveness.
Qualitative interview guides.
DOCX File , 19 KB
CONSORT-EHEALTH checklist (V 1.6.1).
PDF File (Adobe PDF File), 1309 KBReferences
- Gindi R, Holmes JS, Arispe IE. Health, United States 2017: With Special Feature on Mortality. 2018. URL: https://stacks.cdc.gov/view/cdc/59199 [accessed 2022-02-17]
- Pham HH, Schrag D, O'Malley AS, Wu B, Bach PB. Care patterns in Medicare and their implications for pay for performance. N Engl J Med 2007 Mar 15;356(11):1130-1139. [CrossRef]
- Santell JP. Reconciliation failures lead to medication errors. Jt Comm J Qual Patient Saf 2006 Apr;32(4):225-229. [CrossRef]
- Institute for Healthcare Improvement. Medication Reconciliation to Prevent Adverse Drug Events. 2022. URL: http://www.ihi.org/Topics/ADEsMedicationReconciliation/Pages/default.aspx [accessed 2022-02-17]
- Society of Hospital Medicine. Medication Reconciliation for Hospitalists. 2022. URL: https://www.hospitalmedicine.org/clinical-topics/medication-reconciliation/ [accessed 2022-02-17]
- The Joint Commission. National Patient Safety Goals. 2022. URL: https://www.jointcommission.org/standards_information/npsgs.aspx [accessed 2022-02-17]
- World Health Organization. High 5s: Standard Operating Procedures. 2022. URL: https://www.who.int/patientsafety/topics/high-5s/en/ [accessed 2022-02-17]
- Dullabh P, Sondheimer N, Katsh E, Evans MA. How patients can improve the accuracy of their medical records. EGEMS (Wash DC) 2014 Jan 01;2(3):1080 [FREE Full text] [CrossRef] [Medline]
- Andrus MR. Student pharmacist initiated medication reconciliation in the outpatient setting. Pharm Pract (Granada) 2012 Apr;10(2):78-82 [FREE Full text] [CrossRef] [Medline]
- Varkey P, Cunningham J, Bisping S. Improving medication reconciliation in the outpatient setting. Jt Comm J Qual Patient Saf 2007 May;33(5):286-292. [CrossRef]
- Schnipper J, Liang C, Hamann C, Karson A, Palchuk M, McCarthy P, et al. Development of a tool within the electronic medical record to facilitate medication reconciliation after hospital discharge. J Am Med Inform Assoc 2011 May 01;18(3):309-313 [FREE Full text] [CrossRef] [Medline]
- Lesselroth BJ, Felder RS, Adams SM, Cauthers PD, Dorr DA, Wong GJ, et al. Design and implementation of a medication reconciliation kiosk: the Automated Patient History Intake Device (APHID). J Am Med Inform Assoc 2009 May 01;16(3):300-304. [CrossRef]
- Feldman BJ, Fredericksen RJ, Crane PK, Safren SA, Mugavero MJ, Willig JH, et al. Evaluation of the single-item self-rating adherence scale for use in routine clinical care of people living with HIV. AIDS Behav 2013 Jan 30;17(1):307-318 [FREE Full text] [CrossRef] [Medline]
- Lu M, Safren SA, Skolnik PR, Rogers WH, Coady W, Hardy H, et al. Optimal recall period and response task for self-reported HIV medication adherence. AIDS Behav 2008 Jan 19;12(1):86-94. [CrossRef] [Medline]
- Nau DP. Proportion of days covered (PDC) as a preferred method of measuring medication adherence. Pharmacy Quality Alliance. 2011. URL: http://ep.yimg.com/ty/cdn/epill/pdcmpr.pdf [accessed 2022-02-17]
- Drenth-van Maanen AC, Spee J, van Hensbergen L, Jansen PAF, Egberts TCG, van Marum RJ. Structured history taking of medication use reveals iatrogenic harm due to discrepancies in medication histories in hospital and pharmacy records. J Am Geriatr Soc 2011 Oct 18;59(10):1976-1977. [CrossRef] [Medline]
- MARQUIS Investigators. MARQUIS Implementation Manual: A guide for medication reconciliation quality improvement. Society of Hospital Medicine. 2014. URL: https://www.hospitalmedicine.org/globalassets/clinical-topics/clinical-pdf/shm_medication_reconciliation_guide.pdf [accessed 2022-02-17]
- Harris PA, Taylor R, Minor BL, Elliott V, Fernandez M, O'Neal L, REDCap Consortium. The REDCap consortium: Building an international community of software platform partners. J Biomed Inform 2019 Jul;95:103208 [FREE Full text] [CrossRef] [Medline]
- Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap): a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009 Apr;42(2):377-381 [FREE Full text] [CrossRef] [Medline]
- Sandelowski M. Whatever happened to qualitative description? Res Nurs Health 2000 Aug;23(4):334-340. [CrossRef]
- Wolff CM, Nowacki AS, Yeh J, Hickner JM. A randomized controlled trial of two interventions to improve medication reconciliation. J Am Board Fam Med 2014 May 07;27(3):347-355 [FREE Full text] [CrossRef] [Medline]
- Tam VC, Knowles SR, Cornish PL, Fine N, Marchesano R, Etchells EE. Frequency, type and clinical importance of medication history errors at admission to hospital: a systematic review. CMAJ 2005 Aug 30;173(5):510-515 [FREE Full text] [CrossRef] [Medline]
- Leonhardt KK, Pagel P, Bonin D, Moberg DP, Dvorak ML, Hatlie MJ. Creating an accurate medication list in the outpatient setting through a patient-centered approach. In: Henriksen K, Battles JB, Keyes MA, Grady ML, editors. Advances in Patient Safety: New Directions and Alternative Approaches. Rockville, MD: Agency for Healthcare Research and Quality; Aug 2008.
- Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006 Jan;3(2):77-101. [CrossRef]
- Rizopoulos D. GLMMadaptive: Generalized Linear Mixed Models using Adaptive Gaussian Quadrature, R package version 0.5-1. 2022. URL: https://drizopoulos.github.io/GLMMadaptive/ [accessed 2022-02-17]
- Kramer H, Gibson B, Livnat Y, Thraen I, Brody A, Rupper R. Evaluation of an electronic module for reconciling medications in home health plans of care. Appl Clin Inform 2017 Dec 16;07(02):412-424. [CrossRef]
- Wang H, Meng L, Song J, Yang J, Li J, Qiu F. Electronic medication reconciliation in hospitals: a systematic review and meta-analysis. Eur J Hosp Pharm 2018 Sep 08;25(5):245-250 [FREE Full text] [CrossRef] [Medline]
- Mekonnen AB, Abebe TB, McLachlan AJ, Brien JE. Impact of electronic medication reconciliation interventions on medication discrepancies at hospital transitions: a systematic review and meta-analysis. BMC Med Inform Decis Mak 2016 Aug 22;16(1):112 [FREE Full text] [CrossRef] [Medline]
- Marien S, Krug B, Spinewine A. Electronic tools to support medication reconciliation: a systematic review. J Am Med Inform Assoc 2017 Jan;24(1):227-240 [FREE Full text] [CrossRef] [Medline]
- Bassi J, Lau F, Bardal S. Use of information technology in medication reconciliation: a scoping review. Ann Pharmacother 2010 May 01;44(5):885-897. [CrossRef]
- Kripalani S, Hart K, Schaninger C, Bracken S, Lindsell C, Boyington DR. Use of a tablet computer application to engage patients in updating their medication list. Am J Health Syst Pharm 2019 Feb 09;76(5):293-300 [FREE Full text] [CrossRef] [Medline]
- Marien S, Legrand D, Ramdoyal R, Nsenga J, Ospina G, Ramon V, et al. A user-centered design and usability testing of a web-based medication reconciliation application integrated in an eHealth network. Int J Med Inform 2019 Jun;126:138-146. [CrossRef] [Medline]
- Lesselroth B, Adams S, Felder R, Dorr DA, Cauthers P, Church V, et al. Using consumer-based kiosk technology to improve and standardize medication reconciliation in a specialty care setting. Jt Comm J Qual Patient Saf 2009 May;35(5):264-AP1. [CrossRef]
- Lesselroth B, Adams K, Church V, Tallett S, Russ Y, Wiedrick J, et al. Evaluation of multimedia medication reconciliation software: a randomized controlled, single-blind trial to measure diagnostic accuracy for discrepancy detection. Appl Clin Inform 2018 Apr 02;9(2):285-301 [FREE Full text] [CrossRef] [Medline]
- Wright T, Adams K, Church V, Ferraro M, Ragland S, Sayers A, et al. Implementation of a medication reconciliation assistive technology: a qualitative analysis. AMIA Annu Symp Proc 2017;2017:1802-1811 [FREE Full text] [Medline]
Abbreviations
BPMH: best-possible medication history |
cRCT: cluster-randomized controlled trial |
EHR: electronic health record |
IRR: incidence rate ratio |
MT: MedTrue |
OR: odds ratio |
UC: usual care |
Edited by A Mavragani; submitted 09.09.21; peer-reviewed by C Steele Gray, J Walsh; comments to author 30.09.21; revised version received 25.11.21; accepted 13.01.22; published 08.03.22
Copyright©Michael R Gionfriddo, Yirui Hu, Bhumika Maddineni, Melissa Kern, Vanessa Hayduk, William R Kaledas, Nevan Elder, Jeffrey Border, Katie Frusciante, Maria Kobylinski, Eric A Wright. Originally published in JMIR Formative Research (https://formative.jmir.org), 08.03.2022.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.