Published on in Vol 5, No 2 (2021): February

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/19074, first published .
Assessment of Patients’ Ability to Review Electronic Health Record Information to Identify Potential Errors: Cross-sectional Web-Based Survey

Assessment of Patients’ Ability to Review Electronic Health Record Information to Identify Potential Errors: Cross-sectional Web-Based Survey

Assessment of Patients’ Ability to Review Electronic Health Record Information to Identify Potential Errors: Cross-sectional Web-Based Survey

Original Paper

1Patient Safety Translational Research Centre, Institute of Global Health Innovation, Imperial College London, London, United Kingdom

2Center for Health Technology and Services Research / Department of Community Medicine, Health Information and Decision (CINTESIS/MEDCIDS), Faculty of Medicine, University of Porto, Porto, Portugal

3Imperial College Healthcare NHS Trust, London, United Kingdom

*these authors contributed equally

Corresponding Author:

Ana Luisa Neves, MSc, MD, PhD

Patient Safety Translational Research Centre

Institute of Global Health Innovation

Imperial College London

St Mary’s Campus Queen

Elizabeth Queen Mother Wing

London, W2 1NY

United Kingdom

Phone: 44 (0)20 7589 5111

Email: ana.luisa.neves14@ic.ac.uk


Background: Sharing personal health information positively impacts quality of care across several domains, and particularly, safety and patient-centeredness. Patients may identify and flag up inconsistencies in their electronic health records (EHRs), leading to improved information quality and patient safety. However, in order to identify potential errors, patients need to be able to understand the information contained in their EHRs.

Objective: The aim of this study was to assess patients’ perceptions of their ability to understand the information contained in their EHRs and to analyze the main barriers to their understanding. Additionally, the main types of patient-reported errors were characterized.

Methods: A cross-sectional web-based survey was undertaken between March 2017 and September 2017. A total of 682 registered users of the Care Information Exchange, a patient portal, with at least one access during the time of the study were invited to complete the survey containing both structured (multiple choice) and unstructured (free text) questions. The survey contained questions on patients’ perceived ability to understand their EHR information and therefore, to identify errors. Free-text questions allowed respondents to expand on the reasoning for their structured responses and provide more detail about their perceptions of EHRs and identifying errors within them. Qualitative data were systematically reviewed by 2 independent researchers using the framework analysis method in order to identify emerging themes.

Results: A total of 210 responses were obtained. The majority of the responses (123/210, 58.6%) reported understanding of the information. The main barriers identified were information-related (medical terminology and knowledge and interpretation of test results) and technology-related (user-friendliness of the portal, information display). Inconsistencies relating to incomplete and incorrect information were reported in 12.4% (26/210) of the responses.

Conclusions: While the majority of the responses affirmed the understanding of the information contained within the EHRs, both technology and information-based barriers persist. There is a potential to improve the system design to better support opportunities for patients to identify errors. This is with the aim of improving the accuracy, quality, and timeliness of the information held in the EHRs and a mechanism to further engage patients in their health care.

JMIR Form Res 2021;5(2):e19074

doi:10.2196/19074

Keywords



With advancing digitalization and focus on patient centricity, sharing of personal health information with patients has gained increasing importance and prevalence worldwide [1-3]. The UK Department of Health and Social Care [4] has emphasized the need for patients to have access to their records and enhanced control over them. Evidence suggests that sharing personal health information positively impacts quality of care across several domains such as safety and patient-centeredness [5-7].

A potential opportunity to improve patient safety can be realized when patients access and read their health records and identify, and even correct, errors within them [3]. Potential errors include incorrect or missing information concerning administrative details, diagnosis, or treatment, all of which diminish the accuracy, quality, and timeliness of the information held in the medical record, which affects care delivery.

Previous studies have explored the potential of sharing medical records with patients for error correction purposes. Ross and Lin’s [3] review of patient access to medical records indicated that 11 studies reported a facilitation of error correction. However, the studies were based on paper records and the information in 4 studies was anecdotal while that in the other 7 was purely descriptive [3]. Considering electronic health records (EHRs), Bell et al evaluated the impact of an OpenNotes patient reporting tool focused on safety concerns and found that patients and health care partners reported safety concerns in about one-quarter of the reports included in the study, suggesting that a reporting tool could help engage patients as safety partners [8]. However, in order for patients to fully embrace the potential of using EHRs, and particularly to identify and correct potential discrepancies, EHR platforms must support technology acceptance and usability, thereby enhancing a positive user experience and sustained adoption. Previous studies have identified a range of barriers perceived by patients, including attitude and culture challenges [9-11], privacy concerns [12,13], cost concerns [14], poor digital literacy [15], and interface difficulties [11,16]. In a systematic review by Zhao et al (2017), technical or logistical difficulties with the enrollment process (eg, difficult navigation, lack of information technology support) prevented interested patients from completing registration in 15 studies exploring the use of patient portals [15].

Further investigation and understanding of the extent to which patients can identify errors in the EHR will realize current barriers as well as their impact on patients and the safety of their care. The relatively easier access to EHRs compared with paper records might better support patient-initiated correction of errors, although they will likely be presented with a greater granularity of medically related information, which could impact understanding. In addition, patients will need at least some information communication technology skills to navigate an EHR successfully.

Patients’ ability to understand information presented in their EHRs is an essential component for error identification and subsequent correction. If patients cannot interpret information, any proposed advantages of EHR error identification for patient safety will be limited. This pilot study assessed patients’ perceptions of their ability to understand the information contained in their EHRs and the main barriers to their understanding. Additionally, the main types of patient-reported errors were characterized.


Study Design and Setting

A cross-sectional study using electronic survey data collected patients’ use of the Care Information Exchange (CIE). The CIE is a patient-controlled EHR portal in North-West London that allows data sharing between patients and their health care professionals [17]. It was initiated in 2014 at the Imperial College Healthcare National Health Service (NHS) Trust funded by the Imperial College Health Charity [17] and integrates data from sources across health care trusts and care settings. From an organizational perspective, a single patient account is created at the service/department that first registers a patient. However, data from all the departments relevant to each patient’s care are integrated into 1 account. The enrolment of patients in the CIE is optional. Patient information contained in the CIE includes appointment details, test results, care plans, and information on medications. If a patient’s practice has signed up, data such as allergies, medications, and diagnoses will also be visible to them. Clinic letters and discharge summaries can be shared as well. Patients may access their records whenever they wish to review information or when notified about new information such as test results being available. Opportunities and challenges of patient identification of errors within their EHRs were discussed with the National Institute for Health Research (NIHR) Imperial Patient Safety Translational Research Centre’s Research Partners Group, a group of lay partners advising researchers on issues surrounding patient and public involvement and engagement.

Participants and Sampling

All CIE users accessing their records during the data collection period were invited to participate in this study. Upon logging in, a message signposting the survey was displayed on the CIE webpage to all users during the time of the study (n=682). Patients were asked to follow the link to complete a short survey, which they could complete at a convenient time.

Data Collection

The survey was open for completion between March 1, 2017 and September 30, 2017. The web-based survey was implemented using Qualtrics (web-based survey software). It contained both structured (multiple choice) and unstructured (free text) questions (Multimedia Appendix 1). Question topics were selected based on information from the CIE standard operating protocol, which was developed from existing evidence about best practice and the necessary characteristics for EHR systems. Responses to the structured questions provided metrics of the patients’ perceived ability to understand their EHR information (ie, “Did you find any information in your record difficult to understand today?”) and identify errors (ie, “When using CIE today, did you notice any errors in your record?”). Responses to unstructured questions supplemented these metrics with qualitative data. Specifically, unstructured questions allowed respondents to expand on the reasoning for their structured responses and provide more detail about their perceptions of EHRs and identifying errors within them. This pilot survey was tested and it underwent the relevant approval process at Imperial College Healthcare NHS Trust (Reference: 675796). There was no direct contact between the survey team and the participants during data collection. Replies to individual questions were not mandated. Patients could complete the survey more than once as it was related to their current visits to the CIE platform.

Data Analysis

Data were analyzed using Microsoft Excel 2013. Qualitative data were analyzed thematically using the framework analysis method by Ritchie and Spencer [18]. The 6-stage approach was organized as (1) familiarization, (2) identification of a thematic framework, (3) indexing, (4) charting, (5) mapping, and (6) interpretation [18]. A team of 2 researchers (ALN and LF) with previous experience in qualitative research independently and systematically analyzed the data. Subsequently, both discussed their themes together to develop thematic maps for (1) the barriers to understanding EHR information and (2) the kind of errors identified by patients. Themes were supported by quotes from the patients’ free-text survey responses. Data saturation was reached. Results were not returned to participants for comments or feedback. The COREQ (Consolidated Criteria for Reporting Qualitative studies) was used to ensure the study meets the recommended standards of qualitative data reporting [19].


Participant Characteristics

The response rate was 23.5% (160/680). A total of 160 patients confirmed that it was their first time responding to the survey, and 210 survey responses were completed. The additional responses may have been made by patients responding more than once as not all respondents stated how many times they had completed the survey. The average time needed to fill in the survey was 6 minutes. The respondents represented a variety of clinical departments, including Oncology, Colposcopy, Early Intervention Services, HIV Service, Interstitial Lung Disease, Neuro-Oncology, and Renal and Rheumatology. Most responses (134/210, 63.8%) were received from services at 1 out of the 3 acute trusts.

Patients’ Perceptions on Their Ability to Understand the Information in EHRs

Participants were asked whether any information in the record was difficult to understand during their use of the system on the same day. A total of 123 of 210 responses (58.6%) indicated that they did not find it difficult to understand information in the record during their use, while 56 responses (26.7%) indicated at least some difficulties based on the barriers outlined in the section below.

Barriers to Patients Understanding Their EHR Information

When asked to specify the difficulties they had in understanding information in their EHRs, patients’ replies were classified under 2 broad themes. First, difficulties regarding the information itself and, second, the technology with which the information was relayed to the patients. The presence of difficulties with technology indicates the importance of system usability for its usefulness to patients (Table 1). Concerning the information-related barriers, 2 subthemes of “medical terminology” and “knowledge and interpretation of test results—meaning and significance,” were identified (Table 1). Knowledge of medical terminology was considered a necessary prerequisite to understanding the EHR information and therefore a barrier when lacking (Table 1). Furthermore, the interpretation of EHR information in terms of its meaning and significance could be problematic. This was evidenced by patients’ concerns regarding their inability to understand the meaning of tests, interpret their results, understand their clinical implications, and decide whether any action was required. Patients are unsure about what impact test results might have on their care and what could or should be done as a consequence (Table 1).

The difficulties in understanding information were further perpetuated by technological issues. In terms of the technology-related barriers to understanding EHR information, the EHR portal layout and accessibility was another dominant theme that emerged. Patients reported finding it difficult to find and access information in their EHRs, even when they had previously been notified by the system that their information had been updated. Finally, the presentation of test results (ie, plots instead of raw values) emerged as a theme and another barrier to patients understanding their EHR information, as it impacted what information they were visibly able to extract from the EHR.

In order to provide an overview of both information-related and system-related barriers to patients understanding their records, the themes derived from the survey (as well as the interactions between them) were used to build a thematic map (Figure 1).

Table 1. Barriers to patients understanding their electronic health record information.
Barriers, themesIllustrative quote
Information-related barriers

Medical terminologyPlain English needed against various medical terms/acronyms used. [Patient ID 1]

Knowledge and interpretation of test resultsI do not understand […] how worried I need to be when test results are out of range. [Patient ID 204]
…You obviously need reasonable medical knowledge to understand results. [Patient ID 89]
…All test results should have an information sheet attached explaining in layman\'s terms what it means and if any action is required. [Patient ID 94]
Technology-related barriers

Portal layout and accessibilityHaving received an email stating that my records have been updated, I cannot find the stated change once logged in. [Patient ID 136]
…I do not know how to access information correctly. [Patient ID 81]
…I received an email telling me that radiology data had been uploaded to my account, but I was not able to find it. It would have been easier if there had been a path finding link. [Patient ID 146]

Presentation of resultsI would prefer to see [actual] numbers not […] graphs for blood tests. [Patient ID 100]
Figure 1. Thematic map of information-related and system-related barriers to patients understanding their records.
View this figure

Types of Errors Identified by Patients Within Their EHRs

While the majority of the responses did not identify any error in the EHRs (160/210, 76.2%), 26 responses (12.4%) did indicate noticing some form of error in their records, while 24 responses showed no replies to that question. From the free-text responses, 2 main information-based themes were identified as accounting for the errors patients identified within their EHRs: (1) incorrect and (2) incomplete information (Figure 2). Incorrect information included the subthemes of contact information, appointment details, and results and measurements. Moreover, errors in results and measurements were further divided into the classification of results and measurements and entirely false results and measurements. A detailed description of the errors identified, evidenced by participants’ quotes, is provided in Table 2. The subthemes of incomplete information included information on appointment details and results. Patients commented that not all their appointments or results were visible on the system. While most patients appeared to have at least some information with respect to their appointments on CIE, for some, no information at all regarding their results were accessible, thereby limiting the CIE usefulness for the patient.

Figure 2. Types of errors identified by patients within their electronic health records.
View this figure
Table 2. Types of errors identified by patients within their electronic health records.
ThemesIllustrative quotes
Contact information (incorrect information)Incorrect address. [Patient ID 147 and Patient ID 163]
Appointment details (incorrect/incomplete information)Not all my appointments are accurate. [Patient ID 180]
…My appointment admission and discharge times are not accurate [Patient ID 140]
…Not all my appointments are recorded so I can\'t rely on this so I have to keep a separate record myself. [Patient ID 180]
Results and measurements (incorrect/incomplete information)An x-ray scan […] was recorded as a CT scan. [Patient ID 186]
…I had an HRCT scan which had been recorded as an X-ray. [Patient ID 3]
…No information shown on blood test results or anything other than hospital appointments. [Patient ID 144]
…Blood glucose of another patient maybe with the same name seems to be there. [Patient ID 199]

Summary of the Key Findings

The majority of the survey responses (123/210, 58.6%) reported no problems in understanding EHR information. However, for those reports that did, several difficulties seemed to exist that created barriers to the understanding. These were identified as relating to the information presented (terminology and significance) or to technology-based or system-based issues (layout and result presentation). Furthermore, patients recognized issues of incorrect or incomplete information in a range of aspects, including contact information, appointment details, and results and measurements.

Findings as Compared With Previous Studies

In a high percentage of responses, patients were able to understand their EHR information. These results are consistent with the findings of a previous observational study assessing patients’ preferences and perspectives in accessing their records, in which 70.8% of the participants found their records easy to understand [20]. In previous studies, patients reported that they understand notes and that reading notes helps them remember next action points such as tests and referrals [8,21], enables timely follow-up of results, supports family or friend care partners with information [8,21-23], and creates a new mechanism for patients to identify documentation errors [24,25].

The likelihood of patients using EHRs and understanding the information is linked to their digital health literacy, defined as “the ability to search, locate, understand, and use health information through electronic resources and use this knowledge to resolve health-related problems” [26]. Digital health literacy can contribute to more informed decision making and potentially improve health outcomes [27]. Previous studies found that patients with higher digital health literacy levels have a higher likelihood of being a portal user [28-31]. Holt et al (2019) actually suggest that information about patients’ health literacy may provide a better understanding of patients’ reasons for not using digital health services, rather than the sociodemographic data [31]. However, similar to other digital health technologies, providing patients with access to correct errors can also increase health inequities, that is, widen the “digital divide” [32]. While patients with higher level of education and better health literacy may want to get more involved in their health care decision making, patients who are less educated may feel that they will not understand the information or may also feel that their doctors know what is best and be less inclined to get involved [33]. Therefore, realizing the potential of patients correcting errors in their EHRs will require broad outreach, engagement, and training for a diverse group of patients of various ages, races/ethnicities, and educational and health literacy levels [34].

As previously found in other studies [35-38], one of the main barriers to understanding medical information was medical terminology and jargon. Acronyms can vary greatly between individuals and specialties, and their understanding can be confusing for clinicians and even more for patients [20]. Strategies to improve readability include efforts to keep usage of terminology consistent [20], minimization of the use of jargon in clinical notes, as well as use of dictionaries as supporting documents in medical records. Other strategies include providing adequate support and training for patients and activities aiming to improve health literacy, that is, patients’ knowledge and ability necessary to understand and act upon health information [39].

As in this study, technology-related issues have previously been identified as inhibiting patients’ use of health records [40]. The navigation of the system as well as the ease of use have been previously criticized by patients in regard to their EHRs [40]. These findings are consistent with the results of this study showing that patients’ ability to use their EHRs effectively is related to the subthemes “user-friendliness” and “website layout.” The technology-related barriers are likely to further perpetuate the identified information-related barriers as the way medical information is presented is known to influence patients’ ability to interpret its significance and the actions needed as a result [40,41]. There is an opportunity for EHRs to support understanding of health information through user-centered designs in their presentations and provision of linked and easily accessible additional information, to support interpretation of the information shared and to provide guidance for patients on how to use this information for their self-care.

Patients and families have unique knowledge about themselves and their own health care, and their reports have potential for improving both individual and organizational safety [34]. The types of errors identified in our study and in those by Mossaed et al in 2015 and Bell et al in 2020 were similar, including missing test results, medications, and wrong date of birth [20,34]. In our study, the potential severity and impact of the different error types identified varied. While a misclassification of a result (eg, “An x-ray scan […] was recorded as a CT scan”) may be inconvenient for finding it on the CIE, this does not necessarily directly pose a threat to patient safety. However, test results from another patient may directly affect future care provision by providing false data to health care professionals and posing safety risks. Hence, as suggested by Bell et al, patients engaging with their EHRs to monitor its contents and flag up potential inconsistencies could enhance quality of care further down the line and offer an opportunity for patients to engage with their health care [7].

In our study, 12.4% (26/210) of the patients identified errors in their medical records. This value is slightly higher than that observed in similar studies published previously, where 7.7% of the patients reported finding errors in their records [20]. In a recent study by Bell et al in 2020, more than 1 in 5 patients perceived mistakes in their notes, with older patients and those with poorer health twice as likely to identify serious errors, suggesting that note sharing may have particularly important safety implications for those groups [34]. As no information on the actual error rate in the patient-controlled EHR was accessible because of patient privacy reasons, the interpretation of the patient-reported error detection percentage could not be explored further.

It is important to note that some patient-reported errors may refer to disagreements between providers and patients and may not necessarily be errors [34]. However, previous evidence suggests that the errors described by patients as very serious usually appeared to have relevant clinical implications [8,34,42], and therefore, patients perceived and reported that error rates remain central to partnering with patients toward the successful and sustainable patient engagement with their EHRs. Future investigations of errors or inconsistencies as identified by patients in contrast to health care professionals may allow for further estimation of the impact of patient EHR review on safety and quality of care.

Consistent with UK policy guidance emphasizing the need to share health information with patients [4,43], a recent initiative has focused on outpatient clinic letters and the style and language used within them to be better “directed” at patients and thereby written in appropriate language [44]. This initiative highlights the political drive to overcome some of the information-related and knowledge-related barriers to understanding information. These approaches may also help mitigate some barriers inhibiting error identification by patients, thus enhancing the effectiveness of future patient-initiated error identification. In addition, receiving clinic letters almost immediately gives patients the means to keep a convenient record of the letters and to ensure that actions advised in these letters are followed through.

Strengths, Limitations, and Future Work

There are several strengths in this study. The triangulation of interpretations among researchers with expertise in qualitative research, clinical research, and cognitive science resulted in a depth of knowledge and inclusion of different perspectives in this study. Another strength of this study was the diversity in the participant sample, including participants from a range of departments. To ensure the quality of this work, qualitative data were handled with reference to the COREQ checklist, according to best practice recommendations [19]. In order to keep the length of the survey short, encourage responses, and minimize the risk of patient reidentification, the information collected on participant demographics, characteristics, and context of their CIE access was limited. A few demographic factors have been previously identified as playing a role in a patient’s ability to understand the information contained in their EHRs, including female gender, younger age (<60 years), and having a higher level of education [20]. Future work will need to investigate how a wider range of demographic, contextual, and social factors, as well as patient activation and health literacy influence patients’ perspectives and abilities to identify errors in their EHRs, thereby allowing planning and implementation of quality improvement work to support its posited benefits without causing any disadvantage to any patient group.

Respondents could complete the survey more than once, thereby limiting the aspects of the data interpretation. However, exclusion of responses would have significantly reduced the scope of the data, and we wanted to be sure to capture and use all respondents’ feedback, especially when they took the time to provide feedback multiple times. Equally, over time, an individual's feedback could be perceived to become more insightful as their experience with using the patient portal increased. As no information on the scope of errors existing in the questioned patients’ records was available, interpretation of the error detection percentage is limited. Future investigation of errors or inconsistencies as identified by patients in contrast to health care professionals may allow for further estimation of the impact of patient EHR review on safety and quality of care. Additionally, it would be important for future research to examine associations between patient-reported errors and safety outcomes [34]. Finally, future work should include methodologically robust quantitative studies focusing on quantifying how different factors influence both the public willingness and ability to correct errors in their EHRs.

Conclusions

EHR systems have the potential to support patient engagement in health care by providing personal health information to patients. While the majority of the patients reportedly understand the information contained within their EHRs, technology and information-based barriers persist. Future implementation of such systems must consider supporting patients in the interpretation of the information presented, in what concerns both terminology and significance, and partner with a diverse group of patients to co-design solutions with appropriate usability. Additionally, organizations will need to develop systematic mechanisms for triaging and responding to patient-reported errors, particularly as EHR transparency increases. At a moment when public demand for data is growing, along with a greater awareness of health care data ownership, these barriers must be addressed and their solutions incorporated in health information systems that support patients and their care providers to together improve patient safety.

Acknowledgments

The research was supported by the NIHR Imperial Patient Safety Translational Research Centre and the NIHR Imperial Biomedical Research Centre. LF was funded by the Sowerby eHealth Forum. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR, or the Department of Health and Social Care.

Conflicts of Interest

None declared.

Multimedia Appendix 1

Patient survey.

PDF File (Adobe PDF File), 105 KB

References

  1. Fitton C, Fitton R, Hannan A, Fisher B, Morgan L, Halsall D. The impact of patient record access on appointments and telephone calls in two English general practices: a population-based study. London J Prim Care (Abingdon) 2014;6(1):8-15 [FREE Full text] [CrossRef] [Medline]
  2. Pagliari C, Detmer D, Singleton P. Potential of electronic personal health records. BMJ 2007 Aug 18;335(7615):330-333 [FREE Full text] [CrossRef] [Medline]
  3. Ross SE, Lin C. The effects of promoting patient access to medical records: a review. J Am Med Inform Assoc 2003;10(2):129-138 [FREE Full text] [CrossRef] [Medline]
  4. Department of Health UK. Equity and excellence: Liberating the NHS. 2010.   URL: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/213818/dh_118610.pdf [accessed 2021-02-05]
  5. Mold F, de Lusignan S, Sheikh A, Majeed A, Wyatt JC, Quinn T, et al. Patients' online access to their electronic health records and linked online services: a systematic review in primary care. Br J Gen Pract 2015 Mar;65(632):e141-e151 [FREE Full text] [CrossRef] [Medline]
  6. Neves AL, Carter AW, Freise L, Laranjo L, Darzi A, Mayer EK. Impact of sharing electronic health records with patients on the quality and safety of care: a systematic review and narrative synthesis protocol. BMJ Open 2018 Aug 13;8(8):e020387 [FREE Full text] [CrossRef] [Medline]
  7. Bell SK, Mejilla R, Anselmo M, Darer JD, Elmore JG, Leveille S, et al. When doctors share visit notes with patients: a study of patient and doctor perceptions of documentation errors, safety opportunities and the patient-doctor relationship. BMJ Qual Saf 2017 Apr;26(4):262-270 [FREE Full text] [CrossRef] [Medline]
  8. Bell SK, Gerard M, Fossa A, Delbanco T, Folcarelli PH, Sands KE, et al. A patient feedback reporting tool for OpenNotes: implications for patient-clinician safety and quality partnerships. BMJ Qual Saf 2017 Apr;26(4):312-322. [CrossRef] [Medline]
  9. Fix GM, Hogan TP, Amante DJ, McInnes DK, Nazi KM, Simon SR. Encouraging Patient Portal Use in the Patient-Centered Medical Home: Three Stakeholder Perspectives. J Med Internet Res 2016 Nov 22;18(11):e308 [FREE Full text] [CrossRef] [Medline]
  10. Lyles CR, Allen JY, Poole D, Tieu L, Kanter MH, Garrido T. "I Want to Keep the Personal Relationship With My Doctor": Understanding Barriers to Portal Use among African Americans and Latinos. J Med Internet Res 2016 Oct 03;18(10):e263 [FREE Full text] [CrossRef] [Medline]
  11. Tieu L, Sarkar U, Schillinger D, Ralston JD, Ratanawongsa N, Pasick R, et al. Barriers and Facilitators to Online Portal Use Among Patients and Caregivers in a Safety Net Health Care System: A Qualitative Study. J Med Internet Res 2015 Dec 03;17(12):e275 [FREE Full text] [CrossRef] [Medline]
  12. Haun JN, Lind JD, Shimada SL, Martin TL, Gosline RM, Antinori N, et al. Evaluating user experiences of the secure messaging tool on the Veterans Affairs' patient portal system. J Med Internet Res 2014 Mar 06;16(3):e75 [FREE Full text] [CrossRef] [Medline]
  13. Schickedanz A, Huang D, Lopez A, Cheung E, Lyles CR, Bodenheimer T, et al. Access, interest, and attitudes toward electronic communication for health care among patients in the medical safety net. J Gen Intern Med 2013 Jul;28(7):914-920 [FREE Full text] [CrossRef] [Medline]
  14. Yamin CK, Emani S, Williams DH, Lipsitz SR, Karson AS, Wald JS, et al. The digital divide in adoption and use of a personal health record. Arch Intern Med 2011 Mar 28;171(6):568-574. [CrossRef] [Medline]
  15. Zhao JY, Song B, Anand E, Schwartz D, Panesar M, Jackson GP, et al. Barriers, Facilitators, and Solutions to Optimal Patient Portal and Personal Health Record Use: A Systematic Review of the Literature. AMIA Annu Symp Proc 2017;2017:1913-1922 [FREE Full text] [Medline]
  16. Sox CM, Gribbons WM, Loring BA, Mandl KD, Batista R, Porter SC. Patient-centered design of an information management module for a personally controlled health record. J Med Internet Res 2010 Aug 30;12(3):e36 [FREE Full text] [CrossRef] [Medline]
  17. Poovendran D, Wadge H, Roy R, Freise L, Neves A, Fugard A, et al. A qualitative evaluation of the CIE Programme implementation in North West London.   URL: http:/​/www.​imperial.ac.uk/​media/​imperial-college/​institute-of-global-health-innovation/​FINAL-REPORT-2-OCTOBER-FINAL.​pdf [accessed 2020-01-01]
  18. Ritchie J, Spencer L. Qualitative data analysis for applied policy research. Anal Qual Data. London: Routledge; 1994.
  19. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care 2007 Dec;19(6):349-357. [CrossRef] [Medline]
  20. Mossaed S, Leonard K, Eysenbach G. Patient Preferences and Perspectives on Accessing Their Medical Records. J Med Imaging Radiat Sci 2015 Jun;46(2):205-214. [CrossRef] [Medline]
  21. Bell SK, Folcarelli P, Fossa A, Gerard M, Harper M, Leveille S, et al. Tackling Ambulatory Safety Risks Through Patient Engagement: What 10,000 Patients and Families Say About Safety-Related Knowledge, Behaviors, and Attitudes After Reading Visit Notes. J Patient Saf 2018 Apr 27. [CrossRef] [Medline]
  22. Chimowitz H, Gerard M, Fossa A, Bourgeois F, Bell SK. Empowering Informal Caregivers with Health Information: OpenNotes as a Safety Strategy. Jt Comm J Qual Patient Saf 2018 Mar;44(3):130-136. [CrossRef] [Medline]
  23. Wolff JL, Berger A, Clarke D, Green JA, Stametz R, Yule C, et al. Patients, care partners, and shared access to the patient portal: online practices at an integrated health system. J Am Med Inform Assoc 2016 Nov;23(6):1150-1158. [CrossRef] [Medline]
  24. Bell SK, Folcarelli PH, Anselmo MK, Crotty BH, Flier LA, Walker J. Connecting Patients and Clinicians: The Anticipated Effects of Open Notes on Patient Safety and Quality of Care. Jt Comm J Qual Patient Saf 2015 Aug;41(8):378-384. [CrossRef] [Medline]
  25. Herlihy M, Harcourt K, Fossa A, Folcarelli P, Golen T, Bell SK. An Opportunity to Engage Obstetrics and Gynecology Patients Through Shared Visit Notes. Obstet Gynecol 2019 Jul;134(1):128-137. [CrossRef] [Medline]
  26. Norman CD, Skinner HA. eHealth Literacy: Essential Skills for Consumer Health in a Networked World. J Med Internet Res 2006 Jun 16;8(2):e9 [FREE Full text] [CrossRef] [Medline]
  27. Wu AD, Begoray DL, Macdonald M, Wharf Higgins J, Frankish J, Kwan B, et al. Developing and evaluating a relevant and feasible instrument for measuring health literacy of Canadian high school students. Health Promot Int 2010 Dec;25(4):444-452. [CrossRef] [Medline]
  28. Noblin AM, Wan TTH, Fottler M. The impact of health literacy on a patient's decision to adopt a personal health record. Perspect Health Inf Manag 2012;9:1-13 [FREE Full text] [Medline]
  29. Irizarry T, DeVito Dabbs A, Curran CR. Patient Portals and Patient Engagement: A State of the Science Review. J Med Internet Res 2015 Jun 23;17(6):e148 [FREE Full text] [CrossRef] [Medline]
  30. Davis SE, Osborn CY, Kripalani S, Goggins KM, Jackson GP. Health Literacy, Education Levels, and Patient Portal Usage During Hospitalizations. AMIA Annu Symp Proc 2015;2015:1871-1880 [FREE Full text] [Medline]
  31. Holt KA, Karnoe A, Overgaard D, Nielsen SE, Kayser L, Røder ME, et al. Differences in the Level of Electronic Health Literacy Between Users and Nonusers of Digital Health Services: An Exploratory Survey of a Group of Medical Outpatients. Interact J Med Res 2019 Apr 05;8(2):e8423 [FREE Full text] [CrossRef] [Medline]
  32. Makri A. Bridging the digital divide in health care. The Lancet Digital Health 2019 Sep;1(5):e204-e205 [FREE Full text] [CrossRef]
  33. Levinson W, Kao A, Kuby A, Thisted RA. Not all patients want to participate in decision making. A national study of public preferences. J Gen Intern Med 2005 Jun;20(6):531-535 [FREE Full text] [CrossRef] [Medline]
  34. Bell SK, Delbanco T, Elmore JG, Fitzgerald PS, Fossa A, Harcourt K, et al. Frequency and Types of Patient-Reported Errors in Electronic Health Record Ambulatory Care Notes. JAMA Netw Open 2020 Jun 01;3(6):e205867 [FREE Full text] [CrossRef] [Medline]
  35. Earnest MA, Ross SE, Wittevrongel L, Moore LA, Lin C. Use of a patient-accessible electronic medical record in a practice for congestive heart failure: patient and physician experiences. J Am Med Inform Assoc 2004;11(5):410-417 [FREE Full text] [CrossRef] [Medline]
  36. Hassol A, Walker JM, Kidder D, Rokita K, Young D, Pierdon S, et al. Patient experiences and attitudes about access to a patient electronic health care record and linked web messaging. J Am Med Inform Assoc 2004;11(6):505-513 [FREE Full text] [CrossRef] [Medline]
  37. Pyper C, Amery J, Watson M, Crook C. Patients' experiences when accessing their on-line electronic patient records in primary care. Br J Gen Pract 2004 Jan;54(498):38-43 [FREE Full text] [Medline]
  38. Ross SE, Moore LA, Earnest MA, Wittevrongel L, Lin C. Providing a web-based online medical record with electronic communication capabilities to patients with congestive heart failure: randomized trial. J Med Internet Res 2004 May 14;6(2):e12 [FREE Full text] [CrossRef] [Medline]
  39. Sørensen K, Van den Broucke S, Fullam J, Doyle G, Pelikan J, Slonska Z, (HLS-EU) Consortium Health Literacy Project European. Health literacy and public health: a systematic review and integration of definitions and models. BMC Public Health 2012 Jan 25;12:80 [FREE Full text] [CrossRef] [Medline]
  40. Fuji KT, Abbott AA, Galt KA. Personal health record design: qualitative exploration of issues inhibiting optimal use. Diabetes Care 2014;37(1):e13-e14 [FREE Full text] [CrossRef] [Medline]
  41. Fraccaro P, Vigo M, Balatsoukas P, van der Veer SN, Hassan L, Williams R, et al. Presentation of laboratory test results in patient portals: influence of interface design on risk interpretation and visual search behaviour. BMC Med Inform Decis Mak 2018 Feb 12;18(1):11 [FREE Full text] [CrossRef] [Medline]
  42. Khan A, Coffey M, Litterer KP, Baird JD, Furtak SL, Garcia BM, the PatientFamily Centered I-PASS Study Group, et al. Families as Partners in Hospital Error and Adverse Event Surveillance. JAMA Pediatr 2017 Apr 01;171(4):372-381 [FREE Full text] [CrossRef] [Medline]
  43. Department of Health. Liberating the NHS: No decision about me without me - Government response. 2012.   URL: https:/​/www.​gov.uk/​government/​uploads/​system/​uploads/​attachment_data/​file/​216980/​Liberating-the-NHS-No-decision-about-me-without-me-Government-response.​pdf [accessed 2021-02-05]
  44. Academy of Medical Royal Colleges. Writing outpatient clinic letters to patients. 2018.   URL: http://www.aomrc.org.uk/wp-content/uploads/2018/09/Please_write_to_me_Guidance_010918.pdf [accessed 2021-02-05]


CIE: Care Information Exchange
COREQ: Consolidated Criteria For Reporting Qualitative Studies
EHR: electronic health record
NHS: National Health Service
NIHR: National Institute for Health Research


Edited by G Eysenbach; submitted 02.04.20; peer-reviewed by M Smith, L Kayser, R Williams; comments to author 24.09.20; revised version received 01.12.20; accepted 21.01.21; published 26.02.21

Copyright

©Lisa Freise, Ana Luisa Neves, Kelsey Flott, Paul Harrison, John Kelly, Ara Darzi, Erik K Mayer. Originally published in JMIR Formative Research (http://formative.jmir.org), 26.02.2021.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on http://formative.jmir.org, as well as this copyright and license information must be included.