Background: A potential benefit of electronic health records (EHRs) is that they could potentially save clinician time and improve documentation by auto-generating the history of present illness (HPI) in partnership with patients prior to the clinic visit. We developed an online patient portal called AEGIS (Automated Evaluation of Gastrointestinal [GI] Symptoms) that systematically collects patient GI symptom information and then transforms the data into a narrative HPI that is available for physicians to review in the EHR prior to seeing the patient.
Objective: This study aimed to compare whether use of an online GI symptom history taker called AEGIS improves physician-centric outcomes vs usual care.
Methods: We conducted a pragmatic controlled trial among adults aged ≥18 years scheduled for a new patient visit at 4 GI clinics at an academic medical center. Patients who completed AEGIS were matched with controls in the intervention period who did not complete AEGIS as well as controls who underwent usual care in the pre-intervention period. Of note, the pre-intervention control group was formed as it was not subject to contamination bias, unlike for post-intervention controls. We then compared the following outcomes among groups: (1) documentation of alarm symptoms, (2) documentation of family history of GI malignancy, (3) number of follow-up visits in a 6-month period, (4) number of tests ordered in a 6-month period, and (5) charting time (difference between appointment time and time the encounter was closed). Multivariable regression models were used to adjust for potential confounding.
Results: Of the 774 patients who were invited to complete AEGIS, 116 (15.0%) finished it prior to their visit. The 116 AEGIS patients were then matched with 343 and 102 controls in the pre- and post-intervention periods, respectively. There were no statistically significant differences among the groups for documentation of alarm symptoms and GI cancer family history, number of follow-up visits and ordered tests, or charting time (all P>.05).
Conclusions: Use of a validated online HPI-generation portal did not improve physician documentation or reduce workload. Given universal adoption of EHRs, further research examining how to optimally leverage patient portals for improving outcomes are needed.
To facilitate communication between patients and physicians in electronic health record (EHR)–integrated environments, we developed an online patient portal (MyGiHealth) that uses a computer algorithm called Automated Evaluation of Gastrointestinal (GI) Symptoms (AEGIS) to systematically collect patients’ symptom information before the clinic visit. Once collected, the data are transformed into a full narrative history of present illness (HPI) that clinicians can review prior to meeting the patient. While our prior studies noted that AEGIS creates higher quality HPIs and collects more alarm features vs physicians [, ], we found that it did not improve patient satisfaction or shared decision making when compared to usual care in a controlled trial [ ]. Therefore, the objective of this study was to investigate whether AEGIS improved physician-centric outcomes vs usual care.
We performed a pragmatic controlled study among adults aged ≥18 years scheduled for a new patient visit at an academic GI teaching practice and 3 community-based GI clinics at Cedars-Sinai Medical Center. This study was approved by the Cedars-Sinai Institutional Review Board, Los Angeles, CA (Pro45243).
During the intervention period (April 17, 2017-February 7, 2018), patients were invited via email to complete AEGIS via the MyGiHealth app 1 week prior to their visit. We describe AEGIS elsewhere [- ], but in brief, the algorithm systematically assesses patients’ GI symptoms and then transforms the data into a full narrative HPI as shown in . For patients who completed AEGIS, their physicians were notified 1 day before the visit that their HPIs were uploaded to the notes section of our EHR (Epic, Verona, WI) for review (see for the email notification that was sent to physicians by research study staff). To identify individuals for the 2 control groups who were comparable to those in the intervention arm, each patient who completed AEGIS was matched (age ±3 years, sex, race/ethnicity, clinic) with up to 4 patients in the pre-intervention period (October 6, 2015-April 6, 2017) and 1 patient in the intervention period who did not complete AEGIS. Of note, the pre-intervention control group was formed as it was not subject to contamination bias, unlike for post-intervention controls (ie, after physicians reviewed AEGIS reports for those in the intervention arm, they might have been more apt to take and document more thorough HPIs for their control patients). Moreover, age, sex, race/ethnicity, and clinic were selected as matching variables as they were readily available for automated extraction from the EHR and we hypothesized at the outset that they may have correlated with our outcomes.
Once recruitment ceased, 5 resident physicians reviewed patient charts and collected outcomes data using a REDCap data abstraction sheet : (1) documentation of alarm symptom(s) in initial note (hematochezia, melena, hematemesis, unintentional weight loss, fevers); (2) documentation of family history of GI malignancy in initial note (colorectal, esophagus, gallbladder, liver, stomach, pancreas, or throat cancer); (3) charting time (time initial encounter closed minus the appointment time); (4) number of follow-up visits in a 6-month period; (5) number of lab, endoscopy, and imaging tests ordered in a 6-month period.
Statistical analyses were performed using Stata 13.1 (StataCorp LP, College Station, TX). A two-tailed P<.05 was considered significant in all analyses. For bivariate analyses, we used analysis of variance and chi-squared tests to compare continuous and categorical data, respectively. We also conducted multivariable regression analyses on our outcomes and included group assignment, patient age, sex, race/ethnicity, and clinic site as covariates when appropriate. Specifically, logistic regression analyses were performed on the AEGIS completion and documentation of alarm symptoms and GI cancer family history outcomes. We used linear regression analysis to compare charting time between the intervention and control groups. Lastly, numbers of follow-up visits and tests ordered within 6 months of the initial visit were compared using zero-inflated negative binomial and negative binomial regression models, respectively.
Of the 774 patients invited to complete AEGIS (shows demographics), 116 (15.0%) completed it before their clinic visit. shows results from the regression analysis on completion of the app; older individuals and Asians were less likely to complete AEGIS. No significant associations were seen between app completion and the remaining racial/ethnic groups, sex, and clinic. Among those who completed AEGIS, the consultants’ initial notes for 47 (40.5%) of the 116 patients contained at least a portion of the computer-generated report.
|Variable||Completed AEGIS (n=116)||ORa (95% CI)b|
|Age (years), mean (SD)||49.9 (16.1)||0.985 (0.972-0.998)|
|Sex, n (%)|
|Female||71 (15.1)||0.99 (0.66-1.51)|
|Race/ethnicity, n (%)|
|Non-Hispanic white||83 (17.9)||Reference|
|Non-Hispanic black||11 (11.0)||0.57 (0.29-1.13)|
|Latino||10 (13.7)||0.66 (0.32-1.36)|
|Non-Hispanic Asian||5 (7.3)||0.33 (0.13-0.85)|
|Other/unknown||7 (10.5)||0.48 (0.21-1.11)|
|Clinic, n (%)|
|Resident/fellow GIc clinic||19 (17.4)||Reference|
|Physician A||37 (18.2)||1.08 (0.58-2.02)|
|Physician B||29 (12.7)||0.68 (0.36-1.30)|
|Physician C||31 (13.3)||0.74 (0.38-1.42)|
aOR: odds ratio.
bThe logistic regression model adjusted for all covariates in the table.
Patients who completed AEGIS (n=116) were matched with 343 patients from the pre-intervention period and 102 from the intervention period who did not complete AEGIS.lists their demographics; the groups were largely similar in age, sex, race/ethnicity, clinic, and reason for consult.
|Variable||Control group: pre-AEGISa period (n=343)||Control group: did not complete AEGIS (n=102)||Intervention group: completed AEGIS (n=116)||Pb|
|Age (years), mean (SD)||51.4 (16.1)||53.7 (16.2)||49.9 (16.1)||.21|
|Sex, n (%)||.51|
|Male||136 (39.7)||34 (33.3)||45 (38.8)|
|Female||207 (60.4)||68 (66.7)||71 (61.2)|
|Race/ethnicity, n (%)||.25|
|Non-Hispanic white||264 (77.0)||86 (84.3)||83 (71.6)|
|Non-Hispanic black||35 (10.2)||7 (6.9)||11 (9.5)|
|Latino||23 (6.7)||2 (2.0)||10 (8.6)|
|Non-Hispanic Asian||10 (2.9)||5 (4.9)||5 (4.3)|
|Other/unknown||11 (3.2)||2 (2.0)||7 (6.0)|
|Clinic, n (%)||.67|
|Resident/fellow GIc clinic||43 (12.5)||9 (8.8)||19 (16.4)|
|Physician A||112 (32.7)||30 (29.4)||37 (31.9)|
|Physician B||90 (26.2)||28 (27.5)||29 (25.0)|
|Physician C||98 (28.6)||35 (34.3)||31 (26.7)|
|Reason for consult, n (%)|
|Abdominal pain||77 (22.5)||27 (26.5)||28 (24.1)||.69|
|Anemia evaluation||2 (0.6)||1 (1.0)||0 (0)||.60|
|Bloating||30 (8.8)||17 (16.7)||21 (18.1)||.008|
|Blood in stool||18 (5.3)||2 (2.0)||5 (4.3)||.37|
|Bowel incontinence||2 (0.6)||1 (1.0)||1 (0.9)||.90|
|Colorectal cancer screening||101 (29.5)||25 (24.5)||34 (29.3)||.61|
|Constipation||48 (14.0)||13 (12.8)||19 (16.4)||.73|
|Diarrhea||42 (12.2)||10 (9.8)||17 (14.7)||.55|
|Dysphagia||15 (4.4)||1 (1.0)||4 (3.5)||.27|
|Gastroesophageal reflux disease||46 (13.4)||15 (14.7)||33 (28.5)||.001|
|Inflammatory bowel disease||19 (5.5)||4 (3.9)||6 (5.2)||.81|
|Liver disease||2 (0.6)||1 (1.0)||1 (0.9)||.90|
|Nausea/vomiting||25 (7.3)||9 (8.8)||8 (6.9)||.84|
|Rectal pain||2 (0.6)||0 (0)||2 (1.7)||.29|
|Other||47 (13.7)||19 (18.6)||16 (13.8)||.45|
aAEGIS: Automated Evaluation of Gastrointestinal Symptoms.
bP value from analysis of variance test (continuous data) or chi-squared test (categorical data).
In, we present the physician-centric outcomes stratified by group. No differences were seen for documentation of alarm symptoms and GI cancer family history, EHR charting time, or numbers of follow-up visits and ordered tests.
|Variable||Control group: pre-AEGISa period (n=343; reference)||Control group: did not complete AEGIS (n=102)||Adjusted P||Intervention group: completed AEGIS (n=116)||Adjusted P|
|Documentation of an alarm symptom in initial noteb, n (%)||61 (17.8)||22 (21.6)||.18||18 (15.5)||.39|
|Documentation of GIc cancer family history in initial noteb, n (%)||64 (18.7)||20 (19.6)||.86||27 (23.3)||.28|
|Charting time, which is the time until initial EHRd chart encounter was closede (hours), median (IQR)||3.1 (1.4-9.2)||3.3 (1.0-12.7)||.34||3.7 (1.1-10.0)||.58|
|Number of follow-up visits within the 6-month periodf, median (IQR)||0 (0-1)||0 (0-0)||.22||0 (0-1)||.11|
|Number of tests ordered within the 6-month periodg, median (IQR)||1 (1-3)||1 (0-2)||.21||1 (0-3)||.85|
aAEGIS: Automated Evaluation of Gastrointestinal Symptoms.
bLogistic regression model adjusted for group assignment, patient age, sex, race/ethnicity, and clinic.
dEHR: electronic health record.
eLinear regression model adjusted for group assignment, patient age, sex, race/ethnicity, and clinic. Patients seen in the resident or fellow GI clinic (n=71) were excluded from this analysis as trainees first needed to complete their note before attendings could review or edit the note and close the encounter. Patients of Physicians A-C who were seen earlier or later than their originally scheduled appointment time (n=92) were also excluded from this analysis.
fZero-inflated negative binomial regression model adjusted for group assignment, patient age, sex, race/ethnicity, and clinic.
gNegative binomial regression model adjusted for group assignment, patient age, sex, race/ethnicity, and clinic.
We discovered that uptake of AEGIS was low, as only 15% of patients accessed the online portal. Surprisingly, this rate was lower than that seen in our prior AEGIS trial (37%) focused on patient-centric outcomes . This was even despite our use of email invitations (the original study used mailings), which we initially hypothesized would increase uptake as the email included a direct AEGIS hyperlink. Of note, research staff emailed invitations directly to patients; it is possible that sending invitations through the EHR patient portal may have enhanced uptake. Prior literature illustrates that patients are accepting of EHR portals [ - ] and they may be more willing to complete interventions sent through official health system platforms. Further research examining optimal methods for deploying digital interventions in EHR-integrated environments are needed.
While AEGIS was built to enhance patient-physician communication by systematically collecting salient components of the history, one-time use of the app did not increase documentation of alarm symptoms or family history of GI malignancy in the initial note. This suggests that physicians in our study may adequately screen for and document relevant red flags and family history. Alternatively, given our pragmatic design, clinicians may not have reviewed the AEGIS report or incorporated it into their note for some patients. We also noted that the app and its computer-generated HPIs did not impact health care utilization or charting time. In short, we did not find that leveraging an online HPI-generation portal measurably improved physician-centric outcomes in this study.
Of note, we previously found that AEGIS collected more alarm features when compared to physicians . The discordant results likely relate to the different study designs; in the prior observational study, AEGIS was completed by patients after their clinic visit, rather than before the visit as in our current study. It is possible that first consulting with the physician subsequently prompted patients, after further introspection, to report more alarm features through AEGIS than were discussed and documented by the physician in clinic. We also previously found that AEGIS creates higher quality HPIs versus those written by doctors [ ]. However, based on our findings here and in a prior multicenter controlled trial focused on patient-centered outcomes [ ], simply making the comprehensive AEGIS HPIs available for review in the EHR is insufficient for improving care. Further research is needed to determine how best to optimally implement and use these computer-generated data in clinical workflows in order to enhance outcomes.
A limitation of our study was that we could not fully assess whether and how closely physicians reviewed the AEGIS reports in the EHR. While 40.5% of physicians’ notes for the intervention patients contained a portion of the AEGIS report, we do not know how rigorously they reviewed the report after copying it into their notes. On the other hand, for the remaining patients, it is possible that clinicians thoroughly read the AEGIS report in the EHR but chose to not copy and paste it into their official consultant notes. Development of novel, effective methods for alerting and assessing how clinicians use newly uploaded, app-generated data in the EHR and that maximize its use at the point of care are urgently needed. Another limitation was that AEGIS was administered as a one-time intervention; longitudinal use of the app for tracking symptom severity could have impacted outcomes such as numbers of follow-up visits and ordered tests. Notably, longitudinal symptom monitoring via a portal decreased emergency room visits and improved survival among patients with metastatic cancer [, ].
In summary, we found that uptake of AEGIS was low, as less than 1 in 6 patients completed it before their visit. Moreover, one-time use of a carefully developed and validated patient-provider portal did not improve documentation of key elements of the note nor reduce clinician work burden. This is disappointing as it is well known that the EHR has greatly increased physician charting time [, ]; our goal has been to identify ways to reduce physician burden through clinically meaningful, EHR-enabled automation. Yet, even in taking care to maximize the benefits of the EHR to support physician-centric outcomes, we were unable to demonstrate a benefit. Given the near universal adoption of EHRs [ - ], further research examining how best to develop and implement digital interventions in EHR-integrated environments for improving both patient and physician outcomes is critical.
This study was funded by a Junior Faculty Development Grant from the American College of Gastroenterology awarded to CVA. CVA and BMRS are supported by a CTSI grant from the NIH/NCATS UL1TR001881-01. CVA is also supported by a loan repayment award from the NIH/NIDDK L30 DK106734. Support for the MyGiHealth portal that administers AEGIS was obtained from Ironwood Pharmaceuticals. The Cedars-Sinai Center for Outcomes Research and Education (CS-CORE) is supported by The Marc and Sheri Rapaport Fund for Digital Health Sciences & Precision Health.
Conflicts of Interest
BMRS has served on advisory boards and received grant support from Ironwood Pharmaceuticals. CVA has a stock option grant in My Total Health. The remaining authors do not have any relevant disclosures.
Supplementary Figure 1. Sample email sent to physicians the day before their clinic notifying them of the patients that completed AEGIS.PNG File , 171 KB
Supplementary Table 1. Demographics of the cohort invited to complete AEGIS (N=774).DOCX File , 16 KB
- Almario CV, Chey W, Kaung A, Whitman C, Fuller G, Reid M, et al. Computer-generated vs. physician-documented history of present illness (HPI): results of a blinded comparison. Am J Gastroenterol 2015 Jan;110(1):170-179 [FREE Full text] [CrossRef] [Medline]
- Almario CV, Chey WD, Iriana S, Dailey F, Robbins K, Patel AV, et al. Computer versus physician identification of gastrointestinal alarm features. Int J Med Inform 2015 Dec;84(12):1111-1117 [FREE Full text] [CrossRef] [Medline]
- Almario CV, Chey WD, Khanna D, Mosadeghi S, Ahmed S, Afghani E, et al. Impact of National Institutes of Health Gastrointestinal PROMIS measures in clinical practice: results of a multicenter controlled trial. Am J Gastroenterol 2016 Nov;111(11):1546-1556 [FREE Full text] [CrossRef] [Medline]
- Ahmed S, Almario CV, Chey WD, Robbins LA, Chang B, Ahn J, et al. Electronic patient agenda forms: comparing agreement between the reason for specialty consultation reported by referring providers and patients. Inform Health Soc Care 2019 Jan;44(1):105-113 [FREE Full text] [CrossRef] [Medline]
- Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009 Apr;42(2):377-381 [FREE Full text] [CrossRef] [Medline]
- Collins SA, Rozenblum R, Leung WY, Morrison CR, Stade DL, McNally K, et al. Acute care patient portals: a qualitative study of stakeholder perspectives on current practices. J Am Med Inform Assoc 2017 Apr 01;24(e1):e9-e17 [FREE Full text] [CrossRef] [Medline]
- Patel V, Barker W, Siminerio E. Individuals’ access and use of their online medical record nationwide. The Office of the National Coordinator for Health Information Technology (ONC). 2014 Sep. URL: https://www.healthit.gov/sites/default/files/consumeraccessdatabrief_9_10_14.pdf [accessed 2021-04-18]
- Goldzweig CL, Orshansky G, Paige NM, Towfigh AA, Haggstrom DA, Miake-Lye I, et al. Electronic patient portals: evidence on health outcomes, satisfaction, efficiency, and attitudes: a systematic review. Ann Intern Med 2013 Nov 19;159(10):677-687. [CrossRef] [Medline]
- Basch E, Deal AM, Kris MG, Scher HI, Hudis CA, Sabbatini P, et al. Symptom monitoring with patient-reported outcomes during routine cancer treatment: a randomized controlled trial. J Clin Oncol 2016 Feb 20;34(6):557-565 [FREE Full text] [CrossRef] [Medline]
- Basch E, Deal AM, Dueck AC, Scher HI, Kris MG, Hudis C, et al. Overall survival results of a trial assessing patient-reported outcomes for symptom monitoring during routine cancer treatment. JAMA 2017 Jul 11;318(2):197-198 [FREE Full text] [CrossRef] [Medline]
- Baumann LA, Baker J, Elshaug AG. The impact of electronic health record systems on clinical documentation times: A systematic review. Health Policy 2018 Aug;122(8):827-836. [CrossRef] [Medline]
- Sinsky C, Colligan L, Li L, Prgomet M, Reynolds S, Goeders L, et al. Allocation of physician time in ambulatory practice: a time and motion study in 4 specialties. Ann Intern Med 2016 Dec 06;165(11):753-760. [CrossRef] [Medline]
- Adler-Milstein J, Holmgren AJ, Kralovec P, Worzala C, Searcy T, Patel V. Electronic health record adoption in US hospitals: the emergence of a digital "advanced use" divide. J Am Med Inform Assoc 2017 Nov 01;24(6):1142-1148 [FREE Full text] [CrossRef] [Medline]
- Fraccaro P, Vigo M, Balatsoukas P, Buchan IE, Peek N, van der Veer SN. Patient portal adoption rates: a systematic literature review and meta-analysis. Stud Health Technol Inform 2017;245:79-83. [Medline]
- Myrick KL, Ogburn DF, Ward BW. Percentage of office-based physicians using any electronic health record (EHR)/electronic medical record (EMR) system and physicians that have a certified EHR/EMR system, by U.S. state: National Electronic Health Records Survey, 2017. National Center for Health Statistics. 2019. URL: https://www.cdc.gov/nchs/data/nehrs/2017_NEHRS_Web_Table_EHR_State.pdf [accessed 2021-04-18]
|AEGIS: Automated Evaluation of Gastrointestinal Symptoms|
|EHR: electronic health record|
|HPI: history of present illness|
|OR: odds ratio|
Edited by G Eysenbach; submitted 17.08.20; peer-reviewed by G Orshansky, J Chaplin; comments to author 02.11.20; revised version received 18.01.21; accepted 11.04.21; published 04.05.21Copyright
©Natalie J Hall, Sameer K Berry, Jack Aguilar, Elizabeth Brier, Parth Shah, Derek Cheng, Jeremy Herman, Theodore Stein, Brennan M R Spiegel, Christopher V Almario. Originally published in JMIR Formative Research (https://formative.jmir.org), 04.05.2021.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.