To facilitate communication between patients and physicians in electronic health record (EHR)–integrated environments, we developed an online patient portal (MyGiHealth) that uses a computer algorithm called Automated Evaluation of Gastrointestinal (GI) Symptoms (AEGIS) to systematically collect patients’ symptom information before the clinic visit. Once collected, the data are transformed into a full narrative history of present illness (HPI) that clinicians can review prior to meeting the patient. While our prior studies noted that AEGIS creates higher quality HPIs and collects more alarm features vs physicians [1,2], we found that it did not improve patient satisfaction or shared decision making when compared to usual care in a controlled trial [3]. Therefore, the objective of this study was to investigate whether AEGIS improved physician-centric outcomes vs usual care.

We performed a pragmatic controlled study among adults aged ≥18 years scheduled for a new patient visit at an academic GI teaching practice and 3 community-based GI clinics at Cedars-Sinai Medical Center. This study was approved by the Cedars-Sinai Institutional Review Board, Los Angeles, CA (Pro45243).

During the intervention period (April 17, 2017-February 7, 2018), patients were invited via email to complete AEGIS via the MyGiHealth app 1 week prior to their visit. We describe AEGIS elsewhere [1-4], but in brief, the algorithm systematically assesses patients’ GI symptoms and then transforms the data into a full narrative HPI as shown in Figure 1. For patients who completed AEGIS, their physicians were notified 1 day before the visit that their HPIs were uploaded to the notes section of our EHR (Epic, Verona, WI) for review (see Multimedia Appendix 1 for the email notification that was sent to physicians by research study staff). To identify individuals for the 2 control groups who were comparable to those in the intervention arm, each patient who completed AEGIS was matched (age ±3 years, sex, race/ethnicity, clinic) with up to 4 patients in the pre-intervention period (October 6, 2015-April 6, 2017) and 1 patient in the intervention period who did not complete AEGIS. Of note, the pre-intervention control group was formed as it was not subject to contamination bias, unlike for post-intervention controls (ie, after physicians reviewed AEGIS reports for those in the intervention arm, they might have been more apt to take and document more thorough HPIs for their control patients). Moreover, age, sex, race/ethnicity, and clinic were selected as matching variables as they were readily available for automated extraction from the EHR and we hypothesized at the outset that they may have correlated with our outcomes.

Once recruitment ceased, 5 resident physicians reviewed patient charts and collected outcomes data using a REDCap data abstraction sheet [5]: (1) documentation of alarm symptom(s) in initial note (hematochezia, melena, hematemesis, unintentional weight loss, fevers); (2) documentation of family history of GI malignancy in initial note (colorectal, esophagus, gallbladder, liver, stomach, pancreas, or throat cancer); (3) charting time (time initial encounter closed minus the appointment time); (4) number of follow-up visits in a 6-month period; (5) number of lab, endoscopy, and imaging tests ordered in a 6-month period.

Statistical analyses were performed using Stata 13.1 (StataCorp LP, College Station, TX). A two-tailed P<.05 was considered significant in all analyses. For bivariate analyses, we used analysis of variance and chi-squared tests to compare continuous and categorical data, respectively. We also conducted multivariable regression analyses on our outcomes and included group assignment, patient age, sex, race/ethnicity, and clinic site as covariates when appropriate. Specifically, logistic regression analyses were performed on the AEGIS completion and documentation of alarm symptoms and GI cancer family history outcomes. We used linear regression analysis to compare charting time between the intervention and control groups. Lastly, numbers of follow-up visits and tests ordered within 6 months of the initial visit were compared using zero-inflated negative binomial and negative binomial regression models, respectively.

Of the 774 patients invited to complete AEGIS (Multimedia Appendix 2 shows demographics), 116 (15.0%) completed it before their clinic visit. Table 1 shows results from the regression analysis on completion of the app; older individuals and Asians were less likely to complete AEGIS. No significant associations were seen between app completion and the remaining racial/ethnic groups, sex, and clinic. Among those who completed AEGIS, the consultants’ initial notes for 47 (40.5%) of the 116 patients contained at least a portion of the computer-generated report.

Patients who completed AEGIS (n=116) were matched with 343 patients from the pre-intervention period and 102 from the intervention period who did not complete AEGIS. Table 2 lists their demographics; the groups were largely similar in age, sex, race/ethnicity, clinic, and reason for consult.

In Table 3, we present the physician-centric outcomes stratified by group. No differences were seen for documentation of alarm symptoms and GI cancer family history, EHR charting time, or numbers of follow-up visits and ordered tests.

We discovered that uptake of AEGIS was low, as only 15% of patients accessed the online portal. Surprisingly, this rate was lower than that seen in our prior AEGIS trial (37%) focused on patient-centric outcomes [3]. This was even despite our use of email invitations (the original study used mailings), which we initially hypothesized would increase uptake as the email included a direct AEGIS hyperlink. Of note, research staff emailed invitations directly to patients; it is possible that sending invitations through the EHR patient portal may have enhanced uptake. Prior literature illustrates that patients are accepting of EHR portals [6-8] and they may be more willing to complete interventions sent through official health system platforms. Further research examining optimal methods for deploying digital interventions in EHR-integrated environments are needed.

While AEGIS was built to enhance patient-physician communication by systematically collecting salient components of the history, one-time use of the app did not increase documentation of alarm symptoms or family history of GI malignancy in the initial note. This suggests that physicians in our study may adequately screen for and document relevant red flags and family history. Alternatively, given our pragmatic design, clinicians may not have reviewed the AEGIS report or incorporated it into their note for some patients. We also noted that the app and its computer-generated HPIs did not impact health care utilization or charting time. In short, we did not find that leveraging an online HPI-generation portal measurably improved physician-centric outcomes in this study.

Of note, we previously found that AEGIS collected more alarm features when compared to physicians [2]. The discordant results likely relate to the different study designs; in the prior observational study, AEGIS was completed by patients after their clinic visit, rather than before the visit as in our current study. It is possible that first consulting with the physician subsequently prompted patients, after further introspection, to report more alarm features through AEGIS than were discussed and documented by the physician in clinic. We also previously found that AEGIS creates higher quality HPIs versus those written by doctors [1]. However, based on our findings here and in a prior multicenter controlled trial focused on patient-centered outcomes [3], simply making the comprehensive AEGIS HPIs available for review in the EHR is insufficient for improving care. Further research is needed to determine how best to optimally implement and use these computer-generated data in clinical workflows in order to enhance outcomes.

A limitation of our study was that we could not fully assess whether and how closely physicians reviewed the AEGIS reports in the EHR. While 40.5% of physicians’ notes for the intervention patients contained a portion of the AEGIS report, we do not know how rigorously they reviewed the report after copying it into their notes. On the other hand, for the remaining patients, it is possible that clinicians thoroughly read the AEGIS report in the EHR but chose to not copy and paste it into their official consultant notes. Development of novel, effective methods for alerting and assessing how clinicians use newly uploaded, app-generated data in the EHR and that maximize its use at the point of care are urgently needed. Another limitation was that AEGIS was administered as a one-time intervention; longitudinal use of the app for tracking symptom severity could have impacted outcomes such as numbers of follow-up visits and ordered tests. Notably, longitudinal symptom monitoring via a portal decreased emergency room visits and improved survival among patients with metastatic cancer [9,10].

In summary, we found that uptake of AEGIS was low, as less than 1 in 6 patients completed it before their visit. Moreover, one-time use of a carefully developed and validated patient-provider portal did not improve documentation of key elements of the note nor reduce clinician work burden. This is disappointing as it is well known that the EHR has greatly increased physician charting time [11,12]; our goal has been to identify ways to reduce physician burden through clinically meaningful, EHR-enabled automation. Yet, even in taking care to maximize the benefits of the EHR to support physician-centric outcomes, we were unable to demonstrate a benefit. Given the near universal adoption of EHRs [13-15], further research examining how best to develop and implement digital interventions in EHR-integrated environments for improving both patient and physician outcomes is critical.