This paper is in the following e-collection/theme issue:

Formative Evaluation of Digital Health Interventions (2358) Decision Support for Health Professionals (1164) Human Factors and Usability Case Studies (773) Consumer & Patient Education and Shared-Decision Making (667)

Published on in Vol 7 (2023)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/47574, first published .
User-Centered Design and Evaluation of Clinical Decision Support to Improve Early Peanut Introduction: Formative Study

User-Centered Design and Evaluation of Clinical Decision Support to Improve Early Peanut Introduction: Formative Study

User-Centered Design and Evaluation of Clinical Decision Support to Improve Early Peanut Introduction: Formative Study

Authors of this article:

Thinh Hoang Nguyen1, 2 Author Orcid Image ;   Priscila Pereira Cunha1 Author Orcid Image ;   Annabelle Friedman Rowland1 Author Orcid Image ;   Evan Orenstein1, 3 Author Orcid Image ;   Tricia Lee1, 4 Author Orcid Image ;   Swaminathan Kandaswamy1 Author Orcid Image
Article Authors Cited by (2) Tweetations (2) Metrics

    Original Paper

    1Department of Pediatrics, Emory University School of Medicine, Atlanta, GA, United States

    2Division of Immunology, Boston Children's Hospital, Boston, MA, United States

    3Division of Hospital Medicine, Children's Healthcare of Atlanta, Atlanta, GA, United States

    4Department of Allergy and Immunology, Children's Healthcare of Atlanta, Atlanta, GA, United States

    Corresponding Author:

    Swaminathan Kandaswamy, PhD

    Department of Pediatrics

    Emory University School of Medicine

    2015 Uppergate Dr

    Atlanta, GA, 30322

    United States

    Phone: 1 443 720 9337

    Email: swaminathan.kandaswamy@emory.edu


    Background: Peanut allergy has recently become more prevalent. Peanut introduction recommendations have evolved from suggesting peanut avoidance until the age of 3 years to more recent guidelines encouraging early peanut introduction after the Learning Early about Peanut Allergy (LEAP) study in 2015. Guideline adherence is poor, leading to missed care opportunities.

    Objective: In this study, we aimed to develop a user-centered clinical decision support (CDS) tool to improve implementation of the most recent early peanut introduction guidelines in the primary care clinic setting.

    Methods: We edited the note template of the well-child check (WCC) visits at ages 4 and 6 months with CDS prompts and point-of-care education. Formative and summative usability testing were completed with pediatric residents in a simulated electronic health record (EHR). We estimated task completion rates and perceived usefulness of the CDS in summative testing, comparing a test EHR with and without the CDS.

    Results: Formative usability testing with the residents provided qualitative data that led to improvements in the build for both the 4-month and 6-month WCC note templates. During summative usability testing, the CDS tool significantly improved discussion of early peanut introduction at the 4-month WCC visit compared to scenarios without the CDS tool (9/15, 60% with CDS and 0/15, 0% without CDS). All providers except one at the 4-month WCC scenario gave at least an adequate score for the ease of use of the CDS tool for the history of present illness and assessment and plan sections. During the summative usability testing with the 6-month WCC new build note template, providers more commonly provided comprehensive care once obtaining a patient history concerning for an immunoglobulin E–mediated peanut reaction by placing a referral to allergy/immunology (P=.48), prescribing an epinephrine auto-injector (P=.07), instructing on how to avoid peanut products (P<.001), and providing an emergency treatment plan (P=.003) with CDS guidance. All providers gave at least an adequate score for ease of use of the CDS tool in the after-visit summary.

    Conclusions: User-centered CDS improved application of early peanut introduction recommendations and comprehensive care for patients who have symptoms concerning for peanut allergy in a simulation.

    JMIR Form Res 2023;7:e47574

    doi:10.2196/47574

    Keywords

    clinical decision support; peanut; peanut introduction; early peanut introduction; allergy; electronic health records; simulation; user-centered design 


    Peanut allergy is by some reports the most common food allergy among children [1,2], with most recent data showing between 2% to 5% prevalence [3,4]. The prevalence of peanut allergy has nearly doubled in the past decades, posing a growing public health concern [5-7]. Unlike many other types of food allergies, peanut allergy is rarely outgrown [8,9], leading to lifelong impact. Up to 60% of children with peanut allergy experience a severe allergic reaction [3,10-12]. Among all emergency department (ED) visits and hospitalizations for anaphylaxis, including in intensive care units, peanut products are the most common trigger. Although fatality is not common, morbidity and reductions in quality of life remain a concern [13]. In one study, children with peanut allergy reported poorer quality of life than children with insulin-dependent diabetes [14].

    In 2000, the American Academy of Pediatrics released guidance suggesting avoidance of peanuts until the age of 3 years [15], but later revised this, stating in 2008 there was inconclusive data regarding the benefits of delaying peanut introduction [16]. In 2015, the Learning Early about Peanut Allergy (LEAP) study showed that early introduction of peanuts between the ages of 4 months and 11 months was not only safe but also led to lower rates of peanut allergy among high-risk infants [17]. Another study, the Enquiring About Tolerance (EAT) trial, expanded on the previous results by showing that early peanut introduction in infants from a general population, regardless of risk, led to a lower prevalence of peanut allergy [18]. Based on these results, the National Institute of Allergy and Infectious Diseases (NIAID) released new guidelines in 2017, recommending early introduction of peanuts according to risk stratification [19].

    Despite the new guidelines, the rates of early peanut introduction remain low [20-23]. The recommendation for testing high-risk infants prior to early introduction created confusion [21,24], high cost [25], and logistical challenges [21,24,25]. Other barriers to implementation among pediatricians included lack of awareness of guidelines and low comfort discussing them with parents [21,24]. Caregivers have reported fear of introducing peanut products or unawareness of the data, with many saying they relied on their pediatrician for such recommendations [23,26-28]. Interestingly, in Australia, where guidelines recommend introduction around the age of 6 months for all individuals and do not include any testing, there was a 3-fold increase in peanut product introduction before the age of 1 year [29]. In order to address some of the concerns, in 2021 consensus guidance was released by 3 major North American allergy organizations recommending introduction of peanuts to all infants around 6 months of age, with less emphasis on allergy testing [30].

    Though most infants will not have symptoms from peanuts, some may have symptoms suggestive of an immunoglobulin E (IgE)–mediated allergy, which can cause severe, life-threatening anaphylactic reactions. The American Academy of Allergy, Asthma, and Immunology (AAAAI) outlines anticipatory guidance that clinicians should provide to patients and families who are at risk of anaphylaxis. The guidelines include educating patients about anaphylaxis, the risk of recurrence, trigger avoidance, and thresholds for future care, as well as providing a prescription for an epinephrine auto-injector (EAI) and a referral to an allergist [31].

    Although there is not much data available pertaining to the prescription rates of EAIs for patients with a first-time reaction specific to peanuts, one meta-analysis looked at EAI prescription rates for patients with anaphylaxis regardless of the allergic trigger. [32]. Of the 75 studies reviewed, 44 reported less than 50% of the patients who had experienced anaphylaxis had a prescription for an EAI [32]. It remains crucial that EAIs are prescribed at time of diagnosis of any IgE-mediated allergy. More information is needed regarding the prescription patterns of EAIs by pediatric primary care providers for peanut-allergic patients.

    Despite these challenges, one study, from Clark et al [33], described an improvement in providing anticipatory guidance in the ED for patients presenting with food-induced anaphylaxis over a 15-year interval period. Overall, physicians prescribed EAIs, gave allergy referrals, and provided allergen avoidance education at statistically significant higher rates when compared to 15 years prior. However, only 23% of patient encounters met criteria for at least 3 or more of the AAAAI guidelines [33]. This study emphasized the need for addressing provider variability despite standardized guidelines and closing gaps between recognizing food-induced anaphylaxis and taking the steps to prevent its recurrence with proper anticipatory guidance. Little information is available about the success rates of providers in the primary care setting in adhering to the main AAAAI anticipatory guidelines for IgE-mediated reactions, especially in the setting of peanut allergy.

    Clinical decision support (CDS) provides timely point-of-care information to clinicians, who can then share decision-making with patients. CDS design often suffers from poor usability and consideration of workflows [34,35], leading to poor adoption and ultimately to inconsistent effectiveness of CDS [36-38]. Adopting human factors principles and following a user-centered design process with iterative design based on feedback from users and usability evaluations can improve effectiveness of CDS by improving usability as well as ensuring appropriate workflow integration [39-41].

    The aims of this study were to (1) apply user-centered design in a formative evaluation to design and develop a CDS tool to recommend early peanut introduction and (2) evaluate effectiveness of the CDS tool in simulated settings.


    Setting

    The study was conducted at a large academic pediatric primary care clinic within an inner-city freestanding children’s hospital serving diverse, low-income, and historically marginalized populations.

    Ethical Considerations

    This study was a quality improvement project and was deemed non–human subject research by the institutional review board at Children’s Healthcare of Atlanta (STUDY00001218).

    CDS Development

    We developed an initial prototype of the CDS tool after brainstorming with the CDS team and clinical stakeholders. This CDS tool was built as a collection of automated prompts within the well-child check (WCC) note, some of which included reminders with disappearing text that was automatically deleted at the signing of the note [42] with branching logic within the progress note template in a test electronic health record (EHR) environment.

    Formative Usability Testing

    We adopted a think aloud protocol and conducted formative usability testing to iteratively redesign our CDS tool to promote early peanut introduction [43,44]. We reached out to participants in-situ, that is, in their clinical workspace, and asked if they had 10 minutes to provide feedback on a tool for decision support. Participants were free to attend clinical calls or step out for patient care if needed in between the testing sessions. We briefly explained that the goal of the study was to obtain feedback on a CDS tool without explicitly mentioning anything about peanut introduction. After providing them with psychological safety, we provided them with a clinical scenario of an otherwise well child who breastfeeds. Participants were asked to think aloud as they completed documentation of the diet section on a mock patient chart. They were free to ask questions to one of the study team members administering the test who acted as the parent. At the end of documentation participants were debriefed on the aim of the study and qualitative feedback on design of the CDS tool was obtained, with notes checked by a team member with the participant. Based on feedback, we iteratively improved design between participants before summative usability testing.

    Summative Usability Testing

    To estimate the effectiveness of the CDS tool, we conducted summative testing. An updated CDS design based on learnings from the formative testing was used. Similar to formative testing, we reached out to participants in-situ, that is, in their clinical workspace, and asked if they had 10 minutes to test a tool for decision support. Participants were given 1 of the 2 clinical scenarios and were asked to complete the documentation as they would in practice, first with the existing design and then with the updated design for the same scenario. At the end of the test, participants completed a survey with a 4-point Likert scale (1=“not useful,” 2=“adequate,” 3=“useful,” 4=“very useful”) on the usefulness of the CDS tool. They were also asked to provide qualitative feedback. One test administrator took notes and at the end of the testing notes from the feedback were checked by a team member with the participants. One of the test administrators acted as a parent and answered participant questions. Scenario 1 was a 4-month WCC visit with focus on the diet that included breastfeeding. For scenario 1, we defined a task as successfully completed if the participant talked about peanut introduction and discussed options for peanut introduction. Scenario 2 was a 6-month WCC visit that included the parent providing new history that the infant tried peanuts and had symptoms suggesting an IgE-mediated reaction. For scenario 2, we evaluated successful task completion based on 4 actions, including referring to allergy/immunology, prescribing an EAI, instructing on how to avoid peanuts, and providing an emergency treatment plan.

    Statistical Analysis

    All statistical analyses were performed using R (version 3.6.3; R Core Team). We used descriptive statistics to summarize task completion rates with and without CDS in the summative testing. Significant differences in performance were assessed at a P value of .05 using the McNemar test.


    Formative Usability Testing

    A total of 6 pediatric residents participated in the formative testing (2 interns, 3 third-year residents, and 1 chief resident). All 6 residents reported that the CDS tool was overall useful and easy to use. The main concerns from residents were that the tool may need to provide point-of-care education on the definition of an immediate reaction, that the inclusion of a hyperlink that took the user to the literature supporting early peanut recommendation was confusing and tedious, that it was possible to misunderstand the instructions on how to include the early peanut introduction guidance in the after-visit summary (AVS), and that they desired documentation assistance in the plan section of the note about the early peanut introduction discussion with the family. Based on this feedback, 4 major build changes were made to the CDS tool: (1) a disappearing text description with the definition of an immediate reaction was added; (2) the hyperlink to a specific citation was replaced with disappearing text showing a 1-sentence summary of the recommendations; (3) a description of the discussion with the family regarding peanut introduction was added to the plan section; and (4) a drop-down menu was added that automatically populated in the AVS and allowed the user to select instructions for either peanut introduction, continuation, or avoidance (Table 1 and Figure 1).

    Table 1. Major changes to the clinical decision support tool after formative testing.
    Original designResident feedbackEdited design
    At the 6- and 12-month visits, when discussing peanut introduction, if the family had already tried peanut products and reported a reaction, residents were supposed to document if it was an immediate or nonimmediate reaction.Residents reported that they may need point-of-care education on the definition of an immediate reaction.A disappearing text description was added with the definition of an immediate reaction.
    When assessing family readiness and planning introduction of peanut products, residents were given a hyperlink that took them to literature supporting early peanut introduction.Residents found the inclusion of a hyperlink was confusing and tedious.The hyperlink was replaced with a disappearing 1-sentence summary of the recommendations.
    Residents freely typed what they discussed with the family in the note’s assessment and plan section.Residents desired documentation assistance in the plan section of the note about the early peanut introduction discussion with the family.A description of the discussion with the family regarding peanut introduction was added to the plan section.
    At 4-, 6-, and 12-month visits, residents were given a SmartPhrase (an abbreviation that helps to pull long phrases into a note in the Epic system) to type into the after-visit summary that would insert peanut introduction, continuation, or avoidance instructions.Residents misunderstood the instructions on how to include the early peanut introduction guidance in the after-visit summary.A drop-down menu to select instructions for either peanut introduction, continuation, or avoidance automatically populated the after-visit summary.
    ‎
    Figure 1. Screenshots of (A) the original design of the clinical decision support tool and (B) the edited design after major changes were made based on formative testing. Feedback included comments that the tool needed to include point-of-care education on the definition of an immediate reaction, that the hyperlink was confusing and tedious, and that it was possible to misunderstand the instructions for the after-visit summary.

    Summative Usability Testing

    A total of 30 providers (Table 2) participated in the summative testing—15 for scenario 1 and 15 for scenario 2 (Table 3 and Table 4). The demographics of participants in our summative testing are provided in Table 2. The task completion rate in scenario 1 was significantly higher with CDS, at 60% (9/15), compared to 0% without CDS (0/15; P=.008).

    For scenario 2, all participants placed an allergy/immunology referral when using CDS while 2 of 15 did not refer patients with the original note template that did not contain the CDS (P=.48). A total of 87% (13/15) prescribed an EAI when using CDS, while only 53% (8/15) did without CDS (P=.07). There was significant difference in users providing instructions on peanut avoidance, where 100% (15/15) of participants provided anticipatory guidance when using CDS while only 13% (2/15) provided such instruction without CDS (P<.001). All participants provided an emergency treatment plan when using CDS, while only 26% (4/15) completed this element without CDS (P=.003). All participants completed at least 3 of 4 elements in the AAAAI guidelines with CDS, compared to 33% (5/15) without CDS.

    Table 2. Summative testing participant demographics (n=30).
    CharacteristicsParticipants, n (%)
    Role

    		Attending4a (13)

    		Resident26 (87)
    Sex

    		Male10 (33)

    		Female20 (67)
    Race

    		White19 (63)

    		Black5 (17)

    		Asian5 (17)

    		Hispanic/Latino1 (3)
    Experience (years)

    		>102 (7)

    		6-71 (3)

    		4-51 (3)

    		2-38 (27)

    		1-212 (40)

    		<16 (20)

    aIncluding 1 nurse practitioner.

    Table 3. Scenario 1: Participants who discussed peanut introduction with and without CDSa.

    		Participant

    		123456789101112131415
    Without CDS














    With CDS✓b✓




    		✓✓✓
    		✓✓✓✓

    aCDS: clinical decision support.

    b✓: indicates that the participant successfully completed that specific task.

    Table 4. Scenario 2: Actions completed by participants with and without clinical decision support. Checkmarks in the rows labeled “Successful” indicate that a participant completed at least 3 of 4 elements in the American Academy of Allergy, Asthma, and Immunology guidelines.

    		Participant completed the action

    		123456789101112131415
    Actions














    Without clinical decision support

    		Allergy/immunology referral✓a✓✓✓✓✓
    		✓✓✓✓✓
    		✓✓

    		Prescription for epinephrine✓✓✓
    		✓
    		✓
    		✓

    		✓

    		✓

    		Instructions to avoid peanuts

    		✓




    		✓






    		Discuss treatment plan✓


    		✓


    		✓




    		✓

    		Successful✓
    		✓
    		✓


    		✓




    		✓
    With clinical decision support

    		Allergy/immunology referral✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓

    		Prescription for epinephrine✓✓✓
    		✓
    		✓✓✓✓✓✓✓✓✓

    		Instructions to avoid peanuts✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓

    		Discuss treatment plan✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓

    		Successful✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓

    a✓: indicates that the participant successfully completed that specific task.

    Qualitative Feedback From Summative Testing

    Participants had feedback to improve the CDS design. First, they needed information on infant-friendly peanut products that they could suggest to patients’ families. Though this information was available in the AVS, they wanted the information accessible in the progress note at the point-of-care interaction with the family. Second, participants felt the term “early” in reference to the timing of peanut introduction was vague and wanted better-defined parameters. Third, one senior attending health care provider felt that the CDS prompt asking to read the sentence about early peanut introduction was burdensome and inefficient. She instead wanted a hard-stop yes-no question that providers could answer in the note. She felt that would enable her to bring up the topic as they liked but act as a reminder and a place to document conversation about early introduction of peanuts. Fourth, some participants who had missed noticing the CDS tool suggested adding some highlighting elements draw their attention to reading the sentence. Fifth, participants wanted clarifying verbiage in the progress note to refer to the AVS for the peanut introduction instructions. Sixth, participants were confused by the command- or code-like appearance of the note template and asked for clarification. Seventh, participants also suggested clarifying language in the note template to distinguish if the plan to introduce peanuts was mutually agreed between providers or if there was a disagreement between the provider and patient. Eighth, participants also wanted information, likely in the assessment/plan (A/P) section, indicating when peanuts should not be introduced. Ninth, participants suggested to move the prompt to the top of the history of present illness (HPI) section instead of diet so that providers who did not use the diet section could still see and benefit from the CDS. Tenth, participants who missed seeing the CDS tool when talking with family mentioned that CDS was helpful even if they missed it during the simulation. They said that if they saw it at least twice, they were likely to remember the need to answer this section in the note template. They cited examples of similar interventions that needed documentation in other parts of the EHR that became part of their workflow.

    Survey Response

    Participants felt that the CDS tool was easy to use (mean score 3.5 on a scale of 1=“not useful,” 2=“adequate,” 3=“useful,” and 4=“very useful”). The distribution of participant responses to the usefulness of the CDS tool is shown in Figure 2. For the question “Does this clinical decision support help to introduce the topic of peanut introduction to families?” 87% (13/15) of participants gave a response of “useful” or “very useful.” Similarly, 87% (13/15) of participants gave a response of “useful” or “very useful” when asked “Does this automated smartphrase in the AVS about how to introduce peanut products help you to know what is important to tell families?” In the 6-month WCC scenario, all participants indicated that the CDS was at least adequate for helping them to remember to order an allergy referral. All participants responded “useful” or “very useful” when asked about the usefulness of the CDS in reminding them to prescribe an EAI. They also indicated that the automatic SmartPhrase in the AVS was useful to remind them what was important to tell families (mean score 3.47 in the 4-month WCC scenario, mean score 3.8 in the 6-month WCC scenario).

    ‎
    Figure 2. The ease of use of clinical decision support for 4-month well-child check and 6-month well-child check scenarios for (A) history of present illness, (B) assessment and plan, and (C) after-visit summary. Participants were given 4 different choices, including 1=“not useful,” 2=“adequate,” 3=“useful,” and 4=“very useful.”.

    Principal Results

    In this study, we applied user-centered design to develop a CDS tool for early peanut introduction and evaluated the effectiveness of the CDS tool in simulated settings. Formative testing revealed areas for improvement of the tool, such as providing more active point-of-care education to the user in the tool and adding repeated, consistent reminders throughout the note template. We observed that formative testing not only helped inform user-centered design, but also increased user investment in the tool. Key design changes from the qualitative feedback during summative testing included moving the placement of the tool in the established note template higher up to allow for better visibility, improving the uniformity of the language, and adding more point-of-care education. In the summative evaluation, the improved CDS design successfully facilitated discussions on early peanut introduction during 4-month WCC visits. Furthermore, adherence to guideline recommendations improved for the scenario in which the patient had a history of IgE-mediated peanut allergy at the 6-month WCC. This is evidenced by all participants completing at least 3 of the 4 elements in summative testing with CDS, compared to only 23% in past literature [33].

    Although the majority of participants integrated the CDS tool into their workflows, complete adoption was not achieved. This was likely due to differences in provider workflows. Specifically, some providers completed the documentation at the desk away from the family after speaking with the parent, while others completed the documentation while speaking with the family. During summative testing, providers who followed the former workflow did not see the CDS tool in the simulation and therefore did not use it. While this issue may not be completely resolved, we believe that the user-friendly CDS tool, developed with user-centered design and with a note template and education module, could improve utilization rates during real-life implementation.

    Despite recent rapid guidelines changes, there is still a dearth of literature on CDS development and implementation in the field of peanut allergy prevention and treatment, specifically at the primary care level [45]. While there is some data on the implementation of feeding guidelines in Australia, it is unclear whether any informatics techniques were used to encourage these practice changes. There continues to be data to support the lack of implementation of the early peanut recommendations by pediatricians specifically here in the United States [21,27,46]. However, interventions such as the Intervention to Reduce Early (Peanut) Allergy in Children (iREACH) program have been successful in improving pediatrician adherence to early peanut introduction guidelines in Chicago, Illinois [47]. This group has not published the exact build details of their CDS tool, except to say that it includes an order set, prompts, handouts, and a best-practice advisory (BPA). Generally, BPAs are interruptive to workflows. Our build development may be unique in that it has a user-centered design based on formative and summative usability testing. Also, our CDS tool is automated in all WCC encounters, is noninterruptive, provides point-of-care education with disappearing text, and uses branching logic. Moreover, our CDS tool implements more recent consensus recommendations on peanut introduction that do not stratify patients by risk.

    A prior attempt to improve management of food allergy patients in one health system via an interruptive alert in the EHR (triggered by either a diagnosis code of food allergy or EAI prescription) found no significant difference in allergy referral, EAI prescription, or food allergy action planning [48]. While the most common form of CDS tool is an alert, these are not always effective to drive process change or improve outcomes [49]. Furthermore, they are interruptive in nature, adding to alert fatigue and clinician burnout [50]. Our passive CDS tool avoids the risk of alert fatigue but was able to achieve process change, as evident from simulated tests and the favorable scores in the user survey of the final build of our CDS tool. Future real-world scenario testing would further confirm the rate of adoption for our build.

    Limitations

    This study has some limitations. First, the study is limited by the evaluation being performed at a single institution. Second, while we used realistic patient scenarios and had study team members act as family to answer questions, we did not go through the whole visit in simulation. This may have biased conversation about peanut introduction. However, this risk is low, as we confirmed with participants if they would have bought it up in conversation to parents at another time during such an encounter, and they responded negatively. Third, all participants in the formative testing were residents. This may have led us to miss some of the workflow considerations of attending physicians in the CDS design. However, we captured feedback from attending physicians in summative testing and will accommodate them with design changes prior to real-world implementation. Finally, we are limited by the lack of evaluations in clinical practice. As a next step, we plan to test the CDS tool in a real-world clinical setting, which should show the effectiveness of the tool in supporting early peanut introduction as well as standardizing care for newly presenting peanut reactions.

    Conclusion

    Passive CDS tools, as opposed to active alerts, can be effective to help implement evidence-based practices for all patients despite the burdens of busy clinics. User-centered design with feedback from clinical users and rapid prototyping can help develop solutions that work for users and will be seamlessly incorporated into their existing workflows, thus changing clinical outcomes.

    Acknowledgments

    We would like to give special thanks to Dr Melissa Adams for helping us understand clinical workflows and Dr Belise Livingston- Burns for supporting our project as medical director. On the electronic health record build side, we appreciate the assistance and expertise of informatics advisor Tonya Bennett and clinical informaticist-hospitalist Dr Jonathan Beus. We thank our sounding boards in allergy, Dr Roselyn Hicks, Dr Brian Vickery, and Dr Gerald Lee. Lastly, we extend our sincere gratitude to the residents and faculty members at Emory University and Morehouse School of Medicine for their assistance and collaboration during the project. Their contribution was essential to the success of our initiative, and we value their commitment to improving patient care.

    Data Availability

    The data sets generated during and/or analyzed during this study are available from the corresponding author on reasonable request.

    Conflicts of Interest

    EO is a cofounder of Phrase Health, a company that provides clinical decision support analytics, and holds equity in the company but does not receive any direct revenue. Additionally, he serves as the principal investigator for a phase 1 and phase 2 small business technology transfer grant awarded to Phrase Health by the National Library of Medicine and the National Center for Advancing Translational Science. All other authors have no conflicts of interest to disclose.

    1. Gupta RS, Springston EE, Warrier MR, Smith B, Kumar R, Pongracic J, et al. The prevalence, severity, and distribution of childhood food allergy in the United States. Pediatrics. Jul 2011;128(1):e9-17. [CrossRef] [Medline]
    2. Osborne NJ, Koplin JJ, Martin PE, Gurrin LC, Lowe AJ, Matheson MC, et al. HealthNuts Investigators. Prevalence of challenge-proven IgE-mediated food allergy using population-based sampling and predetermined challenge criteria in infants. J Allergy Clin Immunol. Mar 2011;127(3):668-76.e1. [CrossRef] [Medline]
    3. Jiang J, Bushara O, Ponczek J, Warren C, Blumenstock J, Smith B, et al. Updated pediatric peanut allergy prevalence in the United States. Ann Allergy Asthma Immunol. Nov 2018;121(5):S14. [CrossRef]
    4. Bunyavanich S, Rifas-Shiman SL, Platts-Mills TAE, Workman L, Sordillo JE, Gillman MW, et al. Peanut allergy prevalence among school-age children in a US cohort not selected for any disease. J Allergy Clin Immunol. Sep 2014;134(3):753-755. [FREE Full text] [CrossRef] [Medline]
    5. Prescott SL, Pawankar R, Allen KJ, Campbell DE, Sinn JK, Fiocchi A, et al. A global survey of changing patterns of food allergy burden in children. World Allergy Organ J. Dec 04, 2013;6(1):21. [FREE Full text] [CrossRef] [Medline]
    6. Sicherer SH, Muñoz-Furlong A, Godbold JH, Sampson HA. US prevalence of self-reported peanut, tree nut, and sesame allergy: 11-year follow-up. J Allergy Clin Immunol. Jun 2010;125(6):1322-1326. [CrossRef] [Medline]
    7. Pate CA, Zahran HS, Malilay J, Hsu J. The shifting prevalence of asthma and allergic disease in US children. Ann Allergy Asthma Immunol. Oct 2022;129(4):481-489. [CrossRef] [Medline]
    8. Skolnick HS, Conover-Walker MK, Koerner CB, Sampson HA, Burks W, Wood RA. The natural history of peanut allergy. J Allergy Clin Immunol. Feb 2001;107(2):367-374. [CrossRef] [Medline]
    9. Savage J, Johns CB. Food allergy: epidemiology and natural history. Immunol Allergy Clin North Am. Feb 2015;35(1):45-59. [FREE Full text] [CrossRef] [Medline]
    10. Sicherer SH, Muñoz-Furlong A, Sampson HA. Prevalence of peanut and tree nut allergy in the United States determined by means of a random digit dial telephone survey: a 5-year follow-up study. J Allergy Clin Immunol. Dec 2003;112(6):1203-1207. [CrossRef] [Medline]
    11. Leickly FE, Kloepfer KM, Slaven JE, Vitalpur G. Peanut allergy: An epidemiologic analysis of a large database. J Pediatr. Jan 2018;192:223-228.e1. [CrossRef] [Medline]
    12. Deschildre A, Elegbédé CF, Just J, Bruyère O, Van der Brempt X, Papadopoulos A, et al. Peanut-allergic patients in the MIRABEL survey: characteristics, allergists' dietary advice and lessons from real life. Clin Exp Allergy. Apr 2016;46(4):610-620. [CrossRef] [Medline]
    13. Ramsey NB, Guffey D, Anagnostou K, Coleman NE, Davis CM. Epidemiology of anaphylaxis in critically ill children in the United States and Canada. J Allergy Clin Immunol Pract. 2019;7(7):2241-2249. [FREE Full text] [CrossRef] [Medline]
    14. Avery NJ, King RM, Knight S, Hourihane JO. Assessment of quality of life in children with peanut allergy. Pediatr Allergy Immunol. Oct 2003;14(5):378-382. [CrossRef] [Medline]
    15. American Academy of Pediatrics; Committee on Nutrition. Hypoallergenic infant formulas. Pediatrics. Aug 2000;106(2 Pt 1):346-349. [Medline]
    16. Greer FR, Sicherer SH, Burks AW, American Academy of Pediatrics Committee on Nutrition; American Academy of Pediatrics Section on Allergy and Immunology. Effects of early nutritional interventions on the development of atopic disease in infants and children: the role of maternal dietary restriction, breastfeeding, timing of introduction of complementary foods, and hydrolyzed formulas. Pediatrics. Jan 2008;121(1):183-191. [CrossRef] [Medline]
    17. Du Toit G, Roberts G, Sayre PH, Bahnson HT, Radulovic S, Santos AF, et al. LEAP Study Team. Randomized trial of peanut consumption in infants at risk for peanut allergy. N Engl J Med. Feb 26, 2015;372(9):803-813. [FREE Full text] [CrossRef] [Medline]
    18. Perkin MR, Logan K, Tseng A, Raji B, Ayis S, Peacock J, et al. Randomized trial of introduction of allergenic foods in breast-fed infants. N Engl J Med. May 05, 2016;374(18):1733-1743. [FREE Full text] [CrossRef] [Medline]
    19. Togias A, Cooper SF, Acebal ML, Assa'ad A, Baker JR, Beck LA, et al. Addendum guidelines for the prevention of peanut allergy in the United States: Report of the National Institute of Allergy and Infectious Diseases-sponsored expert panel. J Allergy Clin Immunol. Jan 2017;139(1):29-44. [FREE Full text] [CrossRef] [Medline]
    20. Tapke DE, Stukus DR, Prince BT, Scherzer R, Mikhail I. Implementation of early peanut introduction guidelines among pediatricians. J Allergy Clin Immunol. Feb 2019;143(2):AB273. [FREE Full text] [CrossRef]
    21. Gupta RS, Bilaver LA, Johnson JL, Hu JW, Jiang J, Bozen A, et al. Assessment of pediatrician awareness and implementation of the addendum guidelines for the prevention of peanut allergy in the United States. JAMA Netw Open. Jul 01, 2020;3(7):e2010511. [FREE Full text] [CrossRef] [Medline]
    22. Hoffman B, Moreno L, Gerber L, D'Angelo D, Abramson E. OR075 What pediatricians are advising on infant peanut introduction. Ann Allergy Asthma Immunol. Nov 2017;119(5):S10. [CrossRef]
    23. Marget M, Martin V, Virkud Y, Seay H, Rosow R, Yuan Q, et al. Impact of the LEAP Study on age at introduction of peanut in a suburban U.S. cohort. J Allergy Clin Immunol. Feb 2022;149(2):AB105. [FREE Full text] [CrossRef]
    24. Chang A, Cabana MD, LaFlam TN, Patel S, Okumura M. Early peanut introduction and testing: A framework for general pediatrician beliefs and practices. Pediatr Allergy Immunol Pulmonol. Jun 2021;34(2):53-59. [FREE Full text] [CrossRef] [Medline]
    25. Koplin JJ, Peters RL, Dharmage SC, Gurrin L, Tang MLK, Ponsonby A, et al. HealthNuts study investigators. Understanding the feasibility and implications of implementing early peanut introduction for prevention of peanut allergy. J Allergy Clin Immunol. Oct 2016;138(4):1131-1141.e2. [CrossRef] [Medline]
    26. Farhat L, Hanono M, Nagarajan S, Harriz R, Joks R, Vastardi M. Early peanut introduction: Knowledge and current practices of inner-city parents. J Allergy Clin Immunol. Feb 2020;145(2):AB183. [FREE Full text] [CrossRef]
    27. Sandhu S, Hanono M, Nagarajan S, Vastardi M. Knowledge assessment of early peanut introduction in a New York City population. Ann Allergy Asthma Immunol. Sep 2022;129(3):380-382. [CrossRef] [Medline]
    28. Greenhawt M, Chan ES, Fleischer DM, Hicks A, Wilson R, Shaker M, et al. Caregiver and expecting caregiver support for early peanut introduction guidelines. Ann Allergy Asthma Immunol. Jun 2018;120(6):620-625. [CrossRef] [Medline]
    29. Soriano VX, Peters RL, Ponsonby A, Dharmage SC, Perrett KP, Field MJ, et al. Earlier ingestion of peanut after changes to infant feeding guidelines: The EarlyNuts study. J Allergy Clin Immunol. Nov 2019;144(5):1327-1335.e5. [CrossRef] [Medline]
    30. Fleischer DM, Chan ES, Venter C, Spergel JM, Abrams EM, Stukus D, et al. A consensus approach to the primary prevention of food allergy through nutrition: guidance from the American Academy of Allergy, Asthma, and Immunology; American College of Allergy, Asthma, and Immunology; and the Canadian Society for Allergy and Clinical Immunology. J Allergy Clin Immunol Pract. Jan 2021;9(1):22-43.e4. [CrossRef] [Medline]
    31. Shaker MS, Wallace DV, Golden DBK, Oppenheimer J, Bernstein JA, Campbell RL, Collaborators; Chief Editors; Workgroup Contributors; Joint Task Force on Practice Parameters Reviewers; et al. Anaphylaxis-a 2020 practice parameter update, systematic review, and Grading of Recommendations, Assessment, Development and Evaluation (GRADE) analysis. J Allergy Clin Immunol. Apr 2020;145(4):1082-1123. [CrossRef] [Medline]
    32. Miles LM, Ratnarajah K, Gabrielli S, Abrams EM, Protudjer JLP, Bégin P, et al. Community use of epinephrine for the treatment of anaphylaxis: A review and meta-analysis. J Allergy Clin Immunol Pract. Jun 2021;9(6):2321-2333. [CrossRef] [Medline]
    33. Clark S, Boggs KM, Balekian DS, Hasegawa K, Vo P, Rowe BH, et al. MARC-38 Investigators. Changes in emergency department concordance with guidelines for the management of food-induced anaphylaxis: 1999-2001 Versus 2013-2015. J Allergy Clin Immunol Pract. 2019;7(7):2262-2269. [CrossRef] [Medline]
    34. Kilsdonk E, Peute LW, Jaspers MWM. Factors influencing implementation success of guideline-based clinical decision support systems: A systematic review and gaps analysis. Int J Med Inform. Feb 2017;98:56-64. [CrossRef] [Medline]
    35. Ratwani RM, Reider J, Singh H. A decade of health information technology usability challenges and the path forward. JAMA. Feb 26, 2019;321(8):743-744. [CrossRef] [Medline]
    36. Moja L, Kwag KH, Lytras T, Bertizzolo L, Brandt L, Pecoraro V, et al. Effectiveness of computerized decision support systems linked to electronic health records: a systematic review and meta-analysis. Am J Public Health. Dec 2014;104(12):e12-e22. [CrossRef] [Medline]
    37. Georgiou A, Prgomet M, Paoloni R, Creswick N, Hordern A, Walter S, et al. The effect of computerized provider order entry systems on clinical care and work processes in emergency departments: a systematic review of the quantitative literature. Ann Emerg Med. Jun 2013;61(6):644-653.e16. [CrossRef] [Medline]
    38. Strom BL, Schinnar R, Bilker W, Hennessy S, Leonard CE, Pifer E. Randomized clinical trial of a customized electronic alert requiring an affirmative response compared to a control group receiving a commercial passive CPOE alert: NSAID--warfarin co-prescribing as a test case. J Am Med Inform Assoc. 2010;17(4):411-415. [FREE Full text] [CrossRef] [Medline]
    39. Middleton B, Bloomrosen M, Dente MA, Hashmat B, Koppel R, Overhage JM, et al. American Medical Informatics Association. Enhancing patient safety and quality of care by improving the usability of electronic health record systems: recommendations from AMIA. J Am Med Inform Assoc. Jun 2013;20(e1):e2-e8. [FREE Full text] [CrossRef] [Medline]
    40. Carayon P, Hoonakker P, Hundt AS, Salwei M, Wiegmann D, Brown RL, et al. Application of human factors to improve usability of clinical decision support for diagnostic decision-making: a scenario-based simulation study. BMJ Qual Saf. Apr 2020;29(4):329-340. [FREE Full text] [CrossRef] [Medline]
    41. Kandaswamy S, Gill A, Wood S, Mckay L, Hike J, Popkin M, et al. User-centered design of central venous access device documentation. JAMIA Open. Apr 2022;5(1):ooac011. [FREE Full text] [CrossRef] [Medline]
    42. Chong AZ, Lee B, Hollenbach K, Kuelbs CL. Disappearing help text: Implementing a note-based tool for in-line clinical decision support and note bloat reduction. Appl Clin Inform. Oct 2022;13(5):1033-1039. [CrossRef] [Medline]
    43. Orenstein EW, Boudreaux J, Rollins M, Jones J, Bryant C, Karavite D, et al. Formative usability testing reduces severe blood product ordering errors. Appl Clin Inform. Oct 2019;10(5):981-990. [FREE Full text] [CrossRef] [Medline]
    44. Schumacher RM, Lowry SZ. NIST guide to the processes approach for improving the usability of electronic health records. URL: https://nvlpubs.nist.gov/nistpubs/Legacy/IR/nistir7741.pdf [accessed 2023-08-03]
    45. McWilliam V, Venter C, Greenhawt M, Perrett KP, Tang MLK, Koplin JJ, et al. A pragmatic approach to infant feeding for food allergy prevention. Pediatr Allergy Immunol. Sep 2022;33(9):e13849. [FREE Full text] [CrossRef] [Medline]
    46. Soriano VX, Peters RL, Moreno-Betancur M, Ponsonby AL, Gell G, Odoi A, et al. Association between earlier introduction of peanut and prevalence of peanut allergy in infants in Australia. JAMA. Jul 05, 2022;328(1):48-56. [FREE Full text] [CrossRef] [Medline]
    47. Bilaver LA, Martusiewicz MN, Jiang J, Gupta RS. Effectiveness of clinical decision support tools on pediatrician adherence to peanut allergy prevention guidelines. JAMA Pediatr. Dec 01, 2019;173(12):1198-1199. [FREE Full text] [CrossRef] [Medline]
    48. Zelig A, Harwayne-Gidansky I, Gault A, Wang J. Effect of educational and electronic medical record interventions on food allergy management. Allergy Asthma Proc. Sep 2016;37(5):404-408. [FREE Full text] [CrossRef] [Medline]
    49. Coleman JJ, van der Sijs H, Haefeli WE, Slight SP, McDowell SE, Seidling HM, et al. On the alert: future priorities for alerts in clinical decision support for computerized physician order entry identified from a European workshop. BMC Med Inform Decis Mak. Oct 01, 2013;13(1):111. [FREE Full text] [CrossRef] [Medline]
    50. Embi PJ, Leonard AC. Evaluating alert fatigue over time to EHR-based clinical trial alerts: findings from a randomized controlled study. J Am Med Inform Assoc. Jun 2012;19(e1):e145-e148. [FREE Full text] [CrossRef] [Medline]

    ‎
    A/P: assessment and plan
    AAAAI: American Academy of Allergy, Asthma, and Immunology
    AVS: after-visit summary
    BPA: best-practice advisory
    CDS: clinical decision support
    EAI: epinephrine auto-injector
    EAT: Enquiring About Tolerance
    ED: emergency department
    EHR: electronic health record
    HPI: history of present illness
    IgE: immunoglobulin E
    iREACH: intervention to Reduce Early (Peanut) Allergy in Children
    LEAP: Learning Early about Peanut Allergy
    NIAID: National Institute of Allergy and Infectious Diseases
    WCC: well-child check

    Edited by A Mavragani; submitted 24.03.23; peer-reviewed by N Delvaux, J Protudjer; comments to author 04.05.23; revised version received 15.06.23; accepted 21.07.23; published 22.08.23.

    Copyright

    ©Thinh Hoang Nguyen, Priscila Pereira Cunha, Annabelle Friedman Rowland, Evan Orenstein, Tricia Lee, Swaminathan Kandaswamy. Originally published in JMIR Formative Research (https://formative.jmir.org), 22.08.2023.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.