Research has shown that a simultaneous, integrated approach to testing and other services targeting multiple diseases on the same population can be feasible and effective in improving accessibility and health outcomes among patients [1]. This integrated approach can be supported by point-of-care resources such as clinical decision support (CDS) system tools that can combine electronic patient health data and up-to-date guidelines and clinical protocols. This paper describes the usability phase of a study on the integration of a CDS tool for managing gonorrhea, screening for HIV, and identifying patients for HIV pre-exposure prophylaxis (PrEP) at Codman Square Clinic.

Gonorrhea was the second most commonly reported sexually transmitted infection (STI) in the United States in 2020, with a total of 677,769 cases reported to the Centers for Disease Control and Prevention (CDC) and has shown a 71.49% (395,216/677,769) increase since 2015 [2]. Among both men and women, untreated gonorrhea can cause serious and painful health problems, infertility, and in rare cases, even life-threatening conditions [3-7]. To mitigate these risks, annual screening is recommended for all sexually active women younger than 25 years and those at increased risk for infection. Timely diagnosis through routine screening and prompt, effective treatment—adhering to up-to-date CDC treatment guidelines—is paramount to both reducing gonorrhea and slowing down the threat of antimicrobial resistance [3,8-10].

HIV causes an infection that attacks the body’s immune system, specifically the cluster of differentiation antigen 4 T lymphocytes [11,12]. If left untreated, it can lead to AIDS, opportunistic infections, malignancies, and death [11]. HIV PrEP has been shown to reduce the risk of acquiring HIV infection [13,14]. As a result, the CDC has developed guidance for prescribing PrEP to individuals considered at high risk, and gonorrhea infection is identified as one of those risk factors [15,16].

Patients with STIs have a 2- to 3-fold increased risk of having concomitant HIV infection compared to those without any STI [17,18]. Gonococcal infections, specifically, have also been associated with increased risk of HIV [18-23]. Gonococcal infections cause immune reactions that, among others, recruit cluster of differentiation antigen 4 cells, which can potentially enhance HIV in vivo replication and facilitate acquisition and transmission of HIV infection [20,24-26]. Previous studies have further advanced our understanding of this syndemic and describe the biological, behavioral, social, and structural determinants and their synergisms [20,22,25]. As a result, it has been a long-standing recommendation to screen patients diagnosed with STIs, including gonorrhea, for HIV [8,27].

This synergistic and potentially concomitant nature of gonorrhea and HIV and the overlap between gonorrhea treatment, HIV screening, and PrEP prescription present us with the opportunity to develop a CDC guidelines–based synergistic intervention tool that could target adequate treatment for gonococcal infections, screening for HIV based on risk factors associated with the diagnosis of gonorrhea, and PrEP prescription to relevant populations.

CDS systems are computational tools that leverage information available in electronic health records (EHRs) to provide person-specific evidence-based information, intelligently filtered, and presented at appropriate times to help inform decisions regarding a patient’s care to improve patient outcomes and lead to higher quality care [28-31]. A systematic review of 160 articles representing 148 unique studies describing CDS implemented across diverse settings found that CDS interventions were efficacious on health care process outcomes, but data on clinical and economic outcomes were sparse [32]. The Community Preventive Services Task Force conducted a systematic review of 23 studies and reported that CDS increased HIV screening for both the general population and people at higher risk for HIV infections [33]. Based on the strong evidence of effectiveness, the Community Preventive Services Task Force has now recommended CDS to increase HIV screening [33]. The Sexually Transmitted Infections National Strategic Plan has also recommended increasing the implementation of CDS to support high-quality sexual health assessments, screen for STIs, integrated care models, and reduce adverse outcomes [34].

Gonorrhea treatment recommendations are updated regularly to represent the latest evidence to provide adequate treatment and curb the rise of antimicrobial resistance [3,8]. Keeping up to date with the latest guidelines could potentially pose a challenge to clinical providers, as evidenced with varying degrees of adherence to gonorrhea treatment guidelines [9,10]. Coupled with the synergistic nature of the infection with HIV [18,20-26], this presents an opportunity to develop a multifaceted CDS intervention to address the needs of the same patient population and improve clinical provider adherence to CDC guidelines on gonorrhea, HIV screening, and providing PrEP for HIV infection, when indicated.

We partnered with OCHIN, Inc, a nonprofit community-based health center–controlled network as the EHR provider [35,36]. We selected OCHIN’s Epic EHR [37] as it serves as the primary EHR to almost 1000 health care sites with 21,000 clinicians in 45 states who serve over 6 million patients [38]. OCHIN provides 1 instance of Epic consisting of enterprise-wide master patient index; patients have a single medical record across all clinics in the network, and all data are managed centrally.

Within the OCHIN network, we partnered with Codman Square Health Center, a Federally Qualified Health Center in Dorchester, Massachusetts, to pilot the CDS intervention [39]. We selected Codman Square Health Center as both patients with gonorrhea and HIV are regularly managed in their clinics and they have a well-established STI-reporting practice in place. Typically, they have over 115,000 client contacts per year.

While all of Codman Square Health Center had access to the CDS solution, a subset of clinicians within the Internal Medicine clinic, specifically, were selected as the focus of the intervention. This subset of clinicians frequently engaged in the screening and treatment of STIs and provide care under the name “X-Clinic” to allow a level of discretion to the patients they serve. The X-Clinic clinicians were selected for the focus of the intervention due to the high rates of gonorrhea infection and the potential for use of the CDS solution during the pilot period. A clinical champion was identified to provide insight and encourage participation among staff along with 2 project leads, 1 representing the providers and 1 for support staff. While the CDS tool was available to the broader Codman Square Health Center, the X-clinic received targeted training and support to use the tool. This training and support were not extended to the rest of Codman Square Health Center.

An internal review by OCHIN, the implementing partner, determined this work as quality improvement and deemed that an institutional review board review was not necessary. A 12-week study period was planned, beginning August 31, 2021, and ending on November 23, 2021. A mixed methods pre-post design [40] was used to measure the feasibility, use, and usability of the CDS solution. During the time of the planned intervention, COVID-19 altered the typical pattern of patients seeking STI-related care [41,42]. As a result, we determined that the 12-week period immediately prior to the intervention period would be the most representative sample of baseline patient data.

In addition to the direct training provided, the X-Clinic clinicians received targeted and continual support on the use of the CDS solution using the practice coaching methodology. This included supporting adaptive skills and learning to build capacity and capability for the care team to effectively use the CDS solution, explore change ideas, provide feedback to support the team’s progress as the team becomes more self-sufficient, and engage with iterative learning. Practice coaching (also known as practice facilitation) has been demonstrated to be effective in successfully implementing tools and new evidence into clinical practice [43-46]. A key part of the intervention design included a dedicated practice coach who regularly engaged with the clinical champion and project leads at the X-Clinic. The coach was part of the project team’s kick-off meeting, which included a warm welcome from the CDS solution developer, project orientation from the project manager, training by the trainer to demonstrate the solution, and an overview of the coaching engagement from the coach. The facilitation meetings were held biweekly for the duration of the project, and they typically included discussions about general assessment and feedback, use, perceived effectiveness, and clinical integration of the CDS solution. When available, CDS solution–related data were reviewed for discussion of key insights, opportunities for improvement, and when a need was identified, the practice coach engaged subject matter experts to provide added support and facilitated referrals for technical and application support.

OCHIN Epic EHR [37] has native tools to develop CDS solutions in a variety of formats, each performing specific functions. For this CDS solution, we developed four tools: (1) a sexual history data capture form using “SmartForm” (Epic’s name for a method for capturing responses to questions in a structured format) and a patient questionnaire with the same questions that can be made available to the patient prior to the visit, (2) a “best practice advisory” (BPA; Epic’s name for rule-based alerts), (3) an enhanced STI order set—that included diagnoses as well as links to documentation templates—using “SmartSet” (Epic’s name for order sets), and (4) a documentation template using “SmartText” (Epic’s name for a method of notetaking that prompts clinician with standard information collected and presented during a visit).

We developed a CDS solution by integrating translated information from 3 separate CDC guidelines, specifically to guide clinicians to choose appropriate therapy for gonorrhea [3,8] and—as per the CDC’s guidelines to prevent or diagnose new HIV infections [47-49] and link those individuals considered at risk to relevant prevention and medical services [15,16]—prompt them to screen for HIV and consider PrEP, if patients met those criteria (Figure 1). Health data standards were used wherever possible; ICD-10-CM (International Classification of Diseases, Tenth Revision, Clinical Modification) was used to evaluate the diagnosis of gonococcal infections.

The Sexual History SmartForm and corresponding patient questionnaire were developed to collect sexual history and STI-related symptoms from patients. The form and questionnaire were structured as point and click questions and answers for ease of use; patients had access to the questionnaire via MyChart [50] (Epic’s patient portal) prior to the visit (Figure 1, step 1; Table 1). At the beginning of the clinic visit, nurse navigators reviewed the information collected using the SmartForm and completed the questionnaire, if incomplete (Figure 1, step 2). Based on CDC’s guidelines [15,16,48], patients were considered to be at risk for HIV infection for the following criteria: (1) positive result or diagnosis for gonococcal infection in the last 6 months; (2) reason for visiting the clinic was related to STI; and (3) sexual risk behaviors, known partner with any STI, and if patient or partner using or sharing needles. If the patient’s response suggested a high risk for HIV infection, the SmartForm algorithm presented further questions to gauge awareness and interest for PrEP (Table 1).

For patients who (1) had a confirmed positive result for gonococcal infection in the last 14 days, (2) had a chief complaint related to gonorrhea (STI symptoms or concern; penile, urethral, rectal, or vaginal discharge and STI-related screening, testing, follow-up, or visit), or (3) were treated presumptively due to a partner with known gonococcal infection, a BPA would alert the clinical provider (Figure 1, step 3) to use the SmartSet to assist in providing standardized clinical care (Figure 1, step 4; Table 1). The SmartSet was designed to (1) recommend laboratory tests to screen for STI, including HIV, (2) provide PrEP prescription options, (3) present CDC guidelines–based recommendations for medication regimens as preferred options to treat gonococcal infections, and (4) provide a link to gonorrhea treatment guidelines for anything beyond uncomplicated gonorrhea and a link to the SmartText (Figure 1, step 5; Table 1). Opening the BPA did not necessarily mean that clinical providers selected diagnoses or ordered laboratory tests or medication directly from the SmartSet but merely that those options were made available for selection and action. Diagnoses, laboratory tests, and medication orders could also be entered for the patient outside of the SmartSet. The SmartText template was developed to prompt the clinical provider to document in a standardized manner (Table 1). The SmartText included common heading for note taking, prepopulated STI-related laboratory results from the patient’s chart, and risk reduction suggestions.

During the intervention period, 12,048 patients were provided with clinical care in Codman Square Health Center and 37 were diagnosed with gonorrhea; similarly, 40 patients out of 11,269 were diagnosed with gonorrhea in the baseline period. The mean age of patients seen in the baseline and intervention periods was 37 (SD 21.46) years and 36 (SD 21.98) years, respectively (P<.001). A 2-sample t test analysis of the age groups revealed that the intervention period included younger individuals compared to the baseline period and cannot be considered similar; however, the largest age group of 25-44 years old remained the same with 30.78% (3469/11,269) in the baseline period and 29.58% (3564/12,048) in the intervention period. Using Fisher exact test, it was determined that the patients in the baseline and intervention periods were comparable regarding gender (P=.25). While ethnicity distribution was found to be similar (P=.56) using Pearson chi-square test, baseline and intervention groups differed significantly by race (χ₁²=4.36; P=.04). Nevertheless, the largest patient population served by Codman Square Health Center was Black or African American in both baseline (9040/11,269, 80.22%) and intervention (9565/12,048, 79.39%) periods.

Throughout Codman Square Health Center, the BPA was triggered 950 (4.07%) times. Within the X-Clinic specifically, the BPA was triggered in one-tenth of all the clinical encounters (348/3451, 10.08%); these 348 encounters represented 300 patients (Figure 2). The SmartForm was opened for about half of the patients (157/300, 52.33%), and in instances when the SmartForm was opened, it was completed 89.81% (141/157) of the time (Figure 2). Of note, while the BPA did not prompt the user to open the SmartForm, the patient cohort where the BPA presented was used for analysis as this represents the target population for this study. For those patients whose responses determined them as high-risk for HIV, further PrEP-related questions were asked (Table 1). Some information about PrEP was known to 63.31% (88/139) of the patients who were asked that question; 6.62% (9/136) of patients reported having taken PrEP and 16.06% (22/137) patients showed interest in PrEP (Figure 2).

Clinical providers could open SmartSet from the BPA (Figure 1, step 4), and this action was taken for about half of the patients (162/300, 54%) for whom BPA was presented. In the target population, the most common diagnosis was “Vulvovaginitis” (ICD-10-CM: N76.0); it was assigned to 34 patients, 21 were assigned to patients for whom BPA was presented, and 13 were assigned where no BPA was presented (Table 3). Diagnoses were assigned, and laboratories were ordered both from BPA-prompted SmartSet and outside of BPA without using the CDS solution (Table 3). No patients were diagnosed with cervicitis but were diagnosed with urethritis, pharyngitis, and proctitis. The most ordered laboratory test was for HIV. More than half (191/348, 54.89%) of the encounters where a BPA was presented received these orders for HIV testing.

In the X-Clinic, SmartText note template was used in most of the clinical encounters (313/348, 89.9%), where the BPA presented. Clinical providers used SmartText extensively (2343/3103, 75.5%) even during encounters where the BPA did not trigger. The SmartText was accessible either through the SmartSet or a standard “quick button” in the documentation area. The clinical site continued to use the CDS tool beyond the study period.

In the intervention period, among patients where a BPA was triggered, there were fewer patients with a diagnosis of gonococcal infections (33/750, 4.4%) as compared to the baseline period (41/649, 6.32%; Table 4). While roughly four-fifths (34/41, 82.93%) of these patients diagnosed with gonorrhea were prescribed antimicrobials for treating the infection, the difference in the number of patients treated in the intervention (26/33, 78.79%) and baseline periods were not statistically significant (P=.65). Individuals in the intervention period had a 26% decrease in odds (odds ratio 0.74, CI: 0.60-0.91) of being screened for HIV (310/750, 41.33%) compared to the baseline period (317/649, 48.84%). PrEP was prescribed 2 times more to patients in the intervention period (12/750, 1.6%) compared to the baseline period (6/649, 0.92%); however, analysis revealed that these numbers not to be statistically significant (P=.26).

Most of the survey responders reported that the CDS solution was not intrusive to their clinical practice workflow (2/4, 50% strongly agreed; 1/4, 25% agreed; and 1/4, 25% was neutral). They also agreed (0/4, 0% strongly agreed; 3/4, 75% agreed; and 1/4, 25% was neutral) that the CDS solution was easy to navigate and provided sufficient time to use it during the patient consultation. They had neutral responses (0/4, 0% strongly agreed; 1/4, 25% agreed; and 3/4, 75% was neutral) about the CDS solution’s ability to present clear recommendations for treatment.

The interview revealed 3 major emerging themes. First, the interviewees opined that the CDS solution provided a faster and more standardized approach for capturing the sexual history and treating the patient and planned to keep using it after the pilot was complete. Second, while the CDS solution improved the efficiency and streamlined their existing processes, the interviewees did not change their treatment or screening recommendations based on the solution. Third, they also revealed that the full scope of the CDS solution was not clear to them and reported discovering components of the solution on their own that they were not aware were part of the CDS solution.

There were 3 key themes related to the features of the CDS solution. First, the BPAs alone would not prompt behavior change and use. Interviewees emphasized the importance of education of new guidelines and changes in the EHR tools from someone they know and trust and believed this would promote adherence from clinicians. Second, there were mixed reviews regarding the use of the logic that determined to present PrEP-related questions to some patients considered at high-risk for HIV. While 1 interviewee noted that the unique focus of X-Clinic to specifically address sexual health–related questions and conditions, all patients should be asked PrEP-related questions and noted confusion over the questions being presented to only some patients and not to other patients. Another interviewee indicated that this logic was helpful for colleagues who are less familiar with PrEP to highlight relevance in populations not typically targeted for PrEP, such as heterosexual patients. Third, there was a general sense that SmartSets were hard to navigate. Multiple interviewees commented that they were unable to find SmartSets without BPAs, and once in the SmartSet, it was not easy to leave and come back if an issue outside the scope of the SmartSet needed to be addressed, such as discussions regarding intrauterine devices in relation to gonococcal infections. One interviewee noted handwriting details to remember after completing a task within SmartSet.

A patient may be seen by multiple providers from more than one provider type category. In the target population (750/12,048, 6.23%) described in Table 4, 32 Doctors of Medicine and Doctors of Osteopathy (MD/DO) provided care for 281 patients and minimally used SmartForms (20/281, 7.12%) but completed most of them once opened (18/20, 90%); 26 advanced practice providers (APPs) provided care to 362 patients, used SmartForms for 20.99% (76/362) of patients, and 81.58% (62/76) completed most of them. In contrast, the “other” category, which included all other clinical staff such as case manager, dentist, laboratory technician, licensed practical nurse, medical assistants, midwives, and social workers, had the highest use, opening SmartForms for half of the patients they served (99/192, 51.56%) and completed most of the forms (83/99, 83.84%). Similarly, “other” category staff administered the highest number of PrEP-related questions, followed by APPs and MD/DOs administered very few. Providers classified as MD/DOs had 309 clinical encounters with 281 patients and opened SmartSets in half of the encounters (157/309, 50.81%) and those classified as APPs had 398 clinical encounters with 362 patients and opened SmartSets slightly lower than half of the encounters (180/398, 45.23%). “Other” staff opened SmartSets the least (82/126, 39.42%), with 10 staff who interacted with 192 patients in 208 encounters.