Pre-exposure prophylaxis (PrEP) with tenofovir disoproxil fumarate and emtricitabine (TDF/FTC) is used for HIV prevention and is a mainstream pillar of comprehensive HIV prevention programs [1-3]. The National HIV/AIDS Strategy established national targets for PrEP used in 2015, calling for a 500% increase in PrEP use between 2015 and 2020 [4]. These targets have been challenging to measure rigorously; specifically, measurement of uses of antiretroviral therapies (specifically, use of TDF/FTC, the only US Food and Drug Administration (FDA)–approved medication for PrEP from 2015 to 2019) has been far from straightforward. TDF/FTC is a medication that is also used for HIV treatment in combination with other antiretroviral medications, can also be used in regimens for postexposure prophylaxis (PEP), and is sometimes prescribed off-label as for the management of chronic hepatitis B (CHB). For these reasons, to monitor the uptake of TDF/FTC for PrEP indication using data from prescription databases, we must first identify apparent prescriptions for TDF/FTC and then rule out prescriptions that were likely made for indications other than PrEP.

Considering methods to identify PrEP indications for TDF/FTC prescriptions relates to a larger trend in the use of prescription data to measure the use of prescription drugs in populations. Large health care use claims databases and electronic medical records (EMRs) are being increasingly used in pharmacoepidemiology studies [5-7]. There are limitations to such approaches because health care use claims data are primarily used for billing and do not record specific indications. To predict the intended indication for medications like TDF/FTC, for which multiple indications exist, validated algorithms can be used to identify a specific indication with sufficient confidence for use in surveillance. In these algorithms, additional data related to the prescription, such as patient characteristics of interest in the form of codes, procedures, or laboratory results, can help to predict the intended indication. Algorithms using claims data should be developed and validated to support their use for monitoring the use of prescription drugs in populations. Because individuals prescribed TDF/FTC can be identified through commercial claims databases but PrEP indications are not directly available from these databases, we aimed to develop and validate an algorithm of exclusion criteria based on diagnosis indicators.

We report on the development and validation of an early algorithm to identify which people prescribed TDF/FTC were likely PrEP users. We conducted 2 independent, methodologically distinct validation studies in different populations and with different gold standard methodologies to evaluate the performance of the algorithm. Our results have implications for the development and assessment of algorithm-based estimates of PrEP use [13-15]. Data on PrEP use developed using this or similar algorithms have been used for diverse public health purposes, including tracking trends in PrEP use [15,16], conducting ecological analyses of associations between PrEP prescriptions and HIV infection outcomes [17], evaluating policies related to PrEP uptake [18], and evaluating progress of jurisdictions toward programmatic targets of PrEP uptake [18]. We report the data on this historical algorithm to document the development of what later reports referred to as the “Gilead algorithm” [19]. Presenting these data help complete the documentation of iterative process by which current algorithms have been developed. Its crucial contribution was to propose an algorithm that started with an inclusive list of patients receiving TDF/FTC and to then remove patients with other plausible indications. This general framework is still in use today for PrEP agents that can also be used as therapy for HIV infection or hepatitis B infection.

An evaluation of a similar algorithm was published using data from a single medical center in New York, comparing 3 algorithms in that setting; the authors noted that the publication of such assessments was important [19]. The previous evaluation found high sensitivity and specificity of this algorithm (identified in that paper as the “Gilead algorithm”) in the setting of a large health care provider in New York [19]. Importantly, the performance of an algorithm such as that described here, and the alternative algorithms described by Furukawa et al [19], is that they depend on the quality of information available through medical records. Our validation extends other published evaluations by applying this algorithm in medical settings a different urban area; in a clinic setting more focused on sexual health and HIV prevention and care; and to a national EMR data source, which is more like what is being used in practice to produce estimates of PrEP use. Furthermore, the predictive value of PrEP classifications will vary according to the prevalence of the condition; our report also provides predictive value of negative and positive tests in different geographic settings than the New York study. Indeed, our estimates of PPV and NPV were somewhat different than the application of the “Gilead algorithm” by Furukawa et al [19].

Our results indicated that the PrEP algorithms both had high sensitivity but had more modest and more variable specificity compared with gold standard indications of PrEP use. The modest and variable specificity of the algorithm as measured in the 2 evaluations was somewhat surprising, because the algorithm was constructed to be an approach of exclusion. In this setting, any patient record with an indication of at least 1 competing indication for TDF/FTC prescription was considered not to be a PrEP user; however, both approaches had substantial proportions of false positive results. The sensitivity of the algorithm (both 99%) was higher than the specificity in both evaluations (50% and 87%), and in both settings, the number of PrEP users suggested by the algorithms was higher than the actual number of PrEP users.

The high sensitivity of the algorithm also suggests that estimates of PrEP use made using the algorithm should be considered as highly inclusive estimates of TDF/FTC use for PrEP, because nearly all true PrEP prescriptions are classified as PrEP indications by the algorithm. However, the results of the evaluation in the 2 different settings suggest that the algorithm performance might result in information bias for the national estimates of PrEP use [20], for both algorithms. In both algorithms, there were some misclassifications in both directions, but the number of misclassified cases indicating PrEP uses when TDF/FTC was indicated for other purposes was substantial. Thus, both algorithms substantially overestimated PrEP users, and the predictive value of algorithm-identified PrEP use was low, especially for the EMR study. Even in the case of the clinic-based study, for which specificity was quite high, the high specificity was applied to a much larger pool of non-PrEP users, resulting in substantial numbers of false positive results. This latter scenario is likely most akin to the application of such algorithms to large commercial pharmacy databases [21]. We believe that the applications of the algorithm to national prescription datasets will result in performance more like the EMR validation set, rather than the community clinic dataset, because the national prescription data reflects the prescribing patterns across a range of practice types like the commercial EMR data used in that validation. In any case, sensitivity analyses should be considered for ongoing national estimates of PrEP prescription to characterize uncertainty in estimates attributable to algorithm performance, and the algorithm performance characteristics we report here can be used for that purpose. Because we have reported the performance of the algorithm, including CIs, plausible ranges of PrEP users could be calculated when needed.

Our study had important strengths and limitations. In terms of strengths, both studies used rigorous methods to identify gold standard data on PrEP use. Reviews were comprehensive, considering multiple aspects of medical records, and record reviews were conducted independently by 2 clinician investigators (RMG and SB). Two different approaches and clinical settings were used. The first study used a gold standard list created by the investigators by reviewing each subject using a graphical patient profile. The second study used administrative records from within the clinic to classify patients as PrEP users; by using a smaller, highly circumscribed patient population where the authors knew that PrEP was being administered, a “gold standard” disposition was available for all patient records (vs a small sample in the EMR study, since that required new data collection and decision making on PrEP status for each record).

Despite these strengths, our study also had important weaknesses. Our gold standard data were subject to information bias through misclassification; for example, in the first EMR-based study, the clinical reviewers might have had incomplete or inaccurate data on diagnoses or procedures that led them to incorrectly classify a non-PrEP user as a PrEP user (eg, missing HBV [hepatitis B virus] diagnosis), or to classify a PrEP user as a non-PrEP user (eg, mistakenly coded chronic HBV diagnosis). The EMR data source was likely limited by the missingness of some data to exclude non-PrEP indications for TDF/FTC prescription. For example, some patients identified as PrEP users might have been identified on chart review as false positives for PrEP use because they were patients in care for HIV who had incomplete medication or diagnosis data in their medical records. For the clinic study, it is possible that PrEP user status was misclassified (eg, a patient who was on PrEP but was subsequently diagnosed with HIV infection). Similarly, the algorithm relied on relevant codes to classify indications, and missing or miscoded data could have resulted in incorrect categorization in either direction. Miscoding can be substantial [22], including for infectious viral hepatitis outcomes [16,18,23-27]. The relative performance of the algorithms in the 2 evaluations was also affected by the types of data included in the datasets. The EMR validation study had a population that comprised a larger proportion of patients with HIV who were easily and correctly classified; the community health clinic study had a lower ratio of PrEP to patients with HIV because of the nature of the programs offered in the clinic. The data were from 2012 to 2014, and current prescribing practices have likely changed. During the period of the evaluation (specifically the early part of 2012), TDF/FTC was not approved for PrEP use, and off-label use would have lacked appropriate Centers for Medicaid and Medicare Services coding. This research was conducted with ICD-9 codes, which were the most current at the time of the research; confirmation of validation and remapping to ICD-10 codes will be required.

The results presented here are historical, and prescription practices have almost certainly evolved since the data were collected. Despite this, significant aspects of the methods used here are currently in use by both Centers for Disease Control and Prevention [28] and AIDSVu [29] for production of national estimates of PrEP use. These data are being used widely for public health relevant analyses of PrEP [30], and we therefore believe it is important to have a peer-reviewed, transparent documentation of these methods. We recognize that both the medications used for PrEP and the prescriptions patterns issued for PrEP have evolved since this work was conducted and will continue to evolve in the future [28]. Specifically, we recognize the need to develop improved methods to understand how to characterize the use of TDF/FTC as event-driven PrEP [29], to incorporate new formulations of PrEP [30], and to prepare for increased use of long-acting injectable PrEP [31]. There is a need to update the ICD-9 codes used in this evaluation to contemporary ICD-10 codes. To date, we have implemented further changes to the algorithm and provided notes as data methods on the AIDSVu website to account for changes in agents and PrEP guidelines, such as the addition of emtricitabine and tenofovir alafenamide [30] and injectable cabotegravir [31,32]. The AIDSVu link will be maintained as a live repository of current methods used for AIDSVu public use datasets. Given the increasing complexity of agents and prescribing patterns, we identify the need for resources and a consensus and living approach to developing a shared case definition for PrEP use for public health monitoring purposes and tracking of progress toward national prevention priorities.

The algorithm described here is primarily of use for monitoring of state and national progress toward prevention goals using data from national prescription databases [21,22,33]. Clinics might have access to additional clinical data that would help improve such an algorithm for assessment of clinic progress in improving PrEP programs.

Validation studies conducted with 2 different methodologies demonstrate that an algorithm of exclusion using ICD-9 codes, procedure codes, and concomitantly prescribed medications can meaningfully identify PrEP indications in medical claims databases and EMRs, with some defined occurrence of misclassification. The validation provides an important bridge between studies of PrEP uptake and the ability to evaluate PrEP use using real-world data. We recommend that public health researchers using algorithms to estimate numbers of PrEP users conduct sensitivity analyses using the performance data from these studies to evaluate the impact of algorithm-based classification in specific clinical or data settings. Given the complexities of this task, machine learning could be considered for ongoing evaluation of the best ways to monitor PrEP use; such methods have already been applied to identifying potential PrEP indications [34-37]. Whatever methods are used now or in the future, validations should be updated as prescribing patterns change, for new PrEP formulations already available [30], and as new options for PrEP come onto the market [38].