A 3-Tier AI Model for COVID-19 Triage Using Pharyngeal Images: Algorithm Development and Validation

Introduction

SARS-CoV-2 continues to circulate globally and remains a persistent cause of acute respiratory illness among patients presenting with respiratory symptoms [1,2]. Differentiating SARS-CoV-2 from other common viral infections, including influenza and adenovirus, remains challenging because of substantial symptom overlap and its diverse systemic and neurological manifestations [3-5]. Accordingly, when decisions regarding SARS-CoV-2 testing are based solely on symptoms, triage may fail to identify some individuals with COVID-19 for testing while unnecessarily referring low-risk individuals for testing, resulting in increased patient burden and inefficient use of health care resources [6-12]. Although adding information from blood tests or computed tomography can improve predictive accuracy [13,14], lower-burden sources of additional signal are preferable. While traditional nasopharyngeal swabbing remains the standard clinical procedure for COVID-19 testing, its invasive nature and procedural discomfort, combined with the time required for results, can create bottlenecks in high-volume triage settings.

Recent case series and cross-sectional studies have suggested that pharyngeal findings may be observed in patients with COVID-19 [15-19]. Although this examination is simple, its interpretation is subjective and varies with the clinician’s experience. Standardization is also required to mitigate patient-specific factors such as macroglossia, limited mouth opening, or a strong gag reflex, which can hinder consistent visual assessment. Minimizing this variability and standardizing assessment are therefore warranted.

To address these issues, we sought to leverage a recently approved artificial intelligence (AI)–powered medical device which acquires pharyngeal images and integrates clinical information for influenza diagnosis [20]. Building on the existing workflow, we combined pharyngeal images with routine clinical data to estimate the likelihood of SARS-CoV-2 infection and developed an adjunct AI model (“COVID-19-AI”) that outputs high, medium, or low suspicion. This system is designed as a tool to facilitate clinical decision-making regarding whether further invasive SARS-CoV-2 testing should be prioritized or may be deferred (Multimedia Appendix 1). In this study, we evaluate the performance of this 3-tier risk stratification and its clinical utility in optimizing triage workflows.

Methods

Study Design and Participants

We conducted a noninterventional performance evaluation using a multicenter, prospectively collected observational dataset from 26 medical institutions in Japan (December 2023 to March 2024; registered in the UMIN Clinical Trials Registry: UMIN000052896). These institutions were distributed nationwide across 5 geographic regions (from Hokkaido to Kyushu), spanning internal medicine (n=11, 42%), otolaryngology (n=8, 31%), and pediatrics (n=7, 27%). Detailed characteristics are summarized in Multimedia Appendix 2.

The inclusion criteria were as follows: patients with suspected influenza or COVID-19 who underwent testing for both conditions; provided written consent (self or surrogate); met ≥1 of the following clinical triggers—fever ≥37.0 °C, systemic symptoms (eg, joint pain, muscle pain, headache, tiredness, and appetite loss), respiratory symptoms (eg, cough, sore throat, and nasal congestion), or clinician-judged suspicion of influenza or COVID-19 (eg, close contact); and were aged ≥6 years at consent. The exclusion criteria included impaired consciousness or breathing disorders; moderate or severe gingival or dental disease or loose teeth that may be aggravated by the research equipment; trauma, damage, or surgery to the maxillofacial, dental, oral, or pharyngeal regions that might be aggravated by the device; recurrent vomiting; or investigator-determined ineligibility.

Pharyngeal images and clinical information from 2133 patients were used to train a COVID-19-AI model (Multimedia Appendix 1). To evaluate the model, 700 independent patients were enrolled. Of these, 696 (99.4%) were included in the full analysis set after 4 records were excluded due to out-of-window entry dates.

Collected Variables

In addition to pharyngeal images (Figure 1), the following clinical variables were obtained: age, sex, race, vaccination history, close contact history, time from the first symptom onset, use of antipyretics, vital signs at the visit (body temperature, pulse rate, and oxygen saturation), presenting symptoms (joint pain, muscle pain, headache, tiredness, appetite loss, chill, sweating, cough, sore throat, nasal discharge, nasal congestion, respiratory inflammation, abdominal pain, vomiting or diarrhea, sputum production, dyspnea, smell disorder, and taste disorder), and pharyngeal examination features (tonsillar white moss, redness, swelling or exudates, and tender anterior cervical lymphadenopathy). Furthermore, reverse transcription polymerase chain reaction (RT-PCR) and rapid immunochromatographic antigen detection test results for both influenza and COVID-19 were obtained. To ensure a standardized reference across all 26 sites, centralized RT-PCR was performed using residual liquid from rapid antigen test kits, which was frozen and transported to a central laboratory (Mediford Corporation, Tokyo, Japan) for analysis. The RT-PCR result for COVID-19 served as the reference (gold) standard for the diagnosis of SARS-CoV-2 infection. While the initial AI model was trained using high-quality positive labels defined by dual positivity (both RT-PCR and rapid antigen test positive)—with discordant results categorized as negative to minimize noise from cases with low viral loads—the final performance evaluation strictly followed the RT-PCR–confirmed status as the sole reference in accordance with current clinical needs and consultations with the Pharmaceuticals and Medical Devices Agency. In this evaluation, all RT-PCR–positive cases, including those with discordant antigen test results, were categorized as true positives to rigorously validate the model’s utility in real-world clinical settings. All clinicians were blinded to the AI model’s output during the prospective data collection phase to ensure that physical examinations and RT-PCR testing decisions remained independent of the model’s predictions.

‎

Outcomes and Analysis

The outcome measure was the diagnostic accuracy of COVID-19-AI against the RT-PCR reference, expressed as sensitivity and specificity. The COVID-19-AI produces a 3-tier output (high, medium, or low) from pharyngeal images and clinical variables (Multimedia Appendix 1). Performance was evaluated under 2 prespecified operating criteria: inclusive (high or medium=positive and low=negative) and strict (high=positive and medium or low=negative). In a subanalysis, accuracy was further examined across time from symptom onset using 12-hour strata up to 72 hours, with RT-PCR–referenced sensitivity and specificity estimated under both operating criteria.

Ethical Considerations

This study was conducted according to the guidelines of the Declaration of Helsinki and was approved by the central institutional review board of Nihonbashi Sakura Clinic (approved on November 22, 2023, for prospective data collection and on August 21, 2024, for performance evaluation). Written informed consent for research participation and reporting was obtained from all participants or their legally authorized representatives. Patient data were anonymized at the study-ID level prior to transfer to the analysis team. Each participant received a ¥1500 (¥1=US$0.006 as of June 15, 2026) gift card as compensation for their participation. Furthermore, we ensured that no individual participant is identifiable from the images or any other information presented in this manuscript.

Results

Patient Characteristics

Among 696 patients included in the validation of the COVID-19-AI model, 48.7% (n=339) were male, and the mean age was 32.2 years (SD 19.1 years; Table 1). The mean time from symptom onset to presentation was 35.9 hours (SD 30.7 hours). The mean visit temperature was 37.45 °C (SD 0.96 °C), and the mean oxygen saturation was 97.9% (SD 1.3%). All participants were of Asian race. By RT-PCR, 247 (35.5%) patients tested positive for COVID-19 and 449 (64.5%) tested negative.

Main Analyses

Among the total 696 patients, the COVID-19-AI categorized 12.4% (n=86) as “high” suspicion, 72.8% (n=507) as “medium” suspicion, and 14.8% (n=103) as “low” suspicion.

Under the inclusive operating criteria (high or medium vs low), sensitivity was 93.9% (95% CI 90.4%‐96.4%) and specificity was 19.6% (95% CI 16.1%‐23.5%). Thus, the inclusive criteria captured nearly all RT-PCR–positive COVID-19 cases, consistent with a sensitivity-oriented triage role yielding a high negative predictive value (85.4%, 95% CI 77.6%‐91.3%), which is highly effective for ruling out COVID-19 when a low suspicion output is returned (Table 2).

Under the strict criteria (high vs medium or low), sensitivity was 24.7% (95% CI 19.6%‐30.4%) and specificity was 94.4% (95% CI 92.0%‐96.3%). As anticipated, the strict criteria markedly increased specificity, resulting in a positive predictive value of 70.9% (95% CI 60.7%‐79.8%). This makes the high-suspicion output useful for prioritizing infection control measures (Table 2).

Subanalysis: Time-Since-Onset Stratification

With the inclusive criteria, sensitivity remained high (≥92.0% across all 12-hour bins), while specificity was relatively low (approximately 20%). Under the strict criteria, specificity remained high (≥83.3% across all bins) at the expense of lower sensitivity (approximately 25%; Table 3 and Table 4). Neither set of criteria showed a pronounced trend with onset time.

Discussion

Principal Findings

This multicenter evaluation demonstrates that a 3-tier COVID-19-AI model embedded in the AI-powered pharyngeal camera workflow can deliver complementary operating behaviors: very high sensitivity (93.9%) under the inclusive criteria, with the intended use of ruling out COVID-19 as a triage tool, and very high specificity (94.4%) under the strict criteria, with the intended use of guiding immediate infection control measures (Table 2). Stability across 12-hour onset strata suggests robustness to symptom chronology, a known determinant of index test performance in infectious disease triage [21,22]. Feature importance analysis indicates that incorporating pharyngeal images materially improves the prediction of SARS-CoV-2 infection, with pharyngeal image information providing greater predictive value than any single clinical variable (Figure S1 in Multimedia Appendix 1). Furthermore, an ablation study confirmed that the integrated model using both pharyngeal images and clinical information (area under the receiver operating characteristic curve [AUROC] 0.78) significantly outperformed models using only clinical information (AUROC 0.75; P<.001) or only pharyngeal images (AUROC 0.71; P<.001; Figure S2 in Multimedia Appendix 1). The diagnostic value of pharyngeal imaging is corroborated by recent evidence demonstrating the high accuracy of deep learning models in diagnosing respiratory infections and evaluating upper airway conditions [23-26].

Clinical Utility of AI-Driven Triage

Estimating pretest probability from a stratified 3-tier output allows for clear, action-oriented clinical decision-making. By providing instantaneous, noninvasive risk stratification upon a patient’s arrival, this tool enables the immediate categorization of patients, thereby streamlining the diagnostic pathway. Cases with the “low” output, supported by the high sensitivity of the inclusive criteria (93.9%), could be appropriately considered as COVID-19 excluded without additional testing (eg, rapid antigen testing or RT-PCR). Furthermore, our simulation across different COVID-19 prevalence settings demonstrates that the inclusive criteria maintain an exceptionally high negative predictive value (96.7%‐98.4% at 5%‐10% prevalence; Multimedia Appendix 3). This indicates that in general screening populations where COVID-19 is less common, a “low” suspicion result provides a highly reliable rule-out signal. Such a high degree of safety allows clinicians to confidently prioritize other differential diagnoses for low-risk patients, optimizing patient comfort. Conversely, cases with a “high” suspicion output, supported by the high specificity of the strict criteria (94.4%), enable immediate routing for infection control. Although the positive predictive value (70.9%) may not be sufficient for a definitive diagnosis, this high-suspicion signal provides a pragmatic basis for presumptive isolation, substantially enhancing facility safety compared to universal testing.

Resource Stewardship and Implementation

From a practice and resource stewardship standpoint, because medical and human resources are finite, it is not always feasible to perform RT-PCR testing in all potentially eligible patients. In our study, the inclusive criteria resulted in a 14.8% reduction in the testing burden compared to universal testing. While the “high” and “low” categories offer decisive signals, 72.8% of cases are labeled as “medium” suspicion, reflecting the reality of intermediate-risk populations in clinical triage. These medium cases require clinician judgment for additional testing based on clinical priorities and local testing capacity. Specifically, testing should be prioritized for the medium cases where tolerance for false negatives (ie, missed cases) is low. Conversely, testing may be deferred or excluded when patient burden, cost, or local testing capacity are major constraints. Thus, the 3-tier score serves as a decisive reference criterion to support rapid clinician judgment in dynamic workflows. Notably, this adjunct AI model has recently received regulatory approval (October 2025) from Japan’s Ministry of Health, Labour and Welfare (30400BZX00101000) for integration into the NODOCA device (Aillis, Inc). In this approved implementation, the high, medium, and low outputs are officially positioned as “detected,” “detection suspended,” and “undetected,” respectively.

Advantages Over Nasopharyngeal Swabbing

Regarding patient experience and procedural quality, clinical data using this pharyngeal camera confirmed that image capture at the pharynx resulted in significantly less procedural pain than nasopharyngeal swabbing (median numerical rating scale 1 vs 3), supporting its use as a lower-burden diagnostic pathway [27]. Consequently, triaging out low-risk presentations can reasonably reduce patients’ exposure to the discomfort—and occasional complications—associated with nasopharyngeal swab testing. Furthermore, this noninvasive and instantaneous approach eliminates the procedural exposure risks and the mandatory turnaround time inherent to rapid antigen testing. Notably, previous research using the same pharyngeal camera for influenza diagnosis demonstrated that the AI system was effective in identifying positive cases at an earlier stage of illness than rapid antigen testing [28]. This might reflect a fundamental difference in diagnostic methodologies: conventional antigen tests detect pathogen viral load, whereas this pharyngeal AI evaluates the host immune response. Consequently, this AI-powered pharyngeal triage may serve as a valuable complement during the early phase of infection when antigen sensitivity is limited, although larger-scale validation for COVID-19-AI is required given our study’s limited early-onset sample size. Moreover, sampling quality and subsequent positivity rates have been shown to vary with collector experience [29], suggesting that narrowing test eligibility allows limited expert staff to focus on higher-yield cases, which may in turn help reduce the risk of false negatives.

Limitations

Our study has some limitations. First, this study was conducted exclusively in Japan across 26 institutions, and the evaluation population was entirely Asian. As pharyngeal anatomy and mucosal inflammatory responses may vary across ethnic groups and clinical practices differ by country, the generalizability of our results to non-Asian populations requires caution and independent validation. Second, we did not collect several potentially informative predictors for confirming true SARS-CoV-2 infection, which may have limited the model’s performance. For example, our AI-assisted pharyngeal camera is not designed to capture tongue images, even though emerging evidence indicates that tongue imaging features can contribute to predictive performance [30]. Third, our data split was performed at the patient level, meaning the 26 institutions were shared between the training and test sets. While this allowed the model to incorporate diverse clinical environments (eg, lighting and backgrounds), future studies in entirely independent institutions are needed to further confirm its generalizability. Fourth, the use of 10 different rapid antigen test kits and varying sampling techniques across 26 institutions may have introduced intersite heterogeneity; however, such diversity reflects real-world clinical settings. Although centralized RT-PCR was used to provide a standardized reference, the use of frozen residual liquid from the rapid antigen test kits may have impacted analytical sensitivity and potentially led to the misclassification of cases with very low viral loads. Fifth, the small sample size in the late-onset window (eg, n=6 for 60‐72 hours) results in wide CIs, and our findings regarding performance stability should be interpreted with caution. Sixth, patients aged <6 years were excluded due to the difficulty of maintaining stable pharyngeal positioning for imaging, which limits the generalizability of the model to younger pediatric populations. Finally, as the study coincided with the predominance of the Omicron JN.1 lineage in Japan, the model’s performance should be monitored for potential changes in pharyngeal manifestations with future variants.

Conclusions

In conclusion, the 3-tier COVID-19-AI facilitates clinical triage by stratifying patients into 3 actionable risk categories: high, medium, or low suspicion—formally designated as COVID-19 detected, suspended, or undetected, respectively. Our findings demonstrate that this noninvasive tool can achieve high sensitivity for ruling out infection and high specificity for ruling it in, potentially reducing the testing burden and procedural risks. We anticipate that widespread clinical adoption of this triage model will streamline infection control workflows and support clinicians in optimizing diagnostic resource allocation.