This prospective, observational, real-world study employed the RE-AIM evaluation framework to preliminarily evaluate the effectiveness and feasibility of implementation strategies designed to integrate AI-assisted diagnostic software into China’s HMS, following 3 months of implementation. The process evaluation focused on identifying practical challenges, successes, areas for optimization, and the potential to enhance patient outcomes and clinical workflows.

Centralized ethical approval for this study was obtained from the Ethical Review Board of the China Rehabilitation Research Center (approval number: 2024-032-1). Written informed consent was obtained from all patients involved in the referral process. Patient privacy and confidentiality were strictly protected. All potentially identifying information (including names, initials, and patient identification numbers) was removed unless essential for scientific purposes. No identifiable participant information appears in the manuscript or supplementary materials. Participation was entirely voluntary, and no compensation was provided to participants in this study.

The implementation process was illustrated in Figure 1, where the dotted blue line represented patient flow and the solid green line denoted data flow.

The secondary hospital served as the primary site for the implementation of AI-assisted diagnostic software and functioned as the first point of care for patients with suspected pulmonary nodules. Eligible patients presenting with symptoms or undergoing routine cancer screening received an initial clinician consultation, followed by a low-dose CT scan if indicated. The AI-assisted diagnostic software, locally deployed at the secondary hospital, analyzed deidentified imaging data, and the results interfaced directly with the PACS.

When indeterminate cases were identified, physicians at the secondary hospital were responsible for uploading relevant referral information to the ESDR tool, which included:

At the tertiary hospital, a structured referral pathway was established to facilitate the comprehensive evaluation of patients with indeterminate pulmonary nodules. Supported by the ESDR tool, the referral mechanism ensured continuity of care and seamless data flow between institutions. To enhance access to tertiary care, a free registration system was introduced, allowing referred patients to bypass additional registration fees. Moreover, a facilitation team was established to streamline the referral process, assist with appointment scheduling, and provide guidance on accessing tertiary care and navigating the registration system.

Following specialist consultations, the tertiary hospital communicated diagnostic results and follow-up recommendations back to the secondary hospital through the ESDR tool, fostering a continuous feedback loop and building trust in the adoption of AI-assisted diagnostic software.

The integrated strategies were deployed over a 3-month period, from August to November 2024. Following this deployment phase, a transition period was initiated, during which on-site training sessions were held at both hospitals to familiarize physicians with the AI-assisted diagnostic software and the ESDR tool. Concurrently, both systems underwent extensive testing and fine-tuning to ensure effective deployment, with technical challenges addressed and data-sharing protocols refined. Official data collection began in December 2024.

 The study included both physicians and patients. Eligibility criteria for physicians were as follows:

Patients seeking care or undergoing cancer screening at the secondary hospital were assessed for eligibility for AI-assisted interpretation of CT scans based on the following criteria:

Patients with indeterminate findings were subsequently referred to the tertiary hospital for further evaluation based on specific criteria, including:

As a widely recognized framework for evaluating health care interventions, RE-AIM examines 5 dimensions, including Reach, Effectiveness, Adoption, Implementation, and Maintenance, to support the translation of innovations into routine care [46-48].

Given that the current study was designed to assess feasibility and short-term outcomes to inform further refinement of the implementation strategies, the Maintenance dimension was not included. By addressing the remaining dimensions, the RE-AIM framework enabled a multilevel evaluation of the implementation strategies across key stakeholders, including patients, physicians, and health care institutions, providing a comprehensive perspective on their impact. This multidimensional evaluation provided a holistic understanding of the feasibility and effectiveness of the implementation strategies for AI-assisted diagnostic software in pulmonary nodule diagnosis within real-world health care settings, while also identifying opportunities for future optimization and potential scalability.

The specific evaluation indicators are shown in Table 4.

During the study period, deidentified clinical data were prospectively collected for all patients who met the eligibility criteria for pulmonary nodule screening between December 1, 2024, and March 1, 2025. For patients who met the referral criteria, core referral-related data were collected upon obtaining written informed consent and securely transmitted to the ESDR cloud platform to support subsequent diagnostic evaluation and clinical decision-making.

Implementation outcome data were primarily sourced from the hospital information system (HIS), PACS, AI-assisted diagnostic software, and the ESDR tool. To enhance the comprehensiveness and multidimensionality of the data, key feedback information from participating physicians was also collected through electronic questionnaires (Multimedia Appendix 3).

Historical control data were obtained from the same hospital between December 1, 2023, and March 1, 2024. Key outcome indicators included patient representativeness, differences in pulmonary nodule detection rates, and variations in CT image interpretation and reporting times. These indicators were generally stable over time. Given the consistency within the same health care institutions in terms of diagnostic criteria, diagnostic technologies, and the demographic characteristics, disease staging, and severity of the served populations, the use of a historical control group was deemed comparable.

Descriptive statistics for qualitative variables were reported as frequencies and percentages. For quantitative variables, the mean (SD) and median (IQR) were calculated. To compare baseline characteristics between the study participants and historical controls, independent t tests were used for continuous variables, while chi-square tests were applied for categorical variables. All statistical tests were 2-sided, and statistical significance was set at P less than .05.

At the provider level, physicians from relevant clinical departments at both the secondary and tertiary hospitals were invited to participate in the study. Specifically, 6 radiologists and 9 respiratory physicians from the secondary hospital, along with 13 respiratory physicians from the tertiary hospital, were approached. Of those invited, all 15 physicians from the secondary hospital and 8 respiratory physicians from the tertiary hospital agreed to participate. The demographic and professional characteristics of the participating physicians from both institutions are summarized in Table 5.

At the patient level, a total of 1375 patients underwent chest CT examinations at the secondary hospital during the study period. The mean age of patients was 54 (SD 16.6) years, and 604 (43.9%) were male. The mean maximum diameter of the dominant pulmonary nodules detected was 5.4 (SD 2.3) mm, with 953 (69.3%) classified as solid nodules. Compared with the historical control group, there were no statistically significant differences in patient demographics or clinical characteristics of pulmonary nodules (Table 6).

Of the patients who underwent chest CT examinations, 84 (6.1%) were excluded based on the predefined exclusion criteria, including 76 patients under 18 years of age and 8 patients with a history of prior lung surgery. Among the 1291 patients who met the inclusion criteria, 85.6% of chest CT images were successfully analyzed using the AI-assisted pulmonary nodule diagnostic software.

During the study period, 33 patients met the predefined referral criteria. Of these, 25 (75.8%) patients provided written informed consent and were referred to the tertiary hospital. The median age of the referred patients was 63 (IQR 32‐66) years, and 44% (11/25) were male. The median maximum diameter of the dominant pulmonary nodules was 9 (IQR 8‐11) mm, with 28% (7/25) identified as solid nodules. Additionally, 32% (8/25) of referred patients had a history of other pulmonary diseases, 8% (2/25) had a history of cancer, and 24% (6/25) reported a history of smoking (Table 7).

At the provider level, a total of 896 patients at the secondary hospital were identified with at least 1 pulmonary nodule, including micronodules, small nodules, intermediate-sized nodules, and mass-like lesions, resulting in an overall detection rate of 65.2%. The mean time for radiologists to assess and report chest CT images was 24.1 (SD 13.5) minutes. In comparison, during the historical control period, the overall pulmonary nodule detection rate was 32.4% (P<.001), and the mean CT image assessment and reporting time was significantly longer (30.4, SD 19.7 min; P<.001).

At the patient level, of the 25 individuals who consented to referral, 22 successfully completed the referral process by visiting the tertiary hospital, yielding a referral completion rate of 88%. The median interval from referral submission to hospital visit was 3 (IQR 2‐4) days. The 3 patients who did not complete the referral included 2 males and 1 female, with a median age of 67 years. The median maximum diameter of their dominant pulmonary nodules was 13 mm, and 1 had a documented history of cancer.

At the secondary hospital, an electronic questionnaire (Multimedia Appendix 3) was distributed via email and QR code to 9 respiratory physicians and 6 radiologists who participated in the study. All 15 physicians completed the questionnaire, yielding a response rate of 100%. According to the survey, 7/9 (77.8%) of respiratory physicians and all radiologists had used the AI-assisted pulmonary nodule diagnostic software, while 5/9 (55.6%) of respiratory physicians reported using the ESDR tool. In terms of perceived alignment with institutional goals, all respondents agreed that the AI-assisted diagnostic software supported the hospital’s core objectives, and 6/9 (66.7%) of respiratory physicians believed that the use of the ESDR tool was consistent with those objectives.

At the tertiary hospital, all 22 referred patients who arrived were successfully received, resulting in a referral acceptance rate of 100%. Among the 8 participating respiratory physicians, the electronic questionnaire also achieved a 100% response rate. Of these respondents, 5/8 (62.5%) reported prior use of the ESDR tool, and 7/8 (87.5%) indicated that its use aligned with the hospital’s strategic priorities.

In terms of usability, 11/13 (84.6%) of radiologists and respiratory physicians at the secondary hospital who had used the AI-assisted diagnostic software reported that it was easy to use. In contrast, among respiratory physicians who had experience with the ESDR tool, only 3/5 (60%) at the secondary hospital and 4/5 (80%) at the tertiary hospital perceived the tool as user-friendly.

Although both the AI-assisted diagnostic software and the ESDR tool achieved 100% training coverage across the two hospitals, the completeness of referral data collected via the ESDR tool remained suboptimal. Only 28% (7/25) of referred cases had complete data entries, with comprehensive datasets available for 7 patients. Missing information was primarily concentrated in key fields, including clinical symptoms, diagnostic conclusion, and referral-related variables such as insurance information. Despite a relatively high CT image transmission success rate of 90.9% (20/22), system-level failures persisted. Among the 22 patients who successfully completed the referral process, two experienced data transmission failures due to PACS system malfunctions at the secondary hospital, which prevented the upload of AI-generated results and CT images to the ESDR platform. Consequently, these patients were required to carry physical CT films to the tertiary hospital for further diagnostic evaluation.

With respect to integration into clinical workflows, 93.3% (14/15) of physicians at the secondary hospital reported that the AI-assisted diagnostic software did not interfere with routine clinical practice. However, 55.6% (5/9) of respiratory physicians indicated that the ESDR tool imposed a substantial burden on daily tasks. The primary challenge was the lack of interoperability between the ESDR system and the hospital’s internal information systems, which required physicians to manually access the platform via an external web link to upload or retrieve patient data. This additional step increased workload and disrupted workflow efficiency. As a result, continuity of care was affected. Among the 9 respiratory physicians at the secondary hospital, only 44.4% (4/9) reviewed the final diagnostic results of referred patients via the ESDR tool.

At the tertiary hospital, the referral coordination team successfully facilitated the referral process for all 22 patients, achieving a referral completion rate of 100%. Nevertheless, only 50% (4/8) of respiratory physicians uploaded final diagnostic results through the ESDR system. Moreover, 62.5% (5/8) reported workflow disruptions due to challenges similar to those in the secondary hospital, particularly the reliance on external network access, which increased complexity and time demands.

This study designed, implemented, and evaluated an integrated strategy specifically tailored to support the effective implementation of AI-assisted diagnostic software within the context of China’s HMS. In contrast to prior studies that primarily focused on identifying implementation barriers [49-53], this study developed a context-specific implementation framework and demonstrated tangible improvements in both diagnostic capacity and referral processes at the secondary care level. During the study period, 85.6% of chest CT images at the secondary hospital were analyzed using AI-assisted diagnostic software. As a result, the overall detection rate of pulmonary nodules significantly increased from 32.4% in the historical control group to 65.2% in the intervention group (P<.001), indicating a significant enhancement in diagnostic capacity. In addition, the introduction of a standardized referral pathway supported high levels of referral compliance. Supported by a dedicated coordination team, 88% of the 25 patients identified as referral candidates successfully completed the referral process and received timely diagnostic and therapeutic services at the tertiary hospital. The median referral time was approximately 3 days, indicating efficient coordination and strong adherence to the referral process. Moreover, 90.9% of CT images acquired at the secondary hospital were successfully transmitted to the tertiary hospital via the ESDR platform, enabling receiving physicians to access imaging data in advance and facilitating more accurate and timely diagnoses. The high transmission success rate and streamlined interhospital data exchange strengthened institutional coordination while enhancing the continuity of diagnostic and treatment services. Study outcomes collectively supported the feasibility of scaling AI-assisted diagnostic software technologies across lower-tier health care settings and demonstrated their value in strengthening referral processes and optimizing care pathways within the hierarchical health care system.

 The findings aligned with a growing global consensus on the strategic importance of deploying AI in lower-tier health care settings. Internationally, the implementation of AI-assisted diagnostic software technologies varies widely, influenced by differences in software types, national health care infrastructure, policy environments, and the maturity of health information systems. Although technical and organizational challenges remain, recent research has increasingly emphasized the potential of AI to improve diagnostic accuracy for common diseases, streamline clinical workflows, and optimize health care resource use, particularly in settings such as community health centers and secondary hospitals [38,54-57]. AI technologies are especially valuable in supporting primary care providers by enabling large-scale patient management, improving early disease detection, and reducing diagnostic errors in environments where clinical experience and infrastructure are limited [58-60].

Despite the growing recognition of these benefits, most existing studies have concentrated on evaluating clinical performance metrics such as sensitivity, specificity, and referral accuracy [61-63] or have identified implementation challenges like inadequate referral adherence [63]. Relatively few studies have developed and empirically validated comprehensive implementation strategies adapted to lower-tier clinical environments. This gap highlights the need to move beyond performance-oriented evaluations toward theory-informed investigations of how AI technologies can be effectively integrated into routine care.

 From an implementation science perspective, this study extended the application of ERIC strategies by demonstrating how their mechanisms of action vary across health system contexts. While ERIC strategies have largely been derived from Western health systems where hospitals and clinical units have substantial authority to determine implementation priorities, allocate resources, and adapt technologies to local workflows, recent evidence indicates that these strategies are increasingly applied in low- and middle-income country contexts, often requiring contextual adaptation. A systematic review of ERIC strategy use in low- and middle-income countries found that the majority of the 73 ERIC strategies have been selectively adapted to address diverse implementation barriers within resource-limited health systems, underscoring the importance of context-specific specification and tailoring to address structural and policy barriers distinct from those in high-income settings [64]. Consistent with this theoretical framing, the barriers identified in the present study were shaped primarily by system-level regulatory and structural constraints within China’s HMS, leading to prioritization of system-level and inter-organizational ERIC strategies over those commonly emphasized in Western contexts. By explicitly linking CFIR-identified determinants to contextually adapted ERIC strategies, this study underscores the importance of selecting implementation strategies based on the level at which barriers operate. This theory-informed, context-sensitive approach informed the study’s emphasis on feasibility and implementation processes rather than clinical performance outcomes alone.

The need for such implementation-focused research is particularly urgent in China, where structural imbalances within the HMS continue to place substantial pressure on tertiary hospitals. According to the Statistical Bulletin on the Development of China’s Health Sector released by the National Health Commission, outpatient and emergency visits to tertiary hospitals rose from 2.06 billion in 2019 to 2.63 billion in 2023, a 27.7% increase over four years, indicating a sustained upward trend [65,66]. In contrast, patient volume at primary health care institutions has remained relatively stable [67], underscoring the continued underutilization of lower-tier facilities and a growing imbalance in resource allocation. To address these structural inefficiencies, the implementation strategies proposed in this study focused on enhancing diagnostic capacity at the secondary care level and reducing unnecessary reliance on tertiary hospitals.

 Furthermore, inadequate referral mechanisms remain a major challenge in China’s HMS [10,68,69]. A cost-effectiveness analysis of AI-assisted screening for diabetic retinopathy in community health centers in China underscored the pivotal role of referral adherence in determining the overall value of AI-based telemedicine screening [70]. The results indicated that, in the absence of improved follow-up among patients with suspected disease, the adoption of AI tools might offer no significant advantage over manual grading in telemedicine screening, as reduced long-term effectiveness and health utility would offset potential cost savings. The persistence of inadequate referrals partially reflects systemic challenges such as fragmented health information and the lack of standardized referral protocols [71-73]. These systemic gaps could compromise continuity of care, delay timely diagnosis and treatment, and reduce the overall efficiency of health care delivery [74,75]. In recent years, the application of digital tools to referral processes has shown promise in addressing these challenges by facilitating data integration and strengthening communication between institutions [76,77]. Building on this evidence, the current study integrated the ESDR platform into the referral process, which facilitated more efficient data exchange, reduced delays, and enhanced the scalability of referral pathways in lower-tier health care settings. Additionally, the patient-centered design of the referral process helped reduce administrative, logistical, and financial burdens associated with repeated patient registration and diagnostic procedures, promoting timely and high-quality care.

Taken together, the integrated implementation strategies developed in this study reinforced the role of AI in supporting stratified care within China’s HMS. Within this framework, community and secondary hospitals are empowered to perform initial AI-assisted screenings to identify suspected or high-risk cases, which are subsequently referred in a timely and structured approach to collaborating tertiary hospitals for further evaluation and treatment. This allows tertiary hospitals to focus their resources on complex and critical cases, while lower-tier institutions effectively manage less severe conditions. The resulting model of decentralized screening and streamlined referral enhances patient flow across care levels, improves the responsiveness of service delivery, and ensures the timely transition of patients to appropriate levels of care. By enhancing clinical workflow efficiency and improving the use of health care resources, this approach contributes to a more functional HMS and facilitates the delivery of care that is well-coordinated, timely, and equitable across different levels of the health care system.

 However, several barriers were identified that might hinder the broader effectiveness and scalability of the implementation strategies. Notably, 14.4% of cases were not analyzed by the AI software, a gap that appeared to be primarily attributable to operational and workflow-related factors rather than algorithmic limitations. These factors may include interruptions in data transmission, timing mismatches between image acquisition and AI processing, and clinician-driven decisions not to activate the AI tool in specific clinical scenarios. In addition, data incompleteness emerged as a major challenge, as only 28% of referral records contained all required information. Missing data were especially prevalent in key fields, including clinical symptoms, diagnostic conclusions, and insurance information. Poorly structured or incomplete referral documentation has been reported as a main barrier to effective communication and care coordination across different tiers of health care institutions in China [77]. These gaps would impede the clinical capacity of physicians at the tertiary hospital to perform comprehensive clinical assessments, potentially affecting diagnostic accuracy. This finding was consistent with previous studies that have emphasized the ongoing fragmentation of HISs in China, frequently functioning in isolation due to the absence of standardized data standards and limited interoperability across institutions [78,79]. Further, disruptions to routine clinical workflows were commonly reported as a main barrier to the effective adoption of digital health innovations [80-82]. Due to administrative constraints, data security concerns, and technical limitations, the ESDR platform could not be integrated with existing hospital systems (eg, HIS and PACS) and was deployed on a public cloud. As a result, clinicians were required to manually navigate multiple platforms, increasing operational complexity and workload. To address these challenges, future efforts should focus on enhancing the interoperability and operational efficiency of interhospital data exchange systems through seamless integration [78,83,84]. Such interoperability would reduce the operational burden associated with navigating multiple systems and manual data entry, while enhancing the accuracy, completeness, and timeliness of data transmission. Additionally, there is an urgent need for continued examination to determine the adaptability, effectiveness, and long-term sustainability of implementation strategies across diverse health care settings [85]. Such evidence is essential for supporting the widespread, equitable, and context-specific integration of AI technologies into routine clinical practice and for fully realizing their potential to improve diagnostic accuracy, streamline care coordination, and enhance overall health system performance. 

This study had several limitations that should be addressed in future research. The implementation was conducted in a single urban setting in Beijing, involving one secondary and one tertiary rehabilitation hospital, and focused exclusively on pulmonary nodule screening. Although the findings offered clinically relevant insights into the feasibility and potential refinement of implementation strategies, the complexity of other health care settings, such as community health centers or rural clinics, and the applicability to different disease domains remains insufficiently addressed.

In addition, this study did not include a formal organizational-level cost analysis. Although conducted in a real-world clinical environment, the implementation represented an early-phase deployment in a resource-constrained secondary hospital, supported through research collaboration and vendor participation. Under these transitional conditions, short-term cost estimates would be unlikely to reflect routine, institutionally financed adoption. Moreover, key cost drivers, including long-term maintenance and system integration costs, changes in personnel workload, and downstream effects on referral patterns and health care utilization, require longer follow-up periods and more stable operational contexts to be meaningfully assessed.

Further, the single-center, pre-post study design limited the ability to fully account for temporal confounding and restricted external validity. Although the preimplementation and postimplementation periods were matched by calendar months to minimize seasonal variation, unmeasured temporal factors, such as changes in patient characteristics, institutional practices, or broader system-level influences, may still have affected the observed outcomes. Moreover, as the current study primarily focused on feasibility and areas for implementation optimization, both short-term and long-term sustainability were not evaluated. To improve generalizability and strengthen internal validity, future research should adopt multicenter, prospective study designs incorporating concurrent control groups and consider Type II hybrid implementation-effectiveness approaches to enable simultaneous evaluation of implementation outcomes, clinical impact, cost-effectiveness, and long-term scalability of AI-assisted diagnostic software.

This study developed and evaluated an integrated strategy to support the effective implementation of AI-assisted diagnostic software for pulmonary nodule screening in a secondary hospital in China. The findings demonstrated enhanced screening capacity, referral efficiency, and interhospital coordination, highlighting the potential of AI to strengthen the hierarchical health care delivery model. As an early-phase implementation study, these results should be interpreted as preliminary and should primarily be used to inform and refine the proposed implementation strategies. To support future scalability and real-world impact, targeted actions are needed, including promoting standardized interfaces between HISs and third-party platforms to improve interoperability, strengthening governance mechanisms for secure data sharing, and optimizing referral workflows across care levels. Together, these measures offer a practical pathway toward a more precise, efficient, and sustainable health care system, while contributing to more equitable allocation of medical resources. Future research should prioritize multicenter studies and long-term evaluations to inform the broader adoption of AI technologies across diverse clinical settings.