Rates of children diagnosed with autism continue to rise dramatically, with one in 31 eight-year-olds now receiving a diagnosis in the United States [1]. While earlier identification and intervention during the critical neurodevelopmental window can maximize outcomes [2-4], many children experience critical delays to diagnosis [5]. There is a multiyear lag, for example, between the 18‐ to 24-month period where diagnosis is possible [6] and the average US age of diagnosis of nearly 5 years [7]. Delayed diagnoses represent missed opportunities for impactful early intervention, with new evidence emerging that even small delays in treatment initiation (18 vs 27 mo) can reduce its efficacy [8].

In the United States, autism evaluations typically occur in specialist centers, with less than 1% being made in primary care [9,10]. The inadequate number of specialists available to assess a rapidly growing pool of children with questions about developmental delay has created multiyear waitlists for evaluations [11,12]. To help address this crisis, calls to recruit more primary care physicians and practitioners to participate in the diagnostic process are growing [13-15].

While multiple studies show benefits to this approach, unanswered questions remain about the best way to integrate autism evaluations into the primary care workflow [16-20]. Many traditional diagnostic tools, for example, are time-intensive or difficult to administer [21] and thus represent barriers to autism evaluation in primary care. Two of the most commonly used tests in autism diagnosis are the Autism Diagnostic Interview-Revised and the Autism Diagnostic Observation Schedule [21]. The Autism Diagnostic Interview-Revised is a structured parent interview that typically takes 2‐2.5 hours to administer and score. The Autism Diagnostic Observation Schedule is an observational assessment that must be conducted in-person and can take 60‐90 minutes or longer to complete. Both instruments require formal in-person training followed by ongoing practice to maintain scoring reliability. Best results are often achieved by administering both tools to the same child [21], resulting in a time commitment that is unfeasible in the context of a primary care workflow.

Canvas Dx, a Food and Drug Administration–authorized, artificial intelligence (AI)-based Software as a Medical Device, was developed to support streamlined autism evaluations in 18‐ to 72-month-olds with developmental delay concern [22]. The underlying machine learning algorithm was trained on data from thousands of children with diverse demographic backgrounds and behavioral health presentations with the goal of supporting accurate, equitable, and rapid identification or rule out of autism in primary care [23]. Canvas Dx does not require specialist training to administer and has been clinically validated for both remote and in-person administration [24].

The device’s gradient-boosted decision trees algorithm [23,25] integrates data from a brief caregiver questionnaire, a video analyst questionnaire, and a clinical questionnaire that can be administered either remotely or in person, while the clinician observes and interacts with the child. Where sufficient information is provided to make a highly accurate autism prediction or rule out, the device produces a Positive for autism or Negative for autism output. When a highly predictive determination cannot be made, the device abstains from providing a result (Indeterminate output).

The ECHO Autism Primary Care Early Diagnosis (EDx) Model [26,27] is a multifaceted professional development model including in-person and virtual components to support primary care clinicians regarding best practice autism evaluation and longitudinal care. The EDx model involves training and mentoring community-based clinicians to conduct diagnostic assessments for autism, consistent with best practice standards. ECHO Autism: EDx is the first of its kind model designed to professionally develop community-based primary care clinicians to assess and appropriately diagnose young children with a question of autism. However, it requires significant resources of time and professional learning to achieve competency in serving this population well. Therefore, ECHO Autism Communities and Cognoa engaged in this study to evaluate the feasibility of using Canvas Dx in lieu of the model’s usual observational tool, the Screening Tool for Autism in Toddlers and Young Children (STAT) [28], to support primary care clinician diagnostic determination.

The primary study objective was to assess the time from initial clinician concern to diagnosis when using the device as part of the ECHO Autism: EDx diagnostic pathway. The secondary objective was to explore clinician and caregiver experience of the feasibility and impact of device use as part of the diagnostic journey.

The study protocol for this prospective observational feasibility study is described in full in JMIR research protocols [29], with key methodological considerations summarized further. Eligible participants were enrolled consecutively to minimize selection bias, with data collection occurring between April 2022 and November 2023.

The study received institutional review board approval from the University of Missouri-Columbia (project number 2075722), and the protocol was registered with ClinicalTrials.gov prior to study initiation (protocol identifier: NCT05223374). Written informed consent was obtained from all participants, and the participants had an ability to opt out of the study at any time. Device costs were covered by the study sponsor Cognoa, but no other monetary compensation was provided to the participants. Data have been anonymized to protect participant confidentiality, and no identifiable images of participants are included in the manuscript or supplementary material.

Cognoa leadership and ECHO Autism leadership were blinded to the study dataset and had no access to the study end points until after the first data analysis was completed by an independent third party. This was to ensure minimal introduction of influence by either organization involved in the study.

The device inputs consist of the following:

The device output is accessed via the clinical web portal. It consists of a diagnostic determination (Positive for autism, Negative for autism, or Indeterminate); all questions and responses from Inputs 1 and 3; and the caregiver’s submitted videos.

Measure of Processes of Care: This is a validated tool adapted for this study to measure caregivers’ perceptions of the family-centeredness of the care provided.

ECHO Autism Caregiver Presurvey: This collects baseline demographic information and captures caregivers’ initial impressions of the EAC prior to the diagnostic process.

ECHO Autism Caregiver Postdiagnostic Survey: This assesses caregiver satisfaction with the diagnostic services received.

EAC Pre-Project Survey: This evaluates clinicians’ self-efficacy, perceived barriers, and satisfaction with the current ECHO Autism diagnostic process. The current ECHO autism diagnostic workflow consists of the following: (a) completion of a 2-day in-person training on the STAT. This is a direct structured behavioral observation that takes approximately 20 minutes to administer in the physician’s office once training is complete; (b) achievement of administration and coding reliability on the STAT instrument to a certified STAT trainer; (c) participation in twice monthly, 90-minute iterative case-based learning through ECHO Autism: Primary care sessions where additional skills required as part of best practice autism evaluations such as diagnostic interviewing, differential diagnosis, and conceptualization of development and behavior in young children are acquired.

EAC Post-Project Survey: This assesses changes in clinician self-efficacy, perceived barriers, and satisfaction following integration of the device into the diagnostic workflow.

ECHO Autism 3-Month Follow-Up Questionnaires: This was designed to capture data needed to address secondary and exploratory study objectives.

ECHO Autism Interim Visit Form: This was used to document any participant visits occurring between the initial concern and the 3-month follow-up.

Descriptive statistics were used to summarize all collected survey data from caregiver and clinician participants. Continuous variables are reported as n, mean, standard deviation, median, minimum, and maximum values, while categorical variables are presented as counts and percentages where relevant. All available data, including observed and missing values, were tabulated.

Time from initial clinical concern at enrollment to diagnosis by the EAC was calculated in days and compared with the projected wait time if participants had been referred to adjacent specialist centers at first concern instead of following the primary care diagnosis pathway. Both EACs and adjacent specialist centers assess children with concern for developmental delay across the complexity spectrum and labeling age range. To generate comparison estimates for specialty center wait times, we obtained publicly reported median wait times to first appointment from the Missouri Autism Specialty Centers [32]. The Missouri Autism Centers aggregate and track median wait time data from multiple autism specialty centers across Missouri. Data are reported annually by fiscal year and stratified into two age cohorts: children under 60 months and children 60 months and older. Because the current study period spanned two fiscal years (2022‐2023), we calculated the mean of the 2 years’ reported median wait times to derive an overall estimate for each age cohort. For children under 60 months, the reported median wait times were 143 days (FY2022) and 217 days (FY2023), yielding an average of 180 days. For children 60 months and older, the reported medians were 221 days (FY2022) and 306 days (FY2023), yielding an average of 264 days. These averaged values were used as the comparison benchmarks in subsequent analyses. Distance from the families’ home address to the primary care EAC appointment was collected as part of the caregiver surveys and compared with the distance families would have otherwise been required to travel to their nearest specialist center.

Device diagnostic determinations (Positive for autism, Negative for autism, or Indeterminate) were compared with the reference standard diagnosis from the EAC to calculate positive and negative predictive values, as well as sensitivity and specificity in the determinate group. Device performance was also analyzed across race or ethnicity, biological sex, and age to determine if performance metrics varied by demographic. Device-generated timestamps were used to assess the time burden associated with input completion.

Figure 1 summarizes the full study flow.

The protocol anticipated a data-collection period of approximately 12 months and a target of about 100 participants completing the full protocol. These were planning projections and not required minimums or formal stopping rules. In practice, 109 participants were enrolled and 80 completed all study procedures. Because the study had already extended beyond the anticipated 12-month window, we concluded data collection at 80 completers due to operational constraints. This deviation pertains only to study duration and completer count. No changes were made to eligibility criteria, data collection procedures, or the prespecified analysis plan.

Figure 2 summarizes the recruitment and retention funnel for the study.

The overall patient population had an autism prevalence of 60% (48/80) per clinician evaluation and was 30% (24/80) female. The age of participants ranged from 18.29 to 71.28 months at first appointment, with a mean age of 42.25 (SD 14.41) months and median age of 39.91 months. There were no gender or age-related differences between study completers and noncompleters. Just over half (61.76%, 63/102) of participants were primarily insured by Medicaid, while 38.24% (39/102) of participants were privately insured (Table 1).

Demographic data were collected from the caregivers of most enrolled patient participants (n=104; Table 2). Most caregivers reported being White or Caucasian (85.58%, 89/104). This likely reflects the demographic makeup of Missouri where approximately 80% of the population identify as “White Non-Hispanic” [33]. Most caregivers reported a highest education level attained of high school graduate or GED or some college but no degree (28.85% [30/104] and 28.85% [30/104], respectively). The majority of caregivers reported a household income between US $25,000 and 49,999 or US $0‐24,999 (38.46% [40/104] and 26.92% [28/104], respectively). The age of caregivers ranged from 20 to 49 years, with a mean age of 31.86 (SD 6.36) years and median age of 31. The demographic distribution of the caregivers of participants who completed the trial was similar to that of the caregivers of all enrolled participants.

Participating clinicians provided a definitive autism diagnosis or rule out to all study completers in the primary care setting. The device provided determinate results (ie, positive for autism or negative for autism) for 52.50% (42/80) of patients. Determinate device outputs agreed with the clinician diagnosis in 100% of cases, giving a positive predictive value, negative predictive value, sensitivity, and specificity of 100% (Table 3). The device did not produce any false positives or false negatives. Most indeterminate patients received a negative autism diagnosis from their clinician (78.95%, 30/38). There were no significant differences in the rate at which the device output a determinate versus an indeterminate result based on participants’ age (under 48 mo vs over 48 mo; insignificant, P=.36), sex (male vs female; insignificant, P=.81), race or ethnicity (insignificant, P=.78) or socioeconomic status (income brackets 0‐50k, 50k-100k, 100k+; insignificant, P=.87).

The STAT, in contrast, generates a low or high-risk score for each child (binary classifier); however, without a built-in ability to abstain in cases of high uncertainty, its overall predictive performance for identifying or ruling out autism is significantly lower than that of the device. .

Survey data were collected from all seven clinician participants at the start of the study and at the end of the study and in relation to each patient who received an evaluation using the device. Clinicians agreed with the output provided by the device for 100% of determinate cases (Table 4). Most (71.43%, 5/7) clinicians agreed that the device worked well within their clinical practice flow, and more than half (57.14%, 4/7) felt that the device added information to their diagnostic impression.

Clinicians reported that barriers associated with using their previous process (ie, STAT) “process as usual” were reduced when administering the device (Table 5). Time, for example, was identified as a barrier by 71.43% (5/7) of clinicians for the process as usual compared to only 28.57% (2/7) for the device. Similarly, child cooperation was reported as a barrier by 42.86% (3/7) of clinicians for ECHO Autism: EDx process as usual but was decreased to 28.57% (2/7) with the use of the device. Family resistance, reported by 14.29% (1/7) of clinicians as a barrier for process as usual, was not reported as a barrier when using the device. On the other hand, 85.71% (6/7) of clinicians reported technology as a potential barrier with device use, likely because families are required to download an app and independently complete in-app tasks, whereas the STAT was a pen and paper test administered by the clinician.

Additional barriers unique to the device included insufficient clinical information from caregiver videos and limited usefulness of the output generated by the device in cases when an indeterminate result was generated (Table 5). Just over half of clinicians (57.14%, 4/7) reported being unlikely to uniformly recommend the device in lieu of additional direct observation for these cases. While just under half (42.86%, 3/7) of the participating physicians agreed that the device saved them time in the overall diagnostic process, 28.57% (2/7) were uncertain and 28.57% (2/7) disagreed.

This collaborative observational study was the first to evaluate the use of a novel AI-based autism diagnostic device as part of the ECHO Autism: EDx workflow. Compared to specialty center referral, these primary care–based autism evaluations reduced the time from first clinical concern to diagnosis by approximately 5‐9 months. Families, on average, traveled 45 miles less to each appointment than they would have if assessed at their nearest specialty center. Participating caregivers reported strong levels of satisfaction with both the device itself and the ECHO Autism diagnostic process more broadly, highlighting that evaluation in a primary care setting was acceptable to families. High levels of caregiver satisfaction with primary care autism evaluations [34] reduced travel burden [35] and significantly shortened time from first concern to diagnosis [36] are consistent with previously published topical study findings.

While most clinicians endorsed that device use was feasible and provided workflow efficiencies compared to their previous process, clinician participants noted device use did not always obviate the need for additional observations. While there was 100% alignment between determinate device predictions and the final clinical diagnosis, in the roughly half of cases where the device abstained from making an autism prediction or rule out, clinicians were required to rely more heavily on their observations, comprehensive DSM-5 history, and clinical expertise to arrive at a diagnostic determination. This emphasizes the need for clinicians seeking to include autism diagnosis in their practices to obtain and maintain the clinical expertise necessary to interpret varied result outputs in their diagnostic determination process.

Technological barriers to device use noted by a minority of caregiver participants included issues with video uploads, video recording directions, and unreliable internet access. Since the time of study completion, the device manufacturers have implemented a number of updates to improve video uploading functionality and in-app recording directions. Future testing to determine the impact of these updates on user experience is recommended.

The version of the device assessed in this study included only a positive, negative, or indeterminate output but did not include an accompanying detailed report explaining the result. Since the time of study completion, supporting software has been developed that increases the transparency of the device’s AI, mapping collected data to DSM-5 criteria relevant to autism diagnosis and auto-generating a detailed report to reduce clinical documentation burden [25]. Decision threshold updates have also been made to reduce the number of indeterminate outputs that are returned [23]. Future clinician users’ perceptions of device utility and efficiencies (particularly in indeterminate cases) may differ from those reported by study participants, given this significant update in functionality.

Additionally, we note that the clinician sample size (7) was small, and their demographics were not representative of the broader US population, with all being White or Caucasian. Clinician study participation was impacted by EACs being in primarily nonacademic, rural communities with less infrastructure and familiarity with participating in research. Currently, the device is only available in English and Spanish, so findings reported here may not generalize to other non-English or non-Spanish speaking populations. While no significant differences in device abstention rates were noted across age, sex, socioeconomic status, or race and ethnicity, the authors recommend that larger studies be conducted to confirm these findings in more racially diverse samples.

The ECHO Autism: EDx plus device workflow was feasible and reduced barriers to diagnosis for both caregiver and clinician users. Compared to referral to adjacent specialist waitlists, use of the ECHO Autism: EDx plus device workflow resulted in a reduced travel burden for families and significantly shortened patient delays from first clinical question to diagnosis and treatment initiation.

These findings may be of significant interest to families with children stuck on long wait lists for autism evaluations, as well as policymakers, clinicians, and payors seeking to find solutions to the current waitlist crisis. Since the Centers for Disease Control and Prevention began tracking rates of autism among 8-year-olds in 2000, prevalence rates have increased by over 300% [37,38]. Meanwhile, the specialist workforce, typically responsible for conducting autism evaluations, has not kept pace with this increase. Currently, there is only an estimated one developmental behavioral pediatrician per 100,000 children [11] and 14 Child and Adolescent Psychiatrists per 100,000 children [39]. This prospective observational study demonstrated that innovative professional development models like ECHO Autism, combined with emerging diagnostic technologies like Canvas Dx, can help equip the much larger primary care workforce to feasibly play a greater role in diagnosing and managing autism without automatic specialist referral. Strategies to further expand the pool of qualified primary care clinicians able to participate in the diagnostic process, and funding to equip them with evidence-based diagnostic supports, should be pursued to unblock specialist waitlists and support timely treatment initiation.