The use of AI now offers the possibility to examine and extract patterns from large and highly complex datasets. While AI can ensure unprecedented speed in specific tasks such as data mining, predictive analytics, and pattern recognition, its accuracy remains highly variable and dependent on the specific application [1].

Over the past decade, the ongoing evolution of AI has profoundly impacted many aspects of society and clinical practice [2]. By leveraging vast amounts of data, AI provides an essential wealth of resources for research and public health, making medical and scientific information much more accessible. The integration of these technologies into the educational sector has ushered in a new era of technology-supported learning, creating highly innovative prospects for health care education as well [3].

In this context, the integration of natural language processing models is proving to be a game-changer for radically improving the accessibility and management of information in health care. Specifically, large language models (LLMs) have established themselves as cutting-edge tools, endowing machines with the ability to understand, process, and generate text in a way that is almost indistinguishable from human communication [4].

The medical and health care sector represents one of the most promising domains for the application of LLMs. In recent years, these models have garnered significant attention due to their ability to outperform traditional computational approaches in specific natural language processing tasks, such as medical text summarization, conversational triage, and translation [5,6]. However, high performance in these semantic and generative tasks cannot be indiscriminately generalized to domains requiring strict factual retrieval, such as academic research and evidence-based medicine (EBM).

By analyzing massive amounts of unstructured text—from medical records to academic articles—LLMs prove essential in supporting decision-making processes. They assist health care professionals in diagnosis, treatment recommendations, and interpretation of complex cases and reports and even play a key role in patient education and in identifying emerging trends in scientific research [4,5].

However, despite their remarkable potential, the adoption of LLMs in medicine raises several concerns that require careful evaluation by health care professionals and researchers [6].

Research has highlighted that such systems exhibit significant structural and epistemological limitations: they can inherit biases stemming from their training datasets or algorithmic optimization processes. A particularly critical issue in the clinical field is represented by “hallucinations”—the generation of information entirely devoid of factual basis—a phenomenon that tends to be exacerbated when addressing novel, complex, or cutting-edge topics [7].

In the psychological domain, outside the realm of AI, the term “hallucination” refers to a specific perceptual phenomenon consisting of a sensory illusion devoid of any basis in reality, which the subject perceives as authentic. The anomaly wherein AI systems produce inconsistent or erroneous outputs bears striking similarities to the clinical definition of the term, thus justifying this linguistic borrowing. Indeed, “hallucinated” text appears fluent and natural, thereby masking its inherent inaccuracy. Although seemingly logically anchored to the user-provided context, it is actually rooted in fictitious or objectively unverifiable premises. Consequently, within the field of AI, the current standard defines a hallucination as the generation of content that is either baseless or incongruent with the original source material [8].

It should be noted, however, that not all experts agree with the use of this terminology. Indeed, some authors contend that it would be more appropriate to speak of “fabrications and falsifications” rather than hallucinations: a fundamental distinction aimed at avoiding the anthropomorphization of a machine that is merely “inventing” nonexistent data [9]. For the purposes of this paper, the term “hallucination” will conventionally be adopted to identify this phenomenon, in alignment with the prevailing nomenclature in the current technical discourse.

AI-generated hallucinations can originate from multiple interconnected factors, including the reliance on nonspecialized or biased training datasets, the inherently probabilistic nature of text generation, the lack of adequate contextual parameters in user prompts, and the structural limitations of language models in sequential and deductive reasoning [10].

Language models are not inherently designed as academic search engines or relational databases: they operate as “probabilistic engines” trained to generate fluent and plausible text, not to retrieve verified data. Their tendency to “fabricate” seemingly flawless article titles, attributing them to real authors and existing journals, despite being entirely nonexistent in the published literature, is a widely documented phenomenon that threatens the very integrity of the scientific method. While in purely creative or brainstorming contexts, algorithmic hallucinations may be considered a minor flaw, in the field of scientific research and evidence-based clinical practice (EBM), they represent a critical risk [11,12].

Recent studies have underscored the need to critically evaluate the accuracy and reliability of chatbot-generated responses to direct medical queries, emphasizing how the integration of such tools into daily practice requires rigorous validation protocols to ensure their safe and effective deployment [13].

While the phenomenon of AI-generated hallucinations in medical content is increasingly documented, prior research presents significant methodological limitations, particularly concerning prompt design. For instance, recent efforts to quantify AI reliability, such as the reference hallucination score (RHS) proposed by Aljamaan et al [14], have relied on highly structured, engineered prompts. In such studies, the models are explicitly instructed to output predefined metadata fields (eg, authors, title, journal, and DOI). Although valuable for assessing theoretical capabilities under optimal conditions, this heavily guided approach does not reflect the real-world interactions of everyday users—such as students, health care professionals, or laypersons—who typically formulate open, unstructured queries.

Furthermore, earlier studies [11,12] have predominantly focused on single-model evaluations, mostly testing initial versions of ChatGPT, lacking a comparative framework across different algorithmic architectures.

These gaps leave a critical question unanswered: how vulnerable is a standard, unguided user to bibliographic hallucinations when interacting with the newest generation of LLMs? Our study directly addresses this gap by employing an open-ended prompt design. By deliberately omitting instructions on how to structure the output, this study evaluates not only the factual accuracy of the models but also their autonomous capacity to recognize, retrieve, and correctly format scientific evidence without explicit guardrails. This approach provides a more realistic and rigorous assessment of the comparative reliability of newer platforms—such as Grok, Meta AI, and Perplexity—in the context of specialized public health and clinical research.

In light of critical issues highlighted in the literature, the primary objective of this study is to evaluate and compare the outputs generated by different AI platforms in response to specific bibliographic research questions in the medical and health care field. The investigation was designed to simulate typical queries from a broad spectrum of users—ranging from health care professionals to students and even laypersons with no medical background—while subsequently subjecting the obtained results to rigorous methodological validation conducted by domain experts.

Specifically, the research aims to measure the precision and accuracy of the references provided, verifying the exactness of metadata and the actual existence of the proposed articles, in order to quantify the reliability of the models and to highlight the presence of any hallucinations. Beyond factual validation, the study intends to explore the underlying logical and selection mechanisms intrinsically used by the models to compile reference lists. Therefore, it seeks to analyze which parameters guide AI in choosing the retrieved articles, evaluating key factors such as thematic coherence with the user’s query, scientific impact (inferred from citation counts), and the recency of the sources (publication year).

Through this analysis, the study aims to provide a comprehensive overview not only of the accuracy limitations of various chatbots but also of the information retrieval strategies driving their responses, offering valuable insights for their informed use across clinical, academic, and everyday settings.

To assess the effectiveness and reliability of AI in the health care sector, a cross-sectional observational study was conducted.

Four of the most widespread and currently used generative AI platforms have been selected for analysis: ChatGPT [15], Grok [16], Meta AI [17], and Perplexity [18].

The selection of these tools was based on the following criteria: a free-of-charge service, open and unrestricted access, and usability without requiring user account registration.

These inclusion criteria were adopted to simulate the conditions of a standard user seeking immediate access to medical information. The use of “Guest” mode was implemented to mitigate user-history bias and prevent influence by preexisting account cookies, although we acknowledge this does not guarantee absolute algorithmic neutrality, as IP-based geolocation and intrinsic model biases remain active.

In order to assess the bibliographic search capabilities of AI systems, 3 health-related queries were formulated. These were structured to provide a variable gradient of specificity, ranging from highly specialized topics to broader public health themes.

To ensure linguistic uniformity and isolate the topic investigated as the only independent variable of the experiment, the syntactic structure of the questions was kept constant.

The three prompts, administered in Italian, were as follows:

In order to standardize the operational conditions and mitigate technical biases, all queries were performed using a single personal computer and the Google Chrome web browser in “Guest” mode.

All tests were conducted from the geographical region of Italy, with the browser interface language set to Italian. The entire prompt administration phase was completed within a strict 24-hour time window (March 24‐25, 2026). This rigorous methodological precaution was adopted to minimize confounding variables associated with the continuous updates of the underlying LLMs and the machine learning evolution of the AIs over time.

The following free versions, accessible without user authentication (Guest mode), were used: ChatGPT (defaulting to the GPT-4o architecture available for unauthenticated users), Grok (Grok-2 architecture family), Meta AI (powered by the Llama 3 model series), and Perplexity (standard free search model).

The querying protocol was executed in accordance with these standardized steps for each AI and each question.

In the early stages, the phase of data acquisition involved five AI platforms (ChatGPT [15], Grok [16], Meta AI [17], Perplexity [18], and OpenEvidence [19]). However, during prompt administration, the use of OpenEvidence revealed operational limitations related to the platform’s policies: “guest” access (without registration or login) only allowed the generation of a response for the first query (D1), blocking subsequent queries. In order to guarantee the strict homogeneity and statistical comparability of the study sample across all prompts (D1, D2, and D3), it was decided to exclude OpenEvidence from the analysis.

All textual data generated by the AI models were extracted and structured into a dedicated tabular database created using Microsoft Excel software. Based on the bibliographic references provided by the AIs—accounting for 4 platforms, 3 queries (D1, D2, and D3), 3 iterations per query (R1, R2, and R3), and 10 articles requested per iteration—an initial theoretical total of 360 items was collected for the subsequent evaluation phase (4 AIs × 3 prompts × 3 iterations × 10 requested articles).

A rigorous eligibility criterion regarding the source type was applied to these 360 references, based on strict adherence to the literal wording of the administered prompt. In accordance with the explicit stipulations of the prompt, which called for the citation of “10 articles,” only responses containing authentic scientific articles were considered valid and extracted for further analysis. Any other type of source suggested by the AI (including books, manuals, guidelines, institutional reports, or general websites) was classified as “unacceptable” and excluded from the final dataset.

The bibliographic references that were deemed valid following the extraction process were then subjected to a 3-stage analytical process. The purpose of this process was to determine the formal completeness and factual accuracy of the generated outputs.

Statistical analyses were conducted with the aim of describing the sample characteristics and comparatively evaluating the performance of the 4 selected AI models (Grok, ChatGPT, Meta AI, and Perplexity). The sample characteristics were summarized using descriptive statistics. The frequencies and percentages were used to describe categorically measured variables. Median values and IQR, alongside means and SDs, were used to describe quantitative and ordinal variables (latency times, number of citations, publication year, and hallucination score). For the analyses concerning publication year and citation count, only articles verified as existing following the fact-checking procedure were included; “total fabrications” (full hallucinations) were excluded from these computations, as they lack objective, verifiable metadata. The normality of the data distribution was verified using the Shapiro-Wilk tests. For the analysis of categorical variables (eg, the presence or absence of hallucinations across the different models), Pearson chi-square (χ²) test was applied. The nonparametric Kruskal-Wallis test was used to compare the models with respect to continuous and ordinal variables. For statistically significant findings, the analysis was further deepened using Dunn post hoc test, applying the Bonferroni correction for multiple comparisons. Furthermore, to account for potential data clustering and rule out any bias related to pseudoreplication (due to multiple references generated from a single prompt), a sensitivity analysis was conducted by aggregating the data and using the 36 independent query runs as the primary unit of analysis.

Data analysis was performed using R software (version 4.5.1; R Foundation for Statistical Computing). Specifically, data manipulation and descriptive statistics were conducted using the dplyr and gtsummary packages. The Shapiro-Wilk test, Pearson chi-square test, and the Kruskal-Wallis test were executed using the core stats package. Dunn’s post hoc pairwise comparisons were performed using the dunn.test package. The statistical significance level α was set at .05.

This study did not involve human participants, animal subjects, or personal patient data. The research consisted exclusively of interacting with publicly accessible AI platforms; therefore, institutional ethics committee approval was not required.

The data extraction process, which was conducted across 3 iterations for each of the 3 queries administered to the 4 AI platforms under examination, generated a theoretical pool of 360 bibliographic references (30 articles requested for each AI-query combination). However, the analysis of the generated output revealed a significant data dispersion compared to the expected volume. Progressive screening of the sample, as illustrated in Figure 1, yielded the distribution shown in Textbox 1.

Building upon the data dispersion illustrated in Figure 1, the overall performance of the AI models can be stratified and evaluated across 3 distinct operational concepts, using the appropriate denominators for each specific phase:

The analysis of response times, calculated from the moment of prompt submission to the completion of the output, revealed performance differences among the analyzed platforms.

ChatGPT demonstrated the highest overall consistency, maintaining brief processing times across all iterations, ranging from a minimum of 8 to a maximum of 17 seconds, with median values of 15.0 (IQR 9‐15) and mean values of 13.2 (SD 3.8) seconds.

In contrast, Grok required systematically longer processing cycles compared to the other models. Its processing times consistently remained above 28 seconds, with a median time of 47.0 (IQR 39.0‐66.0) seconds and a maximum peak of 97 seconds.

Meta AI exhibited highly fluctuating trends, resulting in an overall median of 15.0 (IQR 9.0‐25.7) seconds and a mean of 18.8 (SD 11.4) seconds. Although it recorded the shortest response time of the entire experiment (6 s in the D3-R1 trial, coinciding with the delivery of an error message), it experienced significant slowdowns during other queries, reaching peaks of up to 40 seconds.

Perplexity emerged as the fastest model, with an average processing time of 12.3 seconds and a median of 10.0 (IQR 8.0‐15.0) seconds.

The first query revealed diametrically opposed reliability rates among the models:

Query D2 severely challenged the models’ ability to discern “scientific articles” from other types of institutional documents (such as guidelines or hospital reports).

The query regarding a topic of broad public health relevance revealed the concerning phenomenon of “hallucinatory drift,” which is characterized by a drop in consistency during serial repetitions of the same prompt.

To scientifically validate the observed divergences among the models, an inferential analysis was conducted on a total sample of 360 observations. Shapiro-Wilk normality tests yielded extremely low P values for all variables (P<.001), confirming a nonparametric data distribution and justifying the use of the Kruskal-Wallis test.

The Kruskal-Wallis test revealed statistically significant differences among the 4 models across all analyzed dimensions (P<.001 in all tests).

Pairwise comparisons with Bonferroni correction allowed for the mapping of statistical distances between the platforms:

To ensure the robustness of our findings and strictly account for data clustering, we reevaluated the data using the 36 independent query runs as the primary unit of analysis (n=36). The Kruskal-Wallis test on this aggregated dataset confirmed the original findings, demonstrating significant performance disparities among the four platforms (P=.004 for the median hallucination score, and P<.001 for the percentage of exact matches). This confirms that the observed differences in AI model accuracy are inherent to the platforms’ architectures and not an artifact of pseudoreplication.

The findings reveal a highly critical scenario: despite the remarkable linguistic fluency and the apparent confidence with which most models generate their responses, the actual factual accuracy of the metadata is alarming. The most significant finding of our investigation lies in the stratified breakdown of the models’ failures. While the overall strict yield was only 35.6% (128/360), a deeper analysis reveals distinct operational bottlenecks: the models failed to complete the task in nearly a quarter of cases, struggled with instruction adherence (generating nonarticle sources 30.4% of the time), and, most critically, hallucinated metadata in 32.6% (62/190) of the formally correct articles. A large proportion of the output was divided between unacceptable formats (gray literature, videos, and blogs) and articles containing at least one inaccuracy (62/360, 17.2%).

In line with recent findings in the literature [9], our data confirm that the use of the term “hallucination” risks downplaying the severity of the phenomenon in a clinical context: the analyzed chatbots engaged in outright data fabrication, generating nonexistent DOI codes or combining real authors with the titles of articles that were never published. This behavior, inherent to the probabilistic nature of LLMs, poses a severe risk to the integrity of EBM, exposing both health care professionals and patients to the dangers of scientific misinformation.

The findings of this study have significant implications for various categories of users. For health care professionals, the uncritical use of LLMs for clinical updates or evidence-based research poses an ethical and legal risk: basing a therapeutic or diagnostic decision on “fabricated” references or incorrect metadata can compromise patient safety and the effectiveness of treatment. In the academic setting, for students and researchers, relying on such tools without meticulous manual verification of sources threatens scientific integrity, exposing them to the risk of unintentional plagiarism or the propagation of nonexistent citations. Finally, for the general public, the apparent authority with which chatbots cite (often nonexistent) medical studies fuels the phenomenon of health misinformation, creating a false perception of scientific rigor that may lead to dangerous self-diagnosis. In summary, the results confirm that, at present, generative AI platforms cannot be considered substitutes for validated bibliographic databases (such as PubMed or Scopus). Instead, they must be regarded exclusively as complementary support tools, subject to constant and rigorous fact-checking by domain experts.

The urgency to address these practical implications is further substantiated by recent scientific evidence. Indeed, the reliability of the results emerging from our study is, in fact, significantly corroborated by a recent analysis conducted by Cabezas-Clavijo and Sidorenko Bautista [21]. Although the authors investigated a different panel of platforms, their research arrived at the same structural conclusions. Specifically, even across varying algorithmic architectures, their study confirmed that none of the tested artificial intelligence tools is currently capable of ensuring absolute precision, reiterating the inadequacy of these models to serve as primary, error-free academic search engines. This convergence demonstrates that the tendency toward bibliographic falsification is not limited solely to the platforms we analyzed but represents a systemic limitation of current generative technology.

Despite the methodological rigor applied, this study is not without its limitations. First, although the analyzed prompts were subjected to triple iteration to test for stability, the sample size is limited compared to the infinite variety of potential clinical queries. Future studies using larger datasets could provide a more comprehensive analytical overview. Second, the rapid technological evolution characterizing LLMs must be considered. As the tested models are updated frequently, the observed performances represent a snapshot of the current technological landscape, which is subject to rapid change.

A further limitation concerns the language used and the way in which the queries were posed to the AI. Since these systems are predominantly trained on English-language text, prompting in Italian may have partially compromised their ability to retrieve international scientific literature accurately.

Furthermore, the decision to use direct queries devoid of contextual instructions on how to structure the output—simulating the behavior of an average user—precluded the opportunity to assess whether more detailed prompt engineering would have guided the models toward more adequate and rigorous responses.

A further significant limitation concerns the selection of the 3 query topics. The topics were chosen subjectively to represent a gradient of specificity, and only a single prompt formulation was used for each domain. Based on the current study design, it is not possible to definitively determine whether the observed performance variations across the three queries were driven by the intended level of specificity, the sheer volume of available literature in each respective field, potential nuances in terminology translation, varying coverage within the training databases, or the intrinsic architectural performances of the AI models. Future studies should employ multiple, validated prompts across standardized topic clusters to isolate these variables more effectively.

Finally, it is important to emphasize that this study analyzed only the free, base versions of the four platforms. Although this methodological choice accurately reflects the tools accessible to the majority of everyday users (eg, students, laypersons, or professionals without paid subscriptions), it prevents the evaluation of these systems’ maximum technological capabilities. Premium versions incorporate higher-performing models and advanced web search capabilities, which could mitigate the tendency to generate fictitious bibliographies.

This study demonstrates that, under the specific testing conditions evaluated, the selective generative AI platforms lack the structural reliability required to replace traditional bibliographic databases in medical and scientific research. Although the conversational interfaces provide a highly intuitive user experience, the alarming rate of hallucinations poses a significant threat to the integrity of EBM.

Specifically, the comparative analysis revealed significant operational differences between the tools tested. While models such as Meta AI and Perplexity proved entirely inadequate for the rigor demanded by the academic setting, ChatGPT demonstrated greater formal stability, anchoring itself to historically established evidence with high bibliometric impact (retrieving, on average, highly cited sources). Conversely, within the confines of our sample, Grok exhibited the best overall performance: although it retrieved articles with lower historical citation volumes, it distinguished itself through high factual accuracy and a pronounced ability to monitor contemporary literature. However, none of the examined platforms proved entirely immune to generating errors or inventing sources, confirming that the risk of data fabrication is currently a limitation of the model versions and configurations tested.

In conclusion, the integration of generative AI into clinical and academic practice should not be impeded; rather, it demands an urgent and profound shift in methodology. As advocated in recent literature, the introduction of structured AI literacy programs [22] throughout the entire educational pathway [23] has become essential: starting from school curricula [24] through university education (especially for medical and scientific degree courses) to continuous medical education programs. In a high-risk sector such as health care, these systems must be framed as complementary exploratory tools rather than primary search engines. The professional’s critical appraisal and rigorous verification against official databases remain indispensable, ethically obligatory steps that cannot be delegated to a machine. Only through formal education regarding the limitations and probabilistic mechanisms of LLMs will it be possible to harness the potential of these technologies without compromising patient safety and the authority of scientific research.