Clinicians, broadly defined in this paper as physicians, advanced practice providers, and nurses involved in patient education and discharge planning, serve as primary translators between medical information and patient understanding. As AI tools increasingly enter clinical environments, health care professionals and clinical informaticists face a pressing question: how do we evaluate whether these tools actually serve our patients, not just in terms of technical accuracy, but also in accessibility, relevance, and responsiveness to diverse patient needs?

The use of large language models (LLMs) in patient-facing applications has grown substantially since the release of publicly accessible generative AI tools. In clinical settings, early use cases have included generating postcare patient education, answering patient portal questions, and producing condition-specific discharge materials. These applications are particularly relevant to clinical practice, where patient education is both a professional standard and a time-intensive responsibility. Inadequate postcare patient education contributes to medication errors, preventable readmissions, and adverse outcomes, with the average cost of a single avoidable readmission estimated at approximately US $15,200 and potentially avoidable emergency department encounters representing approximately US $64.4 billion annually in the United States [1-3].

Research has demonstrated that LLMs can produce patient education content at a level comparable to, and in some cases exceeding, the quality of standard institutional materials [4,5]. However, a critical limitation emerges consistently: most LLM-generated health content defaults to literacy levels that exceed national reading averages [6]. Studies have found that AI-generated patient materials frequently score at a 10th-grade reading level or higher, while health literacy guidelines recommend content at a 6th-grade level for broad accessibility [7]. For patients with limited formal education, older adults, or those navigating health crises, this gap translates directly into reduced comprehension and poorer health outcomes [8].

Equity concerns extend beyond readability. The training data underlying most general-purpose LLMs overrepresent White, English-speaking, and formally educated populations [9]. LLM outputs may therefore reflect cultural assumptions, communication styles, and health beliefs that do not align with the lived experiences of patients from racial and ethnic minority groups, those from lower socioeconomic backgrounds, those with less education, or those navigating language barriers [10]. These barriers reflect what has been described as algorithmic bias by omission, a concept central to the growing literature on health AI [11,12]. These gaps are particularly consequential in regions such as the Louisiana Diabetes Belt and Cancer Alley, where Black, Hispanic, and older adult patients carry disproportionate rates of chronic illness and face compounding barriers to clear health communication [13-17]. In algorithmic development, including these patient demographics alongside conditions and comorbidities is necessary if training corpora are to represent the population they serve.

Our aim in this Viewpoint is to demonstrate how clinician evaluation of LLM-generated patient education should account not only for output quality but also for clinicians’ attitudinal regard toward the medical conditions represented in evaluation scenarios and to propose, as one conceptual approach, a framework pairing the Technology Acceptance Model (TAM) with the Medical Condition Regard Scale (MCRS), which we call the TAM-MCRS LLM evaluation framework. The framework was developed through the National Institutes of Health (NIH) Artificial Intelligence/Machine Learning Consortium to Advance Health Equity and Researcher Diversity (AIM-AHEAD) Clinicians Leading Ingenuity IN AI Quality (CLINAQ) fellowship program at Xavier University of Louisiana in partnership with Ochsner Health, with foundational conceptualization supported through the Veterans Affairs Quality Scholars fellowship program. Within the fellowship, the framework was designed to evaluate the performance of LLMs in generating high-quality postcare patient education across accuracy, appropriateness, clarity, and completeness, using clinical vignettes and evaluation by an expert clinician panel. We write for clinician informaticists, health system AI governance leaders, implementation scientists, and investigators evaluating LLM-generated patient education.

The remainder of this Viewpoint proceeds in 3 moves. We first set out the 2 lenses the framework rests on, explaining what TAM contributes to judging whether patient education is usable and what the MCRS contributes to judging the clinicians’ attitudinal variation. We then describe why the two belong together and what neither achieves alone. Finally, we present the framework itself, a 2-arm design in which clinician and automated evaluation are applied to the same outputs and compared, and we close with what this means for practice, health system governance, and the discipline.

The TAM-MCRS LLM evaluation framework addresses a gap that existing evaluation approaches were not designed to close. Most published clinical LLM evaluations measure performance against clinician standards, asking whether a model’s output is accurate enough for a clinician to trust. This framework reorients the evaluative question in two directions simultaneously: is the output accessible, personalized, and equitable enough for a patient to use, and how might measured clinician attitudinal variance influence the ratings used to answer that question?

The first reorientation reflects a core health informatics principle: technology adoption is not determined by technical performance alone, but by the PU and PEOU experienced by the people the technology is meant to serve [18]. The second reorientation, introducing the MCRS as an attitudinal covariate, reflects the discipline’s accountability to equity. Clinician panels are the gold standard for LLM evaluation in health care, but they are not attitudinally neutral. A framework that does not account for this difference is not fully valid for disparity-responsive research.

The stratified, disparity-responsive analysis built into this framework, which examines model performance across vignette race, language, social determinants of health (SDoH) tier, and age, positions equity not as an add-on but as a core dimension of evaluation. Prior research has documented that LLMs are less accurate for populations underrepresented in their training data [22] and less factually precise for subjects that appear there infrequently [23]. This framework applies that insight to patient demographics, testing whether performance gaps emerge systematically for underserved patient profiles. The MCRS covariate analysis adds a second disparity-responsive validity layer, ensuring that observed performance gaps reflect model differences rather than evaluator attitudinal variance.

Multiple frameworks for evaluating LLMs in health care have emerged in recent years, including comprehensive human-evaluation frameworks such as QUEST (Quality of Information, Understanding and Reasoning, Expression Style and Persona, Safety and Harm, and Trust and Confidence), which organizes clinician evaluation around quality of information, understanding and reasoning, expression style and persona, safety and harm, and trust and confidence [24], and multidimensional benchmarking frameworks such as MEDIC (medical reasoning, ethics and bias, data and language understanding, in-context learning, and clinical safety), which assesses medical reasoning, ethics and bias, data and language understanding, in-context learning, and clinical safety [25]. Recent surveys of LLM evaluation in health care similarly observe that prior evaluation has focused narrowly on accuracy and call for broader, multidimensional criteria spanning safety, reasoning, and clinical applicability [26]. Separately, TAM is widely used to study clinician adoption of health technologies [20], and the MCRS measures clinician attitudes toward specific patient conditions [21]. To our knowledge, however, no existing framework integrates these 2 instruments as paired, measured constructs within a clinician-centered evaluation of LLM-generated patient education. Existing work occupies adjacent space, applying TAM to technology adoption or proposing standardized human-evaluation workflows for health care LLMs. However, none combines TAM and MCRS to model clinician usability perceptions and condition-specific regard as analytic covariates alongside output quality, readability, and understandability.

This framework is designed to serve as the evaluative foundation for a subsequent Readability, Disparity-responsive design, and Personalization prompting framework and a future multisite R01 clinical trial examining AI-enabled postcare communication quality across safety-net health systems. When applying our framework in evaluations, criterion scores would be compared across model groups, with the MCRS score entered as an exploratory covariate to assess whether condition-specific clinician regard is associated with rating variation. Repeated ratings and the measurement properties of each criterion would be addressed using appropriate statistical methods, as detailed in the subsequent empirical report. Exploratory stratification would examine whether performance differences vary by vignette race or ethnicity, language preference, SDoH tier, and age group. Future applications of the framework, whether by our collective or others, should publicly document model versions, model prompts, scoring prompts, rubric specifications, and analytic assumptions before data collection to support reproducibility. Full analytic specification, including model syntax and prespecified hypotheses, is provided in the accompanying materials. We recognize that the primary limitation of the framework is that the PEOU scores are currently designed to be provided by clinician proxies rather than by patients themselves. In the future, we are dedicated to incorporating direct patient evaluators from the target populations, including those in the Louisiana Diabetes Belt and Cancer Alley, to cross-validate clinician proxy assessments of patient-facing accessibility against the health literacy and communication barriers experienced by the populations this framework is designed to serve.

At the bedside, informaticists and clinical educators can use the 4-criterion rubric as a practical vetting tool before deploying any LLM-generated patient education material. The checklist-based completeness scoring and Flesch-Kincaid grade-level targets are straightforward to apply without specialized AI expertise, making the framework accessible to frontline staff in postcare communication workflows.

At the health system level, the framework provides a structured basis for LLM procurement and governance decisions. Health systems currently adopt LLMs with limited standardized evaluation criteria for patient-facing applications. A reproducible, multimethod evaluation framework gives informatics leaders a defensible, evidence-based process for comparing tools before deployment, with a documentation trail that supports institutional review board and regulatory accountability.

At the disciplinary level, this framework advances health informatics as a field that evaluates not only whether AI tools work but also whether they work responsibly and whether our methods for determining that are themselves free from attitudinal confounders. As LLMs become increasingly embedded in clinical workflows, informaticists are uniquely positioned to lead evaluation efforts that center patient experience, clinician accountability, and population health outcomes.