We conducted a formative mixed methods feasibility study using (1) synthetic case simulations to generate chatbot-patient interactions, (2) multidisciplinary expert ratings using a 10-domain binary rubric, and (3) rapid qualitative analysis of rater feedback to guide iterative refinement of prompts and rubric items (Figure 1). Primary outcomes were (1) proportion of rubric criteria met by interaction and domain and (2) interrater reliability metrics. Secondary outcomes included thematic categories of improvement opportunities informing subsequent iterations. A convergent mixed methods design was selected because quantitative ratings identified domains of agreement and divergence in chatbot performance, while qualitative comments supplied the context needed to interpret those domains and guide iterative refinement of both the chatbot and the rubric. Reporting was informed by the Good Reporting of A Mixed Methods Study (GRAMMS) [16] recommendations, the Standards for Reporting Qualitative Research (SRQR) [17], and Developmental and Exploratory Clinical Investigations of Decision Support Systems Driven by AI (DECIDE-AI) guidance [18] for early-stage clinical evaluation of AI systems (Checklist 1).

We developed a 10-criterion evaluation rubric, called PF-HCER, by systematically adapting the 4D health care AI evaluation framework described by Abbasian et al [5] and drawing on rubric-development principles described in HealthBench. The framework described by Abbasian et al [5] evaluates health care chatbots across the dimensions of accuracy, trustworthiness, empathy, and performance dimensions, while HealthBench [6] demonstrates rubric-based evaluation that achieves physician-level agreement. Our adaptation process involved mapping 3 of the dimensions—accuracy, trustworthiness, and empathy—to 10 rubric items tailored to the SDoH-specific use case (Table 1). Each rubric item used binary scoring (met/not met), as used by HealthBench, with a free text box for which the rater could offer a justification for their response. The rubric and rater instructions can be found in Multimedia Appendix 1. The PF-HCER was developed as a formative evaluation instrument for this study and should not be considered a validated measure. Establishing its reliability, validity, and broader applicability will require external testing.

Twenty-seven synthetic clinical scenarios were developed by an emergency medicine physician (AL) and a social worker (RJ-K) to represent the spectrum of SDoH presentations encountered in health care settings. Scenarios were synthetically authored based on common emergency department case archetypes and social work referral patterns drawn from the extensive clinical experience of the research team working within US acute care settings. They were not derived from identifiable patient records. Scenarios were designed to vary systematically across multiple dimensions, including patient age, gender, literacy, health literacy, domain of social need and complexity, and urgency of the need (Table 1).

To promote operational alignment with the emergency department environment, we used the Centers for Medicare and Medicaid Services (CMS) accountable health communities framework, which focuses on housing instability, food insecurity, transportation, and interpersonal safety as our primary taxonomy, as it maps identified social needs to International Classification of Diseases, 10th Revision, Clinical Modification (ICD-10-CM) Z-codes, facilitating billing and supporting health system sustainability [19].

However, we expanded the evaluation beyond these CMS domains to include behavioral health and interactional determinants (English proficiency and health literacy) as critical safety and equity “stress tests” for patient-facing AI. Behavioral health domains were included because they frequently function as proximal drivers of social instability, while interactional determinants were included to evaluate the chatbot’s ability to adapt communication at lower levels of literacy and health literacy. This comprehensive approach to scenario development was designed to better simulate a broad spectrum of real-world cases in which the chatbot would need to collect data necessary for appropriate clinical care, health services referrals, and accurate billing. Because scenarios were designed to reflect US emergency care workflows and billing-relevant SDoH domains (eg, Z-code mapping), transferability to other payment systems or care settings may require scenario adaptation.

We iteratively refined a prompt using common prompt engineering techniques [20] to obtain our desired responses from our SDoH chatbot across 3 model providers, including OpenAI, Anthropic, and Google Gemini. Prompts were created by an emergency medicine physician (AL) in collaboration with the multidisciplinary research team consisting of a social worker, nurse practitioner, emergency medicine and internal medicine physicians, clinical informaticists, and software developers. Prompts emphasized trauma-informed communication principles, cultural sensitivity, and appropriate boundary maintenance between data collection and clinical care. In this study, we operationalized screening as standardized identification and documentation of social needs using domain-appropriate questions and structured closure (handoff to the care team). We operationalized clinical interviewing as dynamic risk assessment requiring interpretation of severity, escalation decisions, and extended probing during crises. The prompt explicitly instructed the chatbot to function as a screener and to initiate an escalation/termination protocol when emergent medical or safety concerns were detected. The model was instructed to systematically explore SDoH domains while adapting communication style to patient responses and recognizing situations requiring immediate clinical attention. The full system prompt and example templates are provided in Multimedia Appendix 2.

The chatbot was designed as a screening support tool, not an autonomous clinical decision-making system. Its intended role was to elicit and structure SDoH information and to communicate findings to the care team; it was not designed to independently diagnose, triage, or manage clinical conditions. Human oversight remained integral to the envisioned workflow, with collected information intended to support subsequent clinician review and action. Within this workflow, the system inputs were simulated-patient utterances during the encounter and its outputs were structured SDoH information, and a handoff or escalation signal to the care team; the underlying foundation model’s training data and preclinical performance are proprietary and were not independently characterized, which is itself a limitation of this evaluation.

The final prompt designed in the chatbot development and refinement phase was provided as the system message to “gpt-4o-realtime-preview-2025-06-03.” A licensed clinical social worker (RJ) with >20 years of clinical experience role-played patients across all 27 scenarios, simulating real patient encounters with the SDoH chatbot. The social worker was provided with detailed scenario backgrounds, including patient demographics, social circumstances, emotional state, and presentation style. Role-playing sessions were conducted to generate authentic patient-chatbot interactions that reflected realistic disclosure patterns, emotional responses, and communication styles typical of patients experiencing social needs. Each interaction continued until the chatbot completed its SDoH assessment or reached a natural conclusion point. The social worker was not provided with the rubric criteria while performing the case role-play simulations. We selected clinician role-play rather than programmatic batch generation to preserve naturalistic disclosure patterns, emotional nuance, and conversational variability typical of real SDoH discussions. However, because LLM outputs are stochastic, our approach captures only a single interaction per scenario in this iteration; future work will repeat each scenario across multiple runs and parameter settings to quantify within-scenario variability.

This study used synthetic clinical scenarios and role-play simulation and did not involve real patients, patient data, or identifiable human-subject information. The licensed clinical social worker role-played fictitious cases that were created for the purposes of tool development and evaluation. This project was part of a larger study that was approved by the University of Colorado Institutional Review Board (COMIRB #22‐1944).

Three internal evaluators from the research team with relevant but diverse professional expertise—a social worker (RJ-K), a nurse practitioner (HC), and an internal medicine physician (AMM)—all with more than a decade of clinical experience, independently evaluated all 27 patient-chatbot interactions using the rubric. The evaluators were masked to each other's rubric responses until the rating was complete. In addition to completing the rubric questions, they offered comments to justify their rubric responses. They also offered comments on the performance of the rubric and how it should be improved.

To minimize potential bias during this formative evaluation, we structurally separated the roles of prompt engineering and rating. The author with a commercial interest in the chatbot (AL, a co-founder of the entity that intends to deploy the tool) was responsible for designing and iterating the system prompt but did not participate as a rater on any of the 27 interactions. All ratings were produced by 3 raters—a licensed clinical social worker (RJ-K), a nurse practitioner (HC), and an internal medicine physician (AMM)—none of whom have a commercial interest in the chatbot. Raters were masked to each other’s ratings during the rating process, and scenario orderings were not disclosed in a manner that would identify prompt iterations under evaluation. Consistent with the formative purpose of this manuscript, the aim of this evaluation was to demonstrate a feasible, low-resource predeployment iteration process and to refine a reusable rubric, rather than to provide a definitive external validation of the chatbot’s accuracy. Independent validation with external raters drawn from outside the study team and patient evaluators is planned as a distinct, subsequent phase of work (see Future Directions).

All raters were internal members of the research team who also contributed to development discussions, and one author (AL) holds a commercial interest in the chatbot. These roles may have shaped scenario design, scoring, and interpretation. The formative framing of this work and the planned involvement of external raters and patient stakeholders are intended to mitigate, but do not eliminate, this influence.

The scenarios represented a broad demographic range, with participants aged 15-80 years, including 7 adolescents (aged ≤18 years) and 8 older adults (aged ≥65 years). Gender distribution included 13 males, 13 females, and 1 nonbinary individual. Four cases involved urgent social needs that required immediate intervention, including suicidal ideation and interpersonal violence (Table 2).

The chatbot received high proportions of positive ratings across several domains, whereas lower agreement was observed for domain focus and completeness, safety, and completeness of data capture (Multimedia Appendix 3). Overall, the chatbot received high proportions of positive ratings for accurate interpretation (agreement=0.98%, 95% CI 0.91‐0.99), communication quality and cultural sensitivity (agreement=0.99%, 95% CI 0.93‐1.00), and adaptive questioning (agreement=0.99%, 95% CI 0.93‐1.00) across 27 cases. For these high-prevalence items, Fleiss κ values were near zero (eg, κ=−0.03 to −0.01), indicating that the near-uniform agreement carried little information beyond chance. Safety performance was more variable overall (agreement=0.69%, 95% CI 0.58‐0.78; Fleiss κ=−0.04), with lower scores concentrated in urgent social needs scenarios (agreement=0.42%, 95% CI 0.19‐0.68) and interpersonal safety scenarios (agreement=0.33%, 95% CI 0.12‐0.65). These lower safety scores did not reflect unsafe, toxic, or discriminatory chatbot language. Instead, they reflect by design behavior, as the chatbot was prompted via a Mandatory Hardstop Protocol (Multimedia Appendix 2) to terminate the interaction and escalate to the clinical team when physical or mental health emergencies were detected, rather than continue a more detailed clinical interview. Completeness of data capture varied by scenario type (agreement=0.59%, 95% CI 0.48‐0.69; Fleiss κ=0.18), with higher scores in transportation (agreement=0.83%, 95% CI 0.55‐0.95) and financial scenarios (agreement=0.75%, 95% CI 0.47‐0.91) and lower scores in substance use (agreement=0.20%, 95% CI 0.07‐0.45) and multidomain scenarios (agreement=0.50%, 95% CI 0.25‐0.75). Across all simulated scenarios, raters did not identify overt privacy violations or discriminatory language and observed generally consistent role adherence.

However, algorithmic bias is difficult to assess comprehensively and is highly context-dependent; the absence of observed bias-related failures in a limited scenario set should not be interpreted as a definitive assessment of fairness [31].

Across all scenarios, simulated interactions ranged from 10 to 35 conversational turns and 300‐1200 words. Based on timestamps, scenarios ranged from 2 to 7 minutes. Across rubric items, percent agreement for overall performance ranged from 51% to 100%, with 95% CIs reflecting uncertainty due to the limited sample. Item-level percent agreement and Fleiss kappa are reported in Multimedia Appendix 3.

This study addresses an urgent gap in health care AI evaluation by providing, to our knowledge, a structured approach to evaluation-driven development of a patient-facing chatbot for SDoH AI data collection. Importantly, this evaluation was conducted exclusively in synthetic, role-played simulations and should be interpreted as an early feasibility exercise to guide iteration of prompts, rubric items, and performance thresholds, rather than evidence that the chatbot is ready for patient-facing deployment in real clinical workflows. In the initial evaluation, it performed well across the majority of rubric domains, achieving near-perfect agreement in accurate interpretation, communication quality, privacy, bias prevention, context responsiveness, role adherence, and patient engagement. However, a performance gap was found for the domains of completeness of data capture, completeness of data collection, and scores for safety. These quantitative findings taken together with the qualitative data clarify the tool’s current operational scope. As intended, it functions effectively as a screening-oriented SDoH tool, capable of accurately interpreting and recording data for social needs, but does not yet meet the safety threshold for the more complex task of autonomous clinical interviewing during crises.

As a formative study, the primary contributions of this work are methodological rather than a claim of deployment readiness. Two contributions are particularly relevant for other groups undertaking similar predeployment evaluation of patient-facing clinical AI. First, we introduce a reusable evaluation artifact, the PF-HCER, a 10-item, binary-scored instrument adapted from established health care–AI evaluation frameworks and purpose-built for patient-facing SDoH use cases. The rubric, rater instructions, and full system prompt are provided in Multimedia Appendices 1 and 2 so the instrument can be adopted and extended by other teams. Second, we describe a feasible, low-resource iteration workflow combining synthetic scenario simulation, multidisciplinary expert ratings, and rapid qualitative analysis into an iteration loop suitable for health systems with limited resources for more elaborate evaluation programs.

Our work makes several novel contributions to the rapidly evolving landscape of health care chatbot evaluation. While recent frameworks, such as the HAICEF [12], provide comprehensive evaluation criteria (271 questions across 60 constructs), our approach uniquely addresses the specific challenges of patient-facing SDoH data collection through a targeted, feasible rubric combined with an iterative development model. Unlike general chatbot evaluation frameworks, our method evaluates prospective patient interactions with explicit attention to health equity and trauma-informed communication. The distinction between screening versus clinical interviewing capabilities—with quantitative performance thresholds guiding deployment decisions—directly addresses findings on chatbot limitations and provides a practical framework for staged implementation analogous to clinical trainee progression.

Our iterative, continuous evaluation-driven development approach aligns with emerging calls for continuous monitoring and adaptation of health care AI systems rather than static, one-time validation [36]. This approach acknowledges that the chatbot’s role will evolve over time as performance improves and clinical needs change. In this formative evaluation, the chatbot showed evidence of screening-oriented functionality, accurately interpreting and recording patient responses while maintaining appropriate boundaries and privacy protections. However, performance gaps in crisis situations highlight the current limits of autonomous operation. This staged approach to capability expansion—analogous to graduated responsibility in clinical training—represents a shift from traditional “deploy and monitor” models toward continuous evaluation and refinement that matches intervention scope to demonstrated competence.

The multidisciplinary approach appeared to be essential for thorough evaluation. Differences in priorities among physicians, social workers, and nurse practitioners revealed that effective SDoH screening requires diverse professional perspectives to evaluate both clinical accuracy and relevance to social care. This finding aligns with recent evidence on multistakeholder preferences for AI in health care and addresses documented challenges with patient trust [37,38], when chatbots are evaluated only from single professional viewpoints, critical safety or usability issues may be overlooked. Including clinical team members in predeployment evaluation also facilitates early integration of end-user perspectives, potentially improving both acceptance and effectiveness when deployed in real-world settings. Future iterations will incorporate patient evaluators, a critical step given evidence that current chatbots often fail to understand patient communication styles and concerns [39,40].

Another advantage of our approach is that our rubric balances comprehensive evaluation with feasibility, a critical consideration given that evaluation complexity can limit real-world implementation and impact. While more extensive frameworks exist, our focused approach enabled rapid iteration using synthetic data and interdisciplinary team expertise, reducing resource barriers that often prevent thorough predeployment evaluation.

Our approach demonstrates both important strengths and limitations that warrant careful consideration. The use of synthetic data, while enabling efficient evaluation across diverse scenarios without patient risk, represents a limitation requiring validation with real patient interactions. Additionally, each scenario was evaluated using a single LLM interaction; this design does not estimate response variability across repeated runs and may over- or underestimate performance in any given domain. Our rubric, though more focused than comprehensive frameworks like HAICEF, was intentionally designed for feasibility and iterative use. However, this trade-off may miss evaluation dimensions that emerge only with broader assessment. The absence of patient perspectives in the current evaluation phase is a significant gap that must be addressed before large clinical deployment. Because this study was formative and was designed to describe and demonstrate a predeployment iteration process, refine a reusable rubric, and identify conceptual refinements, the reported proportions should be interpreted as descriptive inputs to iterative refinement rather than as estimates of the chatbot’s generalizable real-world performance. A single naturalistic role-played interaction per scenario was selected for this phase because it preserved the conversational variability, emotional nuance, and disclosure dynamics that surface the prompt, rubric, and conceptual gaps that are the actual outputs of formative work. Quantifying within-scenario variability across repeated runs and parameter settings is one of the objectives of the next evaluation phase (see Future Directions). Additionally, all raters were drawn from a single institutional context, and their expectations may reflect local workflows and norms. Therefore, findings may not generalize to other health systems, countries, or care settings without further validation. In addition, our team-based evaluation, while multidisciplinary, likely carries inherent bias because the chatbot evaluators were internal to the project and contributed to development discussions. Further, the prompt engineer (AL) who refined the chatbot is co-founder of a business that plans to use the chatbot. However, the purpose of this manuscript is to describe an iterative development and refinement process rather than to demonstrate the chatbot’s fairness. Further evaluations will incorporate independent external raters and patient stakeholders to improve objectivity and generalizability. Finally, these results are bounded by the scope of this formative phase, which included 27 synthetic, role-played scenarios generated by a single role-player and evaluated by 3 internal raters with a single interaction per scenario. These findings should be interpreted as exploratory rather than as evidence of the chatbot’s clinical effectiveness, safety, trustworthiness, or readiness for deployment.

Integration and interpretation of findings were performed by team members who also developed the chatbot and rubric; although this enabled rapid iteration, it may have biased interpretation and the prioritization of changes. Notably, one team member (RJ-K) contributed to scenario authoring, performed the patient role-play, and served as a rater, further concentrating the internal perspective.

Consistent with DECIDE-AI, several implementation factors remain unevaluated. Because assessment relied entirely on synthetic scenario simulation, this study could not assess workflow integration, patient acceptability, clinician adoption, usability, operational reliability, or effects on clinical outcomes; these require prospective evaluation before deployment.

The primary objective of the next phase is independent external validation of both the chatbot and the PF-HCER rubric, incorporating external raters from multiple institutions, patient co-raters, repeated runs per scenario to quantify response variability, and prospective assessment of usability, workflow integration, and acceptability. Future work will address these limitations through sequential validation steps. Immediate next steps include finalizing crisis case analysis with clear safety protocols, recruiting diverse patient evaluators to assess comprehensibility and trust, and conducting a clinical pilot with real-time monitoring and multiperspective evaluation. We will examine whether rubric scores predict patient satisfaction and care outcomes and explore automated scoring approaches for scalable evaluation. The next evaluation phase will also recruit independent external raters from multiple health systems, drawn from outside the study team and without commercial interest in the chatbot, to support unbiased validation; repeat each scenario across multiple LLM runs and parameter settings to quantify within-scenario response variability and stability of rubric-level performance; and incorporate patient evaluators as co-raters to capture the end-user perspective that this predeployment phase intentionally defers. These steps together constitute the independent validation that the present formative study offers preparation for rather than substitutes. These staged validation steps align with recent calls for continuous, context-specific evaluation of health care AI systems and regulatory emphasis on prospective monitoring of generative AI in patient-facing applications [41,42]. Next-phase evaluation will recruit external independent raters across multiple health systems and include patient evaluators to assess trust, clarity, and acceptability.

This study provides a feasible approach for predeployment evaluation of patient-facing SDoH chatbots, addressing documented gaps in health care AI assessment. Our evaluation-driven development model offers a replicable, low-resource approach that balances rigor with feasibility, highly relevant to health systems seeking to implement AI interventions responsibly. By establishing explicit performance thresholds for role-based deployment (screening versus clinical interviewing), incorporating diverse professional perspectives, and using synthetic data for rapid evaluation prior to clinical deployment, we demonstrate a pathway for integrating AI into SDoH data collection that mitigates the risk of unintended consequences, including erosion of patient trust in the health system. Although further optimization will require real-world validation and continuous adaptation to achieve effectiveness in diverse patient populations, this framework offers critical processes and milestones for protecting patients from premature exposure to untested systems. As health care chatbots continue to proliferate rapidly, such systematic yet highly feasible approaches to predeployment assessment are essential to realizing AI’s potential for improving trust and health outcomes in vulnerable populations.