This pilot project was conducted as a voluntary performance improvement process to evaluate the implementation of an AI clinical documentation tool within routine clinical practice by a small subset of our ambulatory APPs. This process replicated a previously implemented physician pilot in which no institutional review board (IRB) approval was sought. Within the organization, performance improvement or quality improvement projects do not require IRB approval if their sole purpose is to evaluate, improve, or maintain the quality of a local health care service or operational process. Participation by APPs was optional and not required as a condition of employment. The pilot was initiated to assess workflow impact and address labor union concerns related to the adoption of new technology rather than to obtain generalizable knowledge. This work did not meet the definition of human subject research, and therefore, IRB review was not sought by the pilot’s governance team, which comprised legal counsel, physician leadership, and other health system leaders. No patients were recruited for research purposes, and patient care was not altered beyond routine documentation processes that were already in place for physician workflows within the organization. Use of the AI scribe technology during the pilot was governed by an existing organizational oversight committee, which provided review and guidance regarding implementation, consent, privacy, and responsible use. All data analyzed were derived from standard operational metrics and were deidentified prior to analysis.

The 15 APPs in primary care, who are our largest ambulatory homogeneous APP team and primarily manage established and follow-up patient types, were identified in partnership with the labor unions as a viable group for initially assessing and addressing the labor concerns about the use of the technology. There were several reasons for this selection. The use of primary care as a care area followed our physician specialty pattern of AI scribe implementation and created an opportunity to evaluate a large number of APP-patient interactions in a short period. Our primary care APPs also have the largest geographic regional separation in the health system, which the unions felt may help elicit a broader range of APP concerns.

The deidentified pilot and nonpilot data were obtained from Epic Signal data, and definitions were applied consistently across clinicians and the reporting period of June 16, 2025, to August 31, 2025. The pilot APP data were compared against those from 98 nonpilot APPs (n=77, 78.6% nurse practitioners; n=21, 21.4% physician assistants) in the institution working in pediatrics, women’s health, and medical and surgical specialty clinics during the same time frame. As the comparison was for descriptive purposes for the union, no rigorous analysis was performed by our organization or requested of the vendor.

Abridge reported an operational metric referred to as “effort reduction,” which was presented to our organization as the percentage of AI-generated note content retained by the APP after review and editing. Higher percentages therefore indicate that a larger proportion of the AI-generated documentation was preserved in the final note, whereas lower percentages indicate more substantial editing by the APP. Abridge did not provide details regarding the specific types or locations of edits performed within the notes, which limits interpretation of this metric.

Same-day encounter closure was defined as completion and final signing of the clinical note on the same calendar day as the patient encounter. Time in the note writer represented the cumulative duration of active clinician interaction with the note-authoring interface.

As part of the pilot process, the vendor invited the pilot APP group to complete a proprietary qualitative-type survey, with questions blinded to our organization, to elicit the APPs’ experience with the AI scribe. The vendor-administered and analyzed survey information was reported to our institution by the vendor. No independent reanalysis of the survey data was conducted by our organization, nor was any survey conducted with the nonpilot group. Furthermore, the questions and validity of the survey were not scrutinized.

In 2024, our organization piloted a limited number of physician AI scribe licenses through the same AI scribe vendor. The positive physician sentiment spurred APP interest in the scribe for their own practices. Following guidance from our employee labor relations team, our office submitted a formal notice to the unions representing our APPs that our institution intended to expand and deploy the AI scribe technology to APPs. Under our agreements, the unions required 30 days’ notice prior to the change in any working conditions.

This notice prompted requests for “meet and confer” meetings with the unions to discuss what the representatives of the labor force considered to be a change in working conditions. This process is a legal obligation for California public higher education institutions and their unionized employees under the Higher Education Employer-Employee Relations Act (HEERA), where public employers must negotiate in good faith over wages, hours, and working conditions before implementing changes [12]. These meetings involve a dialogue meant to exchange information and proposals to reach agreements consistent with an existing labor contract.

During these meetings, a range of concerns were raised by the union leaders, APP union representatives, and members of the APP labor force. Common themes included patient privacy and the use of the audio recordings for worker surveillance and discipline. There were general concerns about the provision of workforce training and availability of health system–supplied devices. The unions also raised concerns that the technology would be used to replace APP clinical decision-making, self-generate diagnoses, and formulate treatment plans.

The unions’ primary concerns were addressed through contractual agreements from the vendor that its product aligns with HIPAA (Health Insurance Portability and Accountability Act) standards [13]. This was further supported with the language found in California’s Confidentiality of Medical Information Act, which restricts use and disclosure of medical information [14].

Our institution and the vendor also follow the California Invasion of Privacy Act, which requires consent from all parties before recording private conversations. For AI scribes to record sessions, explicit consent must be obtained from everyone present for the clinical encounter [15].

Additionally, according to state law, select sensitive and marginalized patient populations are also excluded from the use of an AI scribe. Medical data related to abortion, contraception, fertility, and gender-affirming care are considered sensitive and require electronic health record systems to implement privacy controls and limit data sharing [14]. Electronic health record vendors such as Abridge that offer such data services are subject to the Confidentiality of Medical Information Act strict storage, disclosure, and use protections. All users of the AI scribe technology received formal training from the vendor and our organization on the use of the technology being forbidden with these patient populations and the need to stop the recording should any of these issues be brought up during a patient encounter.

The unions’ concerns regarding AI-generated diagnoses and health care plans were assuaged by describing the functional limitations of the product, which does not self-generate diagnoses or plans. Additionally, state law prohibits anyone other than licensed professionals, including AI, from diagnosing and treating patients [16]. The union representatives were also shown that the AI scribe’s sole function is to transcribe conversations into a progress note; all diagnoses, subsequent orders, and treatment plans remain the purview of the APP. Mandatory training was required to ensure that all providers were educated on the compliant use of the AI scribe, constraints of the technology, and patient consent process.

Prior to being able to access the AI scribe technology, all physicians and APPs were required by our health system to complete an online training module. The module outlines the need and process for obtaining patient consent, the logistics of using the AI scribe, and the regulatory restrictions of its use. It also provides guidance regarding verifying the accuracy of the clinical documentation and a means to provide feedback to the vendor if the scribe is inaccurate. These training modules, containing content from the vendor and the health system, satisfied the unions’ concerns. Compliance with these rules and regulations is monitored by our organization’s health information management team and compliance department.

The ultimate goal of the AI scribe expansion to APPs was to reduce administrative tasks and improve efficiency, not to initiate APP disciplinary proceedings. The audio recordings are saved for 30 days before being deleted from the system. While it is conceivable that the recording could be used in a possible disciplinary process, the health system was also able to convey to the unions that the recording could also be used to vindicate an APP falsely accused of unprofessional behavior by staff or patients. The formal notification to the unions also further clarified our health system’s intention that the recordings generated by the AI scribe would not be used for the purposes of employee monitoring except in the process of due diligence related to policy violations or instances of system misuse. Through our organization’s labor consultant, the unions were formally notified that “It is imperative to emphasize that the recordings generated by this application will not be employed for the purpose of employee monitoring. However, in the event of policy violations or instances of system misuse coming to our attention, we are unequivocally committed to addressing such matters with the utmost diligence and in strict accordance with established policies and procedures.”

The use of an APP’s own device in our institution is optional, and our health system supplied tablet-type devices to employees using the scribe. Additionally, it was reinforced that the use of the new technology was optional and not a requirement of the APPs’ conditions of employment. If, for any reason, an APP did not want to use the AI scribe, they were under no obligation to do so.

The unions and the health system agreed to an 11-week (55-workday) pilot study with a select group of APPs to assess the AI scribe as part of their responsibility to evaluate new technology that could impact their members and lay the groundwork for discussing future broader use of the technology by APPs. Our institution’s analytics oversight committee, comprising physician leadership, legal counsel, and other health system leaders, was also engaged to approve and monitor the expansion.

During the pilot, 3 provider feedback sessions were held to provide training, assistance, and elicit feedback on the AI scribe tool. Weekly emails were also sent to APP leadership to identify issues warranting collective bargaining considerations and general progress of the pilot. The training sessions also reinforced the need to follow regulatory and policy expectations regarding visit types that are not allowed to be conducted with an AI scribe.

The pilot tracked 4 data points to report back to the union as part of the evaluation of this technology. These were the total number of notes written by the AI scribe, the average percentage of the AI note kept by the APP, the time spent on notes, and the percentage of appointments closed on the same day of the encounter. Pilot and nonpilot APP data were reported to the analytics oversight committee and APP leadership, and a postpilot survey of the involved APPs was conducted by the vendor.

The principal findings of this pilot demonstrated that AI scribes can be implemented among unionized APPs when partnered with structured labor engagement, privacy safeguards, and optional adoption. The implementation of a pilot program enhanced trust and transparency and, ultimately, helped with adoption of the AI scribe technology by the labor unions. The novelty of the pilot and its structured meetings and trainings allowed for greater input from the represented APPs on whether and how the AI scribe is deployed. This process gave the employees and unions timely access to those in charge of controlling the technology’s implementation.

Union involvement early in the pilot design phase ensured that APP concerns, particularly regarding job security, scope of practice, workload redistribution, and data governance, were explicitly addressed. This collaborative governance approach reduced skepticism and was helpful in overcoming the union’s own published concern that AI limits clinical judgment and risks patient safety.

This performance improvement process provided an intermediate step that helped meet the APPs’ own requests for documentation assistance and provided a platform to evaluate organization-wide expansion. The postpilot comments from the APPs suggested that the AI scribe was perceived by the APPs as helpful with their documentation burden. For our organization, the pilot process offered a potentially scalable solution to future technology deployments with APPs when managing labor-related change management issues.

Prior investigations have consistently demonstrated that AI scribes can reduce documentation burden, improve clinician satisfaction, and enhance patient-clinician interaction by allowing providers to focus more fully on the encounter [5-8]. However, much of this prior work has focused primarily on technological performance, usability outcomes, and time savings metrics, with comparatively limited attention to the organizational and labor dynamics that impact adoption and sustainability.

Barriers such as concerns about data privacy, accuracy, workflow disruption, and potential surveillance or productivity monitoring have been previously documented [9,10]. In contrast, our experience demonstrates that formal partnership with unions introduces a distinct and highly effective mechanism for addressing these barriers. By involving workforce representation directly with governance and decision-making processes, we were able to operationalize trust building in a way that extends beyond traditional stakeholder engagement models.

The partnership model described in this paper offers a pragmatic pathway for testing the new technology while integrating labor regulations. This contrasts with prior published descriptions where governance is emphasized but implementation of oversight is less defined [7,9].

While the observed efficiency differences align with physician-focused AI scribe literature and lay the groundwork for extending these findings to APPs, a group underrepresented in prior studies, there are several significant limitations when making any comparative or generalization claim [5,7,17]. Unknown at the time of this pilot was the likelihood that there were other unknown concomitant initiatives in the institution that may have also impacted documentation workflows. Given the nonrigorous methodology of this pilot, these findings should be interpreted with significant caution.

Notably, the purposeful sampling of the pilot APP group introduced a selection bias, and their comparison against predominantly non–primary care APPs is significantly limiting. The other nonpilot specialties who use APPs may have different documentation needs, patient encounter cadences, and visit volume and type expectations. Additionally, the clinical specialties and patient needs of the 2 groups may be vastly different. This clinical mismatch of pilot vs nonpilot APPs is a major limitation of our experience. Further compounding these limitations, there were no prior performance levels used as benchmark comparisons within either group.

Limiting the operational effectiveness of this work, no patient-level outcomes, satisfaction indicators, or documentation quality checks were assessed during the pilot. No assessments of differences in performances within each participating APP were performed. Analysis of the type of edits performed by the APPs in their AI-generated notes and in what sections was not performed, which is a significant hindrance to understanding any benefit to the technology.

While our health system reported no breaches of compliance with state regulations during the pilot regarding AI scribe use in prohibited populations, we performed no independent audits to assess the instances of nonuse of the scribe. No process was in place, nor was any delineation made, to determine whether nonuse was related to compliance reasons, patient consent, or APP preference. This was a major limitation of our process.

The survey responses provided by the represented staff along with the observed efficiency differences were sufficient to advance the full implementation of the scribe with the unions’ agreement. However, survey analyses were conducted by the vendor, and the research team received only aggregated results, which limited independent verification and only allowed for thematic analysis. As the vendor has a financial interest in the technology’s success, interpreting proprietary survey data as evidence of quality improvement or a measure of clinician response to the technology should be viewed as a severe limitation. In addition, no surveying of nonpilot APPs was performed as a comparison.

Future quality improvement efforts will focus on understanding variability in AI scribe use across APPs and evaluating the impact of AI-assisted documentation on patient satisfaction, clinician well-being, documentation quality, and vendor-reported effort reduction. Additional work may also explore specialty-specific implementation considerations and factors associated with sustained adoption.

Our experience with implementing the AI scribe with APPs should only be viewed as an example process to achieve an institutional goal, not as reinforcing existing AI efficiency studies or validating its use. Our pilot process serves to highlight a negotiated trust between management and labor rather than elimination of risk.

Public statements from collective bargaining organizations related to technology changes for these represented workforces highlight barriers and potential delays to implementation. Organized labor positions also echo assertions made by other authors that robust regulatory frameworks should be implemented to define responsibility and accountability for multiple areas of concern with this evolving technology [9,17].

In this instance, our collective bargaining organizations have taken a cautious stance toward the adoption of AI scribe technology due to patient privacy, accuracy, uncertainty regarding workload expectations, and surveillance concerns. Open dialogue with the labor unions and their members, our emphasis that AI should augment and not replace our clinicians, and the use of a pilot program all aligned with our existing state regulatory protections to move the technology forward at our institution. Piloting the AI scribe with a select group of APPs significantly limited the creation of generalizable scientific knowledge, but it gave us an opportunity for a structured dialogue with the labor unions and a way for them to feel engaged in the evaluation of the technology’s impact on their members.