We adopted a pragmatist paradigm, focusing on practical questions relevant to community-based clinical education and the monitoring of student-reported learning experiences across clerkship sites [24,25]. We conducted an exploratory observational mixed methods study incorporating quantitative digital log analysis and deductive content analysis of students’ reflection sheets. The quantitative component was the primary analysis and described student-reported clinical experiences, while exploring variation across placement contexts, including the presence of trained CAPs. The reflection sheet analysis provided a complementary descriptive context by mapping students’ learning accounts with predefined MCC-based codes [26]; it was not designed to explain mechanisms or validate the digital log counts. Integration occurred through a joint display that aligned major quantitative findings with MCC-coded reflection patterns and the resulting integrated descriptive interpretation [27]. Reporting and methodological rigor were informed by the Good Reporting of A Mixed Methods Study (GRAMMS) checklist (Multimedia Appendix 1) [28].

All eligible fifth-year medical students at our institution who participated in a community-based clinical clerkship from September to November 2024 were included. Before this 3-week clerkship, students completed 1 year of clinical clerkship at the university hospital. The clerkship cultivated a comprehensive attitude toward patients and community residents and helped students analyze the current state of community medical care, identify issues in local health care, and consider ways to develop and improve community care scientifically and practically.

Community-based clerkships were conducted at 38 institutions across 9 medical districts, including institutions in areas with physician shortages. Allocation was determined by student preference and a needs assessment, considering educational impact and clerkship readiness [29,30]. Since these factors may influence students’ opportunities to participate in clinical procedures and examinations, placement should be interpreted as nonrandom rather than as allocation to an intervention. Of the 112 students, 34 (30.4%) were placed at sites with trained CAPs and 78 (69.6%) at sites without such trained CAPs.

CAPs were considered trained in the university FD program if they had been enrolled in the mandatory program for at least 12 months before the community-based clerkship and were actively involved in supervising students during the clerkship. A site was categorized as including trained CAPs if at least one such CAP actively supervised students during the clerkship; otherwise, it was categorized as a site without trained CAPs. Notably, the latter category does not indicate the absence of FD, supervisor training, teaching workshops, or educational experience among local supervisors.

The FD program was offered by the affiliated university. It was designed to build a shared educational culture within community sites, support community-oriented medical education, and enhance regional health care capacity. In community-based clinical education, CAPs are expected to provide direct student instruction and mentorship and develop and maintain local educational infrastructure [31,32]. The FD program consisted of 2-hour weekly sessions delivered primarily online, with in-person sessions held approximately once per month. Across the 38 available session agendas reviewed for this study, the program incorporated approximately 76 scheduled contact hours. Participation was mandatory for enrolled CAPs. The overall attendance rate was 99.3% on a person-session basis, indicating high and relatively consistent exposure to the FD curriculum among participating CAPs. Sessions were facilitated mainly by faculty members from the affiliated university responsible for community-based medical education and education of health professions, with topic-based contributions from CAPs and invited speakers. The program did not rely on a single standardized textbook or commercial teaching package; rather, sessions were delivered through lectures, workshops, practical exercises, peer sharing, and structured reflection.

Program content included curriculum design and MCC alignment, site-specific goal setting, workplace-based assessment, observation and feedback, microteaching, coaching, simulation-based education, and peer learning. Delivery fidelity was monitored through session agendas, attendance records, documentation of session outputs, and self-assessment and reflection forms. However, the extent to which each FD component was implemented in daily clinical supervision at each site was not directly observed or quantitatively assessed. Hence, this study did not attribute student-reported activity patterns to specific FD components. Additional details are provided in Multimedia Appendix 2.

All 112 students participating in the community-based clerkship were included in the quantitative phase. No formal a priori sample size calculation was performed, as this was an exploratory observational study using a fixed cohort comprising all eligible fifth-year medical students who participated in the clerkship during the study period. Therefore, the sample size was determined based on the number of eligible students and available clerkship placements, rather than a prespecified power calculation.

All students completed baseline surveys before the clerkship. Data on demographic characteristics, academic background, regional quota admission, and baseline rural self-efficacy were collected. Baseline rural self-efficacy was assessed using the rural self-efficacy scale developed by Kawamoto et al. [37] for Japanese medical students. The scale comprises 15 items rated on a 4-point Likert scale and includes four factors: work preferences, evaluation of rural practice, evaluation of rural living, and personal character. The theoretical total score ranges from 15 to 60, with higher scores indicating stronger rural self-efficacy and stronger orientation toward rural practice. Regional quota students were admitted through a prefecture-linked admissions pathway that provided a scholarship of JPY 10.8 million (~US $70,101) over the 6-year medical program and required 9 years of postgraduation service as physicians within the prefecture, including 2 years in medically underserved areas. Assignments to medically underserved areas were determined using the Physician Uneven Distribution Index [38,39].

We constructed a joint display, aligning major quantitative findings with corresponding MCC-coded reflection patterns and the resulting integrated descriptive interpretation in order to describe how the digital log data and written reflection data converged or diverged when representing students’ clinical experiences. However, it did not explain causal mechanisms, differences between placement contexts, or validate the digital log counts.

Descriptive statistics summarized participant characteristics and outcomes. Continuous variables were reported as medians (IQRs), and categorical variables were reported as frequencies and percentages. Differences between the two site categories were assessed using Mann-Whitney U tests for continuous or ordinal variables and chi-square or Fisher’s exact tests for categorical variables. Effect sizes were calculated using rank-biserial correlation coefficients and interpreted according to standard thresholds [42]. Bias-corrected and accelerated 95% CIs for the rank-biserial correlations were estimated using 200,000 bootstrap resamples, with resampling performed separately within the two categories. Furthermore, as a classification-independent sensitivity analysis, we compared the total number of student-reported clinical activity types recorded as performed at least once across all 50 MCC-based items.

To examine whether weekly changes differed between the two site categories, we fitted a Poisson generalized estimating equation (GEE) model including site category, week, and the site category × week interaction, accounting for repeated measures among students. A Gaussian GEE model was fitted as a sensitivity analysis. Since the two site categories differed substantially in placement context, we developed a conceptual diagram to clarify hypothesized contextual overlap and reporting processes involving the presence of trained CAPs, medically underserved placement, home visit exposure, site type, site readiness/local educational culture, baseline rural self-efficacy, regional quota admission, sex, and student-reported clinical experiences (Multimedia Appendix 4).

Exploratory sensitivity analyses used Poisson regression models with robust SEs for the primary outcome, adjusting for home visit exposure, baseline rural self-efficacy, regional quota admission, sex, and site type. Since the presence of trained CAPs and medically underserved placement were strongly associated, with few students in the off-diagonal combinations, medically underserved placement was examined in separate contextual models. A model including both variables was fitted only to assess potential instability arising from sparse off-diagonal cells. Negative binomial models were further fitted because the outcome was count data, with the possibility of overdispersion. Mixed-effects models, facility-level cluster-robust SEs, and region-level clustering were not used as primary analyses because each facility included only 1-4 students and the resulting small clusters made these approaches unreliable for estimating an independent FD-related association. As previous-year or pre-FD student-level digital log data were unavailable for the first year after FD program implementation, before-and-after and difference-in-differences analyses were not performed. All analyses were conducted using R version 4.3.1 (R Foundation for Statistical Computing) and Python 3.12.

This study was approved by the Institutional Ethics Board of Chiba University (approval number: 3425). Participation was voluntary, and all students provided informed consent before data collection. All data collected were anonymized to protect privacy and confidentiality. Participants did not receive compensation for participation.

After excluding absences, holidays or site closure, and days without scheduled clerkship, 1497 eligible student-days were identified. Substantive daily log entries were available for 1491 (99.6%) student-days (99.6%); the remaining 6 (0.4%) student-days had no substantive entries. Delayed-entry duration and postreview correction counts could not be quantified from the retained data, and no standardized supervisor audit was conducted.

A total of 112 fifth-year medical students were included in the analysis. Table 1 summarizes baseline student and placement characteristics for students placed at sites with or without trained CAPs. The two site categories differed in key placement context variables, including medically underserved placement, home visit exposure, and baseline rural self-efficacy. These differences were important contextual confounders when interpreting the findings.

The 50 MCC-based clinical activities were classified into 23 (46%) examination-based, lower-risk clinical activities and 27 (54%) basic clinical and emergency procedures (Multimedia Appendix 3). Figures 1 and 2 compare the median number of student-reported clinical activity types recorded as performed at least once during the 3-week clerkship period.

In the classification-independent sensitivity analysis, students at sites with trained CAPs had a higher total number of student-reported clinical activity types recorded as performed at least once across all 50 items than students at sites without trained CAPs (median 13.5, IQR 9-18, vs median 9, IQR 5-13; Mann-Whitney U=1850.5; P<.001; r=0.40, 95% CI 0.14-0.60).

Weekly trends in the number of student-reported clinical activity types recorded as performed at least once across the 50 MCC-based activities are shown in Figure 3. The category × week interaction was not statistically significant in the Poisson GEE model (P=.45), and the Gaussian GEE sensitivity analysis yielded a similar result (P=.91). Thus, the temporal pattern of change did not show clear differences between the two site categories.

The presence of trained CAPs and medically underserved placement highly overlapped in the observed data, leaving few discordant cases to reliably separate their independent associations. Of the 34 students at sites with trained CAPs, 33 (97.1%) were placed in medically underserved areas, whereas only 4 (5.1%) of 78 students at sites without trained CAPs were placed in such areas. This disparity precluded meaningful descriptive comparison of the sites outside medically underserved areas.

In the adjusted Poisson model with robust SEs, the association for the presence of trained CAPs was attenuated and not statistically significant after adjustment for home visit exposure, baseline rural self-efficacy, regional quota admission, sex, and site type (incidence rate ratio [IRR] 1.24, 95% CI 0.97-1.58; P=.09). Home visit exposure (IRR 1.47, 95% CI 1.08-2.01; P=.01) and baseline rural self-efficacy (IRR 1.17 per SD, 95% CI 1.03-1.32; P=.01) were associated with the primary outcome. In the contextual model with the “university FD program site” category replaced with “medically underserved placement,” the latter was associated with the primary outcome (IRR 1.31, 95% CI 1.02-1.68; P=.04). Models including both variables produced unstable estimates because of their limited overlap. Negative binomial sensitivity analyses showed a similar pattern. Detailed results are provided in Multimedia Appendix 6.

Table 2 presents data extracted from the reflection sheets. The analysis yielded 4097 textual units coded according to MCC learning objectives. Among the top 10 codes, procedure- and examination-related content appeared frequently, including treatment techniques (CS-03-02), physical findings (CS-01-02), investigation techniques (CS-03-01), and medical interviews (CS-01-01). Codes related to organ systems, including the digestive system (PS-02-08) and nervous system (PS-02-03), were less frequent among students at sites with trained CAPs.

In code-level comparisons, CS-03-02 was proportionally more frequent among students at sites without trained CAPs than among students at sites with trained CAPs (206/2780, 7.4%, vs 60/1317, 4.6%; P<.001). In contrast, CS-01-02, CS-03-01, and CS-01-01 did not show statistically significant differences between site categories. Codes related to organ systems, including PS-02-08 and PS-02-03, were proportionally more frequent at sites without trained CAPs (both P<.001). The clinical activity–related subtotal did not differ significantly between site categories.

Representative quotations from students at sites with trained CAPs provided additional context regarding students’ reflections. One student described home care learning as an opportunity to understand patients’ living environments:

In later reflections, the same student described more complex concerns about social context and patient values:

The student further noted:

Another student at a site with a trained CAP reflected on ethical decision-making and interprofessional care. After learning about Jonsen’s four-box method, the student wrote:

After attending a multidisciplinary care conference at a patient’s home, the same student reflected:

The student further noted:

Hence, students reflected on social determinants of health, patient values, ethical decision-making, interprofessional information sharing, and clinical reasoning in home care and community-based contexts.

A joint display (Table 3) was constructed to integrate quantitative digital log findings with the deductively coded reflection sheet findings. The digital log analysis showed student-reported differences in activities across placement contexts, whereas the reflection sheet analysis provided complementary descriptive information on the learning content that students described. The integrated descriptive interpretation indicated that the two sources captured different dimensions of clerkship experience. The reflection sheet findings did not explain activity count differences, establish mechanisms, or validate the digital log counts.

Table 4 summarizes the weekly distribution of code frequencies for clinical procedures and examinations. When the proportion of procedure- and examination-related codes was compared with the total number of codes in the reflection sheets, the highest proportion was observed in the first week, which decreased in the second and third weeks. Therefore, students’ reflections on procedure- and examination-related learning appeared most prominently early in the clerkship, whereas later reflections broadened toward other aspects of community-based clinical learning. For CS-01-02, the proportion was similar between site categories in week 1, lower at sites with trained CAPs in week 2, and directionally higher at those sites in week 3. However, this pattern was inconsistent and was not interpreted as evidence of progressive temporal change attributable to university FD program exposure.

This formative evaluation demonstrated that the student-reported digital daily log workflow achieved 99.6% completion across eligible student-days and provided item- and week-level descriptions of clinical activity experiences across distributed sites. Students at sites with trained CAPs had more student-reported examination-based, lower-risk clinical activity types recorded as performed at least once, with the 50-item sensitivity analysis showing a similar pattern. However, placement category was closely intertwined with other contextual factors, and no statistically significant difference was observed for basic clinical and emergency procedures. Therefore, the findings do not establish an independent effect of supervision by trained CAPs.

Our findings should be interpreted in relation to previous work on clinical participation, community-based clerkships, and FD. Previous studies have reported gaps between medical students’ desired and actual procedural experiences before graduation [2,3]. Furthermore, community-based clerkships provide distinct learning opportunities through broader clinical exposure, care continuity, and direct patient care experiences [9]. In this study, students at sites with trained CAPs had a broader range of student-reported examination-based, lower-risk clinical activity types recorded as performed at least once; however, this pattern was closely intertwined with the placement context.

These findings further relate to the literature on FD in medical education. Reviews have suggested that FD can improve teaching effectiveness, support practice communities, and strengthen educational culture [14,15]. However, our study did not demonstrate the independent effectiveness of supervision by trained CAPs. Rather, the presence of such CAPs is part of a broader educational environment that includes medically underserved placement, home visit exposure, site readiness, local workflow, educator engagement, and local educational culture.

For the next implementation cycle, the digital log workflow should retain the practical daily structure that supported high completion, while prospectively storing completion, delayed-entry, reminder, and postreview correction indicators. Standardized orientation materials should explicitly document the operational definitions of “performed,” “observed/simulated,” and “not performed.”

Since universal item-level verification would impose substantial workload on community supervisors, selected clinical activity items could be confirmed prospectively. The digital log should be linked, where feasible, with CC-EPOC and other workplace-based assessment records to compare student-reported experiences with supervisor-confirmed performance, entrustment, and competency-related outcomes.

Student and supervisor interviews should examine usability, the reporting burden, reporting bias, and how local workflow influences documentation and participation. Future cohorts should include pre- and postimplementation data and better overlap between university FD program exposure and placement context. Furthermore, future research should assess whether performed clinical activities translate into competence, entrustment, professional identity formation, and retention in rural practice [46,47].

This study has several limitations. Students were not randomly assigned to clerkship sites, and the presence of trained CAPs was closely associated with medically underserved placement, home visit exposure, site readiness, local educational culture, and student background. Therefore, the study could not determine whether supervision by trained CAPs was independently associated with student-reported clinical activity experiences. Although we conducted exploratory sensitivity analyses, they could not eliminate the limited overlap between university FD program exposure and placement context. Table 1 includes site-level educational characteristics; however, unmeasured teaching-related exposure among supervisors, especially at sites without trained CAPs, may have affected the contrast between site categories.

The primary outcome was based on student-reported Google Spreadsheet logs. However, it was not independently confirmed by supervisors. Although faculty members monitored entries and the overall completion rate was high, the precise frequency and duration of delayed entries and the number of entries modified after faculty review were not prospectively stored as structured indicators. Furthermore, no standardized audit or partial supervisor confirmation procedure was implemented. CC-EPOC, Mini-CEX, EPA-related records, and other systematically collected workplace-based assessment data were unavailable for the 2024 off-campus cohort. Therefore, the outcome does not directly measure the actual number of times that procedures or examinations were performed, performance quality, procedural competence, entrustment level, or supervisor-confirmed performance. This reliance on student-entered logs may have introduced reporting or registration bias. Accordingly, future implementations should prospectively collect log process indicators and link selected student-reported activities with supervisor-confirmed workplace-based assessment or other educational assessment data.

The reflection sheet analysis used deductive MCC-based coding and was not designed to identify emergent qualitative themes or mechanisms. Moreover, weekly code-level patterns in reflection sheet data, including CS-01-02, were exploratory. Hence, important concepts outside the MCC framework may have been missed. Although all AI-generated candidate codes were reviewed by human researchers and final interrater agreement between the two human reviewers was excellent, the initial AI-human concordance was modest, particularly at the third-tier MCC code level. These findings indicate that Gen AI may be useful for preliminary sorting and consistency checking; however, granular deductive coding of reflective text cannot be delegated to Gen AI without human verification.

No formal a priori sample size calculation was performed because the study used a fixed cohort of eligible students from one academic year. Therefore, the study was not powered to estimate an independent effect of supervision by trained CAPs. Furthermore, the generalizability of this study is limited because it was conducted at a single university within a specific Japanese community-based clerkship system. Hence, the placement structure, FD program, regional quota admission system, medically underserved placement context, Japanese MCC framework, and locally developed Google Spreadsheet–based daily log workflow may differ from those of other institutions or countries. Therefore, the findings should be interpreted as exploratory evidence from one institution and region.

The evaluation period may not capture the long-term sustainability of the digital workflow. As the analysis focused on students’ reported experiences and reflection sheets, whether the observed differences translate into postclerkship performance, objective structured clinical examinations, workplace-based assessments, or later clinical competence remains unclear. The reflection sheet analysis was deductively mapped to MCC learning objectives and did not generate a comprehensive explanatory model of CAP teaching behaviors. Gen AI was used only to support preliminary sorting and candidate coding of deidentified textual units. The final coding decisions and interpretations were made by human researchers. Efficiency, time savings, workload reduction, and usability were not prospectively evaluated; therefore, no conclusions regarding efficiency or workload reduction can be drawn from this study. Hence, AI-generated candidate codes may introduce model-related suggestions or inconsistencies; we addressed this risk by necessitating human review and resolving uncertain cases through researcher discussion.

This exploratory observational mixed methods study illustrated how a digital educational data workflow, combining student-reported daily clinical activity logs with human-verified AI-assisted reflection coding, can be used to describe clinical learning experiences across distributed community-based clerkship sites. The workflow yielded nearly complete longitudinal student-reported data and identified variation in student-reported clinical activity types recorded as performed at least once across placement contexts. However, these differences cannot be attributed independently to the presence of trained CAPs. Reflection sheets provided complementary descriptive information, and the AI component functioned as preliminary coding support that required human verification.