Prior to recruitment and data collection procedures, this study was approved by the institutional review board of the Affiliated Hospital of Guangdong Medical University (PJKT2024-216). All participants provided written informed consent before data collection and were informed that all study data would be deidentified prior to analysis. The data were stored on secure, password-protected online platforms (including BigQuery for OPBot usage data and the Wenjuanxing platform by Tencent for questionnaire data). Access to the data was restricted to authorized members of the research team to ensure participant privacy and data security, in accordance with institutional policies and applicable data protection regulations. Participants did not receive any financial compensation. However, all participants in both groups were provided with equivalent access to free follow-up services and osteoporosis-related health consultations during the study period as part of standard study support.

A formative randomized controlled trial was conducted in a tertiary hospital from February 2024 to March 2025.

The study recruited patients from February 2024 to September 2024, with follow-up completed in March 2025. Patients diagnosed with osteoporosis were recruited from the orthopedic ward of the study hospital. Eligible patients were approached by a research nurse after the diagnosis of osteoporosis had been confirmed and were invited to participate in the study during their hospitalization. Participants’ personal information was obtained from the hospital’s electronic medical record system. Recruitment took place in the hospital ward and continued until the target sample size was reached.

The inclusion criteria were (1) aged 18 years or older and (2) diagnosed with osteoporosis. The exclusion criteria were as follows: (1) unwilling or unable to use smartphones, (2) hearing impairment, (3) impaired consciousness, (4) critical illness, (5) paralyzed in bed, and (6) language barriers.

The sample size for this formative randomized controlled trial was determined based on methodological guidelines for pilot and feasibility studies, rather than formal hypothesis testing. The target sample size for this study was 100 participants (50 per group), determined based on practical factors, such as available resources and recruitment capacity, to support the assessment of feasibility outcomes (including retention, compliance, and acceptability) and to provide an initial estimate for future confirmatory trials.

This study was designed as a randomized controlled trial. Researcher A assessed participants’ eligibility, obtained written informed consent, and collected baseline data. Researcher B, who was not involved in recruitment, generated the random sequence using Microsoft Excel. To ensure allocation concealment, the sequence was placed in sequentially numbered, opaque, sealed envelopes. Prior to the start of the intervention, participants were randomly assigned in a 1:1 ratio to either the intervention group or the control group according to their enrollment order and the random code indicated on the envelope. Each group included 50 participants.

Due to the nature of the intervention, blinding of participants and the intervention coordinator responsible for delivering the intervention was not feasible, as only participants in the intervention group had access to OPBot. However, to minimize potential bias, the data analyst remained blinded to group allocation during statistical analysis.

Data were collected to assess the feasibility and to explore the potential effectiveness of the intervention.

All statistical analyses were performed using R software (version 4.4.3; R Foundation for Statistical Computing). Continuous variables were summarized as mean (SD) or median and IQR, as appropriate, and compared between groups using independent t tests or Mann-Whitney U tests. Categorical variables were analyzed using chi-square tests or Fisher exact tests.

For adherence outcomes measured at 1, 3, and 6 months, mixed-effects models were used to account for within-subject correlations arising from repeated measurements over time. Depending on the nature of the outcome, logistic mixed-effects models were applied for binary outcomes, ordinal mixed-effects models for ordered categorical outcomes, and linear mixed-effects models for continuous outcomes. In all mixed-effects models, fixed effects included group (intervention vs control), time (1, 3, and 6 months), and their interaction (group×time), along with baseline covariates (sex, age, and education level). A random intercept for each participant was included to account for repeated measurements within individuals. The group×time interaction term was used to evaluate whether the effect of the intervention varied over time. To address multiplicity across adherence outcomes, calcium supplement intake was prespecified as the primary adherence outcome and was analyzed without adjustment for multiple testing. The remaining adherence outcomes, including calcium-rich food consumption, sun exposure, weekly exercise frequency, and total adherence score, were treated as secondary outcomes. Holm correction was applied to adjust for multiple comparisons among the secondary outcomes, and adjusted P values are reported accordingly. A 2-sided P<.05 was considered statistically significant.

A total of 643 participants were screened before the study, and 15.6% (100/643) participants who met the study criteria and volunteered to participate were included and randomized into the intervention (n=50, 50%) or control (n=50, 50%) group. Ultimately, 45% (45/100) patients in the intervention group and 43 (43/100) patients in the control group completed the trial, resulting in a dropout rate of 12% (12/100). This corresponds to an overall missing data rate of 12%, as missing data arose from participant dropout. In the experimental group, 2% (2/100) participants did not complete the second knowledge assessment, and 3% (3/100) participants were lost to follow-up. In the control group, 1% (1/100) participant did not complete the second knowledge assessment, 2% (2/100) participants died, and 4% (4/100) participants were lost to follow-up (Figure 2). At baseline, no significant differences were observed in sex distribution, education level, and age between the 2 groups (Table 1). In addition, comparisons between included and excluded participants showed no statistically significant differences in baseline characteristics (Table S1 in Multimedia Appendix 4), suggesting that attrition was not associated with the measured baseline variables.

This formative randomized controlled trial aimed to preliminarily explore the feasibility and potential effects of a chatbot-based SME tool (OPBot) among patients with osteoporosis. For feasibility, nurses in the intervention group spent significantly less time on SME than those in the control group, indicating that OPBot may have the potential to reduce the workload associated with patient education. In addition, the reliability evaluation of the question-and-answer module showed a high level of interrater agreement, suggesting that OPBot responses demonstrated a relatively high level of reliability in a clinical context.

For effectiveness outcomes, the intervention group had significantly higher postintervention knowledge scores compared with the control group. In this study, the time interval between baseline and postintervention assessments was relatively short (median 5 days) and did not differ significantly between groups, which may reduce the likelihood of differential bias. Nevertheless, because the OPBot intervention included a module developed based on the same knowledge assessment tool, participants in the intervention group may have been more exposed to similar content and therefore more likely to recall or recognize test items. This may have led to an overestimation of the intervention effect on knowledge outcomes.

In addition, the intervention group demonstrated a more favorable trajectory over time for calcium supplement intake, as reflected by a significant group×time interaction. This suggests that the pattern of supplement use evolved differently between groups across the follow-up period, rather than showing a consistent difference at individual time points. The intervention group also showed higher odds of consuming calcium-rich foods across time points, although this association did not remain significant after adjustment for multiple comparisons. However, no significant effects were observed for sun exposure, exercise, or total adherence scores, suggesting that the impact of the intervention on adherence was behavior specific rather than uniform across all domains. Overall, these findings indicate that chatbot-based health education may influence certain aspects of self-management adherence, although its effects may vary depending on the specific behavior.

Previous research has shown that LLM-driven chatbots can provide high-quality and easily understandable health information, demonstrating good potential in improving patient understanding and information access [23,25]. Our findings are consistent with these studies, showing that LLM chatbots may have a positive role in improving self-management knowledge.

Different adherence patterns emerged across various behaviors. Specifically, the changes in calcium supplement intake between the intervention and control groups (group×time interaction: OR 1.49, 95% CI 1.08-2.06; nominal P=.02), although this difference was not statistically significant after Holm correction. The intervention group consistently demonstrated higher adherence to calcium-rich foods at all time points. However, no significant differences were observed in other adherence behaviors or overall adherence scores, suggesting that the effectiveness of educational interventions is behavior specific [39]. Comparatively, behaviors such as supplement intake and dietary choices were more susceptible to the influence of chatbot-assisted education, possibly due to their easier integration into daily life, lower implementation costs, and greater consistency with existing health perceptions. For example, in some cultural contexts (including southern China), diet plays a crucial role in health management, which may further promote the adoption of relevant health recommendations. From a practical perspective, these results suggest that chatbot-based health education interventions should consider both the behavioral characteristics and cultural context of the target population when designing them. For behaviors more easily driven by information (such as diet and supplement intake), enhanced personalized recommendations and continuous feedback can improve effectiveness, while for behaviors more influenced by environment or habits (such as exercise or sun exposure), more complex behavioral support strategies may be necessary. Overall, continued access to educational support may help improve specific compliance behaviors, but its impact on overall self-management remains behavior dependent.

Moreover, although general-purpose LLM-based chatbots (eg, ChatGPT; OpenAI) have demonstrated high levels of readability and interactivity in providing general health information [23,25,28], they may still produce responses that are incomplete, insufficiently specific, or potentially inaccurate when addressing disease-specific questions, particularly in clinical contexts [23-26,40]. In contrast, the OPBot in this study was optimized for a specific disease domain, integrating a real clinical knowledge base and undergoing targeted training, enabling it to improve the professionalism and relevance of its answers while maintaining a good interactive experience. This “domain-customized” design may, to some extent, compensate for the shortcomings of general-purpose models regarding professional depth, thus making it more suitable for specific scenarios of chronic disease SME.

Furthermore, integrating OPBot into the health care system, especially in-hospital clinical education, may help support SME delivery. Although there have been studies on LLM-based chatbots in education, there is still a lack of research on their use within hospital settings [17,25,27]. Typically, SME starts when a patient is admitted to the hospital and runs through the entire treatment and follow-up process. However, the introduction of OPBot may help reshape this traditional process. On the basis of our study, OPBot could serve as a knowledge coach, mainly responsible for providing patients with general osteoporosis knowledge and answering specialized questions. Although LLM-powered chatbots have shown great potential in personalized guidance [25,41], the accuracy and reliability of information are still key issues that must be addressed, especially in the medical field, where any misinformation may have a negative impact on patients. Therefore, current chatbots cannot completely replace the role of human providers [42]. In actual applications, neither nurses nor patients rely on OPBot alone but rather regard it as an auxiliary tool.

Furthermore, Beg [43] emphasized that artificial intelligence (AI) research should establish clear standards for ethical review and transparency to safeguard participants’ rights and mitigate potential risks. While AI shows great promise in emotion recognition, dynamic feedback, and remote intervention, it also faces challenges related to trust, ethical responsibility, and role coordination. Accordingly, this study strengthened data privacy protection, information traceability, and mechanisms for human oversight. For instance, nurses reviewed OPBot responses daily and proactively contacted patients when necessary to provide personalized guidance, reflecting the ethical principle of human-AI collaboration.

Moreover, although chatbots can provide real-time feedback and personalized learning based on patients’ needs, a lack of transparency in algorithmic logic and information sources may lead to bias or misinformation. As Beg and Verma [44] further highlighted, personalization and clinical interpretability are essential in AI-driven digital interventions. Therefore, this study implemented safety mechanisms within OPBot. For example, any inquiry related to medication or treatment automatically triggered the response “Please consult your doctor.” This design ensured that patients made medical decisions under professional supervision, embodying the importance of ethical risk control.

However, this study still has certain limitations. First, the reliance on self-reported adherence data is prone to recall and social desirability bias. In addition, although no significant baseline differences were observed between included and excluded participants (Table S1 in Multimedia Appendix 4), the potential impact of missing data and attrition bias cannot be completely ruled out. Second, the participants were recruited from a single tertiary hospital in China, which may limit the generalizability of the findings. Future studies should conduct multicenter, large-sample randomized controlled trials to provide higher-quality evidence. Extending the follow-up duration in future research would allow for a more comprehensive evaluation of the long-term effectiveness of chatbot-based SME, such as its impact on patients’ bone density.

OPBot may have the potential to improve patients’ self-management knowledge of their disease. In health care settings with limited human resources, leveraging educational chatbots to share part of the educational workload and provide patients with basic self-management guidance may be a feasible strategy. Furthermore, to provide better personalized guidance to patients, continuously expanding the chatbot’s corpus is essential to improving its accuracy and credibility. Finally, this study offers valuable insights for SME for other chronic diseases, although further large-scale and long-term studies are needed to confirm these findings.