The chatbot system was designed to address challenges related to sustained user engagement and adherence that have been repeatedly reported in real-world use of dCBT-I programs. Prior dCBT-I systems effectively deliver core CBT-I components—such as sleep restriction, stimulus control, sleep hygiene, relaxation techniques, and cognitive restructuring—through standardized, protocol-driven structures that support structured behavioral training and repeated practice [24,25,27,29,30]. While these fixed designs are well suited for implementing evidence-based CBT-I procedures, they offer relatively limited support for helping users cognitively understand, accept, and integrate the rationale behind CBT-I practices into their daily experiences. Difficulties in meaning-making, motivational alignment, and sustained engagement during treatment practice have been associated with reduced adherence and inconsistent long-term use in real-world dCBT-I settings [25,26].

To address these challenges, the system was designed as a conversational support tool that complements standardized CBT-I protocols rather than replacing them. The design rationale was to provide interactive support during the phase in which users form understanding, motivation, and readiness to engage with CBT-I practices. Socratic questioning is used to facilitate the structured exploration of users’ automatic thoughts and emotional responses, while a self-decision mechanism positions users as the primary agents who determine whether and when to engage with specific CBT-I techniques. The dialogue system reflects users’ utterances and contextual cues to support continuity across conversation turns.

The overall system architecture (Figure 1) consists of five core modules that collectively support conversational exploration, user-driven decision-making, and guideline-aligned advice provision within a CBT-I framework: (1) a dialogue generator that produces Socratic questions and empathic responses using a large language model (LLM), (2) an intent classifier that maps user utterances to CBT-I techniques to support the self-decision process, (3) a retrieval module that identifies clinically grounded counseling cases to support advice generation, (4) education templates containing clinically reviewed CBT-I technique content, and (5) an interface layer that enables interaction through button-based selection and multiturn dialogue.

The conversational interaction is structured into 4 sequential stages—education on CBT-I techniques, Socratic questioning, self-decision, and advice provision—as illustrated in Figure 2. This staged design clarifies how standardized CBT-I content is progressively connected to personalized engagement support through dialogue.

The chatbot operates through a 4-stage conversational flow (education on CBT-I techniques → Socratic questioning → self-decision → advice provision). Each stage has a distinct functional objective and interaction logic, while maintaining continuity such that cognitive and emotional contexts established in earlier stages inform subsequent dialogue. Although a single dialogue engine is used across stages, input handling, prompt structure, output objectives, and interface design are differentiated according to the role of each stage.

The study was approved by the Institutional Review Board of Sungkyunkwan University (SKKU 2025-08-067-001). Written informed consent was obtained from all participants prior to participation, and they were informed of their right to withdraw from the study at any time without penalty. The chatbot did not provide clinical diagnosis or treatment and was explicitly presented as a supportive tool designed to facilitate engagement with digital CBT-I guidelines rather than as a therapeutic intervention. To ensure participant safety, individuals were advised that if they experienced discomfort or psychological distress during the study, they could immediately discontinue participation and contact the research team for support. No automated monitoring or intervention for suicidal ideation was implemented, and the system was clearly described as not intended for emergency use or crisis intervention. All conversation data were anonymized and securely stored in accordance with institutional data protection guidelines. Monetary compensation was provided for participation.

In the pilot study, the chatbot demonstrated generally favorable usability and feasibility. As summarized in Table 5, mean scores indicated moderate-to-high usability (SUS mean 3.67, SD 0.64) and satisfaction (CSAT mean 3.74, SD 0.71). User engagement scores were comparatively lower (UES mean 3.30, SD 0.81), particularly on items related to emotional responsiveness. Cognitive load was moderate (NASA-TLX mean 3.34, SD 0.45).

Exploratory analyses were conducted to examine potential associations between demographic variables and usability ratings. A borderline negative association was observed between age and SUS scores (P=.05); however, the effect size was small and of limited practical significance. No statistically significant associations were found between usability measures and other demographic variables, including sex and academic background. Given the formative nature of the pilot evaluation, analyses focused primarily on overall scale-level descriptive statistics.

Qualitative feedback revealed several recurring patterns. Participants frequently highlighted the clarity of CBT-I information, structured questioning flow, and practical solution suggestions, describing the system as “easy to understand,” “informative,” and “well structured.” Many appreciated the initial provision of sleep-related information and the attempt to integrate empathy with reflective questioning.

At the same time, users consistently reported concerns related to delayed response times, repetitive phrasing, limited contextual continuity across conversational turns, and language perceived as overly scripted or robotic. Several participants also noted difficulties with mobile readability, screen size, and scrolling behavior. Others indicated that the chatbot occasionally struggled to fully interpret nuanced or metaphorical expressions, leading to abrupt topic shifts or perceived conversational disconnection.

These qualitative findings informed refinements implemented prior to the experimental study, including improvements in response latency, conversational continuity, repetition reduction, and mobile interface readability. Detailed quantitative item distributions are provided in Multimedia Appendices 2-5. The AI-assisted qualitative grouping results, generated using the prompt described in Multimedia Appendix 1, are presented in Multimedia Appendix 6.

The mean baseline ISI score was 15.63 (SD 4.06), with approximately half of the participants classified in the moderate insomnia range (50%). No significant demographic imbalances were observed between groups; sex distribution (53% women, 47% men) and age were comparable across conditions. Baseline ISI scores are reported to characterize the sample and were not analyzed as outcome variables.

Among the HBM subscales, perceived severity of sleep problems significantly decreased in the exploratory group, whereas it showed an increasing trend in the directive group, yielding a significant group × time interaction in the mixed-effects model (β=0.467, 95% CI 0.078-0.855; P=.02; Welch Δ: g=0.808). Perceived susceptibility remained relatively stable in both groups. Self-awareness increased over time in both conditions, with no statistically significant between-group difference.

Across TPB constructs—outcome beliefs, subjective norms, self-efficacy, and behavioral intention—scores increased from pre- to postintervention; however, no statistically significant group × time interaction effects were observed. The overall direction of change across TPB constructs was consistent with canonical TPB pathways, indicating increased readiness toward CBT-I practices across conditions.

Stage of change scores shifted toward higher readiness levels, with the mean stage moving from contemplation (2) toward preparation (3) or higher. The proportion of participants reaching the preparation stage or above was comparable across groups (exploratory: 80%; directive: 93%). Baseline stage significantly predicted postintervention stage (ordered logit: odds ratio 3.16, 95% CI 1.12-8.92; P=.03). Figure 3 illustrates changes in key cognitive and behavioral outcomes by group over time.

A comprehensive summary of primary and key secondary outcomes, including effect sizes and multiple-comparison–adjusted results, is presented in Multimedia Appendix 7.

SDT-based autonomy scores were higher in the exploratory group and showed marginal significance after covariate adjustment (ANCOVA P=.07, η²=0.168). No statistically significant group differences were observed in overall UEQ-S scores. However, hedonic quality scores were higher in the exploratory group (Hedges g=0.435), although this difference did not reach statistical significance after adjustment (ANCOVA P=.18).

Figure 4 presents group-level comparisons of autonomy and hedonic quality scores across conditions.

Higher levels of difficulty identifying feelings (DIF) and difficulty describing feelings (DDF), as measured by the PAQ-S, were associated with smaller reductions in HBM perceived severity (pooled Pearson r=0.35 and 0.36, respectively). In contrast, lower levels of externally oriented thinking (EOT) were associated with greater increases in self-awareness and outcome beliefs. Figure 5 illustrates these associations.

Log-derived engagement metrics showed several associations with cognitive outcomes. Greater use of causal self-explanation (mean_cause_words) was associated with larger reductions in perceived severity of sleep problems (r=0.47, P=.009). Higher chatbot question frequency (mean_qturns) was associated with smaller magnitudes of change (r=0.40, P=.03). Fewer turns required to accept advice (turns_to_accept) were associated with greater cognitive change, and longer session duration was associated with larger increases in outcome beliefs.

Comparative log analysis indicated that the exploratory group produced more causal words per session (31.6 vs 15.4) and more words per user turn (3.82 vs 2.83), whereas the directive group exhibited a higher proportion of question-tagged turns (0.208 vs 0.098). Advice acceptance rates were higher in the directive condition (0.292 vs 0.054), while early session termination occurred more frequently in the exploratory condition (0.446 vs 0.000).

This study proposed and examined the usability, feasibility, and interaction characteristics of a conversational digital CBT-I support chatbot that integrates Socratic questioning and a self-decision mechanism. The system was not designed to replace standardized CBT-I protocols or to deliver therapeutic content independently; rather, it was intentionally positioned as a dialogue-based assistive layer that supports how users understand, interpret, and engage with established CBT-I practices within digital environments.

Across a pilot usability study and a short-term, 5-day condition-based comparative experiment, the findings suggest that Socratic questioning–based exploratory dialogue can influence users’ cognitive appraisal of their sleep problems. In particular, participants in the exploratory dialogue condition demonstrated a reduction in HBM perceived severity of sleep problems, whereas no comparable pattern was observed in the directive condition. Importantly, this change occurred without a corresponding decrease in perceived susceptibility, indicating a shift in cognitive appraisal rather than disengagement from sleep-related concerns. This pattern aligns with the system’s design goal of supporting cognitive reappraisal while preserving the perceived relevance and seriousness of insomnia-related difficulties.

In addition, the self-decision structure was associated with higher autonomy-related experiences and trends toward enhanced hedonic quality of interaction. In this study, autonomy should not be interpreted solely as a subjective outcome measure, but also as a design construct: the self-decision mechanism explicitly positions users as active agents who may endorse or defer guideline-based actions following cognitive exploration. In this sense, self-decision operationalizes autonomy at the interaction level rather than attempting to personalize or modify CBT-I treatment content itself. Although not all user experience measures reached statistical significance, the observed effect size patterns suggest that enabling users to cognitively commit to prescribed practices may meaningfully shape engagement-related experiences in digital CBT-I contexts.

The observed changes in HBM constructs and stage of change scores indicate that the chatbot primarily supported readiness for change rather than immediate behavioral performance. This interpretation is consistent with both the feasibility-oriented nature of the study and the intentionally brief intervention duration. Given that CBT-I typically requires several weeks of repeated behavioral practice—particularly for core components such as sleep restriction and stimulus control—the present findings should not be interpreted as evidence of sleep behavior change or clinical improvement.

Rather, the system appears to have functioned as a cognitive catalyst that reorganized how users conceptualized insomnia and their relationship to CBT-I guidelines. The overall shift toward higher readiness levels suggests that the conversational structure supported preparatory cognitive work—such as reframing perceived threat, clarifying personal relevance, and acknowledging ambivalence—that commonly precedes sustained behavioral adherence in longer-term CBT-I programs.

Within CBT frameworks, Socratic questioning has traditionally been used as a therapeutic technique to facilitate cognitive restructuring by guiding clients to examine automatic thoughts and underlying assumptions. In the present system, this approach was adapted not as a therapeutic intervention in itself, but as a conversational scaffold designed to support early-stage cognitive exploration in a digital context. This distinction is important, as the system was not intended to replicate face-to-face therapeutic interaction, but to provide structured support for users’ sense-making processes during initial engagement with CBT-I practices.

Log-derived analyses further contextualized these findings. Greater depth of user self-narrative and causal self-explanation was associated with larger reductions in perceived severity, whereas higher frequencies of chatbot questioning were associated with smaller magnitudes of change. This pattern suggests that the qualitative coherence and perceived relevance of dialogue may be more influential than the sheer quantity of Socratic prompts, underscoring the importance of carefully calibrated dialogue design rather than aggressive questioning strategies in conversational support systems.

From a design perspective, these findings suggest that digital CBT-I systems may benefit from augmenting existing protocol-driven structures with interaction mechanisms that support cognitive exploration and autonomous judgment. Importantly, this does not imply that fixed, standardized CBT-I modules are inherently problematic. On the contrary, structured behavioral protocols are essential for delivering sleep restriction, stimulus control, and other core CBT-I techniques, and their fixed nature often facilitates usability, consistency, and adherence.

The present findings instead highlight a complementary design space: supporting how users cognitively interpret and accept these standardized practices. Because many CBT-I components require effortful behavioral performance over extended periods, difficulties in adherence may arise not from insufficient instruction, but from users’ uncertainty, resistance, or incomplete understanding of why these practices are necessary. The combination of Socratic questioning and self-decision offers one operational pathway for addressing this gap by supporting users’ cognitive endorsement of prescribed practices prior to execution, while preserving the directive structure of CBT-I itself.

The results also indicate a potential trade-off between structural guidance and cognitive engagement. While the directive condition showed higher advice acceptance rates, exploratory dialogue was associated with greater depth of self-reflection but also higher rates of early disengagement. This polarization suggests that future systems may benefit from adaptive strategies that dynamically balance exploration and guidance based on real-time indicators of user readiness, engagement, and cognitive load.

Most established digital CBT-I systems, including Sleepio [33], SHUTi [32], CBT-I Coach [31], Somzz [34], and WELT-I [35], deliver treatment through structured, sequential modules emphasizing behavioral prescriptions such as sleep restriction, stimulus control, and sleep diary monitoring. These systems have demonstrated clinical efficacy in randomized controlled trials and meta-analyses and play a critical role in expanding access to evidence-based insomnia treatment.

A recent large-scale meta-analysis of fully automated digital CBT-I including 32 randomized controlled trials reported a moderate-to-large posttreatment effect on insomnia severity (standardized mean difference −0.71, 95% CI −0.88 to −0.54; P<.001; k=32), although statistical heterogeneity was considerable (I²=91%) [68]. Furthermore, therapist-assisted CBT-I has consistently demonstrated stronger effects than fully automated formats, suggesting that supportive or interactive components contribute meaningfully beyond standardized behavioral content delivery.

Systematic reviews examining commercial dCBT-I applications have highlighted the importance of engagement-oriented design. Uyumaz et al [69] observed that most applications incorporate interactive panels, sleep diaries, feedback loops, reminders, goal-setting modules, and motivational components to enhance user engagement and adherence. Despite these design features, adherence remains a persistent challenge in real-world digital behavioral interventions.

Recent clinical evidence suggests that adherence should not be conceptualized solely in terms of module completion. In a randomized controlled trial of smartphone-based dCBT-I, Lee et al [70] demonstrated that behavioral adherence—such as compliance with sleep efficiency targets and sleep restriction recommendations—was more strongly associated with improvements in sleep outcomes than simple completion of modules. Similarly, meta-regression findings from fully automated dCBT-I analyses indicate that completion rate alone does not significantly predict treatment effect size [68], underscoring the importance of qualitative aspects of engagement.

Taken together, these findings identify a specific gap in current digital CBT-I frameworks: while behavioral content is standardized and empirically supported, support for users’ cognitive engagement with these prescriptions—particularly during early exposure and short-term use—remains comparatively limited. Most systems personalize treatment at the level of content sequencing or progress adaptation. In contrast, SleepPathfinder introduces personalization at the level of interaction. Rather than modifying therapeutic guidelines, it supports users in examining their reasoning, emotional resistance, and readiness prior to committing to guideline-based actions.

The observed associations in this study between dialogue depth, self-explanatory language, and changes in cognitive appraisal suggest that interaction-level personalization may address engagement-related challenges identified in prior systematic reviews [69]. By facilitating cognitive meaning-making before behavioral implementation, the system may complement existing dCBT-I frameworks that already provide strong behavioral scaffolding but limited dialogic exploration.

Finally, emerging perspectives on AI- and LLM-supported CBT-I propose that conversational systems may enhance personalization, real-time feedback, and adaptive motivational support [71]. Within this evolving landscape, SleepPathfinder represents an initial step toward integrating structured CBT-I protocols with dialogue-based mechanisms designed to support adherence-related cognitive processes while maintaining the standardized foundation of established digital treatments.

Several limitations should be considered when interpreting these findings. First, the sample size was modest, the intervention period was limited to 5 days, and no clinical sleep outcomes were assessed. Accordingly, the study was not designed to evaluate treatment effectiveness, symptom reduction, or long-term adherence. The pilot study was intentionally designed as a single-session evaluation to assess immediate usability and interaction quality, rather than sustained engagement or behavioral outcomes. Additionally, the system has not undergone clinical validation as a therapeutic intervention, and its effects on formally diagnosed insomnia populations remain to be established in rigorously controlled trials.

Second, participants were recruited from a university population and were not selected based on formal insomnia diagnoses, limiting generalizability to clinical populations. Moreover, recruitment was voluntary, potentially introducing selection bias toward individuals who are more technologically familiar or open to AI-mediated interaction.

Qualitative feedback from the pilot study was analyzed using an exploratory thematic summarization approach rather than a formal qualitative methodology. Thematic grouping was conducted by the research team without independent multirater coding or calculation of interrater reliability. Accordingly, these qualitative findings should be interpreted as descriptive insights intended to inform iterative system refinement rather than as rigorously validated thematic conclusions. Because participants were aware that they were taking part in a study evaluating the system, engagement patterns and reported experiences may have been influenced by observation effects (ie, a Hawthorne effect).

From a technical perspective, some system components exhibited limitations. In particular, generative inference mechanisms used to support Socratic questioning may be vulnerable to misclassification or abstraction errors when applied to open-ended user input. While additional technical validation was conducted to ensure functional reliability of the system components, these mechanisms were not used for clinical decision-making or outcome evaluation. Their limitations, nonetheless, highlight the need for caution when deploying language model–based reasoning tools in mental health–related contexts. Additionally, although retrieval-based grounding was used to reduce hallucination risk, algorithmic biases in dataset composition or embedding similarity may influence which counseling examples are retrieved and emphasized, potentially shaping recommendation framing in subtle ways.

In addition, the intent classification dataset was partially constructed using LLM-based rewriting and annotation of Reddit posts. Although prompts were iteratively refined and outputs were manually reviewed during development, this process may have introduced systematic biases related to language style, community-specific expression patterns, or model-driven categorization. Future work should incorporate more diverse data sources and, where feasible, clinician-reviewed annotations to strengthen external validity.

Ethically, the system was not designed to detect or respond to acute psychological risk, and AI-mediated dialogue introduces risks related to misinterpretation, overdependence, and data privacy. While these risks were mitigated through study design, informed consent, and data anonymization, future deployments will require more robust safeguards, monitoring strategies, and appropriate clinical oversight.

Future research should evaluate the proposed conversational structure in larger and more diverse samples, including clinically diagnosed insomnia populations, with longer intervention periods and behavior-based outcome measures such as sleep diaries and longitudinal usage logs. Critically, future studies should examine whether conversational support during early engagement phases translates into improved adherence and sustained behavioral practice within full-length CBT-I programs.

Although this study was conducted in nonclinical contexts, the cognitive engagement mechanisms identified here may inform future adaptations for individuals with clinically diagnosed insomnia. In particular, the integration of Socratic questioning and self-decision structures may support how users cognitively interpret and commit to standardized CBT-I protocols in both early-stage and clinical populations.

More broadly, this work suggests that digital CBT-I systems may benefit from being designed not as replacements for therapy, but as usability-oriented interaction layers that support users’ cognitive engagement with established, evidence-based treatment protocols. By clarifying how users make sense of and commit to standardized practices, conversational support systems may play a complementary role in improving real-world uptake and persistence of digital CBT-I interventions.