This paper is in the following e-collection/theme issue:

Formative Evaluation of Digital Health Interventions (2410) Sleep Monitoring, Sleep Quality, Sleep Disorders (183) Usability of Apps and User Perceptions of mHealth (974) Mental Health Issues in Adolescence (442) mHealth for Data Collection and Research (955) Design and Formative Evaluation of Mobile Apps (694) Transdiagnostic Mental Health Interventions (160) Insomnia and Sleep Hygiene (167) Sleep Medicine (26)

Published on in Vol 5, No 11 (2021): November

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/25392, first published .
A Transdiagnostic Self-management Web-Based App for Sleep Disturbance in Adolescents and Young Adults: Feasibility and Acceptability Study

A Transdiagnostic Self-management Web-Based App for Sleep Disturbance in Adolescents and Young Adults: Feasibility and Acceptability Study

A Transdiagnostic Self-management Web-Based App for Sleep Disturbance in Adolescents and Young Adults: Feasibility and Acceptability Study

Authors of this article:

Nicole E Carmona1 Author Orcid Image ;   Aleksandra Usyatynsky1 Author Orcid Image ;   Samlau Kutana2 Author Orcid Image ;   Penny Corkum3, 4 Author Orcid Image ;   Joanna Henderson5, 6 Author Orcid Image ;   Kelly McShane1, 7 Author Orcid Image ;   Colin Shapiro5, 8 Author Orcid Image ;   Souraya Sidani9 Author Orcid Image ;   Jennifer Stinson10, 11, 12 Author Orcid Image ;   Colleen E Carney1 Author Orcid Image
Article Authors Cited by (18) Tweetations (13) Metrics

    Original Paper

    1Department of Psychology, Ryerson University, Toronto, ON, Canada

    2Department of Psychology, Memorial University of Newfoundland, St. John's, NL, Canada

    3Department of Psychology and Neuroscience, Dalhousie University, Halifax, NS, Canada

    4Department of Psychiatry, Dalhousie University, Halifax, NS, Canada

    5Department of Psychiatry, University of Toronto, Toronto, ON, Canada

    6Margaret and Wallace McCain Centre for Child, Youth and Family Mental Health, Centre for Addiction and Mental Health, Toronto, ON, Canada

    7Human Resource Management and Organizational Behaviour, Ted Rogers School of Management, Ryerson University, Toronto, ON, Canada

    8Department of Ophthamology, University of Toronto, Toronto, ON, Canada

    9Daphne Cockwell School of Nursing, Ryerson University, Toronto, ON, Canada

    10Chronic Pain Program, Department of Anesthesia and Pain Medicine, The Hospital for Sick Children, Toronto, ON, Canada

    11Child Health Evaluative Sciences, Research Institute, The Hospital for Sick Children, Toronto, ON, Canada

    12Lawrence S Bloomberg Faculty of Nursing, University of Toronto, Toronto, ON, Canada

    Corresponding Author:

    Colleen E Carney, PhD

    Department of Psychology

    Ryerson University

    350 Victoria Street

    Toronto, ON, M5B 2K3

    Canada

    Phone: 1 416 979 5000 ext 552177

    Email: ccarney@ryerson.ca


    Background: Sleep disturbance and its daytime sequelae, which comprise complex, transdiagnostic sleep problems, are pervasive problems in adolescents and young adults (AYAs) and are associated with negative outcomes. Effective interventions must be both evidence based and individually tailored. Some AYAs prefer self-management and digital approaches. Leveraging these preferences is helpful, given the dearth of AYA treatment providers trained in behavioral sleep medicine. We involved AYAs in the co-design of a behavioral, self-management, transdiagnostic sleep app called DOZE (Delivering Online Zzz’s with Empirical Support).

    Objective: This study tests the feasibility and acceptability of DOZE in a community AYA sample aged 15-24 years. The secondary objective is to evaluate sleep and related outcomes in this nonclinical sample.

    Methods: Participants used DOZE for 4 weeks (2 periods of 2 weeks). They completed sleep diaries, received feedback on their sleep, set goals in identified target areas, and accessed tips to help them achieve their goals. Measures of acceptability and credibility were completed at baseline and end point. Google Analytics was used to understand the patterns of app use to assess feasibility. Participants completed questionnaires assessing fatigue, sleepiness, chronotype, depression, anxiety, and quality of life at baseline and end point.

    Results: In total, 83 participants created a DOZE account, and 51 completed the study. During the study, 2659 app sessions took place with an average duration of 3:02 minutes. AYAs tracked most days in period 1 (mean 10.52, SD 4.87) and period 2 (mean 9.81, SD 6.65), with a modal time of 9 AM (within 2 hours of waking). DOZE was appraised as highly acceptable (mode≥4) on the items “easy to use,” “easy to understand,” “time commitment,” and “overall satisfaction” and was rated as credible (mode≥4) at baseline and end point across all items (logic, confident it would work, confident recommending it to a friend, willingness to undergo, and perceived success in treating others). The most common goals set were decreasing schedule variability (34/83, 41% of participants), naps (17/83, 20%), and morning lingering in bed (16/83, 19%). AYAs accessed tips on difficulty winding down (24/83, 29% of participants), being a night owl (17/83, 20%), difficulty getting up (13/83, 16%), and fatigue (13/83, 16%). There were significant improvements in morning lingering in bed (P=.03); total wake time (P=.02); sleep efficiency (P=.002); total sleep time (P=.03); and self-reported insomnia severity (P=.001), anxiety (P=.002), depression (P=.004), and energy (P=.01).

    Conclusions: Our results support the feasibility, acceptability, credibility, and preliminary efficacy of DOZE. AYAs are able to set and achieve goals based on tailored feedback on their sleep habits, which is consistent with research suggesting that AYAs prefer autonomy in their health care choices and produce good results when given tools that support their autonomy.

    Trial Registration: ClinicalTrials.gov NCT03960294; https://clinicaltrials.gov/ct2/show/NCT03960294

    JMIR Form Res 2021;5(11):e25392

    doi:10.2196/25392

    Keywords

    youth; sleep; technology; mHealth; self-management; adolescents; young adults; mobile phone 


    Background

    Emerging adulthood is a time of risk for sleep problems. An epidemic of shortened sleep times and resultant sleepiness in teens and young adults has been reported worldwide [1,2]. Sleep problems are reported in two-thirds of adolescents and young adults (AYAs) [3-5] and are linked to academic and mental health problems and suicidality [6-11]. AYAs also experience a pubertal shift toward later or delayed circadian rhythmicity, such that they become tired later, resulting in later rise times, although they must maintain early rise times to attend school [12-15]. This biological shift toward a later sleep onset time is further compounded by the fact that AYAs are sensitive to evening light [12,16,17] and are exposed to academic (eg, early school start times) and social pressures (eg, peers reinforcing later activities), which can increase sleeplessness. Moreover, some AYAs exert control over what they eat and drink (eg, caffeine) without an adequate understanding of the consequences of particular behaviors on their sleep and general health [18]. As a result, AYAs have chronic sleep loss, resulting in pathological sleepiness, a condition associated with an increased risk of motor vehicle accidents and poor emotional, interpersonal, and academic functioning [19,20].

    Providing accessible interventions for sleep problems is crucial to mitigate personal, social, and economic costs. Treatment options for most sleep problems typically consist of pharmacological and behavioral approaches [21,22]. Sleep medications are associated with significant concerns regarding use by AYAs [23]. Although cognitive behavioral therapy for insomnia (CBT-I) approaches are the most efficacious for helping individuals with insomnia with few troublesome side effects [24], few AYAs have access to behavioral sleep medicine treatments [25,26]. Furthermore, CBT-I protocols (when accessible) do not target the range of sleep problems that AYAs experience beyond insomnia symptoms, including hypersomnia, excessive daytime sleepiness (eg, from voluntary sleep restriction), delayed phase circadian rhythm, and often uninformed use of sleep-interfering substances, some of which may be experienced concurrently, highlighting the promise of transdiagnostic behavioral sleep treatments for AYAs [27-29]. There are multiple barriers to accessing nonpharmacological evidence-based sleep treatments, including poor dissemination of CBT-I and behavioral sleep treatments in real-world settings [30] and a shortage of providers who are trained in such treatments [31]. It is clear that AYAs have diverse sleep problems (ie, not just insomnia) and require tailored evidence-based strategies that comprehensively address these diverse needs, making transdiagnostic cognitive behavioral treatments [32] a more suitable option than nonadapted CBT-I.

    Smartphone apps are uniquely suited to improve access for those who are not seeking treatment but are nevertheless at risk as well as treatment-seeking groups with poor access to evidence-based treatments, and reportedly increase users’ motivation, goal-setting, and confidence in and control over their ability to enact health-promoting behavior change [33]. AYAs are drawn to technology; therefore, technology provides an opportunity to directly reach this large invisible at-risk cohort for whom sleep disorders contribute to psychopathology and vice versa [34,35]. Although there are web-based programs for infant (aged 6-36 months) [36], pediatric (aged 1-8 years) [37,38], adolescent [39-41], and adult sleep disturbances [42], there are no evidence-based transdiagnostic programs that adequately address the multitude of sleep problems observed in AYAs and that span both adolescence and young adulthood. Put differently, as some of the sleep problems of AYAs are unique and go beyond insomnia (eg, voluntary sleep restriction, circadian phase delay, and poor sleep hygiene), pediatric and adult programs are not suitable for this age group.

    Objective

    In response to a dearth of appropriate treatment options for AYAs with a range of sleep difficulties beyond insomnia, we created a self-management app called DOZE (Delivering Online Zzz’s with Empirical Support), which was co-designed by AYAs in user consultation sessions conducted by the developers of the app. DOZE is based on the principles of cognitive behavioral sleep medicine [32] and allows AYAs access to effective, tailored strategies to address a range of sleep problems (eg, insomnia, daytime sleepiness, and delayed phase circadian rhythms), which also range from subclinical to clinical in terms of their severity. Importantly, consistent with their preferences for technology and autonomy and as revealed by the user consultation sessions, the self-management approach entrusts AYAs to set their own goals and make their own health behavior changes when provided with the tools and education to do so. This study reports on an open trial evaluating the app’s feasibility over a 4-week intervention period. The primary objective of this study is to evaluate whether AYAs regard evidence-based cognitive behavioral strategies in a self-management app (DOZE) as acceptable and credible. Toward the aim of evaluating feasibility, we also want to understand how AYAs use DOZE and which aspects of the app they find most valuable. As AYAs co-designed the app, we hypothesize that AYAs would rate the app as acceptable and credible and report high satisfaction. No a priori hypotheses are made regarding which aspects of the app would be most valuable to AYAs, and an exploratory analysis of app use patterns is conducted to understand which aspects of the app are most useful and to provide further insight into AYAs’ primary sleep complaints and treatment goals.

    The secondary objective is to examine preliminary efficacy, acknowledging that these results may be quite modest because of the range of sleep problems addressed (eg, both insomnia and hypersomnia) and that, without sleep disturbance severity as part of the inclusion criteria, we might recruit a subclinical sample. Outcomes focus on whether DOZE positively affects health-related behavior change, improvements in daytime symptoms (ie, sleepiness, fatigue, energy, and psychological symptoms), and health-related quality of life. We hypothesize that AYAs who use DOZE would see improvements in sleep indices at end point. Furthermore, we hypothesize that the use of DOZE for 4 weeks may contribute to improvements in energy, mood, and health-related quality of life at end point compared with baseline.


    Participants

    Individuals between the ages of 15 and 24 years with self-reported dissatisfaction with their sleep were eligible to participate in this web-based study. Given that a clinical diagnosis of a sleep disorder is not required to use DOZE (ie, any AYA who wants to improve sleep can use it), this study did not have any other inclusion or exclusion criteria to be able to generalize the results of this study to the population that will use DOZE. Self-reported dissatisfaction with sleep was queried with a single question, “How do you feel about your sleep?” and participants could respond using an open textbox. Computer and internet literacy of participants was assumed. Participants were recruited using posters at Ryerson University and in the downtown Toronto community, dissemination of study information to mental health providers via listservs of professional organizations (eg, Canadian Association for Cognitive and Behavioural Therapy and Ontario Psychological Association), and via social media (eg, Twitter and Facebook). Recruitment materials advertised the opportunity for AYAs who were dissatisfied with their sleep or were experiencing difficulty sleeping to participate in a study testing a new sleep app for 4 weeks. Participation in this study was quasi-anonymous. Email confirmation was used to detect repeat participation. All participants provided informed consent and completed a quiz assessing their understanding of critical components of the consent form (detailed in the Procedure section).

    Intervention: DOZE App

    DOZE [43] is a transdiagnostic, self-management, cognitive behavioral web-based app for sleep disturbance among AYAs. It is freely accessed from a web browser on any device (ie, it is not downloaded on a device). DOZE was designed using an iterative, experience-based co-design process that included feedback from AYA users and health care provider stakeholders [44-46]. Initial design consultation interviews conducted by an industry partner with AYA stakeholders revealed that, in addition to including a sleep diary for self-monitoring, AYAs wanted an app that provides feedback on their personal sleep habits (over and above feedback on their sleep), opportunities for goal-setting, and tips on how to make changes to their sleep. Following this initial phase, a user-informed redesign of DOZE was conducted with an industry partner (PIVOT Design), consistent with standard app development processes [47], including 6 user assessments with AYAs (aged 15-24 years) on iterative prototype versions of the app and consultations with health care provider stakeholders. Of note, AYAs who contributed to the co-design of DOZE were not participants in this study. Additional details about the co-design process are presented in Multimedia Appendix 1.

    The final version of DOZE includes a sleep diary, adapted with permission from the Consensus Sleep Diary [48], comprising 8 questions that assess the following: bedtime (ie, time of getting into bed whether or not it was to initiate sleep), time of sleep attempt (ie, time of trying to sleep), time it took to fall asleep (sleep onset latency [SOL]), total time awake in the night (wake after sleep onset), time of final awakening (ie, time of waking in the morning), rise time (ie, time of getting out of bed), amount of time spent napping, and use of sleep-interfering substances within 2 hours of bedtime. DOZE’s onboarding screens instructed participants to complete their sleep diary every morning within 2 hours of waking, and they were reminded of this using a pop-up notification if they completed it that same day outside of the 2-hour window. Completing an entry for the previous day was not permitted. Participants also received an automated reminder email each morning to complete the diary. Sleep indices were calculated based on information provided by the sleep diary, including SOL, variability in rise time (range of earliest to latest rise time), variability in bedtime (range of earliest to latest bedtime), wake after sleep onset, morning lingering in bed (the difference between time of final awakening and rise time), total sleep time (TST; total time spent asleep), total wake time (TWT; total time spent awake during the sleep period), time in bed (TIB; bedtime to rise time), and sleep efficiency (SE; percent of TIB spent asleep). Participants were able to view their average sleep indices over a 2-week period on a personalized dashboard.

    After completing 2 weeks of sleep diaries, DOZE provides personalized feedback on sleep indices (TST, SOL, TWT, TIB, and SE) relative to age-adjusted norms [4] by indicating whether their indices are normal, too high, or too low on the Progress screen. Users then have the opportunity to set goals to improve their sleep in problem areas that are identified (ie, indices are outside the normal range). Feedback areas, which all represent components of evidence-based treatment [32], include naps, too much/too little TIB, sleep-interfering substances, lingering in bed in the morning, sleepiness, and jetlag without traveling (ie, sleep schedule variability). AYAs have the opportunity to set personalized goals in these areas (eg, reduce jetlag by 1 hour, 2 hours, or 3 hours), which are added to their personal dashboard, or they may choose not to set goals in identified areas, accounting for various stages of readiness to change. Notably, if a feedback area is not relevant to the individual based on their sleep indices (eg, they are spending a normal amount of TIB), they are not provided the opportunity to set a goal to improve in that area (eg, they are not shown the feedback area too much/too little TIB). In addition, participants could access a Tips section that included quizzes and psychoeducation about hyperarousal (difficulty winding down), chronotype (night owl living in an early bird’s world), difficulty getting out of bed in the morning, daytime sleepiness (trouble staying awake during the day), and fatigue (exhausted). Tips can also be saved to participants’ dashboards for a personalized user experience, and all participants have access to all of the tips. After setting goals, participants then complete sleep diaries for an additional 2 weeks and, at the conclusion of the second 2-week period, they are able to see a comparison of their progress between weeks 1-2 and weeks 3-4 on their dashboard. Screenshots of the example progress, goal-setting, and tips screens are presented in Figure 1. Participants did not receive any other support or interventions in this study.

    ‎
    Figure 1. Screenshots of Delivering Online Zzz’s with Empirical Support: progress and feedback, goal-setting, and tips.
    View this figure

    Measures

    Outcomes were assessed using analysis of aggregate DOZE use during the study period and through the completion of web-based qualitative and quantitative questionnaires.

    Feasibility and Acceptability

    Treatment acceptability, credibility, and satisfaction were assessed using both quantitative and qualitative measures. A 5-item Treatment Evaluation Questionnaire (TEQ) was administered at baseline and end point and assessed participants’ perceptions of DOZE as a logical treatment, beliefs that it would be successful, confidence in referring it to a friend, willingness to undergo treatment, and whether they thought it was successful in treating others [49]. As reviewed in the Introduction section, DOZE is a self-management intervention rather than a treatment for clinical sleep disorders. The use of the word treatment in evaluating the perceived credibility of DOZE reflects the verbatim wording of each item of the TEQ, which has been used in previous clinical trials of sleep medicine treatments [50]. Each item is scored from 1 (not at all) to 7 (very), with higher scores reflecting greater perceived credibility and satisfaction with DOZE. In addition, a 6-item Acceptability Scale was designed for the purpose of this study and administered at end point. This measure assessed participants’ attitudes regarding ease of use, understandability, enjoyment, perceived helpfulness, time commitment, and overall satisfaction. Participants rated each item on a scale of 1-5, with higher scores denoting greater acceptability and more favorable attitudes toward DOZE. In addition, participants completed a qualitative exit questionnaire about their experience using DOZE at end point, including how they felt about their sleep and the goals they set, what they liked and did not like, and whether there was anything they would change about DOZE. An analysis of qualitative responses will be reported in a separate paper.

    Google Analytics was used to provide additional information about DOZE use, including the number of page views; where, when, and how DOZE was accessed; and the mean length of sessions using DOZE. Consistent with what is available on the Google Analytics website, only averages are provided, as other measures of central tendency are unavailable.

    Preliminary Efficacy

    Improvements in sleep and sleep-related behaviors were evaluated by analyzing changes in the following sleep and behavioral indices derived from the sleep diary: variability in bedtime, variability in sleep attempt, variability in wake time, variability in rise time, morning lingering in bed, SE, TST, TWT, TIB, duration of naps, and frequency of sleep-interfering substance use. In addition, participants completed self-report questionnaires at baseline and end point to investigate whether the use of DOZE for 4 weeks contributed to improvements in insomnia severity, chronotype, daytime symptoms, and health-related quality of life.

    The Insomnia Severity Index (ISI [51]) is a 7-item scale that evaluates self-reported sleep disturbances. Items are scored on a 5-point Likert scale from 0 (no sleep difficulty) to 4 (severe sleep difficulty), with total scores ranging from 0 to 28. A cutoff score of 10 distinguishes between good and poor sleepers in adult community samples [52]. Previous research has suggested a cutoff score of 9 among adolescent samples [53]; however, because of the larger proportion of young adults in our sample relative to adolescents (see the User Characteristics section), we used the more conservative cutoff score of 10 with the understanding that clinical levels of insomnia may be underestimated in our sample. The ISI has been validated in AYA samples and is a validated outcome measure with good psychometric properties [53-56].

    Chronotype, which refers to one’s endogenous circadian preference, was measured using the 13-item Composite Scale of Morningness (CSM [57]). Total scores range from 13 to 55, with higher scores reflecting greater morningness preference; based on established cutoffs, scores between 44 and 55 indicate morning type, between 23 and 43 indicate intermediate type, and between 13 and 22 reflect evening type [57]. The CSM has been validated in samples of adolescents and undergraduate students [57-59].

    Fatigue was measured using the Fatigue Severity Scale (FSS [60]). The FSS is a 9-item scale measuring impairment related to fatigue, and items are rated on a Likert scale from 1 (no impairment) to 7 (severe impairment). A mean item score is calculated, with higher scores indicating more severe fatigue. The FSS has demonstrated convergent validity with a visual analog scale for fatigue [60] and has previously been used to measure fatigue in clinical samples of adolescents [61,62].

    Daytime sleepiness was measured using the Cleveland Adolescent Sleepiness Questionnaire (CASQ) [63], which comprises 16 items scored on a 5-point Likert scale ranging from never to almost every day. Items are scored to yield a total score, as well as a Sleepiness subscale and a reverse-coded Alertness subscale, with higher scores indicating greater daytime sleepiness. The CASQ has been validated in samples of clinical and nonclinical adolescents [63].

    The Center for Epidemiologic Studies Depression Scale–Revised 10-item Version for Adolescents (CESDR-10 [64]) was included to evaluate depression symptom severity. The item assessing suicidality was removed to mitigate risk (because of the nature of the web-based study), leaving a 9-item scale. Participants rated the frequency of each symptom during the past 2 weeks, ranging from 0 (not at all or less than 1 day) to 4 (nearly every day for 2 weeks). In this study, total scores ranged from 0 to 36, with higher scores reflecting greater depression symptom severity. The CESDR-10 has demonstrated excellent internal consistency in adolescents and is a valid screening measure for depression [64].

    Anxiety symptoms were assessed using the State-Trait Inventory of Cognitive and Somatic Anxiety (STICSA [65]), which comprises scales assessing the presence of cognitive and somatic symptoms of anxiety at the moment it is being completed (State) and in general (Trait). The State and Trait scales include 21 statements, and agreement is rated on a scale from 1 (not at all) to 4 (very much so). Total scores on both scales range from 21 to 84. The STICSA has demonstrated reliability and validity in undergraduate and adolescent samples [66-68] and has demonstrated sensitivity to change [68].

    The RAND 36-item Short Form Health Survey 1.0 (SF-36 [69]) was included in this study to evaluate health-related quality of life. The SF-36 consists of 36 items that assess the following domains (scales): physical functioning, role functioning–physical, role functioning-emotional, energy, emotional well-being, social functioning, pain, and general health. Each scale score ranges from 0-110, with higher scores denoting better health functioning. Each scale has demonstrated adequate to excellent internal consistency [70].

    Procedure

    Interested AYAs were directed to the DOZE website, where they were provided with a consent form and details of our privacy policy. As the study took place on the web, a brief quiz followed the consent form to ensure that participants understood key aspects of informed consent, including the purpose of the study, risks and benefits of participation, and limits of confidentiality. After the provision of informed consent, participants received a randomly generated 5-digit study ID and completed a battery of questionnaires using Qualtrics, which included a demographic form and the ISI, FSS, CSM, CASQ, CESDR-10, STICSA State and Trait, SF-36, and TEQ. At the end of the questionnaires, participants received a link to access DOZE and create an account.

    As indicated by the in-app onboarding process, participants were instructed to use DOZE for 2 weeks (ie, period 1), after which they would receive personalized feedback on their sleep, have the opportunity to set goals and access tips, and track their sleep for 2 more weeks (ie, period 2) to examine their progress. Participants were contacted via email by the study coordinator to maintain adherence with sleep tracking if they missed several consecutive days of completing the sleep diary and were sent a standardized email encouraging them to get back on track the next day. Following the completion of the 4-week study period, participants were emailed a link to complete the web-based posttest battery of questionnaires that included the ISI, FSS, CSM, CASQ, CESDR-10, STICSA State and Trait, SF-36, and TEQ, as well as the Acceptability Scale and the Qualitative Exit Questionnaire. Participants were compensated CAD $12.50 (US $9.79) for each study component that they completed; thus, participants who completed all 4 components of the study were emailed a CAD $50 (US $39.17) gift card honorarium for their participation. This study was approved by the research ethics board of Ryerson University, Toronto, Canada. Data collection took place between September 2019 and January 2020. There were no significant changes to the methods of the study or the intervention (eg, bug fixes, downtimes, or content changes) while this study was underway.

    Statistical Analysis

    User characteristics, including self-reported demographic and baseline clinical characteristics as well as sleep indices from period 1, were evaluated using descriptive statistics for all participants who created a DOZE account. The demographic characteristics and scores on all self-report measures at baseline were compared between early dropouts (ie, before, during, and after the baseline questionnaires) and those who went on to create a DOZE account to assess for possible differences, as well as between all dropouts and those who completed the study (study finishers). To evaluate the feasibility of DOZE, including the characteristics and app use patterns of all DOZE users, analyses of app use (ie, number of days of sleep diary completion, use of goals and tips, and Google Analytics) included participants who created an app account regardless of whether or not they completed the study. Histograms and P-P plots were generated for visual inspection of normality, and extreme outliers on sleep indices were removed on the basis of visual inspection of boxplots. Descriptive statistics were used to analyze demographic and clinical characteristics. Patterns of app use were analyzed using descriptive statistics and Google Analytics.

    Owing to insufficient sleep diary data from DOZE users who did not complete the study (32/83, 39%), sleep diary indices are only presented for study finishers. In addition, analyses evaluating acceptability, credibility, and satisfaction with DOZE, and examining the efficacy of DOZE in improving sleep indices and self-reported measures of sleep and daytime functioning were only conducted among study finishers. Paired sample 2-tailed t tests were used to analyze changes in satisfaction with DOZE from baseline to end point, and acceptability was analyzed using descriptive statistics. Changes in sleep indices from period 1 to period 2 were analyzed using paired sample t tests. Similarly, paired sample t tests with 1000 bootstrapped samples were used to evaluate differences in self-reported insomnia severity, daytime symptoms, and health-related quality of life from baseline to end point to account for slight deviations from normality. Effect sizes were calculated for all analyses of change from baseline to end point using Cohen d for repeated measures. Unless otherwise specified, all analyses were performed using SPSS for Mac version 26 (IBM Corporation).


    User Characteristics

    In total, 154 participants consented to participate in this study (Figure 2). Of the 154 participants who consented to participate, 71 (46.1%) dropped out during the baseline questionnaire process. Over half of the participants (83/154, 53.9%) created an app account and completed some tracking. Approximately 33.1% (51/154) of participants completed all 4 components of the study (baseline questionnaires, first 2 weeks of sleep diaries, second 2 weeks of sleep diaries, and end point questionnaires).

    ‎
    Figure 2. Flowchart of participants. DOZE: Delivering Online Zzz’s with Empirical Support.
    View this figure

    Demographic characteristics were analyzed using data from all DOZE users (83/83, 100%; Table 1). Participants were on average 20.17 years (SD 2.49 years). Out of the 83 participants, 20 (24%) were adolescents (between the ages of 15 and 18 years) and 63 (76%) were young adults (between the ages of 19 and 24 years). The sample was predominantly female (57/83, 69%) and European Canadian (37/83, 45%). Most participants were not currently seeing a health care professional for the purpose of a sleep problem (79/83, 95%) or a mental disorder (69/83, 83%). Despite low levels of treatment-seeking, the average score on the ISI indicated sleep disturbance in the mild range (mean 12.48, SD 4.67), and 76% (63/83) of participants exceeded the clinical cutoff on the ISI (ie, score>10). In addition, the sample was characterized by high levels of fatigue, as indicated by the FSS (mean 4.73, SD 1.01) and the energy scale of the SF-36 (mean 35.71, SD 18.24) and daytime sleepiness (mean 45.14, SD 9.47) and on average had an intermediate chronotype on the CSM (mean 30.07, SD 7.16). Levels of state (mean 42.25, SD 12.73) and trait anxiety (mean 42.71, SD 13.15), as well as depression (mean 16.85, SD 8.30), were mild in this sample. Notably, early dropouts of the study (ie, before, during, and after the baseline questionnaires) did not differ from DOZE users on age (t114=0.40; P=.69), gender (χ22=3.0; P=.22), ethnicity (χ29=16.3; P=.06), ISI (t124=1.17; P=.25), FSS (t124=1.08; P=.28), CMQ (t118=0.63; P=.53), CASQ (t113=−0.60; P=.55), STICSA State (t111=0.74; P=.46), STICSA Trait (t104=0.41; P=.69), CESDR-10 (t115=−0.03; P=.98), or any of the SF-36 scales (P values range from .55 to .99). Similarly, study finishers did not differ from participants who dropped out at any point on any of these demographic or clinical characteristics (data not shown).

    Table 1. Demographic characteristics of participants at baseline (N=83).
    Baseline characteristicsParticipants, n (%)
    Sex

    		Female57 (69)

    		Male26 (31)
    Ethnicity

    		Caribbean Canadian1 (1)

    		East or Southeast Asian Canadian15 (18)

    		European Canadian37 (45)

    		Latin American, Central American, or South American Canadian4 (5)

    		South Asian Canadian8 (10)

    		West Asian or Arab Canadian1 (1)

    		Pacific Islander Canadian1 (1)

    		Other16 (19)
    Seeing an HCPa for sleep disorderb4 (5)
    Seeing an HCP for mental disorderb14 (17)

    aHCP: health care professional.

    bReflects the number and percentage of participants who answered "yes" when asked about whether they were currently seeing a health care professional for this issue.

    We analyzed sleep indices for study finishers at periods 1 and 2 (Table 2). After removing outliers (ranging from n=1 to n=6 depending on the sleep index), period 1 sleep indices indicated that despite an average SE in the normal range (ie, >85%), DOZE users spent a clinically meaningful amount of time awake at night. DOZE users also had variable sleep schedules, characterized by >4 hours of variability in their bedtime, sleep attempt, wake time, and rise time over a 2-week period. In addition, DOZE users obtained 7 hours 20 minutes of sleep on average despite spending 8 hours 21 minutes in bed; a one-sample t test indicated that this is significantly lower than 8.5, the minimum recommended amount of TST for AYAs (t50=−8.76; P<.001). In addition, DOZE users spent an average of nearly half an hour lingering in bed in the morning.

    Table 2. Change in sleep indices for DOZE (Delivering Online Zzz’s with Empirical Support) study finishersa.
    Sleep diary variablePeriod 1, mean (SD)Period 2, mean (SD)t testb (df)P valueCohen d
    Bedtime variability (hours)4.26 (2.16)4.96 (4.08)0.52 (49).600.07
    Sleep attempt variability (hours)4.04 (2.19)4.01 (2.13)0.88 (49).380.12
    Wake time variability (hours)4.59 (2.47)4.62 (2.38)−0.22 (49).830.03
    Rise time variability (hours)4.71 (2.41)5.05 (2.83)−0.13 (49).900.02
    Nap duration (hours)0.370 (0.50)0.220 (0.32)1.63 (32).110.28
    Morning lingering in bed (hours)0.402 (0.28)0.353 (0.30)2.24 (46).030.30
    Time in bed (hours)8.35 (1.06)8.49 (0.85)−2.40 (49).020.34
    Total sleep time (hours)7.34 (0.93)7.67 (0.97)−4.23 (49)<.0010.60
    Total wake time (hours)1.00 (0.54)0.837 (0.55)2.54 (47).010.36
    Sleep efficiency (%)87.67 (6.44)90.18 (6.46)−3.43 (49).0010.48
    Frequency of sleep-interfering substance usec0.142 (0.21)0.131 (0.21)0.34 (49)d.740.05

    aTest statistic and Cohen d are for log-transformed variables, whereas means and SDs refer to untransformed variables.

    bTest is 2-tailed.

    cFrequency of substance use refers to the proportion of days within a period in which sleep-interfering substance use was endorsed in the sleep diary.

    dNot log-transformed.

    Feasibility and Acceptability

    Feasibility

    Among all participants who created an app account, sleep diaries were completed for an average of 10.52 days (SD 4.87 days; range 0-14 days; 82/83, 99%) in period 1 and 9.81 days (SD 6.65 days; range 0-2 days; 75/83, 90%) in period 2. Over two-third of app users (57/83, 69%) used DOZE long enough to receive feedback on their sleep and to set goals; of these 57 DOZE users, 46 (81%) chose to set one or more goals to improve their sleep. Participants set an average of 1.54 goals (SD 1.12; range 0-4 goals; 57/83, 69%) following period 1.

    The most frequent goal was to reduce schedule variability (jetlag), followed by reducing naps and reducing lingering in bed in the morning (Table 3). The most frequent tips that participants accessed to help with their goals were how to wind down before bed and how to help with being a night owl (Table 4).

    Table 3. Frequency of goal use by participants (N=83).
    GoalParticipants, n (%)
    Reduce schedule variability (jet lag)34 (41)

    		Reduce to <1 hour11 (13)

    		Reduce to <2 hours18 (22)

    		Reduce to <3 hours5 (6)
    Limit time in bed11 (13)

    		Spend 8.5-10.5 hours on weekdays4 (5)

    		Spend 8.5-10.5 hours on both weekend days and weekdays7 (8)
    Manage sleepiness4 (5)
    Target lingering in beda16 (19)

    		Limit lingering to 30 minutes after alarm11 (13)

    		Set multiple alarms around room1 (1)

    		Ask others to help them get up0 (0)

    		Increase light exposure for first 30-60 minutes3 (4)

    		Get moving for the first 30 minutes5 (6)

    		Take a shower within first 30 minutes4 (5)

    		Remind self not to rely on initial feelings upon waking5 (6)
    Reduce naps17 (20)

    		No naps12 (14)

    		Decrease number of naps1 (1)

    		Decrease length of naps4 (5)

    aParticipants were free to select as many strategies as they desired from a drop-down list.

    Table 4. Frequency of tip use by participants (N=83).
    Tip accessParticipant, n (%)
    Completed night owl quiz17 (20)
    Completed difficulty winding down quiz24 (29)
    Completed difficulty getting up quiz13 (16)
    Completed sleep drunkenness quiz0 (0)
    Completed trouble staying awake quiz5 (6)
    Completed fatigue quiz13 (16)

    Google Analytics indicated that 2659 sessions (ie, instances of app use) took place. The average session duration was 3:02 minutes. During the study period, there were 27,263 page views. On average, 10.91 pages were viewed per session (including multiple visits to the same page). When grouped according to content categories, pages related to the sleep diary accounted for most of the 27,263 page views (18,018/27,263, 66.09%), indicating that it was the most used component of the app, followed by Progress (4098/27,263, 15.03%), Dashboard (1925/27,263, 7.06%), Tips (1061/27,263, 3.89%), Goals (810/27,263, 2.97%), Log-In (777/27,263, 2.85%), onboarding screens (431/27,263, 1.58%), and visits to their user profile (142/27,263, 0.52%). The average time on a page was 18.15 seconds across all pages, ranging from 65.51 seconds for progress pages to 6.68 seconds for onboarding pages. DOZE was most frequently accessed using a mobile device (2,705/2659 sessions, 78.04%), followed by a desktop computer (546/2,659 sessions, 20.53%) or a tablet (38/2,659 sessions, 1.43%). Consistent with the sleep diary and in-app recommendations to complete the sleep diary within 2 hours of waking up, DOZE was most frequently visited between 8 AM and 11 AM (Figure 3), with a modal time of 9 AM.

    ‎
    Figure 3. Users of Delivering Online Zzz’s with Empirical Support by time of day.
    View this figure
    Acceptability, Credibility, and Satisfaction

    To assess the acceptability, credibility, and satisfaction of DOZE, we analyzed the mean, mode, and range of responses for each item of the TEQ and the Acceptability Scale (Table 5). Responses to the TEQ were favorable at baseline and end point, with mean and modal values exceeding 4 for most items. There was no statistical change from baseline to end point on items assessing logic (P=.54), confidence that it would work (P=.48), confidence in recommending the app to a friend (P=.74), and how successful they predicted it would be in helping others with their sleep (P=.78), suggesting that most perceptions of DOZE as a credible and satisfactory intervention were stable over time. However, there was a statistically significant change in the item assessing willingness (P=.008). Responses on the Acceptability Scale indicated agreement or high agreement (mode=4 or 5) that DOZE was easy to use, easy to understand, required a reasonable amount of time, and that users were generally satisfied with the app, whereas responses were mostly neutral (mode=3) for enjoyment using the app and how helpful it was in describing their symptoms and quality of life. Averages for each item on the Acceptability Scale ranged from 3.61 (helpful in describing symptoms) to 4.35 (understandable).

    Table 5. Participants’ responses on the Treatment Evaluation Questionnaire and Acceptability Scale at baseline and during end point assessments (N=51).
    IndexBaselineEnd pointP value

    		Value, mean (SD; range)Value, modeValue, mean (SD; range)Value, mode

    		How logicala4.47 (1.72; 1-7)54.71 (1.63; 1-7)4.54

    		Confident in successb4.04 (1.71; 1-7)43.96 (1.48; 1-7)4.48

    		Recommend to friendc4.37 (1.58; 1-7)54.53 (1.55; 1-7)5.74

    		How willingd5.51 (1.73; 1-7)74.49 (1.59; 1-7)5.008

    		How successful for otherse4.46 (1.49; 1-7)44.53 (1.41; 1-7)5.78

    		Easy to useN/AfN/A4.16 (0.90; 2-5)5—g

    		UnderstandableN/AN/A4.35 (0.89; 2-5)5—

    		EnjoyableN/AN/A3.80 (0.96; 2-5)3—

    		Helpful in describing symptoms and quality of lifeN/AN/A3.61 (1.08; 2-5)3—

    		Amount of time acceptableN/AN/A4.33 (0.91; 2-5)5—

    		Overall satisfactionN/AN/A3.92 (0.82; 2-5)4—

    aHow logical does the treatment you are receiving seem to you?

    bHow confident are you that this treatment will be successful in eliminating your insomnia?

    cHow confident would you be in recommending this treatment to a friend?

    dHow willing are you (were you) to undergo this treatment?

    eHow successful do you think this treatment is for treating other people with insomnia?

    fN/A: not applicable; scale not administered at baseline.

    gSignificance testing not available.

    Preliminary Efficacy

    Sleep Indices

    To assess preliminary efficacy, we compared sleep indices at periods 1 and 2 among study finishers (Table 2). The sleep indices were positively skewed and were therefore log-transformed. They were then re-examined to confirm that the assumption of normality was met (all variables were approximately normal) and subsequently analyzed using paired samples t tests, with the exclusion of frequency of sleep-interfering substance use, which was not transformed because of the presence of 0 values. Significant improvements of small-to-medium effect sizes were seen for log-transformed morning lingering (P=.03), TIB (P=.02), TST (P<.001), SE (P=.001), and TWT (P=.01), whereas variability in sleep schedule (ie, bedtime, sleep attempt, wake time, rise time), naps, and frequency of sleep-interfering substance use did not significantly change (Table 2).

    Self-report Measures

    Finally, we compared self-report measures at baseline and end point among study finishers (Table 6). To account for slight deviations from normality among some of the variables, paired t tests were conducted using bootstrapping. Mean differences (MDs) were resampled 1000 times, and 95% CIs were generated. Statistically significant improvements were seen on the ISI (MD 3.92, SE 0.61; 95% CI 2.82-5.17), CSM (MD −2.15, SE 0.64; 95% CI −3.41 to −0.96), STICSA State (MD 2.36, SE 1.13; 95% CI 0.16-4.64), STICSA Trait (MD 3.97, SE 0.97; 95% CI 2.03-5.87), CESDR-10 (MD 3.10, SE 0.99; 95% CI 1.20-5.18), and SF-36 energy (MD −6.54, SE 2.48; 95% CI −11.28 to −1.79). Changes from baseline to end point were not significant for the remaining measures of fatigue, sleepiness, or health-related quality of life.

    Table 6. Comparison of self-reported measures from baseline to end point for study finishers.
    Self-report measureBaselineEnd pointP valueCohen d

    		Score, mean (SD)Cronbach αScore, mean (SD)Cronbach α

    ISIa11.65 (4.35).778.24 (3.95).74.0010.90
    FSSb4.55 (1.04).854.35 (1.04).86.070.20
    CSMc30.80 (7.37).8632.47 (7.79).88.0060.43
    CASQd total44.14 (9.49).8343.14 (11.09).89.500.15

    		Sleepiness27.42 (7.34).8126.54 (8.49).88.630.14

    		Alertness16.74 (3.80).7716.34 (4.23).81.860.12
    CESDR-10e15.62 (8.34).8912.98 (7.89).90.0040.44
    STICSAf

    		State39.69 (11.39).9037.83 (13.32).94.050.19

    		Trait40.80 (11.60).9138.83 (14.25).94.0020.27
    SF-36g

    		Physical functioning84.22 (22.96).9388.43 (20.70).95.090.28

    		Role functioning-physical65.20 (39.71).8570.59 (39.28).89.270.14

    		Role functioning-emotional43.33 (44.29).8842.16 (43.12).85.850.03

    		Energy38.30 (18.16).7743.92 ()19.17.75.010.38

    		Emotional well-being54.35 (21.27).8356.63 (23.21).90.390.13

    		Social functioning64.22 (27.62).8167.40 (27.17).81.290.14

    		Pain77.45 (15.33).6476.23 (18.48).77.730.08

    		General health62.27 (23.58).8463.95 (22.89).77.080.10

    aISI: Insomnia Severity Index.

    bFSS: Fatigue Severity Scale.

    cCSM: Composite Scale of Morningness.

    dCASQ: Cleveland Adolescent Sleepiness Questionnaire.

    eCESDR-10: Center for Epidemiologic Studies Depression Scale–Revised 10-item Version for Adolescents.

    fSTICSA: State-Trait Inventory of Cognitive and Somatic Anxiety.

    gSF-36: RAND 36-item Short Form Health Survey 1.0.


    Principal Findings

    We tested the feasibility, acceptability, and credibility of a self-management behavioral sleep web-based app designed by and for AYAs (aged 15-24 years). The results suggest that AYAs rated the intervention as logical, had confidence that the self-management approach would work, and had confidence in recommending DOZE to a friend. Willingness significantly decreased from baseline to end point, perhaps reflecting different stages of readiness to implement the strategies provided, environmental limitations that can influence their ability to implement strategies (eg, early school rise times and high academic demands), and intrinsic values that are a barrier to engaging in the intervention (eg, valuing sleeping in on weekends over maintaining a regular schedule). However, it is important to note that the TEQ was completed before participants received any detailed information about the intervention (they were only aware that it was a new sleep app for teens and young adults), representing only naïve expectations of DOZE, whereas at end point they were familiar with the intervention components. DOZE was rated as highly acceptable across indices of ease of use, understandability, time commitment, and satisfaction. Perhaps not surprisingly, ratings about DOZE being enjoyable or succinct in describing their particular symptoms or quality of life issues were acceptable but slightly more modest than ease of use, low time commitment, and overall satisfaction. In support of feasibility, participants engaged with DOZE often (indicated by the number of sessions), most frequently visiting pages pertaining to the sleep diary or their progress and did so using various electronic devices (ie, mobile phones, computers, and tablets). Moreover, although this was not a clinical sample, prospective sleep diary insomnia indices significantly improved from period 1 to period 2 (eg, time spent awake and SE), and the amount of sleep they obtained increased too. AYA users reported significant improvements from baseline to end point in insomnia symptom severity, chronotype, depression, anxiety, and energy. Despite not recruiting a clinical sample, participants in this study overwhelmingly were not treatment-seekers; however, they evidenced very poor health-related quality of life indices and were above clinical cutoffs for insomnia and fatigue, consistent with what we see in the literature [71].

    These results are supportive of self-management. AYAs logged on frequently and completed the diary at the instructed time (ie, within 2 hours of waking up in the morning). Moreover, they worked hard at (1) tracking their sleep (ie, tracking for approximately 10 days in each 14-day period) and (2) setting goals that treatment providers would set (eg, limiting schedule variability and naps and decreasing lingering in bed). They accessed tips to help with their goals, particularly how to wind down before bed, how to help with being a night owl, how to get up regularly in the morning, how to help with fatigue, and how to stay out of bed early in the evening. Thus, we can trust AYAs to set their own goals and make behavioral changes to achieve these goals when they are provided with the tools and education to do so autonomously. Trusting patients to make good choices may not come naturally for providers, but the literature suggests that many patients want autonomy in their health care choices [72] and produce good results when given tools that support their autonomy. As a research group, this was an instructive part of the process, and it was driven by both the literature and collaboration with our AYA advisors and stakeholders throughout the co-design process. Perhaps these findings can inform the tailoring of in-person strategies for AYAs to achieve better outcomes.

    Statistically significant differences in sleep indices from period 1 to period 2 seemed unlikely, as we recruited a sample that was not selected for clinical levels of sleep disturbance, and we assumed there would be too little variability in these indices to detect significant effects. Furthermore, detecting a change in variables, such as SOL, naps, or TIB, in a nonclinical or subclinical sample in which some have opposing problems (eg, hypersomnia vs insomnia) is also surprising. For example, those with hypersomnia or sleepiness have SOL values that are too low, and those with insomnia have elevated SOL values; use of DOZE could result in changes in opposite directions and, when averaged across participants, appear to have affected no change at all. These problems can also occur in the same AYA during the same week, for example, sleepiness on some nights and insomnia on others. Similarly, some AYAs are told to eliminate naps, and others are told to increase them, resulting in low variability. Therefore, it is encouraging that insomnia indices, such as lingering in bed in the morning, TWT, TST, TIB, and SE, improved. Although AYAs set goals to decrease the variability in their sleep schedules, the improvement did not reach statistical significance. Perhaps this is why they accessed tips about how to get up in the morning (ie, as it is difficult for them to rise at a regular time), and our plan is to expand tips and resources for AYA users on the DOZE website.

    An important implication of this research is that this web-based app may be an important tool to increase access to evidence-based interventions. Some AYAs prefer to use technology for health-related behavior change [34,73,74], and smartphone apps appear to have a high uptake potential; 78.04% used a mobile device for DOZE (2,705/2659 sessions), whereas only 20% used a desktop computer (546/2,659 sessions, 20.5%) and 1.4% used a tablet (38/2,659 sessions). Incorporating AYAs in the co-design process is also a notable strength of this web-based app and may promote continued uptake. That said, future research will need to determine whether there would be health disparities based on access to technology, Wi-Fi, and geographical location. Clinicians report favorable attitudes toward CBT-I apps [75], suggesting the possibility of widespread access to evidence-based sleep treatments. Given the limited number of expert providers in this unique group (ie, not pediatric but not fully adult either) and the need for sleep solutions at the level of primary care and community mental health clinics, this web-based app holds the promise of increasing access to tailored, age-specific care. We intend to conduct a health economic study on the impact of DOZE in the future.

    Limitations

    There was a 39% (32/83 participants) dropout rate in the self-management intervention portion of the study, and data were not collected on reasons for discontinuation, although this would certainly be valuable. Although we would prefer a lower rate, the rate was lower than that in other sleep app studies in the literature. For example, digital interventions for insomnia have reported rates >50% [76]. Across adolescent mobile health apps more generally, in a study using an internet-delivered, self-help CBT intervention for chronic pain among adolescents (aged 12-17 years), 52% of participants dropped out of the treatment portion of the study [77]. The dropout rate in our study is perhaps not surprising as they were not treatment-seekers, so if their sleep was not a problem, they might have little motivation except for the CAD $50 (US $39.17) compensation for participation. The brief intervention period in this study may also have contributed to lower rates of dropout compared with other studies. In addition, participants in this study were emailed by the research coordinator if they did not complete sleep diaries for several consecutive days; as such, adherence may be lower when disseminated in the real world. A future trial could evaluate whether dropout rates are lower in a clinical sample. Moreover, much of the dropout in this study occurred during the baseline questionnaire period, suggesting that the self-report battery was too burdensome. These questionnaires are not part of the app, so this barrier to participation is not present with regular, real-world app use, and future studies can verify if the app dropout rate is actually lower in real-world app use.

    Although appropriate for feasibility trials, the use of an open trial suggests caution when interpreting positive outcomes. Without a control group, we could not ascertain whether the results were specific to the app or the increased sleep attention. In addition, we recruited participants using various strategies (eg, through a university, the community, and mental health providers, which are the intended disseminators of DOZE). Although this could be seen as a strength, given that our sample likely reflects those who will use DOZE in the real world, it may also be a limitation as these could be considered 3 distinct samples. Without knowing more about these samples, we could not test for possible differences. In the same vein, although our intended users are all AYAs, we are cautious about stating that this would be effective in patients, as this was a community sample; although most participants experienced symptoms that were above clinical cutoffs, they were neither clinically assessed nor were they treatment-seekers. We intend to study the properties of the app in those who are treatment-seeking. Our treatment provider stakeholders suggested that they would be most interested in using the diary as an adjunct to their treatment, much like sleep diary apps in adults [78]. DOZE could be used as a tracking and posttherapy relapse prevention tool, with providers prescribing behavioral change rather than the AYA deciding their own treatment course. Given the strong preferences for self-management in this group, it would be interesting to test outcomes with providers versus self-management.

    Our findings may also have been influenced by extraneous factors. First, several items in our measure assessing credibility reference specifically to treatment for insomnia and may not have applied equally to all participants, thereby influencing their responses. Moreover, a small percentage of participants were seeing a health care provider for the purpose of a mental disorder or a sleep disorder, which may have also influenced our results given the bidirectional relationship between sleep and psychopathology. Finally, we did not query medication use and could not rule out the possibility that some participants were using sleep medication, which may have influenced our results; although low levels of treatment-seeking for a sleep disorder suggests that this is unlikely, and participants enrolled in the trial as they were dissatisfied with their sleep despite current treatments, it nevertheless remains a possibility among a small minority of our participants.

    Conclusions

    In conclusion, given that there were statistically and clinically significant improvements in a nonclinical sample, it is promising for the tool to be used as designed: as a transdiagnostic tool for young adults to engage in sleep self-management digitally. Even those without sleep disorders were interested and effective in making health-related changes, which can translate into improved sleep habits and potential prevention, as well as potentially improving mental wellness in AYAs. Web-based interventions are an acceptable and effective method of facilitating access to evidence-based care among invisible groups, with the ultimate effect of reducing the public health burden.

    Acknowledgments

    The authors would like to acknowledge the contributions of our adolescent and young adult stakeholders who provided input into the design of DOZE (Delivering Online Zzz’s with Empirical Support) and its evaluation, including Sarah Lagler-Clarke, who is currently helping design an uptake strategy. The authors also thank their stakeholders in community mental health, public health, and pediatric sleep medicine, namely Gloria Chaim, MSW; Joanne Figliano-Scott, RN; Janine Robb, PhD; and Shelly Weiss, MD. Finally, the authors acknowledge the in-kind support from the Office of the Vice President of Research and Innovation and the Dean of Arts at Ryerson University; their industry partners at PIVOT Design, including Ian Chalmers, Brenda Little, Dave Brennan, and Melvyn Loa; as well as the Canadian Institutes of Health Research for developing this mechanism for youth mental health initiatives and for funding this project under the eHealth Innovation Partnership Program (#143551).

    Conflicts of Interest

    CEC discloses the current grant funding from the Canadian government (Canadian Institutes of Health Research). CEC is the creator of DOZE (Delivering Online Zzz’s with Empirical Support) and the principal investigator of this study; however, CEC does not receive any royalties for DOZE. The remaining coauthors have nothing to declare.

    ‎
    Multimedia Appendix 1

    Delivering Online Zzz’s with Empirical Support co-design process.

    DOC File , 28 KB
    1. Gradisar M, Gardner G, Dohnt H. Recent worldwide sleep patterns and problems during adolescence: a review and meta-analysis of age, region, and sleep. Sleep Med 2011 Feb;12(2):110-118. [CrossRef] [Medline]
    2. Owens J, Adolescent Sleep Working Group, Committee on Adolescence. Insufficient sleep in adolescents and young adults: an update on causes and consequences. Pediatrics 2014 Sep;134(3):921-932 [FREE Full text] [CrossRef] [Medline]
    3. Hicks RA, Fernandez C, Pellegrini RJ. Striking changes in the sleep satisfaction of university students over the last two decades. Percept Mot Skills 2001 Dec;93(3):660. [CrossRef] [Medline]
    4. Hysing M, Pallesen S, Stormark KM, Lundervold AJ, Sivertsen B. Sleep patterns and insomnia among adolescents: a population-based study. J Sleep Res 2013 Oct;22(5):549-556 [FREE Full text] [CrossRef] [Medline]
    5. Sivertsen B, Vedaa Ø, Harvey AG, Glozier N, Pallesen S, Aarø LE, et al. Sleep patterns and insomnia in young adults: a national survey of norwegian university students. J Sleep Res 2019 Apr;28(2):e12790 [FREE Full text] [CrossRef] [Medline]
    6. Alvaro PK, Roberts RM, Harris JK. A systematic review assessing bidirectionality between sleep disturbances, anxiety, and depression. Sleep 2013 Jul 01;36(7):1059-1068 [FREE Full text] [CrossRef] [Medline]
    7. Chiu H, Lee H, Chen P, Lai Y, Tu Y. Associations between sleep duration and suicidality in adolescents: a systematic review and dose-response meta-analysis. Sleep Med Rev 2018 Dec;42:119-126. [CrossRef] [Medline]
    8. Hysing M, Harvey AG, Linton SJ, Askeland KG, Sivertsen B. Sleep and academic performance in later adolescence: results from a large population-based study. J Sleep Res 2016 Jun;25(3):318-324 [FREE Full text] [CrossRef] [Medline]
    9. Wong MM, Brower KJ. The prospective relationship between sleep problems and suicidal behavior in the National Longitudinal Study of Adolescent Health. J Psychiatr Res 2012 Jul;46(7):953-959 [FREE Full text] [CrossRef] [Medline]
    10. Yang C, Wu C, Hsieh M, Liu M, Lu F. Coping with sleep disturbances among young adults: a survey of first-year college students in Taiwan. Behav Med 2003;29(3):133-138. [CrossRef] [Medline]
    11. Zhang J, Paksarian D, Lamers F, Hickie IB, He J, Merikangas KR. Sleep patterns and mental health correlates in US adolescents. J Pediatr 2017 Mar;182:137-143. [CrossRef] [Medline]
    12. Carskadon MA. Sleep in adolescents: the perfect storm. Pediatr Clin North Am 2011 Jun;58(3):637-647 [FREE Full text] [CrossRef] [Medline]
    13. Crowley SJ, Acebo C, Carskadon MA. Sleep, circadian rhythms, and delayed phase in adolescence. Sleep Med 2007 Sep;8(6):602-612. [CrossRef] [Medline]
    14. Gradisar M, Wolfson AR, Harvey AG, Hale L, Rosenberg R, Czeisler CA. The sleep and technology use of Americans: findings from the National Sleep Foundation's 2011 sleep in America poll. J Clin Sleep Med 2013 Dec 15;9(12):1291-1299 [FREE Full text] [CrossRef] [Medline]
    15. Randler C, Bilger S. Associations among sleep, chronotype, parental monitoring, and pubertal development among German adolescents. J Psychol 2009 Oct;143(5):509-520. [CrossRef] [Medline]
    16. Aoki H, Ozeki Y, Yamada N. Hypersensitivity of melatonin suppression in response to light in patients with delayed sleep phase syndrome. Chronobiol Int 2001 Mar;18(2):263-271. [CrossRef] [Medline]
    17. Carskadon MA, Vieira C, Acebo C. Association between puberty and delayed phase preference. Sleep 1993 Apr;16(3):258-262. [CrossRef] [Medline]
    18. James JE, Kristjánsson AL, Sigfúsdóttir ID. Adolescent substance use, sleep, and academic achievement: evidence of harm due to caffeine. J Adolesc 2011 Aug;34(4):665-673. [CrossRef] [Medline]
    19. Gibson ES, Powles AC, Thabane L, O'Brien S, Molnar DS, Trajanovic N, et al. "Sleepiness" is serious in adolescence: two surveys of 3235 Canadian students. BMC Public Health 2006 May 02;6:116 [FREE Full text] [CrossRef] [Medline]
    20. Pagel JF, Forister N, Kwiatkowki C. Adolescent sleep disturbance and school performance: the confounding variable of socioeconomics. J Clin Sleep Med 2007 Feb 15;3(1):19-23. [Medline]
    21. Qaseem A, Kansagara D, Forciea MA, Cooke M, Denberg TD, Clinical Guidelines Committee of the American College of Physicians. Management of chronic insomnia disorder in adults: a clinical practice guideline from the American College of Physicians. Ann Intern Med 2016 Jul 19;165(2):125-133 [FREE Full text] [CrossRef] [Medline]
    22. Insomnia. National Institute for HealthCare Excellence (NICE). 2015.   URL: https://cks.nice.org.uk/insomnia#!management [accessed 2020-10-20]
    23. Owens JA, Rosen CL, Mindell JA. Medication use in the treatment of pediatric insomnia: results of a survey of community-based pediatricians. Pediatrics 2003 May;111(5 Pt 1):628-635. [CrossRef] [Medline]
    24. Mindell JA, Kuhn B, Lewin DS, Meltzer LJ, Sadeh A, American Academy of Sleep Medicine. Behavioral treatment of bedtime problems and night wakings in infants and young children. Sleep 2006 Oct;29(10):1263-1276. [CrossRef] [Medline]
    25. Stojanovski SD, Rasu RS, Balkrishnan R, Nahata MC. Trends in medication prescribing for pediatric sleep difficulties in US outpatient settings. Sleep 2007 Aug;30(8):1013-1017 [FREE Full text] [CrossRef] [Medline]
    26. Kloss JD, Nash CO, Horsey SE, Taylor DJ. The delivery of behavioral sleep medicine to college students. J Adolesc Health 2011 Jun;48(6):553-561. [CrossRef] [Medline]
    27. Harvey AG, Hein K, Dolsen MR, Dong L, Rabe-Hesketh S, Gumport NB, et al. Modifying the impact of eveningness chronotype ("Night-owls") in youth: a randomized controlled trial. J Am Acad Child Adolesc Psychiatry 2018 Oct;57(10):742-754. [CrossRef] [Medline]
    28. Gumport NB, Dolsen MR, Harvey AG. Usefulness and utilization of treatment elements from the Transdiagnostic Sleep and Circadian Intervention for adolescents with an evening circadian preference. Behav Res Ther 2019 Dec;123:103504. [CrossRef] [Medline]
    29. Dong L, Gumport NB, Martinez AJ, Harvey AG. Is improving sleep and circadian problems in adolescence a pathway to improved health? A mediation analysis. J Consult Clin Psychol 2019 Sep;87(9):757-771 [FREE Full text] [CrossRef] [Medline]
    30. Koffel E, Amundson E, Polusny G, Wisdom JP. "You're missing out on something great": patient and provider perspectives on increasing the use of cognitive behavioral therapy for insomnia. Behav Sleep Med 2020;18(3):358-371. [CrossRef] [Medline]
    31. Boerner KE, Coulombe JA, Corkum P. Barriers and facilitators of evidence-based practice in pediatric behavioral sleep care: qualitative analysis of the perspectives of health professionals. Behav Sleep Med 2015;13(1):36-51. [CrossRef] [Medline]
    32. Harvey AG, Buysse DJ. Treating Sleep Problems: A Transdiagnostic Approach. 1st Edition. New York, NY: The Guilford Press; 2017.
    33. Crookston BT, West JH, Hall PC, Dahle KM, Heaton TL, Beck RN, et al. Mental and emotional self-help technology apps: cross-sectional study of theory, technology, and mental health behaviors. JMIR Ment Health 2017 Oct 17;4(4):e45 [FREE Full text] [CrossRef] [Medline]
    34. Radovic A, McCarty CA, Katzman K, Richardson LP. Adolescents' perspectives on using technology for health: qualitative study. JMIR Pediatr Parent 2018;1(1):e2 [FREE Full text] [CrossRef] [Medline]
    35. Wetterlin FM, Mar MY, Neilson EK, Werker GR, Krausz M. eMental health experiences and expectations: a survey of youths' web-based resource preferences in Canada. J Med Internet Res 2014 Dec 17;16(12):e293 [FREE Full text] [CrossRef] [Medline]
    36. Mindell JA, Du Mond CE, Sadeh A, Telofski LS, Kulkarni N, Gunn E. Efficacy of an internet-based intervention for infant and toddler sleep disturbances. Sleep 2011 Apr 01;34(4):451-458 [FREE Full text] [CrossRef] [Medline]
    37. Corkum PV, Reid GJ, Hall WA, Godbout R, Stremler R, Weiss SK, et al. Evaluation of an internet-based behavioral intervention to improve psychosocial health outcomes in children with insomnia (Better Nights, Better Days): protocol for a randomized controlled trial. JMIR Res Protoc 2018 Mar 26;7(3):e76 [FREE Full text] [CrossRef] [Medline]
    38. Speth TA, Coulombe JA, Markovich AN, Chambers CT, Godbout R, Gruber R, et al. Barriers, facilitators, and usability of an internet intervention for children aged 1 to 10 years with insomnia. Transl Issues Psychol Sci 2015;1(1):16-31. [CrossRef]
    39. Werner-Seidler A, Wong Q, Johnston L, O'Dea B, Torok M, Christensen H. Pilot evaluation of the Sleep Ninja: a smartphone application for adolescent insomnia symptoms. BMJ Open 2019 May 27;9(5):e026502 [FREE Full text] [CrossRef] [Medline]
    40. Werner-Seidler A, O'Dea B, Shand F, Johnston L, Frayne A, Fogarty AS, et al. A smartphone app for adolescents with sleep disturbance: development of the Sleep Ninja. JMIR Ment Health 2017 Jul 28;4(3):e28 [FREE Full text] [CrossRef] [Medline]
    41. de Bruin EJ, Bögels SM, Oort FJ, Meijer AM. Efficacy of cognitive behavioral therapy for insomnia in adolescents: a randomized controlled trial with internet therapy, group therapy and a waiting list condition. Sleep 2015 Dec 01;38(12):1913-1926 [FREE Full text] [CrossRef] [Medline]
    42. Espie CA, Kyle SD, Williams C, Ong JC, Douglas NJ, Hames P, et al. A randomized, placebo-controlled trial of online cognitive behavioral therapy for chronic insomnia disorder delivered via an automated media-rich web application. Sleep 2012 Jun 01;35(6):769-781 [FREE Full text] [CrossRef] [Medline]
    43. DOZE - delivering online z’s. DOZE App. 2019.   URL: https://dozeapp.ca/ [accessed 2020-10-20]
    44. Hackett CL, Mulvale G, Miatello A. Co-designing for quality: creating a user-driven tool to improve quality in youth mental health services. Health Expect 2018 Dec;21(6):1013-1023 [FREE Full text] [CrossRef] [Medline]
    45. Harrington C, Wilcox L, Connelly K, Rogers W, Sanford J. Designing health and fitness apps with older adults: examining the value of experience-based co-design. In: Proceedings of the 12th EAI International Conference on Pervasive Computing Technologies for Healthcare.: Association for Computing Machinery; 2018 Presented at: PervasiveHealth '18: 12th EAI International Conference on Pervasive Computing Technologies for Healthcare; May 21 - 24, 2018; New York, USA p. 15-24. [CrossRef]
    46. Thabrew H, Fleming T, Hetrick S, Merry S. Co-design of eHealth interventions With children and young people. Front Psychiatry 2018;9:481 [FREE Full text] [CrossRef] [Medline]
    47. Shah LM, Yang WE, Demo RC, Lee MA, Weng D, Shan R, et al. Technical guidance for clinicians interested in partnering with engineers in mobile health development and evaluation. JMIR Mhealth Uhealth 2019 May 15;7(5):e14124 [FREE Full text] [CrossRef] [Medline]
    48. Carney CE, Buysse DJ, Ancoli-Israel S, Edinger JD, Krystal AD, Lichstein KL, et al. The consensus sleep diary: standardizing prospective sleep self-monitoring. Sleep 2012 Feb 01;35(2):287-302 [FREE Full text] [CrossRef] [Medline]
    49. Borkovec TD, Nau SD. Credibility of analogue therapy rationales. J Behav Ther Exp Psychiatry 1972 Dec;3(4):257-260. [CrossRef]
    50. Carney CE, Edinger JD, Kuchibhatla M, Lachowski AM, Bogouslavsky O, Krystal AD, et al. Cognitive behavioral insomnia therapy for those with insomnia and depression: a randomized controlled clinical trial. Sleep 2017 Apr 01;40(4):zsx019 [FREE Full text] [CrossRef] [Medline]
    51. Morin C. Insomnia: Psychological Assessment and Management. New York, NY: Guilford Press; 1993.
    52. Morin CM, Belleville G, Bélanger L, Ivers H. The Insomnia Severity Index: psychometric indicators to detect insomnia cases and evaluate treatment response. Sleep 2011 May 01;34(5):601-608. [CrossRef] [Medline]
    53. Chung KF, Kan KK, Yeung WF. Assessing insomnia in adolescents: comparison of Insomnia Severity Index, Athens Insomnia Scale and Sleep Quality Index. Sleep Med 2011 May;12(5):463-470. [CrossRef] [Medline]
    54. Chahoud M, Chahine R, Salameh P, Sauleau EA. Reliability, factor analysis and internal consistency calculation of the Insomnia Severity Index (ISI) in French and in English among Lebanese adolescents. eNeurologicalSci 2017 Mar 18;7:9-14 [FREE Full text] [CrossRef] [Medline]
    55. Gerber M, Lang C, Lemola S, Colledge F, Kalak N, Holsboer-Trachsler E, et al. Validation of the German version of the Insomnia Severity Index in adolescents, young adults and adult workers: results from three cross-sectional studies. BMC Psychiatry 2016 May 31;16:174 [FREE Full text] [CrossRef] [Medline]
    56. Bastien CH, Vallières A, Morin CM. Validation of the Insomnia Severity Index as an outcome measure for insomnia research. Sleep Med 2001 Jul;2(4):297-307. [CrossRef] [Medline]
    57. Smith CS, Reilly C, Midkiff K. Evaluation of three circadian rhythm questionnaires with suggestions for an improved measure of morningness. J Appl Psychol 1989 Oct;74(5):728-738. [CrossRef] [Medline]
    58. Smith CS, Folkard S, Schmieder RA, Parra LF, Spelten E, Almiral H, et al. Investigation of morning–evening orientation in six countries using the preferences scale. Pers Individ Dif 2002 Apr;32(6):949-968. [CrossRef]
    59. Randler C. Morningness-eveningness comparison in adolescents from different countries around the world. Chronobiol Int 2008 Nov;25(6):1017-1028. [CrossRef] [Medline]
    60. Krupp LB, LaRocca NG, Muir-Nash J, Steinberg AD. The Fatigue Severity Scale. Application to patients with multiple sclerosis and systemic lupus erythematosus. Arch Neurol 1989 Oct;46(10):1121-1123. [CrossRef] [Medline]
    61. Huang Y, Katz BZ, Mears C, Kielhofner GW, Taylor R. Postinfectious fatigue in adolescents and physical activity. Arch Pediatr Adolesc Med 2010 Sep;164(9):803-809 [FREE Full text] [CrossRef] [Medline]
    62. Gordon BA, Knapman LM, Lubitz L. Graduated exercise training and progressive resistance training in adolescents with chronic fatigue syndrome: a randomized controlled pilot study. Clin Rehabil 2010 Dec;24(12):1072-1079. [CrossRef] [Medline]
    63. Spilsbury JC, Drotar D, Rosen CL, Redline S. The Cleveland Adolescent Sleepiness Questionnaire: a new measure to assess excessive daytime sleepiness in adolescents. J Clin Sleep Med 2007 Oct 15;3(6):603-612 [FREE Full text] [CrossRef] [Medline]
    64. Haroz EE, Ybarra ML, Eaton WW. Psychometric evaluation of a self-report scale to measure adolescent depression: the CESDR-10 in two national adolescent samples in the United States. J Affect Disord 2014 Apr;158:154-160 [FREE Full text] [CrossRef] [Medline]
    65. Ree M, MacLeod C, French D, Locke V. The State–Trait Inventory for Cognitive and Somatic Anxiety: development and validation. State–Trait Inventory for Cognitive and Somatic Anxiety (STICSA) 2000:A. [CrossRef]
    66. Deacy AD, Gayes LA, De Lurgio S, Wallace DP. Adaptation of the State-Trait Inventory for Cognitive and Somatic Anxiety for use in children: a preliminary analysis. J Pediatr Psychol 2016 Oct;41(9):1033-1043. [CrossRef] [Medline]
    67. Grös DF, Antony MM, Simms LJ, McCabe RE. Psychometric properties of the State-Trait Inventory for Cognitive and Somatic Anxiety (STICSA): comparison to the State-Trait Anxiety Inventory (STAI). Psychol Assess 2007 Dec;19(4):369-381. [CrossRef] [Medline]
    68. Roberts K. State-Trait Inventory for Cognitive and Somatic Anxiety: psychometric properties and experimental manipulation to evaluate sensitivity to change and predictive validity. Thesis and Dissertations - York University, Toronto. 2013.   URL: https:/​/yorkspace.​library.yorku.ca/​xmlui/​bitstream/​handle/​10315/​31982/​Roberts_Karen_E_2013_PhD.​pdf?isAllowed=y&sequence=1 [accessed 2020-10-20]
    69. Ware JE, Sherbourne CD. The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Med Care 1992 Jun;30(6):473-483. [CrossRef] [Medline]
    70. McHorney CA, Ware JE, Lu JF, Sherbourne CD. The MOS 36-item short-form health survey (SF-36): III. Tests of data quality, scaling assumptions, and reliability across diverse patient groups. Med Care 1994 Jan;32(1):40-66. [CrossRef] [Medline]
    71. Taylor DJ, Bramoweth AD, Grieser EA, Tatum JI, Roane BM. Epidemiology of insomnia in college students: relationship with mental health, quality of life, and substance use difficulties. Behav Ther 2013 Sep;44(3):339-348. [CrossRef] [Medline]
    72. Benbassat J, Pilpel D, Tidhar M. Patients' preferences for participation in clinical decision making: a review of published surveys. Behav Med 1998;24(2):81-88. [CrossRef] [Medline]
    73. Dennison L, Morrison L, Conway G, Yardley L. Opportunities and challenges for smartphone applications in supporting health behavior change: qualitative study. J Med Internet Res 2013 Apr 18;15(4):e86 [FREE Full text] [CrossRef] [Medline]
    74. Gowin M, Cheney M, Gwin S, Franklin Wann T. Health and fitness app use in college students: a qualitative study. Am J Health Educ 2015 Jul 06;46(4):223-230. [CrossRef]
    75. Miller KE, Kuhn E, Owen JE, Taylor K, Yu JS, Weiss BJ, et al. Clinician perceptions related to the use of the CBT-I Coach mobile app. Behav Sleep Med 2019;17(4):481-491. [CrossRef] [Medline]
    76. Cliffe B, Croker A, Denne M, Smith J, Stallard P. Digital cognitive behavioral therapy for insomnia for adolescents with mental health problems: feasibility open trial. JMIR Ment Health 2020 Mar 03;7(3):e14842 [FREE Full text] [CrossRef] [Medline]
    77. Voerman JS, Remerie S, Westendorp T, Timman R, Busschbach JJ, Passchier J, et al. Effects of a guided internet-delivered self-help intervention for adolescents with chronic pain. J Pain 2015 Nov;16(11):1115-1126. [CrossRef] [Medline]
    78. Koffel E, Kuhn E, Petsoulis N, Erbes CR, Anders S, Hoffman JE, et al. A randomized controlled pilot study of CBT-I Coach: feasibility, acceptability, and potential impact of a mobile phone application for patients in cognitive behavioral therapy for insomnia. Health Informatics J 2018 Mar;24(1):3-13. [CrossRef] [Medline]

    ‎
    AYA: adolescent and young adult
    CASQ: Cleveland Adolescent Sleepiness Questionnaire
    CBT-I: cognitive behavioral therapy for insomnia
    CESDR-10: Center for Epidemiologic Studies Depression Scale–Revised 10-item Version for Adolescents
    CSM: Composite Scale of Morningness
    DOZE: Delivering Online Zzz’s with Empirical Support
    FSS: Fatigue Severity Scale
    ISI: Insomnia Severity Index
    MD: mean difference
    SE: sleep efficiency
    SF-36: RAND 36-item Short Form Health Survey 1.0
    SOL: sleep onset latency
    STICSA: State-Trait Inventory of Cognitive and Somatic Anxiety
    TEQ: Treatment Evaluation Questionnaire
    TIB: time in bed
    TST: total sleep time
    TWT: total wake time

    Edited by G Eysenbach; submitted 30.10.20; peer-reviewed by J Yu, D Dunsmuir, B Cliffe; comments to author 18.11.20; revised version received 06.01.21; accepted 13.09.21; published 01.11.21

    Copyright

    ©Nicole E Carmona, Aleksandra Usyatynsky, Samlau Kutana, Penny Corkum, Joanna Henderson, Kelly McShane, Colin Shapiro, Souraya Sidani, Jennifer Stinson, Colleen E Carney. Originally published in JMIR Formative Research (https://formative.jmir.org), 01.11.2021.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.