Published on in Vol 7 (2023)

Preprints (earlier versions) of this paper are available at, first published .
The Dosing of Mobile-Based Just-in-Time Adaptive Self-Management Prompts for Caregivers: Preliminary Findings From a Pilot Microrandomized Study

The Dosing of Mobile-Based Just-in-Time Adaptive Self-Management Prompts for Caregivers: Preliminary Findings From a Pilot Microrandomized Study

The Dosing of Mobile-Based Just-in-Time Adaptive Self-Management Prompts for Caregivers: Preliminary Findings From a Pilot Microrandomized Study

Original Paper

1Department of Biostatistics, School of Public Health, University of Michigan, Ann Arbor, MI, United States

2Michigan Institute for Data Science, University of Michigan, Ann Arbor, MI, United States

3Department of Pediatrics, University of Michigan, Ann Arbor, MI, United States

4Department of Psychiatry, University of Michigan, Ann Arbor, MI, United States

5School of Information, University of Michigan, Ann Arbor, MI, United States

6Department of Physical Medicine and Rehabilitation, University of Michigan, Ann Arbor, MI, United States

7H Ben Taub Department of Physical Medicine and Rehabilitation, Baylor College of Medicine/Harris Health System, Houston, TX, United States

8Clinical Trials Support Office, University of Michigan, Ann Arbor, MI, United States

Corresponding Author:

Noelle E Carlozzi, PhD

Department of Physical Medicine and Rehabilitation

University of Michigan

2800 Plymouth Road

Ann Arbor, MI, 48103

United States

Phone: 1 7347638917


Background: Caregivers of people with chronic illnesses often face negative stress-related health outcomes and are unavailable for traditional face-to-face interventions due to the intensity and constraints of their caregiver role. Just-in-time adaptive interventions (JITAIs) have emerged as a design framework that is particularly suited for interventional mobile health studies that deliver in-the-moment prompts that aim to promote healthy behavioral and psychological changes while minimizing user burden and expense. While JITAIs have the potential to improve caregivers’ health-related quality of life (HRQOL), their effectiveness for caregivers remains poorly understood.

Objective: The primary objective of this study is to evaluate the dose-response relationship of a fully automated JITAI-based self-management intervention involving personalized mobile app notifications targeted at decreasing the level of caregiver strain, anxiety, and depression. The secondary objective is to investigate whether the effectiveness of this mobile health intervention was moderated by the caregiver group. We also explored whether the effectiveness of this intervention was moderated by (1) previous HRQOL measures, (2) the number of weeks in the study, (3) step count, and (4) minutes of sleep.

Methods: We examined 36 caregivers from 3 disease groups (10 from spinal cord injury, 11 from Huntington disease, and 25 from allogeneic hematopoietic cell transplantation) in the intervention arm of a larger randomized controlled trial (subjects in the other arm received no prompts from the mobile app) designed to examine the acceptability and feasibility of this intensive type of trial design. A series of multivariate linear models implementing a weighted and centered least squares estimator were used to assess the JITAI efficacy and effect.

Results: We found preliminary support for a positive dose-response relationship between the number of administered JITAI messages and JITAI efficacy in improving caregiver strain, anxiety, and depression; while most of these associations did not meet conventional levels of significance, there was a significant association between high-frequency JITAI and caregiver strain. Specifically, administering 5-6 messages per week as opposed to no messages resulted in a significant decrease in the HRQOL score of caregiver strain with an estimate of –6.31 (95% CI –11.76 to –0.12; P=.046). In addition, we found that the caregiver groups and the participants’ levels of depression in the previous week moderated JITAI efficacy.

Conclusions: This study provides preliminary evidence to support the effectiveness of the self-management JITAI and offers practical guidance for designing future personalized JITAI strategies for diverse caregiver groups.

Trial Registration: NCT04556591;

JMIR Form Res 2023;7:e43099



Caregiving is often related to negative stress-related health outcomes, especially when the care recipient has a chronic disability or disease that includes several physical, mental, and social challenges [1-4]. Caregiver burden, which can include stress, social isolation, and financial pressures, can have a negative impact on the physical and mental health of both the caregiver and care recipient [5-8]. Currently, several intervention strategies have been made available to caregivers, including psychoeducation, skills training, and therapeutic counseling that can help decrease caregiver burden, promote healthy behavior change, and improve well-being [9-12]. However, these interventions are typically time-intensive, requiring in-person meetings with trained professionals, something that can be difficult for caregivers who are already overwhelmed by the intensity and constraints of the caregiver role.

Recent technological advances in mobile phones and wearable devices provide a new integrated platform to deliver in-the-moment interventions (ie, push notifications) with minimal user burden (eg, a few prompts per day) and maximal spatial and temporal flexibility [13,14]. Mobile devices can collect objective real-time measurements such as physical activity and geographic location as well as self-assessments (eg, self-report surveys) from the user to inform personalized (ie, tailored) just-in-time adaptive interventions (JITAIs) [14-17]. Previous studies have shown that JITAIs based on wearable devices and smartphone data are associated with significant improvements in health outcomes, including improved physical activity, smoking cessation, and reductions in mental health symptoms [18-23]. However, we are unaware of any work that has used JITAIs in caregivers of persons with significant health conditions. Besides, the advantages of low user burden, real-time behavioral support, and high flexibility make the JITAI particularly well suited for these caregivers, who are already overwhelmed by their caregiver role. Therefore, we piloted a microrandomized trial (MRT) to explore the potential effectiveness of JITAIs upon outcome measures obtained from both health-related quality of life (HRQOL) survey data and actigraphy data (ie, physical activity and sleep) [18,19,24-26]. An MRT is a sequentially randomized experimental design that allows random assignment at each of many decision points (eg, every hour or day), which is different from classical randomized controlled trials where subjects are only randomized once before intervention. One of the advantages of the MRT design is that it enables us to examine time-varying moderated intervention effects, which is crucial to personalized JITAI because the variable (eg, potential effect moderator of interest) may change over time, reflecting varying circumstances that influence whether and how much an individual may benefit from interventions [25].

In this study, we have built and deployed the custom-made CareQOL app, which collects and displays passive sensor data (eg, step count, sleep, and heart rate) from a Fitbit and self-reported HRQOL (ie, caregiver strain, depression, and anxiety), and uses these data to deliver personalized JITAIs aimed at improving physical and mental health outcomes (see Table S1 in Multimedia Appendix 1 for example push notifications) [27]. This behavioral intervention consists of personalized “push” notifications that are provided in the form of tips or life insight messages via the app. Tips, consistent with cognitive behavioral theory [28,29], are nondata-based statements that are motivational-focused (why change) or ability-focused (how to change) to promote healthy behaviors. Life insights, targeting enhanced self-monitoring and self-management as a tool of behavior change, include push notifications that summarize personalized data. One key concept for this behavioral JITAI is push notification dosage (ie, how frequently the intervention messages are sent). Excessive push notifications can cause intervention fatigue (ie, burnout), affecting the adherence and retention of users. On the contrary, insufficient push notifications make it harder for the user to create behavioral change, which can result in reduced efficacy of the mobile health (mHealth) intervention [15,30]. Given this, a better understanding of the optimal dosage of push notifications is critical to maximizing positive health outcomes.

Accordingly, we explored the dose-response effect of these push notifications on improving three aspects of caregiver HRQOL (caregiver strain, anxiety, and depression) in three distinct groups of caregivers: (1) caregivers whose partners have chronic conditions caused by a traumatic event (ie, spinal cord injury [SCI]); (2) caregivers whose partners face a chronic, progressive, and neurodegenerative disease (ie, Huntington disease [HD]); and (3) caregivers whose partners have a recurrent cancer condition that commonly requires intensive and repeated hospitalizations (ie, allogeneic hematopoietic cell transplantation [HCT]).

The analyses presented in this paper aim to provide preliminary evidence to inform in-the-moment whether and how many prompts to deliver in future JITAI designs among caregivers to persons with 3 major categories of medical conditions. Specifically, our primary aim was to explore whether the delivery of personalized JITAIs at high (5 or 6 messages per week), medium (3 or 4 messages per week), or low (1 or 2 messages per week) frequency can reduce the level of caregiver strain, anxiety, and depression relative to no prompts (no messages). Our secondary aim was to determine whether JITAI efficacy for each aspect of caregiver health was moderated (varied) by caregiver group (ie, caregivers of people with SCI, HD, or HCT). This prespecified subgroup analysis can provide evidence of differential dosage effects of personalized interventions by caregiver group, a factor that can inform the future design of personalized JITAIs. Finally, our exploratory aim was to examine whether four variables, (1) previous HRQOL measures, (2) number of weeks in study, (3) step count, and (4) minutes of sleep, can modify the effectiveness of personalized JITAIs on HRQOL outcomes. Findings from this exploratory analysis can inform the future design of personalized JITAIs. For example, if we find that longer sleep duration is related to higher JITAI efficacy, we can send more push notifications to those caregivers with longer sleep duration relative to caregivers with fewer sleep hours to maximize the benefit of the JITAI intervention.

Study Design and Settings

The results presented here comprise a subanalysis of data from the JITAI-arm of a larger behavioral randomized controlled trial feasibility and acceptability trial published elsewhere [27]. Of the initial 72 caregivers that were enrolled in this study, a total of 36 caregivers (n=11 HD, n=10 SCI, and n=15 HCT) were randomized to receive the intervention and are the focus of these analyses. Although a detailed description of this sample is reported in the acceptability and feasibility paper that is under review (revision submitted), a brief summary of the sample is provided. Specifically, eligible caregivers had to be at least 18 years old, be able to read and understand English, and be caring for an individual 18 years or older with medically documented HD, SCI, or HCT. Caregivers for individuals with SCI also needed to be caring for someone who was ≥1-year post injury, and caregivers of people with HCT had to be caring for an individual who was receiving, had received, or was scheduled to receive HCT. Professional and paid caregivers were excluded from this study.

Briefly speaking, participation in this behavioral trial was fully remote and consisted of a 2-hour baseline session followed by a 3-month (90-day) home monitoring period. The completion of informed consent, as well as several self-reported measures and instructions for the home monitoring period, were completed during the baseline session, which was conducted via Zoom. The home monitoring period included continuous monitoring (ie, step count, sleep minutes, and heart rate) using a wearable device (ie, Fitbit) and the collection of real-time self-reported ratings of HRQOL measures via the CareQOL study app, which occurred once per day. This feasibility study followed a 2-arm randomized controlled design. For each day during the home monitoring period, those caregivers in the JITAI arm had a 50-50 chance to receive personalized JITAI messages derived from sensor data (eg, accelerometer-based estimates of physical activity and sleep duration) and the daily self-reported ratings of HRQOL (caregiver strain, anxiety, and depression) via the study app, while the control arm did not receive any JITAI messages. The content and type of the messages were randomly drawn from a pool of over 400 messages; message types included data feedback, facts, tips, and support.

Sample Size Considerations

The main purpose of the feasibility and acceptability trial was to establish the feasibility and acceptability of an intensive data collection protocol to inform a future larger effectiveness trial of this self-management JITAI on caregivers of people with chronic medical conditions. Given that there are no formal power analyses for this type of trial, we based our sample size (N=72) on other feasibility and acceptability trials that have been published in the literature for mHealth apps [31-33]. For the purposes of these analyses, we examined the 36 caregivers that were assigned to the JITAI arm.

Outcomes and Measures

The primary outcomes of this study were the daily HRQOL measures of caregiver strain, anxiety, and depression, which were assessed by a single daily question that was drawn from each respective item bank (described below). The CareQOL measures were initially developed and validated in caregivers of people with traumatic brain injury [34-37], but have also been validated for use in caregivers of SCI, HD, and cancer [38]. The CareQOL caregiver strain item bank [35] was used to assess feelings of being overwhelmed or burdened by the caregiver role; the PROMIS (Patient-Reported Outcomes Measurement Information System) Anxiety item bank [39,40] was used to provide a measure of the level of anxiety (eg, fear, anxious misery, hyperarousal, and somatic symptoms related to arousal); and the PROMIS Depression item bank [39,40] provided a measure of perceived depression (ie, negative mood, decrease in positive affect, information-processing deficits, negative views of the self, and negative social cognition). Responses to each of the 3 daily questions were on a 5-point scale, with higher scores corresponding to higher levels of the named construct. Resulting scores were on a T metric (mean 50, SD 10) relative to a reference population (either other caregivers, ie, for caregiver strain, or the US general population for PROMIS Anxiety and PROMIS Depression) [41]. The daily real-time assessments of 3 variables were administered as a computer adaptive test (CAT) [42], which aims to estimate an examinee’s level of the construct (eg, depression) through sequentially administered items (questions), where each item is selected adaptively based on the examinee’s previous response. As a consequence, the estimate of the examinee’s level of the construct typically becomes more precise as more items are administered. A CAT event is composed of an initialization of the test at the beginning and a termination criterion (eg, fixed time limit, certain precision achieved) that ends the test. In our study, the CAT event was administered on a weekly basis and restarted each Monday, with a single item administered each day. Given this, analyses were conducted on a weekly level; JITAI messages were considered over the course of a week, as were HRQOL ratings (ie, CAT events). The final HRQOL scores for each CAT event (eg, a score on Saturday would be treated as the final score of the week if a score on Sunday was not provided and a score on Saturday was assessed) were outcomes of our interest in the analysis. CAT events with fewer than 3 daily responses over the course of a week were considered invalid or unreliable and were removed from the analyses. Daily step count and sleep minutes were recorded through wearable devices, and weekly individual step count and sleep minutes were calculated by taking the average of the daily measures.

Ethics Approval

The procedures of this feasibility trial were approved by IRBMED (Application Approval HUM00184455), and the trial was registered with (NCT04556591). Caregivers provided informed consent prior to the commencement of study activities.

Statistical Analysis


Descriptive statistics of participants were calculated by care recipient type (HCT, SCI, or HD). Continuous variables were reported using means and SDs, and categorical variables were reported by percentage. Additionally, the distributions of the 3 HRQOL scores were examined by histograms.

Primary Aim

The primary objective of this study was to assess whether the delivery of personalized JITAIs at low, medium, and high frequency can impact same-week caregiver HRQOL and to determine if there was a dose-response relationship. The outcomes of interest included HRQOL scores for caregiver strain, anxiety, and depression. Due to the weekly design of the CAT, the intervention of interest was the total number of JITAI messages the caregivers received up to the day before they submitted their last weekly survey response (eg, if one submitted the last survey response on Saturday, only the number of received messages between Monday and Friday would be considered). The number of weekly messages received ranged from 0 to 6. The reason why the message received on the final survey response day was excluded from counting is that we want to make sure the intervention (number of JITAIs received) happened before the outcome was measured (the final survey response resulting in the final score). Furthermore, we categorized the number of intervention messages received per week into four groups: (1) no messages (0 received), (2) low frequency (1 or 2 received), (3) medium frequency (3 or 4 received), and (4) high frequency (5 or 6 received). The primary analysis was performed using a weighted and centered least squares (WCLS) estimator, as proposed by Boruvka et al [43]. The WCLS estimator is asymptotically unbiased for estimating the main causal effect and effect moderation. It can also protect against potential misspecification of the terms that do not interact with the treatment variable in its calculation of parameter estimates. In our statistical analysis, this estimating procedure was readily implemented using generalized estimating equations (GEEs) with an independent working correlation matrix. More specifically, we fit linear models with an outcome (ie, caregiver strain, anxiety, or depression score) as the dependent variable and a treatment indicator (ie, an indicator variable indicating a low, medium, or high dosage of JITAIs received) as the independent variable, solved by GEE techniques. The linear term of week in the study (ie, the value is 5 if a subject is measured at week 5), as well as the caregiver’s sex and baseline age, were included as control variables to increase statistical power. Given that the HRQOL scores were highly right-skewed, a log transformation was used to make the distribution appear more symmetric. Although the symmetry of dependent variables is not required for WCLS estimators to be consistent, being symmetrically distributed can increase the efficiency of estimation, considering the limited sample size in this pilot study.

Secondary Aim

The secondary aim of this analysis was to assess whether the efficacy of JITAIs at low, medium, and high frequency differs among the 3 caregiver groups (HCT, SCI, and HD). Therefore, we performed a prespecified subgroup analysis by adding an interaction term between group (ie, HCT, SCI, and HD) indicators and treatment variables (ie, low, medium, or high dosage of JITAIs) into the linear models used in the primary analysis. Different estimates of coefficients of interaction terms would suggest that the efficacy of delivery of JITAIs differed by subgroup. We also examine the improvement in HRQOL associated with receiving a low, medium, or high dosage of JITAIs compared to receiving no messages; improvements are reported on 3 scales: the log scale, the percentage scale, and the original T score scale.

Exploratory Aim

The exploratory aim of this analysis was to examine whether the four variables, (1) previous week’s HRQOL scores, (2) week in the study, (3) step count, and (4) minutes of sleep, modify the efficacy of the delivery of JITAIs. These variables were called “moderators” because they may affect the direction and magnitude of the relationship between treatment and outcome. We considered these 4 variables as time-varying moderators because the values of these variables can vary across time, representing the varying circumstances of each individual that may inform the best intervention options. The ability to assess time-varying moderators is one of the advantages of MRT. The examination of the treatment effect moderator of the previous week’s corresponding HRQOL score was motivated by the hypothesis that individuals with worse HRQOL might gain more efficacy from future treatment. The inclusion of a week in the study was due to the fact that waning effectiveness over time (ie, the efficacy of treatment decreasing over time) is common in terms of mHealth interventions [19,26,44]. We also hypothesized that the level of physical activity and sleep may affect the efficacy of the JITAI messages [18]. These exploratory analyses were performed by including interaction terms between the treatment variables and moderator variables (eg, treatment X previous week’s HRQOL score). A significant nonzero coefficient of interaction term between treatment and moderator variable X would indicate that X can moderate the efficacy of treatment (ie, different values of the moderator correspond to different efficacy of treatment).

Analysis codes are available at [45]. Deidentified data supporting the results and figures in this manuscript are available upon request from the corresponding author. Data were analyzed using R (R Foundation for Statistical Computing), version 4.0.5 [46], the packages tidyverse, version 1.3.1 [47], ggplot2, version 3.3.5 [48], ggpubr, version 0.4.0 [49], and geepack, version 1.3-2 [50-52]. The study design was preregistered with (NCT04556591).

Study Participants

A total of 36 participants were included in our analysis, with 11 from the HD group, 10 from the SCI group, and 15 from the HCT group. The demographic table is shown in Table 1. In general, the 3 caregiver groups did not differ on demographic variables, with the exception that caregivers of people with HCT reported significantly fewer years (approximately 9) in the caregiving role than the other 2 subgroups. Of the total 36 participants in the study, 33 used the Fitbit Inspire 2 (provided by the study team), while the remaining 3 participants used their own personal Fitbit devices (n=1 Fitbit Versa2, n=1 Fitbit Inspire HR, and n=1 Fitbit Charge 4). Missing data occurred throughout the study. The average missing rate was 9.7% (561/5796 person-days) for the daily HRQOL surveys (caregiver strain, anxiety, and depression), 2.8% (163/5796 person-days) for daily steps, and 14.2% (822/5796 person-days) for daily sleep (from wearing the Fitbit). The participants’ average daily step count was 7802.9, and the average daily sleep duration was 410.6 minutes.

Table 1. Demographics statistics for enrolled caregivers (n=36).
VariablesJITAIa (n=36)HDb (n=11)SCIc (n=10)HCTd (n=15)
Gender, n (%)

Female28 (78)7 (64)8 (80)13 (87)

Male8 (22)4 (36)2 (20)2 (13)
Race, n (%)

African American1 (3)0 (0)1 (10)0 (0)

Asian3 (8)0 (0)2 (20)1 (7)

Caucasian29 (81)11 (100)6 (60)12 (80)

More than 13 (8)0 (0)1 (10)2 (13)
Ethnicity, n (%)

Non-Hispanic33 (92)9 (82)9 (90)15 (100)

Hispanic2 (6)2 (18)0 (0)0 (0)

Missing1 (3)0 (0)1 (10)0 (0)
Marital status, n (%)

Married or cohabitating30 (83)8 (73)10 (100)12 (80)

Single and divorced3 (8)2 (18)0 (0)1 (7)

Single and never married2 (6)0 (0)0 (0)2 (13)

Missing1 (3)1 (9)0 (0)0 (0)
Work status, n (%)

Full-time21 (58)7 (64)7 (70)7 (47)

Part-time2 (6)0 (0)1 (10)1 (7)

Student1 (3)0 (0)0 (0)1 (7)

Retired7 (19)4 (36)1 (10)2 (13)

Unemployed <1 year, looking for work1 (3)0 (0)1 (10)0 (0)

Unemployed >1 year not looking for work2 (6)0 (0)0 (0)2 (13)

Other2 (6)0 (0)0 (0)2 (13)
Age (years), mean (SD)54.4 (13.05)60.00 (10.05)52.9 (15.83)51.20 (12.47)
Time in caregiver role (years), mean (SD)8.17 (7.79)11.00 (8.35)12.20 (7.51)3.26 (4.69)
Age of the care recipient (years), mean (SD)50.3 (14.49)54.36 (11.27)41.4 (14.30)53.50 (14.96)
Relationship to care recipient, n (%)

Spouse or partner22 (61)7 (64)4 (40)11 (73)

Child3 (8)0 (0)2 (20)1 (7)

Parent8 (22)3 (27)2 (20)3 (20)

Sibling2 (6)1 (9)1 (10)0 (0)

Friend1 (3)0 (0)1 (10)0 (0)
Time caregiving, n (%)

1-2 h/d or less12 (33)6 (55)4 (40)2 (13)

3-4 h/d (half a working day)13 (36)3 (27)3 (30)7 (47)

5-8 h/d (full working day)5 (14)1 (9)1 (10)3 (20)

9-12 h/d1 (3)0 (0)0 (0)1 (7)

>12 h/d or round the clock care5 (14)1 (9)2 (20)2 (13)

aJITAI: just-in-time adaptive intervention.

bHD: Huntington disease.

cSCI: spinal cord injury.

dHCT: hematopoietic cell transplantation.

Primary Aim

The primary analysis indicated that the delivery of JITAIs tended to decrease (ie, improve) caregiver strain, anxiety, and depression scores. In addition, the efficacy of the JITAIs tended to increase as the dosage of push notifications increased (ie, higher frequency; see Table 2). More specifically, caregivers who received 1 or 2 messages within a week, on average, would have around a 1-point reduction in caregiver strain, a 1-point reduction in anxiety, and a 1-point reduction in depression in terms of the T score (mean 50, SD 10), compared to those who did not receive any messages. Caregivers with 3 or 4 messages received per week would, on average, have a corresponding 4-point decrease in caregiver strain, a 4-point decrease in anxiety, and a 2-point decrease in depression (ie, improvement) in their T score relative to caregivers that did not receive any messages that week. For those who received more than 4 messages, there were score improvements of around 6 points for caregiver strain, 5 points for anxiety, and 4 points for depression. Taken together, there is preliminary evidence to support the efficacy of the JITAI, and a higher dosage appears to be associated with greater improvements. Although most of the coefficients did not meet traditional cutoffs (ie, P<.05; only the coefficient of high-frequency JITAI on the score of caregiver strain had P=.046), these analyses were underpowered (given the small sample sizes), and thus, the fact that we saw consistent improvements in HRQOL scores indicates that this is a promising approach for improving HRQOL in this population.

Table 2. The estimated effect of delivery of just-in-time adaptive interventions (JITAIs) on health-related quality of life (HRQOL) scores of caregiver strain, anxiety, and depression, along with 95% CI. The improvements were shown in log-scale, percentage, and original score scale.

Beta (95% CI)ImprovementP values

Percentage (95% CI)Score (95% CI)
Caregiver strain

No messagea

Low frequency (1, 2)–0.05 (–0.15 to 0.06)–0.05 (–0.14 to 0.06)–2.36 (–7.11 to 2.96).37

Medium frequency (3,4)–0.08 (–0.18 to 0.03)–0.07 (–0.16 to 0.03)–3.68 (–8.18 to 1.35).15

High frequency (5,6)–0.13 (–0.27 to 0.00)–0.13 (–0.23 to –0.00)–6.31 (–11.76 to –0.12).046

No message

Low frequency (1-2)–0.03 (–0.16 to 0.11)–0.03 (–0.15 to 0.11)–1.33 (–7.77 to 5.84).69

Medium frequency (3-4)–0.08 (–0.20 to 0.05)–0.07 (–0.18 to 0.05)–3.82 (–9.35 to 2.48).23

High frequency (5-6)–0.10 (–0.24 to 0.04)–0.10 (–0.22 to 0.04)–5.15 (–11.27 to 1.87).14

No message

Low frequency (1-2)–0.02 (–0.19 to 0.15)–0.02 (–0.17 to 0.17)–0.83 (–8.47 to 8.21).84

Medium frequency (3-4)–0.04 (–0.20 to 0.12)–0.04 (–0.18 to 0.12)–2.02 (–8.91 to 6.10).61

High frequency (5-6)–0.08 (–0.26 to 0.10)–0.08 (–0.23 to 0.10)–3.72 (–11.08 to 5.05).38

aNot available.

Subgroup Analysis

Our secondary analysis indicated that the 3 caregiver groups tended to respond differently to the JITAI messages (Figure 1 and Table 3). Table 3 shows the estimated effect size (improvement in T score) in log, percentage, and original scale for the HCT, HD, and SCI groups. For example, caregivers in the SCI group who received 3 or 4 messages per week had, on average, a 0.17 (95% CI –0.33 to –0.00) decrease in T score in the log scale, a 16% (95% CI 28%-0%) decrease in percentage, and an 8.05 (95% CI 0.19-14.67) decrease in the original scale when compared to those in the SCI group who did not receive any messages. In general, for all 3 caregiver groups, there was a nonsignificant trend (as indicated by a wide CI), which may indicate that the improvements in HRQOL may increase as the dosage of the JITAI (ie, frequency of push notifications) increases. However, the trajectory of the efficacy of these treatment effects was different among the 3 groups. Specifically, for the HD subgroup, caregivers with high-frequency prompts benefited more from the delivery of JITAI messages, compared to a similar treatment effect for those with low- and medium-frequency prompts. For the SCI group, the efficacy of treatment effects caregivers received did not seem to be affected by how frequently the JITAI messages were sent. Finally, for the HCT group, the estimated treatment effects did not seem to differ by dosage (ie, prompt frequency), and the effect sizes of different treatment dosages were consistently smaller relative to the SCI group.

Figure 1. The estimated effect of delivery of just-in-time adaptive interventions (JITAIs) at low, medium, and high frequencies for 3 caregiver subgroups: HCT, HD, and SCI, along with 95% CIs. HCT: hematopoietic cell transplantation; HD: Huntington disease; SCI: spinal cord injury.
Table 3. The estimated effect of delivery of just-in-time adaptive interventions (JITAIs) on health-related quality of life (HRQOL) scores of caregiver strain, anxiety, and depression by HCT, HD and SCI subgroups, along with 95% CI. The improvements are shown in log-scale, percentage, and original score scale.
Subgroup and prompt frequencyBeta (95% CI)ImprovementP values

Percentage (95% CI)Score (95% CI)
Caregiver strain


Low (1-2)–0.03 (–0.13 to 0.08)–0.02 (–0.12 to 0.08)–1.20 (–6.01 to 4.13).64

Medium (3-4)–0.04 (–0.18 to 0.10)–0.04 (–0.16 to 0.10)–1.81 (–7.81 to 5.03).58

High (5-6)–0.07 (–0.24 to 0.10)–0.07 (–0.21 to –0.11)–3.24 (–10.33 to 5.13).42


Low (1-2)0.02 (–0.10 to 0.14)0.02 (–0.10 to 0.15)1.00 (–5.13 to 7.93).76

Medium (3-4)–0.00 (–0.15 to 0.14)–0.00 (–0.14 to 0.15)–0.05 (–7.05 to 8.06).99

High (5-6)–0.15 (–0.35 to 0.04)–0.14 (–0.29 to 0.04)–7.33 (–15.21 to 2.22).12


Low (1-2)–0.12 (–0.39 to 0.16)–0.11 (–0.32 to 0.17)–5.76 (–16.76 to 8.72).40

Medium (3-4)–0.17 (–0.33 to –0.00)–0.16 (–0.28 to –0.00)–8.05 (–14.67 to –0.19).046

High (5-6)–0.19 (–0.56 to 0.18)–0.17 (–0.43 to 0.19)–9.03 (–22.17 to 9.93).31


Low (1-2)0.03 (–0.11 to 0.16)0.03 (–0.10 to 0.17)1.33 (–4.91 to 8.48).69

Medium (3-4)–0.02 (–0.18 to 0.14)–0.02 (–0.16 to 0.14)–1.01 (–7.92 to 7.03).79

High (5-6)–0.03 (–0.17 to 0.12)–0.02 (–0.16 to 0.13)–1.20 (–7.69 to 6.36).74


Low (1-2)0.04 (–0.13 to 0.20)0.04 (–0.12 to 0.22)1.91 (–6.38 to 11.66).68

Medium (3-4)0.02 (–0.13 to 0.17)0.02 (–0.12 to 0.18)1.10 (–6.18 to 9.60).78

High (5-6)–0.12 (–0.32 to 0.08)–0.11 (–0.27 to 0.09)–5.70 (–14.06 to 4.52).25


Low (1-2)–0.23 (–0.50 to 0.03)–0.21 (–0.40 to 0.03)–10.80 (–20.45 to 1.79).09

Medium (3-4)–0.30 (–0.45 to –0.14)–0.26 (–0.36 to –0.13)–13.30 (–18.8 to –6.91)<.001

High (5-6)–0.28 (–0.62 to 0.06)–0.24 (–0.46 to 0.07)–12.60 (–23.97 to 3.39).11


Low (1-2)0.16 (–0.05 to 0.37)0.17 (–0.05 to 0.44)6.91 (–2.09 to 18.00).14

Medium (3-4)0.14 (–0.05 to 0.34)0.15 (–0.05 to 0.41)6.30 (–2.11 to 16.54).15

High (5-6)0.12 (–0.10 to 0.34)0.13 (–0.09 to 0.40)5.18 (–3.85 to 16.45).28


Low (1-2)–0.04 (–0.16 to 0.07)–0.04 (–0.15 to 0.07)–2.45 (–8.09 to 3.84).43

Medium (3-4)–0.04 (–0.14 to 0.07)–0.04 (–0.13 to 0.07)–2.07 (–7.41 to 3.81).47

High (5-6)–0.17 (–0.33 to –0.01)–0.16 (–0.28 to –0.01)–8.79 (–15.69 to –0.70).03


Low (1-2)–0.06 (–0.31 to 0.19)–0.06 (–0.27 to 0.21)–3.05 (–13.46 to 10.37).63

Medium (3-4)–0.13 (–0.27 to 0.02)–0.12 (–0.24 to 0.02)–5.95 (–11.87 to 0.83).08

High (5-6)–0.10 (–0.40 to 0.19)–0.10 (–0.33 to 0.21)–4.88 (–16.43 to 10.63).50

aHCT: hematopoietic cell transplantation.

bHD: Huntington disease.

cSCI: spinal cord injury.

Exploratory Aim

The exploratory analysis indicated that the previous week’s HRQOL score of depression appears to moderate the effectiveness of the delivery of the JITAI on depression (see Figure 2). There was no evidence to support a moderation effect of week in the study, previous week’s daily step count, or daily minutes of sleep on treatment outcomes (see Figure S1 in Multimedia Appendix 1). However, we found that the JITAI’s efficacy was significantly moderated by the previous week’s HRQOL score of depression. Specifically, for those individuals receiving high-frequency dosage, high levels of depression in the previous week were associated with increased JTAI efficacy. For example, compared to those who received no messages, caregivers who received more than 5 messages per week on average had a 0.66-point reduction in depression score if their previous week’s depression score was 40, a 2.4-point reduction if their previous week’s score was 50, and a 4.4-point reduction if their previous week’s score was 60.

Figure 2. The estimated effect of delivery of just-in-time adaptive interventions (JITAIs) at low, medium, and high frequencies on weekly T score of depression over different weeks, along with shaded area as 95% CIs.


Through this MRT of mHealth intervention, we find that dosage of the messages, caregiver groups, and level of preintervention depression may impact the efficacy of our behavioral JITAI (ie, the delivery of the JITAI at low, medium, and high frequency can decrease levels of caregiver strain, anxiety, and depression) in caregivers of persons with significant health conditions. Specifically, our findings indicate that (1) higher dosage is associated with increased JITAI efficacy; (2) different caregiver groups respond differently to the JITAIs in terms of caregiver strain, anxiety, and depression; and (3) a high level of preintervention depression is associated with increased JITAI efficacy on depression. However, the observed impact is modest and subject to high uncertainty due to the limited sample size, necessitating future large-scale experiments for validation.

Principal Results

Our work provides preliminary evidence to support the idea that self-management–based JITAIs are potentially effective in improving caregiver HRQOL; those individuals in the intervention group demonstrated numerical improvements in caregiver strain, anxiety, and depression. However, due to the limited sample size, the significance of these findings is yet to be determined, which requires future large-scale trials to verify. Even so, this highly scalable intervention expresses the potential to improve caregiver outcomes with minimal burden and cost. Unlike traditional face-to-face interventions, which require time-intensive in-person meetings, an mHealth-based JITAI has the additional advantages of temporal and spatial flexibility, factors that are critically important for interventions targeting individuals that are already overwhelmed by their role as caregivers.

In addition, our research reveals a dosage-response relationship that shows that a higher dosage of JITAI messages is associated with greater improvement in caregiver strain, anxiety, and depression scores. This finding indicates that the dosage of JITAI messages is important and should be considered in future JITAI designs, given that a higher dosage appears to be associated with increased intervention efficacy. However, it is also possible that there may be a ceiling effect where excessive doses of JITAIs no longer improve the effectiveness of the intervention; in fact, it is also possible that excessive JITAI dosing may increase user burden and result in an “overexposure” effect that results in decreased JITAI effectiveness and increased attrition [15,30,53]. Therefore, identifying the optimal JITAI dosage is warranted. Moreover, we found that different caregiver groups respond differently to the JITAI intervention, making that another important consideration for future trial designs where the optimal dosage may differ for different subgroups. For example, we found that caregivers of people with HD appeared to benefit more from high-frequency messages compared to low and medium frequency; thus, a high-frequency dosage might be optimal and could maximize outcomes for HD-specific caregivers.

Our work also indicated that the previous week’s level of depression (ie, before intervention) can moderate the JITAI’s efficacy on depression. This finding also has implications for future JITAI design. Specifically, for those caregivers who experienced high levels of depression in the previous week (preintervention), the app can be optimized to send more frequent JITAI messages focused on mitigating depression in the upcoming weeks. On the contrary, the system can be optimized to minimize the number of JITAI messages sent in subsequent weeks for those individuals with lower levels of depression to reduce their burden. It is also possible that caregivers with high levels of depression benefited more from the JITAI because of a ceiling effect; in essence, those caregivers who were less depressed may have had less room for further improvement [54,55].

Taken together, the reported findings from this trial provide support for the use of JITAIs to promote HRQOL in caregivers of people with significant health conditions. In addition, our work also provides evidence and methodology to guide the future design of a personalized JITAI system.

Limitations and Future Research

First, our analysis is limited by a small sample size. Although there was a trend for significance for the effectiveness of the JITAI in these analyses, most findings did not meet traditional significance level cutoffs (ie, P>.05). Therefore, the replication of these findings in a large-scale trial is warranted to validate the conclusions from these analyses. Second, our study required daily self-reporting of strain, stress, and worry, which may have biased caregiver reporting. Third, we only recruited 3 caregiver groups (ie, HD, HCT, and SCI), which may not represent the whole population of caregivers of persons with significant health conditions. Moreover, the caregivers recruited in our study were mainly Caucasian females (22/36), which may not reflect the caregiver population in the country. Future studies should include more diverse caregiver populations.


This analysis provides preliminary evidence to support both the efficacy of our self-management-based JITAI as well as data to indicate that a greater number of messages each week have the potential to increase the efficacy of this intervention on caregiver HRQOL outcomes. Findings can also be used to inform the future design of a JITAI-based system that personalizes the intervention based on important variables that can impact JITAI efficacy (caregiver group, dosage of the JITAI messages, and preintervention depression level). Ultimately, we believe that highly scalable, inexpensive, low-touch self-management JITAIs have the potential to improve caregiver outcomes by providing a low-burden caregiver intervention that improves overall well-being.


Work on this manuscript was supported by grant R01NR013658 from the National Institutes of Health; the National Institute of Nursing Research; grants R01HL146354 and K24HL156896 from the National Heart, Lung, and Blood Institute; and grant UL1TR002240 from the National Center for Advancing Translational Sciences. This work was also supported by the University of Michigan Institute for Health Care Policy and Innovation. We thank the investigators, coordinators, and research associates and assistants who worked on this study; the study participants; and the organizations that supported recruitment efforts. We also thank Dr Michael A Kallen (Northwestern University) for providing critical suggestions for analyzing computer adaptive test scores.

Conflicts of Interest

None declared.

Multimedia Appendix 1

Additional figures and tables.

DOCX File , 464 KB

  1. Ready RE, Mathews M, Leserman A, Paulsen JS. Patient and caregiver quality of life in Huntington's disease. Mov Disord 2008;23(5):721-726 [] [CrossRef] [Medline]
  2. Anderson MI, Parmenter TR, Mok M. The relationship between neurobehavioural problems of severe traumatic brain injury (TBI), family functioning and the psychological well-being of the spouse/caregiver: path model analysis. Brain Inj 2002;16(9):743-757 [CrossRef] [Medline]
  3. Hanks RA, Rapport LJ, Vangel S. Caregiving appraisal after traumatic brain injury: the effects of functional status, coping style, social support and family functioning. NeuroRehabilitation 2007;22(1):43-52 [Medline]
  4. Ponsford J, Olver J, Ponsford M, Nelms R. Long-term adjustment of families following traumatic brain injury where comprehensive rehabilitation has been provided. Brain Inj 2003;17(6):453-468 [CrossRef] [Medline]
  5. Goren A, Montgomery W, Kahle-Wrobleski K, Nakamura T, Ueda K. Impact of caring for persons with Alzheimer's disease or dementia on caregivers' health outcomes: findings from a community based survey in Japan. BMC Geriatr 2016;16:122 [] [CrossRef] [Medline]
  6. Ferrara M, Langiano E, Di Brango T, De Vito E, Di Cioccio L, Bauco C. Prevalence of stress, anxiety and depression in with Alzheimer caregivers. Health Qual Life Outcomes 2008;6:93 [] [CrossRef] [Medline]
  7. Mittelman MS, Brodaty H, Wallen AS, Burns A. A three-country randomized controlled trial of a psychosocial intervention for caregivers combined with pharmacological treatment for patients with Alzheimer disease: effects on caregiver depression. Am J Geriatr Psychiatry 2008;16(11):893-904 [] [CrossRef] [Medline]
  8. Covinsky KE, Newcomer R, Fox P, Wood J, Sands L, Dane K, et al. Patient and caregiver characteristics associated with depression in caregivers of patients with dementia. J Gen Intern Med 2003;18(12):1006-1014 [] [CrossRef] [Medline]
  9. Northouse LL, Katapodi MC, Song L, Zhang L, Mood DW. Interventions with family caregivers of cancer patients: meta-analysis of randomized trials. CA Cancer J Clin 2010;60(5):317-339 [] [CrossRef] [Medline]
  10. Harding R, Higginson IJ. What is the best way to help caregivers in cancer and palliative care? A systematic literature review of interventions and their effectiveness. Palliat Med 2003;17(1):63-74 [CrossRef] [Medline]
  11. Hudson P. A critical review of supportive interventions for family caregivers of patients with palliative-stage cancer. J Psychosoc Oncol 2008;22(4):77-92 [CrossRef]
  12. Pasacreta JV, McCorkle R. Cancer care: impact of interventions on caregiver outcomes. Annu Rev Nurs Res 2000;18:127-148 [Medline]
  13. Grekin ER, Beatty JR, Ondersma SJ. Mobile health interventions: exploring the use of common relationship factors. JMIR mHealth uHealth 2019;7(4):e11245 [] [CrossRef] [Medline]
  14. Nahum-Shani I, Hekler EB, Spruijt-Metz D. Building health behavior models to guide the development of just-in-time adaptive interventions: a pragmatic framework. Health Psychol 2015;34S:1209-1219 [] [CrossRef] [Medline]
  15. Nahum-Shani I, Smith SN, Spring BJ, Collins LM, Witkiewitz K, Tewari A, et al. Just-in-time adaptive interventions (JITAIs) in mobile health: key components and design principles for ongoing health behavior support. Ann Behav Med 2018;52(6):446-462 [] [CrossRef] [Medline]
  16. Hardeman W, Houghton J, Lane K, Jones A, Naughton F. A systematic review of just-in-time adaptive interventions (JITAIs) to promote physical activity. Int J Behav Nutr Phys Act 2019;16(1):31 [] [CrossRef] [Medline]
  17. Teepe GW, Da Fonseca A, Kleim B, Jacobson NC, Salamanca Sanabria A, Tudor Car L, et al. Just-in-time adaptive mechanisms of popular mobile apps for individuals with depression: systematic app search and literature review. J Med Internet Res 2021;23(9):e29412 [] [CrossRef] [Medline]
  18. NeCamp T, Sen S, Frank E, Walton MA, Ionides EL, Fang Y, et al. Assessing real-time moderation for developing adaptive mobile health interventions for medical interns: micro-randomized trial. J Med Internet Res 2020;22(3):e15033 [] [CrossRef] [Medline]
  19. Klasnja P, Smith S, Seewald NJ, Lee A, Hall K, Luers B, et al. Efficacy of contextually tailored suggestions for physical activity: a micro-randomized optimization trial of heartsteps. Ann Behav Med 2019;53(6):573-582 [] [CrossRef] [Medline]
  20. Saponaro M, Vemuri A, Dominick G, Decker K. Contextualization and individualization for just-in-time adaptive interventions to reduce sedentary behavior. New York, NY, United States: Association for Computing Machinery; 2021 Presented at: Proceedings of the Conference on Health, Inference, and Learning; April 8-10, 2021; Virtual Event USA p. 246-256 [CrossRef]
  21. Thomas JG, Bond DS. Behavioral response to a just-in-time adaptive intervention (JITAI) to reduce sedentary behavior in obese adults: implications for JITAI optimization. Health Psychol 2015;34S:1261-1267 [] [CrossRef] [Medline]
  22. King AC, Hekler EB, Grieco LA, Winter SJ, Sheats JL, Buman MP, et al. Harnessing different motivational frames via mobile phones to promote daily physical activity and reduce sedentary behavior in aging adults. PLoS One 2013;8(4):e62613 [] [CrossRef] [Medline]
  23. Riley W, Obermayer J, Jean-Mary J. Internet and mobile phone text messaging intervention for college smokers. J Am Coll Health 2008;57(2):245-248 [CrossRef] [Medline]
  24. Klasnja P, Hekler EB, Shiffman S, Boruvka A, Almirall D, Tewari A, et al. Microrandomized trials: an experimental design for developing just-in-time adaptive interventions. Health Psychol 2015;34S:1220-1228 [] [CrossRef] [Medline]
  25. Qian T, Walton AE, Collins LM, Klasnja P, Lanza ST, Nahum-Shani I, et al. The micro-randomized trial for developing digital interventionsxperimental design and data analysis considerations. Psychol Methods 2022;27:874-894 [CrossRef]
  26. Wang J, Fang Y, Frank E, Walton MA, Burmeister M, Tewari A, et al. Effectiveness of gamified team competition in the context of mHealth intervention for medical interns: a micro-randomized trial. Npj Digit Med 2023;6(1):1-8 [CrossRef]
  27. Carlozzi NE, Choi SW, Wu Z, Miner JA, Lyden AK, Graves C, et al. An app-based just-in-time adaptive self-management intervention for care partners (CareQOL): protocol for a pilot trial. JMIR Res Protoc 2021;10(12):e32842 [] [CrossRef] [Medline]
  28. Rathbone AL, Clarry L, Prescott J. Assessing the efficacy of mobile health apps using the basic principles of cognitive behavioral therapy: systematic review. J Med Internet Res 2017;19(11):e399 [] [CrossRef] [Medline]
  29. Marciniak MA, Shanahan L, Rohde J, Schulz A, Wackerhagen C, Kobylińska D, et al. Standalone smartphone cognitive behavioral therapy-based ecological momentary interventions to increase mental health: narrative review. JMIR mHealth uHealth 2020;8(11):e19836 [] [CrossRef] [Medline]
  30. Evans W, Nielsen PE, Szekely DR, Bihm JW, Murray EA, Snider J, et al. Dose-response effects of the text4baby mobile health program: randomized controlled trial. JMIR mHealth uHealth 2015;3(1):e12 [] [CrossRef] [Medline]
  31. Athilingam P, Jenkins B, Johansson M, Labrador M. A mobile health intervention to improve self-care in patients with heart failure: pilot randomized control trial. JMIR Cardio 2017;1(2):e3 [] [CrossRef] [Medline]
  32. Bai Y, Winnie S, Wong CL. Feasibility and acceptability of a mHealth intervention to increase colonoscopy uptake among Chinese first-degree relatives: a pilot cluster randomized controlled trial. Annals of Oncology 2019;30(Suppl 5):v846-v847 [CrossRef]
  33. Magee M, Gholamrezaei A, McNeilage AG, Dwyer L, Sim A, Ferreira M, et al. Evaluating acceptability and feasibility of a mobile health intervention to improve self-efficacy in prescription opioid tapering in patients with chronic pain: protocol for a pilot randomised, single-blind, controlled trial. BMJ Open 2022;12(4):e057174 [] [CrossRef] [Medline]
  34. Carlozzi NE, Kallen MA, Hanks R, Hahn EA, Brickell TA, Lange RT, et al. The TBI-CareQOL measurement system: development and preliminary validation of health-related quality of life measures for caregivers of civilians and service members/veterans with traumatic brain injury. Arch Phys Med Rehabil 2019;100(4S):S1-S12 [] [CrossRef] [Medline]
  35. Carlozzi NE, Kallen MA, Ianni PA, Hahn EA, French LM, Lange RT, et al. The development of a new computer-adaptive test to evaluate strain in caregivers of individuals with TBI: TBI-CareQOL caregiver strain. Arch Phys Med Rehabil 2019;100(4S):S13-S21 [] [CrossRef] [Medline]
  36. Carlozzi NE, Boileau NR, Kallen MA, Nakase-Richardson R, Hahn EA, Tulsky DS, et al. Reliability and validity data to support the clinical utility of the Traumatic Brain Injury Caregiver Quality of Life (TBI-CareQOL). Rehabil Psychol 2020;65(4):323-336 [] [CrossRef] [Medline]
  37. Raad JH, Tulsky DS, Lange RT, Brickell TA, Sander AM, Hanks RA, et al. Establishing the factor structure of a health-related quality of life measurement system for caregivers of persons living with traumatic brain injury. Arch Phys Med Rehabil 2020;101(7):1220-1232 [] [CrossRef] [Medline]
  38. Carlozzi NE, Choi SW, Wu Z, Sen S, Troost J, Lyden AK, et al. The reliability and validity of the TBI-CareQOL system in four diverse caregiver groups. J Patient Rep Outcomes 2023;7(1):57 [] [CrossRef] [Medline]
  39. Cella D, Riley W, Stone A, Rothrock N, Reeve B, Yount S, PROMIS Cooperative Group. The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005-2008. J Clin Epidemiol 2010;63(11):1179-1194 [] [CrossRef] [Medline]
  40. Cella D, Yount S, Rothrock N, Gershon R, Cook K, Reeve B, PROMIS Cooperative Group. The Patient-Reported Outcomes Measurement Information System (PROMIS): progress of an NIH Roadmap cooperative group during its first two years. Med Care 2007;45(5 Suppl 1):S3-S11 [] [CrossRef] [Medline]
  41. Rothrock NE, Amtmann D, Cook KF. Development and validation of an interpretive guide for PROMIS scores. J Patient Rep Outcomes 2020;4(1):16 [] [CrossRef] [Medline]
  42. Meijer RR, Nering ML. Computerized adaptive testing: overview and introduction. Appl Psychol Meas 2016;23(3):187-194 [CrossRef]
  43. Boruvka A, Almirall D, Witkiewitz K, Murphy SA. Assessing time-varying causal effect moderation in mobile health. J Am Stat Assoc 2018;113(523):1112-1121 [] [CrossRef] [Medline]
  44. Shcherbina A, Hershman SG, Lazzeroni L, King AC, O'Sullivan JW, Hekler E, et al. The effect of digital physical activity interventions on daily step count: a randomised controlled crossover substudy of the MyHeart Counts Cardiovascular Health Study. Lancet Digit Health 2019;1(7):e344-e352 [] [CrossRef] [Medline]
  45. Wang J, Wu Z, Choi SW, Sen S, Yan X, Yan JA, et al. Source code for JMIR paper: "Does the dosing of mobile-based just-in-time-adaptive self-management prompts matter? Preliminary findings from a pilot micro-randomized study for caregivers.". GitHub. URL: [accessed 2023-08-08]
  46. R: a language and environment for statistical computing. R Core Team, R Foundation for Statistical Computing. URL: [accessed 2023-08-08]
  47. Wickham H, Averick M, Bryan J, Chang W, McGowan LD, François R, et al. Welcome to the Tidyverse. J Open Source Softw 2019;4(43):1686 [] [CrossRef]
  48. Wickham H. Ggplot2: Elegant Graphics for Data Analysis. 2nd Edition. AG, Switzerland: Springer International Publishing; 2016.
  49. Kassambara A. 'ggplot2' Based publication ready plots. ggpubr. URL: [accessed 2023-08-08]
  50. Yan J. Geepack: yet another package for generalized estimating equations. R-News 2002;2(3):12-14 [] [CrossRef]
  51. Yan J, Fine J. Estimating equations for association structures. Stat Med 2004;23(6):859-880 [CrossRef] [Medline]
  52. Halekoh U, Højsgaard S, Yan J. The R package geepack for generalized estimating equations. J Stat Soft 2006;15(2):1-11 [CrossRef]
  53. Golbus JR, Dempsey W, Jackson EA, Nallamothu BK, Klasnja P. Microrandomized trial design for evaluating just-in-time adaptive interventions through mobile health technologies for cardiovascular disease. Circ Cardiovasc Qual Outcomes 2021;14(2):e006760 [ 0pubmed] [CrossRef] [Medline]
  54. Fanning J, Roberts S, Hillman CH, Mullen SP, Ritterband L, McAuley E. A smartphone "app"-delivered randomized factorial trial targeting physical activity in adults. J Behav Med 2017;40(5):712-729 [CrossRef] [Medline]
  55. Sasaki S, Kurokawa H, Ohtake F. Effective but fragile? Responses to repeated nudge-based messages for preventing the spread of COVID-19 infection. Jpn Econ Rev (Oxf) 2021;72(3):371-408 [] [CrossRef] [Medline]

CAT: computer adaptive test
GEE: generalized estimating equation
HCT: hematopoietic cell transplantation
HD: Huntington disease
HRQOL: health-related quality of life
JITAI: just-in-time adaptive intervention
mHealth: mobile health
MRT: microrandomized trial
NIH: National Institutes of Health
PROMIS: Patient-Reported Outcomes Measurement Information System
SCI: spinal cord injury
WCLS: weighted and centered least squares

Edited by A Mavragani; submitted 07.10.22; peer-reviewed by FJ Yang, T Badger, O Sverdlov; comments to author 07.06.23; revised version received 28.06.23; accepted 03.08.23; published 14.09.23


©Jitao Wang, Zhenke Wu, Sung Won Choi, Srijan Sen, Xinghui Yan, Jennifer A Miner, Angelle M Sander, Angela K Lyden, Jonathan P Troost, Noelle E Carlozzi. Originally published in JMIR Formative Research (, 14.09.2023.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.