The Dosing of Mobile-Based Just-in-Time Adaptive Self-Management Prompts for Caregivers: Preliminary Findings From a Pilot Microrandomized Study

Introduction

Caregiving is often related to negative stress-related health outcomes, especially when the care recipient has a chronic disability or disease that includes several physical, mental, and social challenges [1-4]. Caregiver burden, which can include stress, social isolation, and financial pressures, can have a negative impact on the physical and mental health of both the caregiver and care recipient [5-8]. Currently, several intervention strategies have been made available to caregivers, including psychoeducation, skills training, and therapeutic counseling that can help decrease caregiver burden, promote healthy behavior change, and improve well-being [9-12]. However, these interventions are typically time-intensive, requiring in-person meetings with trained professionals, something that can be difficult for caregivers who are already overwhelmed by the intensity and constraints of the caregiver role.

Recent technological advances in mobile phones and wearable devices provide a new integrated platform to deliver in-the-moment interventions (ie, push notifications) with minimal user burden (eg, a few prompts per day) and maximal spatial and temporal flexibility [13,14]. Mobile devices can collect objective real-time measurements such as physical activity and geographic location as well as self-assessments (eg, self-report surveys) from the user to inform personalized (ie, tailored) just-in-time adaptive interventions (JITAIs) [14-17]. Previous studies have shown that JITAIs based on wearable devices and smartphone data are associated with significant improvements in health outcomes, including improved physical activity, smoking cessation, and reductions in mental health symptoms [18-23]. However, we are unaware of any work that has used JITAIs in caregivers of persons with significant health conditions. Besides, the advantages of low user burden, real-time behavioral support, and high flexibility make the JITAI particularly well suited for these caregivers, who are already overwhelmed by their caregiver role. Therefore, we piloted a microrandomized trial (MRT) to explore the potential effectiveness of JITAIs upon outcome measures obtained from both health-related quality of life (HRQOL) survey data and actigraphy data (ie, physical activity and sleep) [18,19,24-26]. An MRT is a sequentially randomized experimental design that allows random assignment at each of many decision points (eg, every hour or day), which is different from classical randomized controlled trials where subjects are only randomized once before intervention. One of the advantages of the MRT design is that it enables us to examine time-varying moderated intervention effects, which is crucial to personalized JITAI because the variable (eg, potential effect moderator of interest) may change over time, reflecting varying circumstances that influence whether and how much an individual may benefit from interventions [25].

In this study, we have built and deployed the custom-made CareQOL app, which collects and displays passive sensor data (eg, step count, sleep, and heart rate) from a Fitbit and self-reported HRQOL (ie, caregiver strain, depression, and anxiety), and uses these data to deliver personalized JITAIs aimed at improving physical and mental health outcomes (see Table S1 in Multimedia Appendix 1 for example push notifications) [27]. This behavioral intervention consists of personalized “push” notifications that are provided in the form of tips or life insight messages via the app. Tips, consistent with cognitive behavioral theory [28,29], are nondata-based statements that are motivational-focused (why change) or ability-focused (how to change) to promote healthy behaviors. Life insights, targeting enhanced self-monitoring and self-management as a tool of behavior change, include push notifications that summarize personalized data. One key concept for this behavioral JITAI is push notification dosage (ie, how frequently the intervention messages are sent). Excessive push notifications can cause intervention fatigue (ie, burnout), affecting the adherence and retention of users. On the contrary, insufficient push notifications make it harder for the user to create behavioral change, which can result in reduced efficacy of the mobile health (mHealth) intervention [15,30]. Given this, a better understanding of the optimal dosage of push notifications is critical to maximizing positive health outcomes.

Accordingly, we explored the dose-response effect of these push notifications on improving three aspects of caregiver HRQOL (caregiver strain, anxiety, and depression) in three distinct groups of caregivers: (1) caregivers whose partners have chronic conditions caused by a traumatic event (ie, spinal cord injury [SCI]); (2) caregivers whose partners face a chronic, progressive, and neurodegenerative disease (ie, Huntington disease [HD]); and (3) caregivers whose partners have a recurrent cancer condition that commonly requires intensive and repeated hospitalizations (ie, allogeneic hematopoietic cell transplantation [HCT]).

The analyses presented in this paper aim to provide preliminary evidence to inform in-the-moment whether and how many prompts to deliver in future JITAI designs among caregivers to persons with 3 major categories of medical conditions. Specifically, our primary aim was to explore whether the delivery of personalized JITAIs at high (5 or 6 messages per week), medium (3 or 4 messages per week), or low (1 or 2 messages per week) frequency can reduce the level of caregiver strain, anxiety, and depression relative to no prompts (no messages). Our secondary aim was to determine whether JITAI efficacy for each aspect of caregiver health was moderated (varied) by caregiver group (ie, caregivers of people with SCI, HD, or HCT). This prespecified subgroup analysis can provide evidence of differential dosage effects of personalized interventions by caregiver group, a factor that can inform the future design of personalized JITAIs. Finally, our exploratory aim was to examine whether four variables, (1) previous HRQOL measures, (2) number of weeks in study, (3) step count, and (4) minutes of sleep, can modify the effectiveness of personalized JITAIs on HRQOL outcomes. Findings from this exploratory analysis can inform the future design of personalized JITAIs. For example, if we find that longer sleep duration is related to higher JITAI efficacy, we can send more push notifications to those caregivers with longer sleep duration relative to caregivers with fewer sleep hours to maximize the benefit of the JITAI intervention.

Methods

Study Design and Settings

The results presented here comprise a subanalysis of data from the JITAI-arm of a larger behavioral randomized controlled trial feasibility and acceptability trial published elsewhere [27]. Of the initial 72 caregivers that were enrolled in this study, a total of 36 caregivers (n=11 HD, n=10 SCI, and n=15 HCT) were randomized to receive the intervention and are the focus of these analyses. Although a detailed description of this sample is reported in the acceptability and feasibility paper that is under review (revision submitted), a brief summary of the sample is provided. Specifically, eligible caregivers had to be at least 18 years old, be able to read and understand English, and be caring for an individual 18 years or older with medically documented HD, SCI, or HCT. Caregivers for individuals with SCI also needed to be caring for someone who was ≥1-year post injury, and caregivers of people with HCT had to be caring for an individual who was receiving, had received, or was scheduled to receive HCT. Professional and paid caregivers were excluded from this study.

Briefly speaking, participation in this behavioral trial was fully remote and consisted of a 2-hour baseline session followed by a 3-month (90-day) home monitoring period. The completion of informed consent, as well as several self-reported measures and instructions for the home monitoring period, were completed during the baseline session, which was conducted via Zoom. The home monitoring period included continuous monitoring (ie, step count, sleep minutes, and heart rate) using a wearable device (ie, Fitbit) and the collection of real-time self-reported ratings of HRQOL measures via the CareQOL study app, which occurred once per day. This feasibility study followed a 2-arm randomized controlled design. For each day during the home monitoring period, those caregivers in the JITAI arm had a 50-50 chance to receive personalized JITAI messages derived from sensor data (eg, accelerometer-based estimates of physical activity and sleep duration) and the daily self-reported ratings of HRQOL (caregiver strain, anxiety, and depression) via the study app, while the control arm did not receive any JITAI messages. The content and type of the messages were randomly drawn from a pool of over 400 messages; message types included data feedback, facts, tips, and support.

Sample Size Considerations

The main purpose of the feasibility and acceptability trial was to establish the feasibility and acceptability of an intensive data collection protocol to inform a future larger effectiveness trial of this self-management JITAI on caregivers of people with chronic medical conditions. Given that there are no formal power analyses for this type of trial, we based our sample size (N=72) on other feasibility and acceptability trials that have been published in the literature for mHealth apps [31-33]. For the purposes of these analyses, we examined the 36 caregivers that were assigned to the JITAI arm.

Outcomes and Measures

The primary outcomes of this study were the daily HRQOL measures of caregiver strain, anxiety, and depression, which were assessed by a single daily question that was drawn from each respective item bank (described below). The CareQOL measures were initially developed and validated in caregivers of people with traumatic brain injury [34-37], but have also been validated for use in caregivers of SCI, HD, and cancer [38]. The CareQOL caregiver strain item bank [35] was used to assess feelings of being overwhelmed or burdened by the caregiver role; the PROMIS (Patient-Reported Outcomes Measurement Information System) Anxiety item bank [39,40] was used to provide a measure of the level of anxiety (eg, fear, anxious misery, hyperarousal, and somatic symptoms related to arousal); and the PROMIS Depression item bank [39,40] provided a measure of perceived depression (ie, negative mood, decrease in positive affect, information-processing deficits, negative views of the self, and negative social cognition). Responses to each of the 3 daily questions were on a 5-point scale, with higher scores corresponding to higher levels of the named construct. Resulting scores were on a T metric (mean 50, SD 10) relative to a reference population (either other caregivers, ie, for caregiver strain, or the US general population for PROMIS Anxiety and PROMIS Depression) [41]. The daily real-time assessments of 3 variables were administered as a computer adaptive test (CAT) [42], which aims to estimate an examinee’s level of the construct (eg, depression) through sequentially administered items (questions), where each item is selected adaptively based on the examinee’s previous response. As a consequence, the estimate of the examinee’s level of the construct typically becomes more precise as more items are administered. A CAT event is composed of an initialization of the test at the beginning and a termination criterion (eg, fixed time limit, certain precision achieved) that ends the test. In our study, the CAT event was administered on a weekly basis and restarted each Monday, with a single item administered each day. Given this, analyses were conducted on a weekly level; JITAI messages were considered over the course of a week, as were HRQOL ratings (ie, CAT events). The final HRQOL scores for each CAT event (eg, a score on Saturday would be treated as the final score of the week if a score on Sunday was not provided and a score on Saturday was assessed) were outcomes of our interest in the analysis. CAT events with fewer than 3 daily responses over the course of a week were considered invalid or unreliable and were removed from the analyses. Daily step count and sleep minutes were recorded through wearable devices, and weekly individual step count and sleep minutes were calculated by taking the average of the daily measures.

Ethics Approval

The procedures of this feasibility trial were approved by IRBMED (Application Approval HUM00184455), and the trial was registered with ClinicalTrials.gov (NCT04556591). Caregivers provided informed consent prior to the commencement of study activities.

Statistical Analysis

Overview

Descriptive statistics of participants were calculated by care recipient type (HCT, SCI, or HD). Continuous variables were reported using means and SDs, and categorical variables were reported by percentage. Additionally, the distributions of the 3 HRQOL scores were examined by histograms.

Primary Aim

The primary objective of this study was to assess whether the delivery of personalized JITAIs at low, medium, and high frequency can impact same-week caregiver HRQOL and to determine if there was a dose-response relationship. The outcomes of interest included HRQOL scores for caregiver strain, anxiety, and depression. Due to the weekly design of the CAT, the intervention of interest was the total number of JITAI messages the caregivers received up to the day before they submitted their last weekly survey response (eg, if one submitted the last survey response on Saturday, only the number of received messages between Monday and Friday would be considered). The number of weekly messages received ranged from 0 to 6. The reason why the message received on the final survey response day was excluded from counting is that we want to make sure the intervention (number of JITAIs received) happened before the outcome was measured (the final survey response resulting in the final score). Furthermore, we categorized the number of intervention messages received per week into four groups: (1) no messages (0 received), (2) low frequency (1 or 2 received), (3) medium frequency (3 or 4 received), and (4) high frequency (5 or 6 received). The primary analysis was performed using a weighted and centered least squares (WCLS) estimator, as proposed by Boruvka et al [43]. The WCLS estimator is asymptotically unbiased for estimating the main causal effect and effect moderation. It can also protect against potential misspecification of the terms that do not interact with the treatment variable in its calculation of parameter estimates. In our statistical analysis, this estimating procedure was readily implemented using generalized estimating equations (GEEs) with an independent working correlation matrix. More specifically, we fit linear models with an outcome (ie, caregiver strain, anxiety, or depression score) as the dependent variable and a treatment indicator (ie, an indicator variable indicating a low, medium, or high dosage of JITAIs received) as the independent variable, solved by GEE techniques. The linear term of week in the study (ie, the value is 5 if a subject is measured at week 5), as well as the caregiver’s sex and baseline age, were included as control variables to increase statistical power. Given that the HRQOL scores were highly right-skewed, a log transformation was used to make the distribution appear more symmetric. Although the symmetry of dependent variables is not required for WCLS estimators to be consistent, being symmetrically distributed can increase the efficiency of estimation, considering the limited sample size in this pilot study.

Secondary Aim

The secondary aim of this analysis was to assess whether the efficacy of JITAIs at low, medium, and high frequency differs among the 3 caregiver groups (HCT, SCI, and HD). Therefore, we performed a prespecified subgroup analysis by adding an interaction term between group (ie, HCT, SCI, and HD) indicators and treatment variables (ie, low, medium, or high dosage of JITAIs) into the linear models used in the primary analysis. Different estimates of coefficients of interaction terms would suggest that the efficacy of delivery of JITAIs differed by subgroup. We also examine the improvement in HRQOL associated with receiving a low, medium, or high dosage of JITAIs compared to receiving no messages; improvements are reported on 3 scales: the log scale, the percentage scale, and the original T score scale.

Exploratory Aim

The exploratory aim of this analysis was to examine whether the four variables, (1) previous week’s HRQOL scores, (2) week in the study, (3) step count, and (4) minutes of sleep, modify the efficacy of the delivery of JITAIs. These variables were called “moderators” because they may affect the direction and magnitude of the relationship between treatment and outcome. We considered these 4 variables as time-varying moderators because the values of these variables can vary across time, representing the varying circumstances of each individual that may inform the best intervention options. The ability to assess time-varying moderators is one of the advantages of MRT. The examination of the treatment effect moderator of the previous week’s corresponding HRQOL score was motivated by the hypothesis that individuals with worse HRQOL might gain more efficacy from future treatment. The inclusion of a week in the study was due to the fact that waning effectiveness over time (ie, the efficacy of treatment decreasing over time) is common in terms of mHealth interventions [19,26,44]. We also hypothesized that the level of physical activity and sleep may affect the efficacy of the JITAI messages [18]. These exploratory analyses were performed by including interaction terms between the treatment variables and moderator variables (eg, treatment X previous week’s HRQOL score). A significant nonzero coefficient of interaction term between treatment and moderator variable X would indicate that X can moderate the efficacy of treatment (ie, different values of the moderator correspond to different efficacy of treatment).

Analysis codes are available at [45]. Deidentified data supporting the results and figures in this manuscript are available upon request from the corresponding author. Data were analyzed using R (R Foundation for Statistical Computing), version 4.0.5 [46], the packages tidyverse, version 1.3.1 [47], ggplot2, version 3.3.5 [48], ggpubr, version 0.4.0 [49], and geepack, version 1.3-2 [50-52]. The study design was preregistered with ClinicalTrials.gov (NCT04556591).

Results

Study Participants

A total of 36 participants were included in our analysis, with 11 from the HD group, 10 from the SCI group, and 15 from the HCT group. The demographic table is shown in Table 1. In general, the 3 caregiver groups did not differ on demographic variables, with the exception that caregivers of people with HCT reported significantly fewer years (approximately 9) in the caregiving role than the other 2 subgroups. Of the total 36 participants in the study, 33 used the Fitbit Inspire 2 (provided by the study team), while the remaining 3 participants used their own personal Fitbit devices (n=1 Fitbit Versa2, n=1 Fitbit Inspire HR, and n=1 Fitbit Charge 4). Missing data occurred throughout the study. The average missing rate was 9.7% (561/5796 person-days) for the daily HRQOL surveys (caregiver strain, anxiety, and depression), 2.8% (163/5796 person-days) for daily steps, and 14.2% (822/5796 person-days) for daily sleep (from wearing the Fitbit). The participants’ average daily step count was 7802.9, and the average daily sleep duration was 410.6 minutes.

Primary Aim

The primary analysis indicated that the delivery of JITAIs tended to decrease (ie, improve) caregiver strain, anxiety, and depression scores. In addition, the efficacy of the JITAIs tended to increase as the dosage of push notifications increased (ie, higher frequency; see Table 2). More specifically, caregivers who received 1 or 2 messages within a week, on average, would have around a 1-point reduction in caregiver strain, a 1-point reduction in anxiety, and a 1-point reduction in depression in terms of the T score (mean 50, SD 10), compared to those who did not receive any messages. Caregivers with 3 or 4 messages received per week would, on average, have a corresponding 4-point decrease in caregiver strain, a 4-point decrease in anxiety, and a 2-point decrease in depression (ie, improvement) in their T score relative to caregivers that did not receive any messages that week. For those who received more than 4 messages, there were score improvements of around 6 points for caregiver strain, 5 points for anxiety, and 4 points for depression. Taken together, there is preliminary evidence to support the efficacy of the JITAI, and a higher dosage appears to be associated with greater improvements. Although most of the coefficients did not meet traditional cutoffs (ie, P<.05; only the coefficient of high-frequency JITAI on the score of caregiver strain had P=.046), these analyses were underpowered (given the small sample sizes), and thus, the fact that we saw consistent improvements in HRQOL scores indicates that this is a promising approach for improving HRQOL in this population.

Subgroup Analysis

Our secondary analysis indicated that the 3 caregiver groups tended to respond differently to the JITAI messages (Figure 1 and Table 3). Table 3 shows the estimated effect size (improvement in T score) in log, percentage, and original scale for the HCT, HD, and SCI groups. For example, caregivers in the SCI group who received 3 or 4 messages per week had, on average, a 0.17 (95% CI –0.33 to –0.00) decrease in T score in the log scale, a 16% (95% CI 28%-0%) decrease in percentage, and an 8.05 (95% CI 0.19-14.67) decrease in the original scale when compared to those in the SCI group who did not receive any messages. In general, for all 3 caregiver groups, there was a nonsignificant trend (as indicated by a wide CI), which may indicate that the improvements in HRQOL may increase as the dosage of the JITAI (ie, frequency of push notifications) increases. However, the trajectory of the efficacy of these treatment effects was different among the 3 groups. Specifically, for the HD subgroup, caregivers with high-frequency prompts benefited more from the delivery of JITAI messages, compared to a similar treatment effect for those with low- and medium-frequency prompts. For the SCI group, the efficacy of treatment effects caregivers received did not seem to be affected by how frequently the JITAI messages were sent. Finally, for the HCT group, the estimated treatment effects did not seem to differ by dosage (ie, prompt frequency), and the effect sizes of different treatment dosages were consistently smaller relative to the SCI group.

‎

Exploratory Aim

The exploratory analysis indicated that the previous week’s HRQOL score of depression appears to moderate the effectiveness of the delivery of the JITAI on depression (see Figure 2). There was no evidence to support a moderation effect of week in the study, previous week’s daily step count, or daily minutes of sleep on treatment outcomes (see Figure S1 in Multimedia Appendix 1). However, we found that the JITAI’s efficacy was significantly moderated by the previous week’s HRQOL score of depression. Specifically, for those individuals receiving high-frequency dosage, high levels of depression in the previous week were associated with increased JTAI efficacy. For example, compared to those who received no messages, caregivers who received more than 5 messages per week on average had a 0.66-point reduction in depression score if their previous week’s depression score was 40, a 2.4-point reduction if their previous week’s score was 50, and a 4.4-point reduction if their previous week’s score was 60.

‎

Discussion

Overview

Through this MRT of mHealth intervention, we find that dosage of the messages, caregiver groups, and level of preintervention depression may impact the efficacy of our behavioral JITAI (ie, the delivery of the JITAI at low, medium, and high frequency can decrease levels of caregiver strain, anxiety, and depression) in caregivers of persons with significant health conditions. Specifically, our findings indicate that (1) higher dosage is associated with increased JITAI efficacy; (2) different caregiver groups respond differently to the JITAIs in terms of caregiver strain, anxiety, and depression; and (3) a high level of preintervention depression is associated with increased JITAI efficacy on depression. However, the observed impact is modest and subject to high uncertainty due to the limited sample size, necessitating future large-scale experiments for validation.

Principal Results

Our work provides preliminary evidence to support the idea that self-management–based JITAIs are potentially effective in improving caregiver HRQOL; those individuals in the intervention group demonstrated numerical improvements in caregiver strain, anxiety, and depression. However, due to the limited sample size, the significance of these findings is yet to be determined, which requires future large-scale trials to verify. Even so, this highly scalable intervention expresses the potential to improve caregiver outcomes with minimal burden and cost. Unlike traditional face-to-face interventions, which require time-intensive in-person meetings, an mHealth-based JITAI has the additional advantages of temporal and spatial flexibility, factors that are critically important for interventions targeting individuals that are already overwhelmed by their role as caregivers.

In addition, our research reveals a dosage-response relationship that shows that a higher dosage of JITAI messages is associated with greater improvement in caregiver strain, anxiety, and depression scores. This finding indicates that the dosage of JITAI messages is important and should be considered in future JITAI designs, given that a higher dosage appears to be associated with increased intervention efficacy. However, it is also possible that there may be a ceiling effect where excessive doses of JITAIs no longer improve the effectiveness of the intervention; in fact, it is also possible that excessive JITAI dosing may increase user burden and result in an “overexposure” effect that results in decreased JITAI effectiveness and increased attrition [15,30,53]. Therefore, identifying the optimal JITAI dosage is warranted. Moreover, we found that different caregiver groups respond differently to the JITAI intervention, making that another important consideration for future trial designs where the optimal dosage may differ for different subgroups. For example, we found that caregivers of people with HD appeared to benefit more from high-frequency messages compared to low and medium frequency; thus, a high-frequency dosage might be optimal and could maximize outcomes for HD-specific caregivers.

Our work also indicated that the previous week’s level of depression (ie, before intervention) can moderate the JITAI’s efficacy on depression. This finding also has implications for future JITAI design. Specifically, for those caregivers who experienced high levels of depression in the previous week (preintervention), the app can be optimized to send more frequent JITAI messages focused on mitigating depression in the upcoming weeks. On the contrary, the system can be optimized to minimize the number of JITAI messages sent in subsequent weeks for those individuals with lower levels of depression to reduce their burden. It is also possible that caregivers with high levels of depression benefited more from the JITAI because of a ceiling effect; in essence, those caregivers who were less depressed may have had less room for further improvement [54,55].

Taken together, the reported findings from this trial provide support for the use of JITAIs to promote HRQOL in caregivers of people with significant health conditions. In addition, our work also provides evidence and methodology to guide the future design of a personalized JITAI system.

Limitations and Future Research

First, our analysis is limited by a small sample size. Although there was a trend for significance for the effectiveness of the JITAI in these analyses, most findings did not meet traditional significance level cutoffs (ie, P>.05). Therefore, the replication of these findings in a large-scale trial is warranted to validate the conclusions from these analyses. Second, our study required daily self-reporting of strain, stress, and worry, which may have biased caregiver reporting. Third, we only recruited 3 caregiver groups (ie, HD, HCT, and SCI), which may not represent the whole population of caregivers of persons with significant health conditions. Moreover, the caregivers recruited in our study were mainly Caucasian females (22/36), which may not reflect the caregiver population in the country. Future studies should include more diverse caregiver populations.

Conclusions

This analysis provides preliminary evidence to support both the efficacy of our self-management-based JITAI as well as data to indicate that a greater number of messages each week have the potential to increase the efficacy of this intervention on caregiver HRQOL outcomes. Findings can also be used to inform the future design of a JITAI-based system that personalizes the intervention based on important variables that can impact JITAI efficacy (caregiver group, dosage of the JITAI messages, and preintervention depression level). Ultimately, we believe that highly scalable, inexpensive, low-touch self-management JITAIs have the potential to improve caregiver outcomes by providing a low-burden caregiver intervention that improves overall well-being.