The results presented here comprise a subanalysis of data from the JITAI-arm of a larger behavioral randomized controlled trial feasibility and acceptability trial published elsewhere [27]. Of the initial 72 caregivers that were enrolled in this study, a total of 36 caregivers (n=11 HD, n=10 SCI, and n=15 HCT) were randomized to receive the intervention and are the focus of these analyses. Although a detailed description of this sample is reported in the acceptability and feasibility paper that is under review (revision submitted), a brief summary of the sample is provided. Specifically, eligible caregivers had to be at least 18 years old, be able to read and understand English, and be caring for an individual 18 years or older with medically documented HD, SCI, or HCT. Caregivers for individuals with SCI also needed to be caring for someone who was ≥1-year post injury, and caregivers of people with HCT had to be caring for an individual who was receiving, had received, or was scheduled to receive HCT. Professional and paid caregivers were excluded from this study.

Briefly speaking, participation in this behavioral trial was fully remote and consisted of a 2-hour baseline session followed by a 3-month (90-day) home monitoring period. The completion of informed consent, as well as several self-reported measures and instructions for the home monitoring period, were completed during the baseline session, which was conducted via Zoom. The home monitoring period included continuous monitoring (ie, step count, sleep minutes, and heart rate) using a wearable device (ie, Fitbit) and the collection of real-time self-reported ratings of HRQOL measures via the CareQOL study app, which occurred once per day. This feasibility study followed a 2-arm randomized controlled design. For each day during the home monitoring period, those caregivers in the JITAI arm had a 50-50 chance to receive personalized JITAI messages derived from sensor data (eg, accelerometer-based estimates of physical activity and sleep duration) and the daily self-reported ratings of HRQOL (caregiver strain, anxiety, and depression) via the study app, while the control arm did not receive any JITAI messages. The content and type of the messages were randomly drawn from a pool of over 400 messages; message types included data feedback, facts, tips, and support.

The main purpose of the feasibility and acceptability trial was to establish the feasibility and acceptability of an intensive data collection protocol to inform a future larger effectiveness trial of this self-management JITAI on caregivers of people with chronic medical conditions. Given that there are no formal power analyses for this type of trial, we based our sample size (N=72) on other feasibility and acceptability trials that have been published in the literature for mHealth apps [31-33]. For the purposes of these analyses, we examined the 36 caregivers that were assigned to the JITAI arm.

The primary outcomes of this study were the daily HRQOL measures of caregiver strain, anxiety, and depression, which were assessed by a single daily question that was drawn from each respective item bank (described below). The CareQOL measures were initially developed and validated in caregivers of people with traumatic brain injury [34-37], but have also been validated for use in caregivers of SCI, HD, and cancer [38]. The CareQOL caregiver strain item bank [35] was used to assess feelings of being overwhelmed or burdened by the caregiver role; the PROMIS (Patient-Reported Outcomes Measurement Information System) Anxiety item bank [39,40] was used to provide a measure of the level of anxiety (eg, fear, anxious misery, hyperarousal, and somatic symptoms related to arousal); and the PROMIS Depression item bank [39,40] provided a measure of perceived depression (ie, negative mood, decrease in positive affect, information-processing deficits, negative views of the self, and negative social cognition). Responses to each of the 3 daily questions were on a 5-point scale, with higher scores corresponding to higher levels of the named construct. Resulting scores were on a T metric (mean 50, SD 10) relative to a reference population (either other caregivers, ie, for caregiver strain, or the US general population for PROMIS Anxiety and PROMIS Depression) [41]. The daily real-time assessments of 3 variables were administered as a computer adaptive test (CAT) [42], which aims to estimate an examinee’s level of the construct (eg, depression) through sequentially administered items (questions), where each item is selected adaptively based on the examinee’s previous response. As a consequence, the estimate of the examinee’s level of the construct typically becomes more precise as more items are administered. A CAT event is composed of an initialization of the test at the beginning and a termination criterion (eg, fixed time limit, certain precision achieved) that ends the test. In our study, the CAT event was administered on a weekly basis and restarted each Monday, with a single item administered each day. Given this, analyses were conducted on a weekly level; JITAI messages were considered over the course of a week, as were HRQOL ratings (ie, CAT events). The final HRQOL scores for each CAT event (eg, a score on Saturday would be treated as the final score of the week if a score on Sunday was not provided and a score on Saturday was assessed) were outcomes of our interest in the analysis. CAT events with fewer than 3 daily responses over the course of a week were considered invalid or unreliable and were removed from the analyses. Daily step count and sleep minutes were recorded through wearable devices, and weekly individual step count and sleep minutes were calculated by taking the average of the daily measures.

The procedures of this feasibility trial were approved by IRBMED (Application Approval HUM00184455), and the trial was registered with ClinicalTrials.gov (NCT04556591). Caregivers provided informed consent prior to the commencement of study activities.

A total of 36 participants were included in our analysis, with 11 from the HD group, 10 from the SCI group, and 15 from the HCT group. The demographic table is shown in Table 1. In general, the 3 caregiver groups did not differ on demographic variables, with the exception that caregivers of people with HCT reported significantly fewer years (approximately 9) in the caregiving role than the other 2 subgroups. Of the total 36 participants in the study, 33 used the Fitbit Inspire 2 (provided by the study team), while the remaining 3 participants used their own personal Fitbit devices (n=1 Fitbit Versa2, n=1 Fitbit Inspire HR, and n=1 Fitbit Charge 4). Missing data occurred throughout the study. The average missing rate was 9.7% (561/5796 person-days) for the daily HRQOL surveys (caregiver strain, anxiety, and depression), 2.8% (163/5796 person-days) for daily steps, and 14.2% (822/5796 person-days) for daily sleep (from wearing the Fitbit). The participants’ average daily step count was 7802.9, and the average daily sleep duration was 410.6 minutes.

The primary analysis indicated that the delivery of JITAIs tended to decrease (ie, improve) caregiver strain, anxiety, and depression scores. In addition, the efficacy of the JITAIs tended to increase as the dosage of push notifications increased (ie, higher frequency; see Table 2). More specifically, caregivers who received 1 or 2 messages within a week, on average, would have around a 1-point reduction in caregiver strain, a 1-point reduction in anxiety, and a 1-point reduction in depression in terms of the T score (mean 50, SD 10), compared to those who did not receive any messages. Caregivers with 3 or 4 messages received per week would, on average, have a corresponding 4-point decrease in caregiver strain, a 4-point decrease in anxiety, and a 2-point decrease in depression (ie, improvement) in their T score relative to caregivers that did not receive any messages that week. For those who received more than 4 messages, there were score improvements of around 6 points for caregiver strain, 5 points for anxiety, and 4 points for depression. Taken together, there is preliminary evidence to support the efficacy of the JITAI, and a higher dosage appears to be associated with greater improvements. Although most of the coefficients did not meet traditional cutoffs (ie, P<.05; only the coefficient of high-frequency JITAI on the score of caregiver strain had P=.046), these analyses were underpowered (given the small sample sizes), and thus, the fact that we saw consistent improvements in HRQOL scores indicates that this is a promising approach for improving HRQOL in this population.

Our secondary analysis indicated that the 3 caregiver groups tended to respond differently to the JITAI messages (Figure 1 and Table 3). Table 3 shows the estimated effect size (improvement in T score) in log, percentage, and original scale for the HCT, HD, and SCI groups. For example, caregivers in the SCI group who received 3 or 4 messages per week had, on average, a 0.17 (95% CI –0.33 to –0.00) decrease in T score in the log scale, a 16% (95% CI 28%-0%) decrease in percentage, and an 8.05 (95% CI 0.19-14.67) decrease in the original scale when compared to those in the SCI group who did not receive any messages. In general, for all 3 caregiver groups, there was a nonsignificant trend (as indicated by a wide CI), which may indicate that the improvements in HRQOL may increase as the dosage of the JITAI (ie, frequency of push notifications) increases. However, the trajectory of the efficacy of these treatment effects was different among the 3 groups. Specifically, for the HD subgroup, caregivers with high-frequency prompts benefited more from the delivery of JITAI messages, compared to a similar treatment effect for those with low- and medium-frequency prompts. For the SCI group, the efficacy of treatment effects caregivers received did not seem to be affected by how frequently the JITAI messages were sent. Finally, for the HCT group, the estimated treatment effects did not seem to differ by dosage (ie, prompt frequency), and the effect sizes of different treatment dosages were consistently smaller relative to the SCI group.

The exploratory analysis indicated that the previous week’s HRQOL score of depression appears to moderate the effectiveness of the delivery of the JITAI on depression (see Figure 2). There was no evidence to support a moderation effect of week in the study, previous week’s daily step count, or daily minutes of sleep on treatment outcomes (see Figure S1 in Multimedia Appendix 1). However, we found that the JITAI’s efficacy was significantly moderated by the previous week’s HRQOL score of depression. Specifically, for those individuals receiving high-frequency dosage, high levels of depression in the previous week were associated with increased JTAI efficacy. For example, compared to those who received no messages, caregivers who received more than 5 messages per week on average had a 0.66-point reduction in depression score if their previous week’s depression score was 40, a 2.4-point reduction if their previous week’s score was 50, and a 4.4-point reduction if their previous week’s score was 60.

Through this MRT of mHealth intervention, we find that dosage of the messages, caregiver groups, and level of preintervention depression may impact the efficacy of our behavioral JITAI (ie, the delivery of the JITAI at low, medium, and high frequency can decrease levels of caregiver strain, anxiety, and depression) in caregivers of persons with significant health conditions. Specifically, our findings indicate that (1) higher dosage is associated with increased JITAI efficacy; (2) different caregiver groups respond differently to the JITAIs in terms of caregiver strain, anxiety, and depression; and (3) a high level of preintervention depression is associated with increased JITAI efficacy on depression. However, the observed impact is modest and subject to high uncertainty due to the limited sample size, necessitating future large-scale experiments for validation.

Our work provides preliminary evidence to support the idea that self-management–based JITAIs are potentially effective in improving caregiver HRQOL; those individuals in the intervention group demonstrated numerical improvements in caregiver strain, anxiety, and depression. However, due to the limited sample size, the significance of these findings is yet to be determined, which requires future large-scale trials to verify. Even so, this highly scalable intervention expresses the potential to improve caregiver outcomes with minimal burden and cost. Unlike traditional face-to-face interventions, which require time-intensive in-person meetings, an mHealth-based JITAI has the additional advantages of temporal and spatial flexibility, factors that are critically important for interventions targeting individuals that are already overwhelmed by their role as caregivers.

In addition, our research reveals a dosage-response relationship that shows that a higher dosage of JITAI messages is associated with greater improvement in caregiver strain, anxiety, and depression scores. This finding indicates that the dosage of JITAI messages is important and should be considered in future JITAI designs, given that a higher dosage appears to be associated with increased intervention efficacy. However, it is also possible that there may be a ceiling effect where excessive doses of JITAIs no longer improve the effectiveness of the intervention; in fact, it is also possible that excessive JITAI dosing may increase user burden and result in an “overexposure” effect that results in decreased JITAI effectiveness and increased attrition [15,30,53]. Therefore, identifying the optimal JITAI dosage is warranted. Moreover, we found that different caregiver groups respond differently to the JITAI intervention, making that another important consideration for future trial designs where the optimal dosage may differ for different subgroups. For example, we found that caregivers of people with HD appeared to benefit more from high-frequency messages compared to low and medium frequency; thus, a high-frequency dosage might be optimal and could maximize outcomes for HD-specific caregivers.

Our work also indicated that the previous week’s level of depression (ie, before intervention) can moderate the JITAI’s efficacy on depression. This finding also has implications for future JITAI design. Specifically, for those caregivers who experienced high levels of depression in the previous week (preintervention), the app can be optimized to send more frequent JITAI messages focused on mitigating depression in the upcoming weeks. On the contrary, the system can be optimized to minimize the number of JITAI messages sent in subsequent weeks for those individuals with lower levels of depression to reduce their burden. It is also possible that caregivers with high levels of depression benefited more from the JITAI because of a ceiling effect; in essence, those caregivers who were less depressed may have had less room for further improvement [54,55].

Taken together, the reported findings from this trial provide support for the use of JITAIs to promote HRQOL in caregivers of people with significant health conditions. In addition, our work also provides evidence and methodology to guide the future design of a personalized JITAI system.

First, our analysis is limited by a small sample size. Although there was a trend for significance for the effectiveness of the JITAI in these analyses, most findings did not meet traditional significance level cutoffs (ie, P>.05). Therefore, the replication of these findings in a large-scale trial is warranted to validate the conclusions from these analyses. Second, our study required daily self-reporting of strain, stress, and worry, which may have biased caregiver reporting. Third, we only recruited 3 caregiver groups (ie, HD, HCT, and SCI), which may not represent the whole population of caregivers of persons with significant health conditions. Moreover, the caregivers recruited in our study were mainly Caucasian females (22/36), which may not reflect the caregiver population in the country. Future studies should include more diverse caregiver populations.

This analysis provides preliminary evidence to support both the efficacy of our self-management-based JITAI as well as data to indicate that a greater number of messages each week have the potential to increase the efficacy of this intervention on caregiver HRQOL outcomes. Findings can also be used to inform the future design of a JITAI-based system that personalizes the intervention based on important variables that can impact JITAI efficacy (caregiver group, dosage of the JITAI messages, and preintervention depression level). Ultimately, we believe that highly scalable, inexpensive, low-touch self-management JITAIs have the potential to improve caregiver outcomes by providing a low-burden caregiver intervention that improves overall well-being.