The full MARS protocol for participant screening includes an initial in-person baseline evaluation comprising personal and family health/medical history, demographics, a physical and neuropsychological examination with a clinical assessor, questions about quality of life, daily living, physical activity, and measures of mood. The clinical and neuropsychological examination is repeated at yearly follow-up visits, as described in the following section.

MARS participants undergo an annual structured clinical evaluation, including a comprehensive neuropsychological battery of cognitive tests [30], such as the Mini-Mental State Examination (MMSE) [32], which is a commonly used standardized cognitive screening test with a maximum score of 30, where scores of 23 or lower indicate possible cognitive impairment. Clinical diagnosis of participant cognitive status is determined through a 3-step diagnosis process: computer scoring of cognitive tests, clinical review and judgment of data by a neuropsychologist blinded to participant demographics and previously collected data, and final diagnostic classification by a clinician upon review of all relevant data and examination of study participants. In this study, each participant’s cognitive status, either no cognitive impairment (NCI) or a diagnosis of MCI, was based on their final diagnostic classification from the annual study visit occurring closest to, and prior to, the onset of Illinois’ COVID-19 stay-at-home order. Participant demographic characteristics presented in this study include age (in years), sex, education (in years), and cognitive status and MMSE score. Age was also determined based on the study visit nearest to, and prior to, Illinois’ COVID-19 stay-at-home order.

In addition to the clinical evaluation, continuous data were collected using the ORCATECH-CART technology platform, which includes (1) weekly online surveys to determine changes in behavioral, social, and physical patterns; and (2) actigraphy watch data to determine changes in physical activity and sleep patterns. A more in-depth description of the ORCATECH-CART technology platform has previously been published [27,28].

Study participants are emailed weekly on Mondays with online surveys automatically via the Qualtrics Survey Platform (Qualtrics). If a survey is not completed, an automatic reminder email is sent on Wednesday, and if the participant does not respond after the reminder, follow-up phone calls are made to improve data capture. Completion of the full survey takes about 5 minutes, and participants can use any internet-capable device (eg, computer, tablet, or smartphone) to complete the survey [14,28,33,34]. The weekly survey includes 13 items (Table 1), which were designed to augment the behavioral data collected by the ORCATECH-CART platform with additional follow-up questions. Eleven of the 13 survey items require a yes/no response for a set of metrics regarding health, behavioral, and social activity patterns, such as away from home overnight, overnight visitors, blue mood, loneliness, medication changes, falls, accidents, hospitalizations, health limitations, living space change, and assistance change. The last 2 questions ask about pain levels and pain interference. Survey metadata are also collected, including submission date and timestamps, submission status (finished vs unfinished), and completion time. Data are stored on the Qualtrics servers and are accessed via application programming interface calls for data analysis.

For the collection of daily step count and sleep duration data, MARS participants were provided with and instructed to wear an actigraphy watch (Withings Activité; Withings) continuously, day and night, on their nondominant wrist. The Withings Activité was selected for its minimal technology demands. Its battery lasts approximately 8 months, eliminating the need for daily or weekly charging, and it enables passive data collection and monitoring without requiring users to press buttons or interact with the device. These low-effort features reduce user burden and are important for technology adoption among older adults [33,34].

Due to substantial differences in participant engagement across the data collection modalities and to ensure data quality and analytic validity, we defined 2 separate analytic subgroups for data analysis: 1 with available weekly survey data and 1 with available actigraphy watch data.

We examined weekly survey submission patterns among the MARS participants who were actively enrolled during the 32-week analysis window (November 30, 2019, to July 11, 2020) surrounding the onset of the Illinois COVID-19 stay-at-home order (March 21, 2020). Survey engagement was assessed across two 16-week periods pre– and post–COVID-19 based on weekly submission counts. The 16-week windows before and after the COVID-19 stay-at-home order were chosen to capture the acute impact of the Illinois stay-at-home order on participants’ survey engagement, behaviors, and wearable-derived activity, not to characterize long-term, seasonal, or chronic postpandemic trends. Each participant’s weekly survey submission status was classified into 1 of 3 categories: finished (when the participant began and submitted a survey), unfinished (when the participant began but did not submit a survey), or no survey (when the participant did not initiate a survey that week). The classification of unfinished vs no survey was used to distinguish partial engagement, as participants may have started but not submitted surveys for various reasons. In this study, this distinction was made for descriptive purposes and was not analyzed separately. However, prior research suggests that as individuals progress from NCI toward MCI, they may initially begin surveys but fail to complete them and, over time, stop initiating surveys, reflecting a gradual decline in engagement [35,36].

Participants were categorized by overall survey engagement. The “high-engagement” group was defined as submitting 50% or more of weekly surveys during the 32-week period, while the “low-engagement” group was defined as submitting fewer than 50%. The ≥50% engagement threshold was a prespecified pragmatic criterion, which is more conservative than completion cutoffs commonly used for ecological momentary assessment (EMA) study inclusion in similar populations [37]. The primary analyses of survey outcomes included all 59 participants under a missing-at-random assumption (see the Statistical Analysis and Missing Data subsections), while engagement status was retained as a sensitivity stratum. Survey start-hour time and survey completion-time analyses were conducted among high-engagement participants only, due to the necessity of sufficient submitted surveys per participant.

From each participant’s submitted weekly surveys, we extracted responses to the 11 binary (yes/no) questions. Response frequencies were summarized weekly across participants for the full analytic cohort. Pre– and post–COVID-19 periods were analyzed separately to examine changes in response trends over time. The 2 pain-rating items (pain intensity on a 1-5 scale and pain interference on a 0-10 scale) were collected as part of the weekly survey but were not analyzed in this study.

To explore potential shifts in daily engagement behavior before and after the COVID-19 stay-at-home order, we analyzed the timing of survey initiation. Changes in survey start-hour patterns may reflect alterations in participants’ daily routines or availability, offering indirect insight into lifestyle disruptions. For each submitted survey, we extracted start times (recorded in hour:minute format) and binned them by hour for an analysis over a 24-hour period. To assess patterns in survey initiation, we computed the distribution of start times and quantified central tendencies using circular statistics appropriate for cyclical time data. Specifically, we calculated the circular mean and SD of survey start times during the pre– and post–COVID-19 periods. The circular mean hour was used to represent the central tendency of engagement time within the 24-hour cycle, and peak start hours were identified as the most frequently occurring hourly bins. All analyses were conducted separately for the pre– and post–COVID-19 time windows.

Additionally, to explore potential changes in participant engagement with the yes/no questions before and after the COVID-19 stay-at-home order, we examined completion time specifically for the 13-item weekly questionnaire. Differences in completion time were examined descriptively, as changes may reflect a range of possible factors, such as cognitive load, emotional burden, shifts in focus and attention, familiarity with the survey interface, or altered daily routines, though the specific underlying causes could not be determined. For each submitted weekly survey, we extracted the total time required to complete the 13-item questionnaire, which was defined as the elapsed duration between the 13-item questionnaire start and finish timestamps. Weekly completion times were averaged across participants and summarized using means and SDs for both the pre– and post–COVID-19 periods. These metrics were computed at the group level. To assess whether completion times differed significantly between the pre– and post–COVID-19 periods, we first applied the Shapiro-Wilk test to evaluate the normality of the data distribution. Depending on the result, we used either the Welch t test (for normally distributed data) or the Mann-Whitney U test (for nonnormally distributed data) to compare group-level completion times pre– and post–COVID-19. Given the small sample size, we anticipated deviations from normality, and distributions were also graphically inspected.

We analyzed actigraphy data over a 10-week window (February 15, 2020, to April 15, 2020) comprising 5 weeks before and 5 weeks after the Illinois stay-at-home order. This shorter window, relative to the 32-week survey window, was necessitated by reduced data availability stemming from COVID-19 lockdown disruptions to in-home technical support and device shipping, along with individual variability in watch wear. Participants were included in the data analysis if they had at least 5 valid days before and after the stay-at-home order, and days with implausible values (sleep durations >15 hours or step counts <50 per day) were excluded as likely nonwear or device error, as these thresholds fall well outside the plausible physiological range for community-dwelling older adults (mean total sleep time approximately 7 hours, with long sleepers conventionally defined as those sleeping more than 9 hours [38]) and are more lenient than the ≥100-step “valid day” floor used in actigraphy validation work [39]. For all included participants, daily step counts and sleep durations were aggregated separately for the pre– and post–COVID-19 periods.

Baseline demographic and cognitive characteristics were summarized for the full analytic sample (N=59) and stratified by weekly survey engagement status (high- vs low-engagement). Across all analyses, continuous variables (eg, age, education, MMSE, daily step counts, and nightly sleep duration) were summarized using means and SDs, and categorical variables (eg, sex and cognitive diagnosis) were summarized using counts and percentages, computed at the participant level or aggregated across participants/weeks as appropriate. Between-group comparisons were conducted using Student t tests for continuous variables and chi-square tests for categorical variables.

The missingness mechanism for weekly survey responses was assessed prior to modeling. The Little test rejected the missing-completely-at-random hypothesis at a monthly aggregation (P=.002), and the test was not computable on the full 32-week response matrix owing to its high-dimensional sparsity, a known limitation of the Little test for intensive longitudinal data. Nonresponse was associated with baseline cognitive status (logistic regression odds ratio [OR] 0.18, 95% CI 0.03-0.89; P=.04; Fisher exact P=.03; 11/51, 22% of NCI vs 5/8, 63% of MCI participants were low-engagement), consistent with a missing-at-random (MAR) mechanism. We further examined whether COVID-19 onset precipitated complete survey cessation: exactly 1 continuously enrolled participant responded before but submitted no surveys after the stay-at-home order, and mean within-person response rates (each participant’s completed surveys out of 16 possible weeks per period, averaged across participants) were essentially unchanged (65.5% to 64.8%; 65.3% to 65.9% excluding that participant). Watch-data missingness was not formally assessed; its qualitative causes are described in the Actigraphy Watch Data Analysis subsection and the Limitations.

Generalized linear mixed models (GLMMs) with a logit link, a participant-level random intercept, and a binary pre– vs post–COVID-19 indicator (Illinois stay-at-home order, March 21, 2020) as the primary predictor, adjusting for age, sex, and education, were used to examine changes in each of the 11 binary survey outcomes. The primary analysis used the full analytic sample (N=59) under an MAR assumption, with high-engagement (n=43) and low-engagement (n=16) strata reported in parallel as sensitivity analyses. Within-participant correlation among repeated weekly observations was captured by the random-effects structure: weekly responses showed modest within-participant autocorrelation (mean lag-1 r=0.10) and substantial between-person variance (intraclass correlation coefficient–like estimate=0.39), both accommodated by the random intercept. Results are reported as ORs with 95% CrIs.

For continuous outcomes (daily step count and nightly sleep duration), linear mixed models with a participant-level random intercept were used to evaluate pre– vs post–COVID-19 changes, adjusting for age, sex, and education. Models were estimated using the PROC MIXED procedure in SAS (version 9.4; SAS Institute [40]). Results were expressed as coefficient estimates and SEs.

To assess robustness of the survey findings under MAR, we conducted 3 sensitivity analyses: multiple imputation (20 imputations, Rubin rules), a single-case analysis excluding the 1 post–COVID-19 nonresponder, and an engagement-stratified analysis comparing the high-engagement (n=43) and low-engagement (n=16) groups separately.

Data analysis and visualizations were conducted using Python (version 3.9), and PROC MIXED and GLIMMIX models were performed using SAS software, with statistical significance evaluated at an α level of .05 for frequentist analyses and 95% CrIs excluding 1 for Bayesian estimates. Additional descriptive and statistical results are presented in Tables S1-S6 in Multimedia Appendix 1. In addition, software concordance checks comparing Python GLMM and SAS GLIMMIX implementations on the high-engagement subsample (n=43) are presented in Tables S3 and S4 in Multimedia Appendix 1.

A total of 59 MARS participants were active during the study period (Table 2). Of these, 43 participants (73%) were classified as the high-engagement group, defined as completing ≥50% of weekly surveys, and 16 participants (27%) were classified as the low-engagement group (<50% completion). Participants in the low-engagement group were significantly more likely to have been diagnosed with MCI (5/16, 31% vs 3/43, 7%; P=.03) and had lower mean MMSE scores (28.1, SD 1.4 vs 28.9, SD 1.0; P=.04) compared with participants in the high-engagement group. No significant differences were found between engagement groups with respect to age, sex, or years of education.

During the 32-week analysis window, 622 surveys were completed in the pre–COVID-19 period (mean 38.88, SD 3.44 per week) and 613 surveys were completed post–COVID-19 (mean 38.31, SD 4.77 per week), reflecting consistent overall participation across time periods. When stratified by engagement status, the high-engagement group maintained stable engagement with 555 surveys completed pre–COVID-19 (mean 34.69, SD 2.89) and 566 surveys completed post–COVID-19 (mean 35.38, SD 4.53). Conversely, the low-engagement group demonstrated a decline in participation, completing 67 surveys before (mean 4.19, SD 1.87) and 47 surveys after COVID-19 onset (mean 2.94, SD 1.48), highlighting a reduction in weekly engagement during the pandemic period. Total and mean weekly submission counts before and after applying the ≥50% completion rate threshold are presented in Tables S1 and S2 in Multimedia Appendix 1.

Figures 1 and 2 visualize these trends, with Figure 1 presenting a heatmap of individual-level weekly survey submissions over the 32-week window, with submission status color-coded as finished (gray), unfinished (yellow), and no survey (white), and participants grouped by engagement classification (high-engagement, n=43; low-engagement, n=16). Figure 2 displays stacked bar charts of mean (SD) weekly survey submissions stratified by engagement group and time period. Each bar depicts the proportions of finished, unfinished, and no survey responses, highlighting the overall scarcity of participation among low-engagement individuals.

Table 3 presents GLMM estimates of the odds of a “yes” response pre–COVID-19 vs post–COVID-19 for all 11 binary survey items, reported for the full analytic sample (N=59) and, in parallel, as sensitivity analyses, for the high-engagement (n=43) and low-engagement (n=16) subsamples. In the full sample, participants were significantly less likely after the stay-at-home order to report being away from home overnight (OR 0.30, 95% CrI 0.20-0.46), having overnight visitors (OR 0.44, 95% CrI 0.31-0.64), a medication change (OR 0.60, 95% CrI 0.42-0.86), or a health-related activity limitation (OR 0.70, 95% CrI 0.52-0.95). Reports of changes to the living space increased modestly (OR 1.41, 95% CrI 1.02-1.95). These 4 reductions were consistent in direction and magnitude in the high-engagement subsample. Estimates in the low-engagement subset (n=16) were unstable, being based on a few outcome-positive participants, and those that were nominally significant were attributable to a small number of repeat responders rather than group-level change. This instability is consistent with the missing-data assessment and is why the full-sample model, valid under an MAR assumption, is the primary analysis. Emotional well-being indicators (blue mood and loneliness) showed no significant change in the full sample. Estimates for rare events (accidents and new assistance needs) were unstable owing to very few responses and are reported descriptively only, and accidents were reported only during the pre–COVID-19 period. Age, sex, and education were included as covariates and were not the focus of inference.

Participants were significantly less likely to report being away from home overnight (OR 0.29, 95% CI 0.17-0.50; P<.001) or having overnight visitors (OR 0.52, 95% CI 0.32-0.86; P=.01) in the post–COVID-19 period compared with the pre–COVID-19 period. The covariates age at baseline, sex, and education were not significant predictors in any of the models. While pre–COVID-19 and post–COVID-19 differences were not statistically significant for other self-reported experiences (Table S3 in Multimedia Appendix 1), descriptive trends (Table S4 in Multimedia Appendix 1) revealed decreases in the group-level prevalence of medication changes, falls, and hospitalizations following the onset of COVID-19. Reports of accidents were rare and occurred only during the pre–COVID-19 period. Physical health limitations remained stable, while small increases were noted in changes to living spaces. Assistance needs remained minimal across both time periods. Table 3 presents GLMM estimates of the odds of a “yes” response pre–COVID-19 vs post–COVID-19 for all 11 binary survey items, reported for the full analytic sample (N=59) and, in parallel, as sensitivity analyses, for the high-engagement (n=43) and low-engagement (n=16) subsamples.

Figure 3A displays the distribution of weekly survey start times over a 24-hour period for the 43 high-engagement participants, comparing the pre–COVID-19 and post–COVID-19 periods. Start times were summarized using 2 key metrics: the peak hour, representing the most frequently observed submission time, and the circular mean hour, capturing the central tendency of engagement within a cyclical 24-hour framework. In the pre–COVID-19 period, the peak submission hour was 16:00, shifting to 18:00 post–COVID-19, indicating a modest delay in preferred engagement time. The circular mean submission time shifted earlier by approximately 2 hours, from 20:48 pre–COVID-19 to 18:42 post–COVID-19 (Figure 3B), reflecting a broader trend toward earlier survey completion. The difference between the circular mean and the peak hour, used as an indicator of distributional spread, decreased from approximately 5 hours pre–COVID-19 to less than 1 hour post–COVID-19, suggesting a tighter clustering of engagement patterns over time.

Figure 4 displays the group-level mean (SD) completion times for the 13-item weekly survey, comparing pre–COVID-19 and post–COVID-19 periods among the 43 high-engagement participants. The average completion time increased from 1.84 (SD 0.44) minutes pre–COVID-19 to 2.08 (SD 0.80) minutes post–COVID-19. This difference was not statistically significant (P=.81), with a negligible effect size (r=0.04), indicating that the overall time required to complete the survey remained stable across the 2 periods.