The key features analyzed for each of the fine motor, speech, and gross motor tasks are presented in Table 1. The following digital features collected from the smartphone tasks were selected a priori for use in all discriminatory and longitudinal analyses, including dominant and nondominant hand total taps from the alternating finger-tapping task, dominant and nondominant hand adjusted scores from the digital pegboard task, articulation rate from the syllable repetition task, speaking rate from the passage reading task, gait speed from the walking task, and total sway area from the static stance balance task.

For the syllable repetition task, acoustic signals were processed using a method adapted from Rong [55]. A Hilbert transform was used to extract acoustic envelopes of the audio signal. Peaks in the envelope were identified with peak detection and corresponded to individual syllables. The acoustic signals were segmented into voiced and unvoiced periods using a voice activity detection algorithm [56]. Articulation rate was calculated by dividing syllables identified during voiced periods by 15 seconds.

For the Bamboo Passage, speaking rate was estimated using an automated transcription approach (AWS_transcribe). Audio files were transcribed using AWS Transcribe (Amazon Web Services), and the transcription was scored against the Bamboo Passage target text using Levenshtein distance edit operations [57] that identify correctly spoken words from the text, missing words, substitutions, and insertions. The timing of the first and last correctly spoken words or substitutions was identified as the start and end of the passage to avoid initial and final silences. After subtracting the duration of any insertions and immediately following pauses, speaking rate (words per minute) was defined as the passage word length (number of words) divided by the passage duration. The quality of automated transcription for estimating speaking rate was evaluated by comparing the results to those obtained from manually transcribed files (n=60), where we found excellent agreement (intraclass correlation [ICC]>0.97).

For the walking task, participants were allowed to use an assistive device, and they indicated if one was used for the task or not. For each participant, we marked each session as “no assistive device” or “used assistive device.” We found that some participants used devices variably across sessions. This small, exploratory study was designed to evaluate whether accelerometry could be used to quantify gait in individuals using and not using assistive devices. The study was not powered to assess intraparticipant variability associated with intermittent use of gait assistive devices over the 24 weeks. Including mixed sessions (some with and without device use) would have inflated within-participant variability, biased longitudinal estimates of gait speed, and undermined the interpretability of reliability metrics, given the size of this study. Therefore, gait speed analyses were restricted to the condition most frequently used by each participant across the study period. If a participant used an assistive device for most sessions, only sessions completed with an assistive device were included in the gait speed analysis, and sessions completed without an assistive device were excluded, or vice versa. Total sway area was observed to be strongly right-skewed with some extreme outliers. To diminish the effect of these very large values and better satisfy normality and other modeling assumptions, sway area observations were capped at 0.5 cm², followed by a Box-Cox transformation with λ=0.2.

The feature extraction and analysis for the mouse clicking task have been described in detail previously [50,51]. In short, we could derive 32 features from the movement trajectories of the mouse during this task. These features are representative of the separate components of the task, including initiation time (from target onset to first mouse movement), execution time (from first to last mouse movement), verification time (time spent on the target once the mouse has stopped to the start of the click), and click time (time from mouse click down to mouse click up). The first trial of each block was used to position the mouse and was excluded from the analyses. Pauses were marked whenever there was a break of 100 ms or more for raw mouse movement. The results for each participant were analyzed as age-specific z-scores (compared with a normative dataset collected previously from over 200,000 healthy volunteers), separately for each of the 32 features. They were calculated separately for each block of trials and later averaged across the blocks.

To assess reliability and measurement sensitivity in the digital features, we computed ICC1 and minimum detectable change (MDC) for each feature. Separate estimates were computed for participants with ALS and HC and used the entire longitudinal dataset (rather than a single pair of measurements) to enable greater estimate precision. For each digital measure, a one-way linear random effects model ignoring time with a subject-specific random intercept was used for each group to estimate between- and within-subject variance (σs2 and σe2, respectively). These quantities were then used to calculate ICC1=σs2σs2+σe2 and MDC95=1.96∗σe∗2. We additionally computed 95% CIs for ICC1 using the parametric bootstrap. MDC was calculated at the group level using the pooled within-subject variance across participants. This approach yields a single MDC value that represents the minimum detectable change for the measure in this sample, rather than individual-specific MDC estimates.

Because the entire longitudinal dataset was used for computing ICC and MDC quantities, we had concerns about temporal trends unrelated to the measurement properties in question (eg, disease-related decline or learning effects) distorting the estimates. In particular, macro-longitudinal trends could inflate the within-subject variance and downwardly bias ICC. To account for this possibility, we estimated ICC and MDC on “detrended” datasets. These datasets were generated by (1) subtracting the fitted participant-specific generalized additive mixed models (GAMM)–estimated mean trajectories from the raw data values (described below in the longitudinal analysis), followed by (2) adding back in the subject-specific intercept, captured as their baseline performance. This process has the effect of removing high-level temporal trends while retaining both the within- and between-subject variability of interest. An example of this detrending process can be seen in Figure S3 in Multimedia Appendix 1. We observed that, as expected, the detrending process resulted in modest increases in estimated ICCs relative to using the raw data. CIs for ICC did not account for uncertainty in the detrending process, so they may be slightly anticonservative.

Baseline performance on the 8 features chosen a priori across the speech, fine, and gross motor tasks was summarized using means and SDs for participants with ALS and HC. Differences in means and corresponding 95% CIs were calculated for each feature, and unadjusted P values were obtained using independent-samples t tests, assuming unequal variances where appropriate. All tests were 2-sided, and statistical significance was evaluated at an alpha level of .05. No adjustment was made for multiple comparisons.

Logistic regression and random forest models were used to predict group (people with ALS vs HC) using the 8 features chosen a priori across the speech, fine, and gross motor tasks. Due to small sample size, tuning parameters in the random forests were fixed a priori (number of trees=300, max depth=3, variables per node=3). Repeated 10-fold cross-validation—stratified by ALS/HC status to ensure class balance in the folds—was used to evaluate model performance. To understand sensitivity of model performance to measurement variability, AUC estimates with 95% CIs were computed using first (baseline) observation only, second only, third only, and fourth only, along with the averages across the first 2, first 3, and first 4 observations.

To assess the concurrent validity of the digital measures, Spearman correlations at baseline were calculated between digital features and each subdomain of the ALSFRS-R, including bulbar (questions 1‐3), fine motor (questions 4‐6), gross motor (questions 7‐9), and respiratory (questions 10‐12). For these, 95% CIs were computed using the nonparametric bootstrap.

Group-specific mean curves were estimated as smooth functions of time using GAMMs, which were preferred over linear mixed models because they allowed for nonlinearity in the longitudinal trajectories. Random effect smooths allow each individual’s estimate to deviate from the mean curves, while simultaneously shrinking individual curves toward the mean curves, similar to traditional random effects. GAMMs were implemented using the gam() function in the mgcv R package [58]. Thin plate regression splines (option bs=“tp”) were used to fit the group-specific smooth fixed effects, factor-smooth interactions with a common smoothness parameter were used for the random effect smooths (option bs=“fs”), and restricted maximum likelihood was used for estimation of the smoothing parameters.

For each digital feature, the emmeans R package [59] was used to compute contrasts of interest and 95% CIs from the fitted GAMMs, including the mean change from baseline to the end of the study in people with ALS and HC group and the mean between-group difference at baseline and at the end of the study. Statistical tests derived from the GAMMs assess whether longitudinal changes differed between patients with ALS and HCs.

For the smartphone app tasks, the longitudinal analysis focused on the same 8 features evaluated at baseline and in the discriminative models. For the computer mouse clicking task, we present results from 8 representative features selected following visual inspection of the spline models for each of the 32 features. Trajectories over time varied substantially among the full feature set. Given the exploratory nature of this analysis and recognizing that features with high short-term repeatability are not necessarily those most sensitive to disease-related change, the selected features illustrate the diversity of longitudinal feature behavior.

Over the 24-week study period, 10,237 digital assessments were performed across all participants (active data collection), and 6801 hours of continuous sensor data were recorded from participant watches (passive data collection). Participants successfully completed a total of 872 (85.07%) of a possible 1025 assessment batteries within the study smartphone app. Completion rates were comparably high between those with ALS and HC (84.4% and 85.6%, respectively). Adherence data for many of the smartphone tasks were presented in Figures 2A-2B. Notably, participants with ALS tended to skip more of the gross motor tasks compared with HC and relative to other tasks, primarily due to task difficulty. Speech tasks presented challenges in data quality, resulting in a higher rate of invalid or unusable data, primarily due to very low speech volume among some participants or rapid task completion among control participants, which limited the ability of the automated approach to distinguish individual syllables.

Participants completed 459 computer mouse clicking sessions. Among the 19 participants with ALS, 14 completed at least one session, averaging 15 sessions (1‐23 range) per participant. Of the 23 HCs enrolled, 18 completed at least one session, averaging 14 sessions (1‐25 range) each.

All participants completed each of the 3 telephone visits with study staff.

The test-retest reliability, presented by the ICCs for each of the features evaluated, is presented in Table 3. For the smartphone fine motor features, total alternating finger taps from dominant and nondominant hands demonstrated the highest ICCs in both participants with ALS (0.95, 95% CI 0.90‐0.97 and 0.96, 95% CI 0.92‐0.98, respectively) and HCs (0.94, 95% CI 0.89‐0.96 and 0.92, 95% CI 0.85‐0.96, respectively). Among the speech features, speaking rate had the highest ICC in participants with ALS (0.97, 95% CI 0.93‐0.98). Syllable articulation rate also demonstrated high ICC in participants with ALS (0.91, 95% CI 0.82‐0.95) and was the highest ICC for the HC group (0.86, 95% CI 0.76‐0.92). Gait speed showed high reliability for the ALS group (0.93, 95% CI 0.85‐0.96) and moderate reliability for HCs (0.68, 95% CI 0.50‐0.78). Notably, many participants with ALS were either unable to complete the walking task or changed the assistive devices they used during the task throughout the study duration, affecting the reliability of this measure.

The minimal detectable change at 95% confidence level (%MDC95) was calculated to identify the smallest detectable change in each feature for each task that exceeded the threshold of measurement error with 95% confidence, representing the minimal change over time in an individual that can be confidently interpreted as a true change rather than measurement variability (Table 3). As expected, the %MDC95 values for most features were similar between people with ALS and HCs, except for the speaking rate and gait speed. For speaking rate, the %MDC95 was 22.1 words per minute for participants with ALS and 31.3 words per minute for HCs, indicating that a greater change in words per minute is required in HCs to be considered a true change beyond measurement error. A similar pattern was observed for gait speed, where a change of 0.18 meters per second was required for people with ALS, compared with 0.28 meters per second for HCs to exceed the MDC threshold.

For the computer mouse clicking task, ICCs across the 32 features varied greatly, ranging from 0.38 to 0.96 in the ALS group and 0.35‐0.78 in HCs, as outlined in Table S1 in Multimedia Appendix 1. The most reliable features for the participants with ALS were movement time (0.96, 95% CI 0.92‐0.98), normalized jerk with pauses (0.94, 95% CI 0.86‐0.97) and without pauses (0.94, 95% CI 0.86‐0.97), execution time with pauses (0.94, 95% CI 0.84‐0.97) and without pauses (0.93, 95% CI 0.83‐0.96), and verification time (0.91, 95% CI 0.81‐0.96). For HCs, the ICCs were lower than those observed in the ALS group, with the highest performing features being movement time (0.76, 95% CI 0.58‐0.85), maximum acceleration (0.70, 95% CI 0.46‐0.81), and verification time (0.78, 95% CI 0.60‐0.87). Additionally, %MDC95 was computed for each of the features and is presented in Table S1 in Multimedia Appendix 1.

Baseline performances on each of the smartphone task features were compared between participants with ALS and HC (Figures 3A-3H and Table S2 in Multimedia Appendix 1). Participants with ALS consistently demonstrated lower average performance compared with HCs across multiple domains. Specifically, participants with ALS completed fewer alternating finger taps, placed fewer pegs during the pegboard task, had slower speaking and articulation rates, slower gait speed, and increased sway area on the balance task compared with the HC group, with all these findings except for sway area and articulation rate reaching statistical significance.

AUC estimates predicting ALS versus HC status using 8 features chosen a priori—dominant and nondominant hand total taps on alternating finger tapping and adjusted scores on digital pegboard, syllable repetition articulation rate, speaking rate, gait speed, and total sway area—show high predictive ability with AUC estimates ranging from 0.75 to 0.92 (Figure 4). Model performance is shown separately using these features at baseline, week 1, week 2, week 3, and the average of the first 4 observations. There is considerable variability seen in the predictive performance depending on which observation or sets of observations were used.

Features from each smartphone task generally showed the strongest Spearman correlations with their corresponding ALSFRS-RSE subdomains (Table 3).

Of the smartphone upper extremity features, dominant-hand alternating finger taps demonstrated the strongest correlation with the fine motor subdomain (questions 4‐6) of the ALSFRS-RSE (0.74, 95% CI 0.41‐0.93). Among the speech features, syllable articulation rate was found to have the highest correlation with the bulbar subdomain (questions 1‐3; 0.51, 95% CI 0.02‐0.81). Gait speed was highly correlated with the gross motor subdomain (questions 7‐9; 0.83, 95% CI 0.61-0.95). Representative scatterplots are provided in Figure S1 in Multimedia Appendix 1.

In the smartphone tasks, both participants with ALS and HC demonstrated improvement over time in the fine motor and speech tasks (Figure 5A). The exception was the nondominant hand digital pegboard mean adjusted score, where participants with ALS showed a decline in function at the end of the study (Figure 5A). Notably, the HC group improved more, widening the performance separation by the end of the study (Figure 5B), perhaps due to ALS blunting the learning effect. This increased mean group difference was most pronounced for the adjusted pegboard scores and speaking rate but did not reach significance in this small sample size.

In gross motor tasks, participants with ALS consistently performed worse than HC, and both groups showed a decline in performance (Figure 5A and B). The difference in sway area widened, as it did for other features. The difference in gait speed between the 2 groups slightly narrowed by the end of the study. Performance variability was observed in both ALS and HC participants throughout the study. This was evident at both the individual participant level and in the aggregated group data (Figure 5C).

We selected 8 features from the computer mouse clicking task by choosing representative patterns from the spline models for each of the 32 features (Figure S2 in Multimedia Appendix 1). These 8 features are (1) noise-to-force ratio per block (SD of the distance from the target center at the end of the first submovement divided by the mean maximum acceleration), (2) distance from the target after the main submovement, (3) target reentries (number of times the cursor leaves and re-enters the target before clicking), (4) movement offset (mean distance of the cursor from the task axis), (5) movement error (mean absolute distance of the cursor from the task axis), (6) normalized jerk (movement smoothness adjusted for movement duration and amplitude), (7) execution time excluding pauses, and (8) number of pauses, representing the different patterns seen across all features. Overall, participants with ALS exhibited a decline in performance in 5 of these 8 features (noise-to-force ratio per block, distance from target after main submovement, target reentries, movement offset, and movement error), with improvement in the other 3 (normalized jerk, execution time without pauses, and number of pauses) over study duration (Figure 6A). In contrast, HCs showed improved performance in all but 2 (movement offset and movement error) of the selected features (Figure 6A). Overall, participants with ALS performed worse than HC in each feature at baseline, and the separation from HC increased over study duration in all features, except for number of pauses (Figure 6B). Only distance from the target after the main submovement reached significance in this small sample size. Similar to the smartphone app tasks, performance variability was observed in both ALS and HC participants throughout the study, at both the individual participant level and in the aggregated group data (Figure 6C).