We created a website to collect structured mouse movement data by having participants complete a series of mouse-tracing tests remotely. We trained a series of deep-learning models using various features derived from the mouse traces. Figure 1 outlines the workflow from data collection to model evaluation and interpretability analysis.

We recruited participants for this study through both online methods (email and social media posts) and offline methods (community meetings and conventions in Hawaii). We collaborated with the Hawaii Parkinson’s Association and Beyond Rehab to post flyers and advertise the study to various PD emailing lists. Additionally, we established a recruitment booth at the 2024 Hawaii Parkinson’s Association Symposium and the 2024 Hawaii Annual Parkinson’s Run, where we provided potential participants with flyers describing how to complete the study.

We collected data via a web application that we developed [40], illustrated in Figure 2. Participants provided demographic and disease-related information, including age and dominant hand. Due to the infeasibility of obtaining official diagnostic documents for all participants under the constraints of the pilot grant that funded this project, we asked participants to self-report their PD status.

Notably, we provided an option to select “suspected PD” for participants who think that they might have PD but are yet to receive a formal diagnosis, as nonmotor symptoms and subtle, nonspecific motor signs may appear before a formal PD diagnosis, which is often referred to as the prodromal phase of PD [41]. For example, in the Rotterdam Study, it was found that among individuals who developed PD after a mean follow-up of 5.8 years, 71.8% had reported at least one early subjective motor symptom [42]. For patients below 50 years, it typically takes over 2 years to receive an official PD diagnosis [43]. Moreover, disparities in the health care system make it harder for people to access a diagnosis, and non-White Americans in the United States are half as likely to be diagnosed with PD [44]. Additionally, in low-income and middle-income countries (ie, Kenya), a diagnosis of PD is often followed by a period of uncertainty where, sometimes, a diagnostic decision is reversed [45].

However, participants who selected “suspected PD” did so based on their personal concerns or perceived symptoms and were not required to report or demonstrate clinically observable motor symptoms or provide official diagnosis documents during participation. Therefore, the “suspected PD” category may include individuals in the prodromal phase of PD, individuals with other motor or neurological conditions, or even individuals without any actual motor conditions.

Participants used a physical mouse on their desktop or laptop to trace a straight line, sine wave, and spiral wave on the website. We visualized their progress and alignment with the lines through highlighted portions and start or end markings. We developed the website using HTML and Bootstrap for the interface and visuals, and JavaScript to track cursor position every 500 ms. The data included mouse positions (x and y axes) and whether the mouse was inside the line (true or false). The web application also captured screen dimensions and operating system details for contextual information. Upon completing the test, all data were securely transmitted and stored in a Google Firebase collection.

Based on the raw data collected, we engineered several features to quantify tracing performance across the 3 tracing tasks (straight line, sine wave, and spiral). These features include the following:

Using the features shown in Table 1, we quantified the consistency and accuracy of a participant’s trace on each test pattern.

To generate mouse-trace images, we created canvases corresponding to each participant’s screen dimensions. The recorded x and y coordinates were then used to render the trace, where segments starting with points inside the target region are colored green and those that fall outside are colored red. Figure 3 illustrates the samples of these plots.

We developed three categories of deep-learning models, as shown in Figure 4, each utilizing different input modalities, to better understand the individual and combined contributions of image and tabular mouse-trace data for PD classification. The first category of models focused on engineered tabular features, processed through a feed-forward neural network. We used this category to determine whether using purely engineered features can reliably classify patients diagnosed with PD. A feed-forward neural network was selected for consistency with the tabular data branch used later in the multimodal models. The second category of models comprised image-based models, where input images were passed through pretrained convolutional neural network (CNN)–based architectures: Densely Connected Convolutional Network-201 (DenseNet-201), Residual Network-50 (ResNet-50), and Mobile Network v2 (MobileNet V2), and two transformer-based architectures: Vision Transformer (ViT) and Shifted Window Transformer (SwinT). CNN-based architectures (ie, DenseNet-201, ResNet-50, and MobileNet V2) excel at capturing edges and textures, which can help detect tremor-related discrepancies in the image trace. On the other hand, transformer-based architectures such as ViT use global self-attention, which can help capture the overall geometry of the traces. Additionally, SwinT uses hierarchical shifting-window attention, which can help capture both fine-grained patterns and the overall trace shape. Specifically, we used ViT-Base-Patch16-224 and swin_base_patch4_window7_224. Each of these models was selected based on their prior performance on different image-based tasks, as both CNN and transformer-based architectures have demonstrated tolerance to variations in scale, resolution, and local noise—properties that are expected in remotely collected mouse-trace images generated from diverse hardware and screen configurations. Transfer learning was applied by unfreezing the final 45 layers of each network and by replacing the classification heads for training. The third category of models consisted of multimodal models, which integrated both data types. Tabular features were passed through a feed-forward neural network, while image inputs were processed via the same pretrained backbones used in the image-based models. The resulting feature representations were concatenated and passed through additional hidden layers for classification. All models were trained as binary classifiers for up to 50 epochs, with early stopping applied using a patience of 5. By structuring the models consistently across these categories, we aimed to isolate and analyze the impact of each data modality and understand how individual components contribute to the performance of the multimodal architectures.

We evaluated model performance across 3 experimental settings, with 1 primary experiment and 2 secondary analyses, designed to reflect different levels of diagnostic certainty and real-world deployment scenarios. The primary experiment focuses on assessing the feasibility of training models on patients who self-reported suspected PD and predicting diagnosed PD.

In the primary experiment, models were trained on participants with suspected PD and 60% of non-PD data and evaluated on all PD cases and the remaining 40% of non-PD samples, split equally into a test set and a validation set. This experiment directly addresses the central research objective by assessing whether models trained on low-specificity, community-recruited labels can generalize to predict diagnosed PD. To mitigate variability caused by nondeterministic behavior observed in some of our image-based and multimodal models, final performance was estimated by averaging results across 20 independently trained models, with average performance reported using 500 bootstrap resamples in each run. Model hyperparameters were tuned using Optuna [46] over 30 trials.

In a secondary analysis, models were trained on all participants with self-reported PD and 60% of the samples without PD and evaluated on participants with suspected PD combined with the remaining 40% of non-PD data. This analysis was designed to assess whether models trained on patients diagnosed with PD can generalize to individuals with uncertain or emerging motor symptoms. Hyperparameters were tuned using Optuna over 30 trials, and final performance was estimated by averaging results across 20 independently trained models, each evaluated using 500 bootstrap resamples. This analysis is not part of the primary study objective and is included only to assess whether the learned signal is bidirectional.

In the cross-validation analysis, we included only participants labeled as either self-reported PD or non-PD, explicitly excluding those marked as suspected PD. Models were trained and evaluated using 5-fold cross-validation. Within each fold, hyperparameters were tuned using Optuna across 10 trials, and performance was estimated using 500 bootstrapped resamples per fold. We repeated each fold 10 times with independently initialized models, reporting average performance across runs. This experiment established a baseline for distinguishing controls with PD from controls without PD using stronger supervisory labels.

Bootstrapping across the different experiments was used to estimate variability and model stability under resampling. However, these approaches do not increase the amount of independent data and therefore do not fully capture uncertainty arising from the limited dataset size.

To evaluate model performance, we used 4 commonly used classification metrics: sensitivity, specificity, accuracy, and F₁-score. These metrics were calculated using the confusion matrix derived from model predictions, consisting of the following:

From these values, we calculated the following metrics:

Each of these metrics was estimated using 500 bootstrapped resamples to compute both the mean and SD, providing a robust measure of model stability and generalizability. We used the mean F₁-score as the base metric to select the best performing models from our analysis. All metric values that we presented included the mean and SD of the metrics.

To identify important features for the best performing model architectures, we created GradSHAP (Gradient Shapley Additive Explanations) saliency plots, as GradSHAP is optimized for identifying complex feature importances in deep-learning models [47]. GradSHAP was applied to both the image and engineered tabular inputs to quantify their respective contributions to the model’s predictions. For the image modality, attributions were computed separately for the sine, straight, and spiral images by comparing the model’s output on actual inputs and shuffled versions of the same batch, used as baselines. Attribution values were summed across spatial dimensions to yield a single importance score for each image input. Importantly, the sign of each attribution was preserved to indicate the direction of influence: positive values reflect contributions toward a PD or suspected PD prediction, while negative values indicate support for a non-PD classification.

Our study was approved by the University of Hawaii Institutional Review Board (protocol 2022‐00857). We collected informed consent forms from all participants at the start of data collection on the website. All data used in this analysis were anonymized by default, as the assessment data did not include any personal information.

We had 261 participants with completed attempts for the mouse-trace section of the website. Among the 261 participants, 73 self-reported as having PD, 155 as non-PD, and the remaining 33 as suspected PD. Attempts with data available for all 3 mouse-trace tasks, irrespective of the age group, were included. The characteristics of the collected data are shown in Figure 5 and Table 2.

We selected the best-performing models from each experimental setting based on the highest mean F₁-scores achieved across runs. In the primary experiment, the multimodal ViT achieved the best performance, and in the secondary analysis, the multimodal ResNet-50 performed best. In the cross-validation analysis, the image-based DenseNet-201 model achieved the highest performance.

We conducted GradSHAP-based feature importance analysis on the 2 multimodal models from the primary experiment and the secondary analysis to better understand the relative contributions of image and engineered features. We did not perform the feature importance analysis on the image-based DenseNet-201 model, as such an analysis would provide limited additional insight: with no tabular features present, the dominance of image inputs is expected and not subject to comparison. Our goal was to explore how much influence tabular features or image features have in a multimodal setting and whether models rely solely on image inputs or leverage both data types.

Figure 7 depicts the feature importance analysis for the multimodal ViT model trained on suspected PD versus non-PD and tested on PD versus non-PD. We found that the model relied heavily on the images for prediction. However, in this case, the engineered features, especially the time required to trace the patterns, exhibited high SHAP (Shapley Additive Explanations) values. The height and width of the screen used by the participant also exhibited relatively high SHAP values.

Figure 8 depicts the feature importance analysis for the multimodal ResNet-50 model trained on PD versus non-PD and tested on suspected PD versus non-PD. We found that the model relied predominantly on image features for prediction, with the spiral, sine, and straight line images showing the highest SHAP values. Among the engineered features, only the time taken to trace the spiral and sine waves, as well as the straight line, exhibited significant SHAP values, while the rest of the tabular features had a negligible impact.

We sought to study the feasibility of training models using weak labels of “suspected PD” to predict confirmed PD in a community-recruited sample. We evaluated model performance using 1 primary experiment and 2 secondary analyses, each reflecting different assumptions about diagnostic certainty and real-world deployment scenarios. The primary experiment involved training models on participants with suspected PD and without PD and evaluating them on patients with self-reported PD and without PD, reflecting a real-world use case in which models trained on low-specificity, community-recruited labels are applied to identify diagnosed PD. We found that training models on patients with suspected PD yielded moderate yet meaningful performance in predicting patients with self-reported PD. These preliminary findings support the feasibility of leveraging community-recruited, low-specificity labels to learn patterns that generalize to predicting diagnosed PD. This suggests that, even in the presence of diagnostic uncertainty in labels, suspected PD data can possibly retain signals relevant to motor symptoms that may still be useful for developing scalable screening tools in global health contexts. However, we emphasize that this claim needs to be supported by future work involving participants with clinically confirmed diagnoses.

We also found that images derived from spiral, sine, and straight line tracing were the dominant contributors to model predictions for the multimodal models. Tabular features such as tracing time showed modest attribution in the ResNet-50 (secondary analysis), while showing much higher influence in ViT (primary analysis with suspected PD). Device-related features (screen height and width) played a minimal to nonexistent role. In the ViT model trained on patients with suspected PD, the increased influence of engineered features suggests that engineered data can complement visual patterns when used to train models to identify real patients with PD.

To the best of our knowledge, we conducted the first pilot feasibility study to assess whether lower-specificity weak labels of “suspected PD” can enable artificial intelligence models to learn useful motor signatures related to PD. From a computational methods perspective, our study also expands upon previous research by introducing an approach relying exclusively on mouse-trace data and images produced from a brief online test for PD classification through the use of deep-learning models. Unlike prior methods that have explored hand and finger movements, keyboard typing patterns, keystroke dynamics, speech analysis, handwriting, drawing tests, and sensor data from accelerometers, gyroscopes, and smartphone interactions, we specifically focus on the remote collection of mouse-tracing data, and our experiments demonstrate the potential of mouse-trace data collected from a heterogeneous array of devices to provide predictive power in classifying PD.

Previous studies by Gil-Martin et al [48] and Pereira et al [49] focused on hand movement dynamics from spiral, meander, and other drawing shapes for PD analysis. However, their data collection was not conducted remotely, and they did not consider handedness, unlike our study. Their best models achieved accuracies of 97.7% and 83.77%, respectively. Goel et al [50] used pen-and-paper methods to collect spiral pattern data but also lacked remote testing and consideration of handedness, achieving an accuracy of 87.36%. Memedi et al [51] used a remote data collection method involving a touchscreen tablet and web interface, but their study spanned 3 years and involved only 65 participants, resulting in an accuracy of around 84%. Wang et al [52] achieved 96.2% accuracy using the data collected during the drawing of Π and ΠΛ shapes. However, their data collection method involved an iPad and stylus and was not conducted remotely. The classification model suggested by Gajos et al [53] detected ataxia or parkinsonism with high sensitivity (0.91) and specificity (0.90) but included only 46 patients with PD in their dataset with 22,946 controls.

These studies served as important and inspirational foundational literature that we aimed to build upon. The results from most prior works were obtained under controlled conditions with specialized hardware; however, our remote approach trades some performance for scalability.

A previous pilot study of ours [54], which explored the feasibility of an earlier version of our web application on a much smaller participant sample, achieved an accuracy of 74.29% and an F₁-score of 73.11% using classical machine-learning models. This work used both keyboard and mouse-trace data as engineered features and did not include patients with suspected PD in the data collection process. Our current models show marked improvements in performance compared to this prior pilot study, possibly due to either our novel machine-learning approach presented here or the significantly expanded dataset size.

Our findings support 2 key observations. First, despite label noise, models trained using patients with “suspected PD” may retain sufficient motor signals to enable the meaningful prediction of diagnosed PD, supporting the feasibility of using community-recruited, low-specificity data for model development. Second, motor patterns observed in individuals with suspected PD share detectable similarities with those diagnosed with PD, allowing the models to generalize even when diagnostic certainty is low. However, several key validation steps are required to fully validate this approach, including evaluation using clinically confirmed prodromal PD cohorts and the inclusion of other related motor disorders for a more rigorous assessment of model specificity (ie, whether the models are actually identifying PD or reporting noise caused by motor dysfunctions). Ultimately, our findings should be interpreted as preliminary feasibility evidence to support further, larger-scale investigation.

As a pilot feasibility study, this work has several limitations that should be considered in future research. First, the sample size of 261, split into 3 diagnostic categories, may not be sufficient to generalize the findings across a diverse population and heterogeneous disease stages. This limitation is particularly relevant for the suspected PD group, which contained only 33 participants. Although we tried to mitigate this through repeated training runs and evaluation across multiple splits, small sample sizes can still affect model generalizability and may have been the reason behind the occasional appearance of extreme values for sensitivity and specificity. Moreover, this small sample size for the suspected PD group limits the statistical power and generalizability of our findings. In the future, we plan to prioritize expanding recruitment and validating these findings in larger and more diverse cohorts. Moreover, while we performed hyperparameter tuning for each model, further optimization, especially regarding model architecture and data fusion strategies, could improve performance and generalization. Some models, particularly those involving image-based and multimodal inputs, exhibited nondeterministic behavior. These models produced variable results across repeated training runs on identical data splits, likely due to the stochastic nature of deep learning, their sensitivity to complex, high-dimensional input spaces, and resampling on the small sample sizes. This reinforces the importance of repeated experimentation and performance averaging in evaluating model robustness. We used a fixed sampling rate of 500 ms to store mouse positions on screen, resulting in some data points with only one time step represented. While we retained those data in this study, future work should explore more granular sampling rates (eg, 50/100 ms) and stricter exclusion criteria.

Notably, a key limitation of this study is that the “suspected PD” category was based solely on self-report and did not require clinical verification. Consequently, this group is heterogeneous and may include individuals with health anxiety, age-related motor conditions, or non-PD neurodegenerative disorders. Future work should incorporate longitudinal follow-up or clinician-verified diagnoses to better characterize which patients with suspected PD progress to PD and to refine early-stage prediction models.

Additionally, our focus on mouse-tracing data collected through a website does not fully capture all aspects of PD symptoms. Importantly, the study did not account for medication usage, specifically distinguishing the “on” phase versus the “off” phase, which can significantly influence the presence and severity of symptoms such as tremors [55-57]. Stress, which is known to exacerbate tremors, was also not accounted for, potentially affecting the results [58]. All data were collected remotely using participants’ own devices, resulting in variability in devices, operating systems, and screen configurations. Participant characteristics, such as age and handedness, were not balanced across groups. These factors may introduce confounding effects, as differences in hardware, cursor behavior, or user interaction patterns could influence both engineered features and generated mouse-trace images. Furthermore, the impact of device type and handedness on the results remains unclear, as models trained on mouse-tracing data tend to reflect device type and handedness bias [59]. In addition, PD often affects one side of the body more than the other, and it is not certain that participants’ dominant hands were the ones most affected by the disease. Future work should include controlled data collection using standardized hardware and display settings to calibrate model performance and validate findings under more reproducible conditions.

In this early-stage pilot feasibility study, we provide exploratory and hypothesis-generating evidence that models trained using weaker labels of “suspected PD” from a community-recruited sample can capture motor patterns that generalize to predicting diagnosed PD. However, due to the use of self-reported and heterogeneous labels, particularly within the suspected PD group, these results should not be interpreted as evidence of clinical screening utility and should serve as a basis for future extramurally funded work conducted using clinically confirmed cohorts in conjunction with lower-resourced community samples.