Globally, the number of older adults is projected to grow substantially over the next 2 decades [1]. Such a significant demographic shift brings increased incidence rates of age-related cognitive decline, mild cognitive impairment, and dementia, highlighting the need for more accessible, efficient, and scalable cognitive screening methods to keep pace with the identification of clinically meaningful decline and fluctuations of cognition [2,3]. While conventional in-clinic, paper-based tests have been the standard form of screening and testing, the recent advent of reliable and user-friendly digital technologies has expanded opportunities to move cognitive evaluations into digital paradigms, increasing in complexity and dimensionality while providing opportunities for richer exploration of behavior [4]. These technologies also look to break systemic and environmental barriers that limit access to underserved populations in both cognitive research and vital health care monitoring [5]. Additionally, digital assessment platforms can automate scoring and data storage, thereby reducing administrative burden and cost, while also providing an opportunity to reach larger populations through remote testing [6]. Transitioning to digital neuropsychological assessment has strong potential to bolster the capacity to detect, monitor, and address cognitive changes in older adults by enabling scalable, longitudinal monitoring that can complement traditional clinic-based assessment and advance research at the population level.

A growing body of evidence supports the digital adaptation of well-established paper-and-pencil neuropsychological tasks and cognitive screening batteries [7-11]. Cognitive screeners, such as the Montreal Cognitive Assessment, have been shown to be reliable in digital form [12]. Digital versions of standard neuropsychological tests, such as the Trail Making Test (TMT), a measure of executive functioning, have shown good correspondence to paper-and-pencil counterparts [13-15]. Similarly, the Clock Drawing Test, a longstanding yet quickly administered dementia screener capturing executive and visuospatial function, has been demonstrated to be a reliable and valid tool when converted to digital modes [16]. These 3 tasks are just a few examples of many tests that have been successfully digitized for use via mobile smartphones or tablets, each demonstrating acceptable sensitivity and specificity in detecting cognitive impairment. However, a broader effort to integrate these digitized assessments into comprehensive digital batteries has emerged, with several platforms not available for use in older adult populations.

Tablet-based platforms for neuropsychological testing, such as Cogstate C3 [17], Tablet-Based Cognitive Assessment Tool (TabCAT) [18], or the Cambridge Neuropsychological Test Automated Battery (CANTAB) [19], offer well-validated, semi–self-administered testing in older adult populations with and without cognitive decline. However, these commonly used systems are narrow in their scope and require proprietary licensing, constraining the range of constructs they can measure and who can collect data using them. In the same vein, remote batteries specifically validated for older adult cognitive monitoring, such as the Boston Remote Assessment for Neurocognitive Health [20] and similar mobile and web-based platforms, focus on a limited number of tasks that primarily function as screeners for clinical cognitive decline. The National Institutes of Health (NIH) Toolbox Cognition Battery [21] offers more tasks for a wider number of age groups; however, a number of studies have identified issues in these tasks’ sensitivity compared to alternative digital testing platforms, especially in older adult populations [22,23]. Critically, all these testing systems suffer from limited configurability in both the tasks they can administer and the degree to which researchers can customize these tasks to fit their research needs, which can be constraining to basic research.

Understanding how reliably these comprehensive assessments perform over time is critical, particularly given that cognitive performance in older adults can fluctuate due to transient factors such as mood, fatigue, and comorbid clinical conditions [24]. Evaluating the stability of scores between 2 closely administered assessments can help to clarify whether changes reflect real cognitive shifts or measurement inconsistencies. Digital tools may be helpful in reliably detecting and modeling these fluctuations, which may be critical for capturing small changes that precede clinical decline or for characterizing specific cognitive phenotypes [25]. Ambulatory platforms such as the Mobile Monitoring of Cognitive Change (M2C2) [26] and the Ambulatory Research in Cognition (ARC) app [27] offer brief, frequently repeated assessments delivered on participants’ own smartphones to capture intraindividual fluctuations in cognition [28]. Although these platforms offer strong ecological validity, their brief, single-domain assessments are not designed to capture the full range of constructs that comprehensive clinical neuropsychological batteries traditionally assess. If digitally adapted assessments that expand on these narrow measurement targets exhibit robust reliability over temporal intervals while accounting for possible fluctuations in individual differences, they may prove ideal for continuous longitudinal monitoring of cognitive trajectories in clinical trials, primary care, and at-home settings. Conversely, evidence of inconsistent scores would underline the need for further technological refinements, novel analysis models to account for this inconsistency, or additional training sessions for less reliable performers to ensure accuracy and participant adherence and acceptance.

Moreover, the transition from paper-based assessments to digital ones requires considerations beyond the cognitive demands the task provides. Accordingly, digital assessments must implement an intuitive and tailored design, require minimal technological expertise to use, and provide adequate instructions to mitigate confounding variables. Establishing the usability and acceptability of these tools among older users is a critical step before they can be fully integrated into regular clinical practice. Several studies have mitigated these confounds by requiring psychometricians or study administrators to assist in person with administration [5,14]. However, there is still a need for platforms that can accommodate diverse testing use cases without sacrificing the usability of the tasks themselves or the underlying cognitive construct the task is trying to measure.

Despite numerous advances, the current literature lacks sufficient studies examining the reliability and feasibility of comprehensive digital neuropsychological testing in remote settings. Our contribution to this emerging field is to evaluate digital versions of traditional paper-and-pencil neuropsychological tests, examining feasibility and repeatability in diverse clinical and research contexts [29]. To achieve this, our laboratory has developed the portable automated rapid testing (PART) [30] platform, a mobile app that facilitates remote data collection without compromising the breadth and depth of the testing schema or the validity of tasks when compared to in-laboratory testing. PART is a freely accessible digital app available via the iOS App Store and the Google Play Store. Where PART builds upon the digital assessment platforms currently available is that it is highly configurable in terms of the parameters of individual tests (eg, stimuli and timing parameters) and the ability to create testing batteries that can combine assessments of different modalities (hearing [31,32], vision [33], executive functions [34], attentional control [35], decision making [36], gamified interventions [37], and, in the current paper, neuropsychological tests). In turn, this allows researchers to have a system that can be used for both clinical evaluation and basic research.

The current work aims to examine test-retest reliability and participant-reported usability of a standard neuropsychological battery within PART administered twice across a 1-month interval. The tasks assess verbal fluency (VF), memory encoding and recall, constructional praxis (CP), and executive functioning. By analyzing any changes in performance, user engagement, acceptability, and data quality, we seek to determine how well older adults adapt to digital neuropsychological testing and whether these tools can consistently capture cognitive data over short intervals.

This study examined the acceptability and short-term reliability of a digital neuropsychological battery administered remotely using the PART app in a cohort of cognitively healthy older adults. Importantly, our sample’s distributions of performance on digital versions of paper-and-pencil tasks were comparable to the ranges found in clinical norming studies, and participant feedback indicated feasibility and self-reported comfort. Below, we explore the implications of these findings and, in turn, discuss the broader contributions of PART to the digital assessment landscape.

The current findings aim to situate PART within the broader landscape of digital cognitive assessment, highlighting unique contributions. PART is a flexible framework for repeated, remote testing across multiple cognitive and perceptual domains within the same ecosystem. This sets a precedent for cross-domain testing which, when further combined with open-access principles, results in a tool capable of richer behavioral phenotyping in digital environments. The configurability of testing schema that combines seemingly disparate constructs has immense promise for multi-domain monitoring of aging. PART is one of many digital tools that are being leveraged by researchers, and future work should aim to harmonize across different software platforms and device types, enabling more direct comparability across studies and settings.

Feasibility was generally acceptable and was consistent with mobile, remote cognitive studies in older adults [60]. Digital TMT may require further investigation, as we excluded a substantial number of individuals using our cutoff criteria (ie, 3 or more errors and total draw times above 160 and 300 seconds for parts A and B, respectively). This could be interpreted as participants struggling with task interfacing, rather than actual cognitive deficits, as all the individuals excluded because of their digital performance would have otherwise not been excluded based on their paper performance, which, in turn, limits the interpretability and generalizability of our TMT reliability estimates. It should be noted that we did see nontrivial participant dropout from screening to consent and consent to T1, possibly due to difficulties in communication and acceptance noted in remote cognitive assessment [61]. Future work should examine strategies to lower overall attrition rate from screening to completion as well as target a larger sample. Stronger retention strategies should be used that account for differential participation based on demographics as well as strategies to increase participants’ perceived value of participation, both shown to affect retention rates [60]. Although the testing was over an hour on average, we did not observe any systematic fatigue effects such as reduced comfortability over the course of testing nor mention of fatigue from participants. Overall, over 75% of participants rated each task as comfortable, with the battery being 88% tolerable during T1 and 84% tolerable for T2 (Table S1 in Multimedia Appendix 1). It should be noted that although retention from T1 to T2 was generally high, our larger study design, which asked for 4 experimental sessions over the course of a week, twice, introduced nontrivial burden.

Several measures demonstrated moderate stability (r=0.50-0.73), while others showed lower agreement and substantial within-person variability, notably clock drawing and CP. Bland-Altman LoA showed nontrivial within-subject variability, notably seen in the TMT (part A=–44.7 to 29.2; part B=–78.1 to 57.9). Of note, bias was generally low across tasks but was noted for both logical memory stories (story A=–2.1; story B=–1.5) and TMT (part A=–7.8; part B=–10.1), leading us to believe that the short retest period may have introduced practice effects in these tasks, especially. Further, several participants showed ceiling effects across time points, including clock drawing (11/63, 17% with maximum scores across time points), CP total recall (7/61, 11% with maximum scores across time points), and BNT (16/62, 11% with maximum scores across time points), indicating limited sensitivity for high performers. Although mean scores fell within 1 SD of published norms (eg, CERAD and CCCS), categorical fluency performance (supermarket items and animals) in our sample trended higher, possibly reflecting the high educational attainment of our participants (mean 16.6 years; SD 4.0 years) [62].

The sample was also predominantly female, which may have influenced fluency performance, as some evidence suggests steeper age-related declines in categorical, but not letter, fluency among men relative to women [63], though other studies report minimal sex differences after accounting for education [64,65]. The overrepresentation of women and highly educated individuals limits the applicability of our findings to broader older adult populations, and, further, our sample was not representative of the Southern California populations from which it was drawn. While age distributions were broadly comparable to those of normative samples, these demographic characteristics should be considered when interpreting observed performance levels. Future studies must aim to stratify sampling to capture trends from both individuals with lower educational attainment and those from historically underserved communities who stand to benefit most from accessible cognitive assessment.

These comparisons serve as group-level descriptions of our current sample’s performance on digital tasks compared with generally comparable samples from past work completing paper versions of the same tasks. For example, our comparisons in phonemic VF were approximated based on normative data from 3 letters (F, A, and S), which was a limitation as each individual letter differed nontrivially, and thus, this comparison should be interpreted with caution. The current normative benchmarking approach, while informative at the group level, does not permit conclusions about equivalence at the individual level. Using the TMT evidence in this study as a benchmark, future validation work should involve administering both digital PART versions and traditional paper-and-pencil versions to the same participants in a counterbalanced order, allowing direct within-individual comparisons.

These results imply that while the PART battery can reproduce normative group patterns in a remote setting, care is needed when interpreting change at the individual level over short intervals, not only due to individual differences but especially for tasks susceptible to practice effects or ceiling effects, something observed in the population-based norming studies leveraged in the current work. Of note, our protocol was partially constrained by logistical considerations, such that our time interval may have been too short to prevent practice effects, as evidenced by significant improvements on logical memory and TMT between time points. Promisingly, our sample’s distributions of performance on our task battery across the 2 time points were generally near the distributions in past studies leveraging paper-and-pencil versions of similar tasks. Although there is benefit in rapid and frequent cognitive testing, future validation studies that leverage PART should induce longer retest periods to ensure data avoid practice effects.

Despite positive findings regarding battery acceptance and reliability, our study has several notable limitations. Our sample was predominantly White, educated, and female, which constrains generalizability. Several studies have long identified differences in neuropsychological task performance depending on demographic factors such as race, ethnicity, sex, education, and socioeconomic status [64,66,67]. Importantly, our comparison data were stratified similarly in terms of age and race, yet not biological sex or education. However, future efforts to use these tasks in PART should aim to include a larger population of individuals with lower education and minority backgrounds. A notable limitation of our protocol was its reliance on mailed researcher-configured hardware. While this approach standardizes the testing platform and ensures more consistent data fidelity across participants, it is resource-intensive and not as scalable to large populations compared with using participant-owned devices.

Of note, our work focused on group-level consistency, which was achieved by comparisons with the normative samples along with group-level reliability analyses. Future studies should investigate using inferential statistics if any differences exist at the levels of clinically meaningful changes in digital versions of paper-and-pencil neuropsychological tasks. Further, we collected limited usability data and, rather, investigated self-reported comfort levels. Older adults often have varying levels of digital literacy and comfort with touchscreen devices, factors that could potentially influence test performance [68]. Moreover, physical limitations such as reduced fine motor skills or impaired vision can exacerbate the challenges of engaging with digital interfaces, such as smaller devices and less interactive feature spaces [69]. Although administering a comfort question after each task was informative, this approach is limited insofar as it is not a validated measure and captures only one dimension. Measures of digital literacy and validated user-experience instruments (eg, senior technology acceptance model [70]) were not administered and should be added to future studies, as they remain critical for future successful digital data collection [11,71]. Moreover, many of our tasks required manual scoring, which can introduce subjectivity that can bolster or attenuate reliability and, although all raters were thoroughly trained, future studies should aim to evaluate interrater reliability of scoring in the analytic data pool or investigate the potential for automatic scoring of these tasks using AI [72,73].

Further, alternative strategies of study design and measurement itself could be used, bolstering ecological validity. Although the present study captures participants’ cognitive performance in a more naturalistic setting (ie, at home vs in the laboratory), participants were tested with unfamiliar devices in potentially atypical settings (ie, long Zoom visits) rather than using their own smartphones or tablets in their day-to-day environment. Shorter, more frequent measurement strategies using participants’ own devices could potentially account for context-varying influences such as mood, fatigue, time-of-day effects, or item-level response patterns, better isolating cognitive change from extraneous sources of variability not directly related to task performance [74]. Largely, digital assessment tools that are highly configurable present an opportunity to tailor design around assessments to account for a variety of ecologically constraining factors.

Importantly, many of the tasks examined in the current work may have alternative outcome measures unique to digital modalities, which could afford increased measurement precision and sensitivity, thereby enhancing researchers’ and clinicians’ ability to detect subtle, within-person cognitive changes and fluctuations associated with aging. For example, work validating alternative digital forms of the TMT has identified outcomes beyond draw time, such as time between circles or number of lifts, as highly predictive of other executive functioning measures and global cognitive screeners [49,50]. It should also be noted that although certain stylus-based tasks (eg, drawing lines or tapping responses) show evidence of reliability, issues in interface design and interaction (eg, digital “smudging”) have been identified as key covariates that require more testing, validation, and fine-tuning [49,75]. Future work should further examine this interplay between cognition and digitally derived motor outcomes as well as the degree to which their measurement sensitivity is affected by these extraneous factors.

Future work using PART can and should address the new avenues digital data collection affords. PART is available on public platforms for desktop, smartphone, and tablet devices (eg, iOS App Store and Google Play Store), allowing for repeated, frequent testing, which may be more sensitive in capturing cognitive changes [28]. Our battery was administered with oversight from trained research administrators, which is both a feature insofar as it may improve data quality and most closely mirrors clinical practice, and a limitation, as fully self-administered batteries may be more scalable. Future work should examine the degree to which these tasks could be fully self-administered to older adult participants on their own devices. Another key advantage of digital assessment platforms is the potential to replace or supplement ceiling-prone tasks with adaptive or more sensitive paradigms [76]. Future studies using PART can incorporate adaptive item difficulty (eg, expanding BNT item sets, using more complex drawing tasks, or adding more challenging executive function conditions) to improve sensitivity and reduce ceiling effects in healthy older adult cohorts.

PART also collects participant audio recordings and temporal tablet touch data, which may provide alternative outcome measures as valuable, or more valuable, when compared to traditional neuropsychological test outcomes. All our nondrawing tasks in the current work involved recording speech and may provide critical information that improves upon measures originally designed under the constraints of paper-and-pencil tasks [11]. Although these speech analyses were out of the scope of the current work, future analyses should investigate how these speech recordings can act as a primary or supplementary data stream, as recent work has shown its utility in categorizing patients with mild cognitive impairment and Alzheimer disease [77,78].

In summary, this study demonstrates that a remotely administered digital neuropsychological battery using PART is feasible, well-tolerated, and produces acceptable short-term reliability in older adults within expected ranges compared to well-established paper-and-pencil counterparts. It advances longitudinal aging measurement through a flexible approach and supports remote screening, longitudinal monitoring, and inclusion of otherwise excluded participants. PART complements in-person assessment while providing potential avenues to expand the capabilities of traditional assessment. Critically, its open-access, configurable framework positions it as a tool that can serve both clinical screening and basic research aims, adaptable across domains and populations. Future research should take advantage of remote assessment frameworks through larger, more diverse sampling procedures, explore the potential for more ecologically valid data schemas, and consider evaluating reliability and feasibility in clinical cohorts.