This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.
There is an extensive library of language tests, each with excellent psychometric properties; however, many of the tests available take considerable administration time, possibly bearing psychological strain on patients. The Short and Tailored Evaluation of Language Ability (STELA) is a simplified, tablet-based language ability assessment system developed to address this issue, with a reduced number of items and automated testing process.
The aim of this paper is to assess the administration time, internal consistency, and validity of the STELA.
The STELA consists of a tablet app, a microphone, and an input keypad for clinician’s use. The system is designed to assess language ability with 52 questions grouped into 2 comprehension modalities (
The time taken to complete the STELA was significantly less than the time for the WAB (mean 16.2, SD 9.4 vs mean 149.3, SD 64.1 minutes;
Our study confirmed significant time reduction in the assessment of language ability and provided evidence for good internal consistency and validity of the STELA tablet-based aphasia assessment system.
While people who sustain brain injury from stroke or cranial trauma have higher survival rates today, remaining disabilities caused by brain damage significantly disturb patients’ daily living. Language ability is one of the important cognitive functions in executing social activities that can be impaired by brain injury. Even if individuals regain the physical ability to perform regular activities of daily living, the impairment in language function often makes it difficult for patients to fully reintegrate into their communities. Language function is fundamental to higher brain function, aiding in communication and underpinning logical thinking, social relationships, self-expression, and even dignity. Early detection and prompt and sufficient rehabilitative interventions specific to deficit severity may help such patients regain their independence and ensure reintegration into society. For this to occur, the timely and accurate assessment of language ability is crucial.
Many standardized tests are widely used for evaluating language dysfunction, such as the Western Aphasia Battery (WAB) and Boston Diagnostic Aphasia Exam in clinical settings. However, these tests take considerable time to be administered. Lengthy assessments can pose difficulties for patients with stroke or cranial trauma with language impairments, who often have trouble with long-term concentration due to effects such as inattention and limited endurance [
Several criteria must be met to evaluate language function in a short period of time and in a treatment-oriented manner. The test should be simplified for quicker completion. Conventional language function assessment tests consist of a significant number of questions. For example, the WAB consists of 225 items. Conversely, screening tests performed in shorter amounts of time have also been developed [
Although these screening tools are useful for briefly checking deficits in language ability, they do not provide sufficient information for planning treatment; to apply test findings in therapy, “language” should be deconstructed and analyzed for its components. For example, aphasia is traditionally assessed across several modalities, such as auditory and reading comprehension, word and sentence production, and repetition. Clinicians construct rehabilitation plans for individual segments based on their understanding of patients’ performance in each segment [
The Short and Tailored Evaluation of Language Ability (STELA) is a newly developed computer-based Japanese language ability assessment system for patients with aphasia, which is compact in number of tasks while having the capacity to assess language ability across multiple components. The purpose of this study was to evaluate the clinical feasibility of the STELA by investigating the time-reduction effect of the STELA compared to the WAB as the gold-standard paper-and-pencil test and its internal consistency and validity in assessing the language ability of patients with aphasia.
This was a prospective methodological study with a repeated-measures design, in which the reliability and validity of the STELA were evaluated. The research was conducted in Fujita Health University Hospital and Nanakuri Memorial Hospital.
This study complied with the principles of the Declaration of Helsinki and was approved by the Medical Ethics Committee of Fujita Health University. All the participants provided written informed consent prior to participation (HM20-060). For privacy protection, the data were deidentified in the analysis.
Study participants included patients from the Fujita Health University Hospital’s Department of Rehabilitation and Nanakuri Memorial Hospital, who were diagnosed with impaired language functioning between July 1, 2020, and December 13, 2020. Participants were diagnosed with aphasia on the basis of the WAB results. Inclusion criteria included being 20 years or older, in healthy condition, and confirmed or suspected to have impaired language function when the informed consent was obtained. Patients were excluded if they had a severe cognitive disorder or disturbed consciousness that compromised their ability to follow the instructions during testing.
The STELA is a functional language assessment system developed to rapidly, yet comprehensively, evaluate language ability (Sysnet Co. Ltd.). Consisting of a tablet app, a microphone, and an input keypad (for clinician’s use), the system is designed to assess language ability in 2 comprehension modalities (
The Short and Tailored Evaluation of Language Ability (STELA).
Variables | Items, n | Raw score |
Auditory comprehension | 16 | 32 |
Reading comprehension | 16 | 32 |
Naming and sentence formation | 10 | 28 |
Repetition | 5 | 10 |
Reading aloud | 6 | 12 |
The patients were assessed with both the STELA and the Japanese version of the WAB. The WAB is a standardized test battery for evaluating aphasia, widely used as the gold standard in language rehabilitation [
The STELA total (global) score and modality (subscale) scores as well as WAB-AQ and the subscales of WAB were used for analysis. The time required from start to finish (completion time) for both scales was also measured.
Total and group-wise comparisons of the completion time for all the tasks of the WAB and the STELA were conducted using the Wilcoxon signed rank test.
The internal consistency and validity of the STELA were evaluated. The STELA’s internal consistency was evaluated for Cronbach α [
The sample size estimated Cronbach α using the Bonett method [
In total, 31 patients participated (n=15, 48% male; n=16, 52% female; mean age 59, SD 13.4 years). Their primary diseases were cerebral infarction (n=12, 39%), cerebral hemorrhage (n=13, 42%), subarachnoid hemorrhage (n=2, 6%), brain tumor (n=4, 13%), and cerebral contusion (n=1, 3%). On average, the STELA was administered 91.5 (SD 128.3) days after the event and 4.5 (SD 3.4) days apart from the WAB. Patients’ global scores were the STELA total score=362.5 (SD 12.2; out of 500) versus WAB-AQ=66.9 (SD 28.5; out of 100). The score of the STELA was not normally distributed (
The time taken to complete the STELA was successfully measured in 27 patients (4 missing values). The time taken to complete the STELA was significantly less than the time for WAB (mean [SD]: 16.2 [9.4] vs 149.3 [64.1] minutes; degree of freedom=26, signed rank statistic (S)=189.0,
The results of the internal consistency evaluation are shown in
The STELA’s total score strongly correlated with WAB-AQ (ρ=0.93: very high correlation,
Internal consistency.
Modalities | Values | ||||||
|
Item-total correlation | Alpha without an item | Cronbach alphaa | ||||
|
|||||||
|
|
.862 | |||||
|
|
|
|||||
|
|
|
1 | 0.34 | .863 |
|
|
|
|
|
2 | 0.31 | .860 |
|
|
|
|
|
3 | 0.50 | .851 |
|
|
|
|
|
4 | 0.38 | .858 |
|
|
|
|
|
5 | 0.52 | .849 |
|
|
|
|
|
6 | 0.45 | .857 |
|
|
|
|
|
|||||
|
|
|
1 | 0.56 | .849 |
|
|
|
|
|
2 | 0.54 | .846 |
|
|
|
|
|
3 | 0.75 | .845 |
|
|
|
|
|
4 | 0.31 | .864 |
|
|
|
|
|
5 | 0.77 | .844 |
|
|
|
|
|
|||||
|
|
|
1 | 0.80 | .839 |
|
|
|
|
|
2 | 0.80 | .848 |
|
|
|
|
|
|||||
|
|
|
1 | 0.49 | .858 |
|
|
|
|
|
2 | 0.52 | .862 |
|
|
|
|
|
3 | 0.50 | .866 |
|
|
|
|
.872 | |||||
|
|
|
|||||
|
|
|
1 | 0.31 | .869 |
|
|
|
|
|
2 | 0.50 | .861 |
|
|
|
|
|
3 | 0.45 | .868 |
|
|
|
|
|
4 | 0.43 | .864 |
|
|
|
|
|
5 | 0.65 | .853 |
|
|
|
|
|
6 | 0.50 | .861 |
|
|
|
|
|
7 | 0.80 | .859 |
|
|
|
|
|
|||||
|
|
|
1 | 0.59 | .855 |
|
|
|
|
|
2 | 0.61 | .873 |
|
|
|
|
|
3 | 0.48 | .869 |
|
|
|
|
|
4 | 0.62 | .867 |
|
|
|
|
|
5 | 0.73 | .855 |
|
|
|
|
|
6 | 0.56 | .859 |
|
|
|
|
|
|||||
|
|
|
1 | 0.54 | .861 |
|
|
|
|
|
2 | 0.56 | .873 |
|
|
|
|
|
3 | 0.41 | .881 |
|
|
|
|||||||
|
|
.902 | |||||
|
|
|
|||||
|
|
|
1 | 0.56 | .899 |
|
|
|
|
|
2 | 0.76 | .891 |
|
|
|
|
|
3 | 0.73 | .888 |
|
|
|
|
|
4 | 0.69 | .891 |
|
|
|
|
|
5 | 0.75 | .888 |
|
|
|
|
|
6 | 0.69 | .895 |
|
|
|
|
|
|||||
|
|
|
1 | 0.68 | .893 |
|
|
|
|
|
2 | 0.75 | .890 |
|
|
|
|
|
|||||
|
|
|
1 | 0.92 | .895 |
|
|
|
|
|
2 | 0.92 | .896 |
|
|
|
|
.787 | |||||
|
|
|
|||||
|
|
|
1 | 0.60 | .752 |
|
|
|
|
|
2 | 0.51 | .783 |
|
|
|
|
|
3 | 0.51 | .783 |
|
|
|
|
|
|||||
|
|
|
1 | 0.94 | .673 |
|
|
|
|
|
2 | 0.92 | .706 |
|
|
|
|
.892 | |||||
|
|
|
|||||
|
|
|
1 | 0.70 | .849 |
|
|
|
|
|
2 | 0.69 | .865 |
|
|
|
|
|
3 | 0.65 | .861 |
|
|
|
|
|
4 | 0.44 | .900 |
|
|
|
|
|
|||||
|
|
|
1 | 0.84 | .857 |
|
|
|
|
|
2 | 0.96 | .895 |
|
aCronbach alpha total=.961.
Time taken for testing. Averaged time taken for WAB (white) and STELA (gray) are shown. STELA: Short and Tailored Evaluation of Language Ability; WAB: Western Aphasia Battery. *
Correlations between the Short and Tailored Evaluation of Language Ability (STELA) and the Western Aphasia Battery (WAB) scores.
STELA vs WAB | Spearman correlation coefficient |
Auditory comprehension vs auditory comprehension | 0.75 |
Repetition vs repetition | 0.96 |
Naming and sentence formation vs naming and word finding | 0.81 |
Reading comprehension and reading aloud vs reading | 0.82 |
Total vs AQa | 0.93 |
aAQ: Aphasia Quotient.
Scatterplots of WAB-AQ and STELA total scores. The correlation coefficient between WAB-AQ and STELA was 0.96 (
This study assessed the clinical feasibility and validity of the STELA, a tablet-based system for evaluating aphasia, by evaluating the administration time, internal consistency, and concurrent validity. The time taken to complete the STELA was significantly less than the time for WAB. The STELA’s total score was strongly correlated with the WAB-AQ, supporting the STELA’s concurrent validity with the WAB as a gold-standard aphasia assessment. Cronbach α coefficients and the values of item-total correlation supported the internal consistency of the STELA.
The STELA took an average of 16 minutes to be administered, approximately one-tenth the duration of WAB, demonstrating a reduced test-taking burden. Long testing sessions are typical of cognitive assessments, including aphasia, causing patients to experience fatigue and stress [
The STELA’s internal consistency was supported for all modalities and overall, by very high Cronbach α coefficients, measured at .96 for the whole scale and ranging from .79 to .90 for its subscales. Furthermore, all item-total correlations measured in each subscale were .30 or more, and statistically significant except three of them; significance level of correlations for the exceptions were marginal (ie, word comprehension item in
The STELA’s total score was strongly correlated with WAB-AQ, supporting the former’s concurrent validity concerning a gold-standard aphasia assessment. The stronger the intertest correlations observed at the subscale level, the further support it provides for the STELA’s validity in the corresponding modalities of language function.
Further integration of digital technology can allow the STELA to assess language ability even more rapidly while keeping the granularity. For example, the employment of computer adaptive testing methodology [
Since the STELA evaluates language ability using a tablet-based system, in severe cases, patients’ performance could be affected by difficulty in operating the tablet due to concurrent cognitive dysfunction. The system’s scope of application requires further investigation, along with usability concerns (eg, steps to take if patients have trouble using the tablet). Additionally, test-retest reliability was not investigated in this study, as our participants were primarily in the subacute phase after their cerebrovascular event, a period wherein aphasia symptoms can fluctuate significantly in a short period. To evaluate the STELA’s test-retest reliability, a study with a patient population in the chronic phase of illness should be further considered.
In this study, clinical feasibility of the STELA tablet-based aphasia assessment system was investigated. The results showed the significantly shorter administration time of the STELA compared with that of the WAB as a gold-standard paper-and-pencil test, and the data also supported the internal consistency and the concurrent validity with WAB. These results support the potential usefulness of the STELA in daily rehabilitation practice.
Supplemental methods.
signed rank statistic
Short and Tailored Evaluation of Language Ability
Western Aphasia Battery
WAB–Aphasia Quotient
The data collected and analyzed during the study are available from the corresponding author upon reasonable request.
The research team leased the STELA from Sysnet Co. Ltd, manufacturer of the STELA. The authors declare no other conflict of interest associated with this manuscript.