This section describes the study’s ethical considerations, the study design, the study sample, the SG and activities, data collection sources and formulation, and the analytical procedure.

This study is part of an ongoing research project titled “Smart Computing Models, Sensors, and Early Diagnostic Speech and Language Deficiencies Indicators in Child Communication,” with the acronym “SmartSpeech” funded by the Region of Epirus and supported by European Regional Development Fund.

The study was conducted in accordance with the Declaration of Helsinki and approved by the Research Ethics Committee of the University of Ioannina, Greece (protocol code 18435, approved on May 15, 2020). The project’s nature, purpose, and procedures were thoroughly explained to parents during an informative meeting, ensuring compliance with the General Data Protection Regulation (GDPR). All participating parents gave informed written consent after being provided with detailed information regarding the study’s objectives, procedures, and ethical guidelines. The informed consent process ensured that parents understood the study’s compliance with GDPR. To protect participants’ privacy, the data collected in this study were deidentified. All necessary measures were taken to ensure the confidentiality of participant information throughout the study. No compensation was provided to participants. Care was taken to ensure that no individual participants or users could be identified in any images included in the paper (or its supplementary materials).

The research was conducted in educational and health care settings across Greece. As this is a randomized trial, the CONSORT (Consolidated Standards of Reporting Trials) 2010 checklist was used to report information included in this study. Participants were recruited through an open call distributed via private and public health and educational establishments across the country. The recruitment focused on parents of typically developing children aged 4 to 12 years. Parents were invited to informational meetings where the study’s objectives, procedures, and ethical guidelines were explained. This session covered the study’s purpose, the nature of the SG “Apsou” by SmartSpeech Project, and the expected involvement of their children. Parents received comprehensive information about the study during these sessions and were asked to provide informed written consent for their children’s participation to ensure compliance with GDPR and ethical considerations. The study included typically developing preschool and school-aged children without NDs, other medical conditions, or medications that could affect the results. Under clinicians’ face-to-face supervision (2021-2022), children played the SG “Apsou,” designed to measure various developmental domains, including speech, language, psychomotor, cognitive, psychoemotional, and hearing abilities [43]. This structured approach ensured comprehensive and ethical recruitment, engaging parents and children while adhering to ethical guidelines and data protection standards.

The study included 229 preschool and school-age children, 118 (51.5%) of whom were boys, and 111 (48.5%) were girls. The mean age of the children was 6.88 (SD 1.94) years. All the children in the sample were attending typical educational settings. We state that all participants included in the sample completed the whole SG.

The SG Apsou, version 1.0.1, developed within the SmartSpeech project, is designed to detect children at risk of NDs. It offers a web-based platform with various activities targeting different developmental skills, making the assessment process engaging and effective. The game is designed around a narrative plot, allowing players to use existing skills or acquire new ones in a scenario focusing on motivation, narrative, and fun [52]. The digital gamified environment uses video game mechanics to turn evaluation into a game, leveraging the capabilities of Unity (Unity Technologies), a cross-platform game engine, for smooth gameplay with high-quality graphics and animations (Multimedia Appendix 1) [53,54]. The game is designed for mobile devices (Android, iOS, Windows) and collects data from parents, clinicians, and the child’s gameplay, providing a comprehensive data set for analysis [53,54]. The validation process for the Apsou game involves several steps to ensure its effectiveness in assessing NDs, including data analysis using various AI algorithms. The game’s effectiveness is evaluated by comparing the performance of typically developing children and children diagnosed with NDs. The integer-bound neural network outperforms other methods, paving the way for future research to improve the game’s detection effectiveness [22,43].

To assess neurodevelopmental skills, an interdisciplinary team designed this SG based on theories that measure speech and language, psychomotor, cognitive, psychoemotional, and hearing skills. There are 18 main variables in total, and each domain is measured by the child’s performance on specific tasks within the SG activities. The SG software assesses NDs in various domains, and Table 1 (“Apsou” SG variables used to assess NDs Domains) describes the variables used to assess NDs domains.

Upon completion, the child’s responses are processed and evaluated using a scoring system [43]. The system collects and examines the scores in the 18 variables of the game assessment to create a developmental individualized profile for the child. All variable scores are computed on a scale of 0 to 100.

The SG was developed to collect children’s responses during playtime [53,55], represented as variables that fall into one of these input sources categories: (1) hand moving on the touch screen (ie, forming sequencing procedures, sentence formation, or reordering objects); (2) verbal responding to questions or executing instructions (ie, verbalization after initiation, word recognition [speech to text], used in articulation, pragmatics perception); and (3) clicking buttons and objects or drag and dropping on screen (ie, pictures with animals and shapes)

The SG automatically generated scoring performance for touch screen hand movement responses according to accuracy. The rest of the variables were analyzed as follows.

SmartSpeech uses speech-to-text (STT) to recognize a child’s verbal responses during the SG. Participants were asked to pronounce characters’ names and objects, with predetermined correct answers. CMUSphinx (Carnegie Mellon University), a free, fast, and offline speech-to-text program, was used [56]. A Greek recognition model was constructed and trained in the software [57], detecting the best word to match the child’s input, to determine a correct answer. The software detected the word best matches what the child had said and, consequently, whether the child gave a correct answer or not. The recognized speech responses by the STT software are evaluated to the predetermined set of words that constitute a correct answer and thus turning into scores by assigning 0 to false and 1 to true.

The system is also designed to connect and collect biometric data from the child, such as heart rate and eye-tracking metrics, but these are outside the scope of this study [22,43].

Descriptive statistics were used to analyze typically developing children’s gaming performance. z scores as standardized scores were calculated to provide information about the distance and direction of each observation from the mean value for that specific variable. We examine our variables with a reliability test analysis via the Cronbach α coefficient, which is used to evaluate the internal consistency of our measurements. To reduce the number of original features present in the data set while retaining crucial information, the widely recognized PCA method for dimensionality reduction was used [58]. PCA was used to examine the data structure and determine the main principles for automated detecting NDs. For the analysis following, we used the IBM SPSS Statistics (version 28).

This study conducted data analysis on typically developed preschoolers and first school-aged children, exploring the main principles toward automated detection of NDs and the underlying theoretical principles. The age range of 4-12 years was selected for this study because NDs often become apparent during the early school years and persist into adulthood. It is important to identify these disorders early for timely intervention. During this age range, children reach important developmental milestones in speech, language, cognition, and motor skills. By examining this age group, we can gain insight into standard developmental paths and the natural variations within them, and we can carry out a thorough assessment during a crucial stage of growth. This ensures that the “Apsou” serious game can efficiently recognize and assist children at a pivotal moment. This strengthens the significance and influence of the study on early childhood and primary education.

The PCA resulted in 5 PCs, as per the Kaiser rule. Only the PCs having eigenvalues greater than 1 were retained. These components describe a cumulative variance of 61.44%, which is due to the intraindividual differences in children’s performance across some domains [60].

As noted by Van Geert and Van Dijk [61], “intraindividual variability can be defined as differences in the level of a developmental variable within individuals and between repeated measurements.” Developmental change is often preceded by variability, which plays a crucial role in exploration and selection processes.

It has been observed that individuals exhibit a wide range of performances in different tasks, such as verbalization after instruction as shown in Table 2. These variations in performance can be attributed to individual skillfulness, level of readiness, and the acquisition of different skills and milestones. These findings emphasize the complexity of typical development, which is characterized by variations in child developmental rhythm, abilities, and performances at each stage. These results are consistent with other research findings [1,62,63].

It is widely acknowledged that the distinctions observed in child development indicate various developmental phases. Although children may attain developmental milestones at varying ages, the sequence in which they achieve these milestones can differ among children [60,64].

Besides, these results pinpoint findings on preschoolers and first school-aged children as this study establishes the fundamental developmental skills that support the automated digital system’s ability to classify the child’s performance per developmental stage. Relevant findings regarding developmental age have been reported [54]. Further, children may temporarily sample advanced milestones but may not fully master them until later and can jump between stages [60,64]. Thus, individuals’ actions are inherently unique, contributing to the observed variability in human behavior. Diversity is crucial for children's development as it manifests unique problem-solving approaches, highlighting interindividual diversity. The recorded total variation is expected to underline typical multifaceted heterogeneous profiles of children with NDs.

Furthermore, researchers from various scientific fields have shown a growing interest in early human development [65]. This growth in interest can be attributed to the recognition of the wide range of NDs, which are characterized by significant genetic and phenotypic differences. Additionally, there is a critical need to enhance our understanding of the similarities and differences among various syndromes, disorders, and disease processes. The primary focus of studying early human development and predicting neurodevelopmental outcomes is to examine critical aspects such as the presence and progression of distinct physical and neurological characteristics, as well as potential abnormalities in neurobehavioral and psychopathological patterns.

The results of the current study revealed 5 main new features (PC1-PC5). These PCs are logically aligned with the original features to establish their compatibility with the domains examined in the traditional manner of assessing NDs. According to the results of the PCA, the first 3 PCs explain about 50% of the total cumulative variance. Further, this study’s findings agree with the globally accepted theoretical principles [21] and are also in line with other research work as follows.

The first new feature (PC1) explaining the most variance in the PCA, includes original features (1) pragmatic perception, (2) syntax, (3) sequencing, (4) phonology, (5) memory, (6) perception or discrimination, and (7) recognition with the most important loads. These skills are pointing toward communication skills and are in line with the theoretical principles of NDs (intellectual disability, global developmental delay, communication disorders, ASD, ADHD, SLD, motor disorders, and other NDs) [1,3,21] that demonstrate deficiencies in “… mental functions in conceptual (language, reading, writing, mathematics, memory, insight, knowledge, interpretation), social (empathy, compassion, social judgment, communication and interaction skills, amicability, harmony) and practical (personal care, school and/or occupational responsibilities and organization, financial management, entertainment, hobby) aspects …” [66].

It has been found that the second new feature (PC2) is responsible for explaining the second highest variance in the PCA, which includes the original features of verbal and intellectual ability, articulation, and empathy, with the second most significant loads. PC2 indicates speech and language development and is in accordance with recent research underlying that observing a child’s speech and language development is crucial for their emotional well-being, especially in the early years [1,66]. Additionally, as children grow, their speech intelligibility improves over time. Typically, it starts at around 25% by the age of 2 and gradually gets better until they reach complete intelligibility by the age of 4. However, if a child is constantly struggling with their speech and language, such as using infantile language, mispronouncing words, or lacking spontaneous speech, it may indicate a delay in their speech and language development [67,68].

The third new feature (PC3) explains the third most variance in the PCA, including original features (1) targeted voicing activities, (2) spatial orientation, and (3) cognitive flexibility, with the subsequent most necessary loads. The interaction of these 3 original features indicates vocal processing, which is also described in accordance with another research. It is also considered a fundamental predictor of cognitive development along with language development [69]. For example, the observed modulation in pleasure vocalizations underscores an increasing association between the act of looking toward the caregiver (spatial-orientated emotional communication), while a gradual evolution undergoes because of the simultaneous presence of early spontaneous behaviors (vocal activities) and intentionally driven behaviors (cognitive flexibility) [70].

The fourth new feature (PC4) points to cognitive skills and sensory functions and explains the following most variance in the PCA, including original features: (1) sustained attention, (2) fine motor skills, and (3) conditioned play audiometry, with subsequent most important loads. In the same direction, current NDs commonly result in cognitive deficits such as attention and executive function problems [71]. For instance, measurements of visual or auditory sustained attention, vigilance, and impulsivity in children with ADHD often exhibit executive dysfunction-like behavior, which impairs their development. ADHD children have long-term neuropsychological deficits, including poor performance on visual or auditory vigilance tasks, response inhibition, attention, and goal-directed behavior [72]. In addition, children with developmental language impairments often display minor motor impairments [73] and reduced coding of spoken syllables and phonemes [74], which are frequently overlooked. However, identifying and treating nonlanguage impairments is advantageous to aid the children’s language development (ie, phonological processing) [75] preventing in the long run the development of cognitive deficits [76].

Lastly, the fifth new feature (PC5) suggesting visual-spatial skills, explains the next most variance in the PCA, including original features: (1) prewriting skills and (2) perception or discrimination with the later most important loads. In the same direction, it is well established that deficiencies in visual-spatial and visual-perceptual abilities, and difficulties solving problems early, associated with reading and attention problems, while language-based skills and IQ remain normal, are the hallmarks of nonverbal SLD [1]. Further, consistent with previous research, weaknesses in visual-spatial abilities, including prewriting skills, are reported in children at risk of SLD [77]. Moreover, consistent with other studies, a spectrum of visuospatial abilities (or visuospatial dimension), ranging from gifted to impaired abilities, exhibit early impairments in various areas, including visual-spatial attention [78], visual-constructive, spatial working memory and fine motor skills, and poor performances in educational subjects such as mathematics, as well as emotional and social problems [79].

Using a sample of typically developing children is fundamental to establishing a robust normative baseline against which deviations can be measured. By understanding the typical developmental trajectories and their inherent variability, atypical patterns that signify NDs can be more accurately pinpointed. This differentiation between normal developmental variations and clinically significant deviations ensures that automated detection methods are both sensitive and specific. It is interesting to note that the 5 PCA factors identified in this study were also found in a parallel relevant study [80] involving typically developing and ND children. Apsou SG, factor analysis, and machine learning with logistic regression were used to create a strong predictive model. The model achieved an area under curve of 0.730, accuracy of 0.815, F₁-score of 0.776, precision of 0.823, and recall of 0.815, clearly indicating the predictive potential of these factors.

A shortcoming of this study is the absence of implementation of a diagnostic test of intelligence. To complement this limitation though, we took into consideration that (1) all the participants were attending the typical educational settings and (2) the Apsou clinical administrators confirmed the nonexistence of NDs in participants using the child's developmental and communicative background provided by the parent, and the observation approach applied during their interaction with the child during the game.

The accuracy of the transcription provided by STT software is a limiting factor in this research. We used the highly accurate CMUSphinx STT program [57], trained to convert adult verbal responses to text, but as the software improves over time, also the accuracy of models relying on these transcripts will improve as well. Further research may also use innovative other open sources for this purpose.

Although PCA is a justified method to reveal the most important features of NDs representing 61.44% of the variance, additional methods can be used to explore features validity like deep learning approaches, genetic and molecular data analysis, machine learning and statistical analysis, biological markers, neurophysiological and electrophysiological measures, nonlinear dimensionality reduction techniques, manifold learning, and more.

These findings have the potential to guide future research toward improving automated classification and prediction. This progress can play a crucial role in the timely and precise identification and treatment of NDs in clinical and educational settings. Understanding typical and atypical developmental patterns, including NDs, can significantly enhance clinical practice with advanced AI tools. Future research should combine these findings with cutting-edge AI and machine learning techniques to explore additional features such as biometric data and longitudinal tracking. This combined effort will not only enhance the accuracy and reliability of automated detection systems but also lead to more effective and personalized interventions. These developments will enable more tailored and holistic approaches to supporting individuals with NDs, ultimately improving their long-term outcomes and quality of life. Furthermore, future studies may strive to validate these findings across diverse populations, ensuring broader applicability and generalizability.

In this study, the main principles for the automated detection of NDs were explored. A sample of 229 typical developed preschoolers and early school-aged children played a unique SG,” Apsou,” developed to measure developmental domains.

Using principal components analysis, we unveiled five core principles, signifying (1) communication skills, (2) speech and language development, (3) vocal processing, (4) cognitive skills and sensory functions, and (5) visual-spatial skills, thus corroborating the theoretical principles regarding typically developmental domains. These findings open the door for future research to harness them for further analysis in refining automated screening, diagnosis, prognosis, and treatment planning to support clinical decision-making on NDs.