The data used in the present research were collected between 2019 and 2022 in a German psychotherapeutic inpatient clinic, which specializes in treating patients from a range of diagnostic groups. The clinic’s standard diagnostic procedure was administered to all patients, which provided the foundation for the data used in this study. The Klenico system was added provisionally to these standard procedures as an extension to the usual care.

Upon admission, all inpatients received their routine diagnoses from the clinic based on an initial interview and received routine clinical self-report questionnaires (Beck Depression Inventory [BDI-II], Patient Health Questionnaire [PHQ]–9, PHQ-15, PHQ–Generalized Anxiety Disorder-7 [GAD-7], and Satisfaction With Life Scale [SWLS]) to further clarify their symptoms. Afterward, all patients were informed about the procedure and were asked to sign informed consent to allow their routine data and data from the Klenico assessment to be used for scientific studies and publications. Patients then received access to the Klenico self-report module.

Patients then completed the Klenico self-report module on a device of their choice (computer or tablet) and the clinics’ routine self-report questionnaires within the first 2 weeks of their stay. The German version of the Klenico system was used. After completion, mental health professionals conducted the Klenico clinical validation module together with the patient in their next session. At discharge, the procedure was repeated while patients again were asked to answer the clinic’s routine self-questionnaires and the Klenico self-report module. Additionally, the Klenico clinical validation module was performed again. No diagnoses were made in the data collection based on the Klenico system.

Mental health professionals consisted of licensed psychotherapists, psychotherapists in training under supervision (with a minimum of 1 year of professional experience), and cotherapists. Professionals received an introductory session and a demonstration of the use of the Klenico system prior to their initial diagnostic validation. They were also provided with instructions on how to interpret and elucidate the results. The data were fully anonymized.

Fully anonymized data from 496 adult inpatients with mental disorders were used (337 female patients and 159 male patients). Data from discharge were available for 59 patients. Data from the Klenico clinical validation module were available for 445 patients at admission and 42 patients at discharge. The mean age was 38.9 (SD 15) years, and the mean duration of stay was 70.2 (SD 38.6) days. A total of 391 patients received a diagnosis of depression. The frequencies of the different depression diagnoses are reported in Table 1 (for a full overview of all given diagnoses, see Multimedia Appendices 2 and 3 [somatic diagnoses]).

Data analysis was performed using RStudio software (version 2021.9.1.372; Posit, PBC). P values <.05 were considered statistically significant.

Exploratory factor analyses (EFA) were conducted to explore the underlying factor structure of the KDD using the lavaan, SEMtools, and psych packages. Kaiser‒Meyer‒Olkin (KMO) and Bartlett tests were used to test the adequacy of the factor analysis using the functions cortest.bartlett and KMO. Parallel analysis and fit estimation were used to identify the optimal number of factors using the functions efaUnrotate and fa.parallel. A maximum likelihood estimator with robust standard errors was applied. Oblique rotation was applied by using the oblqRotate function (the R code for the EFA can be found in Multimedia Appendix 4). The model fit was considered adequate when the comparative fit index >0.90, Tucker‒Lewis coefficient >0.90, standardized root mean square residual <0.10, and root mean square error of approximation <0.08 [53]. Items were assigned to a specific factor if they demonstrated a loading greater than 0.30 on that factor.

Convergent validity for the KDD was tested by performing Pearson correlation analysis with both the PHQ-9 and BDI-II, while divergent validity was assessed by comparison with the PHQ-15, GAD-7, and SWLS. An r value from >0.1 to <0.3 was considered to indicate a small effect, from ≥0.3 to <0.5 was considered to indicate a medium effect, and ≥0.5 was considered to indicate a large effect [54]. Bonferroni-adjustment was performed to correct for multiple testing.

Sensitivity to change of the KDD was examined by categorizing each patient into groups of either improved or unchanged symptoms based on changes in the PHQ-9 or BDI-II sum scores. KDD means for admission and discharge were compared for both improved and unchanged symptom groups by paired 2-tailed t tests. Effect sizes and standardized response means were calculated. Additionally, repeated measures correlations between the KDD and the BDI-II and between the KDD and the PHQ-9 were calculated using the rmcorr package [55]. Bonferroni adjustment was performed to correct for multiple testing.

A dropout analysis was conducted to compare patients who completed the Klenico system and the PHQ-9 and BDI-II questionnaires at discharge with those who did not complete these assessments. The analysis examined potential differences in demographic characteristics, questionnaire scores (KDD, PHQ-9, BDI-II, and SWLS), and length of stay using 2-tailed t tests or Wilcoxon rank sum tests. This approach aimed to identify any systematic biases between the groups.

Post hoc power analyses were conducted to assess the statistical power of the results (using the functions pwr.r.test, pwr.t.test, and power.rmcorr).

Ethical approval was not required for the studies because according to German law, no ethical approval was required for the use of the Klenico system in routine diagnostic procedures. With reference to the Council Directive 93/42/EEC of the European Union and the according German legislation (Medizinproduktegesetz (MPG) § 23b), the test-related use of a medical device with certification (Conformité Européenne marking) does not require an ethics vote if it is carried out within the scope of its intended purpose [56]. Furthermore, the use of the Klenico system in routine diagnostic procedures falls within the scope of the responsibility of the attending physician (Berufsordnung der Ärztekammern §15). According to the ethics committee of the Ludwig Maximilian University Munich, such data collection with the aim of evaluating the performance of an accepted, evidence-based practice (quality assurance) does not require ethical approval. Finally, no ethical approval was required for analysis of the data, since retrospective studies using already available, anonymized data are exempt from needing ethics approval, according to the guidelines of the institutional review board at the Ludwig Maximilian University Munich. The study was conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study. Data were anonymized. No compensation was provided.

Excellent internal consistency was demonstrated by the KDD both at admission (Cronbach α=0.91) and at discharge (Cronbach α=0.93).

A KMO value of 0.91 and a significant Bartlett test indicated eligibility for EFA. Parallel test and fit estimation favored a 7-factor solution (Multimedia Appendices 5 and 6). The 7-factor model revealed good fit (χ²₂₆₉=382.912, comparative fit index=0.971, Tucker‒Lewis coefficient=0.950, standardized root mean square residual=0.025, and root mean square error of approximation=0.031). Based on the rotated loadings presented in Table 2, the 7 factors were interpreted and labeled as “inadequacy,” “sleep disturbance,” “affect,” “anhedonia,” “somatization,” “appetite decrease,” and “self-hatred.” The item “feelings of worthlessness” was cross-loaded onto 2 factors. Four items (“psychomotor agitation,” “psychomotor retardation,” “inner tension,” and “indecision”) did not load adequately on any factor.

Regarding convergent validity, the KDD showed very high correlations with both the BDI-II (r=0.70; P<.001) and PHQ-9 (r=0.68; P<.001). Regarding divergent validity, moderate associations were apparent with the PHQ-15 (r=0.35; P<.001). In contrast, the GAD-7 (r=0.51; P<.001) and the SWLS (r=–0.51; P<.001) were strongly correlated with the KDD.

The KDD mean scores of the improved symptom group (n=28) demonstrated a significant difference (t₂₇=5.36, P<.001) between admission (mean 36.85, SD 16.43) and discharge (mean 23.90, SD 18.54), with a high effect size (Cohen d=0.79) and a standardized response mean of 1.01. The means of the other measurements can be found in Multimedia Appendix 7. Values of the unchanged symptom group could not be reliably calculated due to the small group size (n=4). A total of 27 patients were excluded from the analysis due to missing BDI-II and PHQ-9 scores or deteriorated symptoms. Additionally, the KDD revealed a high repeated measures correlation with both the BDI-II (r=0.61; P<.001) and the PHQ-9 (r=0.59; P<.001). The dropout analysis revealed significant differences between patients who completed the Klenico system and standard questionnaires at discharge and those who did not. Completers showed higher scores in the KDD, BDI-II, and SWLS, and a longer length of stay (see Multimedia Appendix 7 for detailed results). Post hoc analysis revealed statistical power above 0.80.

This research assessed the validity of the KDD by focusing on its construct validity, reliability, and sensitivity to change.

Regarding KDDs’ underlying structure, the EFA suggested a 7-factor model (Table 2). The items on “psychomotor retardation” and “agitation” as well as for “inner tension,” “indecision,” “crying,” “circadian rhythms: worse in the morning,” “loss of libido,” and “suicidality,” did not adequately load on any of the factors.

The 7 factors of the KDD roughly reflect the 10 diagnostic criteria for depression as outlined in the classification of ICD-10. Table 3 summarizes which KDD factors reflect which ICD-10 diagnostic criteria. The KDD factors “inadequacy,” “sleep disturbance,” “self-hatred,” “anhedonia,” “affect,” and “appetite decrease” each reflect a single ICD-10 criterion. The factor “somatization” maps on 3 criteria.

These results indicate that the ICD-10 criteria for depression are subject to a complex and heterogeneous structure composed of different symptom dimensions. This is underlined by the fact that the latent structure of depression is widely disputed [57]. A recent study on the factor structure of the DSM-IV major depression criteria showed comparable results to our findings, as it demonstrated a multidimensional symptom structure with 5 factors for the 24 criteria [58].

The multiple mappings of the “somatiziation” factor on the ICD-10 criteria and the inadequate loading of some items can be explained by the sample-specific characteristics, the Klenico system, and the structure of depression.

The factors “somatization” and “appetite decrease” reflect those ICD-10 criteria categorized as somatic symptoms in depression that have been assigned to a specific somatic factor in various previous studies [57]. This confirms that the heterogeneous group of somatic symptoms has an underlying common structure that is separate from the rest of the affective and cognitive symptoms of depression.

A possible explanation for the split into 2 separate somatic factors is that loss of appetite and the resulting weight loss are related to the overall severity of depression and may be determined by the severity of other symptoms, such as anhedonia, which is usually most pronounced in severely affected patients, potentially leading to a small variance and a possible floor effect [59]. Finally, the separation found between general somatic symptoms and sleep disturbance does not contradict the factorial validity of the KDD, since previous studies have shown that the 2 symptom dimensions impose separate factors [58].

Effects comparable to those at the factor level may also have occurred at the item level. The lack of significant factor loadings for “psychomotor agitation” and “retardation” may be due to their inclusion only in the clinical validation and not in the Klenico self-report module. These symptoms are more accurately assessed through external observation rather than self-report and are often more pronounced in severe depression [23,60], which can lead to reduced variability and a resulting floor effect in the data. This smaller variance might hinder their ability to align clearly with a specific factor in the analysis. The same reasoning accounts for the items for “suicidality” and “loss of libido,” both of which were just below the cutoff of 0.3—“suicidality” with loads just not on “affect” and “loss of libido” with loads just not on “somatization.” Their prevalences are correlated with the severity of depression [59,61], and acute suicidality is mainly an indication for treatment in a psychiatric clinic. Since the present sample comes from a psychotherapeutic clinic in which suicidality is an exclusion criterion, it is to be expected that such patients were less present.

“Inner tension,” “indecision,” “circadian rhythms: worse in the morning,” and “crying” did not load strongly on a single factor, instead showing distributed loadings across multiple factors. “Inner tension” aligns with both the “sleep disturbance” and “somatization” factors, indicating its role in exacerbating both sleep issues and physical symptoms. “Indecision” correlates with the “somatization” and “inadequacy” factors, highlighting its connection to cognitive impairments and reduced self-confidence typical in depressive episodes. “Circadian rhythms: worse in the morning” describes the symptom of morning mood lows in depression. Due to the wording, it is likely that this item is answered in a rather undifferentiated way and can therefore be associated by patients with both poor sleep and physical effects. “Crying” is a universal symptom that may not load strongly on any specific factor, likely due to its general association with various forms of distress across the depressive spectrum. It possibly associated with multiple factors such as “affect” and “self-hatred,” reflecting its role as a common expression of emotional pain, sadness, and feelings of low worth. This pattern suggests these items intersect various dimensions of depression, making their specific factor alignment more complex. Overall, the findings confirm that depression is a multidimensional construct and suggest that the KDD is determined by the heterogeneous structure of the ICD-10 criteria. Since the structure of the KDD faithfully mirrors the ICD-10 criteria, this offers initial support for its factorial validity.

Concerning convergent validity, large associations were found between the KDD and the corresponding BDI-II and PHQ-9, as expected. Additionally, the KDD showed medium associations with divergent PHQ-15. Exceptions were the GAD-7 and SWLS, with high-range correlations.

Similar to the structure of depression, the factor structure for different diagnostic tools is widely debated. The number of factors found for the BDI-II ranges between 2 and 4 [44] and for the PHQ-9 ranges between 1 and 2 [62]. For both, various compositions of the different items loading on the different factors were found. However, although the KDD reflects the factor structure according to ICD-10 criteria, it showed significantly high associations with these 2 instruments, thus indicating the measurement of a comparable construct of depression.

The results regarding divergent validity are not surprising, as comorbidity between depression and various anxiety disorders is highly frequent [63], and depressive symptoms show structural overlap with anxiety disorders [64]. This is further supported by other studies demonstrating strong intercorrelations between the GAD-7 and PHQ-9 [65], as well as significant correlations between the GAD-7 and BDI-II. [66]

Moreover, depressive disorders are strongly negatively associated with life satisfaction and subjective well-being [67]. The strong negative correlation with the SWLS aligns with theoretical expectations and highlights the substantial impairment in life satisfaction that accompanies depressive symptomatology.

The results therefore do not counteract the construct validity of the KDD but rather support its ability to capture the broader context of psychological distress and well-being.

Regarding sensitivity to change, the KDD was able to detect improvements in symptom severity according to PHQ-9 or BDI-II sum scores and showed strong associations with changes in the BDI-II and PHQ-9 sum scores. This demonstrates that the KDD is highly sensitive to symptom change and is therefore able to monitor treatment outcomes, similar to the commonly used self-report questionnaires.

Overall, the results show that the KDD is a reliable and valid instrument for the diagnosis of depression according to ICD-10 criteria while being able to adequately recognize improvements in symptom severity. In contrast to routine diagnostic procedures, the KDD enables a state-of-the-art diagnostic procedure that includes both self-report and clinical validation by experts and is additionally able to assign comorbidities by covering all relevant mental disorders (n=12).

A major strength of this study is that the KDD was tested in a large clinical sample under real-world conditions. It could thus be demonstrated that the KDD measures adequately in patients with frequent comorbidities in clinical reality, providing real-life evidence of its validity.

A limitation of the study was the small sample size at discharge. Despite the fact that the post hoc power analysis has shown sufficient power for the t tests and the repeated measurement correlation, the findings regarding the sensitivity of change should be considered preliminary. Furthermore, the analysis of the sensitivity of change in the unchanged symptom group was not feasible due to the limited sample size and requires further research. In addition, the patients who used the Klenico tool at discharge exhibited greater symptom severity than those who did not. A reason for this higher representation may be that those more heavily burdened patients perceived greater relevance in completing the exit assessment, leading to higher motivation. The composition of the sample also imposed certain limitations. On the one hand, less severe cases were to be expected in the psychotherapeutic setting, and on the other hand, the design of the study could further lead to the possible exclusion of severe depression cases. This was shown by the fact that, for example, “severe depressive episode with psychotic symptoms” were extremely rare, but also “mild depressive episode” were relatively uncommon. Since the completion of the Klenico assessment requires a certain level of motivation and cognitive abilities, patients affected by severe concentration problems and anhedonia may have faced difficulties in participation. These circumstances led to certain restrictions in the factor analysis, for example, the nonloading of the item “psychomotor retardation.” Additionally, the sample exhibited a gender imbalance, with a majority (68%) being female. This mirrors the real-world scenario where women are more commonly affected by mental disorders compared with men [68]. However, this imbalance could also impact the outcome, given that different genders may exhibit variations in the manifestation of depressive symptoms [69]. Moreover, the age distribution of our participants, concentrated primarily in the younger adult segment and in the segment aged around 55 years, suggests a demographic skew that may have influenced the clinical profile observed. For these reasons, the use of the psychotherapeutic sample in this study might restrict the generalizability of the findings to broader populations, particularly individuals with severe depression. Therefore, it would be valuable to assess the functionality and generalizability of the KDD in different settings that involve patients with more severe symptoms and with a balanced gender and age distribution. This would provide a more comprehensive understanding of the measure’s effectiveness across diverse populations.

This analysis focused on examining Klenico assessment of symptom severity, and Klenico-suggested diagnoses were not examined. Therefore, this should be added in future analyses. Finally, the current validation was limited to the German version of the Klenico system. Further validation studies are necessary to assess the system in other languages, such as English. While the EFA provided initial insights into the factor structure of the KDD, these results need to be confirmed through a confirmatory factor analysis. Additionally, the factor structures of other Klenico domains also require assessment to ensure the comprehensive validity of the system across different mental health conditions.

The KDD presents itself as an accurate tool for measuring depressive symptoms and a valuable tool for measuring depressive symptoms and monitoring treatment outcomes in psychotherapeutic settings. As an integral part of the Klenico system, it comprehensively covers the whole range of depression symptoms according to ICD-10 and might therefore be a comprehensive alternative for the diagnosis and course measurement of depression. The current assessment represents an important next step in validating the psychometric properties of the Klenico system. Therefore, it may help to avoid misdiagnosis and incorrect therapies through standardized and evidence-based diagnostic procedures, thus reducing health care costs and minimizing the burden on the patients.