Depression is one of the most widespread mental health disorders, characterized by low mood, loss of interest in activities, reduced energy, and cognitive difficulties persisting for at least 2 weeks. With 280 million people affected worldwide, it is a major contributor to global disability and disease burden [1]. In the United Kingdom, depression is among the most prevalent mental disorders [2]. Older adults are particularly vulnerable due to age-related health decline, cognitive impairment, reduced social interactions, and changes in family dynamics [3,4]. They also have the highest suicide rates of any age group [5].

Demographic shifts have exacerbated this issue. Over the past 40 years in the United Kingdom, the population aged 50 years and older has increased by 47%, and those aged 65 years and older by 52% [6]. Within this significant increase, depression often remains undiagnosed, leading to a reduced quality of life and increased mortality rates [7]. Therefore, early identification of individuals at risk for developing elevated depressive symptoms is critical for timely intervention, but this requires tools tailored to the risk factors most relevant to older adults.

The risk profile for depression in older adults differs from that of younger populations. While emotional symptoms such as stress are highlighted in younger cohorts, older adults with depression often experience cognitive difficulties, physical complaints, and reduced interest in activities [8,9]. Factors such as loss of independence, poor sleep, chronic diseases, and functional limitations are associated with the depression in older adults but do not necessarily predict depressive symptoms at follow-up [10].

Machine learning’s (ML’s) ability to analyze complex, multidimensional data and uncover subtle, nonlinear interactions has made it a powerful tool for predicting depressive symptoms. Prior studies have applied ML in a variety of contexts, ranging from social media monitoring to structured cohort studies. For instance, social media–based approaches have detected depressive language and emotional distress from user-generated content on platforms such as Twitter (X Corp), Facebook (Meta Platforms), and Reddit (Reddit, Inc), though their applicability to older adults is limited due to lower engagement with these platforms [11-13].

Beyond social media, many studies have focused on predicting depressive symptoms through questionnaires and interviews across different cohorts, including military cohorts [14,15], student populations [16], and occupational cohorts [17,18]. For example, in occupational cohorts, elastic net models have identified key predictors such as gender, anxiety disorders, and adverse life events [18]. Notable longitudinal studies have examined the risk of depression in older adults from China and Canada, incorporating variables such as emotional instability, low life satisfaction, perceived health, and nutrition risk, achieving areas under the receiver operating characteristic curves (AUROCs) ranging from 0.62 to 0.79 [7,19,20]. Overall, these studies show the feasibility of predicting depressive symptoms in older adults; despite this difference in cohorts, predictors, and modeling approaches limit the generalizability of these findings to UK older adult populations.

Recent ML approaches for depressive symptoms prediction in older adults have yielded promising results; however, several methodological limitations remain. First, most studies have been conducted outside the United Kingdom in populations with different health care systems and cultural contexts [7,19,20], limiting their applicability to UK clinical settings. Second, while tree-based methods have shown strong performance [18-20], few studies have systematically compared results across diverse algorithmic paradigms (eg, deep learning vs traditional ML), making it unclear which approaches are optimal. Third, categorical feature encoding strategies, which are particularly important for handling ordinal variables such as self-rated health and mobility limitations, have not been systematically evaluated, with most studies relying on one-hot encoding. Finally, deep learning architectures specifically designed for tabular data (eg, TabTransformer and TabNet) remain largely unexplored for depressive symptoms prediction in older adults, despite their potential advantages for handling mixed feature types. The English Longitudinal Study of Ageing (ELSA) offers comprehensive longitudinal data that can address these gaps. Existing ELSA-based research has examined associations between depression and factors such as frailty, biomarkers, and social conditions [21-23], yet no study has systematically applied ML approaches to predict future depressive symptoms at follow-up.

Compared with conventional statistical methods that rely on linearity and predefined interaction structures, ML approaches can capture nonlinear, complex interactions among heterogeneous features in high-dimensional data. We applied 8 ML models across diverse learning paradigms, including tree-based ensembles, deep learning architectures specifically designed for tabular data, distance-based methods, probabilistic methods, and linear methods. This systematic model selection ensures our findings on encoding strategies are algorithmically robust and generalizable rather than constrained to specific model structures. The objective of this study was to apply these diverse models to predict clinically significant depressive symptoms at 2-year follow-up in older adults using data from ELSA and to identify key risk factors contributing to depressive symptoms at follow-up. The 2-year prediction window was chosen to align with the biannual wave structure of ELSA. This timeframe also provides sufficient time for preventive interventions while minimizing uncertainty from longer-term changes in health status and life circumstances that complicate prediction accuracy.

This study makes 3 key contributions. First, this study provides a systematic ML-based approach to predicting depressive symptoms at follow-up among older adults in the United Kingdom using ELSA data, addressing a gap in UK-specific risk prediction models. Second, through a comprehensive comparison of 8 diverse ML algorithms (tree-based, deep learning, distance-based, probabilistic, and linear models), we demonstrate that ordinal encoding consistently outperforms one-hot encoding for health care datasets with naturally ordered categorical features, with average improvements of 0.013 in AUROC and 0.037 in F₁-score. Third, we integrate multithreshold sensitivity analyses of the 8-item Center for Epidemiologic Studies Depression Scale (CES-D 8 score ≥3, ≥4, and ≥5) with individual-level Shapley additive explanations (SHAP) interpretability to identify robust risk factors that remain stable across depression severity levels, enabling targeted clinical interventions.

The best-performing models achieved AUROC values between 0.72 and 0.75 for 2-year depressive symptoms at follow-up. In this setting, traditional ML approaches generally achieved higher performance than the deep learning models in the ELSA dataset. SHAP analysis highlighted loneliness, sleep disturbances, and low social engagement as the robust predictors across all depression thresholds.

Using data from the nationally representative ELSA study, we developed ML models to predict clinically significant depressive symptoms at 2-year follow-up among adults aged 50 years and above. Our goal was to identify individuals at higher risk for developing elevated depressive symptoms and provide a foundation for potential early interventions.

Although statistical models are highly practical for hypothesis testing, they rely on linear assumptions and predefined interaction structures, which may limit their capability to capture the complex relationships underlying depressive symptoms at follow-up in older adults. In contrast, ML and deep learning approaches can model nonlinear interactions among psychological, social, and health-related domains, enabling a more comprehensive representation of depression risk factors. Applying explainability methods such as SHAP allows us to retain interpretability, despite models’ complexity.

In our study, traditional ML models, such as RF, SVM, and XGBoost, demonstrated better performance than deep learning techniques such as TabNet and TabTransformer. These results may be explained by the nature of our dataset, which contains mostly ordinal categorical features. Traditional ML models effectively handle ordinal data through straightforward encoding techniques that preserve the inherent order and relationships within the features. In contrast, TabNet and TabTransformer use complex embedding strategies designed to capture complex feature interactions. However, they may not fully leverage the ordinal characteristics of the data, leading to weaker performance. Furthermore, simpler models such as KNN and LR showed an insufficient ability to recognize complex patterns in the data, leading to lower predictive accuracy. This suggests that while simpler algorithms can perform well in some cases, their effectiveness may decline significantly when applied to complex, multidimensional phenomena like depression.

Although MNB has a probabilistic nature and assumes feature independence, it still yielded competitive results. This suggests that in certain contexts, when applying appropriate preprocessing steps, probabilistic models are capable of capturing depression risk factors. In our study to meet the model’s requirements, numerical variables were discretized into categorical bins. These findings reveal why understanding a dataset’s characteristics is important for selecting the best model. Choosing a model that is inherently aligned with the dataset’s characteristics can improve both performance and interpretability. Selecting the right model is even more important when dealing with datasets that contain a mix of categorical (nominal and ordinal) and numerical variables.

In contrast to earlier studies, our models achieved AUROC values comparable to or higher than those reported in prior longitudinal work, as RF and XGBoost both reached an AUROC of 0.72 [7,19,20]. However, direct comparisons of predictive performance should be interpreted cautiously considering the differences in cohorts, target definitions, and predictor sets. Zheng et al [20] reported an AUROC of 0.673, while Su et al [19] achieved an AUROC of 0.629 using LR with lasso regularization. In our study, we aimed to identify independent predictors of future depressive symptoms by excluding baseline depressive symptom status from the models to minimize data leakage and preserve predictor independence. In contrast, Song et al [7] incorporated baseline psychological scales as predictors and reported a higher AUROC of 0.791. Although this reflects stronger predictive performance, including baseline symptom measures makes interpretation more challenging, as predictions may partly capture symptom persistence rather than future risk. In our study, participants with clinically significant depressive symptoms at baseline were excluded, which minimizes the risk of directly capturing symptom persistence.

Our feature importance analysis identified significant predictors of depressive symptoms at follow-up, including negative social interactions, sleep quality, positive social and spousal support, self-rated health, and life satisfaction. These findings align with prior research. For example, Zheng et al [20] highlighted sex, self-rated health status, physical pain, and marital status as key predictors, consistent with our results, although occupation and eyesight were not emphasized in our analysis. Further supporting the role of functional health and social engagement in predicting depressive symptoms, Su et al [19] had emphasized the significance of self-rated health, activities of daily living, and marital status. Similarly, Song et al [7] pointed out that perceived health, social support, and life satisfaction are key indicators, emphasizing that depression spans physical, social, and emotional areas. Interestingly, they also found that nutritional risk and emotional instability are significant predictors, factors that we did not focus on in our analysis. These differences may arise from variations in study populations, data collection methods, or feature engineering approaches.

The SHAP force plots provide individual-level validation of the aggregate feature importance findings. Key predictors observed across both XGBoost and RF models, including negative social interactions, future outlook and energy, and activity motivation, emerge as the main factors influencing individual predictions in both cases examined. In the depressed participant, poor self-rated health, low future outlook, reduced activity motivation, and digital exclusion collectively push the prediction toward depression, while in the nondepressed participant, low negative social interactions, positive future outlook, good sleep quality, and varied activity engagement provide strong protective effects. This consistency between population-level feature importance and individual-level explanations improves confidence in the clinical relevance of these predictors and implies that the found risk factors operate meaningfully at both aggregate and individual levels.

The sensitivity analysis conducted demonstrated strong consistency between XGBoost and RF across all thresholds, suggesting that results represent a true clinical situation rather than model-specific artifacts. It is important to consider that CES-D 8 is an abbreviated version of the original 20-item scale. Therefore, a one-point difference in threshold implies a significant change in depression severity. This explains the notable performance variation observed: the AUC score increases as the threshold becomes higher, mainly due to the improved ability of the model to detect well-defined cases with obvious symptom profiles. Conversely, F₁-scores decrease with severity, since this parameter is heavily influenced by prevalence. As the threshold increases, the imbalance degree of the dataset increases, which makes it more challenging to maintain both high precision and recall. The main risk factors, self-rated health, psychological well-being, negative social interactions, remained stable across thresholds in both models, implying these predictors are important regardless of the threshold chosen.

In addition to threshold selection, we conducted sensitivity analysis to examine categorical encoding strategies. The consistent outperformance of ordinal encoding over one-hot encoding supports the suitability of our feature engineering strategy. Ordinal encoding preserves natural ordering of many ELSA variables, such as self-rated health, mobility limitations, and pain severity, while one-hot encoding discards it. It was especially important for tree-based models, such as XGBoost and RF, and distance-based methods, such as KNN, which directly benefit from preserved ordinal relationships. The improved F₁-scores with ordinal encoding lead to better identification of at-risk individuals for early intervention.

In order to address high dimensionality and missing data, some recent works have integrated dimensionality reduction into deep learning architectures. Tutsoy and Sumbul [31], for instance, suggested dimensionality reduction based on correlation and target similarity for the diagnosis of thyroid cancer. This method reduced 39-dimensional biomarker data to 5‐10 features while handling up to 88% random missingness and achieving 83% testing accuracy. However, these methods have limitations when applied to epidemiological cohorts with diverse feature types and are best suited to continuous biomarker data. We focus on encoding-aware preprocessing for longitudinal cohort data like ELSA because, in particular, categorical variables need to be encoded before correlation analysis, since correlation measures might not be directly comparable across categorical and continuous data types, and standard correlation does not respect ordinal structure.

Building on the insights from feature importance analysis, clustering provided an additional layer of understanding. Despite negative Silhouette scores across all subsets, highlighting the inherent difficulty of clustering high-dimensional data, t-SNE visualizations revealed depression-related patterns. Among the subsets, social engagement features exhibited the clearest groupings, emphasizing the critical role of social factors such as social support, which were identified as key predictors in our feature importance analysis. While physical health features showed less clear clustering, reflecting their more indirect relationship with depression severity, demographic features showed moderate separation, matching predictors such as self-rated health and marital status.

Our study has several limitations that should be acknowledged. The use of self-reported questionnaires may introduce reporting bias, as participants may underreport or overreport symptoms and behaviors, potentially affecting the reliability of some predictors. Additionally, depressive symptoms in this study were assessed using the CES-D 8 scale, a widely used screening tool with strong psychometric properties; however, it is not a clinical diagnostic tool and may lead to misclassification, particularly among individuals with subthreshold symptoms. Some baseline predictors, such as sleep quality and physical functioning, may overlap conceptually with CES-D items, which could result in the models partly capturing symptom persistence rather than entirely new symptom emergence. To minimize direct information leakage, participants exceeding the CES-D threshold at baseline were excluded, and all predictors were measured prior to the follow-up outcome. Excluding individuals with severe psychiatric conditions may also limit the ability to generalize the results to high-risk populations. Finally, negative silhouette scores indicate that depression risk in older adults may vary gradually rather than forming clear, distinct groups. Apparent groupings observed in t-SNE visualizations should therefore be interpreted cautiously.

There are several sources of uncertainty in model predictions. Sampling uncertainty arises from missingness associated with older age, lower education, and baseline depression, potentially violating missing-at-random assumptions. Methodological uncertainty results from modeling choices such as threshold selection, hyperparameter tuning, and encoding. Feature-related uncertainty relates to both feature reduction (6100 to 120 variables) and unmeasured confounding such as genetic vulnerability or medication use. The 2-year prediction interval introduces temporal uncertainty, as major life events may occur between baseline and follow-up assessments.

To assess the real-world application of the model, further study should validate the predictive model’s performance using a clinical dataset such as general practitioner records. The predictive analysis in this study relied on single-wave data to forecast depressive symptoms at the subsequent wave, which may not fully capture longer-term temporal dynamics of depression; incorporating information from multiple previous waves could provide a more comprehensive understanding of depression trajectories.

Despite these limitations, our findings demonstrate that ML algorithms using the ELSA dataset are feasible in predicting depressive symptoms at 2-year follow-up in the aging population. The results suggested that incorporating a wide range of features from different domains can enhance the model’s prediction capability. Notably, traditional ML models, such as SVM, RF, MNB, and XGBoost, performed better than deep learning techniques, emphasizing the value of aligning model selection and preprocessing with the properties of the dataset. These findings contribute to a deeper understanding of depression risk and support the development of early intervention strategies and scalable risk stratification approaches for aging populations.