Real-world data (RWD), which encompass clinical and health data, are a fundamental asset in health care, providing invaluable insights for research and clinical applications [1,2]. However, for privacy, legal, collaborative, or practical reasons, RWD are typically fragmented into different data sources, such as electronic health records, patient registries, and other clinical repositories. This fragmentation, characterized by the dispersion of RWD across numerous health care systems and geographic locations, hampers the Findability, Accessibility, Interoperability, and Reusability (“FAIRness”) of the RWD [3]. Thus, the capacity to harness these fragmented data sources for evidence-based health care decision-making is hindered and must be improved [4,5].

While limited data sharing impedes the clinical research progress in general, the impact is particularly significant for research into low-prevalence diseases, where data scarcity already poses a challenge [6]. This issue becomes especially critical in urgent situations, such as the COVID-19 pandemic [7,8], where rapid data sharing is vital to quickly understand the impact of a novel disease.

Nevertheless, many challenges stemming from RWD fragmentation are systemic and can be seen as constraints that are difficult to eliminate. Legal data protection frameworks, ethical considerations, and organizational policies of different countries often limit the dissemination of such data both within and across countries [9-12]. These challenges are further complicated by different data registries using different data formats, standards, and infrastructures to reflect their specific needs [13]. This context raises the need for a global data sharing framework to facilitate the secure and efficient sharing of RWD, while accommodating most of the complexities of the clinical data ecosystem.

A key component of such a global data sharing framework is federated learning (FL), which offers a solution by training machine learning models on distributed data sets without centralization [14,15]. Nevertheless, current FL packages crucial for implementing this system fail to cater to the multifaceted requirements of health care stakeholders.

The FL framework ecosystem is characterized by its heterogeneity, prompted by contributions from both the open-source and industry communities, with each framework casted to address specific facets of FL’s inherent challenges. From enhancing security and privacy to ensuring scalability and promoting framework agnosticism to facilitating ease of use and customization, these frameworks collectively foster a conducive environment for the flourishing of FL applications.

Industry solutions such as IBM FL [16], the Clara Training Framework [17], and Sherpa.ai [18] offer specialized features but suffer from restricted accessibility due to their proprietary nature, limiting widespread application and innovation. Open-source FL frameworks, despite their appeal of broader accessibility, grapple with challenges unique to their design and intended use cases. Syft, also known as PySyft [19], in the context of offering advanced privacy-enhancing technologies, focuses on secure and private data analysis across various domains. Despite it emphasizes structured transparency systems, the complex setup process associated with Syft could impede its broad adoption. As a pioneering framework in the realm of privacy technology, Syft has undergone substantial revisions across its iterations. While aimed at improvement, these continuous overhauls may pose challenges for early adopters in terms of staying abreast with the framework’s evolution.

Frameworks such as FATE [20] and OpenFL [21] are distinguished by their strong emphasis on security and privacy, which are critical in sectors such as health care, where data confidentiality is essential. FATE is noted for its robust security and privacy features. However, it requires substantial infrastructural investment and has a steep learning curve, as there are some discrepancies in documentation clarity and availability across different parts of the framework. OpenFL also offers Intel-enabled trusted execution environment known as Safe Guard and mutual Transport Layer Security for all communications to ensure maximum privacy. However, implementing this framework on a broader scale is hindered and may necessitate specific hardware, software, or the activation of certain services [22]. TensorFlow Federated [23], while offering a specialized toolkit for TensorFlow models, limits its use to those committed to a singular modeling framework, potentially sidelining a broader audience that operates across diverse machine learning frameworks.

Flower [24] and FedLab [24] are recognized for their framework agnosticism and scalability. Flower emerges as a user-friendly and lightweight option, albeit still progressing through its developmental stages with real-world applications yet to be fully demonstrated. This reflects a common developmental trajectory among open-source FL frameworks, where achieving a balance between accessibility, ease of use, and broad applicability poses a significant challenge.

FedML [25] distinguishes itself through its versatility and recent architectural improvements, positioning it alongside frameworks such as Plato [26], known for supporting a diverse array of computing paradigms. This dual focus on facilitating FL research and the development of practical applications underscores its strengths. However, integrating significant updates may necessitate substantial maintenance for applications developed on its platform. Additionally, developers may need to reacquaint themselves with the framework’s updated features, presenting a potential challenge in adaptability and ongoing engagement.

Innovative approaches are evident in PaddleFL [27] and Galaxy FL [28], with Galaxy FL pioneering blockchain technology integration for improved data privacy and model confidentiality. This innovation broadens the FL framework landscape, introducing novel data protection and utilization aspects. However, blending FL with blockchain remains a debated topic within the literature, especially concerning the significant computation and communication overheads [29].

Lastly, FLSim [30] specializes in FL simulations, providing a comprehensive framework that incorporates differential privacy and secure aggregation scenarios. Similarly, Leaf [31] focuses primarily on simulation studies. While this specialization in simulation offers valuable insights, it may also narrow their applicability, potentially restricting their direct translation into real-world deployments.

For a more in-depth discussion of the FL frameworks, we guide readers to Multimedia Appendix 1, where Table S1 in Multimedia Appendix 1 [32-51] offers a comprehensive summary of the highlights and key features of various FL frameworks. Additionally, Table S2 in Multimedia Appendix 1 details the developmental progress and recent advancements within these frameworks, providing a clear snapshot of their evolution and current state.

Beyond the challenges already covered, a recent review of FL in the biomedical domain reveals that only some research studies have provided reusable or reproducible source code alongside their research [52]. This highlights a common issue: the tendency to start a unique project for each separate data sharing initiative. This practice not only consumes valuable time and resources but also hinders the evolution of a universal ecosystem. Furthermore, the opportunity to build upon and contribute to an existing initiative is often overlooked.

Despite recent progress in the field, there is a critical demand for a more inclusive, user-friendly, and holistic framework in health care collaborative research [53,54]. A framework that empowers stakeholders from diverse backgrounds to fully engage in collaborative learning opportunities is essential. This requires FL solutions characterized by intuitive interfaces, streamlined workflows, comprehensive documentation, and a design process that actively seeks and incorporates feedback from a wide range of stakeholders. By prioritizing usability and inclusivity in developing FL solutions, we can significantly lower technical barriers. This approach enables wider participation and amplifies the contributions of collaborative health care research.

In this work, we present Federated Learning for Everyone (FL4E)—a dynamic, data-centric framework designed with inclusivity and adaptability at its core. FL4E is dedicated to efficiently converting RWD into actionable evidence, facilitating engagement from a broad spectrum of stakeholders, and enhancing collaborative learning experiences. Its hierarchical architecture offers a customizable data sharing ecosystem, facilitating a tunable balance between centralization and federation. With its modular approach, FL4E is general and provides an ecosystem-based environment that can support a wide range of collaborative research projects. This modularity further allows all stakeholders to contribute to the project by selectively engaging in the relevant part of the data sharing pipeline.

In this section, we rigorously investigate the fundamental principles underlying FL4E, systematically articulating its core concepts. This entails a meticulous examination of the pivotal modules and essential components that constitute FL4E, with a particular focus on their functionalities and interrelationships. Moreover, this section highlights the architectural nuances of FL4E, comprehensively detailing its structural configuration and the dynamics of its operational framework.

First, the concept of “degree of federation” within the FL4E framework introduces a novel, adaptable, and pragmatic approach to collaborative data analysis designed to meet stakeholders’ varied needs and limitations. This feature enables participants to customize their level of involvement and data sharing based on their specific infrastructural capacities, regulatory requirements, and privacy considerations.

FL4E acknowledges that while federated models offer significant privacy advantages, their practicality may vary due to different constraints. Therefore, FL4E is designed to accommodate various levels of participation for stakeholders, ensuring flexibility if one data sharing stream proves unfeasible.

Therefore, the choice between a centralized or federated model within FL4E is presented not as a dichotomy but as a continuum. On the one hand, centralization may simplify operations and enhance efficiency, but it could compromise privacy to some degree. Conversely, full federation prioritizes privacy, possibly adding operational and analytical complexity. FL4E’s design empowers stakeholders to find a balance that best suits their needs, governance frameworks, and regulatory environments.

This adaptable strategy has proven effective in accommodating diverse data sharing needs, as demonstrated by the Global Data Sharing Initiative for Multiple Sclerosis and COVID-19 [55,56]. The deployment of a versatile hybrid data acquisition system, merging traditional data sharing with federated methods, has not only broadened inclusivity and flexibility but also led to a notable (3527/7757, 45.47%) increase in data collection volume compared to initial efforts [57]. These results underscore the “degree of federation” practical value, showcasing the advantages of integrating such adaptability into data acquisition and sharing frameworks. Different configurations within the degree of federation are depicted in Figure 1.

However, it is essential to emphasize that the framework fosters regulatory compliance across all participation methods. Whether stakeholders prefer the simplicity and direct oversight of a more centralized model or the enhanced privacy of a federated approach, stakeholders are reminded of their collective obligation to adhere to prevailing data protection regulations.

Second, ecosystem-based collaborative learning: FL4E framework introduces a paradigm shift from conventional project-specific FL to an “ecosystem-based” approach, underscoring a strategic pivot toward creating a unified, adaptable platform tailored for conducting various analyses centered on specific diseases or health conditions. At the heart of the “ecosystem-based” methodology is its holistic perspective, aiming to formulate a comprehensive and synergistic environment for data sharing and collaborative research.

An ecosystem, in this context, is envisioned not just as a collection of data or series of disjointed projects but as a dynamic, collaborative network of stakeholders. This includes researchers, health care professionals, patients, and policy makers, all united by a common goal: understanding and addressing a particular health issue. By fostering such an ecosystem, FL4E encourages the pooling of diverse resources, expertise, and perspectives. This collaborative efficiency breaks down traditional silos, allowing for more efficient use of resources through shared data and findings.

Moreover, encompassing a wide array of stakeholders and research questions enables the ecosystem to provide a more nuanced and comprehensive understanding of the health condition under investigation. The diversity of perspectives and expertise enriches the research outcomes, offering holistic insights far beyond what could be achieved in isolation.

This approach is inherently adaptable and capable of evolving in response to new research findings, emerging health challenges, and shifts in the regulatory landscape. Such flexibility ensures the long-term sustainability and relevance of the ecosystem, enabling it to continue generating valuable insights over time.

Regarding its structure, FL4E features a nested design. Within each ecosystem, several studies are carried out, each focusing on specific research questions related to the overarching theme. These studies, while operating under their unique regulatory and consent protocols, incorporate the degree of federation. They can include a data set, data quality scripts, and analyses tailored to their specific research queries.

FL4E’s structure also incorporates simplicity and accessibility in its design. The intuitive user interface encourages contributions from stakeholders, fostering a collaborative environment that is integral to the ecosystem’s success. This systematic approach to collaboration and data sharing is visually depicted in Figure 2.

The FL4E framework builds upon the foundational concepts of the Flower framework [32]. Flower’s flexibility, support for multinode execution, and agnosticism to both machine learning frameworks and programming languages significantly contribute to the overall adaptability and versatility of FL4E.

The FL4E framework is divided into server-side and client-side implementations. Server-side, the framework is a microservice-based web application, designed using ASP.NET Core [58]. It adheres to the N-tier architecture and uses the Model-View-Controller design, along with a repository pattern, leading to a scalable and maintainable application. The application is segmented into 4 layers: Application, Model, Utility, and Data Access. Each layer performs distinct tasks, such as managing user authorization, defining data structures, handling requests, and interacting with the database. ASP.NET Core Identity module is used for authentication and authorization, while an admin control panel supervises user registrations in the interface layer of the framework [58].

Client-side, the framework aims for simplicity and ease of use, also leveraging the ASP.NET Core framework. It incorporates 3 primary interfaces for data mounting, script uploading, and script execution. The data mounting interface facilitates user selection and mounting of data files; the script uploading interface enables users to upload scripts acquired from the server; and the script execution interface allows for the running of these scripts, using the Flower framework to secure the communication between the client and server.

Ensuring the seamless operation and accessibility of the FL4E platform, both the client and server components are encapsulated as Docker containers. This strategic choice underpins the platform’s commitment to ease of deployment, consistency across different environments, and portability, thereby, significantly enhancing the user experience for participants in FL projects.

A fully functional prototype of the FL4E server component has been successfully deployed to Microsoft Azure, showcasing the platform’s robustness [59]. This deployment not only demonstrates FL4E’s capability to leverage cloud infrastructure but also its readiness to support FL projects.

To further acquaint potential users and stakeholders with FL4E’s functionalities and user interface, a comprehensive video demonstration has been made available [60]. This visual guide serves as an invaluable resource for understanding the platform’s operational flow, features, and the simplicity of initiating and participating in FL studies.

For those interested in a deeper exploration or customization of FL4E, the source code for both the server and client components is readily accessible [61]. This openness ensures transparency, fosters a community of collaboration, and enables continuous improvement of the platform by allowing researchers, developers, and data scientists to contribute to its development.

This study conducted secondary analyses of openly licensed repository data sets for demonstration purposes in the results section. No sensitive data were used for the cocreation of the framework. All data set sources are cited and adhere to open licensing agreements. For more detailed information about the data sets, please refer to the original sources.

We evaluated our framework using 2 real-world clinical data sets from FLamby [62]: Fed-Heart-Disease and Fed-TCGA-BRCA. The Fed-Heart-Disease data set contains 740 records from 4 centers, detailing 13 clinical features and a binary heart disease indicator for each patient. We used a logistic regression model for prediction. The Fed-TCGA-BRCA data set, sourced from The Cancer Genome Atlas’s Genomics Data Commons portal, has data from 1088 patients with breast cancer across 6 centers. With 39 clinical features and each patient’s time of death, we used a Cox model to predict the risk of death.

Additionally, the Wisconsin Breast Cancer data set was incorporated into the GitHub repository as an extra use case [61,63]. It serves as a set of foundational scripts that can be generalized and act as guidelines for hybrid experiment settings in the FL4E framework.

To demonstrate how FL4E adapts to various research questions, and how it modulates the degree of federation, we conducted 3 experiments using these data sets:

We evaluated our FL models using area under the receiver operating characteristic curve (ROC-AUC) and accuracy metrics for the Fed-Heart-Disease data set and the concordance index for the Fed-TCGA-BRCA data set. These metrics were chosen according to their ability to assess model performance in binary classification and survival analysis task, while accounting for potential class imbalance. For federated experiments, we implemented multiple FL strategies: FedAVG [14], FedOpt [64] (including FedAdagrad and FedYogi), and FedProx [65]. Hyperparameter tuning was performed using grid search on a centralized setting and applied to each experiment of the data sets. To ensure consistency, each experiment was repeated 5 times. The results of these experiments are summarized in Table 1.

In our analysis of the Fed-Heart-Disease data set, as illustrated in Multimedia Appendix 2 and detailed in Table 1, notable behavior patterns emerge from clients 2 and 3. These patterns indicate the effects of class imbalance and quantity skew, suggesting that smaller clients with imbalanced data sets may benefit more from participating in an FL environment than operating in isolation. This insight is harmonious with the foundational principles of the FL paradigm, which aspires to harness the potential of diverse data sources to enhance model robustness and generalizability.

To clarify the evaluation process, we introduce the notation where K represents the total number of clients participating in the FL process, and n_i denotes the size of the data set for the i-th client, where i ranges from 1 to K. Furthermore, E_i signifies the evaluation metric (such as accuracy or ROC-AUC) obtained from assessing the global model by the i-th client. The sum of the data set sizes across all clients is represented by N, calculated as . Therefore, the overall evaluation metric E for the global FL model is calculated using the formula , offering a comprehensive measure of performance across the federated network.

This approach significantly impacts the evaluation process by prioritizing the characteristics of individual clients over the generalizability of the FL model. This provides a more comprehensive understanding of model performance across diverse environments. However, the potential for skewness in federated evaluation, often stemming from imbalanced clients, arises from its reliance on a weighted average of individual client metrics. This introduces challenges, particularly when directly comparing it to centralized performance.

It is important to note that for a direct comparison between federated and centralized evaluations, the ideal scenario involves assessing the FL model using a separate global test set on the server side. While this approach aligns more closely with the methodology used in centralized evaluations, it is not always feasible, especially in real-world settings where maintaining a separate test set on the server can be impractical due to data governance and privacy concerns.

Contrary to initial expectations that centralized models would outperform others, our analysis unveiled that federated and hybrid models exhibit superior performance based on the ROC-AUC measure, even though the accuracy metrics might suggest a different narrative. To ensure the reliability and consistency of our findings, we have conducted rigorous comparisons with the original benchmark [62]. These comparisons demonstrate that our observations, especially concerning the performance metrics of centralized and federated models such as accuracy, are in line with existing research. This alignment reinforces the validity of our analysis despite the inherent challenges in conducting direct comparisons between federated and centralized performance evaluations [62].

Similarly, the Fed-TCGA-BRCA study substantiates the advantage of FL. The heterogeneity of clients in the data set analysis, as highlighted by FLamby [62], indicates that this diversity significantly influences training performance, underscoring the potential benefits of FL.

Notably, the hybrid experiment yields a crucial insight: it offers performance that matches fully federated models while significantly outperforming local settings for smaller clients. Importantly, this is achieved with considerably less complexity and overhead associated with managing federation across multiple data centers. This setup enables centralized management of in-house data, providing analysts with direct access (within regulatory boundaries) and flexibility for advanced analyses. By having data information at their disposal, analysts can develop more effective scripts, enabling advanced analyses with greater flexibility.