The proliferation of digital technologies has had a profound impact on the health, mental health, and social care sectors in recent years [1-3]. Digital technologies are used in services ranging from stand-alone online interventions to blended models that combine digital and face-to-face encounters, and they leverage a diverse toolset (eg, web-based platforms, mobile apps, extended reality, wearables, and artificial intelligence [AI]–enabled systems) for individuals; families; and specific groups such as children, youth, older adults, and persons with disabilities [4,5]. Therefore, the scope of their application is very broad, encompassing different topics, diseases, populations, and technologies. We use the umbrella term “digital care, assistance, and support” to refer to this integration of digital technologies into the provision, facilitation, or enhancement of health, mental health, social, and family-related services.

Because of their omnipresence in our current society, digital technologies are often also promoted as possible solutions to systemic issues in the health, mental health, and social care sectors, such as unmet needs, resource shortages, and personalized care [6,7]. However, their adoption does not come without barriers and challenges [8]. A major concern is safeguarding the quality and reliability of these technologies, which is often complicated by the rapid pace of technological innovation [9]. Key considerations include data security, system interoperability, accessibility for diverse user groups, and the sustainability of digital platforms [10]. Moreover, professionals must be specialized to effectively use these technologies as digital modalities often require additional competencies compared to in-person care [11]. For example, delivering online mental health therapy demands skills in digital communication, privacy management, and remote engagement. Furthermore, organizations and their managers also experience challenges implementing digital care, assistance, and support [12], for instance, organizing technical support for their professionals and end users.

Although European countries can fall back on a plethora of guidelines, regulations, certifications, and assessments, such as the International Organization for Standardization (ISO) guidelines, European Medical Device Regulation (MDR), European Union (EU) AI Act, and General Data Protection Regulation (GDPR) [13-15], there is no consensus on what really constitutes high-quality digital care, assistance, and support.

In addition, it often remains unclear how such quality requirements can be translated into everyday practice by the different actors involved in digital care, assistance, and support. Therefore, the framework developed in this study was intended not only as an assessment instrument but also as a practical guide for multiple user groups within the field. It can support professionals in reflecting on and improving digitally mediated practices; developers in designing and refining technologies in line with sector-specific quality expectations; and policymakers and organizations in shaping implementation, decision-making, and quality governance. In this way, the framework was conceived as a shared point of reference for the development, implementation, and evaluation of digital care, assistance, and support.

The need for such contextualization is particularly relevant in Belgium, where the organization of care, welfare, and support spans multiple policy and practice levels. In this study, we focused on Flanders because the framework was commissioned by the Flemish government and intended for use within the Flemish policy and practice context. This choice reflected the governance, organizational, and implementation setting in which the framework was meant to operate. The framework should also be distinguished from the Belgian validation pyramid for mobile health, which primarily supports the validation of and reimbursement for mHealth apps [16]. In contrast, this study aimed to develop a broader quality assessment framework for digital care, assistance, and support across a wider range of technologies, settings, and target groups. At this stage, the framework is intended primarily to support self-assessment, quality improvement, implementation, and governance in practice; any future reimbursement or formal policy applications fall outside the scope of this study.

Therefore, the primary objective of this study was to codevelop a consensus-based quality assessment framework for digital care, assistance, and support contextualized to practice in Flanders, Belgium. This focus reflected both the Flemish policy context in which the study was commissioned and the Flemish practical setting in which the framework was intended to be implemented. We pursued this objective through a literature review, a 3-round Delphi process, and stakeholder engagement via interviews and focus groups.

The study consisted of 4 phases (Figure 1). First, 10 expert interviews were conducted with officials from the Flemish government, complemented by a narrative review of peer-reviewed and gray literature sources at the regional, national, and international level. Using this as a starting point, we subsequently conducted a 3-round online Delphi study that aimed to develop a consensus across a diverse panel of experts via an iterative, anonymous, and structured process with controlled feedback [17]. A full draft of the quality assessment framework was also presented and critically appraised in in-depth focus groups and follow-up interviews with specialists in AI, regulation, social work, mental health care, and IT.

The preparatory phase commenced with semistructured online interviews with 10 experts from the Flemish government’s policy domain welfare, public health, and family. Participants were recruited within the steering committee of the project. The interviews focused on ongoing initiatives and anticipated needs within digital care, assistance, and support.

In parallel with the interviews, we conducted an exploratory literature search between September 2024 and October 2025 to identify relevant peer-reviewed studies, regulatory documents, and other contextual sources. We searched bibliographic databases, including PubMed, Web of Science, and Scopus, using broad keyword combinations related to digital health, digital mental health, implementation evaluation, and digital competencies. The search strings included the following: (“digital health” OR eHealth OR mHealth OR “health app*” OR “mental health app*”) AND (evaluation OR assessment OR quality OR framework OR usability); (“digital mental health” OR “e-mental health” OR telepsychotherapy OR “online psychotherapy” OR “video consulting”) AND (implementation OR ethics OR accessibility OR regulation OR review OR sustainability OR adoption); and (“digital competenc*” OR “digital skill*” OR “digital literacy”) AND (“social worker*” OR “health care professional*” OR clinician* OR therapist*).

This search was complemented with gray literature, including institutional websites, policy documents, and selected practice-oriented materials, to help contextualize the academic literature and address differences in the availability of quality assessment publications across subfields of digital care, assistance, and support. The aim was two-fold: (1) to determine a limited number of frameworks that could offer a starting point for the all-encompassing quality assessment framework and (2) to find additional relevant sources that have the potential to contextualize and tailor these frameworks to end up with an all-encompassing quality assessment framework.

In parallel with round 3 of the Delphi process, online focus groups and individual interviews were conducted to test topic-specific coverage and implementation issues alongside the Delphi study. Participants were selected for their relevant expertise and involvement in digital care, assistance, and support across policy, practice, technology, and ethics or legal domains.

Before their session, participants received the draft framework and provided annotated comments; completed the “minimum,” “optional,” or “unnecessary” classification used in the Delphi study; and answered brief open-ended questions. Sessions opened with informed consent and a short overview of the framework, followed by a structured discussion on (1) comprehensibility (goals, structure, and wording), (2) completeness and consistency of the criteria (including ethical and legal aspects), and (3) implementation feasibility (roles, support needs, and tools). Recordings, annotations, and researcher notes were collated into session summaries; where changes to the framework were warranted, they were implemented and explicitly flagged when diverging from the original source frameworks.

For the expert interviews, a thematic analysis was conducted. Additionally, the results of the literature review were derived using an exploratory thematic analysis in Citavi (Swiss Academic Software GmbH) [19], complemented by a categorization based on source type, thematic focus, geographical scope, and data characteristics. The various rounds of the Delphi study were assessed through a combination of thematic analysis, clustering techniques, and descriptive statistical methods.

Ethics approval from a medical ethics committee was not required under Belgian national law as operationalized by the Ethics Committee Research University Hospitals Leuven/KU Leuven [20]. All participants provided informed consent for each data collection wave of the Delphi study and before each focus group. All participants who wished to receive a voucher of EUR €30 (US $34.76) for Bol, a Dutch-Belgian online retail platform, were provided with one after completing their participation. Data were processed in accordance with the GDPR [15] and in line with the privacy policy of Thomas More University of Applied Sciences [21].

The semistructured online interviews with 10 government experts found respondents calling for clearer regulation, guidance, and a coherent framework. They also stressed the need to preserve room for locally tailored digital services. Experts widely judged the current approach to be insufficient, citing gaps in knowledge and expertise, limited support, and the absence of an overarching framework. Key concerns concentrated on legal compliance (eg, GDPR) and secure communication. They furthermore stressed accessibility and inclusiveness as nonnegotiable quality criteria, noting barriers related to language, culture, age, disability, and digital literacy. Collectively, the findings pointed to the need for a practical, inclusive, and codeveloped framework with service providers that accounts for sector-specific contexts and diverse target groups.

In the subsequent literature search, 303 sources were identified and coded by source type, thematic focus, geographic scope (regional, national, and international), and data characteristics (Multimedia Appendix 2). From this, 3 overarching pillars emerged to structure the draft framework: technology, organization, and professional competencies. As became clear from the literature and the interviews, the quality of digital care, assistance, and support depends not only on the quality of the technology but also on its contextualization and on the training of professionals. To substantiate each pillar, we anchored the draft in three established source frameworks: (1) ISO/TS 82304-2:2021, “Health software—Part 2: Health and wellness apps,” an internationally recognized ISO framework for assessing the quality and reliability of health apps; (2) the organization domain of the Health Technology Assessment Core Model, a European framework developed within the European Network for Health Technology Assessment to address organizational conditions, workflows, and implementation factors relevant to health technologies; and (3) the framework for digital competences of professional social workers, a practice-oriented framework developed in the Flemish and Belgian social work context that specifies the digital competencies required for blended and digitally supported professional practice [12,22,23].

These 3 source frameworks were selected because they are established and authoritative within their respective domains; are recognized at the international, European, or local professional field level; and provide a sufficiently strong conceptual and empirical basis for developing a framework tailored to the broader Flemish context of care, assistance, and support.

Concurrent with round 3, we engaged experts across policy, well-being practice, technology, and ethics and legal domains in 4 online focus groups and interviews to assess topic coverage and implementation, focusing on comprehensibility, completeness, consistency, and implementation feasibility.

The focus groups largely confirmed the structure and content of the framework while highlighting clarifications needed for usability and proportional adoption. Participants called for a sharper articulation of target users and use cases, a strong separation between binding minimum requirements and guiding optional criteria, and practical guidance tailored to organizational and developer contexts. They also emphasized that the framework should not be understood solely as a formal assessment tool but also as a practical guide to support reflection, implementation, and quality improvement in practice. Input from experts with backgrounds in law and ethics emphasized the need for (1) direct legal signposting rather than paraphrasing; (2) technology-neutral and use case–specific wording to avoid rapid obsolescence; and (3) a proportional, comply-or-explain orientation that permits reasoned trade-offs across principles while preserving legal certainty. Input from technological experts underscored the need for (1) clear accountability and maintenance expectations across the life cycle, (2) feasible baselines for small or medium-sized enterprise and start-ups, and (3) signposting to recognized norms (eg, interoperability and accessibility) alongside lighter risk management baselines as stepping stones. Finally, well-being professionals stressed (1) the translation gap between technical language and day-to-day practice, (2) the central role of service users, and (3) the value of supporting roles to embed the framework. Common implementation needs converged across groups: training professionals, governmental support and resources, simple implementation tools (eg, checklists, templates, and audits), and a feedback loop after launch.

The interviews brought forth similar themes and added emphasis on alignment with EU-level initiatives (eg, the MDR, EU AI Act, GDPR, and European Health Data Space regulation [13-15,24]) and the expectation that technology providers and developers justify state-of-the-art technical and organizational measures without hard-coding specific technologies.

Building on the combined input from Delphi round 3 and these focus groups and interviews, the framework was adapted. This adaptation reflected a qualitative streamlining: 9 new criteria were added (eg, on avoiding dark patterns, affordability for clients, and explicit alignment with EU legislation such as the European Health Data Space), whereas conceptually overlapping criteria were merged (several clusters of 2 to 3 similar items combined into 1 broader criterion) and 9 redundant items were removed because they were fully covered elsewhere. As a result, the final framework is slightly shorter but more precise and up-to-date in legal terms.

All consolidated efforts resulted in a contextualized quality assessment framework for digital care, assistance, and support in Flanders, Belgium (Multimedia Appendix 1) with a final set of 112 criteria (n=53, 47.3% in the technology pillar; n=23, 20.5% in the organizations pillar; and n=36, 32.1% in the professional competencies pillar), with 77 (68.8%) minimum and 35 (31.3%) optional items. Three established source frameworks were taken as a starting point and modified using input from key stakeholders. All these modifications were made to be traceable: each departure from or addition to a source framework is explicitly referenced (either to existing literature or to insights, requests, or recommendations from stakeholders during the course of this study) to ensure provenance, auditability, and transparency regarding the rationale for adaptation. Beyond its assessment function, the resulting framework is intended to support reflection, quality improvement, implementation, and decision-making across different contexts of digital care, assistance, and support.

This study aimed to develop a contextualized quality assessment framework for digital care, assistance, and support in Flanders, Belgium. Drawing on initial interviews with policymakers and a literature review, a 3-round Delphi study with key experts was conducted and complemented by focus groups and interviews. Together, these methods yielded a framework that synthesizes perspectives across 3 pillars: technology, organizations, and professional competencies.

The results of the Delphi study demonstrated a relatively balanced distribution of importance across the pillars, with technology receiving slightly higher priority. This balance underscores the interdependence of technological robustness, organizational aspects, and professional competencies in ensuring the quality of digital care, assistance, and support. Furthermore, the classification of criteria into minimum and optional highlights a dual function of the framework: on the one hand, it provides essential requirements to safeguard quality, and on the other hand, it creates a growth path for continuous improvement. This graded approach enhances the usability of the framework for diverse stakeholders, ranging from small organizations to larger institutions.

Throughout the focus groups and interviews, participants emphasized the importance of the introductory section of the framework, particularly the central role assigned to the user. This focus on user needs, autonomy, and accessibility aligns with broader European and international discourses on digital inclusion and human-centered design [25]. Moreover, the typology based on risk and interactivity offered a pragmatic lens for contextualizing the application of criteria even though participants expressed concerns about its complexity and operationalization for the entire framework. Experts also highlighted the necessity of clarifying terminology, ensuring a clear distinction between binding and guiding criteria, and aligning the framework with existing regulations (eg, GDPR, EU AI Act, and MDR). Importantly, the inclusion of frontline practitioners stressed the need for translation into practice-oriented tools and language. This reinforces that the framework not only provides conceptual clarity but also that derivative instruments were developed.

When compared with existing European assessment and reimbursement frameworks for digital health, this framework shows both substantial overlap and a distinct contribution. At the level of technology assessment, the core domains identified in this study are largely consistent with those found in other frameworks already in use. For example, the German digital health application fast track assesses product qualities such as data protection, information security, interoperability, quality, and user-friendliness in addition to evidence of a positive health care effect [26]. The Belgian validation pyramid similarly focuses on CE marking, GDPR compliance, interoperability, connectivity, usability, and reimbursement requirements for mobile health apps [27]. In Finland, Digi-HTA combines traditional health technology assessment domains such as safety, effectiveness, and cost with digital-specific domains, including usability and accessibility, data security and protection, interoperability, and technical stability [28]. Similarly, the French Prise en Charge Anticipée Numérique (PECAN) pathway is primarily oriented toward reimbursement of sufficiently mature digital medical devices, with emphasis on clinical and/or organizational benefit, as well as compliance with data protection, interoperability, and security requirements [29].

The added value of this framework lies not only in broadening the scope of assessment but also in confirming the relevance of these established technology-related domains within the wider field of digital care, assistance, and support. The expert input gathered in this study indicates that criteria such as usability, interoperability, data protection, and security remain highly relevant in this broader context. At the same time, the framework extends beyond a narrow product assessment or reimbursement logic by integrating 2 domains that are much less explicitly elaborated on in most existing frameworks, namely, organizational conditions and professional competencies. In that sense, it conceptualizes quality as a contextual, practice-based, and sociotechnical issue. This is particularly relevant for digital care, assistance, and support, where quality depends not only on the technical properties of a tool but also on how it is embedded in organizational workflows, governance arrangements, and the capabilities of the professionals who use it. A further contribution of the framework is its contextualization to the Flemish care, welfare, and support landscape while remaining aligned with broader European legal and regulatory requirements. Within the scope of this study, the framework was developed primarily to support quality assessment, self-reflection, implementation, and quality improvement in practice rather than as a reimbursement model. At the same time, its possible future policy applications fall beyond the scope of this study.

A key strength of this study and the resulting quality assessment framework is its interdisciplinary and participatory approach. The diversity of expertise in the Delphi panel, focus groups, and interviews facilitated the development of a shared language and common understanding across different domains of digital care, assistance, and support, whereas the iterative, participatory feedback fostered high engagement and ownership among participants.

Nevertheless, limitations must be acknowledged as well. First, the starting point of the quality assessment framework was based on an exploratory literature review rather than a formal systematic review. Although this approach allowed us to include a broad range of peer-reviewed, regulatory, and gray literature sources, the search and, more specifically, the selection of the source frameworks could have been more structured and standardized. A second limitation is that the time-intensive nature of the process may have deterred broader involvement of stakeholders. This was also reflected in the Delphi method, which does ensure consensus but saw participation rates decline across rounds, potentially limiting representativeness. A third limitation is that the framework is contextually grounded in Flanders. Although it remains sensitive to international developments, this may limit its applicability to other regions as adaptation to local context remains key for technologies and frameworks alike [30]. Nonetheless, for the technology pillar, another recent adaptation of the European Committee for Standardization ISO/TS 82304-2 standard in Catalonia has demonstrated that this can be done in a relatively straightforward process [31].

Furthermore, it remains to be seen how the quality assessment framework will be implemented in practice. Real and lasting impact will only be achieved through proper governance strategies. Finally, digital care, assistance, and support is a rapidly evolving field that necessitates periodic updates to maintain the relevance and validity of the framework. This underscores the need for a structured governance approach that provides clear roles, oversight, and periodic review cycles to keep the framework current and trustworthy.

The framework provides a valuable foundation for assessing and guiding the quality of digital care, assistance, and support in Flanders. To support its implementation, accompanying tools have already been developed, including a practical self-assessment instrument and governance guidelines. These resources aim to facilitate translation into daily practice by offering organizations concrete means to apply the framework in a structured and proportional way. Nevertheless, successful adoption will also depend on adequate governmental support, allocation of resources, and the establishment of sustainable feedback mechanisms. Future research should focus on piloting these tools and the framework in diverse real-world contexts, evaluating their feasibility and impact, and identifying strategies for keeping them up-to-date in light of rapid technological and societal developments. In addition, future studies could explore how the framework may be updated or adapted to the specific contexts and needs of other regions.