This study used a qualitative, exploratory research design to conduct a formative evaluation of a prototype tool for assessing the quality and safety of VCs in primary care. We followed the COREQ (Consolidated Criteria for Reporting Qualitative Research) to support clear and comprehensive reporting of the study (Multimedia Appendix 1) [20].

This study captured the views of GPs with experience delivering VCs in primary care settings. Participants were eligible if they were GPs, either fully qualified or in training; working in primary care settings; and had experience conducting VCs via telephone, video, or asynchronous online platforms. No minimum recency or frequency of VC experience was specified as an eligibility criterion. Recent VC activity was characterized by participants’ reported number and proportion of VCs conducted in the preceding 12 months. Participants were also required to have sufficient English fluency to participate in an online group discussion conducted in English.

Participants were recruited using convenience and snowball sampling strategies; no purposive sampling was applied. A recruitment poster and invitation email were distributed via the researchers’ contact networks, professional networks, academic collaborators, and social media. The final sample comprised GPs practicing in the United Kingdom (n=7, 50%), Portugal (n=3, 21.4%), Türkiye (n=2, 14.3%), Greece (n=1, 7.1%), and Ireland (n=1, 7.1%). Recruitment took place between May and June 2025.

A conceptual framework for assessing the quality and safety of VCs in primary care was previously developed by our team. The framework was derived from a global survey of more than 1600 physicians across 20 countries [18] and validated through international expert consensus [19]. A mock web-based prototype was subsequently developed to operationalize this framework as a practice-level assessment tool. The prototype was not designed to deliver VCs; rather, it was intended to support structured assessment of how VCs were implemented within primary care organizations.

The interface organized assessment statements across 6 domains: contextual considerations, technology infrastructure, awareness and experience, safety and risk management, strategic planning and supporting policies, and monitoring and evaluation. Within each domain, users reviewed a series of statements relating to the implementation of virtual care and indicated whether each criterion was met using a binary yes or no response, with the option to provide a justification (Figure 1A). Example statements addressed whether patients and health care professionals were willing and able to use virtual care, whether contextual barriers to VC access were considered, and whether alternative face-to-face pathways were available when virtual care was not appropriate or preferred.

Responses were summarized at the domain level and visualized using a radar chart to provide an overview of areas in which recommendations had been achieved and areas requiring further attention (Figure 1B). The prototype also included functions for saving assessments and generating or exporting reports. Thus, unlike platforms used to conduct VCs, the prototype functioned as an assessment and quality improvement resource rather than as a system for delivering the consultation itself.

A total of 3 online focus groups were conducted in June 2025 via Microsoft Teams, comprising groups of 5, 5, and 4 participants; group allocation was based primarily on participants’ availability. The final sample size of 14 reflected the number of eligible GPs who were recruited and available to participate during the recruitment period, with group size determined pragmatically by scheduling and feasibility considerations. The first focus group was facilitated by ALN, a clinical associate professor in digital health with extensive research experience in digital health. The subsequent 2 focus groups were facilitated by WZ, with support from SS. WZ had received formal postgraduate training in qualitative research methods, and SS had prior experience in conducting and analyzing qualitative health research. Sessions were held in English, lasted approximately 60 minutes, and were recorded and transcribed. A brief prediscussion survey was used to collect participants’ characteristics. The mock-up prototype of the tool (Figure 1) was presented during the session to prompt feedback on usability, format, and integration into clinical practice and served as a stimulus for feedback on the prototype and to inform its subsequent refinement.

A topic guide was used to facilitate all focus group discussions (Multimedia Appendix 2). The guide was divided into 3 main sections. The first section aimed to gather background information on participants’ professional roles and experience with VCs. The second section focused on obtaining feedback to inform refinement of the practical assessment tool, including participants’ views on optimal timing, usability, and preferred delivery format. The final section invited participants to share any additional reflections, suggestions for improvement, or relevant experiences that might inform future development of the tool.

All study materials were stored on password-protected servers at Imperial College London. Only members of the research team had access to the recordings and transcripts. No follow-up focus groups were conducted.

Transcripts were analyzed using a combined deductive and inductive thematic approach in NVivo 15 (Lumivero) [21]. Meaningful segments of text relevant to the study aims were treated as the unit of coding. Initial deductive codes were informed by the topic guide and study objectives, while additional inductive codes were generated from concepts emerging directly from the data. Two researchers with backgrounds in public health and clinical medicine (SS and WZ) independently coded transcripts. Coding was subsequently compared, and differences in code interpretation or application were discussed and resolved through consensus. The coding framework was iteratively refined throughout the analysis, with overlapping codes merged and code definitions clarified where appropriate. Related codes were grouped into subthemes and subsequently developed into higher-order themes through discussion within the research team [22].

Ethics approval for this study was granted by the Imperial College Research Ethics Committee (ICRER 7756479). All participants provided informed electronic consent before participation. Study materials were stored on password-protected servers at Imperial College London and were accessible only to members of the research team. Transcripts were anonymized before analysis to protect participant confidentiality. Participants received £30 (£1=US $1.34 as of October 21, 2026) in total for participation in the focus group, comprising £25 for 1 hour of participation and £5 for working-from-home or internet expenses.

A total of 14 GPs from 5 countries (the United Kingdom, Portugal, Türkiye, Greece, and Ireland) took part in 3 online focus groups, with an average duration of 46.6 (SD 10.3) minutes. Most participants were female (n=11, 78.6%), younger than 50 years (n=13, 92.9%), and identified as White (n=12, 85.7%; Table 1). Just over half of the participants (n=9, 64.3%) were fully qualified GPs, while the remainder were trainees. Half of the participants (n=7, 50%) had conducted more than 100 VCs in the past year. Telephone remained the dominant modality (n=13, 92.9%), followed by e-consultation forms (n=4, 28.6%), video (n=3, 21.4%), and messaging platforms (n=1, 7.1%). Most participants (n=12, 85.7%) were based in urban settings.

This theme captured participants’ views on the perceived value of the tool, its potential use for reflection and quality improvement, the preferred frequency of use, and opportunities for multidisciplinary engagement.

Positive assessments of the proposed evaluation tool were expressed across all 3 focus groups, particularly in relation to reflective practices and quality improvement. Two participants explicitly described the tool as “comprehensive” and “fundamental”:

Audit, reflection, or quality improvement were identified as potential use cases by more than half of participants. The tool was also considered potentially valuable for professional development, including portfolio-based reflection and appraisal. Some participants stated the following:

In total, 6 participants proposed specific intervals for repeat assessment, with suggestions ranging from monthly to annual use; 6-monthly or annual review was more commonly proposed. Participants linked less frequent assessment to the need to avoid administrative burden and allow sufficient time for changes to be implemented:

Participants generally viewed the tool as more suitable for practice-level or shared use than for assessment by individual clinicians. Suggested users included practice managers, GP partners, multidisciplinary teams, and patient representatives, reflecting a preference for incorporating multiple perspectives rather than relying solely on an individual clinician’s assessment. One participant explained it as follows:

This theme focused on the adaptability of the tool across different health care contexts, preferred response formats, perceived workload, and factors that could support or hinder its use in routine practice.

Participants offered several suggestions on the structure and content of the tool. A consistent theme was the need for flexibility across different health care systems. Given variations in national structures, resources, and digital readiness, many participants felt a “one-size-fits-all” approach would be ineffective. Others stressed the importance of allowing contextual differentiation, particularly for use at the facility or system level. For example, some participants shared the following:

Six participants considered the binary yes or no response format too restrictive and proposed more flexible response options, including Likert scales and open-ended responses. Some participants stated the following:

Three participants also suggested incorporating routinely available quantitative indicators to support interpretation of the assessment, such as rates of repeated telephone consultations or conversion from virtual to face-to-face consultations. One participant stated the following:

Additional workload and time burden emerged as widespread concerns. Completing the tool would add to their existing workload, particularly if not clearly aligned with appraisal or quality improvement requirements. One GP described it as “hidden workload,” stating the following:

Another participant stated the following:

Moreover, participants suggested that uptake could be improved through alignment with existing professional incentives, such as revalidation, audits, or financial support. One UK participant specifically noted that if the tool were not linked to the Quality and Outcomes Framework (QOF) or another remunerated activity, it was unlikely to be prioritized in routine practice. Some other participants stated the following:

This theme addressed participants’ preferred modes of delivery, shared access across practice teams, and integration with existing digital systems and clinical workflows.

Six participants preferred a web-based format as the primary mode of delivery. Key perceived advantages included desktop accessibility, shared practice access, and the ability to retain and compare assessments over time. Mobile apps were discussed as a complementary option, particularly for providing reminders. Some participants expressed as follows:

Integration with existing electronic health record (EHR) systems or clinical systems was also a recurring recommendation. Participants suggested that integration could reduce additional logins and duplicate data entry, facilitate access to routinely collected information, and make the assessment easier to incorporate into existing workflows:

This study explored GPs’ perspectives as part of a formative evaluation of a prototype tool for guiding, monitoring, and evaluating the safe and high-quality implementation of VCs in primary care. Overall, participants regarded the prototype as potentially useful for reflection, audit, and quality improvement and generally envisaged it as a periodic, shared practice-level resource rather than a mechanism for continuous monitoring of individual clinicians. They also emphasized the need for flexible response formats (eg, Likert scales and open-text fields) and contextual adaptability, together with integration into existing EHR systems and alignment with appraisal, quality improvement, or incentive mechanisms to minimize additional burden.

Previous research has addressed both the development and implementation of VC systems in primary care. Studies of online consultation systems have examined how these services are introduced into general practice, how they interact with existing clinical workflows, and how patients and staff experience their use [23,24]. Systematic reviews have further examined the use, acceptability, and experiences of electronic, telephone, and video consultations among patients and health care professionals [22,25]. These studies have been important in identifying the practical and organizational factors that shape the quality and safety of virtual care.

Building on this evidence, a range of frameworks, indicators, and evaluation approaches have been developed to organize relevant considerations into structured domains or measures. For example, the Planning and Evaluating Remote Consultation Services framework captures the clinical, organizational, technological, relational, and ethical complexity of remote consultations [17], while other work has focused on dimensions of health care quality, telehealth performance indicators, or patient safety measures [10,11,26]. Broader virtual care quality frameworks have also sought to organize relevant dimensions of quality into structured domains [27], while a recent scoping review identified a wide range of quality measures used in ambulatory virtual care and highlighted variation in the domains addressed across existing measures [28]. At an organizational and health system level, the World Health Organization Telehealth Quality of Care Tool provides a structured resource to support assessment, reflection, and quality improvement in telehealth services [29]. However, these approaches provide substantial guidance on what aspects of virtual care quality should be considered and how they may be organized for assessment but provide more limited evidence on how a primary care–specific assessment resource is perceived and used by GPs as intended end users. Rather than examining a system used to deliver VCs, this study evaluates a resource intended to assess the quality and safety of VC implementation. More specifically, it examines how intended GP users perceive the relevance, usability, and potential implementation of a concrete primary care assessment prototype and how such a resource might need to be structured and embedded for routine use.

The findings of this study are broadly consistent with wider digital health implementation research. Usability, workflow fit, interoperability, and compatibility with existing EHR systems have previously been identified as important influences on the adoption and sustainability of digital technologies [30-34]. Similarly, additional workload and limited incentives reflect recognized barriers to digital tool use in primary care [35-37]. The contribution of this study therefore lies less in identifying previously unrecognized implementation barriers than in showing how these established considerations translate into requirements for the design and implementation of a VC quality and safety assessment tool. Taken together, the study adds user-focused formative evidence at the prototype evaluation stage, complementing existing evidence on VC delivery systems, quality frameworks, and measurement approaches.

A strength of this study was the direct involvement of GPs, as intended end users, in evaluating the relevance, usability, and potential implementation of the prototype. Importantly, the inclusion of participants from multiple countries offers an additional strength, providing early insight into system-level variation that may shape priorities and use, and supporting the potential transferability of the tool across settings.

However, this study has several limitations. The participant group was small (N=14) and skewed toward White, urban, and early-career clinicians, with limited input from rural or resource-constrained contexts. In addition, the use of convenience and snowball sampling may have limited the breadth of perspectives represented and should be considered when interpreting the transferability of the findings. Conducting the focus groups in English and using voluntary recruitment may have favored those with stronger digital health interests. Finally, the prototype was intentionally illustrative, limiting feedback on technical specifics but enabling open discussion of relevance, usability, and workflow integration.

The findings identify several practical priorities for the further development and implementation of the prototype. For developers, future iterations should support more flexible response formats, shared access across practice teams, and integration with existing EHR or clinical systems to minimize duplicate data entry and additional workload. The tool should also allow sufficient contextual adaptation to accommodate differences in health care organization, digital infrastructure, and local practice.

At the practice and organizational level, the findings support periodic, team-based use rather than continuous completion by individual clinicians. Embedding the tool within existing clinical governance meetings, audit, or quality improvement processes may promote shared responsibility while reducing the risk of creating an additional administrative task. At the policy level, alignment with existing appraisal, quality improvement, or incentive mechanisms may support uptake and sustained use. If subsequently validated for routine use, aggregated assessments could also contribute to broader monitoring of virtual care quality and safety and inform service- or system-level improvement.

Future research should focus on iterative development, piloting, and validation of a fully functional prototype across diverse settings. Testing must consider variations in EHR systems, digital infrastructure, interoperability, staffing models, and national policy contexts. Stakeholder involvement should extend beyond GPs to include nurses, managers, patients, policymakers, and IT teams. Longitudinal evaluation will be essential to assess impact on clinical practice, patient outcomes, and organizational learning, providing evidence to inform national digital health quality strategies and support safer, more equitable digital transformation.

This study explored GPs’ perspectives on a prototype tool for assessing VC quality and safety and identified priorities for its further refinement and implementation. Findings suggest support for a structured yet adaptable resource that can support reflective practice, quality improvement, and clinical appraisal. Participants emphasized flexible response formats, integration with existing systems, and alignment with appraisal and incentive structures, highlighting the importance of minimizing additional workload and embedding evaluation within routine quality improvement and audit processes. Importantly, the tool was viewed not as a top-down performance measure but as a resource to empower clinicians and support shared responsibility for VC quality and safety. By involving GPs directly, this study helps bridge the policy-level digital transformation agenda with frontline needs, providing a foundation for further development and testing and for embedding evaluation mechanisms in everyday digital health care delivery.