Future Health Today (FHT) is a software that can be integrated with a general practice electronic medical record (EMR) [17]. It consists of two core components: a CDSS point of care prompt and a web-based portal, which includes an audit and recall tool, quality improvement management, and access to education and other resources. The prompt also provides direct access to the reasons for the recommendations and evidence-based information relevant to follow-up. FHT has been co-designed with GPs and consumers, with the purpose of optimizing patient care [18]. Different modules are developed for use in FHT. The FHT UWL module was developed to flag patients with a symptom of UWL recorded in the medical record in the past 6 months. The UWL algorithms take into account age, sex, and any recent abnormal test results, with different recommendations dependent on these factors. An example of a recommendation as it appears in the EMR is shown in Figure 1.

The study was carried out in the University of Melbourne’s Digital Health Validitron SimLab at the Centre for Digital Transformation of Health [16,19]. The SimLab has a simulated general practice clinic, incorporating a virtual “sandbox” (a virtual machine that is used to run software in a testing environment), which allows for the UWL algorithm to be used on the Best Practice and Medical Director General Practice EMRs in “near to real” scenarios. It is estimated that approximately two-thirds of Australian general practices use Best Practice and one third use Medical Director, with approximately 3.6% of practices using other EMRs [20].

Four hypothetical patient scenarios were designed to align with the UWL module recommendations. These included one female high-risk and one female low-risk scenario and one male high-risk and one male low-risk scenario. Each GP was exposed to one high-risk and one low-risk scenario according to the sex of the actor available. A summary of the scenario characteristics can be found in Table 1. The full patient scenarios can be found in Multimedia Appendix 1. The simulated consultations were performed by actors from the Department of Medical Education at the University of Melbourne, who were experienced in delivering simulated patient (SP) scenarios. The definition of an actor in this scenario was a professional “trained to reproduce the components of real clinical experience” [21].

Recruitment of GPs occurred using various methods: invitation (email or in-person) using the University of Melbourne Department of General Practice and Primary Care contacts; dissemination of study information through advisory groups (eg, Future Health Today Advisory Group) and groups associated with the University (VicREN, a practice-based primary care research network managed by the University of Melbourne and the Primary Care Collaborative Cancer Clinical Trials group); and GP groups on social media platforms (eg, WhatsApp). The recruitment messaging and materials included information about the simulation study, the location of the study, the time required to participate, and reimbursement. Participants were recruited using purposive sampling, with the aim of recruiting a diverse sample in terms of experience, gender, and age. Inclusion criteria applied the following: registered GP, familiar with either Best Practice or Medical Director EMR, who was able to attend the University of Melbourne SimLab for up to 1 hour.

Lived experience community advocates with a history of a cancer diagnosis (henceforth “community advocates” or “CAs”) were recruited via the Department of General Practice and Primary Care newsletter, word of mouth via friends and family, and through organization newsletters such as the Pancare Foundation, Cancer Voices Australia, and the Victorian Comprehensive Cancer Centre. The recruitment messaging and materials included information about the study, the time required to participate, and reimbursement amount. Similar to GP recruitment, purposive sampling was used. CAs were eligible for inclusion if they were over 40, had experience as a general practice patient, and presented with a nonspecific symptom of cancer before being diagnosed.

Interested participants (GPs and patients) were sent further details of the study using a Plain Language Statement. Informed consent was collected from all participants.

Otter.Ai was used for the initial transcription, which was then checked by a researcher (SC). Transcripts were then uploaded to NVivo 14, coded, and thematically analyzed [25] independently by JMG and SC. The acceptability of health care interventions underpinned the analysis of GP, CA, and patient actor interviews. Video recordings of the simulated consultations were also analyzed. The video analysis provided additional depth to the overall analysis and was conducted using the Sociotechnical Model (Figure 2). Findings from the interview analysis complemented the themes identified in the video analysis. The two frameworks were used to inform relevant themes, and additional themes were added after the initial review of the data. A subset of participants was asked to provide feedback on the analysis. We report results using the COREQ (Consolidated Criteria for Reporting Qualitative Research) checklist (Checklist 1) [26].

Ethical approval was granted by the University of Melbourne Human Research Ethics Committee STEMM Three (ethics ID: 28024). Written informed consent was obtained from all study participants before or on the day of the simulation, and all participants were informed of their ability to opt out at any moment. All transcripts and audio recordings were securely stored on an institutional password-protected server without any personal identifiers to maintain participant confidentiality. GPs and CAs were reimbursed for their time (Aus $150 [US $98]). Actors were compensated in accordance with the University of Melbourne’s established salary scale.

Two researchers were directly involved in data collection (simulation and interviews) and analysis (JMG and SC). JMG is the lead investigator for this study and was assisted by SC. JMG is a female GP and PhD candidate and midcareer researcher at the Department of General Practice and Primary Care, University of Melbourne. SC is a female early career researcher and postdoctoral fellow in the Department of General Practice and Primary Care. Both researchers have experience in qualitative research, in conducting interviews, and in the development of clinical decision support tools.

We conducted a total of 20 simulated consultations involving 10 GPs. Each GP participated in 2 scenarios: 1 high-risk scenario and 1 low-risk scenario. Further, 7 GPs tested the CDSS with the male scenario, and 3 GPs tested it with the female scenario. The patient’s gender in each scenario was determined by the availability of male or female actors on the day of simulation. All GPs were interviewed following the simulations; 2 actors performed the clinical scenarios, and they were interviewed after each GP finished their two clinical interactions (10 interviews with 2 actors). Additionally, 6 CAs were interviewed following their observation of 2 exemplar simulation videos.

GP demographics are summarized in Table 2. All GPs interviewed worked in metropolitan areas of Victoria. There was an even distribution of gender in the sample. Half of the GPs were between 30 and 39 years of age and had <5 years in practice. The age of participating GPs ranged from 31 to 63 years. Interview length ranged from 21 to 32 minutes.

Characteristics of CAs are detailed in Table 3. Half of the CAs were female, and half were between 60 and 69 years of age, with ages ranging from 48 to 71 years. Their year of cancer diagnosis ranged from 2011 to 2018, and they all had consulted their GP with a nonspecific symptom prior to diagnosis, although not necessarily UWL. Interview length ranged from 27 to 57 minutes.

The two actors each played a patient scenario of a middle-aged and older adult. Both patient actors had personal experience as general practice patients. Interviews with the actors were designed to be shorter, with interview length ranging from 5 to 9 minutes.

Characteristics of all participants can be found in Tables 2 and 3.

Consequently, the total number of reported symptoms exceeds the number of CAs interviewed.

We present the results for the two main themes in the theoretical framework of acceptability: affective attitude and perceived effectiveness.

We tested a CDSS to identify patients with UWL at risk of undiagnosed cancer in a simulated environment. The tool was generally deemed acceptable, particularly as a reminder of differential diagnoses and recommended investigations. However, in line with existing literature, concerns about overreferral and overtesting, as well as reluctance to discuss a potential cancer diagnosis, emerged as potential barriers [27].

The clinical content provided was considered adequate. GPs selectively utilized the information, according to their clinical reasoning influenced by internal (personal experience), external (cost and accessibility), and relationship factors (use of a stepwise approach, deciding to approach a cancer, and mental health discussion later), similar to factors described in international literature [28]. Australian cancer clinical guidelines recommend immunochemical fecal occult blood test as a screening tool for asymptomatic individuals in the general population and also in some symptomatic patients [29,30]. Recent research indicates its effectiveness as a rule-out test for colorectal cancer in patients presenting with gastrointestinal and nonspecific symptoms [31]. There was variable awareness of these recommendations by the GPs, with 6 out of 10 GPs recommending it alongside other imaging for their high-risk patients. CDSSs may be used as facilitators to behavior change, encouraging GPs to adopt new approaches in their diagnostic process. The majority of GPs did not access the tool’s embedded resources, which offered additional context and explanation of the recommendations. This could be explained by the additional cognitive load required to navigate and access less-readily available resources. However, it may also indicate a strong trust in the tool’s core content among participating physicians as confirmed in the interviews. Such trust is noteworthy, as overcoming skepticism toward CDSSs has been identified in literature as a major barrier to their adoption [11].

GPs utilized the tool in diverse ways: incorporating it into their own decision-making process without sharing it with the patient, as a reminder of differential diagnosis and suggested follow-up and to aid communication. Recent evidence showed the potential of a CDSS to improve patient–provider communication in cancer patients [32]. Notably, our CDSS showed the potential to either enhance or impede patient–clinician communication, depending on its application, how GPs integrated it into the consultation, and the specific context of use.

Health care delivery has increasingly become “digitalized”. The use of technology has become embedded in health care systems in most high-income countries, and it continues to expand worldwide. Technology can help clinicians to be more efficient and improve patient safety [33].

Multiple guidelines for digital intervention development have been published by international agencies and government bodies, such as World Health Organization [34], National Institute for Health and Care Excellence [35], the Food and Drug Administration [36], and in the academic literature. In 2020, Mathews et al [37] published the digital health score card, where authors propose that digital health solutions should always be validated from a technical, clinical, and system standpoint [37]. Even with these guidelines, there is not one single framework or process required or agreed upon to test digital interventions in similar real-life scenarios before releasing them to the public [38].

Simulation techniques have emerged as a promising approach to bridge the gap between the development and evaluation of digital health solutions before their implementation [38]. Despite growing interest and well-established methods, simulations for digital interventions prior to real-world deployment are not the norm, and this may impact not only their safety but their potential for implementation, resulting in most tools not being adopted into the routine health care system [39].

Barriers to incorporating new software into health care systems can include concerns about potential impacts on patient care quality, apprehensions regarding data privacy and protection, organizational capacity to manage nonmedical tasks, and the ever-evolving landscape of health care delivery [16].

Our simulation provided a controlled environment to test a CDSS designed to identify individuals with symptoms potentially indicative of undiagnosed cancer. By simulating diverse and realistic clinical scenarios, it allowed the system to be rigorously evaluated without risk to actual patients. It also allowed evaluation to occur in a timely manner, rather than waiting for a real-world “needle in a haystack” identification of low prevalence conditions and symptoms. This approach also offered valuable insights into the barriers and facilitators of implementing a tool for early cancer detection [32].

Notably, the simulation served two key functions. First, it directly modeled and observed practical issues, such as usability and workflow integration. Second, it acted as a “priming” tool, enabling practitioners to provide detailed, informed feedback on the CDSS functionality and recommendations. Most importantly, it highlighted that adequate use and implementation need to consider the “human factor.”

A key strength of this study lies in its multimodal analysis approach, combining video recordings of simulations with in-depth interviews with both participants in the consultation and real-live observing CAs. This methodology provided a comprehensive and nuanced understanding of the subject matter, capturing both observable behaviors and subjective experiences. Furthermore, the inclusion of both clinical and lived-experience perspectives ensured a well-rounded analysis, balancing professional insights with authentic patient views. This diverse approach not only enhanced the study’s rigor but also allowed for a comprehensive evaluation that bridges the gap between theoretical design and practical application in health care settings. The additional information gleaned from patients and CAs on how this type of technology and system support can enhance the patient experience in the inclusion of their health management adds an additional perspective to prior research focusing mainly on systems and technological evaluation [37].

All participating GPs were from metropolitan areas. This limitation was primarily due to the logistical requirement for GPs to travel to the simulation laboratory. Additionally, while simulations provide valuable insights, they inherently lack the full complexity and unpredictability of real-world clinical encounters and the pressures of a busy clinical day. This artificial setting may not fully capture the nuances of actual patient interactions, time pressures, or diverse clinical scenarios that GPs encounter in their daily practice, and GPs might act differently than in their regular day-to-day practice when knowing they are being observed.

While most GPs were not previously known to the researchers, and we did not provide information regarding personal goals or reasons for doing the research, two GPs had participated in previous studies and had established working relationships with one or both of the researchers. We acknowledge this may lead to social desirability bias, which we tried to decrease by reassuring them of our intention to assess real implementation, asking questions that could elicit both positive and negative responses toward the tool.

We developed and evaluated a CDSS designed to identify patients with UWL who may be at risk of undiagnosed cancer. The controlled simulation environment provided a safe setting to assess implementation barriers and facilitators while observing the tool in use. Clinicians found the recommendations to be clear and unobtrusive; however, concerns were raised about potential overtesting in patients with a low risk of cancer. The tool was utilized by clinicians in 3 primary ways: to inform their clinical decision-making process, as a reminder or checklist of best practices, and as an aid in communicating potential follow-up options with patients. Both patients and GPs believed the tool could either enhance or impede communication, depending on its integration into consultations, underscoring the potential need for communication-focused training in the use of these technologies. This study’s findings will inform future studies implementing CDSSs for identifying patients with nonspecific symptoms who may be at risk of cancer, potentially improving early detection and patient outcomes.