This study used a multimethod qualitative co-design approach [17] as a way of understanding existing pathology test ordering and decision-making workflows to ensure that the final digital tool was suitable for the clinical context and that user needs were considered at every phase of development. Formative evaluation was also undertaken during the development of the digital tool for the purpose of system improvement [18]. The co-design and evaluation of the decision support tool were conducted across 3 phases.

Phase 1 comprised in-depth interviews and an observation session to understand current laboratory test ordering workflows and issues, decision-making requirements, and expectations for an electronic decision-support tool. Findings from this phase were presented to developers of the tool [19] (laboratory management, a data scientist, and a health services researcher) who created the ED-POST prototype.

Phase 2 comprised demonstrations of the prototype to individual clinicians and the collection of clinician feedback via semistructured interviews. Findings from phase 2 were triangulated at a stakeholder forum during phase 3, which included developers of the tool and representatives from 2 additional EDs in the same public hospital network, to explore the applicability of the tool across hospitals and areas for further development. A summary of each phase is presented in Figure 1.

The study was conducted in the ED of a major regional teaching hospital and Level 1 Trauma Center in New South Wales, Australia. Hospital demographics are presented in Table 1. This hospital was selected based on its designation as the intended pilot site for the intervention.

Ethics approval for the project was granted by the human research ethics committee of the relevant local health district (2022/ETH01809). Written informed consent was obtained from all participants via a signed formal consent form as part of a study information pack. All collected data were deidentified and stored electronically in a secure cloud-based location accessible only to project study team members. Participants were not compensated for their participation in the study.

In phase 1, semistructured interviews were conducted with key informants (2 senior physicians, a senior nurse, and a mid-level physician) to understand the pathology test ordering and decision-making workflow and requirements. Interviews took place in April 2023 (range: 16 minutes 45 seconds to 1 hour 2 minutes 33 seconds), with participants purposively selected based on their involvement in and familiarity with the laboratory test ordering process. This criterion was relevant to this study, as the included participants were the intended users of the prototype. A senior ED physician assisted with identifying and inviting relevant candidates to participate. Interviews were supplemented with a 30-minute observation session in the ED triage area, where the majority of test ordering took place. Observational field notes, nonidentifiable photographs of relevant artifacts, and documentation (eg, guideline documents and blank laboratory test order forms) were also collected to provide further context. Findings from this phase informed the development of the ED-POST prototype.

In phase 2, individual demonstrations of the tool were delivered onsite in October 2023 to a larger sample of frontline clinicians (n=11). Criterion-based purposive sampling [23] was reused in this phase, in which a senior ED physician invited relevant participants based on their knowledge of and engagement with the laboratory test ordering process as a formal component of their clinical role. Immediately postdemonstration, participants were asked to describe the usability, utility, and potential impact of the tool on decision-making and workflow via semistructured interviews (range: 4 minutes 21 seconds to 13 minutes 23 seconds) and to offer any suggestions for improvement. Interview questions included perceived usefulness for ordering tasks, impact on workflow (speed/efficiency and sequence/coordination/number of process steps), influence on decision-making, and recommendations.

In phase 3, outcomes of the demonstration and of the ED-POST tool were presented at a 1-hour forum discussion to stakeholders external to the study site in December 2023. Developers of the ED-POST prototype were invited to attend, and the same senior ED physician who recruited for phases 1 and 2 recruited ED directors from 2 hospitals in the same geographic area. During the resulting stakeholder forum, findings from phase 2 were presented to developers and triangulated with those external clinicians, and the applicability and scalability of ED-POST across sites were explored. Interview and discussion guides for all phases of the study are presented in Multimedia Appendix 1.

Interviews and observations from phases 1 and 2 were audio-recorded and transcribed verbatim for inductive analysis using a general inductive approach to allow findings to emerge from the data and to avoid the constraints of preexisting theories or hypotheses [24]. Minutes were recorded during the stakeholder forum in phase 3 and analyzed in the same manner. Analysis was conducted by an experienced qualitative researcher (JL) who also performed all data collection but had no previous engagement with the study site. Transcript data from all 3 phases were analyzed to develop themes using an inductive, iterative process of initial open coding, constant coding review, and development of categories from open codes [24]. Final themes were presented to the broader research team and reviewed by a key informant from the site for member checking to establish trustworthiness and credibility of findings.

In this study, the Object Management Group’s Business Process Model and Notation (BPMN) [25] was used for mapping work processes in phase 1. BPMN is a standard graphical representation of work processes in a process diagram and plays a crucial role in bridging communication gaps between stakeholders [26]. BPMN’s standardized and intuitive graphical nature enables shared understanding between technical and other stakeholders and supports activities such as process design, implementation, monitoring, and management [26]. BPMN has been used to document the clinical context for digital health at the level of tasks, resources, and workflows and is valuable to the efficacious implementation of digital health [27]. In this study, BPMN was used as a means of presenting contextual workflows to ED-POST developers. Information extracted from interviews was used to identify the tasks, roles, and information requirements of the intended users of the prototype and presented to ED-POST developers alongside the prototype development document (Multimedia Appendix 2).

Demographics of participants across all phases of the study are presented in Table 2. In phase 1, clinicians described existing laboratory ordering work practices, which were mapped and presented to the team that developed the ED-POST prototype. Clinicians’ perceptions on issues, ordering decision-making needs, and expectations for a digital ordering tool were compiled in a prototype development document (Multimedia Appendix 2). A subset of preliminary findings from this phase has been previously published [19]. Across phases 2 and 3, participants responded to demonstrations of the ED-POST tool, raising insights into the following themes: (1) the impact of ordering role on the perceived use and value of the ED-POST tool, (2) occupational hierarchy, role-based priorities, and their broader implications for the ED-POST tool, (3) drivers of test ordering behavior, and (4) the real cost of laboratory testing.

In the absence of an electronic medical record (EMR) at the study site, participants reported that the requesting of laboratory tests remained paper-based. A site-specific characteristic of the ordering workflow compared with other hospitals was the expanded role of registered nurses who held more autonomy and ordering rights (within the scope of a locally developed guideline) and performed the majority of laboratory test ordering for the department at patient triage. Once a paper pathology request form was completed, it was sent with a blood sample to the laboratory via pneumatic chutes situated throughout the department.

ED physicians performed test ordering further along the patient journey through the department and sometimes based their ordering decisions on results already returned from tests ordered at an earlier stage. Additional tests could be added to an existing blood sample, or additional blood draws might be required. Venesection was commonly performed by physicians and nurses and, in less common cases, by a limited number of technical assistants employed by the department. A process model of the laboratory test ordering process in the ED is presented in Figure 2. Participant-reported issues with current test ordering workflows, decision-making requirements, and expectations for an electronic decision-support tool are published elsewhere [19]. In summary, a data-driven dashboard approach to supporting decision-making was raised as a potentially helpful tool, in lieu of guidelines, as peer behavior was reported to have some influence on decision-making. Respondents also requested that any intervention should not prolong or introduce additional steps to the workflow, should be fully integrated into existing systems, and should not impose restrictions on ordering behavior.

The results from the phase 1 exploration were presented to the digital developers and synthesized into a prototype development document (Multimedia Appendix 2), which was shared with the same developers. Table 3 presents an excerpt of the document and shows how findings from phase 1 were operationalized in the design of ED-POST. The ED-POST prototype was then developed by a data scientist, and the user interface was designed by a member of the university research team using Tableau Software LLC (Professional Edition version 2023.2.1). The ED-POST tool drew upon the Australian Pathology Atlas of Variation (Atlas) data source, containing over 8 million encounters and 25 million tests across 4 years in 35 facilities [28,29]. The Atlas analytical model merges ED data such as presenting problem, patient demographics, and triage categories with test ordering profiles. It also measures ED throughput and pathology turnaround times to identify intervals around patient arrival in the ED to triage, arrival to pathology order, the overall length of stay, and mode of separation. This information can be used to create probability scores from 0% to 100% for the likelihood of tests being ordered in scenarios with known variables at the time of triage (presenting problem, triage category, age, sex, and facility).

ED-POST uses patient triage variables to calculate the probability that the top 11 most commonly ordered laboratory tests in the ED will be ordered based on patients with an identical presenting profile from its dataset of 8 million encounters. The tool then displays ED ordering trends for a specified hospital (selectable via a drop-down menu) in comparison with the state average as a means of supporting test ordering decision-making (Figure 3).

A multimethod qualitative co-design approach to understanding clinician workflows illuminated the different roles of various clinical groups in laboratory test ordering in the ED and how their decision support requirements were influenced by their role. Triage nurses who ordered preliminary and routine tests at the beginning of the patient encounter were more likely to use ED-POST as a point-of-ordering decision support tool. Junior physicians also saw value in ED-POST as a form of backup to reinforce individual impressions, as a reminder, and as a guide in cases of uncommon presentations. Senior physicians responsible for determining a differential diagnosis at a later stage of the patient journey through the ED saw less utility in ED-POST for supporting their ordering decisions but recognized potential in its big-data synthesis capabilities to support the departmental management component of their role and quality-improvement activities.

The “one-size-fits-all” fallacy viewpoint of Karsh et al [30] identified the need to acknowledge the substantially different clinical roles extant in a health setting and the differing digital health needs of individual practitioners. Findings of this study indicate that the design of department-wide interventions should account for individual, as well as the collective, requirements of the broader team. After studying workflows in the ED setting during transition from paper medical records to an EMR, Park et al [31] identified the need for digital systems to support both the collective needs and roles and responsibilities of individual medical staff. Our study highlighted the effect of individual roles in test ordering and hierarchical role-based priorities on the perceived use and the impact of a digital health tool within a department.

The validity of departmental trends and statistically derived state averages as benchmarks for guiding test ordering behavior was a key criticism that was raised in this study, with many participants expressing a preference for evidence-based advice. The current digital health landscape is saturated with laboratory test ordering guidelines [15,16], guideline-based alerts and decision support [32], and EMR-based care sets and pathways [33], which have been implemented with varying degrees of success. One study of note by Leis et al [34] on admission order sets reported an increased likelihood of overordering, despite being designed to improve guideline adherence. The authors found that increased visibility of tests included in an order set can act as a behavioral “nudge,” influencing clinician decisions independently of clinical needs. This is relevant to concerns raised by clinical forum participants in this study, who feared that the increased visibility of common tests via ED-POST might encourage overordering by junior physicians, even when probability scores did not depict a substantial majority.

While existing interventions typically provide general and generalizable advice, the distinctiveness of ED-POST is its ability to harness site-specific data to highlight local issues and facilitate quality improvement via the mechanism of tailored feedback. A small number of studies examining feedback-based interventions to address unnecessary or low-value laboratory test ordering have reported positive outcomes [35]. However, such interventions were temporary and relied on manual processes, and there are no current studies reporting feedback-based electronic decision support at the point of ordering. Furthermore, all studies concluded that test ordering stewardship interventions were most effective when feedback was implemented in conjunction with other approaches, most commonly clinician education [35,36]. Given the positive reception to the big data-driven properties and quality improvement potential of ED-POST, the efficacy of the tool might be better achieved if implemented as part of a comprehensive suite of test ordering stewardship measures. The continued development of ED-POST’s real-time or near–real-time data synchronization potential and granular reporting of ordering trends at the staff group or individual level could inform targeted departmental quality improvement activities in addition to the implementation of a point-of-ordering decision support tool alone.

Finally, discussions during the stakeholder forum suggested a lack of clarity regarding the costs and charges incurred and imposed by the laboratory, respectively. A prominent argument against the overordering of tests from senior ED physicians in this study revolved around the issue of cost. However, the true cost of laboratory tests encompasses more than the direct dollar value attributed to the test, with add-ons, in particular, requiring additional resources and potentially creating delays in the testing process [37]. Increased transparency regarding laboratory billing structures and evidence between the real cost of tests against the potential cost of a delayed patient disposition in cases of underordering might reshape overarching attitudes towards the ordering of additional and add-on tests [38].

This study reports user perceptions of an electronic laboratory ordering support tool without quantitative or observational measures of usability, utility, or effectiveness, and outcomes in actual practice might vary. The study was also limited because it did not include the entire population of clinicians involved in laboratory test ordering in the department and was largely a single-site study, limiting transferability to other settings. However, the multimethod, qualitative co-design approach resulted in the in-depth exploration of clinical roles and pathology test ordering workflows within the departmental context and input from representatives from 3 hospital sites.

Requirements for laboratory test ordering decision support can vary substantially across clinical roles within a department. Understanding roles and workflows in the early developmental stages of a digital health tool, and involving users in every phase of development, are necessary to optimize adoption, scalability, and outcomes. Future design and implementation efforts should prioritize role-sensitive, workflow-integrated systems. Incorporating timely, feedback-driven mechanisms using local, context-specific data may facilitate more informed test ordering behaviors. Importantly, digital interventions should support both individual clinical decision-making and broader departmental objectives.