This study followed a usability and efficiency evaluation accompanied by posttask structured feedback and was divided into 2 phases from March 2023 to September 2025. The study was assessed against the CONSORT (Consolidated Standards of Reporting Trials) extension guidelines. Phase 1 involved the construction of a drug compatibility knowledge base and the development of a network visualization system for multidrug compatibility queries based on WeChat Mini Program. In phase 2, the efficiency and quality of the newly developed query system were assessed.

Data were extracted using a standardized form capturing key fields, including drug combination details (name, manufacturer, and concentration), solvent, compatibility result, experimental conditions (eg, temperature, contact duration, and light exposure), container type, and data source. To accommodate variability among the sources, not all fields were mandatory for each entry. To ensure the accuracy and reliability of the knowledge base, data extraction was performed independently by 2 clinical pharmacists (JHY and XQH) for the entire dataset. Any discrepancies were resolved through a third senior clinical pharmacist (ZDH).

We implemented a protocol to ensure data consistency. Drug names were standardized using the international nonproprietary name (INN). All drugs were classified by the World Health Organization (WHO) Anatomical Therapeutic Chemical (ATC) System. All terms from drug labels implying incompatibility (eg, “do not,” “prohibited,” “not recommended”) were uniformly classified as “Incompatible.”

Particularly, if a drug pair had conflicting compatibility results across different references, the system strictly defaulted to a red edge to maximize safety alerts.

The drug compatibility query system was developed as a WeChat Mini Program using WeChat Developer Tools (Stable 1.06.2209190). The user interface was built using WeiXin Markup Language (WXML) and WeiXin Style Sheets (WXSS), which are adaptations of HTML and CSS. The core application logic was programmed in JavaScript. A Cloud database, a JSON-based document database similar to MongoDB, was used to store the structured compatibility data. The ECharts.js library was used to construct a network graph for visualizing multidrug compatibility relationships.

The evaluation consisted of 3 parts: a scenario-based task completion test, a standardized quality rating via the Mobile App Rating Scale (MARS), and a semistructured interview.

Continuous variables were reported as mean (SD) for normally distributed data and as median with IQR for non-normally distributed data. Differences in task completion time between the 2 query modes (mode A and B) were analyzed using Wilcoxon signed-rank tests across the 3 scenarios. To account for multiple comparisons across the 3 scenarios, a Bonferroni correction was applied, setting the significance threshold at α=.0167 (0.05/3). To assess potential carryover or learning effects inherent in the crossover design, a Wilcoxon rank-sum test (Mann-Whitney U test) was conducted to compare Mode A completion times between the two sequence groups (Sequence 1: Mode A followed by Mode B; Sequence 2: Mode B followed by Mode A) across all scenarios, with a P value greater than .05 indicating no significant order effect. The internal consistency of the MARS subscales was assessed using Cronbach α coefficients, interpreted as poor (<0.50), acceptable (0.50‐0.75), good (0.75‐0.90), or excellent (>0.90). Furthermore, Spearman rank correlation analysis was performed to evaluate the relationships between user demographics (age, education level, and work experience), objective efficiency gains (time saved), and subjective usability ratings (MARS scores). All statistical analyses were conducted using R software (version 4.3.1; R Foundation for Statistical Computing).

Posttask structured feedback was transcribed and descriptively analyzed. Two researchers, XQH (a female junior pharmacist) and BRL (a male senior pharmacist), independently categorized the feedback into user-reported strengths, weaknesses, and suggestions using NVivo (Release 15.0.0; Lumivero). A preexisting collegial relationship existed between some pharmacist participants and the interviewers. We addressed this potential bias through proactive reflexivity, specifically by maintaining reflexive journals to document preconceptions and conducting peer debriefing with an independent researcher to verify the findings. The resulting codes were then collaboratively grouped into key descriptive categories. Any discrepancies were resolved through discussion with a third researcher with no preexisting collegial relationships with the participants to reach consensus. Furthermore, member checking was performed with a subset of participants to ensure that the interpreted description accurately reflected their original perspectives. Key descriptive categories were illustrated with direct, anonymized quotes.

Our study protocol was conducted in accordance with the principles of the Declaration of Helsinki and approved by the Research Ethics Committee of the Women and Children’s Hospital, School of Medicine, Xiamen University (KY-2025‐105 K01). All participants were told the study content and could withdraw at any time, and individual data were deidentified. Informed consent was obtained from all participants before data collection. The study was conducted in 2 stages: Phase 1 (March 2023-April 2025) was dedicated to database and software development, which did not involve human participants and thus did not require ethics approval; Phase 2 (commencing May 2025) involved human participants for system evaluation, for which ethics approval was obtained prior to any data collection.

A list of 212 injectable drugs in use at our hospital was constructed, representing 204 distinct active ingredients. Anti-infectives for systemic use were the largest drug category (n=27, 12.74%), followed by drugs for the nervous system (n=24, 11.32%) and the cardiovascular system (n=23, 10.85%) (Multimedia Appendix 2).

Theoretically, 20,706 possible combinations could be formed from these 212 drugs; however, only 3463 drug combinations (16.7%) were documented in at least one of the data sources in this study. The summary of drug compatibility data extracted from the 3 reference sources is shown in Multimedia Appendix 3. Among the documented drug combinations, a total of 1653 (47.7%) had at least one record of incompatibility. Furthermore, 135 drug combinations showed conflicting compatibility results (ie, were listed as both compatible and incompatible) depending on the different data sources or specific trial conditions outlined in the ASHP’s Handbook. In total, 5416 distinct interaction records were identified. This number is higher than the number of drug combinations because some combinations, particularly in the ASHP’s Handbook, had multiple data items corresponding to different trial conditions. The initial interrater agreement for the database extraction between the two clinical pharmacists was excellent, with a Cohen Kappa of 0.91. All identified discrepancies were fully resolved through consensus-building before final integration.

Our system was comprised of three main components: a drug combination input module, a network visualization interface, and a detailed search results panel. Users could input an unlimited number of drugs for analysis. The compatibility relationships were presented as a network graph generated by ECharts.js, where each drug was represented as a node. The edges connecting the nodes indicated the compatibility status, where a green edge indicated that the drug combination was documented as compatible across all data sources, a red edge indicated an incompatibility reported in at least one source, and the absence of an edge signifies a lack of available data. A drug combination was classified as “Compatible” only if supported by at least one source with no conflicting incompatibility data (Figure 1A). The search results panel provides a text-based list detailing the compatibility information for each drug combination from different data sources. By clicking the result button, users could access further details, such as the experimental conditions (Figure 1B).

A total of 47 health professionals from 12 medical institutes participated in the study (Figure 2). Among the 36 invited pharmacists for the quantitative phase, 34 successfully enrolled and completed the MARS assessment, while 2 dropped out entirely due to unexpected clinical emergencies during their scheduled testing slots. Out of the 34 enrolled pharmacists, 33 completed the crossover task completion time analysis (1 participant was unable to complete the time test due to urgent personal affairs but completed the MARS). For the subsequent interviews, 22 pharmacists participated voluntarily, and were joined by 5 physicians and 5 nurses who were additionally invited. Participants primarily worked in Tertiary A-level hospitals (39/47, 83%), with a mean age of 35.8 (SD 7.1) years and an average of 12.3 (SD 7.4) years of work experience. The majority held a bachelor’s degree (24/47, 51.1%). Detailed demographics for each study were presented in Table 1.

The query system (Mode A) demonstrated a statistically significant and substantial reduction in task completion time compared to traditional resources (Mode B) across all scenarios (Figure 3). The median time savings were 2.85 (IQR 1.98‐4.15) minutes for Scenario 1 (V=555, r=0.85; P<.001), 5.45 (IQR 3.95‐7.20) minutes for Scenario 2 (V=545, r=0.82; P<.001), and 31.2 (IQR 27.5‐35.1) minutes for Scenario 3 (V=551, r=0.84; P<.001). Wilcoxon rank-sum tests confirmed the absence of significant order effects for all scenarios (Ps₁=.43, Ps₂=.67, Ps₃=.42), validating the crossover design (Sequence 1 [Mode A first]: N_A=12; Sequence 2 [Mode B first]: N_B=21), though this unequal sequence allocation (12 vs 21) (Multimedia Appendix 4) warrants a cautious interpretation due to potentially limited statistical power in detecting carryover effects.

The system received a high mean overall quality score of 3.88 (SD 0.35) on the MARS. Functionality scored the highest (mean 4.21, SD 0.51), while engagement scored the lowest (mean 3.21, SD 0.62). As shown in Table 2, the overall objective quality demonstrated excellent internal consistency (α=0.87; 95% CI 0.81‐0.92). The functionality (α=0.78) and aesthetics (α=0.77) subscales met the conventional 0.70 reliability threshold [30]. In contrast, the engagement (α=0.61), information (α=0.67), and subjective quality (α=0.48) subscales yielded values below this 0.70 threshold.

Spearman correlation analysis was conducted to examine the relationships between user background, usability ratings, and efficiency gains. No significant correlation was found between the MARS overall quality score and the objective time savings in any scenario (eg, Scenario 3: rs =0.11, P=.58), nor between MARS scores and user demographics such as age (rs =0.021, P=.92), or work experience (rs =0.11, P=.58). Regarding efficiency, while a significant positive correlation was observed between age and time savings in the simplest task (Scenario 1: rs =0.45, P=.01), efficiency gains in the most complex task (Scenario 3) were independent of age, experience, or education (all P>.05).

In this study, we found that a network visualization query system showed promising improvements over traditional, print-based methods for multidrug compatibility checking within this pilot evaluation. By combining objective, crossover usability testing with structured posttask feedback, we observed a substantial and statistically significant reduction in task completion time alongside high ratings for overall quality and functionality on the MARS scale. Crucially, the inclusion of a diverse sample of pharmacists, physicians, and nurses captured a comprehensive view of the system’s potential within a real-world clinical setting. This multiprofessional perspective was highly valuable for identifying divergent workflow needs. While clinicians universally praised the system’s efficiency and intuitive interface, their willingness to adopt was heavily contingent on the transparency and authoritativeness of the underlying data sources. Participants expressed a strong need for features that build trust, such as direct links to evidence and clear explanations for incompatibility, underscoring that successful clinical decision support implementation may benefit from tailored outputs that support the distinct needs of each professional group within the collaborative care process.

In China, the drug label was the first-line reference for compatibility information due to its legal authority, a practice that differs from other countries where ASHP’s Handbook is considered the gold standard [31-33]. However, our study found that across all scenarios, the drug label contained the least amount of information on drug compatibility, a finding consistent with the common issue of data scarcity for drug compatibility in clinical practice, as evidenced by a study in a Chinese obstetrics and gynecology hospital where drug compatibility was a primary issue in telephone drug consultations [34]. Another fact is that drug incompatibility had been one category of off-label uses [35], exposing clinicians to significant legal risks without statutory protection, which further complicated the situation. Authoritative resources such as ASHP’s Handbook could be an alternative solution, but the English-language barrier prevents its widespread adoption in many clinical settings in China. Consequently, to address this dilemma created by a high demand for reliable compatibility data, insufficient legally sanctioned primary reference, and inaccessibility of a Chinese version of ASHP’s Handbook, our system, which integrates multiple data sources, is therefore a potential strategy to address this gap.

Our knowledge base construction revealed that 135 drug combinations showed conflicting compatibility results across different sources. This highlights another critical challenge in clinical practice: data inconsistency. Our system addresses this by transparently presenting data from all sources and flagging incompatibilities if reported in at least one source. This ‘safety-first’ approach empowers clinicians to make more informed decisions by being aware of the existing evidence conflicts, rather than being misled by a single, potentially incomplete data source.

The comparative results of task completion time analysis demonstrated that our query system was substantially more efficient than traditional methods, which were often time-consuming and prone to omission. Across all scenarios, our system enabled pharmacists to rapidly obtain drug compatibility information through easy operations, regardless of their experience or knowledge. Interestingly, while efficiency gains in the highly complex scenario (Scenario 3) were independent of demographic variables, a significant positive correlation between age and time savings was observed in the simpler Scenario 1. This stands in sharp contrast to traditional methods for query efficiency. When relying on drug labels, pharmacists not only face the challenge of reviewing large volumes of text, which can easily lead to overlooking important compatibility data, but they also encounter informational inconsistencies. This suggests that older pharmacists, who may be physically slower at manually cross-referencing thick paper-based books, experienced a more pronounced relative benefit from the digital tool. This aligns with prior research by Manrique-Rodríguez et al [19], which observed that the search order in one drug combination affected the results. In the Lexicomp database, a search for “verapamil” yields compatibility data with “vancomycin,” whereas a reverse search for “vancomycin” provides no such information. Relying on a single-directional query can lead pharmacists to mistakenly believe that compatibility data is unavailable, thereby compromising the accuracy and safety of their clinical decisions. To overcome these challenges, our system performs a bidirectional cross-validation search, extracting data from both drugs’ respective sources. Subsequently, the algorithm summarizes and deduplicates the compatibility information, offering the user comprehensive compatibility data. This process resolves the issue of data omission caused by search order or reliance on a single source and, therefore, ensures the completeness and reliability of the results. Additionally, this substantial improvement in efficiency is not merely a matter of time savings; it directly translates to a reduction in extraneous cognitive load. By automating the burdensome tasks of searching, memorizing, and integrating disparate information, the system alleviates the cognitive burden on clinicians. This allows them to focus on higher-order clinical reasoning and patient-specific factors, which, as cognitive load theory suggests, is crucial for reducing the risk of error [36] and enhancing patient safety [37].

On the MARS scale, our system achieved a high score in the functionality dimension. This result differed from a system review of web-based pharmacy apps [27] in which the aesthetics dimension achieved the highest score (mean 3.62, SD 0.50), followed by functionality. One of the reasons might be that, compared to our system, most commercial apps were developed by large companies where staff were professionally trained in user interface development. Additionally, our system was an early-stage prototype focused on functionality. In contrast, commercial apps were mature products that had been optimized through user feedback. Furthermore, because the engagement, information, and subjective quality subscales did not meet the strict 0.70 reliability benchmark, their corresponding scores must be interpreted with caution, potentially due to the small number of items [38]. The lower internal consistency observed for the MARS subjective quality dimension appears to stem from participants’ divergent views on the system’s value proposition, particularly regarding willingness to pay. Qualitative feedback revealed a clear dichotomy. One group of pharmacists viewed the tool as an essential institutional responsibility, arguing that such a critical safety utility should be provided by the hospital without charge to the individual. This perspective, while acknowledging high functionality, led to lower subjective scores related to personal cost. Conversely, another group perceived the significant time savings and reduced cognitive load as a direct personal and professional benefit that justified a purchase. They framed the app as an investment in their own workflow efficiency and well-being. This highlights that for clinical decision support tools, subjective value is complex, influenced not only by usability but also by perceptions of professional roles and institutional obligations. Future adoption strategies should likely prioritize institutional subscription models over individual payment to align with the expectations of most clinicians.

The inclusion of physicians and nurses in our qualitative analysis revealed critical interprofessional differences in how the system’s utility is perceived, underscoring the need for role-specific adaptations. Pharmacists, as primary users, valued the tool for its comprehensive and authoritative information retrieval capabilities. In contrast, physicians and nurses viewed it more as an action-oriented decision aid, expressing a greater need for immediate clinical guidance, such as management recommendations for incompatibilities or suggestions for alternative drugs. Furthermore, while pharmacists found the mobile format convenient, physicians advocated for integration with desktop EHR systems where prescribing occurs, and nurses highlighted the practical challenges of using smartphones at the busy bedside. These divergent perspectives on utility, workflow integration, and the interpretation of missing data emphasize that successful implementation requires a multiplatform strategy with tailored outputs that support the distinct needs of each professional group within the collaborative care process.

Several studies have reported self-developed drug compatibility tools. Wang et al [39] designed a system named PharmDE for evaluating drug-excipient compatibility. This system constructed a database containing 532 incompatibility data items from 228 articles and established 60 drug-excipient interaction rules, providing functions such as database searching, data matching by similarity, and risk evaluation for both single drugs and formulations. In addition to such complex systems, simpler tools like tables or charts have been implemented in many hospitals. A study in a Japanese ICU [40] developed a drug compatibility chart for 27 drugs, which achieved a high compliance rate and was shown to effectively prevent drug incompatibility incidents. Similarly, other studies [19,41] have developed compatibility charts for 47 and 78 commonly used IV drugs, respectively, for application in neonatal and pediatric ICUs. All those tools have been proven efficient in query speed; however, the results of the query remained in text format and were hard to display the relationship network among all the involved drugs. Additionally, traditional tables and charts are static and limited to pairwise comparisons. In a scenario involving more than two drugs, it still requires more time to query all the potential drug combinations.

The application of visualization techniques to clarify complex pharmacological data is a well-established method to solve this problem. Multiple studies have used such tools in managing multiple drug relationships and proven their efficacy. Zitnik et al [42] used network visualization to present how drugs are connected via shared targets and interaction patterns, enabling a clearer understanding of polypharmacy side effect mechanisms and supporting the interpretation of Decagon’s predictive modeling outcomes. Network visualization in Brattig’s study [43] mapped the complete set of known drug–drug interactions, with nodes sized by probability of interaction and edges colored by gender-specific relative risk. The approach highlighted key drugs and high-risk combinations, revealing interaction patterns and demographic disparities, and providing actionable insights for targeted interventions to reduce adverse drug reaction risks. In the DDInter database developed by Xiong et al [44], network visualization was a core feature for enhancing user experience and clinical decision-making, allowing clinicians to achieve the interaction profile at a glance and helping in quickly identifying high-risk combinations, and ultimately supporting safer prescribing.

As far as we know, there were few studies using network visualization to present complex compatibility relationships among multiple drugs. In this study, we used the visualization tool “ECharts.js,” which had been widely applied across diverse medical fields including neuroscience [45], oncology [46], precision surgery [47], rare diseases [48], and traditional Chinese medicine [49], to present complex biomedical data and relationships through interactive visualizations. By using network visualization, we transformed the query results into a dynamic network visualization. This can help users instantly identify all the compatibility information of involved drugs with diverse linkages connecting nodes.

Several limitations of this study must be acknowledged. First, the evaluation was conducted with a small, geographically localized sample in Xiamen, predominantly involving Tertiary A-level hospitals, which limits generalizability to grassroots medical institutions. Second, the knowledge base was constructed using Chinese-specific reference sources, which may limit the system’s immediate generalizability to other national health care settings with different formularies. Additionally, the lack of connecting lines (representing missing data) was sometimes misunderstood by users as “compatible,” posing a safety risk in high-risk settings. Future iterations will address this by introducing distinct visual cues, such as gray dashed lines or alerts, to clearly denote “No Data Available.” Furthermore, the current system lacks a severity classification for compatibility contraindications, which remains a key objective for future development. Finally, a fundamental constraint of this preliminary usability study is that no formal accuracy validation (eg, sensitivity, specificity, and false-negative testing of the compatibility database) was conducted against an independent clinical gold standard or laboratory assays. Because incorrect decision support outputs can lead to severe patient harm, this prototype is strictly intended for preliminary usability testing, and rigorous clinical accuracy validation is an absolute and mandatory precondition before any bedside clinical deployment is considered. Prospective clinical trials are warranted to validate the system’s direct impact on patient safety outcomes.

This study demonstrates that a query system based on network visualization is a promising and valuable tool for clinical practice. It demonstrates potential to support the efficiency and usability of multidrug compatibility queries. By offering an easily understandable view of complex data, the system has the potential to support medication safety by mitigating cognitive load, which is a known risk factor for clinical errors. Future prospective trials are required to establish its direct impact on reducing clinical medication errors and its economic value.