Telemedicine for Remote Surgical Guidance in Endoscopic Retrograde Cholangiopancreatography: Mixed Methods Study of Practitioner Attitudes

Introduction

Background

Rapid development of surgical techniques and medical technology creates a continual need for retraining among surgeons [1,2]. Remote surgical guidance through telementoring and teleconsulting [3] can be a cost-effective way to facilitate teaching and training for less experienced surgeons [4] and support safe adoption of new clinical methods among experienced practitioners [5-9]. However, telemedicine innovations rarely move from the pilot stage to routine delivery [10,11]. As of yet, the factors contributing to acceptance and adoption of telemedicine are not well understood [12-15].

This study focuses on a promising telemedicine service for remote surgical guidance called teleguidance. The innovation was based on videoconferencing combined with transmission of high-quality endoscopic video and fluoroscopy. In this way, a high-volume clinical center could provide intraoperative consultation to a low-volume center during endoscopic retrograde cholangiopancreatography (ERCP), which is a highly specialized procedure for the diagnosis and treatment of biliary and pancreatic disease. A feasibility study demonstrated the impact on the clinical outcomes [16]. However, when teleguidance was to be scaled up to include more hospitals, some practitioners appeared less interested than anticipated. This raised concerns about implementation and about whether teleguidance could become an accepted way of working. We therefore conducted a theory-driven, user-centered study to gain a deeper understanding of practitioners’ attitudes toward teleguidance.

In the following sections, the clinical procedure and the telemedicine innovation are described. These sections also provide a general background to studying attitudes toward new technologies and a description of our methodological approach.

Teleguidance in ERCP

ERCP is a technically advanced procedure for the diagnosis and treatment of biliary and pancreatic disease. ERCP has a long learning curve in both technical skills and decision making. After the initial specialty training, it is necessary to perform a certain number of cases per year to sustain the acquired skills, which may be difficult in low-volume clinics. Continual retraining, necessary for keeping up with new surgical advances [17], is also sometimes difficult at hospitals with fewer resources for education and research. This can have consequences in the case of unusual conditions or if complications arise during the procedure. Difficulties during ERCP can lead to delays in diagnosis and treatment or painful or even life-threatening complications for patients who already have serious underlying health issues [18].

Practitioners in need of advice during an ERCP procedure, but without the option to ask an experienced colleague on site, can opt for alternative procedures or refer the patient to another hospital. Another option is to get in touch with fellow specialists by telephone. Teleguidance was developed to enhance this practice through videoconferencing coupled with high-quality video transmission of videos and radiographic imagery (Figure 1).

‎View this figure

A feasibility study has shown that teleguidance between a tertiary referral center and a low-volume hospital can result in improved quality of care [16]. This raised the prospect of extending this telemedicine service. A series of user-centered design efforts were initiated [19,20] as well as health-economic modeling [4]. Prior to expanding the service, an exploratory survey (Multimedia Appendix 1) showed variations in how clinicians rated their need of support. Subsequently, we wished to investigate practitioners’ attitudes toward teleguidance more thoroughly.

Related Research

Traditionally, advanced surgical skills are learned by working together with experienced surgeons as mentors, progressing from shadowing to increasingly independent work. Necessary on-site training and retraining is sometimes difficult due to practical reasons and costs [9]. Numerous case studies—the earliest dating from the 1960s—have presented videoconferencing as a safe and efficacious way of providing surgical mentoring, enabling safe adoption of new techniques through remote expert guidance [2,6,7,21]. However, surgical telementoring is not widely used [5] and its impact over time is not well understood [6]. Implementation outcomes in health care are strongly affected by organizational context and how well an innovation answers to user needs [22]. This complexity makes it challenging to identify measurable determinants that provide an adequate image of implementation [23], in particular, regarding the quality and safety of products and services that function across multiple organizations [24].

Technology acceptance, a concept that relates beliefs and attitudes to use, is often considered an important determinant for technology implementation [22]. The technology acceptance model (TAM) [25] hypothesizes that people are more likely to use a technology if they believe it will be useful and easy to use. TAM was developed to provide validated measures for efficient early acceptance testing during development of office information systems [25,26]. The model defines two fundamental constructs: Perceived Usefulness signifying “the degree to which a person believes that using a particular system would enhance his or her job performance” and Perceived Ease of Use, representing “the degree to which a person believes that using a particular system would be free from effort” [27]. These central constructs were to be measured by a parsimonious questionnaire and were also intended to be transferable across different technologies and users [27]. TAM has been applied in many domains and TAM research has provided validation, extension, and elaboration of the central constructs [28]. However, surprisingly, few studies have investigated whether the model actually is a reliable predictor of use [28-31] or what makes a system useful [32]. Despite these weaknesses, the model is so frequently used that it has been described as a paradigm [30,33,34].

Various versions of TAM are commonly used in health care, and TAM has been extensively applied in studies of telemedicine [14,35-39]. While physician acceptance is commonly considered an important success factor [40], there does not appear to be any “optimal” version of TAM for telemedicine [14]. Despite its frequent use, TAM has shown shortcomings in health care [14,33,35]. Some of these have been attributed to the model’s narrow focus on individual users’ needs [39].

Another critique is that TAM invites quantitative treatment of narrowly defined theoretical constructs: the constructs themselves are treated as “black boxes,” which in the end has led to a state of theoretical confusion and chaos “around the TAM’s contribution to the understanding of technology acceptance and use” [32]. TAM was developed for prototype usability testing or system selection for office information technology systems, and the original definitions are grounded in research about cognitive and affective factors affecting the use of single-user computer software [25]. Transferring TAM to health care raises fundamental issues about how the carefully designed and validated TAM scale items (Multimedia Appendix 2) match the concepts being studied [41]. Holden et al [41] posit that acceptance studies in health care could benefit from a broad set of perceptions about usefulness and adapting the variables to the context in question. Many studies have added extensions to include a wider range of situational and social influences than were originally defined [14], and alternative conceptualizations of “Ease of Use” for health care broaden the focus from individual users’ beliefs about usability issues to include an organizational context [39].

The many adaptations and variants of TAM and the discussions about its relevance in health care highlight the importance of carefully considering what “Usefulness” and “Ease of Use” mean in each specific case. However, the model’s construct definitions can support qualitative data collection, analyses, and the interpretation of findings [42,43].

The combination of ambivalent attitudes toward teleguidance and research showing that telemedicine adoption appears problematic motivated us to investigate ERCP practitioners’ attitudes toward teleguidance. Guided by TAM, we studied the ways in which teleguidance might be perceived as “useful” and “easy to use” and how these perceptions vary across ERCP practitioners. These results are intended to inform the design and implementation process and to be valuable for understanding if and how teleguidance will be used at different clinical sites over time.

Methods

Questionnaire Design

Behavioral questionnaires should be grounded in the understandings of what is to be measured through contact with domain experts, and research in the relevant behavior domain should guide the construction of the items [44]. We conducted interviews to understand ERCP work and stakeholder beliefs about teleguidance’s contribution to procedures or other interactions with ERCP work. This was followed by thematic analysis [45], where TAM served as a theoretical framework. The interview findings served as a basis for adapting the standard TAM questionnaire. The interviews and questionnaire are described in more detail in the following sections.

Interview Procedure

Initial key contacts with clinical staff at the different locations were set up by senior physicians at the tertiary referral center providing teleguidance, and a snowballing technique [46] gave us access to additional respondents. A total of 20 semistructured interviews with 10 ERCP specialists, 5 ERCP assistants, 3 technical staff, and 2 administrative staff from 5 hospitals were conducted.

Prior to each interview, the background, design, and purpose of the study, as well as the implications of participation were explained and also presented in printed from in order to gain informed consent [47] (Multimedia Appendix 3). Each participant was given a verbal presentation of the telemedicine service and a printed presentation with text and images describing teleguidance. Interviews were audio recorded and transcribed verbatim and treated as realist accounts. The interview length ranged from 30 minutes to 2 hours.

Coding and analysis proceeded through several iterations of reading the interview transcripts and refining the coding and themes. After coding, the data extracts were collated to help review patterns and relationships. The thematic analysis focused on identifying issues that might affect the use of teleguidance. At the outset, a number of themes were defined from the TAM model: statements related to performance, productivity, and effectiveness were to be coded as “Usefulness” issues, and issues related to expected usability or design issues as “Ease of Use” issues.

Questionnaire Procedure

The 16-item questionnaire used a 7-point Likert-type scale, with a midpoint alternative to respond as “neutral.” It was made accessible as a closed, web-based 1-page survey provided through a web-based survey service during a 6-week period. Email invitations were sent to 25 physicians regularly performing ERCP at 15 different ERCP clinics, providing a link, information about estimated time to complete the survey (5 minutes), and information about data management and analysis (Multimedia Appendix 4). The number of practicing ERCP specialists in Sweden is small, and we made an effort to reach as many specialists in the field as possible that we had not yet interviewed. Analytical themes were operationalized as questionnaire items (Multimedia Appendix 5). Questionnaire items and phrasing were reworked a number of times to provide a succinct format and secure a high response rate. The order of the questions was randomized to avoid order effects. A few questions were also negatively phrased. Subjects’ age, gender, and professional experience, and an option to add comments was included. The questionnaire was pilot tested [46] by 2 ERCP specialists at the University hospital. The results were treated with exploratory data analysis methods, and we created visual representations of the score distributions in the form of stacked columns (Multimedia Appendix 6 and Multimedia Appendix 7). To gain interpretability and improve the stability of the ratings, we dichotomized the Likert scale ratings [48] with a cut between disagreement and neutral (1,2,3,4) and agreement (5,6,7). We also ran a Mann-Whitney U test in SPSS (IBM Corp), a rank-based nonparametric test, to investigate differences in the attitude scores between experts and novices for each questionnaire item. For all tests, a P value less than .05 was considered statistically significant.

Results

Interview Results

The interviews gave us insight into practitioner beliefs about teleguidance’s possible contributions to performance and factors that might interact with implementation. Four analytical themes (Multimedia Appendix 8) were defined through an iterative process of reviewing the transcripts, codes, and themes [49]. As the interview study progressed, it became clear that the respondents were not mentioning standard “Ease of Use” factors related to usability issues or design. What we found instead was mention about how teleguidance might interact with the work system, workflow issues, patients’ and management’s attitudes, and whether the telemedicine service would cause practical/technical or administrative issues. Risk was an additional theme that emerged inductively from the data sets. On this basis, we replaced the concept “Perceived ease of use” with 2 concepts defined in the Consolidated Framework for Implementation Research (CFIR) [50]: compatibility and implementation climate. Compatibility refers to the fit between the innovation and the current work systems. Implementation climate is intended to reflect users’ beliefs about whether the use of teleguidance would be expected and supported among important stakeholders. The themes were used as the basis for the questionnaire design.

Questionnaire Results

In this study, 25 ERCP specialists—14 experts (>500 procedures) and 9 novices (<500 procedures)—provided complete responses (100% completion rate); 2 respondents were removed as they had previously participated in the interviews. The perceived usefulness items and average dichotomized agreement scores are shown in Table 1.

The average agreement ratings for usefulness items were 64% (~9/14) among experts and 75% (~7/9) among novices. The average agreement ratings for compatibility items were somewhat lower (experts 64% [~9/14], novices 69% [~6/9]). The averages were calculated from the sum of several items and therefore, they only approximate the actual values. Experts and novices tended to agree that teleguidance could contribute to better overall ERCP treatment for patients (11/14, 79% and 8/9, 89%; respectively) and improve quality and patient safety during ERCP procedures (11/14, 79% and 8/9, 89%; respectively). However, only 7 of the 14 experts (50%) thought that the procedures they performed were challenging enough for teleguidance to be of value, while 6 of the 9 novices (67%) agreed. The novices also agreed to a higher extent (7/9, 78%) than experts (8/14, 57%) that teleguidance could contribute to their effectiveness. Fewer experts (10/14, 71%) than novices (8/9, 89%) believed teleguidance could help develop ERCP activities at the clinic. Multimedia Appendix 6 shows the score distributions of the Usefulness items.

Experts gave high agreement scores (>75%) on both implementation climate items (Table 2), while relatively few novices agreed that management would be positive toward teleguidance (experts 11/14, 79%; novices 7/9, 56%). There were also concerns about the quality of video and image transmission and administration between hospitals, with relatively low agreements on “The quality of video and image transmission between hospitals will function well” (experts 5/14, 36%; novices 3/9, 33%) and “Administration between hospitals will function well” (experts 6/14, 43%; novices 6/9, 67%). Similarly, only 8 of the 14 experts (57%) and 6 of the 9 novices (67%) agreed that teleguidance would not create new patient safety risks. This contrasts with the scores for “teleguidance would be beneficial for the quality and patient safety of the ERCP that we perform,” where both groups expressed positive expectations about the service’s contribution to quality and patient safety.

The graphs showing score distributions (Multimedia Appendix 7) illustrate that agreement was generally high; however, there was a large portion of neutral ratings on items, namely, “Management would be positive that I use teleguidance,” “The quality of video and image transmission between hospitals will function well,” and “Administration between hospitals will function well.”

Mann-Whitney U tests showed that there were no statistically significant differences between the novices and the experts on any of the 16 items (P<.05) (Multimedia Appendix 9).

Discussion

Principal Results

The interviews provided insight into the types of benefits teleguidance could provide and into the everyday ERCP work practices that might be affected by teleguidance. This served as important input for our questionnaire, where we defined the construct “Usefulness” in terms of how teleguidance might contribute to performance, productivity, and effectiveness. We exchanged the standard TAM construct “Ease of Use” for “Implementation climate and Compatibility” to better reflect concerns about how teleguidance might fit with the existing work system and if teleguidance might introduce new risks. Our main focus was to develop an understanding of the complexity of the domain and of the diversity among respondents, grounded in qualitative data.

The questionnaire allowed us to expand our inquiry to include a larger number of specialists in a domain where access to practitioners can be very difficult [51]. In the interviews, many expressed positive expectations about teleguidance, particularly that it could answer to challenges that novices were facing. However, many staff members also expressed concerns about how teleguidance would fit in with existing work system. The questionnaire results similarly showed that most respondents believed that teleguidance could contribute to the quality and safety of procedures and to the many anticipated technical and administrative issues—possibly even new patient safety issues. This indicates that practitioners had concerns that teleguidance might disrupt work.

While novices were the primary target group of the telemedicine service, teleguidance was designed with both novices and more experienced ERCP specialists in mind. We found that some experts were consistently skeptical toward teleguidance. As senior clinicians are more likely to be key decision makers [52], the attitudes among this group can have a greater relative importance for implementation than novices’ attitudes. Below, we discuss some possible reasons for and consequences of the differences between novices and experts and comment on the methodological concerns. We conclude with some practical suggestions for the implementation of teleguidance.

Differences Between Novices and Experts

Our interviews indicated that novices could be under considerable pressure during key phases of the procedure and they often saw room for improvement in current work processes. This was reflected in the questionnaire results, where novices had higher agreement scores on all the performance items and on the items for individual and organizational effectiveness. Novices may also have a lower threshold to work with videoconferencing than their more experienced colleagues, who also were older; previous use of information and communication technology in everyday life has been seen as a significant predictor for physicians’ telemedicine use [53]. The score distributions show that there were some items with many neutral responses, especially the “Implementation climate and Compatibility” items. This not only draws down the dichotomized score but it also indicates a challenge in asking potential users to form an opinion of a complex intervention, which might have complex outcomes, eg, patient safety issues. Experienced practitioners displayed a lower level of agreement that teleguidance could improve their individual performance, which may be explained by a less imperative need for support. However, developing integrative competence and taking part in a surgical innovation is an important aspect of sustaining acquired surgical competence [54], which is one of the aims of teleguidance.

While more experts than novices believed that management would be positive toward teleguidance, they also expected more administrative challenges. The differences in how experts and novices weighed these organizational aspects of teleguidance may be explained both by differences in roles and in experience: senior practitioners were more likely to have managerial functions and hence might have a different perspective of management priorities and the changes that teleguidance might entail [2]. We can only speculate about the experts who gave negative ratings consistently: senior ERCP specialists with established practices and status may perceive teleguidance more as a challenge to traditional routines [55] or professional autonomy [56], and as a consequence, tend to prefer existing work practices [57]. However, it is likely that negative attitudes are a common source of bias in implementation studies, as these practitioners may very well decline to participate at all. Research has shown that differences in power and politics among professional groups influence the use of new technologies in health care [52]. Our findings underline the importance of including a variety of experience and roles in this type of study: some experts thought that novices would prefer “hands-on” help to remote guidance, while many novices themselves said the opposite. In addition, many of the interviewed nurses mentioned concerns about staff and patient integrity issues, but nurses were not included in the questionnaire nor was this view reflected in the doctors’ questionnaire responses.

Practical Implications for Implementation

Teleguidance was initially developed to meet a wide range of challenges regarding the quality of ERCP procedures; it was not exclusively intended to serve the practical training needs of novices. Negative attitudes, even among a smaller group of experienced practitioners, may offset implementation efforts. However, our findings provide some guidance for design and implementation. The interviews provided insight into ERCP as a time-sensitive, collaborative team effort, highly dependent on medical technologies. Staff concerns that teleguidance might be an extra burden is based on daily experiences of ERCP work. Viewing teleguidance as a service rather than a new technology can widen the design perspective to include considerations what happens when two work systems are bridged by telemedicine. Experts’ concerns about administrative issues and compatibility of work practices should be taken seriously; implementation efforts could benefit from identifying workflow issues, defining staff roles and tasks, and designating scheduling allowances for teleguidance-related tasks. If teleguidance is to be widely used, it is important to define and communicate the value of teleguidance even for experienced practitioners and to investigate incentives for experts’ participation, since teleguidance also aims to support learning among experts. The SAGES telementoring initiative [2] differentiates between telementoring and teleconsultation, answering to different needs among experts and novices. Their definition of telementoring emphasizes a learning relationship between a mentor and a mentee and that telementoring occurs within an educational framework. Framing teleguidance in a similar way could benefit all parties; by clarifying relationships and objectives, teleguidance may be implemented as an explicit training effort for novices. Well-defined educational objectives might serve as incentives for novices to participate as well as increase the management support of telementoring. This could be a way to avoid inadvertently challenging the power and autonomy of incumbent experts.

As a contrast, teleguidance between two qualified experts might be defined as teleconsultation. This would signify different content and implications of the practice, with an emphasis on an exchange between peers, which may be experienced as less of a threat or intrusion by the more experienced ERCP specialists.

Limitations

This study has limitations due to the lack of lack of internal validity tests, which were beyond the scope of the study. This study does not attempt to exhaustively identify themes that may affect attitudes toward teleguidance, as the TAM guided toward predefined factors of interest. The number of respondents may be questioned, but as the total population of practicing ERCP specialists in Sweden is small and our respondents are highly representative, we claim to adequately cover variations among the groups. This study was exploratory, focused on developing an understanding of the complexity of the domain and of diversity among respondents. The quality of our findings is grounded in the qualitative data, rather than in statistical inference [58]. This study represents “the scientific discovery phase” [42], where empirical findings from a complex setting can ground hypotheses about behavior and design. In this sense, items with low agreement or ambiguous findings such as the seemingly contradictory beliefs about patient safety are valuable indications about how similar studies can be refined.

Conclusion

In our interviews, practitioners’ descriptions of ERCP work and their beliefs about teleguidance did not resonate with the classical TAM questionnaire; they had no need to “work more quickly” or “make the job easier” nor did the interviews provide any statements about “Ease of Use” issues such as design features or usability. Instead, staff spoke of organizational demands deeply infused with clinical work, intense team collaboration, and constant organizational pressure for effectiveness and efficiency.

This means that teleguidance does not just have to answer to individual users’ needs but also to organizational demands and priorities. Our findings show that introducing teleguidance is not “just” introducing new technology; teleguidance will change collaborative practices, linking locations that have their own sets of practices and priorities, which also can cause disruptions. We believe that the main cause for negative attitudes toward teleguidance is based on these concerns, which can be addressed during design and implementation. This study is an example of how TAM can support theory-guided user-centered design approaches to telemedicine development [31]. This may be a way to tackle the complexity of introducing technology in health care [59]. Using TAM in this way is also a return to the original intentions of the TAM, namely, to provide early user feedback to the system development processes, so as to gain better understanding of “how to improve user acceptance through design” [26].

Future Work

We suggest that using theories to guide the investigation of relevant user needs and expectations in a specific context is a way to inform the development and implementation of telemedicine. By connecting findings to well-known analytical themes such as usefulness and terminology and concepts in frameworks such as CFIR [50], this type of qualitative approach can contribute to understanding the forces that shape the adoption of telemedicine and contribute to its effects. The complexity of introducing teleguidance across multiple clinical sites and ERCP teams will make evaluation particularly challenging [60,61]. Theories that accommodate complexity in studies of technological change are increasingly emphasized [59,62]. Building on our insights from this study, we hope to apply sociotechnical methods that are developed for understanding changes in complex and adaptive settings [59] and follow the introduction of teleguidance over time in a real-world context to study the ways in which teleguidance affects user behaviors.