The ECHO approach differs from traditional telemedicine, where clinical providers (eg, physicians, physician assistants, and advanced practice providers) assume short-term direct care of individual patients. Further, unlike webinars or conventional didactic lectures, the ECHO model provides a digital discussion of cases with expert panels in real time that is highly contextualized and thereby fulfills key learning theories, such as deliberate practice [22], social cognitive theory [23], and situated learning and communities of practice [24]. As detailed in a previous publication [18], the ECHO model is based on the following key principles: (1) use of internet-based technology for multipoint videoconferencing, which helps leverage scarce resources; (2) use of an established evidence-based disease-management model that has been demonstrated to improve outcomes by reducing variation in processes of care and sharing best practices [17,25-27]; (3) use of case-based learning, based on discussion with experts and peers; (4) use of a digital community of practice, which emphasizes reciprocity in the sharing of skills and information; and (5) use of an internet-based database (ie, iECHO software [Hangzhou IECHO Science & Technology Co, Ltd]) to monitor outcomes.

Professionals involved in miners’ health are invited to attend the program using an emailed web link connected to a REDCap (Research Electronic Data Capture; Vanderbilt University) survey [28]. As published previously [18], the ECHO sessions are scheduled at the same time twice a month for 75 minutes, adhering to a standard format. Following a facilitator’s initial 10 minutes of introduction and announcements, an invited expert delivers a 15-minute didactic followed by a 20-minute facilitated question-answer session and a 30-minute digital facilitated case discussion. The medical director of the Miners’ Wellness ECHO program is trained to facilitate group discussions during a 3-day immersion program and is periodically retrained [29]. The ECHO format, employing adult learning principles, focuses more on active learning through discussion than on didactic training. Upon completing a session-specific survey, participants can receive continuing education credits without charge. A multidisciplinary committee of experts follows a structured curriculum that is continually adapted based on the needs of the virtual community of practice, which are identified through a review of the continuing evaluation reports.

Attendance at ECHO sessions is open and voluntary. Through session attendance, participants can participate in the didactic and case discussions, provide information and insight from their own experience, and receive mentoring from their peers and an expert panel. Outside the program sessions, participants continue to have access to peers and experts for urgent consultation requests via telephone or email. Over time, with iterative practice and feedback, participants gain additional expertise, and become more confident in their skill sets related to caring for miners in their respective fields [14,18,20]. Recorded didactic sessions are made available through a web-based archive. The didactic and case discussion topics were not repeated during this study’s time frame. All cases presented during this study’s time frame were resolved using the collective expertise of the multidisciplinary community of practice, which included experts. The University of New Mexico has a sustainable business model for its ECHO Programs, with the cost of personnel shared between university budgets and grants and contracts.

This 14-month exploratory sequential mixed methods study involved participants in the Miners’ Wellness ECHO Program from July 2018 to September 2019. Participants are invited from across the nation by email, flyers, and word of mouth referrals to join the ECHO sessions. Their session participation is voluntary. At the start of the session, participants are informed that the session is video recorded. Together, the University of New Mexico School of Medicine and its rural partner Miners’ Colfax Medical Center constitute the hub site of experts. The spoke partner sites are located across the pneumoconiosis mortality hot spot regions of the United States [18]. Although participants were allowed to attend multiple sessions, our analyses emphasize cross-sectional comparisons based on participant characteristics.

We asked the following 5 research questions on common ECHO processes based on the experience of this study’s team, which are also related to the engagement theory model that focuses on participation, engagement, and learning within communities of practice [30]:

Each ECHO session during this study time frame was video recorded and professionally transcribed. Participants were informed of the recording at the start of each session.

Using an exploratory approach from a naturalistic point of view, we conducted summative qualitative content analysis to code interprofessional discussions among ECHO participants in each video. Specifically, we examined participants’ verbal utterances, focusing on statements of expertise followed by acceptance or eschewal of the statements by other participants, using latent analysis for interpretation of content. We did not use a guiding theory, as this was exploratory. See Table 1 for the qualitative codebook with definitions of each. See Multimedia Appendix 1 for examples of conversations demonstrating statements of expertise and responses of acceptance and eschewal.

We realize that evaluating others’ acceptance or eschewal can be subjective and difficult to determine by reading transcripts. Additionally, body language and tone of voice can convey meaning that written words cannot. To minimize these risks, our qualitative team consisted of 4 people with varied and complementary areas of expertise: (1) the principal investigator (AS) who is an expert pulmonologist and medical director/session facilitator for the Miners’ Wellness Tele-ECHO Clinic; (2) an expert qualitative methodologist (HRB); and (3) two experts of ECHO methods and processes (director of program operations, Rachelle Rochelle, MPA, and senior program manager, Stephen Murillo, MBA). This team of 4 analysts met frequently during the project to interpret session content and discuss coding, potential biases, and reflexivity. We concurrently coded video recordings of participant behavior alongside transcripts using NVivo (version 12; QSR International). The senior analyst created the preliminary codebook and independently coded the presence of each code across all ECHO sessions. She met with the principal investigator and 2 other team members to review the trial coding of the first 2 sessions as a group, modifying coding and definitions as needed. The principal investigator independently reviewed 2 additional sessions, and the other analysts independently reviewed 5 sessions; therefore, 7 sessions were trial-coded by 3 (5 sessions) or 4 (2 sessions) team members. The team regularly met to discuss coding until they were 100% in agreement, at which point the senior analyst coded the remaining 16 sessions. She continued to meet with the team periodically to discuss her coding generally and anything that was coded to be “unclear” (as defined in Table 1), at which point they jointly decided on whether it was better suited for another code. We removed all statements of expertise, acceptance, and eschewal of the session facilitator (AS), so we only evaluated participants’ utterances. This approach minimized the bias resulting from the effect of the role and relationships of the session facilitator (AS) in the data analysis. There were no unintended consequences from the program participation or analysis. Since thematic saturation was not the goal of this study, the sampling was guided by the number of sessions rather than sampling saturation.

We worked with our senior statistician (OM) to convert our qualitative coding from NVivo into quantitative coding in Excel (Microsoft Corp). We coded additional variables to be analyzed with SAS (version 9.4; SAS Institute Inc). See Table 1 for the quantitative codebook.

We examined the primary outcome variable, “active participation in discussion,” defined by making any utterances (which included statements of expertise, acceptance, and eschewals). We analyzed the association of the outcome on the following predictor variables: (1) participant group status, (2) study time frame, (3) participant ECHO experience status, (4) concordance of participant group identity between presenter and participant, (5) video usage, and (6) attendance frequency.

Participant groups included attorneys, benefits counselors, clinical providers, home health professionals, respiratory therapists, and others. The participants’ ECHO experience status was determined by the participant’s date of joining the ECHO Program. Participants’ ECHO experience status was defined by (1) the experienced group: those who joined the Miners’ Wellness ECHO before the start of this study, May 9, 2018; (2) the intermediate group: those who joined at the start of this study, May 9, 2018; and (3) the less experienced group: those who joined after September 12, 2018. The participant groups were also classified as “silent” and “vocal” based on their preliminary analysis of their rates of utterances during the ECHO sessions. Participant group identity was considered concordant when the didactic presenter and participant belonged to the same participant group. Video status was determined if the video camera was turned on for any duration during a session by the participant.

The number of utterances by type (statements of expertise, acceptance, and eschewal) and overall for each participant and ECHO session were collected. An average number of utterances by participant group, session, and type were computed. Binary variables for any active participation in discussion (1=yes for ≥1 utterance in the session, 0=no for 0 utterances in the session) were made for each utterance type and overall. Frequencies and percentages for active participation variables and other categorical variables were compiled by utterance type, participant group, and session.

We used the generalized estimating equations approach to extend the generalized linear model to handle longitudinal data, including predictor variables. The logit link function was applied for binary outcomes. Repeated statements with the subject option identified each session as a cluster. LSMEANS were used to compute and compare least squares means of fixed effects. A 2-tailed P value less than .05 was considered statistically significant.

This study was an arm of a larger study. This arm of this study included participants who consented and enrolled in the larger study and those who were not enrolled in the larger study. This arm was approved as exempt by the institutional review board, University of New Mexico Human Research Protections Office (HRRC#18‐386). Informed consent was waived by the institutional review board for this arm, separate from the larger study. The study data were deidentified for analysis. No compensation was offered for this arm, separate from the larger study.

We studied 23 ECHO sessions during this study’s time frame, with 158 unique participants and 539 total participants. Further, 78 (49.4%) participants attended a single session, 28 (17.7%) attended 2 sessions, 11 (7%) attended 3 sessions, 13 (8.2%) attended 4‐5 sessions, 13 (8.2%) attended 6‐10 sessions, and 15 (9.5%) attended 11‐20 sessions (overall mean 3.4, SD 4.1 sessions). Individual session attendance averaged 23.4 (SD 5.6; median 24, IQR 8; range=13‐36) participants per session. During these sessions, 23 participants presented 23 patient cases, and 23 invited experts presented 23 didactics.

Clinical providers, the largest participant group constituting 36.7% (58/158) of unique participants, were also the most vocal group (mean 21.74, SD 2.11, average utterances per person-session; Table 2). Attorneys, while the smallest group in number (11/158, 6.9%), were the second most vocal group (mean 6.73, SD 4.29, average utterances per person-session). Benefits counselors and home health professionals were the least vocal groups, with average utterance rates of 0.57 (SD 0.29) and 2.86 (SD 2.24) per person-session, respectively, constituting 8.2% (13/158) and 22.8% (36/158) of unique participants, respectively. Thus, the various participant groups exhibited significantly different rates of utterances across sessions (P=.003).

Some topics generated a lot of discussion (ie, high rates of utterances), while others did not (Figure 1). Overall, utterances per person-session did not change over time (P=.50, Figure 1). It should be noted that in session 17, there were no utterances—this is the only session facilitated by a different person who was an experienced ECHO participant and expert clinical provider but not a trained facilitator. As shown in Table 2, most utterances are statements of expertise, and most statements are accepted. As shown in Figure 1, some topics generate a lot of controversy during discussion (ie, high rates of eschewal), while others do not.

Experienced and intermediate experienced groups accounted for 36.8% (n=104) and 26.9% (n=29) of active participation in discussion, respectively, while the active participation from the less experienced group accounted for 28.6% (n=40). Experienced participants may have, thus, tended to dominate active participation in discussion compared to participants with less or intermediate levels of experience, but this difference was not statistically significant (P=.11, Table 3).

Participant group identity was considered concordant when the didactic presenter and participant belonged to the same participant group. For this analysis, the participant groups were categorized as silent versus vocal, in which the benefits counselors and home health professionals constituted the former group and the clinical providers, attorneys and respiratory therapists in the latter group (with the other or unknown group excluded). When the didactic presenter and participant were from the same participant group, active participation in discussion (measured by any utterances within an individual ECHO session) by the silent group participants was 26.9% (n=7) compared to 9.5% (n=14) when the didactic presenter was from a different group (odds ratio 3.5, 95% CI 1.47-8.54, Table 4). Among vocal group participants, active participation was 46.4% (n=96) when presenters were in the same group as the participant, compared to 46.2% (n=43) when presenters and participants were from different groups (odds ratio 1.01, 95% CI 0.58-1.75), with an interaction P=.003.

Compared to those joining a session by audio, those joining by video tended to have higher rates of active participation in discussion, as measured by any utterance during an individual session. However, this comparison did not reach statistical significance (Table 5, P=.11).

Unique participants’ attendance ranged from 1 to 20 sessions and was categorized into 3 groups (Table 6). Overall, active participation in the discussion was significantly associated with attendance (P=.004); however, no dose response was noted.

In this digital community of practice for managing pneumoconiosis with previously published efficacy outcomes [14,18,20], our study findings show that some participant groups are vocal while others are silent, based on utterance rates during interprofessional discussion. Some topics generate a lot of discussion (ie, relatively high rates of utterances), and some topics generate a lot of controversy during discussion (ie, relatively high rates of eschewal of statements of expertise). Inadequately trained facilitators may impede discussion during a session. Experienced participants may dominate the discussion and inhibit participation by less experienced participants, but this association did not reach statistical significance (P=.11, Table 3). When the didactic presenter and participant are from the same participant group, participation in discussion by the silent group participants rises significantly. Compared to those participating by audio, those participating by video camera may tend to have higher rates of participation in discussion, but this association did not reach statistical significance. Overall, active participation in the discussion was significantly associated with attendance frequency. Our study findings lend themselves to several best practice recommendations, as discussed below.

The primary objective of the Project ECHO movement is to decentralize knowledge for the care of patients through exchanging insights and information using the all teach, all learn principle. All participants have unique knowledge sets, and discussion inside and outside the ECHO session within and across participant groups facilitates the transfer of knowledge that would otherwise remain siloed within individual participant groups. Further, interprofessional discussion in the ECHO model may allow greater access to new and thought-provoking ideas and perspectives that foster learning and other growth-enhancing actions. Our study findings confirm the existence of silent participant groups that, by not fully engaging in discussions, may not be as effective in exchanging knowledge (including providing and acquiring knowledge) within the ECHO session. These groups need to be actively supported by a trained facilitator and by the judicious use of didactic presenters sharing participant group identity concordant with the silent group participants. However, our previous objective measurement of knowledge transfer using social network principles indicates that benefits counselors are among the groups most effective in knowledge transfer [18]. This prior finding would suggest that either benefits counselors convey relevant knowledge succinctly within the ECHO sessions or participate in knowledge transfer outside the strict bounds of the ECHO program via follow-up phone calls, emails, and virtual and physical meetings. This finding warrants further research as it is a possible indication that the informal “social network” system, perhaps provoked by or aided by the ECHO sessions, may be more important than and reinforce the formal sessions.

Project ECHO formally trains its facilitators on the best practices for managing discussions in a digital community of practice. Substituting a trained facilitator by another untrained expert can affect participation in the discussion. Experienced participants usually offer the greatest rates of statements of expertise, acceptance, and eschewal, reflecting their expert status in this complex field. However, the dominance of experienced groups can be avoided by the facilitator actively encouraging less experienced groups to make their minority opinions heard during discussion. Using video cameras during participation in this digital community of practice helps build community and promotes accountability and engagement at the cost of rural internet bandwidth. This practice should be encouraged, even though the findings did not reach statistical significance.

Our study has multiple strengths. To the best of our knowledge, our approach of studying interprofessional discussion in a digital “community of practice” for managing pneumoconiosis has never been used previously by another group of investigators. This study involves the innovative ECHO model intervention that addresses barriers to the care of miners by providing a multidisciplinary community of practice approach, which has been well studied in other diseases [17,25-27]. This study is topical and significant because it addresses a critical gap related to the emerging pneumoconiosis epidemic in rural United States. Since no one refused participation, there was likely no potential participation bias in this study. Since the ECHO model has been adopted nationwide and worldwide to care for patients with numerous chronic diseases, infrastructure already exists to rapidly scale the Miners’ Wellness ECHO Program nationwide and worldwide. Other strengths include the detailed contemporaneous interpretation of videos and transcripts of discussions; strict quality control in qualitative analysis; a mixed methods study design; and the simultaneous use of qualitative, quantitative, and subject content experts as coinvestigators.

Although our study focused on the ECHO program, which emphasizes chronic disease management with a strong focus on mentorship and collaborative learning, our findings have broader implications for other digital communities of practice that may be structured differently and operate differently. Examples include Elpha (Women in Tech Network), a community of practice for women in technology; Stack Overflow (Stack Exchange Inc) for programmers and developers; Digital Nomad World for nonprofessional digital nomads [31]; and various leadership communities of practice that emphasize succession planning and leadership skills development [32]. Such unrelated communities of practice can also benefit from our evidence-based findings and recommendations to reach out to silent groups, break down silos, provide facilitator training, and encourage active participation. On the other hand, the ECHO program could learn from other communities of practice on how to reward active participation and knowledge sharing (Stack Overflow) or create asynchronous platforms to supplement synchronous engagement (Digital Nomad World).

There are also limitations to this study. We cannot correlate interprofessional discussion for knowledge transfer to patient outcomes or changes in provider behavior. However, we have previously published a listing of qualitative changes our ECHO participants reported they would make in their practice, obtained as part of a continuing medical education survey requested at the end of each ECHO session [20]. Participants whose group identity was other or unknown were 11.7% (63/539) of the total participant representation in all sessions. When analyzing participant group identity (as shown in Table 4), the other or unknown group was excluded. Although a small sample size raises the possibility of a type I error, individual professionals and teams of professionals trained in the ECHO model can reach a large number of miners, with the potential for creating exponential change. Miners impacted with pneumoconiosis do not participate in the ECHO model. This model characteristic is, however, not a limitation for the following reasons. The model ensures that knowledge is disseminated widely and effectively by focusing on provider education. By reaching out to more miner professionals, the ECHO model indirectly benefits more miners and is more scalable than traditional telemedicine models focusing on individual patients. Case-based learning, where providers present anonymized patient cases to a panel of experts, allows for in-depth discussion and learning without compromising patient confidentiality while maintaining the traditional provider-patient relationship [25,33].

Our findings provide insight into the mechanics of interprofessional discussion for knowledge transfer in a digital community of practice managing pneumoconiosis and potential recommendations to enhance the same. Our results underscore the capacity of the Project ECHO model to leverage technology and workforce diversity to facilitate interprofessional discussions on the multidisciplinary and complex care of miners and ultimately promote health equity among rural and medically underserved mining communities. Although this approach addresses a critical gap related to the emerging pneumoconiosis epidemic, future research will evaluate whether this translates into improved patient outcomes in mining communities, a priority need in rural United States.