This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.
Successful best practice implementation is influenced by access to peer support and knowledge exchange. The Toronto Stroke Networks Virtual Community of Practice, a secure social media platform, is a knowledge translation tool supporting dissemination and adoption of stroke best practices for interprofessional stroke stakeholders.
The aim of this study is to evaluate the use of a virtual community of practice (VCoP) in supporting regional stroke care best practice implementation in an urban context.
A mixed methods approach was used. Qualitative data were collected through focus groups and interviews with stroke care provider members of the VCoP working in acute and rehabilitation settings. Thematic analysis was completed, and the Wenger Value Creation Model and developmental evaluation were used to reflect practice change. Quantitative data were collected and analyzed using website analytics on VCoP use.
A year after implementation, the VCoP had 379 members. Analysis of web analytics data and transcripts from focus groups and interviews conducted with 26 VCoP members indicated that the VCoP provided immediate value in supporting user networking, community activities, and interactions. Skill acquisition and changes in perspective acquired through discussion and project work on the VCoP were valued by members, with potential value for supporting practice change. Learning about new stroke best practices through the VCoP was a starting point for individuals and teams to contemplate change.
These findings suggest that the VCoP supports the early stages of practice change and stroke best practice implementation. Future research should examine how VCoPs can support higher levels of value creation for implementing stroke best practices.
In Canada, stroke care is guided by the
Communities of practice (CoPs) are groups of people “sharing a concern, a set of problems, or a passion about a topic, and who deepen their knowledge and expertise in this area by interacting on an ongoing basis” [
VCoPs are a potentially valuable tool for supporting the implementation of stroke care best practices, as they provide a mechanism to share knowledge among stakeholders across geographical and organizational boundaries. VCoPs also provide a virtual space for care providers from different disciplines to connect. VCoPs provide stakeholders with a platform to share strategies and outcomes of best practice implementation initiatives in different contexts, allowing them to leverage learning from others to make the process of quality improvement more efficient.
The Toronto Stroke Networks (TSNs) work collaboratively with stakeholders to implement high-quality stroke care and have an established education and KT infrastructure grounded in the
The aim of this study is to evaluate the use of the TSNs VCoP by members and the value created in supporting regional dissemination and implementation of stroke care best practices using a mixed methods approach. This study adds to the small existing body of literature on the use and value of VCoPs to support best practice implementation in health care contexts.
A mixed methods approach was used in this study to evaluate the TSNs VCoP using both quantitative data on site use and qualitative data on value created through the use of the VCoP. Developmental evaluation [
Developmental evaluation seeks to enhance the “understanding [of] the activities of a program operating in dynamic, novel environments with complex interactions” [
Although developmental evaluation provides a structured method of assessment, the Wenger Value Creation Model [
Data on VCoP use were collected 1 year after implementation of the VCoP by 2 university students supervised by a member of the TSNs team (JF), including quantitative, aggregate web analytics data (eg, number of members and number of site visits), quantitative user-level data (eg, number of discussion posts), and qualitative user-level data (eg, questions asked in discussion forums and replies to discussion posts). Quantitative data were collected manually from the VCoP and using Google Analytics. Qualitative user-level data were collected manually from the VCoP and did not contain any identifying characteristics of users.
VCoP members were contacted with a request to participate in a focus group or semistructured interview via email and through the VCoP by a member of the TSNs team (JF). Semistructured interviews and focus groups were conducted by a member of the TSNs team (JF) with support from 2 university students following an interview guide with prompts based on the Wenger Value Creation Model [
To analyze the data collected using web analytics, descriptive analysis of quantitative data on VCoP use was completed using Microsoft Excel (version 2015, Microsoft) to determine the total numbers of page visits, posts, VCoP members, and other indicators of VCoP engagement within 1 year following implementation. Narratives from VCoP discussion forums were analyzed using the analysis framework for this study based on the Wenger Value Creation Model [
To analyze the data collected from interviews and focus groups with VCoP members, transcripts from audio recordings of interviews and focus groups were generated and analyzed using thematic analysis [
The analysis was conducted individually by two university students and two authors (MD and EL) with expertise in KT. Manual coding of transcripts was performed to create frameworks by each individual. Subsequently, individual coding frameworks were compared manually to reach a consensus and develop a preliminary coding framework. This framework was then applied in recoding the transcripts. The resulting themes were collectively reviewed for validation through consensus. The trustworthiness of the findings was supported by review and validation of themes by authors who did not participate in the interviews and focus groups (MD and EL). An audit trail of coding changes was created to document the process of analysis and ensure consistency between coders.
Quantitative and qualitative data from web analytics and interviews and focus groups with participants were integrated using the evaluation framework developed for this study (
Ethical review and approval for this study were provided by the Sunnybrook Health Sciences Centre Research Ethics Board before the collection of data (approval number 123-2013). Quantitative data on VCoP use were anonymous, and qualitative user-level data were anonymized and deidentified at the time of collection. Informed consent to participate in interviews and focus groups was provided by VCoP members interested in participating in the study. Any identifying details presented in interviews were omitted from transcripts.
VCoPa evaluation framework using the Wenger Value Creation Cycles.
Cycle and cycle indicator | Quantitative data (web analytics) | Qualitative data (interviews and narratives from VCoP) | |
|
|||
|
Level of participation | Number of threads, total number of members, number of members in a group, number of members in discussion forums, and number of discussion forums | Questions or statements related to joining a discussion forum |
|
Quality of interaction | Frequency of responses to inquiries, frequency of citing one’s own experience, post length (words), number of debates or differing points of view, and number of suggestions made to a problem | Questions or statements about a clinical issue and sharing of a case example |
|
Networking | Number of group memberships | Question or statement about a need that can be met by the expertise of another member and request connection with a member’s particular knowledge |
|
Collaboration | Number of joint projects and timeliness of responses | Question or statement about a collaborative project and statement about length of time a member waited for response to a question |
|
|||
|
Skills acquired | Number of documents | Question or statement about a document on the VCoP that was used or shared with colleagues |
|
Change in perspective | —b | Statements indicating a shift in understanding or opinion |
|
Confidence building | — | Questions or statements indicating disagreement or challenging posts of other members |
|
|||
|
Implementation of advice, solutions, and insights | — | Questions or statements about successes or challenges related to applying new learning from the VCoP |
|
Use of social connections | — | Questions or statements about networking with other VCoP members |
|
|||
|
Personal performance | — | Statements about personal goals |
|
Organizational performance | — | Statements about organizational goals or accomplishments |
|
Organizational reputation | — | Statements about organizational performance relative to other benchmarks (eg, Ministry of Health and Long-Term Care and Health Quality Ontario) |
|
|||
|
Community aspirations | — | Questions or statements that indicate new purpose for the VCoP or improvements and/or additions to the community |
|
Relationships with stakeholders | — | Questions or statements about relationship building or with others external to the VCoP (eg, patients or other organizations) |
aVCoP: virtual community of practice.
bNo quantitative data identified.
A year after implementation, 379 members had joined the VCoP from 22 organizations in the TSNs representing several professional backgrounds: nursing, occupational therapy, physical therapy, speech-language pathology, medicine, academic research, and other health system stakeholders. Of these 379 members, 26 (6.9%) provided informed consent and participated in 14 interviews and 2 focus groups. Participants included nurses, occupational therapists (OTs), physicians, physiotherapists (PTs), and speech-language pathologists (S-LPs) from 4 rehabilitation and 8 acute care organizations. Overall, 19 participants provided data for number of years in practice, with the mean number of years practicing being 15.6 (SD 8.7; range 2-29.5) years.
The data collected from the VCoP through web analytics reflecting cycles 1 and 2 of the evaluation framework are summarized in
Quantitative data summary.
|
Value | ||
|
|||
|
Virtual community of practice members, n | 379 | |
|
Groups, n | 24 | |
|
Discussion forums, n | 21 | |
|
|||
|
Threads initiated in each discussion forum, mean (SD) | 1.04 (1.4) | |
|
Discussion threads in each group, mean (SD) | 1.0 (1.4) | |
|
Posts in each thread, mean (SD) | 2.5 (5.1) | |
|
Prompts and questions per discussion forum, n | 28 | |
|
Responses to inquiries, n | 27 | |
|
Instances own experience cited, n | 6 | |
|
Debates about a topic, n | 0 | |
|
Length of posts (number of words), mean (SD) | 67.2 (65.1) | |
|
Members in each group, mean (SD) | 17.0 (41.2) | |
|
Members in a discussion forum, mean (SD) | 1.9 (1.7) | |
|
|||
|
Documents shared, n | 117 | |
|
Suggestions made to a problem, n | 31 | |
|
Joint projects, n | 9 | |
|
Timeliness of responses (number of days), mean (SD) | 18.6 (14.2) |
The subjective value of the VCoP and use patterns were collected using interviews and focus groups with VCoP members. Through a qualitative thematic analysis of interview and focus group transcripts, the research team identified 5 themes discussed in the next sections.
Participants noted that a primary function of the VCoP is to provide a more effective platform for contacting individuals and building networks within and across disciplines. This enhanced connectivity was primarily achieved through the use of the VCoP as a mechanism to find members’ organizational contact information and directly message them through this platform. Participants noted that they leveraged the VCoP network to assist colleagues who were not VCoP members, suggesting the spread of VCoP value beyond the immediate member network:
...I helped a colleague here who needed to contact somebody...and you know, through the VCoP I knew exactly who she needed to contact. I got her number through the member directory.
What is meaningful to me is directly messaging other members, I can network with people in the same discipline.
Participants also discussed how they leveraged the VCoP to support collaborations within and between professions on ongoing stroke projects. Members also used the VCoP to collaborate with individuals on posters for conferences, which contributed to knowledge transfer outside the VCoP:
We’ve been using [the] VCoP for the development of posters...to share ideas & information about the process, and then post the actual creation of posters on VCoP...that was our only means of communication at that point...I was able to liaise with different [colleagues] about different practice concerns...the two posters I worked on.
This project-based collaboration on the VCoP was noted to provide additional educational opportunities. For example, individuals not directly involved in he projects benefited from observing the collaborative process in public groups, leading to the development of conference posters. In addition, VCoP members identified that conducting project-based work on the VCoP led to the development of practice resources:
I have learned things from reviewing the other posters that I haven’t been involved in because I was part of the online group and saw what was posted.
[The] VCoP was the means of communication for a group project...afterwards, I created additional assessment checklists...and shared it with my team.
Participants identified that access to up-to-date information and resource sharing improved with VCoP use. The VCoP was noted as beneficial for identifying topic-specific resources more easily and providing a central location for resources. This increased accessibility saved members’ time that would otherwise have been spent searching for these resources:
Now with the VCOP, I will not have to go back to my desk to refer to the assessment checklist as it is all online now.
I printed the triage tool that someone uploaded to the VCoP and gave it to new people on my team for reference.
VCoP members also noted that these resources had an impact beyond their immediate use. Resource sharing on the VCoP provided a catalyst for in-person knowledge sharing about best practices across institutions. Although a direct link to changes in practice was not identified, participants noted that VCoP resources started discussion about changes in practice that could lead to future changes:
I shared information with a group that sparked discussion about [specialized] assessments, the comparison between hospitals prompted discussions here with our [professional group] about whether we should change our ways of doing things.
Participants identified that the VCoP helped them understand the broader picture of stroke best practices within their region beyond their disciplinary boundaries. Participants noted that the VCoP helped them access information to support learning about stroke Quality-Based Procedures (QBPs) [
Looking up the information about the QBP is very helpful to get someone new to understand how it impacts the [stroke] program...it gives them a leverage point of how they can advocate for the stroke patients.
Multiple participants noted that the information shared on the VCoP went beyond what was available from other web-based sources and was more regionally relevant. Learning about activities and concerns outside one’s practice was made possible through a diverse membership on the VCoP from across professions, sectors, and organizations within the region:
So if you really want to find out what’s happening across the stroke community, you have that opportunity more at your fingertips than what you did before.
It’s information that you’re not going to necessarily find on the internet because it’s about current practice.
I have a better awareness of the best practices and what the bigger picture is. When you work in one area (e.g. rehab) you don’t really have a big sense of what was going on in acute care, what they were looking at for outpatient services, community service...being a part of the community and through looking at some of the resources.
Although themes of benefits were identified, challenges with accessing resources were also recognized as barriers to VCoP use. Members noted that the navigation path to certain resources was too long, making them difficult to find. Frustration over inability to find resources and time spent searching for them was noted as a deterrent to future use:
If it takes less time to find it, right, it’s like you’re spending like 10 minutes and you’re like, “Okay, I’m not going to do this again,” right?
Email notifications were suggested as a strategy to increase engagement in preferred topics and previous discussions in which individuals had participated. This suggested that improvement may decrease the time spent searching the site for information and challenges with navigation:
...if there was a new tag or a new thing, something pops up into my e-mail, to, kind of, go answer it...what about an option to receive updates via email, or notifications based on individuals’ interests?
The results from both quantitative and qualitative analyses were applied to the evaluation framework. These results provide evidence for value creation associated with VCoP reflective of cycles 1 to 3 of the Wenger Value Creation Model (Immediate Value, Potential Value, and Applied Value, respectively;
Mapping of quantitative data and qualitative themes to an evaluation framework reflecting the Wenger Value Creation Model (cycles 1-3). VCoP: virtual community of practice.
The results of this evaluation strongly support the creation of
Although evidence for value creation reflective of cycle 3 was weaker than that of cycles 1 and 2 in this study, some narratives and data on the number of projects, number of resources shared, and number of solutions to problems offered by members support the initial steps toward best practice implementation. Several VCoP member narratives described sharing of resources as helping individuals and teams make progress toward changes in practice. For example, VCoP members interacting in a group (cycle 1) supported the generation of knowledge capital such as an educational poster (cycle 2). In the process, members learned from each other and brought information back to their teams, prompting discussions about practice changes (cycle 3).
Evidence of the value created by a VCoP to support stroke best practice implementation in this study aligns with existing KT frameworks that identify the importance of increasing awareness of best practices and identifying practice gaps. For example, the KTA Framework [
This evaluation found few instances of members sharing their experiences or debating issues, suggesting that this community is still in the process of building social capital. Social capital is created in social networks [
The patterns of and reflections on VCoP use identified in this study may pose challenges to value creation on the VCoP. Web analytics data indicated slow response times for users to address questions posed by others on the VCoP and instances of as few as 2 individuals having web-based exchanges in some discussion forums, whereas other forums had extensive resource sharing. Slower response times and inconsistent engagement across different discussion forums could challenge value creation on the VCoP by discouraging members from posing questions if they do not think they will receive a timely response or any response at all. Although some VCoP members identified that the VCoP saved them time in the long run when it came to finding resources, others, as supported by previous literature [
The self-selection of VCoP members to participate in the focus groups and interviews, rather than random sampling, creates a potential sampling bias that could impact the generalizability or transferability of this work to different regional contexts or KT applications. Additional insights provided by less active members could improve understanding of VCoP value, as active members outweighed nonactive members participating in focus groups and interviews.
Participants included health care providers in acute and rehabilitation hospital settings, which represents VCoP membership demographics at the time of data collection. Membership has since expanded to include more VCoP members from nonhospital settings. Although the VCoP has an interprofessional membership, most participants in focus groups and interviews were allied health professionals. A more diverse group of participants may affect the value expressed by VCoP members for health professions involved in stroke care.
In addition, this study was conducted to evaluate a regional best practice implementation tool. Study participation included users in a large metropolitan area. Although there are existing face-to-face opportunities to build social capital in this region, challenges for stroke care providers to access these opportunities exist (eg, taking time away from care provision to commute to in-person events) [
Through a developmental evaluation approach, the value of a VCoP in supporting the initial phases of stroke best practice implementation was demonstrated in this study. Members reported enhanced networking, more efficient identification of resources, and enhanced collaboration on joint projects to build and maintain professional relationships. VCoP use contributed to a broader understanding of the stroke continuum of care and a better understanding of the priorities in stroke care that supported individuals and teams to contemplate best practice change. Future VCoP design changes to improve functionality and user experience were suggested to increase collaboration and the quality of engagement. The evaluation framework used in this study will continue to be used to collect evidence of the value created by the TSNs VCoP as an ongoing KT initiative. The use of a VCoP in other care networks and regions should be investigated to gauge the potential value of this educational strategy for health professionals working with other populations seeking to enhance best practice implementation.
community of practice
knowledge translation
Knowledge to Action
occupational therapist
physiotherapist
Quality-Based Procedure
speech-language pathologist
Toronto Stroke Network
virtual community of practice
The authors would like to acknowledge Joanne Fortin for leading the conceptual plan to establish and implement the Toronto Stroke Networks Virtual Community of Practice.
None declared.