This paper is in the following e-collection/theme issue:

Formative Evaluation of Digital Health Interventions (2491)

Published on in Vol 3, No 3 (2019): Jul-Sep

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/12216, first published .
An Interactive Website for Whiplash Management (My Whiplash Navigator): Process Evaluation of Design and Implementation

An Interactive Website for Whiplash Management (My Whiplash Navigator): Process Evaluation of Design and Implementation

An Interactive Website for Whiplash Management (My Whiplash Navigator): Process Evaluation of Design and Implementation

Authors of this article:

Aila Nica Bandong1, 2 Author Orcid Image ;   Martin Mackey1 Author Orcid Image ;   Andrew Leaver1 Author Orcid Image ;   Rodney Ingram1 Author Orcid Image ;   Michele Sterling3, 4 Author Orcid Image ;   Carrie Ritchie3, 4 Author Orcid Image ;   Joan Kelly5 Author Orcid Image ;   Trudy Rebbeck1, 6 Author Orcid Image
Article Authors Cited by (6) Tweetations (1) Metrics

    Original Paper

    1Discipline of Physiotherapy, Faculty of Health Sciences, The University of Sydney, Sydney, Australia

    2Department of Physical Therapy, College of Allied Medical Professions, The University of the Philippines, Manila, Philippines

    3Recover Injury Research Centre, The University of Queensland, Brisbane, Australia

    4Centre of Research Excellence in Road Traffic Injury Recovery, The University of Queensland, Brisbane, Australia

    5Menzies Health Institute Queensland, Griffith University, Gold Coast, Australia

    6Kolling Institute, John Walsh Centre for Rehabilitation Research, The University of Sydney, Sydney, Australia

    *all authors contributed equally

    Corresponding Author:

    Aila Nica Bandong, BS, MPT

    Discipline of Physiotherapy

    Faculty of Health Sciences

    The University of Sydney

    75 East Street

    Lidcombe

    Sydney, 2141

    Australia

    Phone: 61 452578705

    Email: aban9213@uni.sydney.edu.au


    Background: Whiplash is a health and economic burden worldwide. Contributing to this burden is poor guideline adherence and variable management by health care professionals (HCPs). Web-based tools that facilitate clinical pathways of care are an innovative solution to improve management.

    Objective: The study aimed to develop, implement, and evaluate a Web-based tool to support whiplash management following a robust process.

    Methods: This study followed the first 3 processes of a research translation framework (idea generation, feasibility, and efficacy) to inform the development, implementation, and evaluation of a website that supports HCPs in whiplash management. Development followed the idea generation and feasibility processes to inform the content, design, features, and functionality of the website. This involved stakeholder (eg, industry partners, website developers, and HCPs) consultations through face-to-face meetings, surveys, and focus group discussions. Implementation followed the feasibility process to determine the practicality of the website for clinical use and the most effective strategy to promote wider uptake. Implementation strategies included classroom education, educational meetings, educational outreach, reminders, and direct phone contact. The analysis of website use and practicality of implementation involved collection of website metrics. Evaluation followed the feasibility and efficacy processes to investigate the acceptability and extent to which the website assisted HCPs in gaining knowledge about whiplash management. Surveys were conducted among student, primary, and specialist HCPs to explore ease of access, use, and satisfaction with the website, as well as self-rated improvements in knowledge of risk assessment, management, and communication between HCPs. Website logs of specialist management decisions (eg, shared care, specialist care, and referred care) were also obtained to determine actual practice.

    Results: The development process delivered an interactive, user-friendly, and acceptable website, My Whiplash Navigator, tailored to the needs of HCPs. A total of 260 registrations were recorded from June 2016 to March 2018, including 175 student, 65 primary, and 20 specialist HCPs. The most effective implementation strategies were classroom education for students (81% uptake, 175/215) and educational meetings for primary HCPs (43% uptake, 47/110). Popular pages visited included advice and exercises and risk assessment. Most HCPs agreed that their knowledge about risk management (79/97, 81%) and exercises (85/97, 88%) improved. The specialists’ most common management decision was shared care, an improvement from a previous cohort. Areas to improve were navigation and access to outcome measures.

    Conclusions: A robust process resulted in an innovative, interactive, user-friendly, and acceptable website, the My Whiplash Navigator. Implementation with HCPs was best achieved through classroom education and educational meetings. Evaluation of the website showed improved knowledge and practice to be more consistent with a risk-based clinical care pathway for whiplash. The positive results provide sufficient evidence to scale implementation nationally and involve other target markets such as people with whiplash, insurers, and insurance regulators.

    JMIR Form Res 2019;3(3):e12216

    doi:10.2196/12216

    Keywords

    primary health care; whiplash injuries; clinical decision support; clinical pathways; rehabilitation 


    Background

    Whiplash-associated disorders (WAD) remain a huge health and economic burden internationally [1-3], with 50% of people experiencing persistent pain and disability [4-6]. One possible contributor to this burden is that WAD guidelines are inconsistently applied by health care professionals (HCPs) [7,8]. Inconsistent and costly practices include high rates of imaging, excessive use of passive treatments, and delayed and poorly directed specialist referral [8-13]. These practices persist despite extensive and strategic guideline implementation [14-16], highlighting the need for adopting other strategies to promote guideline-based care.

    Clinical pathways of care are a promising strategy to improve HCP practice by incorporating guideline recommendations into health care processes [17]. A clinical pathway of care for WAD was developed that uses risk assessment [18,19] and guideline-based treatments matched to the risk of nonrecovery [20]. People at low risk require less intensive treatment and are provided with existing guideline-based resources [21,22]. People at medium and high risk of nonrecovery are referred early to HCPs who specialize in WAD (specialist HCPs), as recommended by WAD guidelines [21]. The specialist HCP more thoroughly assesses the physical and psychological factors associated with nonrecovery. This model of care has been successfully implemented in people with low back pain [23] but is yet to be tested and implemented in people with WAD.

    Web-based tools such as computerized decision support systems are one of the most effective strategies to support the implementation of clinical pathways and guidelines, thereby improving HCP practice [24-26]. In musculoskeletal health, these tools are used to provide clinical information, guide triage, and match patients to appropriate resources and treatments [27]. These tools are shown to be effective in improving HCP practice in the management of conditions such as hypertension, diabetes, and osteoarthritis [28,29]. The positive results in other conditions suggest that a Web-based tool might also be an effective strategy to support the WAD clinical pathway of care.

    To be effective, Web-based tools need to be developed and implemented using a robust process that ensures maximum uptake by users. Research translation frameworks describe a process from idea generation right through to implementation, evaluation, and then monitoring. For most innovations, such frameworks are followed over decades until an innovation is fully implemented and operational. Although several frameworks exist to guide the process [30], the New South Wales (NSW) Translational Research Framework was applied to develop a suitable Web-based tool for HCPs within the NSW and similar Queensland (QLD) health schemes [31]. Important initial steps of the framework are idea generation, feasibility, and efficacy [31]. Idea generation involves an integrated, multidisciplinary, collaborative approach to develop an innovative intervention [31]. Feasibility determines acceptability and viability of the intervention before further testing and attempts to answer the question of whether the intervention is practical to implement [31]. Implementation strategies known to change practice [14,16,32-35] may be used in this step to determine the practicality of implementing the intervention. During both idea generation and feasibility, stakeholder engagement is crucial to ensure suitability and acceptability of the intervention [31]. Finally, efficacy assesses whether the intervention can deliver expected outcomes under best circumstances. Results of these initial steps gather useful insights to inform further improvements and determine the capacity of the intervention to be successfully scaled up for greatest impact.

    Process evaluation is an essential methodology that is used to evaluate the change process at each stage of intervention development and implementation [36,37]. This methodology assesses elements such as reach, effectiveness, and influencing factors to optimize the design and evaluation of the intervention [36,37]. Interventions developed following similar processes have been successful in delivering acceptable, easy-to-use, and effective Web-based tools for conditions such as rheumatoid arthritis [38], obesity [39], and cancer [40].

    Objective

    This study used process evaluation methodology to report on the development, implementation, and evaluation of a Web-based tool to support HCPs in the management of WAD.


    Design

    This study followed the stages of development, implementation, and evaluation, incorporating the first 3 processes of the NSW Translational Research Framework [31] within each stage (Figure 1). The study involved both quantitative and qualitative methodologies. Ethics approval was provided by the University of Sydney (2015-444) and Griffith University (2015-707) Human Research Ethics Committees.

    ‎
    Figure 1. Process diagram summarizing the key stages, research translation process, and target markets to deliver the My Whiplash Navigator website. WAD: whiplash-associated disorder.
    View this figure

    Development

    The aim of the development stage was to deliver a suitable website to support HCPs in WAD management. During this stage, the idea generation and feasibility processes of the framework were followed.

    The idea generation process first involved stakeholder consultation. Initial consultations between researchers from the University of Sydney and University of Queensland identified key stakeholders, including industry partners, primary HCPs, specialist HCPs, and people with WAD. Second, face-to-face meetings were held with industry partners, the compulsory third party (CTP) insurance regulators (ie, NSW State Insurance Regulatory Authority and QLD Motor Accident Insurance Commission). Third, a survey (idea generation questionnaire) was conducted among primary and specialist HCPs and people with WAD to explore their perspectives about existing and ideal WAD resources. Primary and specialist HCPs and people with WAD were invited from our existing research databases. The survey was administered using the Research Electronic Data Capture (REDCap) tool hosted at the University of Sydney [41]. Participants rated their opinions on current resources using a 5-point Likert scale (1=strongly disagree to 5=strongly agree), and data were analyzed using descriptive statistics. The final step involved selecting and then consulting with a vendor who had experience in building interactive health education websites. A transparent tendering process was followed to select a suitable vendor to design the website.

    The feasibility process in the development stage involved face-to-face meetings with industry partners and website designers, followed by focus groups discussions among primary and specialist HCPs. A total of 20 work-in-progress meetings were held over 8 months to develop the website, involving 4 policy makers, 8 researchers, 6 clinicians, and 4 website designers. Discussions included design concepts, content, functionality, maintenance and support, milestones, and administration. The researchers had input to the content and sought feedback from key experts during this process. The vendor’s role was to ensure the content was adapted to be suitable for website and target market use.

    A total of 6 semistructured focus group discussions were conducted among primary and specialist HCPs to explore perceptions on ideal features, functionality, and content that would facilitate HCP uptake and use. The discussions were run by 2 members of the research team, who were guided by a set of key questions (Multimedia Appendix 1). Proceedings were audio-recorded, transcribed verbatim, and analyzed thematically [42]. Focus group methods have been described in detail elsewhere [43,44].

    Before implementation, the website was piloted with 5 HCPs to test the website features and functionality, resolve issues, and further refine the website.

    Implementation

    The aim of the implementation stage was to determine the most effective and feasible strategy that could inform future scalable implementation of the website before public release. At this stage, the feasibility process of the framework was followed to also evaluate the practicality of the website. Target markets for this stage were students and primary and specialist HCPs, ensuring a range of clinical experience from novice to experienced HCPs.

    Student HCPs were engaged through classroom education. The website was integrated as a learning resource in a unit of study for physiotherapy students at the University of Sydney (n=215). Key components of the educational module included standard assessment for WAD, risk assessment, provision of treatments for people with WAD at low risk of nonrecovery, and timely and appropriate referral to specialist HCPs for people with WAD at medium and high risk of nonrecovery.

    Primary HCPs were engaged through educational courses, educational outreach, and reminders. We targeted 3 professional development courses in WAD that were conducted by the Australian Physiotherapy Association over a 12-month period. A total of 110 physiotherapists attended these courses. The key features and components of the website were explained and integrated into the WAD educational modules. In addition, the website was promoted through 4 conference presentations. Primary HCPs who attended the courses and consented to participate were followed up with educational outreach, either face-to-face or over the phone, and provided with standard educational packages. In addition, reminders were sent through social media (eg, Facebook) and regular newsletters.

    Specialist HCPs (n=20) were engaged through 2 targeted educational workshops. The content of the first educational workshop was based on results of a qualitative study, focusing on the role of specialist HCPs, management decisions for people with WAD at medium and high risk of nonrecovery, and effective communication with primary HCPs [43]. Key features and use of the website that facilitate the above were also discussed during the educational workshop. A follow-up face-to-face meeting was conducted after 6 months to address issues such as access to certain resources and documentation of management decisions.

    Analysis of website use and the practicality of website implementation involved assessment of reach and extent of engagement by HCPs. Website traffic and trends (eg, registered users, total visits, total page views, and top landing pages) were collected through Google analytics and built-in website reports.

    Evaluation

    The aim of the evaluation stage was to investigate acceptability and preliminary efficacy of the website among HCPs. The framework processes followed were feasibility and efficacy. This was achieved by inviting all HCPs registered on the website to complete an evaluation questionnaire and determining actual practice through website logs.

    The feasibility process in the evaluation stage assessed acceptability of the website by students and primary and specialist HCPs. The evaluation questionnaire asked questions regarding ease of access, use, and satisfaction with the website. At least 10 website users were necessary to obtain enough data for evaluation. This number was determined a priori and based on available evidence and previous studies that explored usability of Web-based resources for management of other conditions [38,45,46].

    Efficacy was evaluated by measuring the extent to which the website assisted HCPs in gaining knowledge of key components of WAD management. Students and primary and specialist HCPs were asked about their self-rated improvement in knowledge of risk assessment and management of people with WAD at low risk and those at medium and high risk of nonrecovery. Furthermore, primary and specialist HCPs were asked about their opinion whether the website facilitated communication between HCPs. This item was specifically asked to primary and specialist HCPs, given their access to the website feature that allows documentation of management decisions in patient care. Actual practice was documented by reviewing website logs of decisions made by specialist HCPs.

    Close-ended questions were rated using a 5-point Likert scale (1=strongly disagree to 5=strongly agree). Open-ended questions explored opinions on the best features of the website, barriers to use, and suggestions for further improvement. Survey data were analyzed using descriptive statistics, and responses to open-ended questions were grouped into common categories.


    Development

    The stakeholder consultation generated ideas that informed website design and implementation. Key features and content of the website were identified successively as a result of the consultation process.

    Researchers and CTP insurance regulators determined that the website should assess the risk of nonrecovery and facilitate risk-based management. The clinical prediction rule (CPR) developed by researchers from the University of Queensland [18,19] was agreed to be used and automated on the website. Concurrently, consultation between researchers from the University of Sydney and CTP insurance regulators had developed a decision matrix for specialist HCPs to assist in the management of people with WAD at medium and high risk of nonrecovery. The decision matrix included 3 management decisions: (1) shared care (ie, continued treatment with the primary HCP monitored by a specialist), (2) specialist care (ie, direct treatment from specialist), and (3) referred care (ie, referral to other disciplines).

    Researchers and CTP insurance regulators further identified that the website should consolidate evidence-based and risk-based resources for people with WAD and HCPs. Before the development of the website, patient versions of the WAD guidelines, educational and exercise videos, factsheets, and booklets were previously developed by the CTP insurance regulators with extensive involvement of people with WAD. Resources for HCPs who manage people at low risk were based on the most recent WAD guidelines and were available in various formats (eg, booklets, factsheets, and videos) [21,22]. Resources for HCPs who manage people at medium and high risk were to facilitate referral to specialist HCPs and assist in further assessment of physical and psychological factors.

    The idea generation survey was returned by 94 HCPs (94/671, 14.0%) and 26 people with WAD (26/50, 52%). About half of the HCPs (53/94, 56%) and most of the people with WAD (25/26, 96%) surveyed had not seen the existing WAD resources. When these resources were shown to them, most HCPs (59/94, 63%) and people with WAD (21/26, 81%) agreed that these would help WAD management and applied to their injury (Table 1). In addition, the majority of the people with WAD who completed the survey agreed that they would appreciate access to the resources. About half of the HCPs (50/94, 53%) also indicated that a website would be the best platform to facilitate access to the resources.

    Results of the focus group discussions, attended by 28 HCPs (16 primary and 12 specialist HCPs), endorsed some key features of the website and informed the types of resources that were added to the website (Multimedia Appendix 2). Themes that were generated can be broadly categorized as those relating to risk assessment, management of people at low risk of nonrecovery, and management of people at medium and high risk of nonrecovery. For example, HCPs suggested that if risk assessment was to be automated on the website, then guidance on what to say to people at different risk levels should be provided. In the low risk section of the website, primary HCPs suggested downloadable and customizable exercise sheets be provided. In the high-risk section of the website, primary HCPs indicated that a database of specialist HCPs incorporated into the website would facilitate the referral process. Finally, specialist HCPs suggested accessible and downloadable psychological screening tools (eg, Depression, Anxiety and Stress Scale [47], and Pain Catastrophizing Scale [48]) and other outcome measures (eg, Self-report Leeds Assessment of Neuropathic Symptoms and Signs pain scale [49], and Central Sensitization Inventory [50]) as well as incorporating a mechanism to document management decisions and communicate with primary HCPs. These results were considered, and the website incorporated content that provided suggestions on communicating risk level, downloadable resources (eg, screening tools, exercise sheets, and outcome measures) and a database of specialist HCPs.

    The vendor consultation resulted in the delivery of an interactive, user-friendly website, the My Whiplash Navigator (Figure 2). Within the website, 3 sections were developed: patient, health care practitioner, and specialist practitioner. Each section was developed with the target market considered and the key components of WAD management embedded (ie, risk assessment and risk-based management; Multimedia Appendix 3). The patient section guides people with WAD through various steps within the clinical pathway, including risk assessment, and provides access to useful information, advice, and exercises. The health care practitioner section comprises guideline-based resources to assist in WAD management, particularly management of people at low risk and referral of those at medium and high risk of nonrecovery. The specialist practitioner section was designed to assist the specialist in making appropriate decisions for people who are at medium and high risk of nonrecovery.

    Table 1. Perceptions of health care professionals (N=94) and people with whiplash-associated disorder (N=26) about available resources on whiplash before the project.
    Target groupStrongly disagree/disagree, n (%)Neutral, n (%)Strongly agree/agree, n (%)
    Health care professionals

    		Factsheet


    		Helpful in WADa management12 (13)26 (28)56 (60)


    		Would use the resource in practice19 (20)20 (21)55 (59)

    		Educational video


    		Helpful in WAD management9 (10)23 (25)62 (66)


    		Would use the resource in practice22 (23)26 (28)46 (49)

    		Exercise video


    		Helpful in WAD management12 (13)22 (23)60 (64)


    		Would use the resource in practice24 (26)25 (27)45 (48)
    People with WAD

    		Factsheet


    		Applicable to my injury1 (4)3 (12)21 (84)


    		Would want access to the resource1 (4)1 (4)23 (92)

    		Educational video


    		Applicable to my injury0 (0)5 (19)20 (80)


    		Would want access to the resource1 (4)4 (15)20 (80)

    		Exercise video


    		Applicable to my injury0 (0)5 (19)20 (80)


    		Would want access to the resource2 (8)4 (15)19 (76)

    aWAD: whiplash-associated disorder.

    ‎
    Figure 2. The My Whiplash Navigator website landing page.
    View this figure
    Format, Layout, and Features

    The content of the website was written in a professional and patient-friendly tone, included interactive and intuitive pages, and presented a positive and encouraging color scheme and layout. Pages were interactive, intuitive, and user friendly to facilitate ease of navigation. Information contained within the website were presented as navigation links and in a drop-down/accordion format, where appropriate. The interactions and access to various pages within the website are summarized in Figure 3.

    The homepage of the website provided information about the My Whiplash Navigator and links to register and capture login details. Once registered or logged in, the patient or HCP is directed to their relevant page. The patient section includes an automated version of the whiplash CPR. The patient completes a questionnaire, and the score/risk status is computed. Feedback about the level of risk is provided to the patient and the treating HCP, with a link to information for matched treatments according to the risk of nonrecovery. Suggestions on how to communicate risk level to patients are provided to the treating HCP. A database of specialist HCPs is also incorporated on the website to facilitate referral of people with WAD at medium and high risk of nonrecovery. In addition, the website links the patient with the treating HCP, as well as the specialist HCP, allowing access to information and monitoring of recovery. Communication between the treating HCP and patient is facilitated through a comments section in the management page. Communication between the treating and specialist HCPs is further facilitated in the management page, where the specialist HCP logs the decisions made and advice for the treating HCP. The management section also allows the specialist HCP to upload relevant documents or reports that the treating HCP is able to view. Resources for patients and HCPs are downloadable, and a customizable exercise dosage chart is also available for HCPs to download.

    ‎
    Figure 3. The My Whiplash Navigator wireframes overview.
    View this figure
    Permissions

    The website was built on a Web platform called Drupal, which is a proven, secure application framework that has role-based permissions system. A tiered permission system was used for the My Whiplash Navigator. First-tier included administrators (eg, researchers and research assistants) who were able to access all information on the website. Other tiers include patients and HCPs. Patients were tagged as belonging to a particular HCP. Only those HCPs with a tagged relationship with a patient were able to review all the data on that particular patient. No facility exists for them to access other patient data, except through aggregated and anonymized data reports. No patient is able to access another patient’s data. The data are stored on a MySQL (Oracle Corporation) server cluster that is run by a secure Australian data center hosted in Sydney. The physical location for this server has 24/7 on-site security and is ISO27001:2005 certified.

    Implementation

    Implementation of the website took place primarily in NSW, QLD, and Australian Capital Territory from June 2016 to March 2018. A total of 260 HCP registrations were recorded (Table 2). The majority of the registrations were from NSW (214/260, 82.3%) followed by QLD (26/260, 10.0%).

    A total of 65 primary HCPs registered on the website, with most registrations recorded following educational meetings (47/110, 42.7%). All specialist HCPs engaged through educational workshops registered on the website (20/20, 100%). The majority of the student HCPs registered on the website (175/215, 81.4%) following classroom education.

    Table 2. Summary of registrations on the My Whiplash Navigator website.
    Target groupImplementation strategyTotal approached, NTotal registered on the website, n (%)
    Student HCPaClassroom education215175 (81.4)
    Primary HCPEducational meeting11047 (42.7)
    Educational outreach and reminders5118 (35)
    Specialist HCPEducational workshops2020 (100)

    aHCP: health care professional.

    Table 3. Summary of the number and duration of page views for the My Whiplash Navigator website.
    Page titleUnique page views, NAverage time (seconds)
    Assessment of risk

    		Health professional assessment of a whiplash injury280160.94

    		Prognosis: identifying people at risk of poor recovery200128.46
    Provision of appropriate management for people at low risk of nonrecovery

    		Whiplash information and exercises339145.48

    		Whiplash exercises192260.59

    		Helpful facts and advice102161.99
    Provision of appropriate management for people at medium and high risk of nonrecovery

    		Specialist assessment of whiplash94143.38

    		Specialist care for whiplash8466
    Other frequently visited pages

    		Whiplash Navigator steps37583.68

    		Managing whiplash in your patients217199.23

    		What is Whiplash Navigator?16991.7

    		FAQsa for patients88119.31

    aFAQs: frequently asked questions.

    The website pages most commonly viewed were related to risk assessment, exercises, and information about WAD (Table 3). There were 1229 times when users were actively engaged with the website. The majority of website engagement was from new visitors (858/1229, 69.81%), and 30.19% (371/1229) were returning visitors. The total page views of the website were 7508 views, of which 5027 were unique page views. Users view, on average, 6 pages during a visit and stay approximately 3 minutes on each page.

    Evaluation

    The website evaluation survey was administered after the implementation stage, between April and June 2018, and returned by 24 primary HCPs (24/65, 37%), 13 specialist HCPs (13/20, 65%), and 60 student HCPs (60/175, 34.3%).

    Most HCPs agreed that the website was accessible, easy to navigate, and was a useful resource to assist in WAD management (Table 4). Posthoc analyses showed that there was no difference among students and primary and specialist HCPs regarding the acceptability of the website (Table 5). Most HCPs agreed that they would use the website in their clinical practice (82/97, 85%). Furthermore, the majority of the primary and specialist HCPs would recommend the use of suitable Web-based tools to other practitioners (32/37, 87%). The best features identified were accessibility of the website, simple interface promoting ease of use and navigation, and capacity to link people with WAD with HCPs. Many perceived that the website was comprehensive and provided a one stop shop for HCPs.

    In contrast, features that needed improvement were initial navigation difficulties and complicated registration process. A few HCPs suggested potential limitation of the reach of the website for HCPs or people with WAD who do not have access to technology. Accordingly, HCPs identified a number of additional features and content to further improve the website. Some HCPs suggested adding more exercise and assessment videos that can be used to further assist patient care. Features such as a search bar, site navigator, and information for first time users would further enhance navigation. Finally, some HCPs wanted to be able to customize exercises and resubmit forms within the website to make monitoring progress and designing exercise programs easier.

    Table 4. Perceptions of health care professionals about acceptability and self-rated improvement in knowledge of whiplash-associated disorder management after using the My Whiplash Navigator website (N=97).
    OutcomeStrongly disagree/disagree, n (%)Neutral, n (%)Strongly agree/agree, n (%)
    Acceptability of the website

    		Easy to access screening tools5 (5)20 (21)72 (74)

    		Easy to access outcome measures3 (3)21 (22)73 (75)

    		Easy to navigate1 (1)28 (29)68 (70)

    		Easy to understand0 (0)13 (13)84 (87)

    		Useful resource for WADa0 (0)11 (11)86 (89)

    		Will use the website in clinical practice1 (1)14 (14)82 (85)
    Self-rated improvement in knowledge

    		Risk assessment0 (0)18 (19)79 (81)

    		Using the C-Spine ruleb1 (1)14 (17)69 (82)

    		Standard assessmentb1 (1)12 (14)71 (85)

    		Risk-based advice0 (0)18 (19)79 (81)

    		Appropriate exercises1 (1)11 (11)85 (88)

    		Referral of high-risk patientsb1 (1)13 (16)70 (83)

    aWAD: whiplash-associated disorder.

    bn=84.

    Table 5. Posthoc analyses of differences among student, primary, and specialist health care professionals regarding acceptability and self-rated improvements in knowledge of whiplash-associated disorder management after using the My Whiplash Navigator website.
    OutcomeDifference between groupsa, chi-square (df)Significance, P value
    Acceptability of the website

    		Easy to access screening tools4.2 (2).12

    		Easy to access outcome measures1.8 (2).41

    		Easy to navigate1.8 (2).41

    		Easy to understand0.6 (2).74

    		Useful resource for WADb0.6 (2).71

    		Will use the website in clinical practice1.0 (2).59
    Self-rated improvement in knowledge

    		Risk assessment0.7 (2).67

    		Risk-based advice0.0 (2).99

    		Appropriate exercises0.6 (2).74

    aTest statistic: Kruskal-Wallis test.

    bwhiplash-associated disorder.

    Most HCPs agreed that the website helped them gain knowledge about key aspects of WAD management (Table 4). The highest self-rated improvement was around provision of appropriate exercises (85/97, 88%), whereas the lowest was in the identification of risk of nonrecovery and provision of risk-based advice (79/97, 81%). Similarly, the majority of the specialist HCPs agreed that the website assisted in gaining knowledge about management decisions for people with WAD at medium and high risk of nonrecovery (11/13, 85%). Posthoc analyses demonstrated that there was no difference among student, primary HCPs, and specialist HCPs regarding self-rated improvements in knowledge (Table 5). The majority of the primary and specialist HCPs agreed that the website facilitated communication between the primary health care provider and specialist (25/37, 68%).

    In terms of actual practice, there were a total of 24 recorded management decisions logged by specialist HCPs on the website. The most commonly chosen pathway decision was shared care (15/24, 63%). Provision of specialist care was chosen by specialist HCPs in 33% of the decisions logged (8/24), mainly because the people with WAD that they saw did not have primary HCPs at the time of consultation.


    Principal Findings and Comparison With Previous Work

    Results of this study demonstrate that the My Whiplash Navigator is innovative, practical to implement, acceptable, and assisted in improving knowledge and practice of HCPs in WAD management. The rigorous process of development, implementation, and evaluation involving extensive consultations among stakeholders has led to the first Web-based tool for WAD that puts together comprehensive, evidence-based resources, automated risk assessment, and risk-based management. Stakeholder consultation from the outset is key in research translation [31,51], and incorporating the suggestions of each target market during idea generation and feasibility supported uptake and acceptability of the website. These results demonstrate that the My Whiplash Navigator may be scaled for wider implementation.

    Most of the development and implementation strategies used for this website have succeeded in engaging the target markets and can be used in a future wider implementation program. Known implementation strategies such as classroom education [52,53] and educational meetings [14,16,32] used to improve practice in other conditions were also effective in engaging HCPs to use the website. Incorporating the website as part of assessable educational content further enhanced implementation, given higher uptake among student HCPs compared with primary HCPs. Second, an effective and efficient way to facilitate website use among people with WAD might be through primary HCPs and insurers. Compared with the fast-paced, high-stress environment in hospital emergency departments [54], the typical clinical encounter and the therapeutic relationship between people with WAD and their primary HCPs (Griffin et al, under review) would more likely assist in engaging people with WAD to use the website. Primary HCPs are also the first point of contact of people with WAD, with general practitioners consulted as early as 4 days and physiotherapists at 3 weeks after injury [8]. Finally, insurance regulators are equally in a position to engage people with WAD because most people who get injured from motor vehicle crashes in NSW and QLD submit claims after the accident to access benefits under the CTP insurance scheme.

    Results also showed that implementation of the website improved HCP knowledge and practice across the health service delivery spectrum (ie, from novice to experienced HCPs) that could potentially improve outcomes for people with WAD. Preliminary efficacy demonstrated that the website improved self-rated knowledge of HCPs about key aspects of WAD management, specifically, self-rated knowledge of risk assessment and provision of risk-based advice. Although previous studies have shown that it has been difficult to improve knowledge on risk assessment among primary HCPs [14,16,32], use of the website appeared to support the delivery and implementation of these key messages. Similarly, use of the website appeared to improve actual practice of specialist HCPs through the promotion of a shared care approach in managing people with WAD at medium and high risk of nonrecovery. This is in contrast to the results of a previous study where the same group of specialist HCPs preferred to provide direct treatment (60%) over shared care (22%) [43]. A shared care approach in management was perceived by primary HCPs to facilitate the referral process and recovery for people with WAD [43]. The strategies used to engage specialist HCPs along with the interactivity of the website may have contributed to the improvements in practice observed.

    Next Steps

    The positive results related to the feasibility and preliminary efficacy of the website demonstrated the potential for broader uptake and implementation. Results further identified website content and features that need to be improved before more widespread implementation. The next process in translation would be that of replicability and scalability [31]. Replicability involves testing whether the website could deliver the same outcomes in other circumstances. Ideally, the aim is to implement the website in other states in Australia to test its application for WAD in different motor accident CTP insurance schemes. In addition, implementation of the website could involve other key target markets such as people with WAD, the insurance industry, and HCPs from other disciplines involved in WAD management (eg, chiropractors, osteopaths, and general practitioners). Finally, the suggestions from HCPs to incorporate additional features such as search or navigator bar and access to outcome measures would be considered to facilitate easier navigation and better clinical decision making.

    Limitations

    The response rate for the idea generation survey among HCPs was low; however, the survey was meant to be exploratory to supplement the other processes used during the development stage of the website. Despite a low response rate, the responses received from HCPs captured divergent opinions allowing us to understand the needs and expectations related to resources to assist HCPs in WAD management. People with WAD were not included in the implementation and evaluation stage, mostly because access to the website was limited to patients enrolled in a concurrent randomized controlled trial (RCT) [20] that the website supports. However, before this work, people with WAD were extensively involved in the development of the patient version of the WAD guidelines, education and exercise videos, factsheets, and booklets for people with WAD. These resources formed the patient section of the website. Implementation and evaluation of the website among people with WAD will be conducted and reported on after completion of the RCT.

    Conclusions

    In summary, a robust process resulted in an innovative, interactive, user-friendly website, the My Whiplash Navigator. Implementation with HCPs was best achieved through classroom education and educational meetings. Evaluation suggested that the website was acceptable and improved knowledge and practice of HCPs in WAD management. These positive results provide sufficient evidence to scale implementation nationally and involve other target markets such as people with WAD, insurers, and insurance regulators.

    Acknowledgments

    The research team would like to acknowledge the State Insurance Regulatory Authority of New South Wales, Australia, and the Motor Accident Insurance Commission of Queensland, Australia, for funding this project. The funding bodies were not involved in any aspect of the project (data collection, analysis, interpretation of results, and review of the paper). The researchers are also grateful to the University of Sydney Musculoskeletal Laboratory, Chiropractors Association of Australia, Australian Physiotherapy Association, and the Central and Eastern Sydney Primary Health Network for promoting the study. ANB was supported by scholarships from the University of the Philippines Expanded Modernization Program—Doctoral Studies Fund and the University of Sydney Non-established Postgraduate Award. TR was supported by an early career NHRMC Research Fellowship and a University of Sydney SOAR Fellowship.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Focus group discussion questions for health care professionals during the idea generation process of website development.

    PDF File (Adobe PDF File), 63KB
    ‎
    Multimedia Appendix 2

    Summary of themes, codes, and illustrative quotes from the focus group discussions.

    PDF File (Adobe PDF File), 63KB
    ‎
    Multimedia Appendix 3

    Design features, content, and functionality of key pages of the My Whiplash Navigator website.

    PDF File (Adobe PDF File), 489KB
    1. Galasko C, Murray P, Stephenson W. Incidence of whiplash associated disorder. B C Med J 2002;44:237-240 [FREE Full text]
    2. Connelly LB, Supangan R. The economic costs of road traffic crashes: Australia, states and territories. Accid Anal Prev 2006 Nov;38(6):1087-1093. [CrossRef] [Medline]
    3. Holm LW, Carroll LJ, Cassidy JD, Hogg-Johnson S, Côté P, Guzman J, Bone and Joint Decade 2000-2010 Task Force on Neck Pain and Its Associated Disorders. The burden and determinants of neck pain in whiplash-associated disorders after traffic collisions: results of the bone and joint decade 2000-2010 task force on neck pain and its associated disorders. Spine (Phila Pa 1976) 2008 Feb 15;33(4 Suppl):S52-S59. [CrossRef] [Medline]
    4. Carroll LJ, Holm LW, Hogg-Johnson S, Côté P, Cassidy JD, Haldeman S, Bone and Joint Decade 2000-2010 Task Force on Neck Pain and Its Associated Disorders. Course and prognostic factors for neck pain in whiplash-associated disorders (WAD): results of the bone and joint decade 2000-2010 task force on neck pain and its associated disorders. Spine (Phila Pa 1976) 2008 Feb 15;33(4 Suppl):S83-S92. [CrossRef] [Medline]
    5. Casey PP, Feyer AM, Cameron ID. Course of recovery for whiplash associated disorders in a compensation setting. Injury 2015 Nov;46(11):2118-2129. [CrossRef] [Medline]
    6. Kamper SJ, Rebbeck TJ, Maher CG, McAuley JH, Sterling M. Course and prognostic factors of whiplash: a systematic review and meta-analysis. Pain 2008 Sep 15;138(3):617-629. [CrossRef] [Medline]
    7. Nikles J, Yelland M, Bayram C, Miller G, Sterling M. Management of whiplash associated disorders in Australian general practice. BMC Musculoskelet Disord 2017 Dec 29;18(1):551 [FREE Full text] [CrossRef] [Medline]
    8. Bandong AN, Leaver A, Mackey M, Ingram R, Shearman S, Chan C, et al. Adoption and use of guidelines for whiplash: an audit of insurer and health professional practice in New South Wales, Australia. BMC Health Serv Res 2018 Dec 8;18(1):622 [FREE Full text] [CrossRef] [Medline]
    9. Lurie JD, Birkmeyer NJ, Weinstein JN. Rates of advanced spinal imaging and spine surgery. Spine (Phila Pa 1976) 2003 Mar 15;28(6):616-620. [CrossRef] [Medline]
    10. Webster BS, Cifuentes M. Relationship of early magnetic resonance imaging for work-related acute low back pain with disability and medical utilization outcomes. J Occup Environ Med 2010 Sep;52(9):900-907. [CrossRef] [Medline]
    11. Flynn TW, Smith B, Chou R. Appropriate use of diagnostic imaging in low back pain: a reminder that unnecessary imaging may do as much harm as good. J Orthop Sports Phys Ther 2011 Nov;41(11):838-846. [CrossRef] [Medline]
    12. Smith-Bindman R, Miglioretti DL, Johnson E, Lee C, Feigelson HS, Flynn M, et al. Use of diagnostic imaging studies and associated radiation exposure for patients enrolled in large integrated health care systems, 1996-2010. J Am Med Assoc 2012 Jun 13;307(22):2400-2409 [FREE Full text] [CrossRef] [Medline]
    13. Carroll LJ, Cassidy JD, Côté P. The role of pain coping strategies in prognosis after whiplash injury: passive coping predicts slowed recovery. Pain 2006 Sep;124(1-2):18-26. [CrossRef] [Medline]
    14. Rebbeck T, Macedo LG, Maher CG. Compliance with clinical guidelines for whiplash improved with a targeted implementation strategy: a prospective cohort study. BMC Health Serv Res 2013 Jun 13;13:213 [FREE Full text] [CrossRef] [Medline]
    15. Rebbeck TJ, Refshauge KM, Maher CG. Use of clinical guidelines for whiplash by insurers. Aust Health Rev 2006 Nov;30(4):442-449. [CrossRef] [Medline]
    16. Rebbeck T, Macedo L, Paul P, Trevena L, Cameron ID. General practitioners' knowledge of whiplash guidelines improved with online education. Aust Health Rev 2013 Nov;37(5):688-694. [CrossRef] [Medline]
    17. Rotter T, Kinsman L, James E, Machotta A, Gothe H, Willis J, et al. Clinical pathways: effects on professional practice, patient outcomes, length of stay and hospital costs. Cochrane Database Syst Rev 2010 Mar 17(3):CD006632. [CrossRef] [Medline]
    18. Ritchie C, Hendrikz J, Jull G, Elliott J, Sterling M. External validation of a clinical prediction rule to predict full recovery and ongoing moderate/severe disability following acute whiplash injury. J Orthop Sports Phys Ther 2015 Apr;45(4):242-250. [CrossRef] [Medline]
    19. Ritchie C, Hendrikz J, Kenardy J, Sterling M. Derivation of a clinical prediction rule to identify both chronic moderate/severe disability and full recovery following whiplash injury. Pain 2013 Oct;154(10):2198-2206. [CrossRef] [Medline]
    20. Rebbeck T, Leaver A, Bandong AN, Kenardy J, Refshauge K, Connelly L, et al. Implementation of a guideline-based clinical pathway of care to improve health outcomes following whiplash injury (Whiplash ImPaCT): protocol of a randomised, controlled trial. J Physiother 2016 Apr;62(2):111 [FREE Full text] [CrossRef] [Medline]
    21. SIRA: State Insurance Regulatory Authority. 2014. Guidelines for the Management of Acute Whiplash Associated Disorders For Health Professionals   URL: https:/​/www.​sira.nsw.gov.au/​resources-library/​motor-accident-resources/​publications/​for-professionals/​whiplash-resources/​SIRA08104-Whiplash-Guidelines-1117-396479.​pdf [WebCite Cache]
    22. MAIC: Motor Accident Insurance Commission. 2015. Whiplash Injury Recovery: A Self Help Guide   URL: https://maic.qld.gov.au/wp-content/uploads/2016/02/Whiplash-Injury-Recovery-booklet-2015.pdf [WebCite Cache]
    23. Hill JC, Whitehurst DG, Lewis M, Bryan S, Dunn KM, Foster NE, et al. Comparison of stratified primary care management for low back pain with current best practice (STarT Back): a randomised controlled trial. Lancet 2011 Oct 29;378(9802):1560-1571 [FREE Full text] [CrossRef] [Medline]
    24. Lenz R, Blaser R, Beyer M, Heger O, Biber C, Bäumlein M, et al. IT support for clinical pathways--lessons learned. Int J Med Inform 2007 Dec;76(Suppl 3):S397-S402. [CrossRef] [Medline]
    25. Sucher JF, Moore FA, Todd SR, Sailors RM, McKinley BA. Computerized clinical decision support: a technology to implement and validate evidence based guidelines. J Trauma 2008 Feb;64(2):520-537. [CrossRef] [Medline]
    26. Blaser R, Schnabel M, Biber C, Bäumlein M, Heger O, Beyer M, et al. Improving pathway compliance and clinician performance by using information technology. Int J Med Inform 2007;76(2-3):151-156. [CrossRef] [Medline]
    27. Gross DP, Armijo-Olivo S, Shaw WS, Williams-Whitt K, Shaw NT, Hartvigsen J, et al. Clinical decision support tools for selecting interventions for patients with disabling musculoskeletal disorders: a scoping review. J Occup Rehabil 2016 Dec;26(3):286-318 [FREE Full text] [CrossRef] [Medline]
    28. Jaspers MW, Smeulers M, Vermeulen H, Peute LW. Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings. J Am Med Inform Assoc 2011 May 1;18(3):327-334 [FREE Full text] [CrossRef] [Medline]
    29. Umapathy H, Bennell K, Dickson C, Dobson F, Fransen M, Jones G, et al. The web-based osteoarthritis management resource My Joint Pain improves quality of care: a quasi-experimental study. J Med Internet Res 2015 Jul 7;17(7):e167 [FREE Full text] [CrossRef] [Medline]
    30. Milat AJ, Li B. Narrative review of frameworks for translating research evidence into policy and practice. Public Health Res Pract 2017 Feb 15;27(1):pii: 2711704 [FREE Full text] [CrossRef] [Medline]
    31. Sax Institute. 2016. Translational Research Framework: Testing Innovation in Policy, Programs and Service Delivery   URL: https://www.saxinstitute.org.au/wp-content/uploads/Translational-Research-Framework.pdf [accessed 2018-09-16] [WebCite Cache]
    32. Rebbeck T, Maher CG, Refshauge KM. Evaluating two implementation strategies for whiplash guidelines in physiotherapy: a cluster randomised trial. Aust J Physiother 2006;52(3):165-174 [FREE Full text] [CrossRef] [Medline]
    33. Cheung A, Weir M, Mayhew A, Kozloff N, Brown K, Grimshaw J. Overview of systematic reviews of the effectiveness of reminders in improving healthcare professional behavior. Syst Rev 2012 Aug 16;1:36 [FREE Full text] [CrossRef] [Medline]
    34. Flodgren G, Parmelli E, Doumit G, Gattellari M, O'Brien MA, Grimshaw J, et al. Local opinion leaders: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2011 Aug 10(8):CD000125 [FREE Full text] [CrossRef] [Medline]
    35. Forsetlund L, Bjørndal A, Rashidian A, Jamtvedt G, O'Brien MA, Wolf F, et al. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2009 Apr 15(2):CD003030. [CrossRef] [Medline]
    36. Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. Br Med J 2015 Mar 19;350:h1258 [FREE Full text] [CrossRef] [Medline]
    37. Hulscher ME, Laurant MG, Grol RP. Process evaluation on quality improvement interventions. Qual Saf Health Care 2003 Feb;12(1):40-46 [FREE Full text] [CrossRef] [Medline]
    38. Srikesavan CS, Williamson E, Eldridge L, Heine P, Adams J, Cranston T, et al. A web-based training resource for therapists to deliver an evidence-based exercise program for rheumatoid arthritis of the hand (iSARAH): design, development, and usability testing. J Med Internet Res 2017 Dec 13;19(12):e411 [FREE Full text] [CrossRef] [Medline]
    39. Park MH, Skow A, Puradiredja DI, Lucas A, Syrad H, Sovio U, et al. Development and evaluation of an online tool for management of overweight children in primary care: a pilot study. BMJ Open 2015 Jun 12;5(6):e007326 [FREE Full text] [CrossRef] [Medline]
    40. Pauwels E, van Hoof E, Charlier C, Lechner L, de Bourdeaudhuij I. Design and process evaluation of an informative website tailored to breast cancer survivors' and intimate partners' post-treatment care needs. BMC Res Notes 2012 Oct 3;5:548 [FREE Full text] [CrossRef] [Medline]
    41. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009 Apr;42(2):377-381 [FREE Full text] [CrossRef] [Medline]
    42. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006 Jan;3(2):77-101. [CrossRef]
    43. Bandong AN, Leaver A, Mackey M, Sterling M, Kelly J, Ritchie C, et al. Referral to specialist physiotherapists in the management of whiplash associated disorders: perspectives of healthcare practitioners. Musculoskelet Sci Pract 2018 Dec;34:14-26. [CrossRef] [Medline]
    44. Kelly J, Sterling M, Rebbeck T, Bandong AN, Leaver A, Mackey M, et al. Health practitioners' perceptions of adopting clinical prediction rules in the management of musculoskeletal pain: a qualitative study in Australia. BMJ Open 2017 Aug 11;7(8):e015916 [FREE Full text] [CrossRef] [Medline]
    45. Lewis JR. Sample sizes for usability studies: additional considerations. Hum Factors 1994 Jun;36(2):368-378. [CrossRef] [Medline]
    46. Virzi RA. Refining the test phase of usability evaluation: how many subjects is enough? Hum Factors 2016 Nov 23;34(4):457-468. [CrossRef]
    47. Lovibond SH, Lovibond PF, editors. Manual for the Depression Anxiety Stress Scales. Second Edition. New South Wales, Sydney: Psychology Foundation of Australia; 1995.
    48. Sullivan MJ, Bishop SR, Pivik J. The pain catastrophizing scale: development and validation. Psychol Assess 1995;7(4):524-532. [CrossRef]
    49. Bennett MI, Smith BH, Torrance N, Potter J. The S-LANSS score for identifying pain of predominantly neuropathic origin: validation for use in clinical and postal research. J Pain 2005 Mar;6(3):149-158. [CrossRef] [Medline]
    50. Mayer TG, Neblett R, Cohen H, Howard KJ, Choi YH, Williams MJ, et al. The development and psychometric validation of the central sensitization inventory. Pain Pract 2012 Apr;12(4):276-285 [FREE Full text] [CrossRef] [Medline]
    51. Murtagh MJ, Minion JT, Turner A, Wilson RC, Blell M, Ochieng C, et al. The ECOUTER methodology for stakeholder engagement in translational research. BMC Med Ethics 2017 Dec 4;18(1):24 [FREE Full text] [CrossRef] [Medline]
    52. Overmeer T, Boersma K, Main CJ, Linton SJ. Do physical therapists change their beliefs, attitudes, knowledge, skills and behaviour after a biopsychosocially orientated university course? J Eval Clin Pract 2009 Aug;15(4):724-732. [CrossRef] [Medline]
    53. Domenech J, Sánchez-Zuriaga D, Segura-Ortí E, Espejo-Tort B, Lisón JF. Impact of biomedical and biopsychosocial training sessions on the attitudes, beliefs, and recommendations of health care providers about low back pain: a randomised clinical trial. Pain 2011 Nov;152(11):2557-2563. [CrossRef] [Medline]
    54. Cameron KA, Engel KG, McCarthy DM, Buckley BA, Mercer KL, Donlan SM, et al. Examining emergency department communication through a staff-based participatory research method: identifying barriers and solutions to meaningful change. Ann Emerg Med 2010 Dec;56(6):614-622. [CrossRef] [Medline]

    ‎
    CPR: clinical prediction rule
    CTP: compulsory third party
    HCP: health care professional
    NSW: New South Wales
    QLD: Queensland
    RCT: randomized controlled trial
    WAD: whiplash-associated disorder

    Edited by G Eysenbach; submitted 16.09.18; peer-reviewed by P Williams, S Sarbadhikari, G Kolostoumpis; comments to author 07.02.19; revised version received 02.05.19; accepted 20.05.19; published 26.08.19

    Copyright

    ©Aila Nica Bandong, Martin Mackey, Andrew Leaver, Rodney Ingram, Michele Sterling, Carrie Ritchie, Joan Kelly, Trudy Rebbeck. Originally published in JMIR Formative Research (http://formative.jmir.org), 26.08.2019.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on http://formative.jmir.org, as well as this copyright and license information must be included.