Background: Implementation of clinical guidelines into routine practice remains highly variable. Strategies to increase guideline uptake include developing digital tools and mobile apps for use in clinical practice. The National Heart Foundation of Australia in collaboration with the Cardiac Society of Australia and New Zealand published 3 key cardiac clinical guidelines, including the Australian clinical guidelines for the (1) prevention and detection of atrial fibrillation, (2) detection and management of heart failure, and (3) management of acute coronary syndromes. To improve access and uptake for health care providers, we developed the Smart Heart Guideline App.
Objective: This study aims to evaluate the acceptability, implementation, and usability of an Australian-specific cardiac guidelines mobile app.
Methods: We used an iterative multiple methods development and implementation approach. First, we conducted a cross-sectional web-based survey with end users (n=504 health professionals) in 2017 to determine the acceptability of an Australian-specific cardiac clinical guidelines mobile app. Second, the Smart Heart Guidelines app was created using a design, user testing, and revision process. The app includes interactive algorithms and flowcharts to inform diagnosis and management at the point of care. The freely available app was launched in October 2019 on iOS and Android operating systems and promoted and implemented using multiple methods. Third, data from 2 annual national cross-sectional general practitioner (GP) surveys in 2019 and 2020 were evaluated to understand the awareness and use of the clinical guidelines and the app. Fourth, data from the app stores were analyzed between October 1, 2019, and June 30, 2021, to evaluate usage.
Results: Most health professionals surveyed (447/504, 89%) reported accessing resources electronically, and most (318/504, 63%) reported that they would use an Australian-specific cardiac guidelines app. GPs surveyed in 2019 were aware of the heart failure (159/312, 51%) and atrial fibrillation (140/312, 45%) guidelines, and in 2020, a total of 34 of 189 (18%) reported that they were aware of the app. The app was downloaded 11,313 times (7483, 66% from the Apple App Store; 3830, 34% from Google Play) during the first 20-month period. Most downloads (6300/7483, 84%) were a result of searching for the app in the stores. Monthly download rates varied. App Store data showed that people used the app twice (on average 2.06 times) during the 20 months. Many (3256/3830, 85%) Android users deleted the app.
Conclusions: Health professionals supported the development of the Smart Heart Guidelines app. Although initial downloads were promising, the frequency of using the app was low and deletion rates were high. Further evaluation of users’ experience of the most and least useful components of the app is needed.
Cardiovascular disease is the leading cause of death in Australia and a nationwide priority for research . National clinical guidelines have been developed to inform decision-making among health professionals in the diagnosis and management of cardiovascular disease. Implementation of clinical guidelines can prevent avoidable harm, improve resource use, and reduce variation in care [ ]. Despite this, implementation of clinical guidelines into routine practice remains highly variable, suboptimal, and concerningly low in some areas of health care [ ]. In Australia, the National Health and Medical Research Council advises improving access and uptake of clinical guidelines by presenting content in multiple formats that are tailored to users’ needs [ ].
Smartphone apps are a convenient way for health professionals to access health information in a timely manner. In recent years, international organizations including the European Society of Cardiology and the American Heart Association have developed mobile apps providing digital access to their guidelines and specific apps for interactive decision support tools for real-time use in clinical practice and cardiovascular risk calculation [, ]. Similarly, other health disciplines including anesthesiology, pediatrics, and dermatology have developed apps to improve access and adherence to guideline recommendations in clinical practice [ - ]. With the advent of “living guidelines” and its methods of continuous evidence surveillance and recommendation updates, digital authoring, and web-based publication platforms such as Making Grade the Irresistible Choice (MAGICapp) have been developed, publishing over 190 living guidelines, including the Australian guidelines for care for people with COVID-19 [ ]. These developments compliment a trend toward greater utilization of digital platforms and smart phone use by doctors in Australia [ , ].
The National Heart Foundation of Australia (Heart Foundation) is an independent, not-for-profit organization that funds cardiovascular research and, in partnership with the Cardiac Society of Australia and New Zealand and other organizations, publishes clinical practice guidelines and position statements in areas where guidance will have the biggest impact.
The study aim was to evaluate the acceptability, implementation, and usage of an Australian-specific mobile app to improve the awareness and use of 3 Australian cardiac clinical guidelines for the (1) prevention and detection of atrial fibrillation 2018, (2) detection and management of heart failure 2018, and (3) Australian clinical guidelines for the management of acute coronary syndromes 2016 [- ].
We used an iterative multiple methods approach, drawing on established methods in mobile app development, implementation, and evaluation . First, we conducted a cross-sectional web-based survey of end-users to determine the acceptability of an Australian-specific mobile app to access cardiac clinical guidelines. Second, the Smart Heart Guidelines app was developed and launched on Google Play and Apple App Store (App Store) and promoted using multiple methods. Third, the app was promoted and implemented. Fourth, data from 2 annual cross-sectional national general practitioner (GP) surveys in 2019 and 2020 were evaluated to assess awareness and use of the clinical guidelines and the app. Fifth, data from the app stores were collated to evaluate app use. Each of these steps are now described.
Health Professional Acceptability Survey
A web-based survey designed for health professionals likely to use a cardiology guidelines app was developed and distributed in 2017. The survey contained 10 questions including the frequency of searching for information about the prevention or management of cardiovascular disease, methods of searching, and the likelihood of using an Australian-specific mobile app to access cardiac clinical guidelines. The survey was distributed through the Heart Foundation’s social media accounts and promotion in the organization’s monthly newsletter (the mailing list comprised of approximately 15000 health professionals: average open rates of 21%, 3150/15000). Survey responses informed the development of the app.
Development of the Smart Heart Guidelines App
The Heart Foundation collaborated with a global app developer, experienced with international cardiology apps in Europe and the United States. The app was created in iOS (Apple) and Android formats.
Three clinical guidelines were converted from publication in a PDF format in a peer-reviewed journal into the app. The app included the guidelines for acute coronary syndromes, atrial fibrillation, and heart failure [- ]. The app was organized into 3 sections for each clinical guidelines and contained a main page, table of contents, interactive tools and algorithms (to support clinical decision-making), and a table of key recommendations, as shown in . The app was created using a design, user testing, and revision process.
The app was registered as a class 1 software–based medical device with the Australian Register of Therapeutic Goods in 2019, in accordance with national legislation . Registration requirements include ongoing monitoring of safety, quality, and performance. Users had to declare that they were a health professional to download the app. The freely available (no cost) Smart Heart Guidelines app was launched on both the App Store and Google Play in October 2019.
Promoting and Implementing the App
Implementation strategies used to promote the app included printed and electronic flyers, containing a quick response code distributed at health professional educational meetings and events. Direct email and newsletters to multiple health professional groups and paid advertising in health professional journals was used. Promotion of the app on multiple webpages associated with the clinical guidelines was also undertaken.
Annual Cross-sectional Awareness Surveys to General Practitioners in 2019 and 2020
The Heart Foundation develops and distributes a nationwide cross-sectional survey to GPs annually since 2010. These surveys collect feedback about views, attitudes, awareness and the use of Heart Foundation resources and clinical guidelines. The 2019 and 2020 surveys were distributed to a sample of approximately 4000 GPs identified from the Medical Directory of Australia . The 2019 annual survey contained 24 items and was distributed and open between October and November 2019. GPs could respond on the web, using the software platform Typeform or via a paper version [ ]. In 2020, the 21-item survey was only available on the web and was open from October 2020 to November 2020 [ ]. Both surveys contained a combination of closed-ended questions and Likert rating scales.
Evaluation of App Usage From the App Store and Google Play
Usage data for the app was retrieved from the App Store and Google Play during the period between October 1, 2019, and June 30, 2021. Data from both app stores showed the total number of downloads of the app and the conversion rate (number of downloads divided by the number of impressions) during this period. Data from Google Play showed the retention rate (percentage of users who had not uninstalled the app from their device). The App Store report provided the number of impressions (the number of times an app appears in an App Store search), product page views (the number of times a user viewed the apps product page in the App Store), and average use (number of sessions per device divided by the total number of users).
Heart Foundation surveys are approved through routine governance organizational processes. Data collected from the surveys were anonymized and informed consent was assumed at the time of survey participation. Therefore, there was no ethical application made for this multi-methods study.
Data Handling and Statistical Analyses
Descriptive statistics were used to summarize the health professional acceptability survey from 2017 and the GP surveys in 2019 and 2020. Google Analytics was used to evaluate app data from the App Store and Google Play.
Health Professional Acceptability Survey in 2017
There were 504 respondents, from all 6 states and 2 territories in Australia. Most were nurses (198/504, 39%), allied health professionals (132/504, 26%), GPs and cardiologists (45/504, 9%), researchers (11/504, 2%), or identified as other (118/504, 23%). Respondents were from a variety of disciplines including public health (131/504, 26%), private practice (121/504, 24%), community health (86/504, 17%), research institute (55/504, 11%), private hospital (35/504, 7%), health promotion (25/504, 5%), and others (50/504, 10%). Most (447/504, 89%) reported accessing the Heart Foundation’s resources electronically. Most health professionals reported using Microsoft software (246/504, 63%) Apple iOS (162/504, 32%), or Android devices (63/504, 13%). The majority of health professionals (369/504, 73%) reported they would use a mobile device (a tablet or mobile phone) to access health professional resources, although the majority (465/504, 92%) currently did not. The majority (314/504, 62%) reported that they would be likely or very likely to use an Australian-specific mobile app to access the clinical guidelines and resources.
Promotion and Implementation the App
Implementation and promotional strategies for the app are summarized in. Promotion strategies had a potential reach to over 73,000 individuals.
|Direct email to NHFAa staff||November 22, 2019||90 staff members|
|NHFA all staff newsletter||November 27, 2019||272 staff members|
|NHFA webpage on the Smart Heart Guidelines app||Live from November 11, 2019||6721 page views (November 11, 2019 to June 30, 2021)|
|NHFA newsletter to health care professionals||Monthly promotion between November 2019 and September 2020||Approximately 20,000 heath care professionals subscribed|
|Direct email to health organizations (eg, Stroke Foundation)||November 11, 2019||69 unique IP addresses |
|NHFA advisory committees and guideline writing groups||November 22, 2019 ||40 members|
|Promoted in the Australian Primary Health Care Nurses Association newsletter||November 2019||Approximately 3500 members|
|Direct email to the CSANZb||December 2019||Approximately 3000 members|
|Print the advertisement in the RACGPc newsletter||December 1, 2019||Approximately 40,000 general practitioners|
|Society page in Heart, Lung, Circulation||February 2020||>2000 cardiologist and cardiac surgeons in Australia|
aNFHA: National Heart Foundation of Australia.
bCSANZ: Cardiac Society of Australia and New Zealand.
cRACGP: Royal Australian College of General Practitioners.
Annual Cross-sectional Awareness Surveys to General Practitioners in 2019 and 2020
From the 4000 GPs on the distribution list, 312 GPs responded in 2019, and 189 GPs responded in 2020. Respondents’ awareness of specific cardiac clinical guidelines and reported the use of guidelines and the Smart Heart Guideline app are presented in.
|Surveys||Heart failurea, n (%)||Atrial fibrillationb, n (%)||Appc, n (%)|
|GP survey 2019|
|Aware of the resource||159 (51)||140 (45)||Not asked|
|Frequently uses the resource||9 (7)||19 (6)||Not asked|
|GP survey 2020|
|Aware of the resource||113 (60)||102 (54)||34 (18)|
|Frequently uses the resource||11 (6)||11(6)||10 (5)|
aGuidelines for the prevention, detection, and management of heart failure in Australia 2018 .
bGuidelines for the prevention and management of atrial fibrillation in Australia 2018 .
cSmart Heart Guideline app.
Data From the App Store and Google Play
During the period from October 1, 2019, to June 30, 2021, the Smart Heart Guideline app had 11,313 downloads (n=7483, 66% from the App Store; n=3830, 34% from Google Play). Data from the App Store indicated over 59,900 impressions (the number of times the Smart Heart Guideline App appeared in an App Store search) with 9000 product page views (number of times a user viewed the apps product page in the App Store), resulting in 7483 downloads. Of these downloads, the majority (6300/7483, 84%) were a direct result of searching for the Smart Heart Guideline app in the App Store. The remaining downloads resulted from accessing the app’s product page via the organization’s webpage (610/7483, 8%) from browsing the App Store (n=346, 5%) or from an app referrer (a link within another app; n=214, 3%). The app had consistent conversion rates (number of downloads divided by the number of impressions) throughout this period in both the App Store and Google Play. Data from the App Store showed that most downloads were on a mobile device (6951/7483, 93%) with fewer downloads on a tablet (532/7483, 7%). The App Store data showed low average use of the app, with an average of 2.06 sessions per user during the 20-month period. Additionally, Google Play data showed that from the 3830 downloads, most (3260/3830, 85%) resulted in user losses (app deletion by users). Download data from the App stores are presented in.
Following a survey among end users, which revealed that approximately two-thirds (318/504, 63%) of health professional reported being “likely” or “very likely” to use an Australian-specific mobile app to access cardiology guidelines, the Smart Heart Guidelines app was launched in October 2019. Downloads of the app during the first 20 months from users deliberately searching for the Smart Heart Guideline app in the App Store and Google Play indicate that promotional and implementation activities were useful in raising awareness of the app. The total number of app downloads (>11,000), primarily in smartphones in the first 20 months from app launch, is encouraging. However, usage data show people tended to use the app only a couple of times and many deleted the app, indicating it may not be meeting users’ needs. Responses from the 2019 and 2020 GP surveys demonstrate a consistent lack of awareness and use of clinical guidelines, despite efforts to disseminate broadly and improve accessibility, including open access publishing. Although 18% (34/189) of GPs reported awareness of the Smart Heart Guidelines app in the 2020 GP survey, fewer (10/189, 5%) reported frequently using it.
Comparisons With Prior Work
Guideline implementation is notoriously challenging . Internationally, there have been mixed experiences with the development and use of clinician-facing guideline apps. The National Institute for Health and Care Excellence, United Kingdom, a major developer of clinical guidelines, launched a guideline app in 2012 only to retire it in 2018 because app use stagnated and more people were directly accessing their website for information [ ]. It is important to consider the purposes of a guideline app, and if it is created for voluntary download and passive access to guideline information, it may not be meaningful to clinicians and may have low impact on practice and behavioral change. Contrastingly, apps involving more interactive and specific decision support tools may be better in improving guideline adherence and changing patient outcomes. The European Society of Cardiology Atrial Fibrillation clinical guideline app has demonstrated the value of integrating novel digital technology into clinical practice, with potential for optimizing health care professionals’ adherence to recommendations of pharmacological and interventional therapy for patients with atrial fibrillation [ ]. Our App included a combination of guideline information and interactive decision support tools.
In a literature review of medical smartphone apps in clinical decision support, Watson et al  identified 48 trials and one Cochrane review finding that while diagnostic accuracy studies are plentiful, studies to determine whether guideline apps improve adherence to guidelines are only beginning to emerge, usually in the form of before-and-after studies, often in global health. For example, in India, in a study of over 6000 participants, a nurse‐facilitated smartphone‐enabled hypertension and diabetes mellitus intervention in primary care was associated with significant improvements in blood pressure and blood glucose control over 18 months [ ]. The nurse examined and entered patient parameters into a mobile phone–based clinical decision support system to generate a prescription, which was reviewed by a physician [ ]. In contrast, in a small randomized controlled trial of pediatric doctors in the United States in a hospital simulation study, a guideline app developed to support and drive cardiopulmonary resuscitation in real time improved guideline adherence compared with other tools including pediatric advanced life support pocket cards. The app was associated with a shorter time to first and subsequent defibrillation attempts and fewer medication errors.
A reported barrier to health professionals’ uptake of guideline apps is the lack of an international regulatory framework to ensure that the apps are evidence-based and held to a quality standard [, ]. In Australia, therapeutic goods including medical software must be entered in the Australian Register of Therapeutic Goods, and registration of the app as a class 1 medical device may have positively influenced the uptake of the app. Studies investigating if the investment in creating and maintaining a guideline app is superior to other guideline implementation strategies are lacking [ , ]. It was beyond the scope of this study to undertake a cost-benefit analysis of the app; however, reviews of cost-effectiveness of digital health interventions in the management of cardiovascular disease are generally promising [ ].
Strengths and Limitations
A strength of this study is the use of multiple methods used to identify the acceptability of an Australian-specific cardiology clinical guidelines app and comparing App Store and Google Play usage data and GP experience survey data over 2 years (2019 and 2020). The app included both guideline information along with clinical decision support tools and interactive algorithms. However, this study has limitations. First, although the acceptability survey had over 500 respondents, only a small proportion of them were GPs and cardiologists, the main intended users of the app, potentially limiting the representativeness of this sample. Second, response rates to the 2019 and 2020 surveys were low, limiting the generalizability of the results. Third, comparisons between the app’s use across the App Store and Google Play was restricted as data capture is not consistent across these commercial platforms. Fourth, we were unable to comment on health professional behavior change resulting from app use, as it was beyond the scope of this study, and alternate methods are required to understand how access to the app influenced clinical practice healthcare service delivery.
This formative research provides a summary of why and how an Australian-specific cardiac guideline app was developed, launched, promoted, and initially used. Future research could involve qualitative interviews with app users to explore which aspects of the app they find most and least useful and to discover any unmet needs of health professionals using the app. More tailored research into the use of clinical decision aids and support tools in the app would help better understand whether the app changes health professional behaviors and improves patient outcomes.
The development of the Smart Heart Guideline app, a cardiology-specific guideline app in Australia, was indicated from surveying health professionals. The app incorporated guideline information and clinical support decision aids. Although downloads of the app from among >11,000 users in the first 20 months was a promising finding, the frequency of using the app was low and deletion of the app was high. Further evaluation of the app is needed to understand the most and least useful aspects and to understand if using the app improves guideline adherence and impacts patient outcomes.
The National Heart Foundation of Australia provided financial support for the development of this project.
Conflicts of Interest
Smart Heart App user interface.PNG File , 374 KB
Number of Smart Heart guidelines app downloads by month from the Apple app store and Google Play store.PNG File , 26 KB
- Deaths in Australia. Australian Government. 2022 Jun 09. URL: https://www.aihw.gov.au/reports/life-expectancy-death/deaths-in-australia/contents/leading-causes-of-death [accessed 2021-09-28]
- Guidelines for guidelines. National Health and Medical Research Council. 2018. URL: https://nhmrc.gov.au/guidelinesforguidelines/plan/implementability [accessed 2021-09-28]
- Elshaug AG, Rosenthal MB, Lavis JN, Brownlee S, Schmidt H, Nagpal S, et al. Levers for addressing medical underuse and overuse: achieving high-value health care. Lancet 2017 Jul 08;390(10090):191-202. [CrossRef] [Medline]
- Kotecha D, Chua WWL, Fabritz L, Hendriks J, Casadei B, Schotten U, European Society of Cardiology (ESC) Atrial Fibrillation Guidelines Taskforce‚ the CATCH ME consortiumthe European Heart Rhythm Association (EHRA). European Society of Cardiology smartphone and tablet applications for patients with atrial fibrillation and their health care providers. Europace 2018 Feb 01;20(2):225-233 [FREE Full text] [CrossRef] [Medline]
- Siebert JN, Lacroix L, Cantais A, Manzano S, Ehrler F. The Impact of a Tablet App on Adherence to American Heart Association Guidelines During Simulated Pediatric Cardiopulmonary Resuscitation: Randomized Controlled Trial. J Med Internet Res 2020 May 27;22(5):e17792 [FREE Full text] [CrossRef] [Medline]
- McEvoy MD, Hand WR, Stiegler MP, DiLorenzo AN, Ehrenfeld JM, Moran KR, et al. A Smartphone-based Decision Support Tool Improves Test Performance Concerning Application of the Guidelines for Managing Regional Anesthesia in the Patient Receiving Antithrombotic or Thrombolytic Therapy. Anesthesiology 2016 Jan;124(1):186-198 [FREE Full text] [CrossRef] [Medline]
- Schoemans HM, Goris K, Van Durm R, Fieuws S, De Geest S, Pavletic SZ, EBMT Transplantation Complications Working party. The eGVHD App has the potential to improve the accuracy of graft-versus-host disease assessment: a multicenter randomized controlled trial. Haematologica 2018 Oct;103(10):1698-1707 [FREE Full text] [CrossRef] [Medline]
- Hejjaji V, Malik AO, Peri-Okonny PA, Thomas M, Tang Y, Wooldridge D, et al. Mobile App to Improve House Officers' Adherence to Advanced Cardiac Life Support Guidelines: Quality Improvement Study. JMIR Mhealth Uhealth 2020 May 19;8(5):e15762 [FREE Full text] [CrossRef] [Medline]
- Tendal B, Vogel JP, McDonald S, Norris S, Cumpston M, White H, National COVID-19 Clinical Evidence Taskforce. Weekly updates of national living evidence-based guidelines: methods for the Australian living guidelines for care of people with COVID-19. J Clin Epidemiol 2021 Mar;131:11-21 [FREE Full text] [CrossRef] [Medline]
- Nerminathan A, Harrison A, Phelps M, Alexander S, Scott KM. Doctors' use of mobile devices in the clinical setting: a mixed methods study. Intern Med J 2017 Mar;47(3):291-298. [CrossRef] [Medline]
- Akl EA, Meerpohl JJ, Elliott J, Kahale LA, Schünemann HJ, Living Systematic Review Network. Living systematic reviews: 4. Living guideline recommendations. J Clin Epidemiol 2017 Nov;91:47-53. [CrossRef] [Medline]
- Brieger D, Amerena J, Attia JR, Bajorek B, Chan KH, Connell C, et al. National Heart Foundation of Australia and Cardiac Society of Australia and New Zealand: Australian clinical guidelines for the diagnosis and management of atrial fibrillation 2018. Med J Aust 2018 Oct 15;209(8):356-362. [CrossRef] [Medline]
- NHFA CSANZ Heart Failure Guidelines Working Group, Atherton JJ, Sindone A, De Pasquale CG, Driscoll A, MacDonald PS, et al. National Heart Foundation of Australia and Cardiac Society of Australia and New Zealand: Guidelines for the Prevention, Detection, and Management of Heart Failure in Australia 2018. Heart Lung Circ 2018 Oct;27(10):1123-1208 [FREE Full text] [CrossRef] [Medline]
- Chew DP, Scott IA, Cullen L, French JK, Briffa TG, Tideman PA, NHFA/CSANZ ACS Guideline 2016 Executive Working Group:. Heart Lung Circ 2016 Sep;25(9):895-951 [FREE Full text] [CrossRef] [Medline]
- Llorens-Vernet P, Miró J. Standards for Mobile Health-Related Apps: Systematic Review and Development of a Guide. JMIR Mhealth Uhealth 2020 Mar 03;8(3):e13057 [FREE Full text] [CrossRef] [Medline]
- Theraputic Goods Administration. Department of Health. Australian Government. URL: https://www.tga.gov.au/ [accessed 2021-09-28]
- What is MDA Online? Australasian Medical Publishing Company. URL: https://www.mdaonline.com.au/ [accessed 2021-09-28]
- Tools that get you closer to your audience. Typeform. URL: https://www.typeform.com/product/ [accessed 2021-09-28]
- NICE to retire Guidance app. National Institute for Health and Care Excellence. 2018. URL: https://www.nice.org.uk/news/article/nice-to-retire-guidance-app [accessed 2021-09-28]
- Watson HA, Tribe RM, Shennan AH. The role of medical smartphone apps in clinical decision-support: A literature review. Artif Intell Med 2019 Sep;100:101707. [CrossRef] [Medline]
- Ajay VS, Jindal D, Roy A, Venugopal V, Sharma R, Pawar A, et al. Development of a Smartphone-Enabled Hypertension and Diabetes Mellitus Management Package to Facilitate Evidence-Based Care Delivery in Primary Healthcare Facilities in India: The mPower Heart Project. J Am Heart Assoc 2016 Dec 21;5(12) [FREE Full text] [CrossRef] [Medline]
- Whitelaw S, Pellegrini DM, Mamas MA, Cowie M, Van Spall HGC. Barriers and facilitators of the uptake of digital health technology in cardiovascular care: a systematic scoping review. Eur Heart J Digit Health 2021 Mar;2(1):62-74 [FREE Full text] [CrossRef] [Medline]
- Frederix I, Caiani EG, Dendale P, Anker S, Bax J, Böhm A, et al. ESC e-Cardiology Working Group Position Paper: Overcoming challenges in digital health implementation in cardiovascular medicine. Eur J Prev Cardiol 2019 Jul;26(11):1166-1177. [CrossRef] [Medline]
- Jiang X, Ming W, You JH. The Cost-Effectiveness of Digital Health Interventions on the Management of Cardiovascular Diseases: Systematic Review. J Med Internet Res 2019 Jun 17;21(6):e13166 [FREE Full text] [CrossRef] [Medline]
|GP: general practitioner|
Edited by T Leung; submitted 10.12.21; peer-reviewed by D Madhusudhan, SS Amritphale; comments to author 13.01.22; revised version received 03.03.22; accepted 20.06.22; published 05.08.22Copyright
©Stacey Matthews, Brooke Atkins, Natalie Walton, Julie-Anne Mitchell, Garry Jennings, Amanda K Buttery. Originally published in JMIR Formative Research (https://formative.jmir.org), 05.08.2022.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.