Published on in Vol 8 (2024)

This is a member publication of University of Cambridge (Jisc)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/54747, first published .
Machine Learning and Symptom Patterns in Degenerative Cervical Myelopathy: Web-Based Survey Study

Machine Learning and Symptom Patterns in Degenerative Cervical Myelopathy: Web-Based Survey Study

Machine Learning and Symptom Patterns in Degenerative Cervical Myelopathy: Web-Based Survey Study

Original Paper

1University of Manchester, Manchester, United Kingdom

2University of Cambridge, Cambridge, United Kingdom

3Icahn School of Medicine at Mount Sinai, New York, NY, United States

*these authors contributed equally

Corresponding Author:

Benjamin M Davies, BSc, MPhil, MRCS

University of Cambridge

The Old Schools, Trinity Ln, Cambridge

Cambridge, CB2 1TN

United Kingdom

Phone: 44 01223 337733

Email: bd375@cam.ac.uk


Background: Degenerative cervical myelopathy (DCM), a progressive spinal cord injury caused by spinal cord compression from degenerative pathology, often presents with neck pain, sensorimotor dysfunction in the upper or lower limbs, gait disturbance, and bladder or bowel dysfunction. Its symptomatology is very heterogeneous, making early detection as well as the measurement or understanding of the underlying factors and their consequences challenging. Increasingly, evidence suggests that DCM may consist of subgroups of the disease, which are yet to be defined.

Objective: This study aimed to explore whether machine learning can identify clinically meaningful groups of patients based solely on clinical features.

Methods: A survey was conducted wherein participants were asked to specify the clinical features they had experienced, their principal presenting complaint, and time to diagnosis as well as demographic information, including disease severity, age, and sex. K-means clustering was used to divide respondents into clusters according to their clinical features using the Euclidean distance measure and the Hartigan-Wong algorithm. The clinical significance of groups was subsequently explored by comparing their time to presentation, time with disease severity, and other demographics.

Results: After a review of both ancillary and cluster data, it was determined by consensus that the optimal number of DCM response groups was 3. In Cluster 1, there were 40 respondents, and the ratio of male to female participants was 13:21. In Cluster 2, there were 92 respondents, with a male to female participant ratio of 27:65. Cluster 3 had 57 respondents, with a male to female participant ratio of 9:48. A total of 6 people did not report biological sex in Cluster 1. The mean age in this Cluster was 56.2 (SD 10.5) years; in Cluster 2, it was 54.7 (SD 9.63) years; and in Cluster 3, it was 51.8 (SD 8.4) years. Patients across clusters significantly differed in the total number of clinical features reported, with more clinical features in Cluster 3 and the least clinical features in Cluster 1 (Kruskal-Wallis rank sum test: χ22=159.46; P<.001). There was no relationship between the pattern of clinical features and severity. There were also no differences between clusters regarding time since diagnosis and time with DCM.

Conclusions: Using machine learning and patient-reported experience, 3 groups of patients with DCM were defined, which were different in the number of clinical features but not in the severity of DCM or time with DCM. Although a clearer biological basis for the clusters may have been missed, the findings are consistent with the emerging observation that DCM is a heterogeneous disease, difficult to diagnose or stratify. There is a place for machine learning methods to efficiently assist with pattern recognition. However, the challenge lies in creating quality data sets necessary to derive benefit from such approaches.

JMIR Form Res 2024;8:e54747

doi:10.2196/54747

Keywords



Degenerative cervical myelopathy (DCM) is a progressive spinal cord injury caused by spinal cord compression from degenerative pathology and consists of various subcategories of pathology, including cervical spondylotic myelopathy, ossification of the posterior longitudinal ligament, ossification of the ligamentum flavum, and degenerative disc disease [1-4]. It is estimated to affect 2% of adults, although fewer than 10% are currently diagnosed [5,6]. Surgery is the mainstay of treatment for DCM, aiming to decompress the spinal cord [2,7-9].

DCM often presents with neck pain, sensorimotor dysfunction in the upper or lower limbs, gait disturbance, and bladder or bowel dysfunction [2,10-14]. Examination findings include upper motor neuron signs in the limbs, such as positive Babinski sign, positive Hoffman sign, hyperreflexia, and increased tone [2,10-13]. Its symptomatology is very heterogeneous, making early detection difficult. This heterogeneity makes it difficult to measure or understand what drives consequences. For instance, the heterogeneity has made it harder to understand health-related quality of life [15]. This has also hindered comparisons between studies and the development of clinical practice guidelines and recommendations for DCM [16,17]. Additionally, once detected, DCM is unpredictable due to a lack of reliable methods to determine prognosis.

Increasingly, evidence suggests that DCM may consist of subgroups of the disease, which still need to be defined [18-20]. Machine learning can help in finding them. In fact, machine learning has shown potential in predicting health-related quality of life after surgery for mild DCM and outcome after surgery, although external validation and prospective analysis are still needed [21,22]. The use of machine learning in identifying these subgroups is dependent on the data set. Munro et al [23] (2023) provide a unique and comprehensive description of the effects of DCM from the perspective of people living with DCM [24,25]. This is a data set that could lend itself to machine learning analysis due to its comprehensiveness.

The objective of this study was to explore whether machine learning can identify clinically meaningful groups of patients based on solely clinical features.


Data Set

A mixed methods cross-sectional study was conducted by a team from the University of Cambridge through Myelopathy.org [26], a global charity dedicated to DCM. A focus group session of people with DCM and their supporters was used to inform the development of a web-based survey to explore the consequences of living with DCM. The survey was advertised using the Myelopathy.org website, an international nonprofit organization dedicated to promoting understanding and awareness of DCM. Survey participants were asked to specify the clinical features they had experienced, their principal presenting complaint, and time to diagnosis as well as demographic information, including disease severity, age, and sex. The data consist of 189 yes or no responses to a list of 76 clinical features. This was published in a paper, titled “Targeting earlier diagnosis: what symptoms come first in degenerative cervical myelopathy?” [23], wherein the full methodology is detailed.

Analysis

Patients were grouped into subsets with similar characteristics using k-means clustering. K-means clustering is a method that groups data into “k” nonoverlapping, distinct subsets by finding centroids in the data representing each cluster’s center and allocating data points to each cluster by minimizing within-cluster variance around centroids. K-means clustering was used due to its efficiency for small data sets and explainability, aiming to group respondents into clusters based on their clinical features, using the Euclidean distance measure and the Hartigan-Wong algorithm [27]. The optimal number of clusters (k) was determined through the inspection of 3 ancillary methods, namely, the elbow, silhouette, and gap statistic methods [28]. The clinical significance of groups was subsequently explored by comparing their time to presentation, time with disease severity, and other demographics. DCM severity was assessed using total Modified Japanese Orthopaedic Association scores [29]. Noncomplete records were not excluded, and missing data were not imputed. All analyses were conducted in R (version 4.1.0; R Foundation for Statistical Computing) [30].

Ethical Considerations

This study was conducted with ethical approval from the University of Cambridge (HBREC.2019.14). At the start of the survey, participants were provided with an overview of the study and definition of DCM, and by continuing into the survey, participants were confirming their diagnosis of DCM and providing informed consent to participate. All data collected were anonymous. No incentives were offered for the completion of the surveys.


Cohort Demographics

Of the 189 participants, 134 were female and 49 were male (6 did not report biological sex). Respondents were on average 54.1 years of age. A total of 29 of them had mild DCM, 68 had moderate DCM, and 92 had severe DCM. The majority (131/189, 69%) reported having had surgery for DCM.

Cluster Analysis

Ancillary methods suggested different optimal numbers of clusters (k). Elbow, silhouette, and gap statistic methods identified k=3, k=2, and k=5, respectively (Figure 1A). The data were hence clustered into multiple values of k and inspected (Figure 1B). After a review of both ancillary and cluster data, it was determined by consensus between AYT and BD that the optimal number of DCM response groups was 3. The reasoning behind this was that the ancillary curves in 2 out of 3 ancillary methods plateaued from k≥3 (Figure 1A), but clusters above k≥4 overlapped (Figure 1B).

Figure 1. (A) Determining the optimal number of clusters; (B) k-means clustering (euclidean).

Characterization Analysis

In Cluster 1, there were 40 respondents, and the ratio of male to female participants was 13:21. In Cluster 2, there were 92 respondents, with a male to female participant ratio of 27:65. Cluster 3 had 57 respondents, with a male to female participant ratio of 9:48. A total of 6 people did not report biological sex in Cluster 1. The mean age was 56.2 (SD 10.5) years in this cluster; in Cluster 2, it was 54.7 (SD 9.63) years; and in Cluster 3, it was 51.8 (SD 8.4) years.

A spider chart was subsequently generated to explore the clinical significance of the clusters, wherein the curves did not cross (Figure 2A; Multimedia Appendix 1).

It was also checked if patients in the different groups experienced differing numbers of features (Figure 2B). Patients across clusters significantly differed in the total number of clinical features reported, with more clinical features in Cluster 3 and the least clinical features in Cluster 1 (Kruskal-Wallis rank sum test: χ22=159.46; P<.001).

To check whether patients with more clinical features had a more severe form of DCM, patterns of clinical features against severity were compared. The results showed no relationship between the pattern of clinical features and severity (Figure 2C).

Patterns of clinical features against both time since diagnosis and time with DCM were also analyzed. As shown in Figures 2D and 2E, there did not seem to be any differences between clusters in these distributions.

Figure 2. (A) Spider charts showing survey responses across clusters (the abbreviations along the circumference are detailed in the table in the Multimedia Appendix 1); the radius represents relative frequency, normalized to 1; (B) total number of clinical features reported across clusters; (C) proportions of degenerative cervical myelopathy (DCM) severity across clusters (based on the Modified Japanese Orthopaedic Association scores); (D) distribution of time taken to be diagnosed with DCM in each cluster; (E) distribution of time with DCM in each cluster.

Principal Findings

Cluster analysis suggested 3 optimal subgroups based on clinical features. When exploring why these groups differed in terms of cohort demographics, only the number of reported symptoms differed significantly. The pattern of clinical features within each of the 3 groups was similar. Notably, the 3 curves in the spider chart appear to peak and trough in a similar pattern, suggesting that there was no difference in the pattern of clinical features. The concentricity of curves, however, suggested that clustering may be due to the total number of features experienced. This possibility was statistically significant (Kruskal-Wallis rank sum test). Finally, there was no link between the groups and disease severity, time with DCM, and time since diagnosis.

Limitations

This study has several limitations. The data represent a single time point cross-sectional survey of an internet-recruited cohort of patients, which could limit the generalizability of the findings. Additionally, information on disease characteristics, used for exploring the clinical significance of clusters, was limited to time with symptoms and a self-reported modified Japanese Orthopaedic Association score [31]. A more diverse data set would be more insightful, especially in DCM, wherein the nuances of symptom presentation and progression are critical. The sample size is also relatively small by machine learning standards. Finally, only 1 analysis method (k-means clustering) was performed, which may prevent us from capturing the full complexity of DCM symptomatology, especially with the increasing prominence of personalized approaches [32].

That being said, this is a unique data set, formed from the unrestricted perspectives of almost 200 patients; it was formed without any preconceptions regarding what symptoms were considered related to DCM. The result is also not unexpected. Standard analytical approaches, using more traditional data sets, have failed to stratify patients by symptoms [33]. Consequently, although a clearer biological basis for the clusters may have been missed, the findings are consistent with the emerging observation that DCM is a heterogeneous disease, difficult to diagnose or stratify [15-17]. This has been highlighted by the work of Cook et al [34] (2022) and is perhaps reflected in our inability to explain the variability in the quality of life in DCM [15].

This study shows that there is certainly a role for machine learning methods to efficiently assist with pattern recognition, but data sets must be large, valid, and comprehensive. In DCM, the challenge and priority appear to be less focused on data set size and more focused on the type of data [35]. For example, our redefinition of DCM in terms of time, mechanical stress, and vulnerability to sustain a spinal cord injury has highlighted the potential significance of various disease factors; these factors range from frailty and genetics to the type of pathology causing compression, encompassing the likely heterogeneous mechanical loading they induce [20]. Further, there are few valid and reliable outcome measures available, with most relying on face-to-face presentations to measure changes over the course of months, exhibiting low statistical power. The work of Cook et al [34] (2022) has highlighted that the experience of DCM is driven by social determinants—features such as ethnicity as well as educational, and economic status [34]. This means subjectivity in outcomes will drive current variability. Novel biomarkers, including imaging, blood, and digital biomarkers, are likely to hold value in this context, offering more disease-specific and sensitive disease indicators [36]. The need for more comprehensive and improved measurement is a firm priority in DCM [16]. Therefore, artificial intelligence undoubtedly has an important role in the future of DCM research and care. To our knowledge, such measures do not currently exist. Analysis of one of the most detailed cohorts also failed to identify biologically significant strata [22,37]. Therefore, the short-term challenge for our community lies in creating quality data sets necessary to derive benefit from these emerging analytical approaches.

Conclusions

Using machine learning and patient-reported experience, 3 groups of patients with DCM were defined. These groups differed in the number of clinical features reported but not in the severity of DCM, time since diagnosis, or time with DCM. The significance and generalization of this study remain uncertain. Overall, this study confirms the role of machine learning in DCM research, but more pressingly, it confirms the need to curate the right data sets.

Acknowledgments

BMD is supported by a National Institute for Health Research (NIHR) Clinical Doctoral Fellowship.

The views expressed in this publication are those of the authors and not necessarily those of the National Health Service (NHS), the NIHR, or the Department of Health.

Data Availability

The data sets generated and analyzed during this study are available from the corresponding author upon reasonable request.

Conflicts of Interest

BMD is the director of MoveMed Ltd.

Multimedia Appendix 1

Additional material regarding survey completion.

DOCX File , 19 KB

  1. Nouri A, Tetreault L, Singh A, Karadimas S, Fehlings M. Degenerative cervical myelopathy. Spine. 2015;40(12):E675-E693. [FREE Full text] [CrossRef]
  2. Badhiwala JH, Ahuja C, Akbar M, Witiw C, Nassiri F, Furlan J, et al. Degenerative cervical myelopathy - update and future directions. Nat Rev Neurol. Feb 2020;16(2):108-124. [FREE Full text] [CrossRef] [Medline]
  3. Davies BM, Khan D, Barzangi K, Ali A, Mowforth O, Nouri A, et al. We choose to call it 'degenerative cervical myelopathy': findings of AO spine RECODE-DCM, an international and multi-stakeholder partnership to agree a standard unifying term and definition for a disease. Global Spine J. Jun 29, 2022:21925682221111780. [FREE Full text] [CrossRef] [Medline]
  4. Davies BM, Mowforth O, Smith E, Kotter MR. Degenerative cervical myelopathy. BMJ. Feb 22, 2018;360:k186. [FREE Full text] [CrossRef] [Medline]
  5. Grodzinski B, Stubbs D, Davies B. Most degenerative cervical myelopathy remains undiagnosed, particularly amongst the elderly: modelling the prevalence of degenerative cervical myelopathy in the United Kingdom. J Neurol. Jan 2023;270(1):311-319. [FREE Full text] [CrossRef] [Medline]
  6. Smith SS, Stewart M, Davies B, Kotter M. The prevalence of asymptomatic and symptomatic spinal cord compression on magnetic resonance imaging: a systematic review and meta-analysis. Global Spine J. May 2021;11(4):597-607. [FREE Full text] [CrossRef] [Medline]
  7. Fehlings MG, Ibrahim A, Tetreault L, Albanese V, Alvarado M, Arnold P, et al. A global perspective on the outcomes of surgical decompression in patients with cervical spondylotic myelopathy: results from the prospective multicenter AOSpine international study on 479 patients. Spine (Phila Pa 1976). Sep 01, 2015;40(17):1322-1328. [FREE Full text] [CrossRef] [Medline]
  8. Nachiket V, Bjoern P, Hernan E. G, Lisaweta R, Herbert W, Christian H, et al. Efficacy and safety of surgical decompression in patients with cervical spondylotic myelopathy: results of the aospine north america prospective multi-center study. Fehlings MG, Wilson JR, Kopjar B, Yoon ST, Arnold PM, Massicotte EM, Vaccaro AR, Brodke DS, Shaffrey CI, Smith JS, Woodard EJ, Banco RJ, Chapman JR, Janssen ME, Bono CM, Sasso RC, Dekutoski MB, Gokaslan ZL. J Bone Joint Surg Am. 2013 Sep 18;95(18):1651-8. doi: 10.2106/JBJS.L.00589. Spine J. Jan 2014;14(1):197. [CrossRef]
  9. Wilson JR, Tetreault L, Kim J, Shamji M, Harrop J, Mroz T, et al. State of the art in degenerative cervical myelopathy: an update on current clinical evidence. Neurosurgery. Mar 01, 2017;80(3S):S33-S45. [FREE Full text] [CrossRef] [Medline]
  10. 10 TJ, Bartleson B. Cervical spondylotic myelopathy. Neurologist. 2010;16:176. [FREE Full text] [CrossRef]
  11. Kalsi-Ryan S, Karadimas S, Fehlings M. Cervical spondylotic myelopathy: the clinical phenomenon and the current pathobiology of an increasingly prevalent and devastating disorder. Neuroscientist. Aug 2013;19(4):409-421. [FREE Full text] [CrossRef] [Medline]
  12. Harrop J, Naroji S, Maltenfort M, Anderson D, Albert T, Ratliff J, et al. Cervical myelopathy. Spine. 2010;35(6):620-624. [FREE Full text] [CrossRef]
  13. 13 TL, Goldstein C, Arnold P, Harrop J, Hilibrand A, Nouri A. Degenerative cervical myelopathy: a spectrum of related disorders affecting the aging spine. Neurosurgery. 2015;77:S51-S67. [FREE Full text] [CrossRef]
  14. Boerger T, Alsouhibani A, Mowforth O, Hamilton J, Lalkhen A, Davies B, et al. Moving beyond the neck and arm: the pain experience of people with degenerative cervical myelopathy who have pain. Global Spine J. Sep 2022;12(7):1434-1442. [FREE Full text] [CrossRef] [Medline]
  15. 15 HY, Mowforth O, Davies B, Kotter M. Determinants of quality of life in degenerative cervical myelopathy: a systematic review. Br J Neurosurg. 2021:71-81. [FREE Full text] [CrossRef]
  16. Tetreault L, Garwood P, Gharooni A, Touzet A, Nanna-Lohkamp L, Martin A, et al. Improving assessment of disease severity and strategies for monitoring progression in degenerative cervical myelopathy [AO Spine RECODE-DCM Research Priority Number 4]. Global Spine J. Feb 2022;12(1_suppl):64S-77S. [FREE Full text] [CrossRef] [Medline]
  17. Davies BM, McHugh M, Elgheriani A, Kolias A, Tetreault L, Hutchinson PJA, et al. Reported outcome measures in degenerative cervical myelopathy: a systematic review. PLoS One. 2016;11(8):e0157263. [FREE Full text] [CrossRef] [Medline]
  18. Pope DH, Davies B, Mowforth O, Bowden A, Kotter M. Genetics of degenerative cervical myelopathy: a systematic review and meta-analysis of candidate gene studies. J Clin Med. Jan 20, 2020;9(1):A. [FREE Full text] [CrossRef] [Medline]
  19. Liu Z, Bian B, Wang G, Tian C, Lv Z, Shao Z, et al. Evaluation of microstructural changes in spinal cord of patients with degenerative cervical myelopathy by diffusion kurtosis imaging and investigate the correlation with JOA score. BMC Neurol. May 13, 2020;20(1):185. [FREE Full text] [CrossRef] [Medline]
  20. Davies BM, Mowforth O, Gharooni A, Tetreault L, Nouri A, Dhillon R, et al. A new framework for investigating the biological basis of degenerative cervical myelopathy [AO Spine RECODE-DCM Research Priority Number 5]: mechanical stress, vulnerability and time. Global Spine J. Feb 2022;12(1_suppl):78S-96S. [FREE Full text] [CrossRef] [Medline]
  21. Khan O, Badhiwala J, Witiw C, Wilson J, Fehlings M. Machine learning algorithms for prediction of health-related quality-of-life after surgery for mild degenerative cervical myelopathy. Spine J. Oct 2021;21(10):1659-1669. [FREE Full text] [CrossRef] [Medline]
  22. Merali ZG, Witiw C, Badhiwala J, Wilson J, Fehlings M. Using a machine learning approach to predict outcome after surgery for degenerative cervical myelopathy. PLoS One. 2019;14(4):e0215133. [FREE Full text] [CrossRef] [Medline]
  23. Munro CF, Yurac R, Moritz ZC, Fehlings MG, Rodrigues-Pinto R, Milligan J, et al. Targeting earlier diagnosis: what symptoms come first in degenerative cervical myelopathy? PLoS One. 2023;18(3):e0281856. [FREE Full text] [CrossRef] [Medline]
  24. Davies BM, Munro C, Khan D, Fitzpatrick S, Hilton B, Mowforth O, et al. Outcomes of degenerative cervical myelopathy from the perspective of persons living with the condition: findings of a semistructured interview process with partnered internet survey. Global Spine J. Apr 2022;12(3):432-440. [FREE Full text] [CrossRef] [Medline]
  25. Boerger TF, Davies B, Sadler I, Sarewitz E, Kotter M. Patient, sufferer, victim, casualty or person with cervical myelopathy: let us decide our identifier. Integr Healthc J. 2020;2(1):e000023. [FREE Full text] [CrossRef] [Medline]
  26. Myelopathy.org. URL: https://myelopathy.org/ [accessed 2024-01-05]
  27. Hartigan JA, Wong MA. Algorithm AS 136: a k-means clustering algorithm. Applied Statistics. 1979;28(1):100. [FREE Full text] [CrossRef]
  28. K-means cluster analysis. UC Business Analytics R Programming Guide. 2018. URL: https://uc-r.github.io/kmeans_clustering [accessed 2024-01-05]
  29. Witiw CD, Fehlings MG. Degenerative cervical myelopathy. CMAJ. Jan 23, 2017;189(3):E116-E116. [FREE Full text] [CrossRef] [Medline]
  30. R: The R Project for Statistical Computing. URL: https://www.r-project.org/ [accessed 2024-01-05]
  31. 30 RJ, Shi W, Cyriac M, Kim J, Zhou F, Easley K. The P-mJOA: a patient-derived, self-reported outcome instrument for evaluating cervical myelopathy comparison with the mJOA. Clin Spine Surg. 2018;31:E115-E120. [FREE Full text] [CrossRef]
  32. Hejrati N, Moghaddamjou A, Marathe N, Fehlings M. Degenerative cervical myelopathy: towards a personalized approach. Can J Neurol Sci. Nov 2022;49(6):729-740. [FREE Full text] [CrossRef] [Medline]
  33. Badhiwala JH, Wilson J. The natural history of degenerative cervical myelopathy. Neurosurg Clin N Am. Jan 2018;29(1):21-32. [FREE Full text] [CrossRef] [Medline]
  34. Rethorn ZD, Cook C, Park C, Somers T, Mummaneni PV, Chan AK, et al. Social risk factors predicting outcomes of cervical myelopathy surgery. J Neurosurg Spine. Jan 28, 2022:1-8. [FREE Full text] [CrossRef] [Medline]
  35. Yanez Touzet A, Bhatti A, Dohle E, Bhatti F, Lee K, Furlan J, et al. Clinical outcome measures and their evidence base in degenerative cervical myelopathy: a systematic review to inform a core measurement set (AO Spine RECODE-DCM). BMJ Open. Jan 19, 2022;12(1):e057650. [FREE Full text] [CrossRef] [Medline]
  36. Touzet AY, Houhou T, Rahic Z, Laufer I. Validity of a smartphone app to objectively monitor performance outcomes in degenerative cervical myelopathy?: an observational study. Validity of a smartphone app to objectively monitor performance outcomes in degenerative cervical myelopathy?: an observational study. 2023:2023. [CrossRef]
  37. Tetreault LA, Côté P, Kopjar B, Arnold P, Fehlings M. A clinical prediction model to assess surgical outcome in patients with cervical spondylotic myelopathy: internal and external validations using the prospective multicenter AOSpine North American and international datasets of 743 patients. Spine J. Mar 01, 2015;15(3):388-397. [FREE Full text] [CrossRef] [Medline]


DCM: degenerative cervical myelopathy


Edited by A Mavragani; submitted 20.11.23; peer-reviewed by S Yordanov, M Porter; comments to author 12.12.23; revised version received 18.12.23; accepted 19.12.23; published 25.01.24.

Copyright

©Alvaro Yanez Touzet, Tanzil Rujeedawa, Colin Munro, Konstantinos Margetis, Benjamin M Davies. Originally published in JMIR Formative Research (https://formative.jmir.org), 25.01.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.