Published on in Vol 5, No 2 (2021): February

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/14179, first published .
Ecological Momentary Assessment Using Smartphones in Patients With Depression: Feasibility Study

Ecological Momentary Assessment Using Smartphones in Patients With Depression: Feasibility Study

Ecological Momentary Assessment Using Smartphones in Patients With Depression: Feasibility Study

Original Paper

1Sorbonne Université, AP-HP, Service de psychiatrie adulte de la Pitié-Salpêtrière, Institut du Cerveau, ICM, F-75013, Paris, France

2National Institute for Health Research Health Protection Research Unit in Healthcare Associated Infections and Antimicrobial Resistance, Imperial College London, London, United Kingdom

3French Institute for Medical Research (Inserm), Infection Antimicrobials Modelling Evolution, UMR 1137, University Paris Diderot, Paris, France

*these authors contributed equally

Corresponding Author:

Redwan Maatoug, Dr med, MSc

Sorbonne Université, AP-HP

Service de psychiatrie adulte de la Pitié-Salpêtrière

Institut du Cerveau, ICM, F-75013

47-83 Boulevard de l'hôpital

Paris, 75013

France

Phone: 33 682476484

Email: redwanmaatoug@gmail.com


Background: Ecological momentary assessment (EMA) is a promising tool in the management of psychiatric disorders and particularly depression. It allows for a real-time evaluation of symptoms and an earlier detection of relapse or treatment efficacy. The generalization of the smartphone in the modern world offers a new, large-scale support for EMA.

Objective: The main objective of this study was twofold: (1) to assess patients’ compliance with an EMA smartphone app defined by the number of EMAs completed, and (2) to estimate the external validity of the EMA using a correlation between self-esteem/guilt/mood variables and Hamilton Depression Rating Scale (HDRS) score.

Methods: Eleven patients at the Pitié-Salpêtrière Hospital, Paris, France, were monitored for 28 days by means of a smartphone app. Every patient enrolled in the study had two types of assessment: (1) three outpatient consultations with a psychiatrist at three different time points (days 1, 15, and 28), and (2) real-time data collection using an EMA smartphone app with a single, fixed notification per day at 3 pm for 28 days. The results of the real-time data collected were reviewed during the three outpatient consultations by a psychiatrist using a dashboard that aggregated all of the patients’ data into a user-friendly format.

Results: Of the 11 patients in the study, 6 patients attended the 3 outpatient consultations with the psychiatrist and completed the HDRS at each consultation. We found a positive correlation between the HDRS score and the variables of self-esteem, guilt, and mood (Spearman correlation coefficient 0.57). Seven patients completed the daily EMAs for 28 days or longer, with an average response rate to the EMAs of 62.5% (175/280). Furthermore, we observed a positive correlation between the number of responses to EMAs and the duration of follow-up (Spearman correlation coefficient 0.63).

Conclusions: This preliminary study with a prolonged follow-up demonstrates significant patient compliance with the smartphone app. In addition, the self-assessments performed by patients seemed faithful to the standardized measurements performed by the psychiatrist. The results also suggest that for some patients it is more convenient to use the smartphone app than to attend outpatient consultations.

JMIR Form Res 2021;5(2):e14179

doi:10.2196/14179

Keywords



A promising development in the treatment of unipolar depression consists of improving the monitoring of depressive symptoms at home. Indeed, a personalized follow-up and continuous assessment of the symptomatology and its contextual influences are paramount to the management of depression.

Ecological momentary assessment (EMA) is a method used in psychiatric research that collects real-time data on symptoms, microenvironmental fluctuations, and medication intake in the everyday environment of patients.

Several reviews have confirmed an interest in EMA in various psychiatric disorders [1-3], particularly mood disorders [4-6] such as depression [7,8]. In the context of mood disorders, symptoms are subjective and usually fluctuate from day to day. Consequently, the evaluation by a clinician at each consultation is necessary but is less precise than a continuous self-assessment. Furthermore, studies such as the one by Ben-Zeev et al [9] highlighted that during consultation patients tended to talk only about the negative aspects of their recent history and forgot about the positive ones. EMA helps reduce this bias and improves our understanding of mood fluctuations, including their links with the environment and medication adherence [10,11].

The first EMA studies were published in the 1980s by Csikszentmihalyi and LeFevre [12]. The authors were interested in knowing whether the quality of human experience was more influenced by whether a person was at work or at leisure or by whether a person was in “flow” (ie, a condition of optimal experience created when one’s environment presents high challenges that are met by one’s skills). The first EMA studies in depression, conducted using paper-and-pencil daily diaries, were published in the 1990s [13].

In previous EMA studies, data were mostly collected using a notebook that the patient had to complete. To avoid missing data, patients were reminded by a signal to write in the notebook. This method suffered from the risk of recall bias, as it was never certain when the patient filled in the questionnaire. In that regard, Stone et al [14] showed that “although patients reported high compliance, actual compliance was low.” Using the patient’s smartphone makes it possible to record the time of data collection. Over the last 10 years, smartphone use has become widespread in the population. While the penetration rate of smartphones was 29% in the French population in 2012, it exceeded 65% in 2018, offering an interesting tool for real-time monitoring of symptoms [15,16]. However, EMA studies using a smartphone app associated with an online platform are still scarce [17,18], although a few studies have used EMA apps on the smartphones of participants with depressive symptoms and/or bipolar disorder [19].

Recent EMA studies that included data restitution to patients have demonstrated that patients’ knowledge of their mood fluctuations and their context can assist them in understanding and managing their pathologies and, consequently, allow them to switch from passive consumers into active participants in their own care [20]. This innovative use of EMA may have the potential to improve prediction of relapse and remission [21,22].

Finally, a few studies have been published exploring the benefits of using a dashboard to provide data restitution and feedback from the clinician to the patient using the EMA data collected. Simons et al [20,23] demonstrated the therapeutic interest in restoring patient results from the EMA in a randomized trial involving approximately 102 patients with depression. They explored not only the collection of clinical information and their predictive values in terms of prognostic evaluations but also the return of this information to patients living with depression and the economic consequences of such a method.

Thus, coupling EMA with the patient’s smartphone has three main goals: (1) the possibility of real-time and continuous evaluation of symptoms [24], (2) early detection of relapse and treatment efficiency [25], and (3) generalization of the tool on a large scale [26,27].

The aim of our work was to highlight the relevance of using EMA with smartphones to track patients with depression. The main objective of this study was twofold: (1) to assess patients’ compliance with an EMA smartphone app defined by the number of EMAs completed, and (2) to estimate the external validity of the EMA using a correlation between self-esteem/guilt/mood variables and Hamilton Depression Rating Scale (HDRS) score.


This 28-day, single-center prospective study took place between August and October 2019. During the study, patients had continuous full access to the EMA app on their smartphone.

Population

All patients included in the study were experiencing a major depressive episode according to the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition criteria, with a recent introduction (less than one month) or modification of an antidepressant. Additional inclusion criteria were ownership of a smartphone and age over 18 years. Patients were included over a 2-week period.

The study took place at the psychiatry department of the Pitié-Salpêtrière Hospital (Paris, France). With the help of the psychiatrist, the patients downloaded the EMA app onto their personal smartphone. Additionally, they were informed that the clinical data were processed in real time and reviewed during the outpatient consultations with the psychiatrist.

Assessments

Every patient enrolled in the study was expected to have two forms of assessment: (1) three outpatient consultations with a psychiatrist at three different time points (days 1, 15, and 28), and (2) real-time data collection using the EMA smartphone app, with a single, fixed notification daily at 3 pm.

Outpatient Consultations

During the three outpatient consultations, the psychiatrist assessed anxiety and depressive symptoms using the HDRS. The HDRS is a 17-item questionnaire designed for adults and used to rate the severity of their depression from 0 (no symptoms of depression) to 52 (severe depression) by probing mood, feelings of guilt, suicide ideation, insomnia, agitation or retardation, anxiety, weight loss, and somatic symptoms [28].

Smartphone App

The EMA smartphone app is available in both iPhone (iOS) and Android operating systems (Figure 1).

Every day at 3 pm, each patient received a notification on their smartphone to remind them to respond to the EMA questions. However, patients could choose which questions they wished to answer. Indeed, a drop-down menu is present on the app and allows the patient to choose between different questions. This tool was developed to allow the patient to answer any particular question according to their mood or preference. Therefore, patients were not obliged to answer all of the questions and could stop at any time.

Figure 1. Illustration of the ecological momentary assessment smartphone app.
View this figure

Each patient was asked to answer 10 questions per day for a period of 28 days, for a total of 280 EMAs to be completed over the course of the study. In order to assess the compliance of patients to the smartphone app in their home environment, no external reminders were sent to the patients.

The data collected with the smartphone app were as follows: antidepressant compliance (binary answer); potential side effects of the antidepressant (binary answer); mood evaluation (“Could you please rate your mood over the last 3 days?”); self-esteem (“How self-confident have you been during the last 3 days?”); guilt (“How guilty did you feel during the last 3 days?”); sleep parameters (bedtime and time to fall asleep, sleep quality); appetite (binary answer and weight changes); and general interest.

Each visual analog scale used semantic differentials as anchor points for the patient, which were converted to 0 to 100 for the clinician.

Study Design

The study was developed in collaboration with Ad Scientiam, a French start-up specialized in real-life data, an initiative of the Brain and Spine Institute at the Pitié Salpêtrière Hospital in Paris, France.

No direct nominative data were collected in this study. All participants were identified by an identification code in the app. The study was approved by the local ethics committee for the protection of persons at Sorbonne University and by the Chair of the data protection commission (Commission nationale de l’informatique et des libertés). Written informed consent was obtained from all participants in the study.


General Description

Eleven patients experiencing a major depressive episode were included in this feasibility study. All patients downloaded the EMA app onto their personal smartphone (Table 1).

Table 1. Summary of compliance with the smartphone app and outpatient consultations for each patient.
PatientGenderAge (years)Days of follow-upaNumber of EMAsb completedcNumber of outpatient consultations attendedd
1Female45353002
2Male52352523
3Male51351063
4Female41332803
5Female37302972
6Male45292103
7Male5628563
8Female5717923
9Male7011531
10Male678781
11Male695321

aThe duration of follow-up was 28 days. Some patients exceeded 28 days because they continued to use the app after the end of the study.

bEMAs: ecological momentary assessments.

cThe number of EMAs completed during the total follow-up period; data collected after the 28th day were included in the total number, as the objective was to measure patient compliance with the smartphone app.

dThree outpatient consultations were scheduled (on days 1, 15, and 28).

Of the 11 participants, 6 patients attended the three outpatient consultations with the psychiatrist and completed the HDRS at each consultation. Seven of the 11 patients responded to the EMAs for a duration of 28 days or longer; the mean duration of follow-up was 24 days. It should be noted that of the 7 patients who completed the EMAs for a minimum of 28 days, 6 patients continued to complete them after the end of the study; patients 1 and 5 continued to complete EMAs without attending outpatient consultations with the psychiatrist (Figure 2).

Figure 2. Monitoring of each patient for the 28 days of the study. The first value for each patient corresponds to the Hamilton Depression Rating Scale (HDRS) score at the first outpatient consultation (day 1); the second and third values correspond to the HDRS scores at the second (day 15) and third (day 28) outpatient consultations, respectively. The absence of a value indicates that the patient missed the consultation. The intensity of the green bar correlates with the number of questions out of a total of 10 that were answered in a given day.
View this figure

EMAs and HDRS Score Correlation

On average, 175 (175/280, 62.5%) EMAs were completed by each patient. The least compliant patient in the smartphone app responded to 53 EMAs and the most compliant patient responded to 278 EMAs.

When we focused on the six patients who completed the HDRS three times, we found a positive correlation between the average HDRS score and the average score for the variables of self-esteem, guilt, and mood (Spearman correlation coefficient 0.57).

Furthermore, there was a correlation between the number of responses to EMAs and the duration of follow-up. In fact, the higher the response rate to EMAs, the longer the follow-up (Spearman correlation coefficient 0.63).


This is the first feasibility study of an EMA tool with daily notifications for 28 days in patients experiencing a major depressive episode.

Our first objective was to assess the compliance to the smartphone app using the number of EMAs answered. Of the 11 patients, 7 patients responded to the EMAs for a duration of 28 days or longer; the mean duration of follow-up was 24 days. Moreover, on average, 175 responses (out of 280 possible responses; 62.5%) to EMAs were given. Surprisingly, we observed that two patients preferred answering the EMAs rather than attending the outpatient consultations with the psychiatrist.

The second objective was to estimate the external validity of the EMA using a correlation between self-esteem/guilt/mood variables and HDRS score (Figure 3). Interestingly, patients seemed to assess their self-esteem, guilt, and mood as accurately as a psychiatrist using the standardized HDRS (Spearman correlation coefficient 0.57); these results are consistent with the literature, particularly the papers by Faurholt-Jepsen et al [29,30] and Dogan et al [4]. According to Cuijpers et al [31], self-report measures and clinical assessment are not equivalent but may provide complementary information.

Figure 3. Dashboard used by the psychiatrist to visualize the patient’s data during the outpatient consultation. The psychiatrist chose which variable to display (eg, antidepressant compliance, mood evaluation), on which statistical graph (eg, pie chart, histogram, box plot), and over which period of time (eg, daily, weekly, monthly). Aug: August; HDRS: Hamilton Depression Rating Scale; Sept: September.
View this figure

By comparison, a feasibility study by Husky et al [32], which was conducted over 3 consecutive days with a preprogrammed personal digital assistant, showed an average compliance of 82% in the group of patients with major depression and 86.7% in the group of patients with any mood disorder. Johnson et al [33] found a 78% compliance rate after 1 week of follow-up in patients with severe psychiatric disorders.

Although a few studies, such as one by Vachon et al [34], have reported a high completion rate in patients with depression after 5 months of follow-up, the lower completion rate in our study may have been a result of the long duration of follow-up. In fact, in our study, patients had to complete the EMAs for 28 days, while in the studies by Husky et al [32] and Johnson et al [33], the follow-up periods were 3 days and 1 week, respectively. In addition, fixed notifications at precise times may be less disruptive than random notifications if they appear at a convenient time to the respondent.

Our results are encouraging, as they tend to confirm our initial hypothesis that data collected by patients using their own smartphone are more reliable than data obtained by a clinician during standard medical follow-up. Additionally, real-time data collection highlights the variability in the symptomatology of depression during the introduction or modification of an antidepressant. The analysis of patterns of emotional variability might lead practitioners to improve their understanding of mood disorders and might help us to identify risk factors for depression relapse.

At the patient level, the results obtained for the various self-assessment criteria—in this case mood, self-esteem, guilt, and sadness—allowed us to obtain, thanks to the large volume of data collected, a reliable representation of the symptoms expressed by the patient during the entire study. The availability of such a user-friendly tool could actually lead to a greater investment by the patient in his/her own care.

Our study also points out the difficulties for some patients in using smartphones. In our data, we observed many inconsistent results, especially for older patients, suggesting that they could not master the slide function on their device. In the future, more emphasis should be placed on assisting patients in the use of smartphones.

Acknowledgments

The authors would like to thank Fondation Pierre Deniker and the start-up Ad Scientiam for their contribution to this study.

Conflicts of Interest

Author TB owns stocks of Adscientiam. The other authors declare no conflicts of interest.

References

  1. Ferreri F, Bourla A, Mouchabac S, Karila L. e-Addictology: An Overview of New Technologies for Assessing and Intervening in Addictive Behaviors. Front Psychiatry 2018;9:51 [FREE Full text] [CrossRef] [Medline]
  2. Schueller S, Aguilera A, Mohr D. Ecological momentary interventions for depression and anxiety. Depress Anxiety 2017 Jun;34(6):540-545. [CrossRef] [Medline]
  3. Van Singer M, Chatton A, Khazaal Y. Quality of Smartphone Apps Related to Panic Disorder. Front Psychiatry 2015;6:96 [FREE Full text] [CrossRef] [Medline]
  4. Dogan E, Sander C, Wagner X, Hegerl U, Kohls E. Smartphone-Based Monitoring of Objective and Subjective Data in Affective Disorders: Where Are We and Where Are We Going? Systematic Review. J Med Internet Res 2017 Jul 24;19(7):e262 [FREE Full text] [CrossRef] [Medline]
  5. Dubad M, Winsper C, Meyer C, Livanou M, Marwaha S. A systematic review of the psychometric properties, usability and clinical impacts of mobile mood-monitoring applications in young people. Psychol Med 2018 Jan;48(2):208-228. [CrossRef] [Medline]
  6. Wenze S, Miller I. Use of ecological momentary assessment in mood disorders research. Clin Psychol Rev 2010 Aug;30(6):794-804. [CrossRef] [Medline]
  7. Armey M, Schatten H, Haradhvala N, Miller I. Ecological momentary assessment (EMA) of depression-related phenomena. Curr Opin Psychol 2015 Aug 01;4:21-25 [FREE Full text] [CrossRef] [Medline]
  8. Frank E, Pong J, Asher Y, Soares CN. Smart phone technologies and ecological momentary data: is this the way forward on depression management and research? Curr Opin Psychiatry 2018 Jan;31(1):3-6. [CrossRef] [Medline]
  9. Ben-Zeev D, Young M, Madsen J. Retrospective recall of affect in clinically depressed individuals and controls. Cognition & Emotion 2009 Aug;23(5):1021-1040 [FREE Full text] [CrossRef]
  10. Hung S, Li MS, Chen YL, Chiang JH, Chen YY, Hung GCL. Smartphone-based ecological momentary assessment for Chinese patients with depression: An exploratory study in Taiwan. Asian J Psychiatr 2016 Oct;23:131-136. [CrossRef] [Medline]
  11. Silk J, Forbes E, Whalen D, Jakubcak J, Thompson W, Ryan N, et al. Daily emotional dynamics in depressed youth: a cell phone ecological momentary assessment study. J Exp Child Psychol 2011 Oct;110(2):241-257 [FREE Full text] [CrossRef] [Medline]
  12. Csikszentmihalyi M, LeFevre J. Optimal experience in work and leisure. J Pers Soc Psychol 1989 May;56(5):815-822. [CrossRef] [Medline]
  13. Barge-Schaapveld D, Nicolson NA, van der Hoop RG, De Vries MW. Changes in daily life experience associated with clinical improvement in depression. J Affect Disord 1995 May 17;34(2):139-154. [CrossRef] [Medline]
  14. Stone A, Shiffman S, Schwartz JE, Broderick JE, Hufford MR. Patient compliance with paper and electronic diaries. Control Clin Trials 2003 Apr;24(2):182-199. [CrossRef] [Medline]
  15. Groot P. Patients can diagnose too: How continuous self-assessment aids diagnosis of, and recovery from, depression. J Ment Health 2010 Aug;19(4):352-362. [CrossRef] [Medline]
  16. Wichers M, Simons C, Kramer I, Hartmann J, Lothmann C, Myin-Germeys I, et al. Momentary assessment technology as a tool to help patients with depression help themselves. Acta Psychiatr Scand 2011 Oct;124(4):262-272. [CrossRef] [Medline]
  17. Zhang M, Ho R. Enabling Psychiatrists to Explore the Full Potential of E-Health. Front Psychiatry 2015;6:177 [FREE Full text] [CrossRef] [Medline]
  18. Seppälä J, De Vita I, Jämsä T, Miettunen J, Isohanni M, Rubinstein K, M-RESIST Group, et al. Mobile Phone and Wearable Sensor-Based mHealth Approaches for Psychiatric Disorders and Symptoms: Systematic Review. JMIR Ment Health 2019 Feb 20;6(2):e9819 [FREE Full text] [CrossRef] [Medline]
  19. Simons CJP, Hartmann JA, Kramer I, Menne-Lothmann C, Höhn P, van Bemmel AL, et al. Effects of momentary self-monitoring on empowerment in a randomized controlled trial in patients with depression. Eur Psychiatry 2015 Nov;30(8):900-906. [CrossRef] [Medline]
  20. Simons C, Hartmann J, Kramer I, Menne-Lothmann C, Höhn P, van Bemmel AL, et al. Effects of momentary self-monitoring on empowerment in a randomized controlled trial in patients with depression. Eur Psychiatry 2015 Nov;30(8):900-906. [CrossRef] [Medline]
  21. Geschwind N, Nicolson N, Peeters F, van Os J, Barge-Schaapveld D, Wichers M. Early improvement in positive rather than negative emotion predicts remission from depression after pharmacotherapy. Eur Neuropsychopharmacol 2011 Mar;21(3):241-247 [FREE Full text] [CrossRef] [Medline]
  22. Kramer I, Simons C, Hartmann J, Menne-Lothmann C, Viechtbauer W, Peeters F, et al. A therapeutic application of the experience sampling method in the treatment of depression: a randomized controlled trial. World Psychiatry 2014 Feb;13(1):68-77 [FREE Full text] [CrossRef] [Medline]
  23. Simons C, Drukker M, Evers S, van Mastrigt GAPG, Höhn P, Kramer I, et al. Economic evaluation of an experience sampling method intervention in depression compared with treatment as usual using data from a randomized controlled trial. BMC Psychiatry 2017 Dec 29;17(1):415 [FREE Full text] [CrossRef] [Medline]
  24. Solhan M, Trull T, Jahng S, Wood P. Clinical assessment of affective instability: comparing EMA indices, questionnaire reports, and retrospective recall. Psychol Assess 2009 Sep;21(3):425-436 [FREE Full text] [CrossRef] [Medline]
  25. Peeters F, Berkhof J, Rottenberg J, Nicolson N. Ambulatory emotional reactivity to negative daily life events predicts remission from major depressive disorder. Behav Res Ther 2010 Aug;48(8):754-760. [CrossRef] [Medline]
  26. Ebner-Priemer U, Trull T. Ecological momentary assessment of mood disorders and mood dysregulation. Psychol Assess 2009 Dec;21(4):463-475. [CrossRef] [Medline]
  27. Myin-Germeys I, Klippel A, Steinhart H, Reininghaus U. Ecological momentary interventions in psychiatry. Curr Opin Psychiatry 2016 Jul;29(4):258-263. [CrossRef] [Medline]
  28. Lee E, Kim J, Shin I, Lim K, Lee S, Cho G, et al. Current use of depression rating scales in mental health setting. Psychiatry Investig 2010 Sep;7(3):170-176 [FREE Full text] [CrossRef] [Medline]
  29. Faurholt-Jepsen M, Vinberg M, Frost M, Debel S, Margrethe Christensen E, Bardram J, et al. Behavioral activities collected through smartphones and the association with illness activity in bipolar disorder. Int J Methods Psychiatr Res 2016 Dec;25(4):309-323 [FREE Full text] [CrossRef] [Medline]
  30. Faurholt-Jepsen M, Frost M, Vinberg M, Christensen EM, Bardram JE, Kessing LV. Smartphone data as objective measures of bipolar disorder symptoms. Psychiatry Res 2014 Jun 30;217(1-2):124-127. [CrossRef] [Medline]
  31. Husky MM, Gindre C, Mazure CM, Brebant C, Nolen-Hoeksema S, Sanacora G, et al. Computerized ambulatory monitoring in mood disorders: feasibility, compliance, and reactivity. Psychiatry Res 2010 Jul 30;178(2):440-442. [CrossRef] [Medline]
  32. Husky M, Gindre C, Mazure C, Brebant C, Nolen-Hoeksema S, Sanacora G, et al. Computerized ambulatory monitoring in mood disorders: feasibility, compliance, and reactivity. Psychiatry Res 2010 Jul 30;178(2):440-442. [CrossRef] [Medline]
  33. Johnson E, Grondin O, Barrault M, Faytout M, Helbig S, Husky M, et al. Computerized ambulatory monitoring in psychiatry: a multi-site collaborative study of acceptability, compliance, and reactivity. Int J Methods Psychiatr Res 2009;18(1):48-57 [FREE Full text] [CrossRef] [Medline]
  34. Vachon H, Viechtbauer W, Rintala A, Myin-Germeys I. Compliance and Retention With the Experience Sampling Method Over the Continuum of Severe Mental Disorders: Meta-Analysis and Recommendations. J Med Internet Res 2019 Dec 06;21(12):e14475 [FREE Full text] [CrossRef] [Medline]


EMA: ecological momentary assessment
HDRS: Hamilton Depression Rating Scale


Edited by G Eysenbach; submitted 28.03.19; peer-reviewed by E Kohls, S Choemprayong, C Simons; comments to author 12.12.19; revised version received 18.04.20; accepted 17.01.21; published 24.02.21

Copyright

©Redwan Maatoug, Nathan Peiffer-Smadja, Guillaume Delval, Térence Brochu, Benjamin Pitrat, Bruno Millet. Originally published in JMIR Formative Research (http://formative.jmir.org), 24.02.2021.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on http://formative.jmir.org, as well as this copyright and license information must be included.