Original Paper
Abstract
Background: Mobile health (mHealth) apps can innovatively diagnose, prevent, and treat many diseases. The increasing use of mHealth apps necessitates an appropriate assessment standard.
Objective: This study aimed to translate the User Version of the Mobile Application Rating Scale (uMARS) into Polish, followed by validation, cultural adaptation, and examination of its reliability and validity.
Methods: The Polish version of uMARS was adapted, translated, and validated based on the free STOP COVID ProteGO Safe app available for Android and iOS platforms. A total of 117 participants rated the app using the translated scale and rerated it 1 week later
Results: The conceptual equivalence of all items and subscales of the original uMARS and its Polish version was confirmed. The translated uMARS scale exhibited high reliability (Cronbach α=0.95). The perceived test-retest reliability and total influence score were acceptable, with intraclass correlation coefficient values of 0.59 and 0.65, respectively
Conclusions: The translated scale matched the reliability of the original scale. It can help users choose the best mHealth apps available in Poland and report their quality. The Polish version of uMARS was cross-culturally validated and was found to be as reliable as the original uMARS. The translated and validated uMARS tool can be used to evaluate mHealth apps in various aspects. App developers can reliably assess app components and determine areas that require further improvement and development to deliver higher-quality apps. The Polish version of the uMARS can become a standard tool for evaluating the quality of mHealth apps in Poland.
doi:10.2196/65067
Keywords
Introduction
Mobile health (mHealth) apps provide innovative approaches [] to health promotion [-], education [-], disease diagnosis [-], prevention [], and management [-]. In 2017, the number of mHealth apps surpassed 325,000 [], with an expected annual growth of approximately 41% from 2015 to 2020. The COVID-19 outbreak further accelerated the adoption of digital technologies [,], such as telemedicine [,-], eHealth [,], and mHealth [,,,], resulting in an exponential increase in mHealth users [,,].
According to the report by the IQVIA Institute for Human Data Science [], app stores had approximately 350,000 health-related apps globally in 2020, with more than 90,000 being mHealth apps. The global mHealth app market size is expected to expand at a compound annual growth rate of 11.6% from 2023 to 2030 [].
In Poland, the number of smartphone users reached 50.6 million in 2021 (132.1 subscriptions per 100 inhabitants), comprising 13.8 and 36.8 million prepaid and postpaid users, respectively []. According to a survey by Deloitte Digital [], in 2022, 75% of the respondents in Poland used mHealth apps, primarily dedicated to fitness, lifestyle, and diet.
Developers, researchers, health care professionals, and web application library managers require appropriate, consistent, and objective tools to assess the quality and safety of mHealth apps. Users frequently choose mHealth apps based on star ratings and short comments in mobile app stores and repositories. Such star ratings may not adequately reflect the quality of the app [,]. Generally, user reviews posted on the app download sites provide biased and unreliable descriptions of the apps. These reviews often fail to specify the nature of the app and the circumstances under which it is used and may not cover all helpful features for an accurate evaluation. The traditional star-rating system lacks clarity and is ambiguous in its interpretation. A reliable rating system can help clinicians and patients identify valuable mHealth apps [-], streamlining the overwhelming range of available apps to a select few that merit consideration in clinical settings [,,,,,-]. Many studies have examined the mainstream market status of mHealth apps [,,] by evaluating functional distribution, without refined classifications of mHealth apps and information integrity []. In addition, the COVID-19 pandemic complicated the development of many mHealth apps, requiring close attention []. The User Version of the Mobile Application Rating Scale (uMARS) targeted this gap by providing an empirical basis for objectively assessing the critical quality criteria and offering users, practitioners, and researchers a rigorous measure of app quality []. It helps select and recommend apps [], owing to growing interest in using objective and sustainable user ratings for mHealth apps, and clinicians and patients can use a measurable, repeatable, clustered evaluation system to ensure that a particular mHealth app can meet their specific needs effectively.
The uMARS provides an empirical basis for the objective evaluation of critical quality criteria. It offers a rigorous metric for evaluating app quality [] and increasing user confidence [,]. With the increasing interest in objective and sustainable user ratings, using tools such as uMARS is well justified. [].
The considerable international interest garnered by the original uMARS (English) [] is evident from its translated and validated versions in Italian [], Japanese [], Greek [], Portuguese (Brazil) [], Spanish [], and Turkish []. Many researchers have highlighted the significance of uMARS as an essential instrument in the evaluation of mHealth apps, with its international use growing continuously [,,-,-]. Although alternative scales [,] have been developed in response to the limitations of uMARS [,], the uMARS unquestionably fulfills its task and remains the most widespread tool for evaluating mobile apps worldwide [,,,-,-].
Users’ interest and demand for tools, such as uMARS, are also growing in Poland owing to the increasing number of mHealth apps in the country [,]. Consequently, the need for prioritizing individual apps when multiple apps may serve the same purpose has arisen [].
A translated and validated version of the uMARS was unavailable in Polish despite the increasing number of Polish mHealth apps. Therefore, this study aimed to translate and culturally adapt the uMARS into Polish and test its reliability and validity. The strengths of this study lie in the standardized methods used in all procedures and the rigorous and reliable translation process.
Methods
Study Design and Participants
Validation Procedure, Participants, and Protocol
We followed the original methodology outlined by Stoyanov et al [] to validate the final Polish version of uMARS. We assessed the internal consistency and test-retest reliability of the scale.
This cross-sectional methodological study followed STROBE (Strengthening the Reporting of Observational studies in Epidemiology) and COSMIN (Consensus-Based Standards for the Selection of Health Measurement Instruments) guidelines for the validation of patient-reported outcome measures.
The recruitment process focused on the general adult population, and participants who provided written informed consent were considered. The authors recruited volunteers from Polish universities. The inclusion criteria for participants were as follows: individuals aged 18 years or older, individuals who owned and confidently used a smartphone, individuals who were motivated to participate in the study and were fluent in Polish. Participants unable to comprehend or respond to the questionnaires, unable to understand Polish, or unavailable during the study period were excluded.
Procedure and Data Collection
Questionnaire
The uMARS is a widely recognized tool designed to assess the quality and usability of mobile health apps from the perspective of end users. This study used uMARS to evaluate user perceptions and experiences with a mobile app. The original uMARS comprises 4 objective domains—engagement, functionality, aesthetics, and information—and 1 subjective quality subscale []. This structured questionnaire provides valuable insights into an app’s strengths and areas for improvement, ensuring that it meets user expectations and effectively supports the intended health outcomes. By offering a nuanced understanding of user interaction and perception, uMARS contributes significantly to the app’s effectiveness and adoption. Notably, uMARS is used worldwide and does not require special training, making it accessible and practical for diverse research settings.
The uMARS ratings of participants who responded twice per the study protocol were included in the analysis. Singular ratings were excluded from further analysis. The participants evaluated the selected apps. The decision to include only uMARS ratings from participants who responded twice while excluding single ratings was driven by the need to ensure reliability and validity in assessing mobile health apps. The methodology was crucial for ensuring reliability and reducing variability. By requiring participants to rate the app twice, we aimed to increase the consistency and trustworthiness of the collected data. Consistent ratings across the 2 time points helped confirm that user ratings were stable and not influenced by transient factors, such as mood or immediate environment. Excluding single ratings helped reduce the variability resulting from 1-time impressions and ensured that the data reflected a more stable and accurate user experience with the app. The STOP COVID ProteGO Safe app was rated most frequently. Finally, the research team agreed to validate the STOP COVID ProteGO Safe app []. The selected app parameters were timeliness, necessity, cost-effectiveness, practicality, and compatibility with Android and iOS platforms.
Considering the assumptions described by the authors of the original method [], participants received task instructions in the Polish language. They familiarized themselves with the app for at least 10 minutes before providing the ratings using uMARS. This procedure was repeated twice to assess intrarater reproducibility. The sensitive data of the participants were well protected.
Adaptation and Translation
The process involved several stages: preparation, double translations from the original version, reconciliation, back translation, review and harmonization, and final proofreading [,-]. Internationally accepted guidelines and checklists [-] were used to ensure thorough translation and validation processes [,,]. A bilingual translator professionally translated the English version of uMARS into Polish. The translated output was compared with a validated expert scale. After discussions, minor adjustments were made. Back translation was performed to ensure compatibility and accuracy of meaning between the source and target languages and was reviewed for consistency with the original [,]. Finally, the Polish uMARS questionnaire was approved.
Validation of the Polish Version of the uMARS
The Polish version of the uMARS questionnaire was validated following the recommendations described by Stoyanov et al []. Cronbach α coefficient, which estimates questionnaire homogeneity, was used to measure internal consistency [,]. A high internal consistency coefficient was indicated by a value greater than 0.70. Participants completed the translated uMARS questionnaire online and then completed it again after 1 week. The intraclass correlation coefficient (ICC) was calculated to assess the reliability of the translated scale.
App Selection
This study was conducted from February 2022 to March 2022, and the STOP COVID ProteGO Safe app was used in this study []. The app was freely accessible to Android and iOS users aged 17 years or older.
Similar COVID-19 mHealth apps were available globally [,]. The STOP COVID ProteGO Safe app’s Polish version was considered due to its universality and timeliness. The STOP COVID ProteGO Safe app became available in March 2020 [] and remained active until March 31, 2022. The app received a rating of 4.6 stars for the iOS version (7600 user ratings) and 4.2 stars for the Android version (22,000 ratings from more than 1 million downloads). The official Polish app was developed in collaboration with the Chief Sanitary Inspectorate, which aided the fight against COVID-19. The developer certified the app’s safety, anonymity, data security, and privacy. The app did not collect or share user data with third parties, track the user’s location, or access any files or information on the user’s phone. After the test, the app assessed the user’s most likely COVID-19 risk group based on the guidelines of the World Health Organization and the Chief Sanitary Inspectorate.
Statistical Analysis
Statistical tests were conducted considering the satisfaction of the assumptions of these tests, including examining whether the distributions of the studied variables conformed to normality. The Shapiro-Wilk W test was used to assess the normality of variable distributions. Normal distribution of quantitative variables entailed calculations of mean differences (95% CIs with 1–α=.95 confidence level). External agreement between the 2 measurements was tested for each parameter at P<.05. The correlation between each domain and the uMARS total score was assessed using the Pearson correlation coefficient, where a value greater than 0.7 (significance level below 0.05) was considered sufficient to evaluate test-retest reliability. Finally, the ICC between the test and retest was calculated to determine weighted rater agreement values and evaluate the closeness rather than equality of scores.
The reliability of the uMARS questionnaire was assessed via the Student 2-tailed t test for dependent variables. The analysis used the final test results and individual categories (results of individual scale parts). Reliability was calculated using Pearson linear correlation and ICC of each variable. Cronbach α (0.95% CI) was calculated to assess internal consistency and reliability. The Cronbach α values were interpreted as excellent (≥0.90), good (0.80-0.89), acceptable (0.70-0.79), doubtful (0.60-0.69), weak (0.50-0.59), and unacceptable (<0.50). Furthermore, the results were supplemented with descriptive statistics (means and SDs).
All analyses were performed using TIBCO Software Inc (version 2017) and Statistica (version 13; Data Analysis Software System) []. The results were considered statistically significant at the significance level (α) of .05 if the critical level P value was less than .05.
Ethical Considerations
This study was approved by the Bioethical Commission (AKBE/60/2022). The study was conducted in accordance with the principles of scientific ethics and the Declaration of Helsinki. The Bioethical Commission approved the study protocol on February 21, 2022, and confirmed that the study complies with ethical standards for scientific research. All participants were adults and took part in the study voluntarily. Before participating in the study, participants received information about the purpose and procedures of the study and gave their informed consent in electronic form. Participants were informed that participation in the study was voluntary and that they could withdraw at any time without any consequences. No personally identifiable data were collected. All data were collected and analyzed anonymously. Data storage and processing complied with applicable data protection regulations, and access to the data was restricted to the research team. Participants did not receive any financial or other compensation for their participation in the study.
Results
Overview
This study enrolled 117 participants who fulfilled the inclusion criteria (women: n=81, 69.2%; men: n=36, 30.8%; mean age 20, SD 1.66 years). Among them, 90 (76.9%) participants were associated (academically or professionally) in the medical field; 19 (16.2%) were in biomedical engineering, 4 (3.4%) were in nursing, and 4 (3.4%) were in dentistry. Most respondents were undergraduate and postgraduate students enrolled at Cardinal Stefan Wyszyński University in Warsaw (n=39, 33.3%), the Medical University of Warsaw (n=60, 51.3%), and the Silesian University of Technology (n=18, 15.4%). A total of 71 (60.7%) respondents reported daily use of mobile apps, and 59 (50.4%) respondents used Apple, iOS, or other Android devices.
All respondents performed the initial app assessment twice with a 1-week gap using uMARS. The questionnaire was administered twice with a 1-week break to assess the repeatability of the Polish uMARS questionnaire. First, a paired t test was conducted to observe the differences between the 2 measurements for each subscore and total score. Test-retest reliability was assessed for each variable using the Pearson 2D correlation coefficient.
The participants rated the STOP COVID ProteGO Safe app twice. Every participant received clear and concise information regarding the study objectives, rules of participation, and procedures.
Translation Process
The translation process involved several stages: preparation, double translations from the original version, reconciliation, back translation, review and harmonization, and final proofreading [,]. Internationally accepted guidelines and checklists [-] were used to ensure thorough translation and validation processes [,]. Two native Polish-speaking translators independently translated the original English survey into Polish. A consensus meeting agreed upon a standard version as a unified translation. Similarly, 2 independent back translations into English were conducted anonymously using the Polish version of the uMARS. The interdisciplinary group reviewed and completed the questionnaires. No significant difficulties were encountered during the translation process, and a consensus was reached regarding minor discrepancies that reflected the cultural context of the Polish language or semantic issues. The final consensus on the Polish version of the uMARS was achieved after a group of pretesters tested the questionnaire. They confirmed its comprehensibility and transcultural adaptation. A translation history review and examination of the original questionnaire confirmed that all items in the original uMARS were flawlessly translated without major substantive problems. Consequently, the final version was presented to respondents unfamiliar with uMARS, seeking their input on the translated words, understandability, interpretability, and cultural relevance. A ready-to-use version of the Polish uMARS was made available after a final review.
Test-Retest Reliability
Participants completed the questionnaire twice, providing responses for an initial assessment (test) and a reassessment after a week (retest), considering the values obtained by pairs of the same respondents (P>.05; ).
A paired t test was performed to compare the results of the test and retest to assess the reliability of the test and retest of the Polish version of uMARS.
The external consistency of the 2 measurements was statistically significant for each measured parameter (P<.05). The highest agreement was in category F (perceived impact; r=0.73; P<.001). The concordance for the total scale score was moderate (r=0.40; P<.001); however, concordance was not observed for category C12 (visual appeal; P=.18; ).
| Domain and subdomain | Paired t test | ||||||
| Test, mean (SD) | Retest, mean (SD) | P value | |||||
| A (engagement) | |||||||
| A1 (entertainment) | 3.10 (1.16) | 3.19 (1.16) | .20 | ||||
| A2 (interest) | 3.61 (0.94) | 3.68 (1.01) | .49 | ||||
| A3 (customization) | 3.37 (0.87) | 3.48 (1.08) | .37 | ||||
| A4 (interactivity) | 3.45 (0.92) | 3.53 (0.93) | .53 | ||||
| A5 (target group) | 4.09 (0.80) | 4.15 (0.88) | .52 | ||||
| A (engagement total) | 17.02 (3.86) | 17.29 (4.51) | .58 | ||||
| B (functionality) | |||||||
| B6 (performance) | 4.11 (0.80) | 3.99 (0.81) | .11 | ||||
| B7 (ease of use) | 3.95 (0.85) | 3.98 (0.84) | .92 | ||||
| B8 (navigation) | 3.87 (0.81) | 3.90 (0.93) | .91 | ||||
| B9 (gestural design) | 3.88 (0.78) | 3.97 (0.88) | .55 | ||||
| B (functionality total) | 14.90 (3.95) | 15.09 (3.87) | .81 | ||||
| C (aesthetics) | |||||||
| C10 (layout) | 4.04 (0.80) | 3.92 (0.97) | .17 | ||||
| C11 (graphics) | 3.96 (0.88) | 4.02 (0.84) | .70 | ||||
| C12 (visual appeal) | 3.83 (0.82) | 3.78 (0.79) | .54 | ||||
| C (aesthetics total) | 11.63 (2.67) | 11.65 (2.24) | .97 | ||||
| D (information) | |||||||
| D13 (quality of information) | 3.92 (0.80) | 4.02 (0.77) | .39 | ||||
| D14 (quantity of information) | 3.76 (0.76) | 3.78 (0.88) | .91 | ||||
| D15 (visual information) | 3.95 (0.82) | 3.92 (0.84) | .48 | ||||
| D16 (credibility of source) | 4.11 (0.79) | 4.21 (0.71) | .45 | ||||
| D (information total) | 14.51 (3.92) | 15.02 (3.76) | .12 | ||||
| Quality | |||||||
| (A+B+C+D)/4 | 14.47 (2.90) | 14.75 (2.98) | .43 | ||||
| E (subjective quality) | |||||||
| E17 (recommendation to others) | 3.48 (1.14) | 3.49 (1.01) | .75 | ||||
| E18 (use and relevance) | 3.30 (1.30) | 3.41 (1.18) | .48 | ||||
| E19 (payment) | 2.46 (1.23) | 2.54 (1.22) | .54 | ||||
| E20 (overall rating) | 3.62 (0.79) | 3.64 (0.87) | ˃.99 | ||||
| E (subjective quality total) | 12.56 (3.88) | 12.91 (3.59) | .49 | ||||
| F (perceived impact) | |||||||
| F1 (awareness) | 2.72 (1.35) | 2.87 (1.23) | .21 | ||||
| F2 (knowledge) | 2.82 (1.34) | 2.81 (1.13) | .85 | ||||
| F3 (attitudes) | 2.80 (1.36) | 2.61 (1.20) | .07 | ||||
| F4 (intention to change) | 2.88 (1.32) | 2.87 (1.27) | .85 | ||||
| F5 (help seeking) | 2.92 (1.38) | 3.02 (1.26) | .40 | ||||
| F6 (behavior change) | 2.93 (1.32) | 2.94 (1.30) | .60 | ||||
| F (perceived impact total) | 16.71 (7.22) | 16.51 (5.91) | .39 | ||||
| Total score | 85.66 (21.87) | 88.05 (18.58) | .29 | ||||
| Domain and subdomain | Reliability | ||||
| Pearson correlation | P value | ||||
| A (engagement) | |||||
| A1 (entertainment) | 0.52 | <.001 | |||
| A2 (interest) | 0.41 | <.001 | |||
| A3 (customization) | 0.33 | <.001 | |||
| A4 (interactivity) | 0.20 | .041 | |||
| A5 (target group) | 0.27 | .004 | |||
| A (engagement total) | 0.37 | <.001 | |||
| B (functionality) | |||||
| B6 (performance) | 0.31 | .001 | |||
| B7 (ease of use) | 0.41 | <.001 | |||
| B8 (navigation) | 0.49 | <.001 | |||
| B9 (gestural design) | 0.33 | <.001 | |||
| B (functionality total) | 0.39 | <.001 | |||
| C (aesthetics) | |||||
| C10 (layout) | 0.44 | <.001 | |||
| C11 (graphics) | 0.35 | <.001 | |||
| C12 (visual appeal) | 0.13a | .166a | |||
| C (aesthetics total) | 0.29 | .001 | |||
| D (information) | |||||
| D13 (quality of information) | 0.35 | <.001 | |||
| D14 (quantity of information) | 0.45 | <.001 | |||
| D15 (visual information) | 0.27 | .009 | |||
| D16 (credibility of source) | 0.24 | .02 | |||
| D (information total) | 0.52 | <.001 | |||
| Quality | |||||
| (A+B+C+D)/4 | 0.39 | <.001 | |||
| E (subjective quality) | |||||
| E17 (recommendation to others) | 0.35 | <.001 | |||
| E18 (use and relevance) | 0.53 | <.001 | |||
| E19 (payment) | 0.49 | <.001 | |||
| E20 (overall rating) | 0.30 | .001 | |||
| E (subjective quality total) | 0.37 | <.001 | |||
| F (perceived impact) | |||||
| F1 (awareness) | 0.61 | <.001 | |||
| F2 (knowledge) | 0.64 | <.001 | |||
| F3 (attitudes) | 0.76 | <.001 | |||
| F4 (intention to change) | 0.67 | <.001 | |||
| F5 (help seeking) | 0.67 | <.001 | |||
| F6 (behavior change) | 0.63 | <.001 | |||
| F (perceived impact total) | 0.73 | <.001 | |||
| Total score | 0.40 | <.001 | |||
aStatistically significant difference between test and retest scores (P<.001).
The reliability analysis of Cronbach α confirmed the very high reliability of the scale at 0.95. No relevant factor lowered or increased the reliability of the entire tool ().
The ICC values were overall moderate. The highest ICC values were for the category perceived impact (ICC=0.59 for 0.29-0.53). The intraclass correlation coefficient for the total uMARS score was moderate (ICC=0.65), indicating moderate temporal stability. This level of agreement is consistent with comparable validation studies of user-rated app evaluation tools, where subjective perceptions of design and engagement may vary naturally between assessments.
presents a scatterplot depicting final test values (total test vs complete retest). The scatterplot shows the relationship between the overall scale scores on the test and retest. The results were positively correlated, as evidenced by the upward slope of the solid red regression line.
Participants who gave higher ratings on the test tended to give higher ratings on the retest, whereas those who gave lower ratings on the test tended to give lower ratings on the retest. The dots running in an even pattern along the regression line illustrate this observation. The dashed lines indicate CIs of the mean scores.
shows the percentage distribution of individual ratings (1-5 points) provided by the respondents in each category. The left and right sides represent the test and retest results, respectively. Different colors denote each rating. Matching colors indicate specific assessments in both measurements with similar frequencies. The test results mirrored the retest results, thereby forming a mirror image pattern.
The overall ICC values and those for the entire scale were moderate (ICC=0.65). The highest ICC values were for the category perceived impact (ICC=0.59 for 0.29-0.53).
| Domain and subdomain | Cronbach α | ||
| A (engagement) | |||
| A1 (entertainment) | 0.95 | ||
| A2 (interest) | 0.95 | ||
| A3 (customization) | 0.95 | ||
| A4 (interactivity) | 0.95 | ||
| A5 (target group) | 0.95 | ||
| A (engagement total) | —a | ||
| B (functionality) | |||
| B6 (performance) | 0.95 | ||
| B7 (ease of use) | 0.95 | ||
| B8 (navigation) | 0.95 | ||
| B9 (gestural design) | 0.95 | ||
| B (functionality total) | — | ||
| C (aesthetics) | |||
| C10 (layout) | 0.95 | ||
| C11 (graphics) | 0.95 | ||
| C12 (visual appeal) | 0.95 | ||
| C (aesthetics total) | — | ||
| D (information) | |||
| D13 (quality of information) | 0.95 | ||
| D14 (quantity of information) | 0.95 | ||
| D15 (visual information) | 0.95 | ||
| D16 (credibility of source) | 0.95 | ||
| D (information total) | — | ||
| Quality | |||
| (A+B+C+D)/4 | — | ||
| E (subjective quality) | |||
| E17 (recommendation to others) | 0.95 | ||
| E18 (use and relevance) | 0.95 | ||
| E19 (payment) | 0.95 | ||
| E20 (overall rating) | 0.95 | ||
| E (subjective quality total) | — | ||
| F (perceived impact) | |||
| F1 (awareness) | 0.95 | ||
| F2 (knowledge) | 0.95 | ||
| F3 (attitudes) | 0.95 | ||
| F4 (intention to change) | 0.95 | ||
| F5 (help seeking) | 0.95 | ||
| F6 (behavior change) | 0.95 | ||
| F (perceived impact total) | — | ||
| Total score | 0.95 | ||
aNot available.


Discussion
Principal Findings
The Polish version of uMARS exhibited excellent internal consistency and good short-term (weekly) test-retest reliability throughout the validation process. The subscales demonstrated either excellent or good reliability in all domains. This version enabled Polish users to obtain reliable measurements of mHealth app quality. The app evaluation system integrated evidence, user experience, and content values into a comprehensive evaluation system. The partial and total results of the assessed domains were reliable and correctly evaluated using the questionnaire.
This study used a single mobile app (STOP COVID ProteGO Safe) as a standardized reference for validation, ensuring linguistic consistency and minimizing contextual bias in participant responses. However, the use of a single app did not permit the assessment of factorial or convergent validity. This limitation aligns with early-phase validation studies of the original uMARS and its adaptations in other languages. Future research should incorporate multiple app categories to confirm construct validity and investigate potential subgroup differences.
The overall internal consistency of the Polish uMARS was high (Cronbach α=0.95). While this supports strong item homogeneity, uniformly high α values across subscales should be interpreted with caution, as they may reflect item redundancy within conceptually overlapping domains. Similar Cronbach α magnitudes have been observed in other uMARS validations and in multi-item satisfaction scales, suggesting that high interitem correlations are typical for this type of measure. Nevertheless, this potential redundancy represents a limitation inherent to user perception instruments. The Polish uMARS questionnaire responds to the growing need for standardization in the evaluation of health-related mobile apps, informed by user feedback. It offers a structured approach to assessing key quality aspects of apps in Polish, benefiting both users and developers.
The app in question is no longer available, as it was designed specifically for use during the COVID-19 pandemic and was withdrawn after it ended. This circumstance makes it impossible to repeat the study using this app, as the app was used only during the COVID-19 pandemic. However, this study aimed to verify the Polish language version of the app. Despite the limited duration of the app’s operation, our goals were successfully achieved, as the aim of this study was to validate the Polish version of the scale.
This validation was conducted on a sample of young university students pursuing health-related education. Although this group was appropriate for the initial phase of linguistic and psychometric validation, it may not fully represent older or less experienced users of digital technology. However, the study group reflects the demographic most engaged in the use of health-related mobile apps. Therefore, focusing on this group was appropriate for initial linguistic and psychometric validation, ensuring that all participants were experienced smartphone users capable of providing reliable app ratings. Future research should evaluate the performance of the Polish version of uMARS among broader and more diverse populations. Validation efforts need to include participants from a wider range of age groups and varying levels of digital literacy to ensure the Polish uMARS’s broader applicability.
Conclusions
Cross-cultural validation studies of the Polish version of uMARS were conducted, showing its reliability comparable to that of the original version. The Polish version of uMARS can be a valuable tool for assessing the quality of mHealth apps in Poland. From a user perspective, the Polish version of uMARS demonstrates appropriate metric properties for evaluating the quality of mHealth apps, providing comprehensive access to mHealth app ratings from different user perspectives.
Polish researchers can use the translated and verified Polish tool to collect end-user feedback and ratings, helping to identify highly rated apps. In addition, app developers can benefit from reliable app component ratings and gain valuable insights for further improvement and development, increasing the overall quality and impact of mHealth apps. Developers and app stores can present more comprehensive and documented evaluation results along with a star rating to provide users with better insight into the app’s quality.
Acknowledgments
The authors would like to thank Editage for English language editing and reviewing this manuscript.
Funding
No funding was received for this research.
Data Availability
The datasets generated and analyzed during this study are not publicly available due to ethical and privacy considerations but are available from the corresponding author on reasonable request.
Authors' Contributions
Conceptualization: WMG, TC, BG, SS
Data curation: WMG, TC, JN
Formal analysis: WMG, TC, JN, SS
Investigation: WMG, TC, JN, RD, BM, BG
Methodology: WMG, JN, BG, SS
Project administration: WMG, TC, BG
Resources: WMG, TC, BG
Supervision: WMG, SS
Validation: WMG, TC, JN, RD, BM, BG, SS
Writing—original draft: WMG, JN, BG
Writing—review and editing: WMG, TC, JN, RD, BM, BG, SS
Conflicts of Interest
None declared.
References
- Lee G, Raghu TS. Determinants of mobile apps' success: evidence from the app store market. J Manag Inf Syst. Dec 07, 2014;31(2):133-170. [FREE Full text] [CrossRef]
- Lee M, Lee H, Kim Y, Kim J, Cho M, Jang J, et al. Mobile app-based health promotion programs: a systematic review of the literature. Int J Environ Res Public Health. Dec 13, 2018;15(12):2838. [FREE Full text] [CrossRef] [Medline]
- Mateo-Orcajada A, Vaquero-Cristóbal R, Mota J, Abenza-Cano L. Physical activity, body composition, and fitness variables in adolescents after periods of mandatory, promoted or nonmandatory, nonpromoted use of step tracker mobile apps: randomized controlled trial. JMIR Mhealth Uhealth. Jul 30, 2024;12:e51206. [FREE Full text] [CrossRef] [Medline]
- Stark AL, Geukes C, Dockweiler C. Digital health promotion and prevention in settings: scoping review. J Med Internet Res. Jan 28, 2022;24(1):e21063. [FREE Full text] [CrossRef] [Medline]
- Johnson SG, Potrebny T, Larun L, Ciliska D, Olsen NR. Usability methods and attributes reported in usability studies of mobile apps for health care education: scoping review. JMIR Med Educ. Jun 29, 2022;8(2):e38259. [FREE Full text] [CrossRef] [Medline]
- Medina Aguerrebere P, Medina E, Gonzalez Pacanowski T. Promoting health education through mobile apps: a quantitative analysis of American hospitals. Healthcare (Basel). Nov 08, 2022;10(11):2231. [FREE Full text] [CrossRef] [Medline]
- Rusatira JC, Tomaszewski B, Dusabejambo V, Ndayiragije V, Gonsalves S, Sawant A, et al. Enabling access to medical and health education in Rwanda using mobile technology: needs assessment for the development of mobile medical educator apps. JMIR Med Educ. Jun 01, 2016;2(1):e7. [FREE Full text] [CrossRef] [Medline]
- Costa WD, de Oliveira AM, Aguilar GJ, Dos Santos LM, Dos Santos LR, Donato DD, et al. A review of software and mobile apps to support the clinical diagnosis of hansen disease. JMIR Dermatol. Aug 18, 2023;6:e47142. [FREE Full text] [CrossRef] [Medline]
- Gheisari M, Ghaderzadeh M, Li H, Taami T, Fernández-Campusano C, Sadeghsalehi H, et al. Mobile apps for COVID-19 detection and diagnosis for future pandemic control: multidimensional systematic review. JMIR Mhealth Uhealth. Feb 22, 2024;12:e44406. [FREE Full text] [CrossRef] [Medline]
- Lee C, Witkowski A, Żychowska M, Ludzik J. The role of mobile teledermoscopy in skin cancer triage and management during the COVID-19 pandemic. Indian J Dermatol Venereol Leprol. Dec 08, 2022;89:347-352. [CrossRef]
- Palozzi G, Antonucci G. Mobile-Health based physical activities co-production policies towards cardiovascular diseases prevention: findings from a mixed-method systematic review. BMC Health Serv Res. Mar 01, 2022;22(1):277. [FREE Full text] [CrossRef] [Medline]
- Rathbone AL, Prescott J. The use of mobile apps and SMS messaging as physical and mental health interventions: systematic review. J Med Internet Res. Aug 24, 2017;19(8):e295. [FREE Full text] [CrossRef] [Medline]
- Wang Y, Min J, Khuri J, Xue H, Xie B, A Kaminsky L, et al. Effectiveness of mobile health interventions on diabetes and obesity treatment and management: systematic review of systematic reviews. JMIR Mhealth Uhealth. Apr 28, 2020;8(4):e15400. [FREE Full text] [CrossRef] [Medline]
- Wen Y, Ruan Y, Yu Y. Mobile health management among end stage renal disease patients: a scoping review. Front Med (Lausanne). Jul 11, 2024;11:1366362. [FREE Full text] [CrossRef] [Medline]
- Cozad MJ, Crum M, Tyson H, Fleming PR, Stratton J, Kennedy AB, et al. Mobile health apps for patient-centered care: review of united states rheumatoid arthritis apps for engagement and activation. JMIR Mhealth Uhealth. Dec 05, 2022;10(12):e39881. [FREE Full text] [CrossRef] [Medline]
- The mHealth apps market is getting crowded. Research 2 Guidance. Berlin. Research2Guidance The Digital Health Market Analyst And Strategy Consulting Company Business Consulting and Services Berlin; 2017. URL: https://research2guidance.com/mhealth-app-market-getting-crowded-259000-mhealth-apps-now/ [accessed 2025-05-29]
- Martin-Payo R, Carrasco-Santos S, Cuesta M, Stoyan S, Gonzalez-Mendez X, Fernandez-Alvarez MD. Spanish adaptation and validation of the user version of the Mobile Application Rating Scale (uMARS). J Am Med Inform Assoc. Nov 25, 2021;28(12):2681-2686. [FREE Full text] [CrossRef] [Medline]
- Holl F, Flemisch F, Swoboda W, Schobel J. Apps for COVID-19 in Germany: assessment using the German Mobile App Rating Scale. JAMIA Open. Dec 2022;5(4):ooac082. [FREE Full text] [CrossRef] [Medline]
- Furlepa K, Śliwczyński A, Kamecka K, Kozłowski R, Gołębiak I, Cichońska-Rzeźnicka D, et al. The COVID-19 pandemic as an impulse for the development of telemedicine in primary care in Poland. J Pers Med. Jul 18, 2022;12(7):1165. [FREE Full text] [CrossRef] [Medline]
- de Batlle J, Benítez ID, Moncusí-Moix A, Androutsos O, Angles Barbastro R, Antonini A, et al. GATEKEEPER project. GATEKEEPER's strategy for the multinational large-scale piloting of an eHealth platform: tutorial on how to identify relevant settings and use cases. J Med Internet Res. Jun 28, 2023;25:e42187. [FREE Full text] [CrossRef] [Medline]
- Więckowska-Rusek K, Danel J, Deja G. The usefulness of the nutrition apps in self-control of diabetes mellitus - the review of literature and own experience. Pediatr Endocrinol Diabetes Metab. 2022;28(1):75-80. [FREE Full text] [CrossRef] [Medline]
- Ochnik D, Cholewa-Wiktor M, Jakubiak M, Pataj M. eHealth tools use and mental health: a cross-sectional network analysis in a representative sample. Sci Rep. Mar 02, 2024;14(1):5173. [FREE Full text] [CrossRef] [Medline]
- Płaciszewski K, Wierzba W, Ostrowski J, Pinkas J, Jankowski M. Factors associated with the use of public eHealth services in Poland - a 2022 nationwide cross-sectional survey. Ann Agric Environ Med. Mar 31, 2023;30(1):127-134. [CrossRef] [Medline]
- Gawałko M, Duncker D, Manninger M, van der Velden RM, Hermans AN, Verhaert DV, et al. TeleCheck-AF investigators. The European TeleCheck-AF project on remote app-based management of atrial fibrillation during the COVID-19 pandemic: centre and patient experiences. Europace. Jul 18, 2021;23(7):1003-1015. [FREE Full text] [CrossRef] [Medline]
- Liew MS, Zhang J, See J, Ong YL. Usability challenges for health and wellness mobile apps: mixed-methods study among mHealth experts and consumers. JMIR Mhealth Uhealth. Jan 30, 2019;7(1):e12160. [FREE Full text] [CrossRef] [Medline]
- Vaidya A. Key features of mHealth apps and trends in use. Xtelligent Healthcare Media. Newton, Massachusetts (Headquarters) 275 Grove Street, Newton, MA 02466. The Informa TechTarget URL: https://www.techtarget.com/virtualhealthcare/feature/Key-Features-of-mHealth-Apps-Trends-in-Use [accessed 2025-05-29]
- Aitken M, Nass D. Digital health trends 2021 innovation, evidence, regulation, and adoption. the IQVIA Institute for Human Data Science. 100 IMS Drive Parsippany, NJ 07054 United States. the IQVIA Institute for Human Data Science; Jul 22, 2021. URL: https://www.iqvia.com/insights/the-iqvia-institute/reports-and-publications/reports/digital-health-trends-2021 [accessed 2025-05-29]
- Guzowski W, Juszczak K, Mordan P. Telecommunications in 2021. The Statistical Council. 00-925 Warsaw, Aleja Niepodległości 208. Statistics Poland; 2022. URL: https://stat.gov.pl/files/gfx/portalinformacyjny/en/defaultaktualnosci/3324/2/6/1/telecommunications_in_2021.pdf [accessed 2025-05-29]
- Pacjent XXI wieku szanse, wyzwania i możliwości. Deloitte Digital. Deloitte Poland Foundation al. Jana Pawła II 22, 00-133, Warsaw, Poland. Deloitte Consulting LLP, a subsidiary of Deloitte LLP; Jun 15, 2023. URL: https://raportorium.pl/wp-content/uploads/2023/08/pl_patient-of-the-future-2023.pdf [accessed 2025-05-29]
- Lee J, Lee MA. Validation and usability study of the framework for a user needs-centered mHealth app selection. Int J Med Inform. Nov 2022;167:104877. [FREE Full text] [CrossRef] [Medline]
- Peng W, Kanthawala S, Yuan S, Hussain SA. A qualitative study of user perceptions of mobile health apps. BMC Public Health. Nov 14, 2016;16(1):1158. [FREE Full text] [CrossRef] [Medline]
- Stoyanov SR, Hides L, Kavanagh DJ, Wilson H. Development and validation of the user version of the Mobile Application Rating Scale (uMARS). JMIR Mhealth Uhealth. Jun 10, 2016;4(2):e72. [FREE Full text] [CrossRef] [Medline]
- Peek J, Hay K, Hughes P, Kostellar A, Kumar S, Bhikoo Z, et al. Feasibility and acceptability of a smoking cessation smartphone app (My QuitBuddy) in older persons: pilot randomized controlled trial. JMIR Form Res. Apr 14, 2021;5(4):e24976. [FREE Full text] [CrossRef] [Medline]
- Nogueira FM, Martins RP, Nery EC, Silva AG. A systematic search and assessment of the quality and characterisation of free mobile applications targeting knee pain. Musculoskeletal Care. Mar 03, 2023;21(1):212-220. [CrossRef] [Medline]
- Welch WA, Solk P, Auster-Gussman L, Gavin KL, Whitaker M, Cullather E, et al. User-centered development of a smartphone application (Fit2Thrive) to promote physical activity in breast cancer survivors. Transl Behav Med. Feb 16, 2022;12(2):203-213. [FREE Full text] [CrossRef] [Medline]
- Meyer CH, Stanzel BV, Moqaddem S, Brohlburg D. [Healthcare "apps" for smartphones: relief or toy for patients and the visual impaired?]. Ophthalmologe. Jan 26, 2012;109(1):21-29. [FREE Full text] [CrossRef] [Medline]
- Fiordelli M, Diviani N, Schulz PJ. Mapping mHealth research: a decade of evolution. J Med Internet Res. May 21, 2013;15(5):e95. [FREE Full text] [CrossRef] [Medline]
- Klonoff DC. The current status of mHealth for diabetes: will it be the next big thing? J Diabetes Sci Technol. May 01, 2013;7(3):749-758. [FREE Full text] [CrossRef] [Medline]
- Martínez-Pérez B, de la Torre-Díez I, López-Coronado M, Sainz-de-Abajo B, Robles M, García-Gómez JM. Mobile clinical decision support systems and applications: a literature and commercial review. J Med Syst. Jan 8, 2014;38(1):4. [FREE Full text] [CrossRef] [Medline]
- Armstrong KA, Semple JL, Coyte PC. Replacing ambulatory surgical follow-up visits with mobile app home monitoring: modeling cost-effective scenarios. J Med Internet Res. Sep 22, 2014;16(9):e213. [FREE Full text] [CrossRef] [Medline]
- Chow CK, Ariyarathna N, Islam SM, Thiagalingam A, Redfern J. mHealth in Cardiovascular Health Care. Heart Lung Circ. Aug 2016;25(8):802-807. [FREE Full text] [CrossRef] [Medline]
- mHealth apps market size, share and trends analysis report by type (medical apps, fitness apps), platform (Android, IOS), region (Asia-Pacific, Middle East and Africa) with growth forecasts, 2025-2030. Grand View Research. Research and Markets, Guinness Centre, Taylors Lane, Dublin 8, D08 AH31, Ireland. Grand View Research; 2025. URL: https://tinyurl.com/38sc2z5e [accessed 2025-05-29]
- Komunikat Ministra Zdrowia w sprawie przyznawania aplikacji tytułu „Aplikacja Certyfikowana MZ” oraz włączania do „Portfela Aplikacji Zdrowotnych” (PAZ). Ministry of Health, Poland. ul. Miodowa 15 00-952 Warszawa. Ministry of Health; Aug 08, 2022. URL: https://tinyurl.com/9ehppt5u [accessed 2025-05-29]
- Morselli S, Sebastianelli A, Domnich A, Bucchi C, Spatafora P, Liaci A, et al. Translation and validation of the Italian version of the user version of the Mobile Application Rating Scale (uMARS). J Prev Med Hyg. Mar 2021;62(1):E243-E248. [FREE Full text] [CrossRef] [Medline]
- Shinohara Y, Yamamoto K, Ito M, Sakata M, Koizumi S, Hashisako M, et al. Development and validation of the Japanese version of the uMARS (user version of the mobile app rating system). Int J Med Inform. Sep 2022;165:104809. [CrossRef] [Medline]
- Chasiotis G, Stoyanov SR, Karatzas A, Gravas S. Greek validation of the user version of the Mobile Application Rating Scale (uMARS). J Int Med Res. Mar 16, 2023;51(3):3000605231161213. [FREE Full text] [CrossRef] [Medline]
- Vasconcelos JM, Viana YG, Duarte NP, Polese JC. Translation and cross-cultural adaptation into Portuguese-Brazil of NuHISS, Enlight, and uMARS. J Bodyw Mov Ther. Apr 2024;38:437-448. [FREE Full text] [CrossRef] [Medline]
- Haas R, Aşan H, Doğan O, Michalek CR, Karaca Akkan O, Bulut ZA. Designing and implementing the MySusCof app-a mobile app to support food waste reduction. Foods. Jul 26, 2022;11(15):2222. [FREE Full text] [CrossRef] [Medline]
- Deady M, Glozier N, Collins D, Einboden R, Lavender I, Wray A, et al. The utility of a mental health app in apprentice workers: a pilot study. Front Public Health. Sep 4, 2020;8:389. [FREE Full text] [CrossRef] [Medline]
- Song Y, Chen H. Evaluating Chinese mobile health apps for ankylosing spondylitis management: systematic app search. JMIR Mhealth Uhealth. Jul 14, 2021;9(7):e27234. [FREE Full text] [CrossRef] [Medline]
- Lull C, von Ahnen JA, Gross G, Olsavszky V, Knitza J, Leipe J, et al. German mobile apps for patients with psoriasis: systematic search and evaluation. JMIR Mhealth Uhealth. May 26, 2022;10(5):e34017. [FREE Full text] [CrossRef] [Medline]
- Miró J, Llorens-Vernet P. Assessing the quality of mobile health-related apps: interrater reliability study of two guides. JMIR Mhealth Uhealth. Apr 19, 2021;9(4):e26471. [FREE Full text] [CrossRef] [Medline]
- Azad-Khaneghah P, Roduta Roberts M, Liu L. Alberta rating index for apps: study of reliability and validity. Can J Occup Ther. Sep 16, 2022;89(3):326-338. [FREE Full text] [CrossRef] [Medline]
- Baptista S, Oldenburg B, O'Neil A. Response to "development and validation of the user version of the Mobile Application Rating Scale (uMARS)". JMIR Mhealth Uhealth. Jun 09, 2017;5(6):e16. [FREE Full text] [CrossRef] [Medline]
- Shetty VB, Soon WH, Roberts AG, Fried L, Roby HC, Smith GJ, et al. A novel mobile health app to educate and empower young people with type 1 diabetes to exercise safely: prospective single-arm mixed methods pilot study. JMIR Diabetes. Oct 14, 2021;6(4):e29739. [FREE Full text] [CrossRef] [Medline]
- Lambrecht A, Vuillerme N, Raab C, Simon D, Messner E, Hagen M, et al. Quality of a supporting mobile app for rheumatic patients: patient-based assessment using the user version of the Mobile Application Scale (uMARS). Front Med (Lausanne). Jul 22, 2021;8:715345. [FREE Full text] [CrossRef] [Medline]
- Herbuela VR, Karita T, Francisco ME, Watanabe K. An integrated mHealth app for dengue reporting and mapping, health communication, and behavior modification: development and assessment of mozzify. JMIR Form Res. Jan 08, 2020;4(1):e16424. [FREE Full text] [CrossRef] [Medline]
- Piderit MC, White Z, Wenhold FA. The development and usability of a web-based mobile application as a dairy intake screener for South African adults. J Dairy Res. Nov 16, 2022;89(4):453-460. [CrossRef] [Medline]
- Petros NG, Hadlaczky G, Carletto S, Martinez SG, Ostacoli L, Ottaviano M, et al. Sociodemographic characteristics associated with an eHealth system designed to reduce depressive symptoms among patients with breast or prostate cancer: prospective study. JMIR Form Res. Jun 08, 2022;6(6):e33734. [FREE Full text] [CrossRef] [Medline]
- Becker ER, Shegog R, Savas LS, Frost EL, Coan SP, Healy CM, et al. Parents' experience with a mobile health intervention to influence human papillomavirus vaccination decision making: mixed methods study. JMIR Pediatr Parent. Feb 21, 2022;5(1):e30340. [FREE Full text] [CrossRef] [Medline]
- Olesch A. OtwartySystem Ochrony Zdrowia OSOZ Rapord Specjalny Edycja 2019. KAMSOFT S.A. ul. 1 Maja 133 40-235 Katowice. KAMSOFT S.A; 2019. URL: https://www.osoz.pl/static_files/kom-linki/aplikacje_2019_03.pdf [accessed 2025-05-29]
- Żarnowski A, Jankowski M, Gujski M. Use of mobile apps and wearables to monitor diet, weight, and physical activity: a cross-sectional survey of adults in Poland. Med Sci Monit. Aug 25, 2022;28:e937948. [FREE Full text] [CrossRef]
- Raeesi A, Khajouei R, Ahmadian L. Evaluating and rating HIV/AIDS mobile apps using the feature-based application rating method and mobile app rating scale. BMC Med Inform Decis Mak. Oct 30, 2022;22(1):281. [FREE Full text] [CrossRef] [Medline]
- STOP COVID - archival material about the app STOP COVID. Ministry of Digitalization of Poland. 2022. URL: https://www.gov.pl/web/protegosafe/protego-safe [accessed 2022-05-29]
- Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A, et al. ISPOR Task Force for Translation and Cultural Adaptation. Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (PRO) measures: report of the ISPOR task force for translation and cultural adaptation. Value Health. Mar 2005;8(2):94-104. [FREE Full text] [CrossRef] [Medline]
- Konstantynowicz J, Abramowicz P, Glinkowski W, Taranta E, Marcinowicz L, Dymitrowicz M, et al. Polish validation of the SarQoL, a quality of life questionnaire specific to sarcopenia. J Clin Med. Oct 04, 2018;7(10):323. [FREE Full text] [CrossRef] [Medline]
- Glinkowski W, Żukowska A, Dymitrowicz M, Wołyniec E, Glinkowska B, Kozioł-Kaczorek D. Translation, cross-cultural adaptation, and psychometric properties of the polish version of the hip disability and osteoarthritis outcome score (HOOS). Medicina (Kaunas). Sep 20, 2019;55(10):614. [FREE Full text] [CrossRef] [Medline]
- Katarzyna K, Glinkowski WM. Patient-reported outcomes of carpal tunnel syndrome surgery in a non-industrial area. Ann Agric Environ Med. Jun 17, 2019;26(2):350-354. [FREE Full text] [CrossRef] [Medline]
- Stoyanov SR, Hides L, Kavanagh DJ, Zelenko O, Tjondronegoro D, Mani M. Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR Mhealth Uhealth. Mar 11, 2015;3(1):e27. [FREE Full text] [CrossRef] [Medline]
- McNeish D. Thanks coefficient alpha, we'll take it from here. Psychol Methods. Sep 2018;23(3):412-433. [CrossRef] [Medline]
- Statistica (version 13): data analysis software system. StatSoft. URL: https://www.statsoft.pl/programy/statistica/ [accessed 2025-05-29]
- Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976). Dec 15, 2000;25(24):3186-3191. [FREE Full text] [CrossRef] [Medline]
- Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol. Jul 2010;63(7):737-745. [CrossRef] [Medline]
Abbreviations
| COSMIN: Consensus-Based Standards for the Selection of Health Measurement Instruments |
| ICC: intraclass correlation coefficient |
| mHealth: mobile health |
| STROBE: Strengthening the Reporting of Observational Studies in Epidemiology |
| uMARS: User Version of the Mobile Application Rating Scale |
Edited by J Sarvestan; submitted 05.Aug.2024; peer-reviewed by K Stawarz; comments to author 08.Jul.2025; revised version received 10.Nov.2025; accepted 10.Nov.2025; published 12.Jan.2026.
Copyright©Wojciech Michał Glinkowski, Tomasz Cedro, Joanna Nowicka, Rafał Doniec, Bartłomiej Michalak, Bożena Glinkowska, Stoyan Stoyanov. Originally published in JMIR Formative Research (https://formative.jmir.org), 12.Jan.2026.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.

