Statistical tests were conducted considering the satisfaction of the assumptions of these tests, including examining whether the distributions of the studied variables conformed to normality. The Shapiro-Wilk W test was used to assess the normality of variable distributions. Normal distribution of quantitative variables entailed calculations of mean differences (95% CIs with 1–α=.95 confidence level). External agreement between the 2 measurements was tested for each parameter at P<.05. The correlation between each domain and the uMARS total score was assessed using the Pearson correlation coefficient, where a value greater than 0.7 (significance level below 0.05) was considered sufficient to evaluate test-retest reliability. Finally, the ICC between the test and retest was calculated to determine weighted rater agreement values and evaluate the closeness rather than equality of scores.

The reliability of the uMARS questionnaire was assessed via the Student 2-tailed t test for dependent variables. The analysis used the final test results and individual categories (results of individual scale parts). Reliability was calculated using Pearson linear correlation and ICC of each variable. Cronbach α (0.95% CI) was calculated to assess internal consistency and reliability. The Cronbach α values were interpreted as excellent (≥0.90), good (0.80-0.89), acceptable (0.70-0.79), doubtful (0.60-0.69), weak (0.50-0.59), and unacceptable (<0.50). Furthermore, the results were supplemented with descriptive statistics (means and SDs).

All analyses were performed using TIBCO Software Inc (version 2017) and Statistica (version 13; Data Analysis Software System) [71]. The results were considered statistically significant at the significance level (α) of .05 if the critical level P value was less than .05.

This study was approved by the Bioethical Commission (AKBE/60/2022). The study was conducted in accordance with the principles of scientific ethics and the Declaration of Helsinki. The Bioethical Commission approved the study protocol on February 21, 2022, and confirmed that the study complies with ethical standards for scientific research. All participants were adults and took part in the study voluntarily. Before participating in the study, participants received information about the purpose and procedures of the study and gave their informed consent in electronic form. Participants were informed that participation in the study was voluntary and that they could withdraw at any time without any consequences. No personally identifiable data were collected. All data were collected and analyzed anonymously. Data storage and processing complied with applicable data protection regulations, and access to the data was restricted to the research team. Participants did not receive any financial or other compensation for their participation in the study.

This study enrolled 117 participants who fulfilled the inclusion criteria (women: n=81, 69.2%; men: n=36, 30.8%; mean age 20, SD 1.66 years). Among them, 90 (76.9%) participants were associated (academically or professionally) in the medical field; 19 (16.2%) were in biomedical engineering, 4 (3.4%) were in nursing, and 4 (3.4%) were in dentistry. Most respondents were undergraduate and postgraduate students enrolled at Cardinal Stefan Wyszyński University in Warsaw (n=39, 33.3%), the Medical University of Warsaw (n=60, 51.3%), and the Silesian University of Technology (n=18, 15.4%). A total of 71 (60.7%) respondents reported daily use of mobile apps, and 59 (50.4%) respondents used Apple, iOS, or other Android devices.

All respondents performed the initial app assessment twice with a 1-week gap using uMARS. The questionnaire was administered twice with a 1-week break to assess the repeatability of the Polish uMARS questionnaire. First, a paired t test was conducted to observe the differences between the 2 measurements for each subscore and total score. Test-retest reliability was assessed for each variable using the Pearson 2D correlation coefficient.

The participants rated the STOP COVID ProteGO Safe app twice. Every participant received clear and concise information regarding the study objectives, rules of participation, and procedures.

The translation process involved several stages: preparation, double translations from the original version, reconciliation, back translation, review and harmonization, and final proofreading [65,72]. Internationally accepted guidelines and checklists [66-68] were used to ensure thorough translation and validation processes [72,73]. Two native Polish-speaking translators independently translated the original English survey into Polish. A consensus meeting agreed upon a standard version as a unified translation. Similarly, 2 independent back translations into English were conducted anonymously using the Polish version of the uMARS. The interdisciplinary group reviewed and completed the questionnaires. No significant difficulties were encountered during the translation process, and a consensus was reached regarding minor discrepancies that reflected the cultural context of the Polish language or semantic issues. The final consensus on the Polish version of the uMARS was achieved after a group of pretesters tested the questionnaire. They confirmed its comprehensibility and transcultural adaptation. A translation history review and examination of the original questionnaire confirmed that all items in the original uMARS were flawlessly translated without major substantive problems. Consequently, the final version was presented to respondents unfamiliar with uMARS, seeking their input on the translated words, understandability, interpretability, and cultural relevance. A ready-to-use version of the Polish uMARS was made available after a final review.

Participants completed the questionnaire twice, providing responses for an initial assessment (test) and a reassessment after a week (retest), considering the values obtained by pairs of the same respondents (P>.05; Table 1).

A paired t test was performed to compare the results of the test and retest to assess the reliability of the test and retest of the Polish version of uMARS.

The external consistency of the 2 measurements was statistically significant for each measured parameter (P<.05). The highest agreement was in category F (perceived impact; r=0.73; P<.001). The concordance for the total scale score was moderate (r=0.40; P<.001); however, concordance was not observed for category C12 (visual appeal; P=.18; Table 2).

The reliability analysis of Cronbach α confirmed the very high reliability of the scale at 0.95. No relevant factor lowered or increased the reliability of the entire tool (Table 3).

The ICC values were overall moderate. The highest ICC values were for the category perceived impact (ICC=0.59 for 0.29-0.53). The intraclass correlation coefficient for the total uMARS score was moderate (ICC=0.65), indicating moderate temporal stability. This level of agreement is consistent with comparable validation studies of user-rated app evaluation tools, where subjective perceptions of design and engagement may vary naturally between assessments.

Figure 1 presents a scatterplot depicting final test values (total test vs complete retest). The scatterplot shows the relationship between the overall scale scores on the test and retest. The results were positively correlated, as evidenced by the upward slope of the solid red regression line.

Participants who gave higher ratings on the test tended to give higher ratings on the retest, whereas those who gave lower ratings on the test tended to give lower ratings on the retest. The dots running in an even pattern along the regression line illustrate this observation. The dashed lines indicate CIs of the mean scores.

Figure 2 shows the percentage distribution of individual ratings (1-5 points) provided by the respondents in each category. The left and right sides represent the test and retest results, respectively. Different colors denote each rating. Matching colors indicate specific assessments in both measurements with similar frequencies. The test results mirrored the retest results, thereby forming a mirror image pattern.

The overall ICC values and those for the entire scale were moderate (ICC=0.65). The highest ICC values were for the category perceived impact (ICC=0.59 for 0.29-0.53).

The Polish version of uMARS exhibited excellent internal consistency and good short-term (weekly) test-retest reliability throughout the validation process. The subscales demonstrated either excellent or good reliability in all domains. This version enabled Polish users to obtain reliable measurements of mHealth app quality. The app evaluation system integrated evidence, user experience, and content values into a comprehensive evaluation system. The partial and total results of the assessed domains were reliable and correctly evaluated using the questionnaire.

This study used a single mobile app (STOP COVID ProteGO Safe) as a standardized reference for validation, ensuring linguistic consistency and minimizing contextual bias in participant responses. However, the use of a single app did not permit the assessment of factorial or convergent validity. This limitation aligns with early-phase validation studies of the original uMARS and its adaptations in other languages. Future research should incorporate multiple app categories to confirm construct validity and investigate potential subgroup differences.

The overall internal consistency of the Polish uMARS was high (Cronbach α=0.95). While this supports strong item homogeneity, uniformly high α values across subscales should be interpreted with caution, as they may reflect item redundancy within conceptually overlapping domains. Similar Cronbach α magnitudes have been observed in other uMARS validations and in multi-item satisfaction scales, suggesting that high interitem correlations are typical for this type of measure. Nevertheless, this potential redundancy represents a limitation inherent to user perception instruments. The Polish uMARS questionnaire responds to the growing need for standardization in the evaluation of health-related mobile apps, informed by user feedback. It offers a structured approach to assessing key quality aspects of apps in Polish, benefiting both users and developers.

The app in question is no longer available, as it was designed specifically for use during the COVID-19 pandemic and was withdrawn after it ended. This circumstance makes it impossible to repeat the study using this app, as the app was used only during the COVID-19 pandemic. However, this study aimed to verify the Polish language version of the app. Despite the limited duration of the app’s operation, our goals were successfully achieved, as the aim of this study was to validate the Polish version of the scale.

This validation was conducted on a sample of young university students pursuing health-related education. Although this group was appropriate for the initial phase of linguistic and psychometric validation, it may not fully represent older or less experienced users of digital technology. However, the study group reflects the demographic most engaged in the use of health-related mobile apps. Therefore, focusing on this group was appropriate for initial linguistic and psychometric validation, ensuring that all participants were experienced smartphone users capable of providing reliable app ratings. Future research should evaluate the performance of the Polish version of uMARS among broader and more diverse populations. Validation efforts need to include participants from a wider range of age groups and varying levels of digital literacy to ensure the Polish uMARS’s broader applicability.

Cross-cultural validation studies of the Polish version of uMARS were conducted, showing its reliability comparable to that of the original version. The Polish version of uMARS can be a valuable tool for assessing the quality of mHealth apps in Poland. From a user perspective, the Polish version of uMARS demonstrates appropriate metric properties for evaluating the quality of mHealth apps, providing comprehensive access to mHealth app ratings from different user perspectives.

Polish researchers can use the translated and verified Polish tool to collect end-user feedback and ratings, helping to identify highly rated apps. In addition, app developers can benefit from reliable app component ratings and gain valuable insights for further improvement and development, increasing the overall quality and impact of mHealth apps. Developers and app stores can present more comprehensive and documented evaluation results along with a star rating to provide users with better insight into the app’s quality.