The following devices were included: Fitbit Charge 6, Fitbit Inspire 3, Garmin Vivosmart 5, Garmin Vivoactive 5, Apple Watch SE, Google Pixel Watch 2, Polar Ignite 3, Polar Pacer, Xiaomi Watch 2, and Oura (Gen 3). Thus, this selection contained 9 watches and 1 ring. The Zephyr BioHarness 3.0 chest strap was used as the gold standard reference measure. The Zephyr BioHarness 3.0 allows for collecting the raw electrocardiogram data and therefore to evaluate the data quality of the derived HR time series. It has demonstrated validity across many exercise modalities and contexts [19].

A total of 45 healthy adult volunteers (23 male, 22 female) participated in this study. The participants ranged in age from 21 to 68 years, with a mean age of 34 (SD 12) years. All participants were free from known cardiovascular, metabolic, or musculoskeletal conditions that could interfere with data collection or affect physiological responses. The participants completed an hour-long experimental protocol. During each session, participants wore 2 watches and 1 chest strap. To ensure that there was no optical or electromagnetic cross-interference, only 2 watches were worn at any time, 1 on the left wrist and 1 on the right wrist. The independent operation and data storage of all devices also mitigated any potential data interference. They were instructed to wear the smartwatch snugly, just above the wrist bone, ensuring the optical sensor maintained direct skin contact. In addition, 10 individuals wore the Oura ring on their nondominant hand. The ring (size 10) was worn on the finger that ensured stable sensor contact. Each of the 10 wearable devices was tested 10 times across different participants, yielding a total of 100 unique testing sessions (10 per device). To control for potential placement effects, the location of the devices was counterbalanced across the left and right hands. This sample size balanced device coverage against the logistical constraints of the multisession environmental protocol.

The study was approved by the Social and Societal Ethics Committee of KU Leuven (case G-2024-8195-R4(AMD)). Written informed consent was obtained from all study participants prior to participation after the nature and possible consequences of the studies were explained. To ensure privacy and confidentiality, all participant data were deidentified and stored on secure, password-protected institutional servers. No compensation was provided to participants.

All participants performed a standardized protocol in each of 3 environmental chambers: a neutral condition (23 °C, 50% relative humidity), a hot condition (36 °C, 70% relative humidity), and a cold condition (10 °C, 40% relative humidity). The 3 climate chambers were dimensionally identical (3.6 m by 2.4 m) and were located adjacent to one another, allowing for rapid participant transition. Half of the participants completed the protocol in the sequence neutral-hot-cold, while the other half followed the sequence neutral-cold-hot.

Within each environmental condition, participants completed six tasks lasting in total 22 minutes: (1) a 6-minute seated resting phase to acclimate to the chamber, (2) a 4-minute cognitive stress task (Montreal Imaging Stress Task [MIST] [20]) performed at a desk using a PC, (3) a 2-minute seated recovery phase, (4) a 4-minute steady walking task on a treadmill (Focus Fitness Jet 1) at 6 km/h, (5) a 4-minute intermittent walking task involving alternating 30-second walking and standing intervals, and (6) a final 2-minute standing rest phase. The sequence of these 6 tasks was fixed for all participants to maintain a standardized progression from rest to increasing cognitive and physical exertion, allowing for direct comparison of device performance across the controlled states. To mitigate systematic carry-over effects (eg, fatigue or learning) that might accumulate over the testing day, the order of the 3 main environmental conditions was systematically counterbalanced across participants. This experimental protocol was designed to simulate a range of ecologically valid use cases including cognitive and physical stressors, steady-state and intermittent movement, and environmental variation. The timeline of the research protocol is visualized in Figure 1.

HR data from the Zephyr BioHarness and each wearable device were recorded continuously throughout all tasks. The data were exported using manufacturer-provided software platforms or application programming interfaces. This yielded the HR data at the highest possible sampling frequency for each device. The Polar Ignite 3, Polar Pacer, and Oura collected data at 1 Hz. For the other devices, the sampling frequency was variable with the intervals varying from 1 to 7 seconds. To ensure alignment, HR time series from each wearable were synchronized with the reference series using timestamp-based matching. Any reference data points that could not be matched due to sparse wearable sampling were excluded from analysis. No additional data smoothing or outlier removal was performed.

Three key metrics were used to assess HR accuracy. First, mean absolute error (MAE) and MAPE were calculated between the wearable and reference HR signals for each condition and device. Second, we used the concordance correlation coefficient (CCC) [21]. Because the data consisted of repeated measurements over time for each participant, we extended the CCC using a repeated-measures framework [22]. The repeated-measures CCC accounts for the hierarchical structure of the data by estimating the variance components associated with between-subject and within-subject variability. This was done by fitting a linear mixed-effects model with a random intercept per participant, allowing variance components due to between-subject and within-subject variability to be separated. The repeated-measures CCC was then calculated using the model-based estimates of means, variances, and correlations, ensuring an unbiased agreement estimate under repeated measures. Third, we calculated the mean bias and limits of agreement for the Bland-Altman analysis [23] to assess the agreement between HR values recorded by the consumer wearable and the gold-standard chest strap. Similar to the calculation of the CCC, a mixed-effects adaptation of the original Bland-Altman analysis as described in Bland and Altman [24] was used to account for the repeated measurements.

The statistics outlined above were calculated both for the complete time series and for segments of the data corresponding to isolated activities and environmental conditions. This approach allowed for a statistical comparison to determine whether different conditions influenced the accuracy of the devices.

Before formal analysis, the normality of the MAE and MAPE values for the different subsets was tested using Shapiro-Wilk testing. The significance level (α) was set to .05 for these tests. Given that the null hypothesis of the normally distributed data was rejected in most cases, it was decided to apply robust nonparametric testing. To determine whether certain devices yielded significantly differing MAE and MAPE values, Kruskal-Wallis testing was performed. When the test yielded a significant result, multiple Wilcoxon rank sum tests were performed to identify which pairs of devices differ significantly. Similarly, it was studied whether the experimental conditions (ie, the type of activity or thermal environment) yielded significantly differing MAE and MAPE values for each device. For this purpose, Friedman testing was performed. When the test yielded a significant result, multiple Wilcoxon signed-rank tests were performed to identify which pairs of experimental conditions differed significantly. To account for the family-wise error rate of repeated testing, Bonferroni-corrected α-values were considered. Specifically, the Bonferroni correction was applied to all multiple pairwise Wilcoxon rank sum tests and Wilcoxon signed-rank tests to maintain a family-wise error rate of α=.05 across the multiple device and condition comparisons performed. To enhance the interpretability of statistically significant findings, we calculated and reported effect sizes for the pairwise nonparametric tests. For all pairwise Wilcoxon signed-rank and rank sum tests, the rank-biserial correlation (r) was used to quantify the magnitude of the difference. These metrics are indicated in the main text of the paper when presenting the results.

All analyses were performed in MATLAB using the fitlme function to fit mixed-effects models and custom scripts to compute the extracted parameters.

This study evaluated the HR accuracy of 10 commercially available wearable devices (9 watches and 1 ring) under varying environmental and activity conditions. Accuracy was assessed using multiple statistical approaches, including absolute and relative error metrics (MAE and MAPE), concordance with a reference chest strap (repeated-measures CCC), and agreement intervals (Bland-Altman analysis). To interpret these outcomes, established standards were applied. For example, the American National Standards Institute specifies a boundary of ±10% MAPE or ±5 bpm, whichever is greater, as acceptable error for cardiac monitors [25]. Correlation coefficients were interpreted as weak (≤0.5), moderate (0.5‐0.7), or strong (≥0.7) [26,27], while a stricter criterion of CCC ≥0.80 has also been proposed in previous work [13,28]. Within this framework, our findings indicate differences in device performance: the Fitbit Inspire 3 and Oura Ring exhibited significantly higher errors and wider limits of agreement, whereas the Fitbit Charge 6 and Google Pixel Watch 2 met both the criterion of CCC ≥0.80 and the American National Standards Institute error boundaries. Interestingly, these results demonstrate that accuracy is not only brand-dependent but also device-specific, as even wearables from the same manufacturer, despite relying on similar PPG technology, can yield different outcomes.

Participants underwent repeated testing across 3 thermal environments, neutral (23 °C), hot (36 °C), and cold (10 °C), but we found no statistically significant effect of temperature condition on device accuracy after the Bonferroni correction. However, the type of physical activity did significantly affect HR measurement accuracy in several devices. Specifically, walking intermittently elicited larger errors in a subset of wearables. These results support our initial hypothesis that motion irregularity is a strong determinant of PPG-based HR accuracy.

The significant variability in accuracy across commercial wearables is consistent with earlier validation studies. Düking et al [29], for example, found marked differences in HR accuracy among 4 commercial wrist-worn devices (Apple Watch 4, Polar Vantage V, Garmin Fenix 5, and Fitbit Versa) during a structured treadmill protocol. They observed that vigorous or intermittent activities, particularly those involving sudden directional changes, were prone to introducing motion artifacts that degraded PPG signal quality. Our findings extend this line of work by demonstrating that even seemingly modest variations in movement patterns (eg, intermittent walking) can impact accuracy and that these effects differ between devices.

Interestingly, we did not find significant accuracy differences between climate conditions, which contradicts some theoretical expectations and anecdotal observations. Cold-induced vasoconstriction and heat-induced vasodilation are known to affect peripheral blood flow and optical signal amplitude [3,8,30]. However, it is possible that our selected temperature ranges (10-36 °C) were not extreme enough to meaningfully affect PPG signal quality, particularly in a short-term indoor protocol. Alternatively, modern signal-processing algorithms embedded in the wearables may have compensated for these physiological variations.

Prior studies have also highlighted the limitations of wrist-worn devices for certain populations and use cases. For example, variability in skin tone, arm hair, or wrist circumference can influence optical HR signal integrity [13,31], and these factors were not explicitly accounted for in our study. Nevertheless, the consistency of our findings across participants and the robustness of the repeated-measures statistical approach support the generalizability of our device comparisons.

Notably, the Fitbit Charge 6 consistently showed high accuracy across all conditions, with low MAE, low MAPE, and high repeated-measures CCC (>0.90). The Google Pixel Watch 2 and Garmin Vivosmart 5 also performed well, particularly during steady-state walking. In contrast, the Polar Ignite 3, Polar Pacer, and Oura Ring exhibited wider limits of agreement and lower CCC values, particularly during irregular activity. These differences may reflect varying PPG sensor designs, sampling rates, and proprietary signal-cleaning algorithms [32,33].

PPG is inherently sensitive to movement artifacts, particularly when motion occurs along the axis of the light source or in tissues with variable compressibility (eg, wrists with varying musculature) [34]. Devices such as the Oura Ring, which is worn on the finger, may face additional challenges due to the vasomotor sensitivity of the digits and the smaller optical contact area. While rings offer the advantage of less wrist movement, they may be more susceptible to temperature-driven changes in blood flow [35], which could explain the relatively poorer performance of the Oura device in this study.

Furthermore, activity context plays a significant role. The MIST task, a standardized cognitive stressor with minimal physical exertion, was associated with lower HR variability and motion. This is consistent with evidence that cognitive demands and psychological stress typically reduce HR variability [36]. As a result, wearables generally performed well under these conditions. In contrast, intermittent walking introduced frequent postural and muscular transitions, posing a challenge for devices dependent on motion-compensated PPG. These findings align with previous observations that accuracy degrades in nonsteady-state conditions and that current algorithms may overfit to predictable, repetitive activity patterns [37,38].

Several limitations should be acknowledged. First, the duration of each activity phase was relatively short (4‐6 min). Longer monitoring periods might have revealed delayed effects of thermal stress or device drift. Second, while we designed the environmental conditions to simulate realistic hot and cold exposures, extreme climates or outdoor settings were not replicated. Third, we did not directly measure skin temperature or blood flow perfusion, which could have elucidated underlying physiological causes of error.

Moreover, our participant sample, although balanced across devices and arms, did not include diverse skin tones or body compositions, which may limit generalizability. We note from the literature that variations in skin tone are known to bias PPG accuracy [11], suggesting that device performance may differ in diverse populations. The skin tone of the participants in our study was not included in the data collection. In hindsight, we note a relatively homogenous population of lighter skin phototypes (Fitzpatrick type I-III). Additionally, while the sample had a broad age range (21‐68 y) and a balanced sex distribution, the sample was predominantly younger adults, and only 5 older participants (ages 47‐68) were involved. Hence, a formal analysis to quantify the isolated impact of skin tone, age, and sex on device accuracy was not performed, which may be a focus for future research. Finally, as with any study of commercial wearables, device firmware and algorithms may change over time, making it difficult to extrapolate our results to future updates of the same models.

Our findings offer practical insights for researchers and health professionals selecting wearable devices for HR monitoring in mixed conditions. Devices such as the Fitbit Charge 6 and Google Pixel Watch 2 may offer sufficient accuracy for light-to-moderate physical activities in typical environmental conditions. However, researchers should exercise caution when using wearables for irregular activities or for applications that require high precision (eg, clinical monitoring or dose-response studies).

The study protocol can easily be replicated with other wearable devices, enabling comparisons between new devices and the 10 devices that have already been evaluated. For future validation studies, the inclusion of diverse populations, extended durations, and wearable signal-quality indices (eg, raw PPG, accelerometer-derived movement indices) would allow for a more granular understanding of error sources. Open-science practices, including the publication of raw validation data and code for statistical methods, will further support reproducibility and transparency in wearable health technology research.

In this comparative study of 10 commercial wearable devices, significant differences in optical HR accuracy were observed between brands and activity types. Climate conditions did not produce statistically significant effects on accuracy, suggesting robustness within the moderate temperature ranges tested. However, irregular movement patterns degraded performance in multiple wearables, highlighting the need for careful selection and context-aware interpretation of HR data in applied health and sports settings. Devices such as the Fitbit Charge 6 and Google Pixel Watch 2 demonstrated strong agreement with reference chest strap measurements, even across varied protocols. The device-specific findings should be interpreted with caution, as they pertain to the investigated models. Accuracy may vary considerably across other devices within the same brand, despite the use of similar PPG technology.