The procedures followed in this study were conducted according to the principles of the World Medical Association Declaration of Helsinki and were approved by the University of New South Wales and St Vincent’s Hospital Human Research Ethics Committee jointly (16/229). The study was independently audited.

This is a cross-sectional blinded study comparing the fall detection classification provided by a smartwatch algorithm with a reference standard’s classification, in this case, a video record of induced falls.

A total of 22 volunteer participants deemed to be medically healthy were recruited after satisfying all the inclusion and exclusion criteria. Participants were recruited by distribution of a leaflet on the university campus and compensated for their time. The inclusion criteria were males/females older than 18 years willing and able to provide written informed consent prior to initiation of any study-related procedures. Participants were excluded if they had any of the following: disability that may prevent them from completing the study (eg, severe illness), being suspected of or having a known allergy to any components of the smartwatch, having any injury or medical condition that would be adversely affected by an induced fall, and being pregnant.

This study used a threshold-based algorithm programmed for different smartwatches. The threshold-based algorithm running on the smartwatch app uses threshold values, or settings, to automatically detect a fall. The frequency of the smartwatch accelerometer is 2 kHz with the algorithm of the app collecting data every 0.01 seconds. The algorithm follows strict rules for the three phases of a fall, as shown in Figure 1. The algorithm was supplied by My Medic Watch.

T1 is defined as the time during which the smartwatch is moving toward the earth (fall time) recording a low acceleration, lower than 1G. T2 is the time during which the smartwatch hits the ground, recording a very high positive acceleration for a short period of time. T3 is the time during which the smartwatch is “almost” immobile on the ground for a long period of time. These threshold values are optimized in the app according to the particular smartwatch and body morphology, including body weight and height. Optimization was performed during the test falls.

A near fall can be recognized when all, or one, of the accelerometer data are close to one of the thresholds, as depicted in Figure 2. We have arbitrarily defined “close” as 20% lower than the fall threshold value.

Participants were randomly assigned to have either smartwatch model A or model C on one wrist and model B or no device on the other wrist. Model A and C were running one operating system, while model B was running on a different operating system. Every smartwatch contained the fall detection app that was programmed to detect and record falls paired with a smartphone located at the study site. The same app was used for each model. The smartwatches and smartphones used one of two operating systems: android or iOS. Two smartwatches were connected to iOS and one to Android. The versions of iOS and Android were the latest available at the time of the test. The version of the operating system on the smartphone and smartwatch were the same for all participants. The smartphones were linked to the smartwatch (according to the operating system) to communicate stored data of the time-stamped recorded episodes to secure cloud servers that were then compared to the video-recorded events.

Before starting the trial, participants were placed in a crash mat protected area, the smartwatches were placed on the participants’ wrists, and a helmet was provided to be used during the tests; no other safety devices were used. Once the trial started, the smartwatch app was set up in monitoring mode and two rounds of four falls were induced in the blindfolded participants. A fall was defined as an event that results in a person coming to rest inadvertently on the ground, floor, or other lower level. A nonfall was defined as any event occurring while both the smartwatch app and the video record were active but excluding a fall or near fall (defined later). In every round, a frontward fall, a right side fall, a left side fall, and a backward fall were induced. These were induced by pushing the participant while standing. The method of fall induction was the same for all participants, executed by the same person. The participants were told of the impending direction of the push. Each assessment took approximately 5 minutes with 8 falls: 2 backward, 2 forward, 2 right, and 2 left. Additionally, up to 3 test falls were performed before the first round to ensure the participants were feeling comfortable with the procedure. Test falls were not included in the analyses. Further, prior to the test falls and between the falls, the participants wore the smartwatches and walked around freely. Near falls where the participant took one or more steps in the direction of the push without falling were also recorded, as there is some evidence that they may presage a fall [12]. This definition is in accord with the traditional definition as applied to this experimental scenario: “a stumble event or loss of balance that would result in a fall if sufficient recovery mechanisms were not activated” [12]. Importantly, the fall-triggering settings were optimized for each participant during the test falls. A non–near fall was defined as any event occurring while both the smartwatch app and the video record were active but excluding a fall or near fall.

During the fall, the algorithm was collecting the acceleration data and the time of the fall. The data collected were in three phases: “prefall” (preparation and walking to the crash mat, several minutes) as soon as the smartwatches were on the participants wrist, “induced fall” (8 falls around 5 minutes), and “post fall,” walking back from the crash matt to the area to remove the smartwatches. In addition to this, the falls were recorded by built-in motion-detecting cameras (recording at 50 frames/second) available at the study site, the National Facility for Human Robot Interaction Research, University of New South Wales. Motion detection data were used to indicate when a fall was observed. The video of the falls also contained a timestamp that was used to compare it with the falls detected by the smartwatch app. In this case, the video recorded event was used as a reference standard, and the falls detected by the smartwatches were compared against it.

After all the falls had been induced, the smartwatches and safety equipment were removed, and participants were observed for approximately 10 minutes: the heart rate, blood pressure, and symptoms (if any) were assessed.

To perform the analysis of the falls, data were first retrieved from video records of the built-in motion-detecting cameras and coded as a fall or near fall by the authors and a person independent of the conduct of the study. Where there was disagreement, a majority opinion was taken. These were then compared independently by an external person with data retrieved from a fall detection database built to register the falls detected by the smartwatch algorithm. Each fall was classified as a true positive if the smartwatch app detected a fall at the time when the event was recorded on the video, a false positive if the smartwatch detected a fall event that was not recorded on the video, a false negative if the smartwatch did not detect a fall event recorded on the video, and a true negative if neither the smartwatch nor the video recorded a fall. Near falls were similarly analyzed. Results were computed for sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, positive predictive value, negative predictive value, and accuracy. CIs for sensitivity, specificity, and accuracy are “exact” Clopper-Pearson CIs. CIs for the likelihood ratios are calculated using the “Log method.” To compare fall and near fall detection by smartwatch model and direction of fall only, sensitivity data were used with chi-square tests and a significance value of P<.05. Further data are available on request. Sample size calculations were not formally performed beyond an approximate anticipated number of 20 to 25 participants that could be accommodated for the study given the constraints of the availability of the study site and personnel time.

A total of 22 participants were enrolled in the study: 14 (63%) females and 8 (36%) males; 20 (91%) completed the whole procedure. Two (9%) females abandoned the study during the process: one after a soft tissue injury and the other for unstated reasons. An average of 7.2 falls was performed for each participant; however, one of the participants withdrew from the study after having performed 5 sets of 8 falls, and another after having performed 1 set of 8 falls. Of the induced 226 falls, 64 were backward, 51 were forward, 55 were left sided, and 56 were right sided. Two participants reported postfall self-limiting symptoms associated with soft tissue injuries, 1 required medication and physiotherapy, and their symptoms resolved after 6 weeks.

Demographic characteristics of the participants are shown in Table 1. With regard to BMI, 1 (6%) female was classified as underweight, 1 male and 1 female were classified as overweight (9%), and 1 (6%) male was classified as obese.

A total of 12 participants were wearing two smartwatches, model A device on one wrist and model B on the other wrist; 10 participants were wearing only one smartwatch, model C, on one wrist. The overall performances of the algorithm, disregarding the model of the smartwatch, are detailed in Tables 2 and 3. There was no difference in the performance of the algorithm according to which wrist if both were used. Tables 4 and 5 represent the results of near fall detection and the associated statistics. The overall test outcomes are summarized in the following section.

In general, the direction of the fall or near fall did not significantly influence sensitivity. Nonetheless, there was a trend for better detection of backward falls: of the 64 backward falls, 11 were false negatives, giving a sensitivity of 82%, versus forward falls, of which there were 51 with 12 false negatives, giving a sensitivity of 76%. Further, there was a significant difference in fall detection if the fall was to the same side versus opposite side of the wrist that had the smartwatch (left sided and right sided sensitivities combined: 92.5% vs 76.3%; P=.009). The same held true for near falls. If the fall was to the same side as the wrist with the smartwatch, there was a 95% sensitivity for left sided falls (55 with 3 false negatives) and 89% sensitivity for right sided falls (56 with 11 false negatives) versus if the fall was on the opposite side as the wrist with the smartwatch, there was 84% sensitivity for left sided falls (55 with 9 false negatives) and 80% sensitivity for right sided falls (56 with 11 false negatives).

The number of responses for each smartwatch model were A=186, B=186, and C=122. Model A was used 173 times on the left wrist and 13 times on the right wrist. As per Table 6, there were differences among the models according to sensitivity and specificity, but none were significant. This was also true of the operating system. Similar results were found for near falls.