A cross-sectional, laboratory-based study was conducted to assess the criterion validity (SORD vs direct observation) and convergent validity (SORD vs activPAL). Adults were recruited to take part in this laboratory-based study through print and email advertisements at a university campus. Adults aged 18 years or older, without gait abnormalities, able to walk on a treadmill easily, with no skin sensitivity to plasters or tapes, and able to communicate in English were included.

Upon arrival, participants completed a demographic questionnaire including age, sex, ethnicity, job status, marital status, education, and the Physical Activity Readiness Questionnaire [34] for safe exercise. Anthropometric measures, including height to the nearest 0.1 cm and weight to the nearest 0.1 kg, were taken using a stadiometer (Seca 213) and Tanita scale (Tanita Innerscan 50), respectively.

Participants were given a printed activity protocol to help familiarize them with the required activities and the order in which they were to be performed. Textbox 1 presents a range of different states of activities included in the study protocol to mimic typical postures that may be encountered during everyday life.

Hypoallergenic retention dressing tape (Hypafix) was used to attach the SORD and activPAL devices on the midline of the right thigh. Participants were then instructed to engage in a combination of activities in the order of sitting, reclining, sitting, standing, walking, standing, sitting, lying, and walking on a treadmill. Each activity variation lasted for a minimum of 2 minutes and a maximum of 3 minutes and 30 seconds, except walking, which involved participants walking at their regular walking pace along a 10-m-long path. Participants had 2 minutes of optional resting to break up the activities if needed. Ground truth, or the true time spent on each of the activities, was measured by a researcher with the help of a video camera for direct observation.

ActivPAL is a thigh-worn triaxial accelerometer that classifies an individual’s activity into periods of time spent sedentary (lying or sitting), standing, and walking, as well as the number of steps and stepping speed [29,35]. ActivPAL devices were initialized before the data collection and date-time stamped 1-second epoch files were used for comparative analysis.

True time spent engaging in activities was logged by a trained researcher (RDK). This was assisted by a video camera positioned in the room and checked by another researcher (JM). If there was any discrepancy, RDK and JM reviewed the camera data together to achieve consensus. No formal intra- or interrater reliability was conducted.

SORD data were transmitted to a computer through Bluetooth Low Energy. A program was developed in MATLAB (MathWorks) and run on a Microsoft Windows (Microsoft Corp)–based computer to receive data from the SORD devices in real time and store it into a Microsoft Excel (Microsoft Corp) file. The program starts by initializing relevant variables and a communications port, creates a file name based on the current date and time, continuously receives data from the SORD devices, and stores the incoming data in the Microsoft Excel file in real time. Each data packet received from the SORD devices includes values obtained from the onboard sensors at the current time. For a data packet, the following information is then stored in the file in real time: date, time, angle, accelX, accelY, accelZ, gyroX, gyroY, gyroZ,magX, magY, magZ, and battery voltage. To avoid potential Bluetooth transmission package loss, this study used the time-stamp data from the SORD device instead of the computer receiver (ie, the sending time stamp rather than the receiving time stamp). Thus, we had computer receiver and accelerometer data, along with their timestamps. Based on the real sampling rate, the computer calculated the time stamp difference between each data point (ΔT). In this research, the number of missing data points was defined by missing = ΔT/(1/25Hz) – 1. The values of these data points were filled by the average of the 2 data points before and after the missing data points (eg, Vi[missing] = [Vi – 1 + Vi + 1]/2). Before sending data to the server for inference, the phone app waits until all required data have been received (processing buffer length). Using the VANE (standard) classification algorithms, activPAL data were processed and collected using proprietary software (activPAL Professional Research Edition, PAL Technologies). The software-generated event file was used. This file contains a chronological list of all episodes of sedentary, standing, and stepping (ie, walking) activities recorded at 1-second intervals. The frequency of the recorded signals from SORD was subsequently reduced to 1 Hz (ie, 1-second epochs) for comparative analysis. This reduction in frequency simplifies data processing and facilitates direct comparison with activPAL, which was also sampled at 1 Hz. Furthermore, outliers or irregular data points were identified and removed. Once individual data sets were cleaned, they were combined for subsequent comparative analysis. The combining process involved aligning the data sets temporally so that corresponding data points from both devices were synchronized for direct comparison.

Due to multiple limitations, we did not use the available open-source activity recognition algorithms. These limitations include (1) inconsistency in the data format and ranges, (2) differences in the frequency of raw data assumed by these algorithms compared to SORD (which is 28 measurements per second), and (3) the variations of activities considered by these algorithms were not exactly the same as the ones we wanted to address in this research at this stage and in the future. Thus, we developed the data engineering and activity recognition models. Although ensemble learning techniques outperform deep learning, they demand higher computation resources and have longer processing latency [36,37]. Therefore, for practical reasons and real-world applications of SORD, this study used deep neural network models—a combination of convolutional neural network and recurrent neural network—to develop algorithms. Deep neural network can learn features automatically from the raw data, therefore performing better than statistical and basic machine learning methods, and they are suitable for recognizing complex activities [38].

A data scientist developed deep learning algorithms to classify activity type and postural states from preprocessed motion sensor data using the Python programming language [39]. First, machine learning classifiers were developed, trained, and tested for the SORD device. A dynamic sliding window approach was used for machine learning [40], where each window was related to a particular activity and multiple variables were examined within each window to identify patterns. When a particular activity was detected in the sensor readings, features were extracted to classify activities between the previous one and the current one (further details are provided below). Then, criterion validity (against direct observation) and convergent validity (against activPAL micro) were evaluated. Using Python, the Bland-Altman method was used to assess the level of agreement between SORD and each reference measure (criterion agreement=directly observed time and convergent agreement=activPAL). Mean difference represents the systematic bias, and the limits of agreement (LoA) show the range of agreement between SORD and reference methods, where a positive value indicates underestimation and a negative value indicates overestimation by SORD. For all activity states, we predefined the acceptable LoA between ±10%.

A single data set included SORD, activPAL, and direct observation data for 1 participant. Deep learning was used to randomly select 6 data sets for training, 1 for validation, and 7 for testing. In the training set, similar patterns were identified for the previous 35 data points to specify an activity. Confusion matrices were used to visualize the model’s performance. In a confusion matrix, each row represents the instances in the predicted activity, and each column represents the instances in the actual activity.

Ethics approval was granted by the Deakin University Human Research Ethics Committee’s Human Ethics Advisory Group (HEAG-H 109_2019). All participants provided written informed consent. All research data were anonymized before cleaning and analysis. Participants were remunerated with an Aus $20 (US $14) gift voucher.

In total, 15 adults (12 female adults) aged between 20 and 62 years completed the experimental study. Table 1 presents the demographic characteristics of the participants.

A total of 4 models were presented for SORD. Model 1 classified 3 activities, including sedentary (lying, reclining, or sitting), standing, and walking separately. As illustrated in Figure 2, model accuracy, sensitivity, and specificity for detecting sedentary time were 0.92, 0.99, and 0.87; for standing, they were 0.95, 1.00, and 0.91; and for walking, they were 0.96, 0.92, and 1.00, respectively.

Model 2 included 4 activities: sitting, reclining, standing, and walking; lying was excluded (ie, lying moments observed by video camera were omitted from the data set). As illustrated in Figure 3, model accuracy, sensitivity, and specificity for detecting sitting and reclining were 1.00, 1.00, and 1.00; for standing, they were 0.99, 0.99, and 1.00; and for walking, they were 0.98, 1.00, and 0.95, respectively.

Model 3 included 3 activities: sitting, standing, and walking; reclining and lying were excluded. Respectively, model accuracy, sensitivity, and specificity for detecting sitting were 0.97, 1.00, and 0.94; for detecting standing, they were 0.95, 0.91, and 1.00; and for walking, they were 0.98, 1.00, and 0.97 (Multimedia Appendix 1).

Model 4 included all 5 activities: lying, sitting, reclining, standing, and walking. Respectively, model accuracy, sensitivity, and specificity for detecting lying were 0.70, 0.54, and 1.00; for sitting and reclining, they were 0.85, 1.00, and 0.75; for standing, they were 0.75, 0.63, and 0.93; and for walking, they were 0.99, 1.00, and 0.98 (Multimedia Appendix 2).

For models 1 and 2, results of the Bland-Altman analysis comparing second-by-second data on sedentary, standing, and walking time between direct observation versus SORD and activPAL versus SORD are presented in Figures 4 and 5. Multimedia Appendices 3 and 4 illustrate Bland-Altman for the other models. Percentage values are presented in the text (see Figures 4 and 5 and Multimedia Appendices 3 and 4 for true values).

Mean differences (biases) between SORD model 1 and direct observation were 6.4% for sedentary (LoA=–4.3% to 17.1%), –8.7% for standing (LoA=–23.5% to 6.1%), and –8.9% for walking (LoA=–22.2% to 4.4%). Results of model 1 show wide limits, although the mean biases were below 10% for all activities. Relative to total activity durations, mean biases between SORD model 1 and activPAL were –2.5% for sedentary (LoA=–15.0% to 9.9%), 1.7% for standing (LoA=–23.3% to 26.9%), and 7.4% for walking (LoA=–2.3% to 17.1%). Results of model 1 comparing SORD to activPAL show wide limits.

Mean biases between SORD model 2 and direct observation were 0.3% for sitting and reclining (LoA=–0.3% to 0.9%), 1.19% for standing (LoA=–1.05% to 3.42%), and –4.71% for walking (LoA=–9.26% to –0.16%). Model 2 showed the narrowest LoA for “sitting and reclining,” standing, and walking, denoting excellent agreement with direct observation. All the mean biases were within ±10%. Relative to total activity durations, mean biases between SORD Model 2 and activPAL were –3.45% for sitting and reclining (LoA=–11.59% to 4.68%), 7.45% for standing (LoA=–5.04% to 19.95%), and –5.40% for walking (LoA=–11.44% to 0.64%). Results of model 2 comparing SORD to activPAL show a wider LoA, although mean biases are relatively low for “sitting and reclining” and walking.

Mean biases between SORD model 3 and direct observation were –6.4% for sitting (LoA=–18.6% to 5.7%), 12.4% for standing (LoA=–6.6% to 31.5%), and –4.9% for walking (LoA=–12.5% to 2.5%). Therefore, sitting and walking were overestimated, while standing was underestimated. The mean bias was acceptable for sitting and walking but not standing. A narrow LoA were observed for walking.

Mean biases between SORD model 4 and direct observation were 52.2% for lying (LoA=–6.9% to 111.4%), –32.0% for sitting and reclining (LoA=–78.2% to 14.1%), 48.8% for standing (LoA=–13.7% to 111.4%), and –2.2% for walking (LoA=–6.7% to 2.2%). Therefore, “sitting and reclining” and walking were underestimated while lying and standing were overestimated. Model 4 shows the broadest LoA for “sitting and reclining” and standing, while the narrowest LoA were observed for walking in this model.