This section details the end-to-end development of the proposed SRS and how its performance was evaluated (Figure 1). The first part describes qualitative aspects of the software system, whereas the second part describes quantitative aspects in terms of system properties. The “Nonfunctional Requirements” section specifies the properties and characteristics of the developed system’s software architecture. The “Software Architecture” section describes the system’s components, how they are structured, and how they interact with each other. The “Field Experiments” section describes the acquisition of data to evaluate the SRS system, which involves running versions of the proposed software, referred to as instances. In the “System Properties” section, a documentation of the metrics to evaluate the performance of the system properties is provided.

The technical aspects of a software system (ie, its technical properties and constraints) can be described in terms of nonfunctional requirements. In the context of software engineering, nonfunctional requirements are defined as the constraints imposed on a system that specify the quality attributes of the software [33]. These requirements therefore affect how the software architecture of a system is designed and implemented. To design the software architecture of the SRS, the nonfunctional requirements and attributes in Textbox 1 were considered.

A total of 4 SRS instances, hereafter referred to as instance 1, instance 2, instance 3, and instance 4, were used and evaluated in 3 observational studies. These studies will be broadly elaborated on to highlight how the SRS system can be used and the associated preliminary findings will be shown.

First, instance 1 was used in a study, hereafter referred to as study 1 (n=67 for 21 days). In this study, participants had to perform various gait-related tasks in an instrumented apartment. The purpose of this study was to create a medical data set that can be used to develop novel algorithms to extract gait parameters. Second, instance 2 and instance 3 were deployed in a study, hereafter referred to as study 2 (n=43 for 20 days), in which the system was expanded to work as a mobile and portable SRS to act as a mobile gait analyzer and to track activities of daily living (ADL). The purpose of this study was to analyze these activities in a population of healthy participants using different sensors. Notably, the results of both studies, study 1 and study 2, have yet not been published. Lastly, instance 4 was used in a pilot case study, hereafter referred to as study 3 (n=4 for 30 days), in which, Gerber et al [65] measured ADL during a 12-hour overnight stay in an instrumented apartment. The purpose of this use case study was to demonstrate that the proposed system can be used to continuously assess digital biomarkers (eg, gait parameters and ADL) during a prolonged period and therefore represents a reliable measuring method for conducting future clinical studies in the apartment (eg, measure the change in motor functions in patients with Parkinson disease or multiple sclerosis).

Study-specific demographic information, a list of installed sensors, and the hardware specifications of each SRS instance used in each study can be found in Table 2. All sensor systems were assembled using consumer-grade hardware, without the need for any connection to the internet.

To date, a variety of different sensors have been tested and integrated into the SRS system, demonstrating the flexibility of the sensor interface. The sensors used to generate the data sets (Table 2) were Doppler radars, seismographs, LIDARs, magnetic door sensors, a motion tracking camera system, a pressure mattress for tracking activity in bed, an infrared camera, night and day cameras, passive infrared sensors, a microphone system, devices for measuring power consumption, and environmental sensors (eg, temperature, humidity, and water flow sensors). These sensors have been shown to be particularly useful in biomedical research concerning the analysis of motor and nonmotor functions [65,66] and were thus integrated into the SRS system. The performance of the SRS was statistically evaluated based on the data collected through deployed instances used in the studies. Finally, to demonstrate that the SRS has a wide area of application with respect to different hardware configurations, the software was tested on a Raspberry Pi 4 in combination with 3 radar sensors. This SRS instance was run for 24 hours with its radars configured to send 1 data packet per second from the sensors to the backend. In this proof-of-concept test, only the error rates of incoming packets and the CPU utilization were analyzed.

The referred case studies were conducted according to the guidelines of the Declaration of Helsinki and approved by the Ethics Committee of the Canton of Bern, Switzerland (Kantonale Ethik Kommission [KEK] numbers 2020-02771, 2021-00965, and 2021-01420).

This section details the statistical aspects of the analyzed system properties and experimental findings of the SRS on a Raspberry Pi 4. Next, quantitative findings in the form of the statistical outcomes of the evaluated system properties (reliability, throughput, delay, and usability) are presented. Finally, the results of a proof-of-concept 24-hour measurement performed with a Raspberry Pi 4 are demonstrated. A summary of our main quantitative findings, together with reference values from the literature, is presented in Table 3.

HTML response codes were logged and analyzed for more than 420 days to determine the reliability performance (Table 4). In total, more than 320 million requests were sent across all instances, with an average rate of approximately 9 requests per second. A total of 7342 errors occurred, and thus, the average and maximum error percentages were low (0.002% and 0.012%, respectively). Both error percentages were within the threshold for an acceptable error rate (0.02%) [68].

During our experiments, a total of 19.8 million LIDAR packets were processed by all instances of the SRS. Their internal processing times are listed in Table 5. The mean package processing time for all instances was 25.09 (SD 0.15) ms.

Test results of GET and POST throughput experiments are shown in Figure 6. For both request types, across all experiments, the average request time was short, ranging between 4.72 and 8.30 ms (mean 5.98 ms, SD 1.82 ms). Furthermore, independent of the instance and request type, on average, as the number of requests increased, the SD of the throughput rate increased (test A: SD 48.00 ms; test B: SD 3.84 ms; test C: SD 2.39 ms). Similarly, the average maximum throughput rate increased (test A: maximum 60.49 ms; test B: maximum 54.00 ms; test C: maximum 25.90 ms). Both request types reported very similar and low average minima (GET: minimum 3.28 ms; POST: minimum 3.29 ms). The average and maximum request times (5.98 and 60.49 ms, respectively) were below the threshold of an instantaneously reacting system (100 ms) [71]. For all instances, the median was very close to the corresponding mean, with an average difference of 0.67 ms. There was no significant difference (t₄₈₀=–0.0571; P=.045) in throughput between GET (mean 5.97 ms, SD 3.64 ms) and POST (mean 5.99 ms, SD 3.50 ms). No packet losses were observed during the testing.

The evaluation of instance latency was realized via box plot representations of long-term delay measurements (Figure 7). Across all instances, the average delay was low (between 9.9 and 11.6 ms, mean 10.7 ms, SD 2.0 ms). Instance 3 exhibited the lowest delay (mean 9.55, SD 1.33 ms), and instance 4 exhibited the highest delay (mean 12.21 ms, SD 2.18 ms). The average delay across all instances (10.88 ms) was below the definition of a low-latency system (100 ms) [73]. Moreover, there were no significant differences in delay between the instances (t_33,600=–1.54 × 10^–15; P<.001). Finally, few outliers were reported, and no packet losses were observed during the testing.

The system usability was assessed by analyzing the responses of 10 participants to the SUS and PSSUQ (Figure 8). The mean participant SUS score yielded an excellent (>80.3) usability rating [74], with scores between 80.0 and 97.5 (mean 89.5, SD 4.8). The mean overall PSSUQ score (mean 1.62, SD 0.36), as well as its system usability (mean 1.35, SD 0.27), information quality (mean 1.80, SD 0.54), and interface quality (mean 1.90, SD 0.63) scores attained an acceptable level (overall: <2.62) [75,76].

According to the participants’ qualitative feedback, the installation script was helpful and simple to use. By contrast, the experiment UI could be made easier to use by adding colors to functional components, such as buttons and drop-down menus.

The SRS was installed on a Raspberry Pi 4 in combination with 3 radar sensors. The resulting system was run for 24 hours. In the measurement, only reliability aspects, along with CPU usages, were analyzed. The system did not report any errors during the measurement. In total, more than 250,000 packets were processed under high load (CPU usage: mean 95%). No packets were lost during the measurement.

In this paper, we reported the development, functional components, and performance evaluation of a novel lightweight, open-source, and cost-efficient software system. This system was built to enable sensor data collection that offers both cloud and on-premises deployment. In line with our objective, the SRS can be reliably used to integrate multiple sensors on consumer-grade hardware to perform offline measurements in clinical studies. This was shown by analyzing different performance metrics applied to various long-term measurements.

To demonstrate that the SRS helps to address technical challenges that arise during multimodal sensor measurements (eg, unsynchronized sensor signals), the following three utility programs were developed and integrated: (1) a timestamp-annotation program to mark events that are synchronized across measurements; (2) resource monitoring software, called the TICK stack, to alert administrators to the presence of hardware errors related to the server, nonfunctioning sensors, or network problems; and (3) a timestamp analyzer to detect and understand delays in the recorded data due to potential sensing device issues.

With respect to the quantitative findings, it was found that the SRS system operated with an acceptable error rate while running at a high request rate. This was determined by analyzing the error rates of more than 320 million requests transmitted over 420 days and relying on Horner’s definition of acceptable error rates [68]. It is noteworthy that most errors arose due to problems in the power supply to the system caused by a power outage that lasted several hours. Moreover, a long-term measurement of the internal packet processing time showed a low deviation in time. This indicates that the SRS system was internally processing data packets at a stable rate. Those two findings are in line with our objective, which was to make the internal system operate reliably.

Next, the throughput performance of the system was examined. The results demonstrate that, regardless of the request type, the SRS was capable of efficiently functioning at a high request rate (2000 requests per second) on consumer-grade hardware. Based on the literature, the results meet the criteria of an instantly responding system [71]. This means that the SRS system can handle high-frequency data streams in real-time and on consumer-grade hardware, which makes the system promising for monitoring. Furthermore, for all the systems and request types, the average request time was determined to be close to the median request time. This is an indicator of stable request processing, suggesting that the throughput rate was stable over a prolonged period. This processing property leads to the conclusion that the throughput rate of serving incoming API requests was stable over a prolonged period.

In addition to running stably, the delay results indicate that the SRS corresponds to a low-latency system, as defined by Jiang et al [73], which implies that the network topology does not affect the transmission rates between the sensors and the SRS. Furthermore, the average SD was small, which indicates that the latency was stable over time. Both findings imply that the SRS did not introduce any significant delay during the sensor recordings and, thus, did not affect the measurement process.

The overall usability was assessed, including through the rating of the complexity of setting up an SRS instance and its applicability for measurements. An SUS score indicating excellent usability was achieved [74], suggesting that the system is highly usable. Moreover, according to the results of the PSSUQ, the usefulness, information quality, and interface quality of the SRS system are at acceptable levels [75], indicating that the UI is easy to use and exhibits helpful information pertaining to its use. Furthermore, the results of the 2 usability tests show that, in line with the initial objective, the overall usability of the SRS is effective, efficient, and satisfactory. The participants particularly appreciated that the SRS can be installed via Docker, as it simplified the installation effort. However, a major weakness identified by the participants was the unintuitive nature of the UI components. If this issue is eliminated, the overall usability of the interface could be improved.

Finally, to demonstrate the SRS’s broad integration capability, it was operated on a Raspberry Pi 4, in combination with several integrated sensor devices, over a period of 24 hours. During this period, no packets were lost, and no errors were reported in the logs. However, due to the exploratory nature of this test, no other system properties were analyzed. Nevertheless, this finding indicates that the SRS could be a promising software solution with which to conduct biomedical research on cost-efficient consumer-grade hardware as basic as a Raspberry Pi 4.

The SRS system offers a simple and cost-efficient solution for on-premises deployment that requires only minimal technical expertise and provides high usability. The SRS may provide numerous benefits for researchers conducting early stage research, or working with limited resources, compared with cloud-based solutions [23,24,27] that require on-premises deployment and normally face economic, technical, and regulatory challenges. This is mainly because the SRS runs stably on a variety of consumer-grade hardware, even as basic as that on a Raspberry Pi 4. It is also due to the fact that the system supports a simple mechanism for integrating multiple and diverse high-frequency data streams from various sensing devices. The proposed system is therefore a feasible option for long-term measurements that takes into account biomedical research requirements, such as privacy protection. Integrating Docker in the system may help users of the SRS replicate studies as the utilized containers consist of replicable configurations (ie, the source code of the executed program, the execution order in form of a Dockerfile, and defined environment variables). Furthermore, by providing a digest of the data stored in the database of a particular study (eg, by exporting a database dump or hashing the persisted data), users of the SRS are not only able to replicate that study, but also able to reproduce it. Additional aspects of the system enable straightforward deployment and enhanced maintainability while easing its operationalization (ie, integration and management). It further promotes proper error handling, which reduces the complexity of the run-time diagnostics used to determine the cause and extent of errors. Consequently, this lightweight recording software system allows researchers to focus resources on research questions rather than on developing technology.

Although it was successful in reaching its desired objective, the SRS system still has some limitations. For example, although a variety of different hardware combinations were tested, the extent to which these results can be extrapolated to low-cost hardware (eg, ESP32 [Espressif Systems 32 Microcontroller] microcontrollers) is not clear. From a performance perspective, this becomes particularly challenging when integrating many sensors, as the hardware specifications of low-cost systems may be too low to handle a large number of incoming requests. Furthermore, although the SRS achieved high scores for usability, the generalizability of these results is limited by the low number of participants. Another usability hurdle is the integration of new sensor types, as this currently requires adherence to interface specifications, which can be tedious.

To expand the system for applications beyond clinical needs and to mitigate potential scaling issues, the SRS could be extended to scale horizontally by operating as a distributed service in the cloud. Future studies could investigate how, and to what extent, the SRS system could be extended to properly scale in multiple clinical settings, in which many SRS instances might be running simultaneously while sharing a central database. In addition, the performance could be improved using streaming technologies (eg, broker-based solutions). To benchmark the system, the methods used to measure the performance could be further improved by correlating the measurements with more system resources (eg, by comparing the throughput with the CPU and RAM usage). Finally, the sensor integration could be simplified by relying on standardized serialization mechanisms to represent structured data (eg, Google’s Protocol Buffers). This would reduce the effort to integrate new sensor devices even further.

Despite the significant potential of these new digital technologies, adopting sensor-enhanced biomedical solutions is not simple due to the complex and expensive nature of most existing SRS. This paper presents a lightweight software recording system for integrating arbitrary sensing devices on consumer-grade hardware to perform reliable long-term measurements. The system has significant potential to address the economic, technical, and data regulatory challenges associated with earlier systems, thereby enabling sensor measurements and objective assessments in the realm of biomedical research. Moreover, the system facilitates the testing of sensor systems, as well as the development and validation of algorithms for the extraction of digital measures. Overall, this allows researchers to focus on their research questions rather than on developing the technology needed to collect their data.