This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.
The use of wearable biosensor devices for monitoring and coaching in forensic psychiatric settings yields high expectations for improved self-regulation of emotions and behavior in clients and staff members. More so, if clients have mild intellectual disabilities (IQ 50-85), they might benefit from these biosensors as they are easy to use in everyday life, which ensures that clients can practice with the devices in multiple stress and arousal-inducing situations. However, research on (continuous) use and acceptance of biosensors in forensic psychiatry for clients with mild intellectual disabilities and their caretakers is scarce. Although wearable biosensors show promise for health care, recent research showed that the acceptance and continuous use of wearable devices in consumers is not as was anticipated, probably due to low expectations.
The main goal of this study was to investigate the associations between and determinants of the expectation of usability, the actual experienced usability, and the intention for continuous use of biosensors.
A total of 77 participants (31 forensic clients with mild intellectual disabilities and 46 forensic staff members) participated in a 1-week trial. Preceding the study, we selected 4 devices thought to benefit the participants in domains of self-regulation, physical health, or sleep. Qualitative and quantitative questionnaires were used that explored the determinants of usability, acceptance, and continuous use of biosensors. Questionnaires consisted of the System Usability Scale, the Technology Acceptance Model questionnaire, and the extended expectation confirmation model questionnaire.
Only the experienced usability of the devices was associated with intended continuous use. Forensic clients scored higher on acceptance and intention for continuous use than staff members. Moderate associations were found between usability with acceptance and continuous use. Staff members showed stronger associations between usability and acceptance (
Contrary to expectations, it was the actual perceived usability of wearing a biosensor that was associated with continuous use and to a much lesser extent the expectancy of usability. Clients scored higher on acceptance and intention for continuous use, but associations between usability and both acceptance and continuous use were markedly stronger in staff members. This study provides clear directions on how to further investigate these associations. For example, whether this is a true effect or due to a social desirability bias in the client group must be investigated. Clients with mild intellectual disabilities might benefit from the ease of use of these devices and their continuing monitoring and coaching apps. For these clients, it is especially important to develop easy-to-use biosensors with a minimum requirement on cognitive capacity to increase usability, acceptance, and continuous use.
The use of wearable biosensor devices for monitoring and coaching in forensic psychiatric settings for people with intellectual disabilities and their caretakers yields high expectations for improved self-regulation of emotions and behavior. This is based on the expectation that wearable biosensor devices can be used to detect changing levels of emotional states [
A potentially complicating factor in the use of these biosensors are the mild intellectual disabilities and borderline intellectual functioning (MID-BIF; IQ 50-85) of the user. Clients with MID-BIF might not benefit from cognitive behavioral therapies (eg, anger management) like people with average intelligence [
Although wearable biosensors show promise for health care, research into the use and acceptance of wearable biosensors is almost absent in forensic psychiatry, let alone in clients with MID-BIF. Recent consumer research, however, showed that the acceptance of wearable devices in consumers is not as was anticipated [
To increase the usability, acceptance, and continuous use of biosensors, user preferences, needs, and wishes, especially for people with MID-BIF, must be known. In addition, it is necessary to determine the goals of the user and the tasks for which the biosensors will be used. Finally, the functions of user interfaces and biosensor devices should be evaluated to make them more attractive, desirable, and efficient for the users by integrating the outcomes of the evaluation [
Research on the use of biosensors for clients with MID-BIF is scarce while the potential benefits might be significant. Therefore, we investigated the use of everyday wearable biosensors to establish what would lead to their (continuous) use and acceptance. Biosensor information could potentially benefit not only clients but staff members as well. To this end, qualitative and quantitative questionnaires were used to explore the psychological (preferences, needs, wishes, and goals) and functional (tasks and functions) determinants of usability, acceptance, and continuous use of biosensors in both staff members and clients. The main goal of this study was to investigate the expectation–experience–continuous use connection to see whether there is a gap between expectations of usability and the actual experience that will preferably lead to continuous use and whether there are differences between clients and staff members. In addition, we investigated the key determinants involved in the usability, acceptance, and continuous use of biosensors using validated questionnaires. The following research questions were formulated:
Are there differences between clients and staff members in expectations of usability and the actual experienced usability that will lead to continuous use of biosensors (expectation–experience–continuous use)?
Which key determinants contribute to the usability, acceptance, and continuous use of biosensors in forensic psychiatry for clients with MID-BIF and staff members?
The participants for this small-scale study consisted of two groups of users: clients with MID-BIF who are residents of forensic psychiatric living units and staff members who work as nurses or sociotherapists on these forensic psychiatric living units. Clients are often referred to the units as a result of aggressive and violent behavior and are at an increased risk for severe behavior problems, offending behavior, and recidivism [
After multiple sessions with a user group consisting of staff members and clients, we selected 4 devices that were thought to benefit the participants in domains of self-regulation, physical health, or sleep. All 4 devices used in this study are US Food and Drug Administration and CE approved and can be bought in regular stores (commercially available).
The Spire Stone (Spire Health) is a wearable device in the form of a stone that can be attached to a belt (men) or bra (woman). It measures the contraction of the torso to indicate the rate of breathing. The device comes with an app and classifies the respiration rate as calm, focused, or tense (see Holt et al [
The Charge 3 (Fitbit Inc) is a physical activity tracker with a heart rate monitor that provides users with real-time feedback on heart rate and physical activity. In addition, it can provide users with information on sleep and exercise. The app provides users with detailed information on stress, sleep, and activity (see Schrager et al [
The vívosmart 4 (Garmin Ltd) is a physical activity tracker with a heart rate monitor that provides users with real-time feedback on energy expenditure, stress indications based on heart rate, and sleep quality assessment. The app provides users with detailed information on sleep, stress, and energy expenditure [
The TicWatch E (Mobvoi Inc) is a smartwatch running WearOS with a heart rate sensor. It can be used as a biofeedback device when running the Sense-It app [
To assess determinants of usability, we evaluated user satisfaction with the wearable biosensors [
As the questionnaires were not available in Dutch, they were translated by 3 researchers and 8 staff members who work in forensic psychiatric settings with MID-BIF clients. The questionnaires were then back-translated by native English speakers. As the formulation of the questions was deemed too complex for the MID-BIF clients, an easier version of all 3 questionnaires was constructed for MID-BIF clients consisting of fewer, more easily formulated questions. The choice of which questions to select for the short version for clients was made by the researchers and staff members based on two key principles: the question should easily be understood by the MID-BIF client and represent the implied construct to be measured.
The SUS is a 10-item questionnaire with good reliability (.85) [
To answer the first research question on the expectation–experience–continuous use connection, the questionnaire was administered twice. The SUS questionnaire was administered preceding the study to measure the expectation of the participants. Following 1 week, the SUS was administered to measure the actual experience with the biosensors.
The TAM questionnaire [
Pal et al [
The full TAM and EECM questionnaires would be too much of a burden for the clients with MID-BIF. Therefore, we selected one question from each factor on the TAM and EECM for the clients to answer. The staff members completed the full version of the questionnaire. For ease of reporting and interpretation for both clients and staff members, the results reported in this paper consist of the SUS, the short version of the TAM, and the short version of the EECM (
The qualitative questionnaires consisted of an individually administered semistructured interview based on the quantitative questionnaires. Participants were asked to elaborate on thoughts they had on the aspects of usability, acceptance, and continuous use. Additional determinants of usability, acceptance, and continuous use were derived from the semistructured interview in order to further explore the second research question.
The research was conducted from May to August 2019. Recruitment of the participants was done at the sites of the living units. Participants were invited to participate and informed about the aim of the study through posters, flyers, and email. After a participant agreed to participate in the study, wearable biosensors were given to the participants with instructions on how to use them. If a user did not own a phone to connect to the app, a P smart (Huawei Device Co Ltd) was provided to the participant (although some participants were not allowed to use a phone due to the nature of their sentence and only used the app in the presence of their caretakers). One of four commercially available devices was randomly assigned to the participant. Before wearing the device, they completed the SUS questionnaire to assess their expectations of usability. The research coordinator completed the SUS questionnaire with the participant if necessary. Sheehan and Hassiotis [
The participants were given time to get familiar with the biosensors. The research coordinator functioned as a contact person in case the participant had any technical problems [
Descriptive statistics were used to describe the devices that were worn and the age, education, and gender of participants. A 2-way mixed analysis of variance (within: pre-post and between: client-staff) was used to test the main outcome on the expectation of usability with the actual experience. The SUS scores were calculated both preassessment and postassessment for each participant to determine their correlation with continuous use to answer the first research question. To test whether there was an association between the expected and experienced usability with (the intended) continuous use of the EECM questionnaire, we used Spearman correlations to answer the second research question, as the scores on the SUS were not normally distributed. To determine the key determinants that contribute to the usability, acceptance, and continuous use of biosensors, the proportion and number of responses for all questionnaires was computed for clients and staff members.
Further exploratory statistical analyses consisted of a 2-way analysis of covariance (ANCOVA) to test which demographic factors were associated with the judgments of the participants concerning the usability, acceptance, and continuous use of biosensors. Last, an analysis was performed on the qualitative questions of usability, acceptance, and continuous use regarding word frequency. All analyses were done in R (version 3.6.1, R Foundation for Statistical Computing) software [
To investigate whether there were differences between clients and staff members in expectations of usability and the actual experienced usability that will lead to continuous use, 77 participants were included (31 clients and 46 staff members), with an age range varying from 18 to 63 (mean 34.9 [SD 10.8]) years. Participants were included from 4 mental health institutions in the Netherlands that provide forensic care for clients with MID-BIF.
Participants wore a Charge 3 (31/77), vívosmart 4 (21/77), Spire Stone (14/77), or TicWatch E (11/77;
Descriptive statistics.
Participant | Client (n=31), n (%) | Staff (n=46), n (%) | |||
|
|||||
|
Charge 3 | 16 (52) | 15 (33) | ||
|
vívosmart 4 | 7 (23) | 14 (30) | ||
|
Spire Stone | 3 (10) | 11 (24) | ||
|
TicWatch E | 5 (16) | 6 (13) | ||
|
|||||
|
Male | 20 (65) | 21 (46) | ||
|
Female | 10 (32) | 25 (54) | ||
|
|||||
|
Primary | 16 (52) | 0 (0) | ||
|
Secondary | 13 (42) | 17 (37) | ||
|
Higher | 0 (0) | 28 (61) |
A small proportion of clients with MID-BIF were not allowed to use a mobile phone (6/77). They were therefore unable to answer questions regarding the use of the biosensor in combination with the app. The missing values were therefore imputed with the “don’t know or neutral” categories of the questionnaires.
Descriptive statistics of System Usability Scale scores.
Group | Product Start | n | Start | SD | End | SD end | Min start | Max start | Min end | Max end |
Client | Charge 3 | 16 | 56.88 | 20.01 | 60.31 | 18.66 | 15.00 | 90.00 | 15.00 | 82.50 |
Client | vívosmart 4 | 7 | 66.43 | 16.00 | 67.86 | 17.82 | 45.00 | 87.50 | 47.50 | 95.00 |
Client | Spire Stone | 3 | 64.17 | 10.10 | 63.33 | 14.65 | 55.00 | 75.00 | 52.50 | 80.00 |
Client | TicWatch E | 5 | 51.50 | 8.59 | 57.50 | 27.33 | 37.50 | 60.00 | 20.00 | 82.50 |
Staff | Charge 3 | 15 | 69.17 | 10.42 | 75.50 | 12.00 | 55.00 | 87.50 | 55.00 | 92.50 |
Staff | vívosmart 4 | 14 | 74.11 | 5.24 | 76.25 | 8.31 | 65.00 | 82.50 | 65.00 | 92.50 |
Staff | Spire Stone | 11 | 64.77 | 6.56 | 59.77 | 20.05 | 55.00 | 77.50 | 12.50 | 75.00 |
Staff | TicWatch E | 6 | 70.83 | 16.93 | 44.58 | 13.73 | 50.00 | 97.50 | 30.00 | 70.00 |
The mean for the clients increased over time while the mean for the staff decreased (
The correlation between the expected usability and the EECM was .18 (
Descriptive statistics of System Usability Scale scores per group.
Group | Time | Variable | n | Mean (SD) |
Client | Start | Score | 31 | 58.87 (17.19) |
Staff | Start | Score | 46 | 69.84 (9.77) |
Client | End | Score | 31 | 61.85 (19.09) |
Staff | End | Score | 46 | 67.94 (17.45) |
As shown in
Proportion and number of responses for clients on the System Usability Scale questionnaire.
Proportion and number of responses for staff on the System Usability Scale questionnaire.
Proportion and number of responses for clients on the Technology Acceptance Model questionnaire.
Proportion and number of responses for staff on the Technology Acceptance Model questionnaire.
Proportion of responses and number of responses for clients on the extended expectation confirmation model.
Proportion of responses and number of responses for staff on the extended expectation confirmation model.
We explored which demographic variables contributed to the usability (SUS), acceptance (TAM), and continuous use (EECM) of biosensors. We included age, level of education, and gender in separate 2-way ANCOVAs as the dependent variables were correlated, which prohibited a multifactorial multivariate analysis of covariance. In addition, we performed a Box-Cox transformation on the SUS and TAM as these were not normally distributed; the ANCOVAs were adjusted for age. There was a significant difference for both acceptance (F1,69=9.214,
Spearman correlations between variables.
Questionnaire | SUSa | EECMb | TAMc |
EECM | 0.54 | —d | — |
TAM | 0.58 | 0.86 | — |
Age | –0.24 | –0.15 | –0.03 |
aSUS: System Usability Scale.
bEECM: extended expectation confirmation model.
cTAM: Technology Acceptance Model questionnaire.
dNot applicable.
For analysis of qualitative questionnaires, a sample size between 5 and 50 is required [
Word frequencies for the Spire Stone and TicWatch E.
In this study, we investigated whether the expectancy or the actual experience was most important for an intended continuous use of biosensor devices for monitoring and coaching in forensic psychiatry. In addition, we investigated what contributes to the usability, acceptance, and intended continuous use. The main result of the study is that it was the actual experience of wearing a biosensor that was associated with intended continuous use, and to a much lesser extent, the expectancy. This is contrary to the hypotheses of Pal et al [
In addition, a strong association between the acceptance (TAM) of wearable devices and the intention of continuous use (EECM) was found. This seems to indicate that these two questionnaires measure overlapping constructs, and the question arises whether both need to be administered. Especially when the load on participants should be kept to a minimum as in our sample with forensic MID-BIF clients. These results must be interpreted with care as the design of our study, without proper counterbalancing, and the use of short questionnaires limit the conclusions that can be drawn from the study.
As far as the determinants for usability, acceptance, and continuous use are concerned, answers from the usability scale indicated that most of the clients and staff members felt confident using the biosensors and after they wore the devices and thought that most people would learn to use the product very quickly and want to use it frequently. The acceptance scale indicated that the majority have positive attitudes toward technology, their affective quality, relative advantage, mobility, availability, and perceived ease of use. The continuous use scale showed that the majority of staff members and clients gave positive answers on satisfaction, self-socio motivation, perceived comfort, and hedonic motivation. However, the majority had doubts on the perceived accuracy and functional limitations. It is also interesting to note that a minority of staff members and clients were not positive about the usability, acceptance, and continuous use of the devices. These people might not want to use the devices or think that they need help in using the devices. For instance, a minority of people think that they would need help from a technical person to use the device. It might well be that providing them with proper support might increase their intention to use the device. Also, some find that wearing the device is uncomfortable and the accuracy of the fitness data could be improved. These devices might thus benefit from developments in accuracy and form factor [
Two particular strengths of this study are the use of simply worded questionnaires adapted for clients with MID-BIF as this was not available in the literature. In addition, we used qualitative questionnaires and a diverse and heterogeneous sample as Kalantari [
Future research should focus on longitudinal research investigating usability, acceptance, and continuous use, should include a counterbalanced design in which all devices are worn at least once, and should investigate measurement invariance for the short questionnaires [
Another important topic for future research is the reliability and validity of the sensors, especially in comparison with gold standard equipment used in laboratories. Peake et al [
Last, for people with MID-BIF, it is especially important to develop easy-to-use biosensors with a minimum requirement on cognitive capacity to increase usability, acceptance, and continuous use in the future. It must be noted, however, that clients scored similar to staff members on ease of use of available devices and higher on acceptance and (intended) continuous use. Whether clients indeed grasped the information provided by the sensors must be investigated further.
Actual perceived usability of wearing a biosensor and to a much lesser extent the expectancy of usability were associated with continuous use. Clients with mild intellectual disabilities might benefit from the ease of use of wearables devices and their continuing monitoring and coaching apps. Clients scored higher on acceptance and intention for continuous use, but associations between usability and both acceptance and continuous use were markedly stronger in staff members. For clients, it is especially important to develop easy-to-use biosensors with a minimum requirement on cognitive capacity to increase usability, acceptance, and continuous use.
Short questionnaires used in the study.
analysis of covariance
extended expectation confirmation model
mild intellectual disabilities and borderline intellectual functioning
System Usability Scale
Technology Acceptance Model questionnaire
We would like to thank the project partners of the Sense-IT project for making the Sense-IT available for use in this study: University of Twente, Scelta/GGNet, VUmc, Arkin, and Pluryn.
None declared.