Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?


Journal Description

JMIR Formative Research publishes peer-reviewed, openly accessible papers containing results from process evaluations, feasibility/pilot studies and other kinds of formative research and preliminary results. While the original focus was on the design of medical- and health-related research and technology innovations, JMIR Formative Research publishes studies from all areas of medical and health research.

Formative research is research that occurs before a program is designed and implemented, or while a program is being conducted. Formative research can help

  • define and understand populations in need of an intervention or public health program
  • create programs that are specific to the needs of those populations
  • ensure programs are acceptable and feasible to users before launching
  • improve the relationship between users and agencies/research groups
  • demonstrate the feasibility, use, satisfaction with, or problems with a program before large-scale summative evaluation (looking at health outcomes)

Many funding agencies will expect some sort of pilot/feasibility/process evaluation before funding a larger study such as a Randomized Controlled Trial (RCT).

Formative research should be an integral part of developing or adapting programs, and should be used while the program is ongoing to help refine and improve program activities. Thus, formative evaluation can and should also occur in the form of a process evaluation alongside a summative evaluation such as an RCT.

This journal fills an important gap in the academic journals landscape, as it publishes sound and peer-reviewed formative research that is critical for investigators to apply for further funding, but that is usually not published in outcomes-focused medical journals aiming for impact and generalizability.

Summative evaluations of programs and apps/software that have undergone a thorough formative evaluation before launch have a better chance to be published in high-impact flagship journals; thus, we encourage authors to submit - as a first step - their formative evaluations in JMIR Formative Research (and their evaluation protocols in JMIR Research Protocols). 

JMIR Formative Research has been accepted for indexing in PubMed and PubMed Central.


Recent Articles:

  • Copilot: an mHealth intervention to enhance exacerbation-related self-management in patients with COPD. Source: Panton B.V. Deventer; Copyright: Panton B.V. Deventer; URL:; License: Licensed by the authors.

    Early-Stage Feasibility of a Mobile Health Intervention (Copilot) to Enhance Exacerbation-Related Self-Management in Patients With Chronic Obstructive...


    Background: There is an emergence of mobile health (mHealth) interventions to support self-management in patients with chronic obstructive pulmonary disease (COPD). Recently, an evidence-driven mHealth intervention has been developed to support patients with COPD in exacerbation-related self-management: the Copilot app. Health care providers (HCPs) are important stakeholders as they are the ones who have to provide the app to patients, personalize the app, and review the app. It is, therefore, important to investigate at an early stage whether the app is feasible in the daily practice of the HCPs. Objective: The aim of this study is to evaluate the perceived feasibility of the Copilot app in the daily practice of HCPs. Methods: A multimethods design was used to investigate how HCPs experience working with the app and how they perceive the feasibility of the app in their daily practice. The feasibility areas described by Bowen et al were used for guidance. HCPs were observed while performing tasks in the app and asked to think aloud. The System Usability Scale was used to investigate the usability of the app, and semistructured interviews were conducted to explore the feasibility of the app. The study was conducted in primary, secondary, and tertiary care settings in the Netherlands from February 2019 to September 2019. Results: In total, 14 HCPs participated in this study—8 nurses, 5 physicians, and 1 physician assistant. The HCPs found the app acceptable to use. The expected key benefits of the app were an increased insight into patient symptoms, more structured patient conversations, and more tailored self-management support. The app especially fits within the available time and workflow of nurses. The use of the app will be influenced by the autonomy of the professional, the focus of the organization on eHealth, costs associated with the app, and compatibility with the current systems used. Most HCPs expressed that there are conditions that must be met to be able to use the app. The app can be integrated into the existing care paths of primary, secondary, and tertiary health care settings. Individual organizational factors must be taken into account when integrating the app into daily practice. Conclusions: This early-stage feasibility study shows that the Copilot app is feasible to use in the daily practice of HCPs and can be integrated into primary, secondary, and tertiary health care settings in the Netherlands. The app was considered to best fit the role of the nurses. The app will be less feasible for those organizations in which many conditions need to be met to use the app. This study provides a new approach to evaluate the perceived feasibility of mHealth interventions at an early stage and provides valuable insights for further feasibility testing.

  • Source: Pexels; Copyright: Julia M Cameron; URL:; License: Licensed by JMIR.

    Technology-Enabled Solutions for Australian Mental Health Services Reform: Impact Evaluation


    Background: Health information technologies (HITs) are becoming increasingly recognized for their potential to provide innovative solutions to improve the delivery of mental health services and drive system reforms for better outcomes. Objective: This paper describes the baseline results of a study designed to systematically monitor and evaluate the impact of implementing an HIT, namely the InnoWell Platform, into Australian mental health services to facilitate the iterative refinement of the HIT and the service model in which it is embedded to meet the needs of consumers and their supportive others as well as health professionals and service providers. Methods: Data were collected via web-based surveys, semistructured interviews, and a workshop with staff from the mental health services implementing the InnoWell Platform to systematically monitor and evaluate its impact. Descriptive statistics, Fisher exact tests, and a reliability analysis were used to characterize the findings from the web-based surveys, including variability in the results between the services. Semistructured interviews were coded using a thematic analysis, and workshop data were coded using a basic content analysis. Results: Baseline data were collected from the staff of 3 primary youth mental health services (n=18), a counseling service for veterans and their families (n=23), and a helpline for consumers affected by eating disorders and negative body image issues (n=6). As reported via web-based surveys, staff members across the services consistently agreed or strongly agreed that there was benefit associated with using technology as part of their work (38/47, 81%) and that the InnoWell Platform had the potential to improve outcomes for consumers (27/45, 60%); however, there was less certainty as to whether their consumers’ capability to use technology aligned with how the InnoWell Platform would be used as part of their mental health care (11/45, 24% of the participants strongly disagreed or disagreed; 15/45, 33% were neutral; and 19/45, 42% strongly agreed or agreed). During the semistructured interviews (n=3) and workshop, participants consistently indicated that the InnoWell Platform was appropriate for their respective services; however, they questioned whether the services’ respective consumers had the digital literacy required to use the technology. Additional potential barriers to implementation included health professionals’ digital literacy and service readiness for change. Conclusions: Despite agreement among participants that HITs have the potential to result in improved outcomes for consumers and services, service readiness for change (eg, existing technology infrastructure and the digital literacy of staff and consumers) was noted to potentially impact the success of implementation, with less than half (20/45, 44%) of the participants indicating that their service was ready to implement new technologies to enhance mental health care. Furthermore, participants reported mixed opinions as to whether it was their responsibility to recommend technology as part of standard care.

  • Used with permission from the Mayo Foundation for Medical Education and Research. Source: Mayo Clinic Foundation for Medical Education and Research; Copyright: Mayo Clinic Foundation for Medical Education and Research; URL:; License: Licensed by the authors.

    A Cardiovascular Health and Wellness Mobile Health Intervention Among Church-Going African Americans: Formative Evaluation of the FAITH! App


    Background: In light of the scarcity of culturally tailored mobile health (mHealth) lifestyle interventions for African Americans, we designed and pilot tested the Fostering African-American Improvement in Total Health (FAITH!) App in a community-based participatory research partnership with African American churches to promote cardiovascular health and wellness in this population. Objective: This report presents the results of a formative evaluation of the FAITH! App from participants in an intervention pilot study. Methods: We included 2 semistructured focus groups (n=4 and n=5) to explore participants’ views on app functionality, utility, and satisfaction as well as its impact on healthy lifestyle change. Sessions were audio-recorded and transcribed verbatim, and qualitative data were analyzed by using general inductive analysis to generate themes. Results: In total, 6 overarching themes emerged among the 9 participants: overall impression, content usefulness, formatting, implementation, impact, and suggestions for improvement. Underpinning the themes was a high level of agreement that the intervention facilitated healthy behavioral change through cultural tailoring, multimedia education modules, and social networking. Suggestions for improvement were streamlining the app self-monitoring features, prompts to encourage app use, and personalization based on individuals’ cardiovascular risk. Conclusions: This formative evaluation found that the FAITH! App had high reported satisfaction and impact on the health-promoting behaviors of African Americans, thereby improving their overall cardiovascular health. Further development and testing of the app among African Americans is warranted. Trial Registration: NCT03084822;

  • Source: Delmaine Donson; Copyright: iStock Photo; URL:; License: Licensed by the authors.

    Adaptation and Evaluation of a Symptom-Monitoring Digital Health Intervention for Patients With Relapsed and Refractory Multiple Myeloma: Pilot Mixed-Methods...


    Background: Relapsed and refractory multiple myeloma (RRMM) is a bone marrow cancer that requires systemic treatment, which often results in severe symptom burden. Recent studies have found that electronic patient-reported outcome (ePRO) interventions implemented in the clinic setting have had positive outcomes for other oncology populations. Evidence of the efficacy of a similar approach is lacking for patients with RRMM. Objective: Recent recommendations for digital health interventions call for the publication of descriptions of iterative development processes in order to improve reproducibility and comparability. This study is an implementation pilot aiming to evaluate the acceptability and appropriateness of an ePRO intervention for patients with RRMM and to explore its impact on clinic workflow. Methods: A total of 11 patients with RRMM were recruited from the John Theurer Cancer Center in Hackensack, New Jersey. Patients used a mobile app to report on 17 symptoms at 4 sessions, each a week apart. Patients could also report symptoms ad hoc. When reports met predefined thresholds, the clinic was alerted and patients received automated guidance. Study end points were assessed using qualitative and quantitative methods. Results: A total of 9 patients (mean age 69.7 years) completed the study. Overall, 83% (30/36) of weekly sessions were completed. Patients found the frequency and time required to complete reporting acceptable. All patients agreed that the app was easy to use and understand. Providers felt the alerts they received required refinement. Patients and providers agreed it would be beneficial for patients to report for longer than 4 weeks. Patients felt that the training they received was adequate but contained too much information for a single session. All patients found the symptoms tracked to be appropriate; providers suggested shortening the list. All patients understood how to use the app for weekly reporting but had confusion about using it ad hoc. Providers felt the ad hoc feature could be removed. Neither patients nor providers viewed the in-app data reports but agreed on their potential value. Patients reported benefitting from symptom reporting through increased awareness of their symptoms. Clinic staff reported that app alerts were too numerous and redundant. They had difficulty responding to alerts within their existing workflow, partially because the data were not integrated into the electronic medical record system. Conclusions: Overall, the intervention was found to be acceptable and appropriate for patients with RRMM. Points of friction integrating the intervention into the clinic workflow were identified. Clinic staff provided recommendations for addressing these issues. Once such modifications are implemented, ePRO data from patients with RRMM could be used to inform and improve clinical research and care. This study underlines the importance of an iterative approach to implementation that includes all stakeholders in order to ensure successful adoption.

  • Source: Unsplash; Copyright: yunmai; URL:; License: Licensed by JMIR.

    Arabic Translation of the Weight Self-Stigma Questionnaire: Instrument Validation Study of Factor Structure and Reliability


    Background: While it is most often associated with its effects on physical health, obesity is also associated with serious self-stigmatization. The lack of a suitable, validated tool to measure weight-related self-stigma in Arabic countries may be partly responsible for the scarcity of literature about this problem. Objective: This study investigated the reliability and validity of an Arabic version of the Weight Self-Stigma Questionnaire (WSSQ). Methods: Data on the Arabic-translated version of the 12-item WSSQ were collected using two cross-sectional electronic questionnaires distributed among Saudi nationals through the Sharik Association for Health Research’s database in June 2020. Internal consistency, test-retest reliability, and exploratory factor analysis of the Arabic WSSQ were assessed and compared with the original English version and other translations. Results: For reliability analysis, 43 participants completed the Arabic WSSQ during two time periods. Internal consistency was α=.898 for the overall survey, α=.819 for the fear of enacted stigma subscale (factor 1), and α=.847 for the self-devaluation subscale (factor 2). The test-retest reliability of the intraclass correlation coefficient was α=.982. In the factor structure analysis, 295 participants completed the questionnaire. The Arabic WSSQ loading of the items was consistent with the original WSSQ, except for the loading of item 9, which was stronger in factor 2 than in factor 1. The two factors accounted for the observed variances of 47.7% and 10.6%. Conclusions: The Arabic version of the WSSQ has good internal consistency and reliability, and the factorial structure is similar to that of the original WSSQ. The Arabic WSSQ is adaptable for clinicians seeking to assess weight-related self-stigma in Arabic-speaking people.

  • Source:, Inc; Copyright:, Inc; URL:; License: Licensed by the authors.

    Use of the Consumer-Based Meditation App Calm for Sleep Disturbances: Cross-Sectional Survey Study


    Background: Over 30% of Americans report regular sleep disturbance, and consumers are increasingly seeking strategies to improve sleep. Self-guided mindfulness mobile apps may help individuals improve their sleep. Despite the recent proliferation of sleep content within commercially available mindfulness apps, there is little research on how consumers are using these apps for sleep. Objective: We conducted a cross-sectional survey among subscribers to Calm, a popular, consumer-based, mindfulness-based meditation app, and described and compared how good sleepers, poor sleepers, and those with self-reported insomnia use the app for sleep. Methods: Participants who were paying subscribers of Calm and had used a sleep component of Calm in the last 90 days were invited to complete an investigator-developed survey that included questions about sleep disturbance and the use of Calm for sleep. Based on self-reports of sleep disturbances and of insomnia diagnosis, participants were categorized as “good sleepers,” “poor sleepers,” or “those with insomnia diagnosis.” Chi-square tests compared reasons for downloading the app and usage patterns across participants with and without sleep disturbance. Results: There was a total of 9868 survey respondents. Approximately 10% of participants (1008/9868, 10.21%) were good sleepers, 78% were poor sleepers (7565/9868, 77.66%), and 11% reported a diagnosis of insomnia (1039/9868, 10.53%). The sample was mostly White (8185/9797, 83.55%), non-Hispanic (8929/9423, 94.76%), and female (8166/9578, 85.26%). The most common reasons for sleep disturbances were racing thoughts (7084/8604, 82.33%), followed by stress or anxiety (6307/8604, 73.30%). Poor sleepers and those with insomnia were more likely than good sleepers to have downloaded Calm to improve sleep (χ22=1548.8, P<.001), reduce depression or anxiety (χ22=15.5, P<.001), or improve overall health (χ22=57.6, P<.001). Respondents with insomnia used Calm most often (mean 5.417 days/week, SD 1.936), followed by poor sleepers (mean 5.043 days/week, SD 2.027; F2=21.544, P<.001). The most common time to use Calm was while lying down to sleep (7607/9686, 78.54%), and bedtime use was more common among poor sleepers and those with insomnia (χ22=382.7, P<.001). Compared to good and poor sleepers, those with insomnia were more likely to use Calm after waking up at night (χ22=410.3, P<.001). Most participants tried to use Calm on a regular basis (5031/8597, 58.52%), but regular nighttime use was most common among those with insomnia (646/977, 66.1%), followed by poor sleepers (4040/6930, 58.30%; χ22=109.3, P<.001). Conclusions: Of the paying subscribers to Calm who have used one of the sleep components, approximately 90% have sleep difficulties, and 77% started using Calm primarily for sleep. These descriptive data point to areas of focus for continued refinement of app features and content, followed by prospective trials testing efficacy of consumer-based meditation mobile apps for improving sleep.

  • Source: X2 AI / PlaceIt; Copyright: X2 AI / PlaceIt; URL:; License: Licensed by JMIR.

    Artificial Intelligence Chatbot for Depression: Descriptive Study of Usage


    Background: Chatbots could be a scalable solution that provides an interactive means of engaging users in behavioral health interventions driven by artificial intelligence. Although some chatbots have shown promising early efficacy results, there is limited information about how people use these chatbots. Understanding the usage patterns of chatbots for depression represents a crucial step toward improving chatbot design and providing information about the strengths and limitations of the chatbots. Objective: This study aims to understand how users engage and are redirected through a chatbot for depression (Tess) to provide design recommendations. Methods: Interactions of 354 users with the Tess depression modules were analyzed to understand chatbot usage across and within modules. Descriptive statistics were used to analyze participant flow through each depression module, including characters per message, completion rate, and time spent per module. Slide plots were also used to analyze the flow across and within modules. Results: Users sent a total of 6220 messages, with a total of 86,298 characters, and, on average, they engaged with Tess depression modules for 46 days. There was large heterogeneity in user engagement across different modules, which appeared to be affected by the length, complexity, content, and style of questions within the modules and the routing between modules. Conclusions: Overall, participants engaged with Tess; however, there was a heterogeneous usage pattern because of varying module designs. Major implications for future chatbot design and evaluation are discussed in the paper.

  • Source: freepik; Copyright: torwaiphoto; URL:; License: Licensed by JMIR.

    Evaluation of Treatment Descriptions and Alignment With Clinical Guidance of Apps for Depression on App Stores: Systematic Search and Content Analysis


    Background: The use of apps for the treatment of depression shows great promise. However, there is uncertainty regarding the alignment of publicly available apps for depression with clinical guidance, their treatment fidelity and evidence base, and their overall safety. Objective: Built on previous analyses and reviews, this study aims to explore the treatment and safety issues of publicly available apps for depression. Methods: We conducted a content analysis of apps for depression in the 2 main UK app stores (Google Play and Apple App Store). App store listings were analyzed for intervention content, treatment fidelity, and fit with the National Institute for Health and Care Excellence (NICE) guidelines for the treatment of depression in adults. Results: A total of 353 apps for depression were included in the review. App descriptions reported the use of 20 treatment approaches and 37 treatment strategies. Many apps used transdiagnostic (155/353, 43.9%) and multitheoretical interventions to treat multiple disorders including depression. Although many interventions appeared to be evidence-informed, there were issues with treatment fidelity, research evidence, and fit with clinical guidelines. None of the apps fully aligned with the NICE guidelines for depression. Conclusions: App developers have adopted many evidence-informed treatments in their interventions; however, more work is needed to improve clinical validity, treatment fidelity, and the safety of apps. We urge developers to consult relevant guidelines and standards, and to engage in reflective questioning on treatment and safety to address these issues and to improve treatment content and intervention design.

  • Source: Image created by the Authors/Placeit; Copyright: The Authors/Placeit; URL:; License: Licensed by the authors.

    Assessing the Efficacy and Acceptability of a Web-Based Intervention for Resilience Among College Students: Pilot Randomized Controlled Trial


    Background: College students are at elevated risk for developing mental health problems and face specific barriers around accessing evidence-based treatment. Web-based interventions that focus on mental health promotion and strengthening resilience represent one possible solution. Providing support to users has shown to reduce dropout in these interventions. Further research is needed to assess the efficacy and acceptability of these interventions and explore the viability of automating support. Objective: This study investigated the feasibility of a new web-based resilience program based on positive psychology, provided with human or automated support, in a sample of college students. Methods: A 3-armed closed pilot randomized controlled trial design was used. Participants were randomized to the intervention with human support (n=29), intervention with automated support (n=26), or waiting list (n=28) group. Primary outcomes were resilience and well-being, respectively measured by the Connor–Davidson Resilience Scale and Pemberton Happiness Index. Secondary outcomes included measures of depression and anxiety, self-esteem, and stress. Outcomes were self-assessed through online questionnaires. Intention-to-treat and per-protocol analyses were conducted. Results: All participants demonstrated significant improvements in resilience and related outcomes, including an unexpected improvement in the waiting list group. Within- and between-group effect sizes ranged from small to moderate and within-group effects were typically larger for the human than automated support group. A total of 36 participants began the program and completed 46.46% of it on average. Participants were generally satisfied with the program and found it easy to use. Conclusions: Findings support the feasibility of the intervention. Preliminary evidence for the equal benefit of human and automated support needs to be supported by further research with a larger sample. Results of this study will inform the development of a full-scale trial, from which stronger conclusions may be drawn. Trial Registration: International Standard Randomized Controlled Trial Number (ISRCTN) 11866034;

  • Source: Freepik; Copyright: freepik; URL:; License: Licensed by JMIR.

    Diagnosing Chronic Obstructive Airway Disease on a Smartphone Using Patient-Reported Symptoms and Cough Analysis: Diagnostic Accuracy Study


    Background: Rapid and accurate diagnosis of chronic obstructive pulmonary disease (COPD) is problematic in acute care settings, particularly in the presence of infective comorbidities. Objective: The aim of this study was to develop a rapid smartphone-based algorithm for the detection of COPD in the presence or absence of acute respiratory infection and evaluate diagnostic accuracy on an independent validation set. Methods: Participants aged 40 to 75 years with or without symptoms of respiratory disease who had no chronic respiratory condition apart from COPD, chronic bronchitis, or emphysema were recruited into the study. The algorithm analyzed 5 cough sounds and 4 patient-reported clinical symptoms, providing a diagnosis in less than 1 minute. Clinical diagnoses were determined by a specialist physician using all available case notes, including spirometry where available. Results: The algorithm demonstrated high positive percent agreement (PPA) and negative percent agreement (NPA) with clinical diagnosis for COPD in the total cohort (N=252; PPA=93.8%, NPA=77.0%, area under the curve [AUC]=0.95), in participants with pneumonia or infective exacerbations of COPD (n=117; PPA=86.7%, NPA=80.5%, AUC=0.93), and in participants without an infective comorbidity (n=135; PPA=100.0%, NPA=74.0%, AUC=0.97). In those who had their COPD confirmed by spirometry (n=229), PPA was 100.0% and NPA was 77.0%, with an AUC of 0.97. Conclusions: The algorithm demonstrated high agreement with clinical diagnosis and rapidly detected COPD in participants presenting with or without other infective lung illnesses. The algorithm can be installed on a smartphone to provide bedside diagnosis of COPD in acute care settings, inform treatment regimens, and identify those at increased risk of mortality due to seasonal or other respiratory ailments. Trial Registration: Australian New Zealand Clinical Trials Registry ACTRN12618001521213;

  • Source: Flickr; Copyright: CollegeDegrees360; URL:; License: Creative Commons Attribution + ShareAlike (CC-BY-SA).

    Effectiveness of a Smartphone App (BioBase) for Reducing Anxiety and Increasing Mental Well-Being: Pilot Feasibility and Acceptability Study


    Background: The prevalence of workplace-related stress and anxiety is high, resulting in stress-related physical and mental illness. Digital self-guided interventions aimed at key areas of workplace design may be able to provide remote anxiolytic effects. Objective: The aim of this feasibility study is to assess changes in anxiety and mental well-being after use of the BioBase programme, a mobile phone platform for psycho-educational modules, tools, and real-time feedback of physiological data. Methods: A 4-week observational study was carried out in 55 healthy adults who were screened for stress with the Depression Anxiety Stress Scale (DASS) Stress subscale. Participants completed anxiety (6-item State-Trait Anxiety Inventory [STAI]) and mental well-being (Warwick-Edinburgh Mental Well-being Scale [WEMWBS]) questionnaires at baseline and at 4 weeks. Feedback questionnaires were administered after 4 weeks. Results: After 4 weeks of using the programme and controlling for any effect of being paid to take part in the study, STAI significantly decreased (baseline mean 45.52 [SD 13.2]; 4-week mean 39.82 [SD 11.2]; t54=–3.51; P<.001; CI –8.88 to –2.52; Cohen d=0.96) and WEMWBS significantly increased (baseline mean 48.12 [SD 6.4]; 4-week mean 50.4 [SD 6.9]; t53=2.41; P=.019; CI 0.44-4.23; Cohen d=0.66). Further, higher baseline stress was significantly associated with a greater decrease in STAI (t53=–3.41; P=.001; CI –8.10 to –2.10; R2=0.180) and a greater increase in WEMWBS (t52=2.41; P=.019; CI 0.38-4.11, R2=0.101). On feedback, participants found the programme easy to use/navigate, with the content being acceptable and relevant to workplace-related stressors; 70% (21/30) of participants would recommend the programme to a friend. Conclusions: The BioBase programme is a potentially effective intervention in decreasing anxiety and increasing mental well-being, with larger changes in those with higher baseline levels of stress.

  • Source: Pexels; Copyright: Andrea; URL:; License: Licensed by JMIR.

    Co-Designing a Web-Based Decision Aid Tool for Employees Disclosure of Mental Health Conditions: A Participatory Study Design Using Employee and...


    Background: Decisions of whether to disclose mental health conditions are extremely personal and require the consideration of multiple factors associated with the disclosure process (eg, weighing the risks and benefits). Decision aid tools help people make these complex decisions. Such an aid needs to be confidential, easily accessible, and easy to use with the potential to access the tool on multiple occasions. Web programs are well suited to meet these requirements and, if properly developed, can provide feasible, accessible, affordable, and effective workplace interventions. Objective: This study aims to gain insights from potential end users, in this case both employees and organizations, into what type of components including language, style, and content would avoid potential stigma and ensure that elements of clear value for users would be built into a web-based decision aid tool that aims to assist employees in making decisions about the disclosure of their mental health condition at work. Methods: A participatory design approach was used to allow developers, researchers, experts, and end users to collaborate in co-designing the tool. During the user research phase of the development of the web-based tool, a participatory design workshop approach was selected as a part of a larger study of focus groups. Australian employees and managers in rural, suburban, and urban locations participated in an exploratory qualitative study involving participatory workshops designed to elicit their perspectives and preferences for a decision aid tool. Results: A total of 2 workshops were conducted with 13 participants. The majority were from a transport company (9/13, 69%), male (8/13, 62%), and employed full time (11/13, 85%). Six employees had previous experience disclosing their own mental health condition, and 7 were in a supervisory role and had previously been disclosed to. In any co-design development, there are certain trade-offs that need to be made between the views of experts, developers, end users, and the available budget. In this specific instance of a very delicate, personal decision, the end users provided valuable design insights into key areas such as language, and they were very antipathetic to a key feature, the avatar, which was thought to be desirable by experts and developers. Findings including aspects of the tool where all stakeholders were in agreement, aspects where some stakeholders disagreed and adaptations were implemented, where disagreements could not be implemented because of financial constraints, and misalignment between stakeholders and how to decide on a balance were shared. Conclusions: The co-design with a lived experience approach is useful for contributing much to the design, language, and features. The key in this study was balancing the needs of the workers and the potential impact for the managers and organizations, while ensuring legislation and regulation requirements were upheld. Trial Registration:

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Latest Submissions Open for Peer-Review:

View All Open Peer Review Articles
  • Automated Reinforcement Management System (ARMS): Phase I Methods and Development

    Date Submitted: Nov 16, 2020

    Open Peer Review Period: Nov 16, 2020 - Jan 11, 2021

    Background: Alcohol use is directly related to over 3 million deaths worldwide every year. Contingency management (CM) is a cost-effective treatment for substance use disorders; however, few studies h...

    Background: Alcohol use is directly related to over 3 million deaths worldwide every year. Contingency management (CM) is a cost-effective treatment for substance use disorders; however, few studies have examined the efficacy of CM for alcohol use disorder (AUD). Recent technological advances have enabled the combined use of mobile applications (apps) and low-cost electronic breathalyzer devices to remotely monitor alcohol use. Leveraging this type of technology, our group has recently developed an integrated CM system that would enable community treatment programs to deliver CM remotely to anyone who owns a smartphone. Objective: In this study, we present a full description of our integrated CM system, Automated Reinforcement Management System (ARMS), and describe the protocol that will evaluate its feasibility and acceptability. Methods: Initially, six clinicians will participate in a one-hour focus group where study staff will navigate through ARMS as it would be used by clinicians and patients. Clinicians will then provide feedback on the intervention in general. This information will be used to modify ARMS to make it more user friendly, time-saving, and relevant to treatment. A second focus group will summarize the changes made following the initial clinician feedback and will provide additional input regarding the potential utilization of ARMS. At the end of the second focus group, the clinicians’ acceptability of ARMS will be evaluated using the System Usability Scale (SUS). Following the clinician assessments of ARMS and after the final modifications are made, the system will be evaluated in terms of feasibility and patient acceptability using an A-B-A within-subject experimental design where 20 treatment-seeking individuals with AUD will be recruited. The two A phases will each last two weeks and the B phase will last four weeks. During all phases, participants will be asked to use the ARMS app to submit three breathalyzer samples per day (at 10am, 2pm, and 8pm). Participants will be prompted by their ARMS app at these pre-determined times to record and submit their breathalyzer samples. During the A phases (control conditions), participants will earn vouchers for every breathalyzer sample submitted, independent of sample results. During the B phase (CM condition), vouchers will be provided contingent upon the submission of alcohol-negative breathalyzer samples (BAC = 0.00). At the end of the A-B-A experiment trial, patients’ acceptability of ARMS will be evaluated with the SUS. Feasibility will be measured by whether or not ARMS could significantly increase alcohol abstinence. Results: This study will begin recruitment in January 2021 and is expected to be completed by December 2021. Conclusions: This study will provide the baseline capability for the implementation of a remotely monitored contingency management platform. If successful, ARMS has the potential to provide effective treatment for AUDs to those living in remote rural areas.

  • A Digital Platform for Facilitating Personalized Dementia Care in Nursing Homes: A Formative Evaluation Study

    Date Submitted: Nov 12, 2020

    Open Peer Review Period: Nov 12, 2020 - Jan 7, 2021

    Background: Person-centered care is key to the wellbeing of people with dementia. A large quantity of personal data can be collected with the development of the Internet of Things, which has the poten...

    Background: Person-centered care is key to the wellbeing of people with dementia. A large quantity of personal data can be collected with the development of the Internet of Things, which has the potential to facilitate person-centered care for people with dementia. Yet, there are limited assistive technologies developed for this purpose, and the user acceptance for assistive technologies is low in nursing homes. Through a data-enabled design approach, a digital platform was developed for helping the care team to personalize the management of behavioral and psychological symptoms for people with dementia in nursing homes. Objective: This study aims to evaluate the digital platform from three aspects, in a real-life context with potential users. First, its technical feasibility in collecting sufficient data for pattern analysis; second, the types of insights and actions generated from the potential users by using it, if any; third, its perceived usefulness and its future improvements that potential users would like to see. Methods: The digital platform was deployed in a nursing home for seven weeks, and the data collected were first analyzed by the researchers for a technical feasibility check. The data were then visualized and presented to the potential users via the digital platform. The potential users were asked to analyze the visualizations and were interviewed on 1) the insights and actions generated, if any; 2) the usefulness of the digital platform and 3) what could be improved. Results: The data collected in the digital platform demonstrate its technical potential to reveal behavior patterns for PwD. The insights generated by the potential users were categorized into “client level”, “ward level” and “team level”. The actions taken by the potential users were classified into “investigation” and “implementation”. The user acceptance varied across potential users, and three aspects of improvements for the digital platform were identified. Conclusions: This study provides the first evidence for the technical feasibility of the digital platform; besides, it offers future researchers some recommendations on how to integrate assistive technologies in the nursing home context from exploring the types of insights and actions identified, the varied perceived usefulness, and the areas of improvement for the digital platform.