JMIR Formative Research

Process evaluations, early results, and feasibility/pilot studies of digital and non-digital interventions

Editor-in-Chief:

Amaryllis Mavragani, PhDc, Scientific Editor at JMIR Publications, Canada


Impact Factor 2.0 CiteScore 2.7

JMIR Formative Research (JFR, ISSN 2561-326X, Journal Impact Factor™ 2.0 (Clarivate, 2024)) publishes peer-reviewed, openly accessible papers containing results from process evaluations, feasibility/pilot studies and other kinds of formative research and preliminary results. While the original focus was on the design of medical- and health-related research and technology innovations, JMIR Formative Research publishes studies from all areas of medical and health research.

Formative research is research that occurs before a program is designed and implemented, or while a program is being conducted. Formative research can help

  • define and understand populations in need of an intervention or public health program
  • create programs that are specific to the needs of those populations
  • ensure programs are acceptable and feasible to users before launching
  • improve the relationship between users and agencies/research groups
  • demonstrate the feasibility, use, satisfaction with, or problems with a program before large-scale summative evaluation (looking at health outcomes)

Many funding agencies will expect some sort of pilot/feasibility/process evaluation before funding a larger study such as a Randomized Controlled Trial (RCT).

Formative research should be an integral part of developing or adapting programs and should be used while the program is ongoing to help refine and improve program activities. Thus, formative evaluation can and should also occur in the form of a process evaluation alongside a summative evaluation such as an RCT.

JMIR Formative Research fills an important gap in the academic journals landscape, as it publishes sound and peer-reviewed formative research that is critical for investigators to apply for further funding, but that is usually not published in outcomes-focused medical journals aiming for impact and generalizability.

Summative evaluations of programs and apps/software that have undergone a thorough formative evaluation before launch have a better chance to be published in high-impact flagship journals; thus, we encourage authors to submit - as a first step - their formative evaluations in JMIR Formative Research (and their evaluation protocols to JMIR Research Protocols). 

JMIR Formative Research is indexed in MEDLINEPubMed, PubMed CentralDOAJ, Scopus, Sherpa/Romeo, EBSCO/EBSCO Essentials, and the Emerging Sources Citation Index (ESCI).

Recent Articles

Article Thumbnail
Formative Evaluation of Digital Health Interventions

Despite the availability of antiretroviral therapy (ART), only 66% of people with HIV in the United States achieve viral suppression, largely due to suboptimal ART adherence. Barriers such as limited access to care and forgetfulness impact adherence rates, which must be maintained at ≥95% to prevent viral load rebound. Combination interventions leveraging community health worker (CHW) support and mobile health (mHealth) technologies have the potential to overcome previously identified barriers and provide cost-effective support for improving adherence and viral suppression outcomes in people with HIV.

|
Article Thumbnail
Formative Evaluation of Digital Health Interventions

Despite the availability of prophylactic HPV vaccines, uptake remains suboptimal among young Black adults. Social media is a platform for the dissemination of health information and can be used to promote HPV vaccination among young Black adults.

|
Article Thumbnail
Formative Evaluation of Digital Health Interventions

Lyra Health’s short-term blended care therapy model, Lyra Care Therapy (LCT), has demonstrated effectiveness at scale. In LCT, clients participate in synchronous telehealth sessions and asynchronous guided practice sessions, in which they are provided with digital tools to reinforce key concepts and skills. These digital tools include animated video lessons that use storytelling to show characters learning and implementing new skills from therapy, written psychoeducational materials, interactive exercises that prompt reflection and skills practice, symptom assessments, and messaging with therapists. Past research on LCT found that time spent in therapy sessions and viewing digital video lessons predicts improvements in depression and anxiety symptoms.

|
Article Thumbnail
Pilot studies (ehealth)

Access to mental health services continues to pose a global challenge, with current services often unable to meet the growing demand. This has sparked interest in conversational artificial intelligence (AI) agents as potential solutions. Despite this, the development of a reliable virtual therapist remains challenging, and the feasibility of AI fulfilling this sensitive role is still uncertain. One promising approach involves using AI agents for psychological self-talk, particularly within virtual reality (VR) environments. Self-talk in VR allows externalizing self-conversation by enabling individuals to embody avatars representing themselves as both patient and counselor, thus enhancing cognitive flexibility and problem-solving abilities. However, participants sometimes experience difficulties progressing in sessions, which is where AI could offer guidance and support.

|
Article Thumbnail
Formative Evaluation of Digital Health Interventions

The use of virtual reality (VR) technology in rehabilitation therapy has been growing, leading to the development of VR-based upper-limb rehabilitation software. To ensure effective utilization of such software, usability evaluations are critical to enhance user satisfaction and identify potential usability issues.

|
Article Thumbnail
Formative Evaluation of Digital Health Interventions

In the modern economy, shift work is prevalent in numerous occupations. However, it often disrupts workers’ circadian rhythms and can result in shift work sleep disorder. Proper management of shift work sleep disorder involves comprehensive and patient-specific strategies, some of which are similar to cognitive behavioral therapy for insomnia.

|
Article Thumbnail
Formative Evaluation of Digital Health Interventions

Chemotherapy cycle prescription is generally carried out through a multi-step manual process which is prone to human errors. Clinical decision support tools can provide patient-specific assessments that support clinical decisions, improve prescribing practices, and reduce medication errors.

|
Article Thumbnail
Formative Evaluation of Digital Health Interventions

Family caregivers of individuals with dementia face significant mental health challenges. Acceptance and commitment therapy (ACT) has emerged as a promising intervention for improving these caregivers’ mental health. While various delivery modes of ACT have been explored, there is a need for evidence on the efficacy of videoconference-delivered ACT programs for this population.

|
Article Thumbnail
Formative Evaluation of Digital Health Interventions

The optimal response to a major incident in a road tunnel involves efficient decision-making among the responding emergency services (fire and rescue services, police, and ambulances). The infrequent occurrence of road tunnel incidents may entail unfamiliarity with the tunnel environment and lead to uncertain and inefficient decision-making among emergency services commanders. Ambulance commanders have requested tunnel-specific learning materials to improve their preparedness.

|
Article Thumbnail
Research Letter

Systematic reviews are recognized as a high-level source of evidence in medical research but are often constrained by time-consuming manual screening of vast numbers of citations. The exponential growth of biomedical literature further complicates researchers’ ability to remain updated. Artificial intelligence (AI) may offer a solution, particularly through Large Language Models (LLMs), which excel at processing complex text. In this pilot study, we explored the feasibility of using five distinct LLMs to screen citations extracted from an existing systematic review on trauma hemorrhage. We selected ChatGPT 3.5, ChatGPT 4, Google Bard, Meta Llama 2 (70b parameters), and Claude AI 2, chosen for their widespread availability and text-comprehension abilities. In the original systematic review, 1,186 citations were screened by human reviewers, identifying 21 for full-text inclusion and excluding 1,165. From these excluded citations, we randomly sampled 100, yielding a total dataset of 121 records (21 included, 100 excluded). Each LLM received the original inclusion and exclusion criteria in a single-run format. We assessed performance by calculating sensitivity (correct identification of included abstracts), specificity (correct exclusion of irrelevant items), and overall accuracy. Sensitivity varied considerably, ranging from 57% (Claude2) to 100% (ChatGPT 3.5, Bard). Specificity was similarly diverse, from 18% (Llama2) to 79% (ChatGPT 3.5). ChatGPT 3.5 thus achieved the highest combined sensitivity and specificity. While these findings suggest that AI-driven LLMs could potentially streamline systematic review screening—perhaps even replacing a second human reviewer—significant limitations persist. Our pilot nature and single-run design curb generalizability, and some LLMs were prone to overinclusion. Larger, multi-run investigations are needed to assess probabilistic variability and refine prompt-engineering approaches. Nonetheless, this pilot evaluation highlights the emerging role of LLMs in citation screening and sets the stage for broader adoption, ensuring that ethical and procedural frameworks keep pace with AI’s rapid evolution.

|
Article Thumbnail
Pilot studies (non-ehealth)

Food insecurity (FI) is a risk factor for type 2 diabetes (T2D) that disproportionately affects Latinas. We propose that FI cycles over the course of a month according to disbursement of food assistance benefits and seek to understand whether this cycling is related to diabetes risk. We conducted a micro-longitudinal study to examine the relationship of monthly cycling of FI and diabetes risk factors.

|

Preprints Open for Peer-Review

|

Open Peer Review Period:

-

We are working in partnership with