Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Monday, March 11, 2019 at 4:00 PM to 4:30 PM EST. We apologize in advance for any inconvenience this may cause you.
Who will be affected?
Readers: No access to all 28 journals. We recommend accessing our articles via PubMed Central
Authors: No access to the submission form or your user account.
Reviewers: No access to your user account. Please download manuscripts you are reviewing for offline reading before Monday, March 11, 2019 at 4:00 PM.
Editors: No access to your user account to assign reviewers or make decisions.
Copyeditors: No access to user account. Please download manuscripts you are copyediting before Monday, March 11, 2019 at 4:00 PM.
JMIR Formative Research
Process evaluations, early results, and feasibility/pilot studies of digital and non-digital interventions
(Toronto, Nov 29th, 2018) We are pleased to report that JMIR Formative Res has been accepted for indexing in PubMed Central and PubMed. The staff at PubMed Central is currently finalizing the setup and we expect all articles published since Vol 1 / Iss 1 (2017) to be in PubMed Central and PubMed within the next couple of weeks. The acceptance is the result of a rigorous scientific and technical evaluation by the US National Library's (NLM’s) Library Operations Division, which decided that the scientific and editorial character and quality of a JMIR Formative Research merits its inclusion in PMC. In making this decision NLM has considered the suitability of the journal for the NLM collection as well as the opinions of expert consultants.
JMIR Formative Research (JFR) (a PubMed-indexed sister journal of J Med Internet Res (JMIR) and JMIR mHealth & uHealth, the leading eHealth and mHealth journals by impact factor) publishes peer-reviewed, openly accessible papers containing results from process evaluations, feasibility/pilot studies and other kinds of formative research and preliminary results. While the original focus was on the design of medical and health-related research and technology innovations, JFR publishes studies from all areas of medical and health research.
Formative research is research that occurs before a program is designed and implemented, or while a program is being conducted. Formative research can help
define and understand populations in need of a intervention or public health program
create programs that are specific to the needs of those populations
ensure programs are acceptable and feasible to users before launching
improve the relationship between users and agencies/research groups
demonstrate the feasibility, use, satisfaction with, or problems with a program before large-scale summative evaluation (looking at health outcomes)
Many funding agencies will expect some sort of pilot/feasibility/process evaluation before funding a larger study such as a Randomized Controlled Trial (RCT).
Formative research should be an integral part of developing programs or adapting programs, and should be used while the program is on-going to help refine and improve program activities. Thus, formative evaluation can and should also occur in form of a process evaluation alongside a summative evaluation such as a RCT.
This journal fills an important gap in the academic journals landscape, as it publishes sound and peer-reviewed formative research that is criticial for investigators to apply for further funding, but that is usually not published in outcomes-focussed medical journals aiming for impact and generalizability.
Summative evaluations of programs and apps/software that have undergone a thorough formative evaluation before launch have a better chance to be published in our high-impact flagship journals (JMIR mHealth, J Med Internet Res, JMIR Serious Games), thus we encourage authors to submit - as a first step - their formative evaluations in JMIR Formative Res (and their evaluation protocols in JMIR Res Protoc).
Background: NutriQuébec is a Web-based prospective study on the relationship between diet and health as well as the impact of food-related health policies in the adult population of Québec, Canada. Recruitment and retention of individuals with a low socioeconomic status (SES) in such a study are known to be challenging, yet critical for achieving representativeness of the entire population.
Objective: This study aimed to identify the behavioral, normative, and control beliefs of individuals with a low SES regarding participation in the NutriQuébec project and to identify their preferences regarding recruitment methods.
Methods: A total of four focus groups were conducted in community centers located in low-income areas of Québec City, Canada. On the basis of the theory of planned behavior, participants’ beliefs associated with attitude, subjective norm, and perceived behavioral control regarding hypothetical participation in the NutriQuébec project were identified. Focus groups were recorded, transcribed, and coded by two analysts.
Results: Participants (16 men and 12 women) were aged between 28 and 72 years, and a majority of the participants had an annual household income of Can $19,999 or less. The main perceived advantages of participating in the NutriQuébec project were contributing to improved collective health and supporting research. The only disadvantage identified was the risk of having to fill out too many questionnaires. Participants could not, in general, identify persons from their entourage who would approve or disapprove their participation in the study. The main facilitators identified were obtaining a brief health assessment and the ability to complete questionnaires in a way that is not Web-based. The main barrier was the lack of internet access. The preferred means of recruitment were through social media, television, and community centers.
Conclusions: These results provide insightful information regarding the best methods and messages to use in order to recruit and retain individuals with a low SES in a population-based prospective study on lifestyle and health on the internet.
Background: There is need for more timely data to inform interventions that address the growing noncommunicable disease (NCD) epidemic. With a global increase in mobile phone ownership, mobile phone surveys can bridge this gap.
Objective: This study aimed to explore the acceptability and use of interactive voice response (IVR) surveys for surveillance of NCD behavioral risk factors in rural Uganda.
Methods: This qualitative study employed user group testing (UGT) with community members. The study was conducted at the Iganga-Mayuge Health and Demographic Surveillance Site (IM-HDSS) in Eastern Uganda. We conducted four UGTs which consisted of different categories of HDSS members: females living in urban areas, males living in urban areas, females living in rural areas, and males living in rural areas. Participants were individually sent an IVR survey, then were brought in for a group discussion using a semistructured guide. Data were analyzed thematically using directed content analysis.
Results: Participants perceived that IVR surveys may be useful in promoting confidentiality, saving costs, and raising awareness on NCD behavioral risk factors. Due to the clarity and delivery of questions in the local language, the IVR survey was perceived as easy to use. Community members suggested scheduling surveys on specific days and sending reminders as ways to improve their use for surveillance. Social issues such as domestic violence and perceptions toward unknown calls, technological factors including poor network connections and inability to use phones, and personal issues such as lack of access to phones and use of multiple networks were identified as barriers to the acceptability and use of mobile phone surveys. However, incentives were reported to motivate people to complete the survey.
Conclusions: Community members reflected on contextual and sociological implications of using mobile phones for surveillance of NCD behavioral risk factors. The opportunities and challenges that affect acceptability and use of IVR surveys should be considered in designing and implementing surveillance programs for NCD risk factors.
Background: The integration of Behavioral Activation Treatment for Depression (BAT-D) into smoking cessation interventions is a promising approach to address depression as a barrier to quitting. However, this approach has only been tested as a face-to-face intervention, which has low reach.
Objective: The aims of the study were to develop a BAT-D mobile health app with high potential reach and determine its feasibility, acceptability, and preliminary effects on theory-based behavioral processes of behavioral activation, reduced depressive symptoms, and smoking cessation.
Methods: Following a user-centered design process consisting of competitive analysis, focus groups, and prototype testing, we conducted a single-arm pilot trial of Actify!, a BAT-D app for depressed smokers. Participants used SmokefreeTXT along with Actify! to provide cessation content that had not yet been built into the app for this initial phase of pilot testing. Participants in the trial were current, daily smokers with mild to moderate depressive symptoms. We examined use outcomes for all enrolled participants and process and cessation outcomes at 6 weeks postenrollment for study completers (16/17, 94% retention).
Results: Regarding acceptability, average number of log-ins per participant was 16.6 (SD 13.7), and 63% (10/16) reported being satisfied overall with the app. Posttreatment interviews identified some usability challenges (eg, high perceived burden of planning and scheduling values-based activities). There was a significant decrease in depressive symptoms from baseline to follow-up (mean change in Patient Health Questionnaire–9 scores was –4.5, 95% CI –7.7 to –1.3; P=.01). Additionally, carbon monoxide (CO)-confirmed, 7-day point prevalence abstinence (PPA) at 6-week follow-up was 31% (5/16), and the 30-day PPA was 19% (3/16).
Conclusions: Results demonstrate promising engagement with Actify! and potential for impact on theory-based change processes and cessation outcomes. Preliminary quit rates compare favorably to previous trials of smoking cessation apps for the general population (ie, short-term, self-reported 30-day quit rates in the 8% to 18% range) and a previous trial of face-to-face BAT-D for depressed smokers (ie, CO-confirmed, 7-day PPA rate of 17% at end of treatment).
Background: The Dialysis Guide (DG) is a patient decision aid (PDA) available as an app and developed for mobile phones for patients with chronic kidney disease facing the decision about dialysis modality.
Objective: The aim of this study was to uncover the applicability of the DG as a PDA.
Methods: The respondents completed a questionnaire before and after using the DG. The respondents' decisional conflicts were examined using the Decisional Conflict Scale, and the usability of the app was examined using the System Usability Scale (SUS). The change in decisional conflict was determined with a paired t test.
Results: A total of 22 respondents participated and their mean age was 65.05 years; 20 out of 22 (90%) had attended a patient school for kidney disease, and 13 out of 22 (59%) had participated in a conversation about dialysis choice with a health professional. After using the DG, the respondents' decisional conflicts were reduced, though the reduction was not statistically significant (P=.49). The mean SUS score was 66.82 (SD 14.54), corresponding to low usability.
Conclusions: The DG did not significantly reduce decisional conflict, though the results indicate that it helped the respondents decide on dialysis modality. Attending a patient school and having a conversation about dialysis modality choice with a health professional is assumed to have had an impact on the decisional conflict before using the DG. The usability of the DG was not found to be sufficient, which might be caused by the respondents’ average age. Thus, the applicability of the DG cannot be definitively determined.
Background: Internet-based cognitive behavior therapy (iCBT) can be effective in mental and somatic health care. Research on the feasibility of internet interventions in clinical practice is, however, still scarce. Studies with a focus on the patient regarding usability of interventions and digital health literacy skills are especially lacking. Objective: The goal of this study was to assess the usability of an iCBT for chronic pain, Master Your Pain, and the relationship between its usability outcomes and the factors age, educational level, and digital health literacy skills. The aims were to determine what changes were needed in the program for sufficient usability and which individual characteristics were related to the usability of the program. Methods: Patients were recruited from two mental health care practices. A mixed methods approach was used in this study. A qualitative observational study comprising performance tasks in the iCBT program was used to test usability. A quantitative questionnaire was used to measure possible related constructs. Usability was operationalized as the number of tasks that could be completed and the type and number of problems that occurred while doing so. Performance tasks were set up to measure 6 digital skills: (1) operating the computer and internet browser, (2) navigation and orientation, (3) using search strategies, (4) evaluating relevance of content, (5) adding personal content, and (6) protecting and respecting privacy. Participants were asked to think aloud while performing the tasks, and screen activities and webcam recordings were captured. The qualitative observational data was coded using inductive analysis by two independent researchers. Correlational analyses were performed to test how usability relates to sociodemographics and digital health literacy. Results: A total of 32 patients participated, with a mean age of 49.9 years and 84% (27/32) being female. All performance tasks except one (fill in a diary registration) could be completed independently by more than 50% of the participants. On operational, navigation, and search levels, participants struggled most with logging in, logging out, and finding specific parts of the intervention. Half of the sample experienced problems evaluating the relevance and adding content to the program to some extent. Usability correlated moderately negatively with age and moderately positively with digital health literacy skills but not with educational level. Conclusions: The results provide insight into what is essential for proper usability regarding the design of an iCBT program considering variations in age, educational level, and digital health literacy. Furthermore, the results provide insight into what type of support is needed by patients to properly use the intervention. Tailoring support among the needs of certain age groups or skill levels could be beneficial and could range from no extra support (only online feedback, as intended) to practical support (an additional usability introduction session) to blended care (combined face-to-face sessions throughout the therapy).
Background: The availability of both digital and traditional mental well-being interventions is rising, but these interventions typically do not consider people with intellectual disabilities as potential users.
Objective: The study aimed to explore the acceptability and feasibility of a new digital intervention, developed with and for people with intellectual disabilities, to improve their subjective well-being.
Methods: Using a single-group pre-post design, participants with intellectual disabilities and their caregivers completed the 4-week intervention. Mixed methods questionnaires assessed the acceptability of the intervention, in addition to self-report and proxy-report measures of subjective well-being and behavioral problems.
Results: A total of 12 men with mild to moderate intellectual disabilities enrolled in and completed the study alongside 8 caregivers. Participant acceptability of the intervention was high, and feedback covered multiple aspects of the intervention, including (1) program concept and design, (2) program content, and (3) intervention usage. Self-rated mood barometers indicated mood improvements for 5 participants, deteriorations for 2 participants, and no observed changes for the remaining participants. Statistical analyses yielded no difference from pretest (median=79; range 39-86) to posttest (median=79; range 21-96) for subjective well-being in people with intellectual disabilities (W=10.5; P=.17) and for behavioral problems (W=14; P=.05).
Conclusions: People with intellectual disabilities and their caregivers are receptive to using digital well-being interventions, and this research shows such interventions to be feasible in routine practice. Given the acceptability of the intervention, its potential efficacy can now be evaluated in people with intellectual disabilities and symptoms of reduced mental well-being.
Background: Family-based drug prevention programs that use group-based formats with trained facilitators, such as the Strengthening Families Program (SFP), are effective in preventing underage drinking and youth drug use. However, these programs are resource-intensive and have high costs and logistical demands. Tailoring them for Web-based delivery is more cost-effective and makes it easier to scale these programs for widespread dissemination. This requires the active involvement of all key stakeholders to determine content and delivery format.
Objective: The aim was to obtain consumer, agency stakeholder, and expert input into the design of a Web-based parenting skills training and youth drug prevention program.
Methods: We conducted 10 focus groups with 85 adults (range 4-10, average 8 per group), 20 stakeholder interviews with family services agency staff, and discussed critical design considerations with 10 prevention scientists and e-learning experts to determine the optimal program content and technology features for SFP Online. Focus group participants also answered survey questions on perceived barriers to use, desired navigational features, preferred course format, desired content, preferred reward structures, course length, interactive components, computer efficacy, and technology use. Descriptive statistics were used to examine consumer characteristics; linear regression was used to examine relations between SFP exposure and four continuous outcome measures, including desired program content, interactive technology, and concerns that may inhibit future use of SFP Online. Logistic regression was used as a binary measure of whether consumers desired fun games in the SFP Online program.
Results: Three broad thematic categories emerged from the qualitative interviews enumerating the importance of (1) lesson content, (2) logistics for program delivery, and (3) multimedia interactivity. Among the many significant relations, parents who viewed more SFP lessons reported more reasons to use an online program (beta=1.48, P=.03) and also wanted more interactivity (6 lessons: beta=3.72, P=.01; >6 lessons: beta=2.39, P=.01), parents with less interest in a mixed delivery format (class and online) reported fewer reasons to use the online program (beta=−3.93, P=.01), comfort using computers was negatively associated with concerns about the program (beta=−1.83, P=.01), having mobile phone access was related to fewer concerns about online programs (beta=−1.63, P=.02), willingness to view an online program using a mobile phone was positively associated with wanting more online components (beta=1.95, P=.02), and parents who wanted fun games wanted more interactivity (beta=2.28, P=.01).
Conclusions: Formative evaluation based on user-centered approaches can provide rich information that fuels development of an online program. The user-centered strategies in this study lay the foundation for improving SFP Online and provide a means to accommodate user interests and ensure the product serves as an effective prevention tool that is attractive to consumers, engaging, and can overcome some of the barriers to recruitment and retention that have previously affected program outcomes in family-based prevention.
Background: Increasing pressure from governments, public health bodies, and consumers is driving a need for increased food-based information provision in eating-out situations. Meals eaten outside the home are known to be less healthy than meals eaten at home, and consumers can complain of poor information on the health impact and allergen content of meals eaten out.
Objective: This paper aimed to describe the development and early assessment of a mobile phone app that allows the provision of accurate personalized food-based information while considering individual characteristics (allergies, diet type, and preferences) to enable informed consumer choice when eating out.
Methods: An app was designed and developed to address these requirements using an agile approach. The developed app was then evaluated at 8 public engagement events using the System Usability Scale (SUS) questionnaire and qualitative feedback.
Results: Consideration of the literature and consultation with consumers revealed a need for information provision for consumers in the eating-out situation, including the ability to limit the information provided to that which was personally relevant or interesting. The app was designed to provide information to consumers on the dishes available in a workplace canteen and to allow consumers the freedom to personalize the app and choose the information that they received. Evaluation using the SUS questionnaire revealed positive responses to the app from a range of potential users, and qualitative comments demonstrated broad interest in its use.
Conclusions: This paper details the successful development and early assessment of a novel mobile phone app designed to provide food-based information in an eating-out situation in a personalized manner.
Background: Owing to the revolution in technology, the internet has become an important aspect of people’s lives. Modern technology is enabling people from diverse educational backgrounds to use the internet for several purposes, one of which is health information seeking. Recently, Web-based health information has become more popular among patients all over the world and among the general public. Objective: This study aimed to investigate the use of Web-based health resources among undergraduate students at Kuwait University. Methods: The study employed a cross-sectional design with students selected from 8 faculties of Kuwait University, 4 faculties of Literature and 4 faculties of Science. Data were collected using structured questionnaires, and analysis was done using a chi-square test and binary logistic regression to determine the factors associated with seeking health information on the Web. Results: The sample size obtained was 1132 with a response rate of 90.27% (1132/1254). Overall, the prevalence of students seeking Web-based health information was 92.66%. (1049/1132) The most significant factors associated with seeking health information on the Web were age, gender, faculty, year of study, primary source of internet, and level of experience with internet use. In total, 90.0% (325/361) of students who were aged older than 21 years used Web-based health information compared with 82.8% (275/332) of those who were aged 18 years. In addition, female students showed a higher prevalence (829/934, 88.8%) of Web-based health information seeking than males (210/270, 77.8%). All the differences found in the study were significant (P value <.05). Conclusions: The study concluded that many people use the internet for seeking health information. Sociodemographic factors have a significant association with Web-based health information seeking. Therefore, doctors must educate the public about the health information websites that can be trusted.
Background: Virtual reality (VR) has demonstrated efficacy for distraction from pain-related thoughts and exposure to feared movements. Little empirical VR research has focused on chronic pain management. Objective: The purpose of this study was to examine the feasibility of VR as an adjunctive intervention for Veterans with chronic pain. We designed a hierarchy ranging from low-intensity pain distraction to high-intensity movement-based exposure for this purpose. VR apps were mapped onto the hierarchy. Methods: Sixteen Veterans receiving inpatient chronic pain rehabilitation participated in daily VR sessions over a 3-week period. Trajectories across the distraction-to-exposure hierarchy and Veteran-reported intensity ratings were described and evaluated over time. Minimum clinically important differences (MCIDs), pre-post effect sizes, and 95% confidence intervals were examined for fear of movement using the Fear of Daily Activities Questionnaire (FDAQ) and Pain Outcomes Questionnaire-VA (POQ-VA; fear scale). This approach was applied to secondary outcomes: POQ-VA (pain intensity, interference, negative affect), Pain Catastrophizing Scale, and Patient-Specific Functioning Scale (PSFS). Session attendance, completion, and VR experiences were described. Results: Ten of 14 Veterans (71%) who participated in three or more VR sessions completed the distraction-to-exposure hierarchy. Only three trajectories emerged more than once. Due to high completion rates, Veterans that completed the hierarchy could self-select nonhierarchy apps. Veterans rated all hierarchy levels (low, medium, high) near medium intensity. Self-selected activities were rated as high intensity. For kinesiophobia, six Veterans (38%) exceeded the MCID on the FDAQ and a small effect size improvement was observed (Cohen d=−0.35). The confidence interval (95% CI −0.71 to 0.01) indicated the possibility of a null effect. The POQ-VA fear scale yielded no effect (Cohen d=0.06, 95% CI −0.43 to 0.54). For secondary outcomes, Veterans exceeding MCID were calculated with complete data: pain intensity (1/15, 7%), pain catastrophizing (5/14, 36%), and patient-specific functioning (10/15, 67%). Effect sizes were large for patient-specific functioning (Cohen d=1.14, 95% CI 0.50-1.78), medium for mobility interference (Cohen d=−0.56, 95% CI −0.96 to −0.16), and small for pain intensity (Cohen d=−0.40, 95% CI −0.69 to −0.12) and catastrophizing (Cohen d=−0.41, 95% CI −0.79 to −0.02). No effects were observed for interference in daily activities (Cohen d=0.10, 95% CI −0.27 to 0.47) and negative affect (Cohen d=0.07, 95% CI −0.26 to 0.40). Veterans attended 85.2% (98/108) of VR sessions and completed 95% (93/96) of sessions attended. Twenty-minute sessions were rated as too short. No significant adverse events were reported. Conclusions: Findings support the feasibility of VR as an adjunct for Veterans with chronic pain. However, the hierarchy will require modification, as evidenced by homogeneous intensity ratings. Veteran-selected activities presented the highest intensity ratings, largest outcome effect size (PSFS), and MCID. This highlights the important role of utilizing Veteran stakeholders in hierarchy modification, design of VR interventions, and outcome selection.
Background: Health apps for the screening and diagnosis of mental disorders have emerged in recent years on various levels (eg, patients, practitioners, and public health system). However, the diagnostic quality of these apps has not been (sufficiently) tested so far. Objective: The objective of this pilot study was to investigate the diagnostic quality of a health app for a broad spectrum of mental disorders and its dependency on expert knowledge. Methods: A total of 2 psychotherapists, 2 psychology students, and 2 laypersons each read 20 case vignettes with a broad spectrum of mental disorders. They used a health app (Ada—Your Health Guide) to get a diagnosis by entering the symptoms. Interrater reliabilities were computed between the diagnoses of the case vignettes and the results of the app for each user group. Results: Overall, there was a moderate diagnostic agreement (kappa=0.64) between the results of the app and the case vignettes for mental disorders in adulthood and a low diagnostic agreement (kappa=0.40) for mental disorders in childhood and adolescence. When psychotherapists applied the app, there was a good diagnostic agreement (kappa=0.78) regarding mental disorders in adulthood. The diagnostic agreement was moderate (kappa=0.55/0.60) for students and laypersons. For mental disorders in childhood and adolescence, a moderate diagnostic quality was found when psychotherapists (kappa=0.53) and students (kappa=0.41) used the app, whereas the quality was low for laypersons (kappa=0.29). On average, the app required 34 questions to be answered and 7 min to complete. Conclusions: The health app investigated here can represent an efficient diagnostic screening or help function for mental disorders in adulthood and has the potential to support especially diagnosticians in their work in various ways. The results of this pilot study provide a first indication that the diagnostic accuracy is user dependent and improvements in the app are needed especially for mental disorders in childhood and adolescence.
Background: Traditional dietary recommendations for achieving optimal gestational weight gain are ineffective for pregnant women due to the lack of real-time communication and tedious consultation processes. Objective: In this pilot study, we aimed to determine the feasibility of a novel food-coaching smartphone app for controlling gestational weight gain and macronutrient intake among overweight and obese pregnant women. Methods: We designed a randomized controlled trial and recruited 30 overweight and obese pregnant women (1:1 ratio) during 18-20 weeks of gestation and followed them up after 4 and 8 weeks, respectively. Both groups received standard pregnancy dietary orientation at recruitment, while the intervention group received 8 weeks of real-time food coaching via a smartphone app. This food coaching smartphone app (Glycoleap, Holmusk, Singapore) aimed to improve care and outcomes for people with diabetes. Pregnant women using this app were able to upload food images (eg, a picture of a meal, a drink, or a dessert) and received real-time and detailed food-coaching comments and guidance provided by professional dietitians during the day (8 AM to 8 PM). We recorded detailed characteristics during recruitment and examined anthropometry at all visits. We compared the mean differences of the 8-week gestational weight gain and macronutrient intake between the two groups. Results: Upon study completion, three subjects dropped out from the intervention, and one gave birth prematurely in the control group. The acceptance rate of the smartphone app was 90%. More participants achieved optimal gestational weight gain per week in the intervention group (8/12, 67%) than in the control group (5/14, 36%). After the 8-week intervention, women in the intervention group appeared to have lower gestational weight gain (mean difference=–0.08 kg; 95% CI –1.80 to 1.63) and cholesterol intake (mean difference=–31.73 mg; 95% CI –102.91 to 39.45) than those in the control group. Conclusions: Our findings showed that this food-coaching smartphone app is feasible and favorable for weight gain control and cholesterol intake control among overweight and obese pregnant women. Although our results were not significant (perhaps, attributed to the small sample size), it provided proof of concept for the feasibility of applying such technology in future randomized controlled trials with a larger sample size, an earlier intervention onset, and a longer follow-up for overweight and obese pregnant women.
