Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Monday, March 11, 2019 at 4:00 PM to 4:30 PM EST. We apologize in advance for any inconvenience this may cause you.
Who will be affected?
Readers: No access to all 28 journals. We recommend accessing our articles via PubMed Central
Authors: No access to the submission form or your user account.
Reviewers: No access to your user account. Please download manuscripts you are reviewing for offline reading before Monday, March 11, 2019 at 4:00 PM.
Editors: No access to your user account to assign reviewers or make decisions.
Copyeditors: No access to user account. Please download manuscripts you are copyediting before Monday, March 11, 2019 at 4:00 PM.
JMIR Formative Research
Process evaluations, early results, and feasibility/pilot studies of digital and non-digital interventions
Background: The NHS’ ‘Long term plan’ details plans to make digital interactions available to all patients in five years. Teleconsultations can improve access to specialist services however there is a lack of UK evidence for the use of teleconsultations in an Oncology setting. Here we describe a service evaluation of teleconsultations for patients attending a regional brain metastases (BM) clinic. These patients have unique travel restrictions that prevent them from driving. Objective: To assess the feasibility and acceptability of a teleconsultation service in this setting. Methods: From April 2018 to October 2018 all patients attending the BM clinic were offered the choice of a teleconsultation in place of a face-to-face appointment. Feedback was assessed using a satisfaction questionnaire and data was collected on all clinic attendances. Results: Sixty-nine individual patients had 119 appointments over the duration of the pilot. Of these, 36 were new patient appointments and 73 were follow ups. Of these, 24 patients (35%) took part in 41 (34%) teleconsultations. User satisfaction was high and no patients who took part in a teleconsultation reverted to face-to-face appointments. These patients avoided 2521 miles (61.6 miles per appointment) of hospital associated travel and travel costs of £441.48 (£10.78 per appointment). Conclusions: Teleconsultations appear to be acceptable in this cohort of patients who have brain metastases attending a regional stereotactic radiosurgery service with the potential for significant savings in travel and expenses evident.
(Toronto, Nov 29th, 2018) We are pleased to report that JMIR Formative Res has been accepted for indexing in PubMed Central and PubMed. The staff at PubMed Central is currently finalizing the setup and we expect all articles published since Vol 1 / Iss 1 (2017) to be in PubMed Central and PubMed within the next couple of weeks. The acceptance is the result of a rigorous scientific and technical evaluation by the US National Library's (NLM’s) Library Operations Division, which decided that the scientific and editorial character and quality of a JMIR Formative Research merits its inclusion in PMC. In making this decision NLM has considered the suitability of the journal for the NLM collection as well as the opinions of expert consultants.
JMIR Formative Research (JFR) (a PubMed-indexed sister journal of J Med Internet Res (JMIR) and JMIR mHealth & uHealth, the leading eHealth and mHealth journals by impact factor) publishes peer-reviewed, openly accessible papers containing results from process evaluations, feasibility/pilot studies and other kinds of formative research and preliminary results. While the original focus was on the design of medical and health-related research and technology innovations, JFR publishes studies from all areas of medical and health research.
Formative research is research that occurs before a program is designed and implemented, or while a program is being conducted. Formative research can help
define and understand populations in need of a intervention or public health program
create programs that are specific to the needs of those populations
ensure programs are acceptable and feasible to users before launching
improve the relationship between users and agencies/research groups
demonstrate the feasibility, use, satisfaction with, or problems with a program before large-scale summative evaluation (looking at health outcomes)
Many funding agencies will expect some sort of pilot/feasibility/process evaluation before funding a larger study such as a Randomized Controlled Trial (RCT).
Formative research should be an integral part of developing programs or adapting programs, and should be used while the program is on-going to help refine and improve program activities. Thus, formative evaluation can and should also occur in form of a process evaluation alongside a summative evaluation such as a RCT.
This journal fills an important gap in the academic journals landscape, as it publishes sound and peer-reviewed formative research that is criticial for investigators to apply for further funding, but that is usually not published in outcomes-focussed medical journals aiming for impact and generalizability.
Summative evaluations of programs and apps/software that have undergone a thorough formative evaluation before launch have a better chance to be published in our high-impact flagship journals (JMIR mHealth, J Med Internet Res, JMIR Serious Games), thus we encourage authors to submit - as a first step - their formative evaluations in JMIR Formative Res (and their evaluation protocols in JMIR Res Protoc).
Background: The impact of mental ill-health on every aspect of the lives of a large number of Australian Defence Force (ADF) personnel, their partners, and their families is widely recognized. Recent Senate inquiries have highlighted gaps in service delivery as well as the need for service reform to ensure appropriate care options for individuals who are currently engaged with mental health and support services as well as for those who, for a variety of reasons, have not sought help. To that end, successive Australian governments generally and the Department of Veterans’ Affairs specifically have prioritized veteran-centric reform. Open Arms is an Australia-wide service that provides counseling and support to current and former ADF personnel, and their family members, for mental health conditions. Objective: The aim of this study was to develop and configure a prototypic Web-based platform for Open Arms—Veterans & Families Counselling (formerly Veterans and Veterans Families Counselling Service) with the Open Arms community to enhance the quality of mental health services provided by Open Arms. Methods: The study aimed to recruit up to 100 people from the Open Arms community (current and former ADF personnel and their families, health professionals, service managers, and administrators) in regions of New South Wales, including Sydney, Canberra, Maitland, Singleton, and Port Stephens. Participants were invited to participate in 4-hour participatory design workshops. A variety of methods were used within the workshops, including prompted discussion, review of working prototypes, creation of descriptive artifacts, and group-based development of user journeys. Results: Seven participatory design workshops were held, including a total of 49 participants. Participants highlighted that the prototype has the potential to (1) provide the opportunity for greater and better-informed personal choice in relation to options for care based on the level of need and personal preferences; (2) ensure transparency in care by providing the individual with access to all of their personal health information; and (3) improve collaborative care and care continuity by allowing information to be shared securely with current and future providers. Conclusions: Our findings highlight the value of actively engaging stakeholders in participatory design processes for the development and configuration of new technologies.
Background: Psychiatry research has begun to leverage data collected from patients’ social media and smartphone use. However, information regarding the feasibility of utilizing such data in an outpatient setting and the acceptability of such data in research and practice is limited. Objective: This study aimed at understanding the outpatients’ willingness to have information from their social media posts and their smartphones used for clinical or research purposes. Methods: In this survey study, we surveyed patients (N=238) in an outpatient clinic waiting room. Willingness to share social media and passive smartphone data was summarized for the sample as a whole and broken down by sex, age, and race. Results: Most patients who had a social media account and who were receiving talk therapy treatment (74.4%, 99/133) indicated that they would be willing to share their social media posts with their therapists. The percentage of patients willing to share passive smartphone data with researchers varied from 40.8% (82/201) to 60.7% (122/201) depending on the parameter, with sleep duration being the parameter with the highest percentage of patients willing to share. A total of 30.4% of patients indicated that media stories of social media privacy breaches made them more hesitant about sharing passive smartphone data with researchers. Sex and race were associated with willingness to share smartphone data, with men and whites being the most willing to share. Conclusions: Our results indicate that most patients in a psychiatric outpatient setting would share social media and passive smartphone data and that further research elucidating patterns of willingness to share passive data is needed.
Background: Persons living with dementia represent a significant and growing segment of the older adult (aged 65 years and older) population. They are often challenged expressively and may experience difficulties with sharing their feelings or moods. Availability of, and easy access to, tablets facilitates the use of information and communication technologies (ICTs) as a delivery mechanism for nonpharmacological interventions, especially for persons living with dementia. Evidence of the impact of ICTs in different community settings on mood with older adults and the impact of engagement on their caregivers is needed to promote broader adoption and sustainment of these technologies in the United States. Objective: This study aimed to determine the extent of the effects of tablets on positive mood change and examine the effects of study variables on care recipients’ mood changes and caregivers’ daily interactions. Methods: The tablet intervention was developed and evaluated in five programs. The primary outcome was caregivers’ assessment of care recipients’ mood (n=1089) before and after a tablet engagement session using an eight-point mood visual analog scale. Session influence on caregivers’ daily activities was captured for a subsample of participants (n=542). Frequency distributions were computed for each study variables. Chi-square tests of association were calculated to determine the association of the variables on mood changes for all care recipients, as well as those being treated in skilled nursing facilities and in-home, and then for those that affected caregivers’ daily activities. Results: The study sample comprised 1089 care recipient and caregiver engagement sessions. Cumulatively, 50.78% (553/1089) of care recipients showed a transition from negative to positive moods, whereas another 41.78% (455/1089) maintained an already-positive mood after the caregiver engagement session. Chi-square analyses demonstrated that positive mood changes resulted from using music (χ210=72.9; P<.001), using YouTube as the sole app (χ212=64.5; P<.001), using multiple engagement strategies (χ22=42.8; P<.001), and when cared for in a skilled nursing facility (χ24=236.8; P<.001) across the entire care recipient sample. In addition, although many features of the engagement session positively influenced the caregivers’ day, the largest effect was observed when care recipients’ mood was considered to have improved following the session (χ24=234.7; P<.001). Conclusions: The study is one of the first in the United States to explore the impact of ICTs, in particular managed tablets and Web-based video services that can be used on a tablet through an app, on improving mood in persons living with dementia, and enhancing caregivers’ perceptions about their care recipient interactions. Importantly, these pilot data substantiate ICTs as part of a personalized engagement approach, as beneficial alternatives to pharmaceutical interventions for mood enhancement. However, a more comprehensive study that explores the ICT’s impact on additional clinical outcomes is needed to confirm these preliminary findings.
Background:
Dengue is a mosquito-borne viral disease that has increasingly affected Sri Lanka in recent years. To address this issue, dengue surveillance through increasingly prevalent digital surveillance applications has been suggested for use by health authorities and the general public. Epihack Sri Lanka was a 5-day hackathon event organized to develop a digital dengue surveillance tool.
Methods:
Ethnographic observation and qualitative informal interviews were conducted with 58 attendees from 11 countries over the 5-day Epihack to identify the main factors that influence a collaborative hackathon. Interviews were transcribed and coded based on grounded theory.
Results:
Three major themes were identified during the Epihack Sri Lanka event: engagement, communication, and current disease environment. Unlike other hackathons, Epihack had no winners or prizes and was collaborative rather than competitive, which worked well in formulating a variety of ideas and bringing together volunteers with a sense of civic duty to improve public health. Having health and IT experts work together concurrently was received positively and considered highly beneficial to the development of the product. Participants were overall very satisfied with the event, although they thought it could have been longer. Communication issues and cultural differences were observed but continued to decrease as the event progressed. This was found to be extremely important to the efficiency of the event, which highlighted the benefit of team-bonding exercises. Bringing expert knowledge and examples of systems from around the world benefited the creation of new ideas. However, developing a system that can adapt and cater to the local disease environment is important in successfully developing the concepts.
Conclusions:
Epihack Sri Lanka was successful in bringing together health and IT experts to develop a digital solution for dengue surveillance. The collaborative format achieved a variety of fruitful ideas and may lead to more hackathons working in this way in the future. Good communication, participant engagement, and stakeholder interest with adaptation of ideas to complement the current environment are vital to achieve the goals of the event.
Background: Whiplash is a health and economic burden worldwide. Contributing to this burden is poor guideline adherence and variable management by health care professionals (HCPs). Web-based tools that facilitate clinical pathways of care are an innovative solution to improve management. Objective: The study aimed to develop, implement, and evaluate a Web-based tool to support whiplash management following a robust process. Methods: This study followed the first 3 processes of a research translation framework (idea generation, feasibility, and efficacy) to inform the development, implementation, and evaluation of a website that supports HCPs in whiplash management. Development followed the idea generation and feasibility processes to inform the content, design, features, and functionality of the website. This involved stakeholder (eg, industry partners, website developers, and HCPs) consultations through face-to-face meetings, surveys, and focus group discussions. Implementation followed the feasibility process to determine the practicality of the website for clinical use and the most effective strategy to promote wider uptake. Implementation strategies included classroom education, educational meetings, educational outreach, reminders, and direct phone contact. The analysis of website use and practicality of implementation involved collection of website metrics. Evaluation followed the feasibility and efficacy processes to investigate the acceptability and extent to which the website assisted HCPs in gaining knowledge about whiplash management. Surveys were conducted among student, primary, and specialist HCPs to explore ease of access, use, and satisfaction with the website, as well as self-rated improvements in knowledge of risk assessment, management, and communication between HCPs. Website logs of specialist management decisions (eg, shared care, specialist care, and referred care) were also obtained to determine actual practice. Results: The development process delivered an interactive, user-friendly, and acceptable website, My Whiplash Navigator, tailored to the needs of HCPs. A total of 260 registrations were recorded from June 2016 to March 2018, including 175 student, 65 primary, and 20 specialist HCPs. The most effective implementation strategies were classroom education for students (81% uptake, 175/215) and educational meetings for primary HCPs (43% uptake, 47/110). Popular pages visited included advice and exercises and risk assessment. Most HCPs agreed that their knowledge about risk management (79/97, 81%) and exercises (85/97, 88%) improved. The specialists’ most common management decision was shared care, an improvement from a previous cohort. Areas to improve were navigation and access to outcome measures. Conclusions: A robust process resulted in an innovative, interactive, user-friendly, and acceptable website, the My Whiplash Navigator. Implementation with HCPs was best achieved through classroom education and educational meetings. Evaluation of the website showed improved knowledge and practice to be more consistent with a risk-based clinical care pathway for whiplash. The positive results provide sufficient evidence to scale implementation nationally and involve other target markets such as people with whiplash, insurers, and insurance regulators.
Background: The majority of people with spinal cord injury (SCI) in the United States are not meeting the recommended guidelines for regular physical activity. Behavior change techniques (eg, goal setting and action planning) that are framed within the principles of the social cognitive theory (self-efficacy and self-regulation) have the potential to enhance physical activity behavior. Objective: The aim of the study was to develop and test the usability of an electronic learning (e-learning) program for improving social cognitive factors related to physical activity behavior among people with SCI. Methods: The program was created through an iterative process of development and refinement, using a modification of a similar methodology used to develop evidence-informed guidelines in health promotion for people with disabilities (Guidelines, Recommendations, and Adaptations Including Disability; GRAIDs framework). The study included 4 phases: (1) initial product creation, (2) national survey, (3) expert review, and (4) usability testing. Usability testing included both quantitative and qualitative data collection and analyses. Results: The review of the program by an expert panel (n=5) and the results from a national survey (n=142) led to several refinements. Usability testing demonstrated that the program could be completed in a timely manner (<30 min). Participants reported 5 themes: (1) the program improves social cognitions related to physical activity participation; (2) reflection of physical activity behavior; (3) positive perceptions of the quality of the program; (4) positive perceptions of the program operation and effectiveness; and (5) recommendations for improvement. Each item was incorporated into a revised program version 1.0. Conclusions: This study incorporated an evidence-based framework for developing a brief 30-min e-learning program for increasing the physical activity behavior among people with SCI. The Exercise Strategies Through Optimized Relevant Interactive E-learning Storytelling (e-STORIES) program could be completed in a timely manner and was reported by participants as valuable and useful for enhancing intent-to-perform physical activity in individuals with SCI. The program has the potential to be applied in a variety of settings, but feasibility testing is required before implementing in a larger trial.
Background:
Accelerometers, often in conjunction with heart rate sensors, are extensively used to track physical activity (PA) in research. Research-grade instruments are often expensive and have limited battery capacity, limited storage, and high participant burden. Consumer-based activity trackers are equipped with similar technology and designed for long-term wear, and can therefore potentially be used in research.
Methods:
A total of 50 participants wore 2 ActiGraphs (wrist and hip), 2 Actihearts (upper and lower chest position), and 1 Polar M430 sport watch for 1 full day. We compared reported time (minutes) spent in sedentary behavior and in light, moderate, vigorous, and moderate to vigorous PA, step counts, activity energy expenditure, and total energy expenditure between devices. We used Pearson correlations, intraclass correlations, mean absolute percentage errors (MAPEs), and Bland-Altman plots to assess criterion validity.
Results:
Pearson correlations between the Polar M430 and all research-grade instruments were moderate or stronger for vigorous PA (r range .59-.76), moderate to vigorous PA (r range .51-.75), steps (r range .85-.87), total energy expenditure (r range .88-.94), and activity energy expenditure (r range .74-.79). Bland-Altman plots showed higher agreement for higher intensities of PA. MAPE was high for most outcomes. Only total energy expenditure measured by the hip-worn ActiGraph and both Actiheart positions had acceptable or close to acceptable errors with MAPEs of 6.94% (ActiGraph, 3 axes), 8.26% (ActiGraph, 1 axis), 14.54% (Actiheart, upper position), and 14.37% (Actiheart, lower position). The wrist-worn ActiGraph had a MAPE of 15.94% for measuring steps. All other outcomes had a MAPE of 22% or higher. For most outcomes, the Polar M430 was most strongly correlated with the hip-worn triaxial ActiGraph, with a moderate or strong Pearson correlation for sedentary behavior (r=.52) and for light (r=.7), moderate (r=.57), vigorous (r=.76), and moderate to vigorous (r=.75) PA. In addition, correlations were strong or very strong for activity energy expenditure (r=.75), steps (r=.85), and total energy expenditure (r=.91).
Conclusions:
The Polar M430 can potentially be used as an addition to established research-grade instruments to collect some PA variables over a prolonged period. However, due to the high MAPE of most outcomes, only total energy expenditure can be trusted to provide close to valid results. Depending on the variable, the Polar M430 over- or underreported most metrics, and may therefore be better suited to report changes in PA over time for some outcomes, rather than as an accurate instrument for PA status in a population.
Background: Mountain biking is an aerobic physical activity that has experienced rapid growth. The emergence of the electric pedal-assist mountain bike (eMTB), while not without its critics, presents the potential for an even larger segment of the population to enjoy the health benefits of mountain biking. Although the research focused on the use of e-bikes generally is growing, there is limited research specifically targeting eMTB use. Research is needed exploring the potential exercise response of riding an eMTB, together with the beliefs and perceptions of mountain bikers who have and have not experienced eMTB riding. Objective: This study aimed to compare conventional mountain bike and eMTB use. This was done by investigating 2 questions: (1) What proportion of exercise response is retained for an experienced mountain biker while using an eMTB when compared with a conventional mountain bike? and (2) What are the perceptions and beliefs of experienced mountain bikers toward eMTBs both before and after riding an eMTB? Methods: A convergent mixed methods data collection approach was used in the study. Participants completed both a pre- and postride questionnaire, and data regarding heart rate were collected. Heart rates from each ride were compared against each other. Results: The average heart rate during eMTB use was 94% (31/33) of the average heart rate during conventional mountain bike use. Therefore, eMTB use in this study achieved a majority of the exercise response and exceeded established biometric thresholds for cardiovascular fitness. Paired t test statistics were calculated to compare beliefs of conventional mountain bikes and eMTBs and to compare mean heart rate and speed between conventional mountain bike and eMTB use on the study loop. Participants overwhelmingly perceived the potential impact of eMTB use to be positive on both pre- and post-eMTB ride questionnaires. Conclusions: Despite the measured benefit, participants’ perceived exertion while riding the eMTB was low.
Background: Although we know that many young people watch online pornography and engage in sexting, there is limited literature examining their needs in relation to information on these highly sensitive and complex issues. Online resources exist; however, we can find no evidence of any of them having been formally tested for usability within the target population. Objective: This study aimed to test the usability of a resource about online pornography and sexting among young people. Methods: Semistructured interviews were conducted with 17 participants aged 15 to 29 years. Results: We found that the SCOPE resource was perceived as trustworthy and credible because of its evidence-based content, nonjudgmental tone, and balanced perspectives. Multimedia and video content enhanced the layout and usability of the resource; however, content relevance could be improved by targeting age and developmental stages. Participants identified resource sections such as Real Stories from young people as relevant and engaging. However, they raised issues with the translation of formative research findings relating to these stories into their final presentation. Conclusions: Our findings suggest that young people prefer online resources about complex issues, such as online pornography and sexting, if they are balanced in content and tone. Most importantly, in the context of responding to complex and sensitive issues such as these, co-design methods can ensure that young people are central to the development of resources and avoid gaps in translating research into practice. In the context of limited literature focusing on the usability of online resources about these topics, this paper provides important insights for public health practitioners working in this emerging space.
Background: An estimated one in eight cervical cancer cases are due to a lack of follow-up care for abnormal Pap test results. Low rates of completion of follow-up care particularly affect low-income minority women. The burden of cervical cancer could be reduced through interventions that improve timely colposcopy follow-up and treatment of abnormal screening results. Mobile communications via text messaging present a low-cost opportunity to increase rates of clinic return among women referred to follow-up after obtaining abnormal screening results. Objective: Our aims were to determine the acceptability and feasibility of using text messaging to increase completion of follow-up care following abnormal cervical cancer screening (Pap test) results and to examine factors that may affect the acceptability and use of text messaging to increase communications between health care providers (HCP) and low-income minority women. Methods: The study participants were 15 low-income women who had undergone a Pap test within the preceding 12 months. Semistructured interviews, including open- and closed-ended questions from a validated questionnaire, were conducted by phone or in person. Responses to closed-ended survey items were tabulated, and descriptive statistics were generated using Microsoft Excel. Responses to the open-ended questions were coded and analyzed using NVivo 11 qualitative analysis software. Results: Nearly all participants (14/15, 93%) were comfortable receiving a text message from an HCP stating that their Pap test results were available (<40 years: 100%; ≥40 years: 86%). Over half (8/15; 53%) of the participants were comfortable receiving a text message stating that their Pap test results were abnormal, although many preferred to receive such information via a phone call (6/15; 40%). Most participants (9/15; 60%) believed that receiving a text reminder would make them more likely to attend their appointment. The preferred method for receiving a reminder appeared to vary by age, with older women preferring telephone reminders over text messaging reminders. Analysis of open-ended questions suggested that text messaging appeals to some women due to its wide use and convenience for communicating with HCPs. However, women cited concerns about the confidentiality of messages and barriers to understanding the messages, including the physical capacity to read and accurately interpret the content of the messaging. Conclusions: Most participants indicated a willingness to receive text messages from their HCPs about cervical cancer screening results and believed that text messages were the best way to remind them of appointments for follow-up care. Potential concerns could be addressed by excluding explicit references to the nature of the appointment in the text message in order to avoid disclosure of sensitive health information to unauthorized individuals. Although text messaging seems promising to improve adherence to timely follow-up, personal preferences should be considered by allowing patients to opt-out of text communications.
Background: The NHS’ ‘Long term plan’ details plans to make digital interactions available to all patients in five years. Teleconsultations can improve access to specialist services however ther...
Background: The NHS’ ‘Long term plan’ details plans to make digital interactions available to all patients in five years. Teleconsultations can improve access to specialist services however there is a lack of UK evidence for the use of teleconsultations in an Oncology setting. Here we describe a service evaluation of teleconsultations for patients attending a regional brain metastases (BM) clinic. These patients have unique travel restrictions that prevent them from driving. Objective: To assess the feasibility and acceptability of a teleconsultation service in this setting. Methods: From April 2018 to October 2018 all patients attending the BM clinic were offered the choice of a teleconsultation in place of a face-to-face appointment. Feedback was assessed using a satisfaction questionnaire and data was collected on all clinic attendances. Results: Sixty-nine individual patients had 119 appointments over the duration of the pilot. Of these, 36 were new patient appointments and 73 were follow ups. Of these, 24 patients (35%) took part in 41 (34%) teleconsultations. User satisfaction was high and no patients who took part in a teleconsultation reverted to face-to-face appointments. These patients avoided 2521 miles (61.6 miles per appointment) of hospital associated travel and travel costs of £441.48 (£10.78 per appointment). Conclusions: Teleconsultations appear to be acceptable in this cohort of patients who have brain metastases attending a regional stereotactic radiosurgery service with the potential for significant savings in travel and expenses evident.
