Principal Findings
In this internet-based, pilot study of predominantly young and middle-age, educated, White patients, using a semiautomated, edge-based, mHealth app that uses individualized data for tailored management of CRMCs, we made several key observations with regard to both the internet-based recruitment approach to the study of mHealth apps, as well as the specific usage rates and patterns of use by participants. First, we found that while an internet-based recruitment approach was superior to “grass-roots” local methods of recruiting participants on our campus using fliers and word-of-mouth, the 3-month follow-up rates were only slightly above 50% (103/180), indicating that future studies using this type of methodology targeted to achieve a prespecified degree of statistical power will need to account for a high number of dropouts. Second, we found that among those with complete follow-up, the 3-month adoption rate of the iMTracker app was about 70.9% (73/103), with the most common CRMCs that users chose to self-apply the app being chronic pain, headaches, and mental health conditions. This information not only provides conditions and clinical settings in which to target future clinical trials, but also indicates that there may be a need for better tools to manage these conditions beyond what is presently available in clinical practice. Importantly, we found that on completion of this study, more patients had increased perceptions of the safety, privacy, and time demands with the use of an mHealth app for self-management of CRMCs. Finally, we found that the main barrier to use, based on both subjective and quantitative feedback, was related to the design and workflow of iMTracker itself, which was reflected in the decrease in perception of efficacy noted on completion, and has important implications for future development and testing of this app as well as other mHealth technologies. In other words, this finding indicates that patients are likely to be receptive to the semiautomated N-of-1 trial methodology employed by the iMTracker app, but that greater attention to design and function is needed before moving forward with clinical testing targeted toward improvement in outcomes.
mHealth apps have increased significantly in frequency over the years, with iOS apps including health and fitness groups increasing from 43,000 in 2013 to 98,000 in 2015 [32]. Unfortunately, these tools do not consistently employ best practices for self-management [33], and many of these approaches have failed to reach any meaningful level of adoption across the medical community [32,34], likely due to a lack of formal clinical testing. Our finding that there was an increased concern among patients about the effectiveness of iMTracker is consistent with prior studies of mHealth app for self-management of CRMCs. In a previous investigation, it was found that only 3.4% of apps on the iTunes and Google Play stores promoted for management of depression and anxiety had research to justify their claims of effectiveness, with only 30.4% having expert input in development [35]. A study by Devan et al [36] of 19 apps available commercially for self-management of pain found that only 2 had been validated to improve health outcomes. A similar lack of scientific support for commercially available mental health–targeted [37-39] and pain [25,40] apps has been reported by other investigators. Although we did not inquire about prior use of mHealth apps for self-management, one can infer that most participants in this study had tried prior apps without success. Clinical validation of any mHealth app should be required before integration into the clinical care process, and our study further suggests that while users are optimistic that self-management using an app is possible, follow-up clinical studies will be needed.
Among the characteristics of the specific CRMCs that patients identified for use of an mHealth app, recurrent pain, headaches, and mental health were highly represented. While these diagnoses were self-identified by users, and not validated with clinicians or clinical data (ie, chart review), it does help to identify potential clinics and providers for testing mHealth apps, as would be needed before an app such as the iMTracker could be incorporated into routine clinical care. In addition, the majority of patients noted a high frequency of recurrence of their condition, which is key in determining the number of patients needed for a prospective study to demonstrate efficacy, as a condition that occurred less frequently, for example, once or twice a year, would require an extended amount of time to identify correlation of episodes with other lifestyle and behaviors, or assess the impact of an intervention.
Although our study did not specifically examine clinical efficacy of the iMTracker app in terms of a reduction in the primary CRMC, we did note that there was enthusiasm about future uses, particularly if barriers related to design and function could be addressed. This finding is in line with prior work, such as that by Neuhauser et al [41,42], who noted that participatory methods linked with traditional health communication theory and methods can create effective health communication using artificial intelligence, highlighting the role of design science theory in the development and refinement of mHealth apps. Such insights highlight the challenge that is unique to mHealth, and other health IT apps, in which consideration of user-based preferences and desires must be merged with information and guidance grounded in biology and evidence-based medicine principles. In terms of design life cycles, this requires an integrated design approach with features of both top-down (ie, waterfall) strategies and bottom-up, user-driven design (eg, agile) strategies. Our team has already begun efforts to improve the design and function of the iMTracker app, and future studies will examine the improvement in these changes to make the app more in line with the level of commercial design that many patients have come to expect from all mobile apps, in addition to mHealth.
Despite results to suggest a high degree of potential for clinical application of the iMTracker app, there were several key limitations in our study. First, we generally had a limited amount of follow-up of patients, with roughly half of the patients who enrolled being lost to follow-up. We suspect that this limitation highlighted the challenges of using the internet for recruitment, and the trade-off between use of network-based recruitment methods (ie, online) and in-person clinic recruitment. As an extreme example of the potential of the former, the Apple Watch study recruited over 400,000 participants in an 8-month period using online methods, although only 450 actually returned the confirmatory patches in follow-up [43]. As such, future studies of mHealth technology should consider that the potential benefit in terms of recruitment numbers using internet-based recruitment may accompany a relatively high degree of dropouts. A second limitation was that the population we studied were primarily White, educated, and young/middle age adults; these were individuals who engage regularly with providers using technology, post to social media, and perform self-management with regular weight checks. Missing from our population are older patients, those with less education, and those from underrepresented populations—the type of populations that have also been shown to have less close clinical follow-up for their conditions [44-46], and who might stand to benefit the most from an app that allows self-management. This population bias is critical in considerations of further app development as the design and functionality changes that would typically guide app development would be needed for successful integration with clinical care. Further work is needed on methods to include less represented populations in mHealth studies. Among the issues with biased recruitment was the omission of sex from the baseline variables we collected. Qualitatively (based on the first names of the participants), we suspect recruitment was not heavily imbalanced toward 1 specific sex, although formal assessment would have been beneficial in terms of statistical analysis. In future studies, we plan to examine sex along with other user characteristics in terms of both usage patterns and study participation. Finally, a key limitation of the study was the inability to confirm diagnoses or response to therapy among users of iMTracker. While we have ongoing studies to examine the app prospectively toward clinical outcomes, the findings from this investigation provided important information about which specific conditions, and which types of clinics, to target for recruitment. This finding was critical as the iMTracker was originally developed toward treatment of recurrent cardiac symptoms (ie, palpitations), and yet we identified other conditions that included chronic pain, headaches, and mental health as being more heavily favored by patients in this investigation.
In future studies we plan to examine the impact of specific design and function improvements on iMTracker, including an examination of the use of industry-standard Agile development life cycles to make iterative improvements to the user interface within the Scrum methodology [47]. Our team has already employed a newer user interface and data entry design based on emojis, although additional work is needed to ensure all users of all levels of education and medical literacy can use the app comfortably. Following this step, we plan to target clinical trials of iMTracker to primary care, neurology, and psychiatry clinics to assess the impact of N-of-1 management on recurrence of chronic pain, headaches, and mental health conditions. Finally, our team has continued work on integration of more sophisticated data management approaches using federated and distributive learning, as well as Bayesian-based analytical frameworks, and we plan to examine the improvement in accuracy with these innovations. Ultimately, much work is needed before the iMTracker app can be used routinely in clinics to manage CRMCs, although feedback from this study has helped target efforts toward high-yield conditions and modifications to improve the chances of success.