The study was designed as a prospective development and usability study at St. Olavs University Hospital in Trondheim, Norway, from December 2019 to March 2020. Adults with migraine were recruited from the outpatient headache clinic and the municipality using the hospital intranet and an advertisement in the news. A total of 18 participants were included in the study. We did not conduct a formal a priori sample size calculation. The sample size was based on recommended guidelines for a sample size of usability studies [21,22]. All diagnoses were confirmed by a consultant neurologist with headache expertise. Because there were only 10 pairs of sensors available for the study, participants were divided into two groups before completing a 4-week period of app testing at home (10 participants in the first group and 8 participants in the second group). The 4-week period was preceded by a preusability expectations interview and succeeded by a postusability experience interview. Participants also completed an evaluation questionnaire at weeks 2 and 4 of the test period. The study was approved by the regional committee for medical and health research ethics (REK Midt 7166) and the Norwegian Medicines Agency for trials of medical equipment (19/11730-9).

Inclusion criteria were aged 18 to 65 years; migraine with or without aura diagnosed according to the International Classification of Headache Disorders, Third Edition [23]; 2 to 8 attacks per month; experience with using a smartphone; and signed written informed consent. Exclusion criteria were lack of proficiency in the Norwegian language; reduced vision, hearing, or sensibility to a degree that hampered study participation; or any severe neurological or psychiatric disorders.

The Cerebri (Nordic Brain Tech AS) biofeedback setup consisting of a smartphone app connected to wearables was used for self-administered biofeedback treatment. The setup was developed based on similar equipment used in 2 previous studies at St. Olavs University Hospital. In the first of these studies, a biofeedback treatment app was developed in an iterative and incremental fashion, in which adolescents completed 3 cycles of usability testing, and improvements and changes to the app interface were completed between cycles [24,25]. A small compact SEMG sensor was used for measuring muscle tension from the upper trapezius muscle fibers. A combined device including 2 sensors was attached to the right index finger to measure peripheral skin temperature and heart rate (Figure 1 displays the sensors). Both devices transmitted signals to the app via Bluetooth Smart.

The app included biofeedback training, instructions for self-delivery, and a headache diary. Participants were given a push reminder daily to complete a headache diary entry and biofeedback session of 10 minutes. Prior to commencing treatment, participants were provided with a brief explanation of how biofeedback treatment works and how to complete the biofeedback training sessions. One of the investigators helped participants run through a trial session using both sensors and the app. Participants were encouraged to try to relax and sit comfortably during the sessions. No detailed instructions on how to perform the biofeedback were given. Figure 2 displays screenshots of the app. All participants were provided with a box containing both sensors, a power charger, and additional SEMG electrode patches. Participants used their own smartphones throughout the trial.

Upon inclusion, participants met with one of the researchers to download and start using the app. A semistructured preusability expectations interview was completed (Multimedia Appendix 1) before starting a 4-week period using the app and sensors.

During the 4 weeks, participants were prompted to complete daily biofeedback sessions and daily headache diary entries (Multimedia Appendix 2). Participants were given an evaluation questionnaire at 2 weeks and 4 weeks after commencing use (Multimedia Appendix 3). The evaluation questionnaire was similar to the one used in a recent adolescent usability study of a similar biofeedback setup [25]. The evaluation questionnaire included the following 5 main domains, engagement, functionality, design, information, and biofeedback, corresponding to the mobile app rating scale [26]. Questions were answered on a 5-point Likert scale, ranging from 1 = completely disagree to 5 = completely agree. Also, participants were encouraged to take notes of any adverse events and discomforts during the treatment period.

At the end of the 4-week home use period, participants met with one of the researchers to return the equipment and complete a semistructured postusability experience interview (Multimedia Appendix 1). At this point, participants were explicitly asked to report any skin reactions, dizziness, and nausea and openly questioned on other adverse events. All adverse events were recorded with their seriousness and the potential grade of causality.

We reported the number of completed sessions and calculated the adherence rate as the proportion of sessions completed within the first 28 days. Only sessions in which the full 10 minutes of biofeedback training was finished were considered as completed sessions. Sessions were automatically marked as completed within the app software, and the information was transferred to the database. No self-reported measure of adherence was made. We also described what weekdays and what time of the day sessions were completed.

Self-reported overall hours of phone use and familiarity with apps and sensors were averaged over the 2 evaluation questionnaires for each participant. Scores were averaged over each domain for all participants for the week 2 and week 4 evaluation questionnaires and summarized with medians and interquartile ranges. We only used complete data in the evaluation questionnaire analyses.

All usability interviews were recorded on an Olympus WS-853 recorder (Olympus America Inc). All recordings were transcribed and coded using NVivo 12 (QSR International) and stored in the software for qualitative analyses. A general inductive method was used to code transcripts. The transcripts were read repeatedly, and text segments were coded for potential themes. We used thematic content analysis to assess both preusability expectations and postusability experience [27]. We performed an inductive thematic content analysis for the expectations usability interview by generating codes that emerged naturally based on the participant responses to the semistructured interview guide. For the postuse usability interview, we conducted a problem-based deductive thematic content analysis to assess patterns of experience with the app and sensors and potential technical difficulties with the equipment. As the coding framework developed, transcripts were reanalyzed in light of new themes that emerged. Finally, we derived and summarized major themes that were relevant to the usability experience.

This is the primary analysis of data collected in this study. A priori, we planned for exploratory descriptive and qualitative analyses of usability data. No a priori hypothesis testing was planned, and none were conducted. Data were reported as means, standard deviations, medians, and interquartile ranges. Normality assumptions were based on visual inspection of histograms. Descriptive statistics were calculated, and figures were made using Python v3.7.7 (Python Software Foundation) with the following open-source packages: matplotlib 3.1.1, NumPy 1.17.2, pandas 0.20.3, and seaborn 0.9.0.

A total of 18 participants were recruited, attended, and completed the preusability and postusability evaluations. All participants had prior experience using health or wellness apps (such as headache diaries or meditation apps). Patient demographics are summarized in Table 1. Headache diary entry and biofeedback session data were not successfully transmitted to the data storage server for one participant and were thus not available for analyses.

One case of prespecified adverse events was recorded, in which a patient developed a skin rash from the sticky SEMG electrodes. The rash was mildly painful and lasted for a week. No other adverse events were detected. Based on the evaluation questionnaire and qualitative analysis, some of the participants raised a discomfort concern regarding the bulkiness of the SEMG sensor when trying to lay down.

This study explored the usability and feasibility of a biofeedback app treatment for migraine using a mixed methods approach with both quantitative and qualitative data. The preusability assessment indicated that all participants were positively attuned to the biofeedback app and expected to learn more about their bodily signals and how these affect their migraine. On the other hand, the problem-based postusability analysis revealed that the setup had several shortcomings, including technical difficulties and inaccurate measurements. Most participants had some recommendations for specific improvements to the setup but reported an overall positive user experience and good adherence rates.

Three main findings in this study illustrate the usefulness of usability testing when developing mHealth interventions. First, qualitative analysis revealed that participants felt they had difficulties with increasing their skin temperature. This is noteworthy because peripheral skin temperature appears to be one of the most effective biofeedback parameters for migraine [7,10] and is believed to provide an indirect measure of activity in the sympathetic nervous system, partially explaining the treatment effect of biofeedback [9,10,28]. A reduction in arousal and autonomic tone leads to increased peripheral blood flow and skin temperature, while increased arousal increases sympathetic outflow, thereby constricting peripheral blood flow and lowering skin temperature [29]. Migraine patients seem to have an interictal sympathetic impairment and ictal adrenoreceptor hypersensitivity [30], which suggest that regular training in reducing sympathetic tone could lower migraine burden [9]. Therefore, the importance of finger temperature in biofeedback training indicates that the perceived lack of influence over temperature lies in the app/sensor itself and not in the choice of the physiological parameter. An explanation for the mentioned difficulties with raising their finger temperature could be that the biofeedback sessions were completed during the cold winter in Norway. Thereby, the participants’ index finger might have been colder than the average body temperature prior to the biofeedback sessions.

Second, our mixed methods approach revealed that participants required more guidance to understand the association between biofeedback and the app’s illustration of the physiological measurements. The quantitative evaluation questionnaires revealed high scores for functionality and design but low scores for engagement and biofeedback, both in line with qualitative findings. Preexpectation analysis indicated that participants wished for the app to be simple and understandable. However, the postexperience analysis revealed that participants had difficulties perceiving how biofeedback is associated with migraine and better health outcomes. This suggests that upcoming iterations of the treatment should be focused on ensuring a tight correlation between users’ perception of the biofeedback training and true physiological measurements. This is likely to increase motivation and potentially treatment effect further [31].

The third significant usability finding was the desire for relaxation training and stress management techniques to be included in the app. We decided not to include such features to investigate if a therapist might be omitted from the usual treatment and see if the app itself may replace the therapist [25]. However, both qualitative and quantitative findings indicated that more guidance would have improved user experience and adherence. On the other hand, previous studies investigating the use of minimal therapist contact treatments have discovered that they often generate results equivalent to therapist-led treatment [32] and are more cost-effective [33]. Andersson and colleagues [34] investigated the role of therapist-initiated contact in a telephone study where adults suffering from headaches were randomized to either a web-based self-help program or the same program with additional therapist-initiated telephone calls. They found that therapist-initiated phone calls did not influence the dropout rate or improvements in headache index. This indicates that therapist-assisted treatment is not necessarily superior to pure self-management. The immediate idea to counteract this uncertainty in app-based biofeedback treatment is to implement a combination of home training and therapist contact. The problem, however, with such an approach is that it is not much different from traditional treatment and will not ensure the desired cost-benefit and widespread distribution. To keep in line with trends of mHealth development, we propose a program where minimal to no guidance is included in the app for the first few sessions, and then specific tips and techniques are progressively included for participants who have problems with achieving improvement in their biofeedback scores.

As the mHealth approach to pain treatment has grown over the past year, several shortcomings in development processes have been identified. Lalloo and colleagues [16] found that only 8.2% of pain self-management apps included a health care professional in the development and the majority of the apps (58.5%) implemented only a single self-management function. Similarly, Rosser and Eccleston [20] found that 86% reported no involvement of health care professionals in the development process. This could explain some of the general deficiencies they found, such as a lack of psychological and behavioral components underlying many self-management interventions, thereby causing an absence of validated expertise and content underpinning the available pain apps [20]. We took several measures to combat these limitations by including a wide range of health care professionals, including medical doctors and psychologists, in the development process.

Even though there are no similar studies of biofeedback for migraine, development studies of biofeedback for other purposes highlight the importance of a thorough development processes. A study of a sensor-based exercise biofeedback system found that using a systematic combined quantitative and qualitative assessment improved the system [35]. Another study of a mHealth biofeedback device for borderline personality disorder was also significantly improved through user-centered design with usability assessment [36]. Together with this study, both of these studies demonstrate that meticulous usability and feasibility assessments can mitigate the unwanted effects of poor development processes that often hamper mHealth apps [37].

Several limitations should be considered when interpreting the results of this study. First, participants used the equipment over a relatively short period, whereas the International Headache Society recommends 3 months for clinical trials [38]. Second, each group of participants only completed one cycle of usability testing. The study could have benefitted from iterative cycles with several rounds of testing for each individual to identify the effect of improvements in the app directly [39]. This was not possible at the time because there were only 10 available sensors for the study. Finally, the semistructured interview guide and evaluation questionnaire used in the study have not been systematically validated, which decreases confidence in usability findings. Nevertheless, the interview guide was based on recommended guidelines for the development of mHealth apps [18], and the evaluation questionnaire was based on a validated mHealth app rating scale [40].

The study also has several strengths. The greatest strength is the rigorous usability and feasibility development process. As discussed above, we believe this will mitigate the unwanted effects of poor development processes. In addition, the study was designed based on recommended guidelines for the development of mHealth apps and known shortcomings in existing mHealth usability studies. Finally, although the sample size was seemingly small, the authors feel that the study reached a level of data saturation with the high adherence rate and collecting quantitative and qualitative data from all participants. We believe this study has uncovered the majority of essential usability issues.

In this study, we performed usability and feasibility testing of a new biofeedback treatment app targeted at adults with migraine. The treatment underwent a rigorous development process specifically for the target population. Participants were overall satisfied with the treatment, had a high adherence rate, and provided several suggestions worthy of inclusion in future iterations. Our findings highlight the importance of usability testing, revealed shortcomings with the intervention that would otherwise have been difficult to discover, and built a solid foundation for future efficacy trials. Future research should assess the efficacy of the proposed biofeedback treatment app among adults with migraine.