The flow diagram of the search method is depicted in Figure 1. Initial searches produced 2827 unique records following the removal of duplicates. A total of 50 records were identified following screening of the abstracts, and their references were also searched for further relevant studies. Following the search procedure described above, 13 records were identified for inclusion in this review; study characteristics are listed in Table 1. Findings of each study are listed separately (Table 2). The assessments of the risk of bias are described in Multimedia Appendix 1.

A total of 4 papers identified for inclusion assessed the concurrent validity of mobile app–based self-report tools, all compared against the Young Mania Rating Scale (YMRS) and either the Hamilton Depression Rating Scale (HDRS) or the Montgomery-Åsberg Depression Rating Scale (MADRS) [25,27,29,31]. All 4 studies found a statistically significant association between mood ratings collected via self-report using a mobile app and clinical assessment using the HDRS or MADRS. In addition, 3 studies found a statistically significant association between mood ratings collected via self-report using a mobile app and clinical assessment using the YMRS [25,29,31]. The fourth study, however, did not observe a statistically significant relationship [27]. One study also found a statistically significant relationship between self-reported mixed symptoms and clinically rated mixed symptoms, as well as a statistically significant relationship between self-reported irritability and YMRS scores [31]. One study examined mood ratings that were reported using a paper-and-pencil tool as well [27]. They did not find a statistically significant correlation between mood ratings reported using a paper-and-pencil tool and either the MADRS or YMRS [27].

A total of 3 studies examined the ability of self-report scales administered via a mobile app to differentiate between known groups, a form of construct validity [26,35,37]. Of these, 2 studies evaluated the differences in the variability of symptoms (mood, irritability, energy, speed of thoughts, impulsivity, or sleep) between patients with bipolar disorder and healthy controls [26,35]. These studies found statistically significant differences in the variability of symptoms between the 2 groups, with the exception of variability of positive mood [26]. One study also compared the variability of negative mood, positive mood, and irritability between patients with bipolar disorder and patients with borderline personality disorder; this study observed a statistically significant difference between the 2 groups for all 3 variables studied [26]. One study examined the difference in the 14-day mean of participants’ mood and energy, as well as the 14-day range of mood, thoughts, and impulsivity between patients with bipolar disorder and healthy controls [37]. Statistically significant differences were observed between the 2 groups for all 5 of these variables [37].

Additionally, 2 studies examined the convergent validity of self-report symptom assessments administered via a mobile app with instruments assessing related factors: the Functional Assessment Short Test (FAST), the Cohen Perceived Stress Scale (PSS), and the abbreviated World Health Organization Quality of Life scale (WHOQoL-BREF) [31,32]. A statistically significant relationship was observed between self-reported mixed symptoms and PSS scores, but not with WHOQoL-BREF or FAST scores [31]. A statistically significant association was observed for both irritability and mood instability determined using self-report compared with the FAST, PSS, and WHOQoL-BREF [31,32].

Varying levels of adherence to the reporting protocol, ranging from 42% to 95%, were reported among studies in which measures were administered once or twice daily, with all but 1 study that used once-daily administration having adherence rates >70% [25,27,28,30,31,33,35-37]. Two studies reported high dropout rates [26,34]. In 1 study, participants were asked to complete a 6-item assessment 10 times daily, with 59 out of 113 (52.2%) of participants dropping out across all 3 study groups [26]. The other study reported that 70 out of 201 (34.8%) participants dropped out during the first month, which was higher than the percentage of participants dropping out in another study using a similar mobile app [33,34]. Compliance rates were substantially higher for the paper-and-pencil conditions in the 2 studies reported by Depp et al [27,28]. However, the frequency of measure completion was not the same between the 2 groups, and the paper-and-pencil condition could complete the measure at any time, whereas the phone condition was time limited [27,28]. These differences may have contributed substantially to the differences in completion rates between conditions.

The overall results of this review suggest that mobile app–based self-report questionnaires demonstrate concurrent validity when compared with established measures of depression and mania and convergent validity when compared with other related assessment tools. Furthermore, current evidence indicates that mobile app–based self-report questionnaires are able to differentiate between patients with bipolar disorder and patients with borderline personality disorder or healthy controls. In terms of protocol adherence, variability was observed in completion rates, with higher overall adherence rates in participants completing questionnaires daily compared with twice daily. High dropout rates were observed when participants were asked to complete the measure 10 times per day.

In this review, 4 studies analyzed the association between the self-reporting of symptoms via a mobile app and clinical assessment tools. While all 4 studies found a statistically significant association between mood ratings collected via self-report and clinical assessment tools for depression, only 3 out of 4 studies found a statistically significant association between mood ratings collected via self-report and the YMRS. Of note, the study in which no statistically significant correlation was found compared YMRS scores to data collected over the entire study duration and to those collected during the first 6 weeks of the study [27]. As the YMRS assesses symptoms over the preceding 48 hours, the poor correlation may be at least partly attributable to the difference in time periods observed. Only 1 other study reported the period of data used in the comparison, comparing YMRS scores to data collected over the preceding 3 days [29]. This may be a more appropriate comparison, especially as one goal of app-based self-report scales is the detection of acute mood states and changes in symptoms over time.

Furthermore, data collected via the paper-and-pencil condition did not have a statistically significant correlation with either the MADRS or YMRS [27]. This suggests that app-based self-report scales may more accurately collect data on depressive symptoms compared with their paper-based counterparts. While there are few data comparing mobile assessments with rating scales administered via paper and pencil, it has been suggested elsewhere that participants may be more forthcoming when reporting symptoms through mobile assessments [38]. In addition, it has been shown that participants completing measures via paper and pencil may complete the entries retrospectively and hence, outside the specified time frame being assessed [17]. This may explain the seemingly increased accuracy of symptoms reported via app-based measures compared with paper and pencil.

A manic or depressive episode at study onset was an exclusion criterion for many of the studies identified [27-33]. In addition, 3 other studies indicated that patients were euthymic for the duration of the study [25,26,36]. The remaining studies did not state whether any participants experienced acute episodes of mania or depression. As such, it is unclear whether mobile app–based self-report tools can detect acute mood episodes in patients with bipolar disorder.

Some studies assessed the ability of mobile app–based self-report tools to differentiate between known groups [26,35,37]. These studies found statistically significant differences between patients diagnosed with bipolar disorder and healthy controls. While differences in mean mood and mean energy were observed between the 2 groups in 1 study, the magnitude of the difference in range of thoughts and range of impulsivity between the 2 groups was higher [37]. The 2 other studies comparing 2 known groups also observed differences in the variability of symptoms associated with bipolar disorder [26,35]. These findings suggest that the range and course of symptoms measured using mobile app–based self-report tools may allow us to distinguish patients with bipolar disorder from healthy controls.

Studies comparing data collected via self-report assessments administered via a mobile app to the FAST, PSS, and WHOQoL-BREF observed statistically significant associations between some data collected and these measures. As the FAST, PSS, and WHOQoL-BREF assess functional impairment, psychological distress, and quality of life, these findings suggest that data collected via self-report using a mobile app may also reflect other factors of clinical importance [39-41].

Lower rates of adherence to the protocol were observed in most studies in which assessments were administered twice daily compared with studies in which assessments were administered once daily. Furthermore, 1 study in which assessments were administered 10 times per day observed high dropout rates during its 3-month course [26]. These findings suggest that users may have difficulty completing multiple assessments per day but are able to manage assessments occurring once daily. Different proportions of participants dropped out in 2 studies administering similar mobile apps [33,34]. The reason for this is unclear. Previous data indicate that users value apps that are simple and intuitive to use [42]. The study in which higher dropout rates were observed used a version of the app containing numerous additional features, so it is possible that users found the app more complicated and were less willing to continue regular use as a result [34].

In this review, only English studies from peer-reviewed journals were considered. As very few (n=49) non-English papers were identified prior to screening, this was felt to have minimal impact on overall results. As there were large numbers of protocol papers identified, for which it is not possible to exclude unpublished data, it is also possible that publication bias may have resulted in missed negative findings. While 13 papers were identified for inclusion in this review, only 5 different research groups seem to be represented, based on the names and affiliations of authors. One group is represented in 5 studies, which is over one-third of those identified for inclusion [25,29-32]. This may contribute to bias; however, it is reassuring that the reported findings appear to be fairly consistent across the different groups included. As noted above, no study reported on the ability of mobile app–based self-report tools to detect acute mood episodes. As such, it is unclear whether these tools are suitable for this purpose.

Further studies on the validity of mobile app–based assessment tools, especially studies evaluating the ability of these tools to detect acute mood states, will better inform us about the potential utility of these tools in clinical settings. Future research into the course of symptoms measured using these tools may also provide insights into the differences between patients with bipolar disorder and healthy controls. Furthermore, the use of repeated self-report questionnaires combined with physiological and behavioral monitoring, which have been examined elsewhere [43], and with other biomarkers also bears further investigation and may further our understanding of bipolar disorder.

These findings suggest that mobile app–based self-report tools are valid in the assessment of symptoms of mania and depression in euthymic patients with bipolar disorder. These findings also suggest that data on the range and variability of symptoms collected using a mobile app differ between patients with bipolar disorder and healthy controls and are significantly associated with other clinically important measures. It is unclear at this time whether these tools can be used to detect acute episodes of mania or depression in patients with bipolar disorder. Adherence data indicate that patients with bipolar disorder show good adherence to self-report assessments administered daily for the duration of the study periods evaluated.