Depression is the leading cause of global disability [1] and poses a significant public health burden [2-7]. Following the COVID-19 outbreak, approximately one-third of Americans reported clinically significant symptoms of depression [8], highlighting the substantial need for effective treatment. Evidence-based treatments (EBTs) for depression exist; however, access to EBTs remains limited. Less than 35% of individuals with depression receive “minimally adequate” treatment [9-11], and the ratio is even lower for ethnic minority groups [9,10,12,13]. “Stay-at-home” orders during the peak of the COVID-19 pandemic disrupted traditional health care delivery, and our already-overburdened health care system struggled to meet patient needs [14]. Consequently, the pandemic not only increased the prevalence of depression but also reduced treatment accessibility.

Smartphone-based mobile health (mHealth) apps are well suited to address treatment barriers given the relative accessibility of smartphones. In the United States, 85% of adults own a smartphone, and 91% of users report that their smartphone is within arm’s reach 24 hours per day [15]. Consequently, mHealth apps can be accessed at any time, allowing for real-time intervention. In addition, 15% of Americans reported that their only internet-accessible device was their smartphone, and marginalized communities reported higher proportions of smartphone-only internet access [16]. Furthermore, emerging mHealth interventions—particularly self-guided interventions—alleviate the burden on the mental health care system by providing contactless, automated interventions and facilitating self-management.

Unfortunately, many mHealth apps purported to treat depression are not informed by evidence-based care. Specifically, only 10% of publicly available depression apps use evidence-based principles of cognitive behavioral therapy or behavioral activation (BA) [17]. Of those that are empirically informed (eg, MoodTools [18]), few have been empirically evaluated. A systematic review found that only 2% of publicly available mental health apps were empirically supported [19]. Thus, it is unclear to what extent specific mHealth apps are efficacious or effective treatments for depression.

A minority of depression apps have been informed by EBTs and subjected to at least an initial empirical evaluation. For example, Moodivate is a BA self-help app involving psychoeducation, identification of personal values, activity scheduling, and daily mood ratings [20]. A pilot trial found initial evidence of its feasibility and efficacy [21], and a large-scale randomized clinical trial is underway [22]. Nevertheless, too few app-based interventions are subjected to rigorous empirical testing before they reach the market.

In addition, existing evidence-informed mHealth apps, such as Moodivate, target depressive symptoms by delivering an app version of a traditional face-to-face psychotherapy protocol using a varied set of nonoverlapping skills (eg, values clarification, cognitive restructuring, and activity scheduling). These tend to be presented in a nonstepwise fashion, with all psychoeducational material available at the outset in a handout format. This approach is inconsistent with how consumers use apps: in frequent, short bursts [23]. Treatment engagement may suffer if mHealth interventions require lengthy engagement, are overly didactic, or lack a clear sequential structure. Therefore, we argue that it might be most useful for an mHealth app to target a specific psychological process using a limited set of scaffolded skills [23]. Each targeted skill builds on the skills previously developed as the user progresses through the app content. The intervention would include minimal, targeted psychoeducation and narrowly focus on brief, sequential skill-oriented tasks that are “gamified.”

To date, no empirically supported mHealth apps have focused on rumination, despite it being a potentially promising treatment target. According to response styles theory [24], rumination is a pattern of behaviors and cognitions that focus attention on depressive symptoms, including their causes, consequences, and implications [25]. Rumination causes excessive focus on negative emotional states, inhibits mood-enhancing behaviors, and exacerbates and prolongs depression [24,25]. Although various theories of rumination conceptualize its content, antecedents, and functions somewhat differently [26,27], most converge on key features: rumination is a specific, depression-related form of repetitive negative thinking that occurs in response to a triggering event, and it is experienced as distressing and difficult to control [28-30].

Overwhelming evidence demonstrates a strong relationship between rumination and depression. Rumination prospectively predicts higher levels of depressive symptoms over time [31-33], is a trait of vulnerability to depression [34-37], contributes to the maintenance of depressive episodes [24,25], is a precursor to the onset of clinical depression [32,35,37,38], and is a risk factor for relapse and recurrence [39-42]. Experimental studies have causally linked rumination to the maintenance of depression [25,43,44] and have demonstrated its co-occurrence with other forms of psychopathology [45-52]. Rumination functions as an experiential avoidance strategy (ie, avoidance of uncomfortable private experiences [53]) wherein ruminators distract themselves from emotionally arousing material (eg, sadness) through repetitive thinking, which contributes to negative sequelae [54].

Several EBTs explicitly target rumination, including BA, mindfulness-based cognitive therapy, and rumination-focused cognitive behavioral therapy (RFCBT). Although these interventions are well supported by research [55-57], they are neither widely disseminated nor easily accessible. Considering that depression is so pervasive and problematic, rumination is strongly implicated in depression, and effective treatments for rumination exist but are not widely disseminated, rumination is a promising target for mHealth intervention. To address the need for an easily accessible evidence-based intervention targeting rumination, we designed RuminAid—a new mHealth app.

An overwhelming number of mHealth apps claim to treat depression, but a minority are empirically supported [19]. To address this issue directly, we have and will continue to incorporate scientific inquiry into RuminAid from development to dissemination in an iterative, data-driven process. This study aimed to estimate the initial acceptability of RuminAid by evaluating its user-perceived quality and identifying the potential modifications required. We distributed a storyboard presentation of the initial version of RuminAid to potential consumers and collected quantitative and qualitative feedback via individual surveys and focus group interviews. This feedback was used to facilitate improvements in RuminAid before testing its feasibility, acceptability, and effectiveness.

We recruited participants from the University at Buffalo Psychology 101 courses. Individuals were eligible to participate if they were native English speakers and adults (age ≥ 18 years). Participants were oversampled for moderate, moderately severe, or severe depression, but individuals with lower depression scores were also allowed to participate. A total of 22 individuals completed all the required surveys, viewed the complete storyboard presentation, passed the attention check items, and attended a focus group interview. We conducted a total of 7 focus group sessions with groups ranging in size from 1 to 7 participants (mean 3.14, SD 2.04; median 3).

Our sample comprised 77% (17/22) men and 23% (5/22) women. The mean age of the participants was approximately 19 (mean 18.86, SD 0.83) years. The sample was primarily heterosexual (19/22, 86%), and 14% (3/22) of the participants were bisexual. None of the participants endorsed any other sexual orientation. The sample consisted mainly of first- and second-year students: 68% (15/22) were first-year students, 23% (5/22) were second-year students, and 9% (2/22) were third-year students. In terms of marital status, most (20/22, 91%) participants were single and a minority (2/22, 9%) were married or partnered. In terms of religious background, 46% (10/22) identified as Catholic, 9% (2/22) as Protestant, 5% (1/22) as Muslim, and 9% (2/22) as some other religion (Christian nondenominational, n=1; spiritual, n=1), whereas 32% (7/22) of the participants reported that they did not identify with a religion. Participants were allowed to select all ethnic and racial identities with which they identified. The ethnic and racial makeup of the sample was as follows: 50% (11/22) White, 36% (8/22) Black, 18% (4/22) Latin American, and 9% (2/22) Asian American and Pacific Islander.

Regarding depressive symptoms, 41% (9/22) of the participants reported minimal symptoms of depression based on the Patient Health Questionnaire (PHQ; PHQ-8), 41% (9/22) reported mild depression, 9% (2/22) reported moderate depression, and 9% (2/22) reported moderately severe depression. None of the participants reported experiencing severe depression. The participants had an average PHQ-8 score of 6.55 (SD 5.12) and a median score of 5, which indicated that, overall, the participants experienced mild depression.

Participants were screened through mass testing using the University at Buffalo’s Sona Systems to determine whether they met the eligibility criteria. Eligible participants signed up for this study on the University at Buffalo’s Sona Systems and attended a web-based focus group session. The sessions were conducted on a web-based videoconferencing platform (Zoom). Upon enrollment, the participants completed a demographics questionnaire and the PHQ-8 via Qualtrics. Next, participants were provided with a link to view the RuminAid storyboard using Panopto, which is a lecture recording and streaming platform. Participants’ progress in viewing and listening to the presentation was tracked using Panopto, which allowed study staff to view the percentage of the presentation the participants had completed. The participants had to complete 100% of the storyboard to proceed with the study. After viewing the storyboard presentation for 2 hours, 2 minutes, and 53 seconds, participants completed the follow-up Qualtrics survey, which consisted of the uMARS and attention-check questions that the participants needed to answer correctly to participate in the focus groups. The correct answers to the attention-check items were embedded within the audio of the storyboard presentation, with clear instructions to make a note of the information for the attention-check items. This ensured that the attention-check items would be easy to answer correctly for participants who attended to the presentation and quite difficult for those who did not. Next, participants attended a scheduled focus group session and provided feedback in a discussion-based format. Focus group sessions were video- and audio-recorded and then transcribed and coded.

The University at Buffalo Institutional Review Board deemed this project exempt from review; all documents used were reviewed and approved by the institutional review board. Before participating, participants were informed that this research was being conducted to investigate the user-perceived quality of a new mHealth smartphone app called RuminAid and that their feedback would be used to improve the quality of the app. We told participants that their responses would be deidentified and stored on a password-protected server. We did not collect identifying information (eg, name, email, and phone number), but participants could provide their email address to opt in to receive a free download of RuminAid once it reached the marketplace; this was entirely optional. Similarly, participants could opt out of their feedback being anonymously quoted in publications about RuminAid. Participants were told that they had the right to end the study at any time and were compensated with credits that partially fulfilled the course research requirement.

In our sample, engagement (mean 3.58, SD 0.65), functionality (mean 3.86, SD 0.64), information (mean 4.07, SD 0.46), app quality (mean 3.63, SD 0.47), and perceived impact (mean 3.63, SD 0.47) were rated as “good.” In contrast, aesthetics (mean 3.02, SD 0.66) and subjective app quality (mean 2.89, SD 0.64) were rated as “acceptable.”

This study used a mixed methods approach and aimed to estimate the initial acceptability of RuminAid by evaluating its user-perceived quality and identifying potential modifications needed. The results of this study suggested that, overall, RuminAid was perceived as “acceptable” to “good” by the focus group participants. In terms of strengths, participants highlighted the information and perceived impact of RuminAid: focus group members reported that the quality of information was quite good and had the potential to help adults who struggle with depression and rumination. In contrast, focus group members expressed concern that poor aesthetics could interfere with users’ desire to continue using the app. Indeed, focus group members highlighted RuminAid’s aesthetics as its primary weakness.

In particular, participants felt that the visual elements of the app—especially the color scheme and graphics—appeared amateur, outdated, and dull. Specific suggestions for improvement included a brighter color scheme, modern aesthetics, and professionally designed graphics. To address these concerns, we hired a graphic designer to redesign all RuminAid screens. The graphic designer was instructed to revise the app screens to look more professional, modern, and colorful, based on participant recommendations. Comparisons of the RuminAid app screens before and after focus group testing are displayed in Figures 1-3 to demonstrate how the data collected in this study directly informed the modifications.

In addition to redesigning the app screens so that they were more aesthetically pleasing, we modified the RuminAid timeline. Although only a minority of focus group participants commented on the “time gating” involved in RuminAid (ie, restricting users from moving from one quest to the next for a period of 1 week), this feedback highlighted potential frustration and loss of interest that might result from this restriction. This is reflected in the literature, which indicates that people use phone apps sporadically [23,88-90], and in lay commentary, which suggested that people dislike apps and games with time-gated content [91]. In addition, attrition tends to be high in mHealth interventions [92-94]. Thus, to prevent frustration and attrition, we reduced RuminAid from a 5-week intervention restricting users to completing 1 quest per week to a 5-day intervention with instructions (but no time gates) to complete 1 quest per day. Notably, we retained the calendar feature for scheduling quests but modified the instructions to fit the revised timeline.

It is possible that demand characteristics and group factors may have played a role in participants reporting that they liked the app in a focus group setting led by a researcher [95]. During the interviews, positive statements about the subjective app quality were made by all but 1 participant. In contrast, the subjective app quality scores obtained via uMARS suggested less satisfaction. However, the uMARS subjective quality scale includes items such as, “would you pay for this app?” The sample of focus group participants consisted entirely of college students, who may be unlikely to pay for any app. More broadly, monetary commitment may not indicate an individual’s true feelings regarding app quality. For example, some individuals may never feel comfortable paying for a smartphone app, regardless of what the app is. Notably, the app quality domain, based on mean scores of engagement, information, functionality, and aesthetics, was more consistent with the feedback provided during focus group interviews. Nevertheless, it is possible that demand characteristics and group factors may have played a role in participants reporting that they liked the app in a focus group setting led by a researcher. In the future, it might be helpful to have the research assistant running the focus group explicitly state that they were not involved in app development. Furthermore, it may be more effective to collect supplemental qualitative feedback via one-on-one interviews to help reduce feedback biases owing to group factors, such as groupthink or reluctance to dissent [96].

Another potential limitation of our study was the use of a sample of college students. We intentionally selected a sample of college students because this population tends to be young and highly digitally literate. As such, college students often have extensive experience using a variety of high-quality smartphone apps with which they could meaningfully compare RuminAid to provide detailed qualitative feedback informed by the current standards of app quality. However, the use of a college student sample may have potentially limited the amount of critical feedback related to app functionality, particularly flow logic. For example, older adults with less digital literacy may have identified more potential difficulties with app navigation and functionality. As we plan for the next stages of this research, it is important to seek feedback from stakeholders of all ages to identify a broader spectrum of potential modifications before making RuminAid available on the public marketplace.

Finally, participants interacted with a story board rather than a beta version of the app itself, and there were limitations to seeking feedback on an app that participants did not have the opportunity to use. Without having first-hand experience of using RuminAid, the participants may have relied on personal biases, expectations, and perceptions to provide feedback on the app’s usability.

In the next phase of this research, we are most interested in evaluating the core therapeutic content contained within RuminAid as an intervention. As such, we will conduct a pilot trial of RuminAid as an intervention for depression and rumination, which will allow us to examine the app’s feasibility and acceptability and provide initial estimates of its effectiveness among a community sample of adults with depression and rumination. Subsequently, we may conduct a case series study of a beta version of RuminAid. To do so, we may have a community sample of adults with depression install RuminAid and use it for a week before interviewing them individually about their experience. The rich qualitative data collected in such a study would inform any additional modifications made to RuminAid before it is made available to the public.

Once the feasibility, acceptability, and effectiveness of the basic RuminAid approach are established with support for its core therapeutic content, additional features could be incorporated. For example, a substantial percentage of the participants stated that integration of measurement-based care features would be favorable. Measurement-based care has been shown to improve outcomes of treatment for depression [97,98]. Therefore, a longer-term future direction would be to develop measurement-based care features and evaluate whether outcomes are enhanced. Likewise, several participants mentioned wanting a social media component to help them feel more engaged. The incorporation of social media has been shown to improve app engagement [98] and may help to improve long-term app use [99]; therefore, we may also develop and include social features in future iterations of RuminAid.

RuminAid was intended to address the need for accessible EBT for depression by targeting a key risk factor and maintenance factor—rumination. Although the overwhelming majority of mHealth apps are neither empirically tested nor empirically informed, RuminAid incorporates recent research findings and elements of EBTs for rumination and depression. Should research support its feasibility, acceptability, and effectiveness, RuminAid stands to increase access to treatment for rumination, depression, and their negative sequelae. Importantly, this intervention involved no therapist contact, making it inexpensive and timely, given the recent COVID-19 pandemic and the need for remote service delivery options. The self-guided nature of RuminAid may also minimize stigma-related concerns and "better engage individuals who have limited access to or interest in traditional face-to-face interventions.

In summary, the overall quality and treatment approach used in RuminAid was acceptable to potential users. The feedback obtained in this study directly informed the modifications to RuminAid, and the usability, acceptability, and feasibility of this modified version will be tested in a pilot trial. In terms of its potential impact, RuminAid has broad applications. It is a contactless intervention, based on EBTs, and could reduce the burden on the mental health care system by offsetting care to automated service delivery. If effective, RuminAid could be implemented and tested in a variety of settings such as primary care, behavioral health integrated care, stepped care facilities, or remote care clinics or offered to individuals on clinic waitlists. Potential future augmentations to RuminAid could include peer coaching, therapist coaching, or group facilitation. Consequently, pending empirical evaluation, RuminAid may be an accessible and effective intervention for depression and rumination with positive public health consequences, and its development process could serve as a road map for developing evidence-based mHealth apps and empirically supported mHealth apps.