This paper is in the following e-collection/theme issue:

Development and Evaluation of Research Methods, Instruments and Tools (439) Mobile Health (mhealth) (2777) mHealth for Symptom and Disease Monitoring, Chronic Disease Management (1317) mHealth for Wellness, Behavior Change and Prevention (2754) Depression and Mood Disorders; Suicide Prevention (1290) Focus Groups and Qualitative Research for Human Factors Research (781)
Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/40045, first published .
Development of an mHealth App–Based Intervention for Depressive Rumination (RuminAid): Mixed Methods Focus Group Evaluation

Development of an mHealth App–Based Intervention for Depressive Rumination (RuminAid): Mixed Methods Focus Group Evaluation

Development of an mHealth App–Based Intervention for Depressive Rumination (RuminAid): Mixed Methods Focus Group Evaluation

Authors of this article:

Eve A Rosenfeld1, 2 Author Orcid Image ;   Cassondra Lyman3 Author Orcid Image ;   John E Roberts4 Author Orcid Image
Article Authors Cited by Tweetations (29) Metrics

    Original Paper

    1Dissemination and Training Division, National Center for PTSD, VA Palo Alto Healthcare System, Menlo Park, CA, United States

    2Department of Psychiatry and Behavioral Sciences, Stanford University, Stanford, CA, United States

    3Department of Psychology, University of South Florida, Tampa, FL, United States

    4Department of Psychology, University at Buffalo, The State University of New York, Buffalo, NY, United States

    Corresponding Author:

    Eve A Rosenfeld, PhD

    Dissemination and Training Division, National Center for PTSD

    VA Palo Alto Healthcare System

    NCPTSD – 334

    795 Willow Road

    Menlo Park, CA, 94025

    United States

    Phone: 1 908 907 4135

    Email: earosenf@stanford.edu


    Background: Depression is a common mental health condition that poses a significant public health burden. Effective treatments for depression exist; however, access to evidence-based care remains limited. Mobile health (mHealth) apps offer an avenue for improving access. However, few mHealth apps are informed by evidence-based treatments and even fewer are empirically evaluated before dissemination. To address this gap, we developed RuminAid, an mHealth app that uses evidence-based treatment components to reduce depression by targeting a single key depressogenic process—rumination.

    Objective: The primary objective of this study was to collect qualitative and quantitative feedback that could be used to improve the design of RuminAid before the software development phase.

    Methods: We reviewed empirically supported interventions for depression and rumination and used the key aspects of each to create a storyboard version of RuminAid. We distributed an audio-guided presentation of the RuminAid storyboard to 22 individuals for viewing and solicited user feedback on app content, design, and perceived functionality across 7 focus group sessions.

    Results: The consumer-rated quality of the storyboard version of RuminAid was in the acceptable to good range. Indeed, most participants reported that they thought RuminAid would be an engaging, functional, and informational app. Likewise, they endorsed overwhelming positive beliefs about the perceived impact of RuminAid; specifically, 96% (21/22) believed that RuminAid will help depressed ruminators with depression and rumination. Nevertheless, the results highlighted the need for improved app aesthetics (eg, a more appealing color scheme and modern design).

    Conclusions: Focus group members reported that the quality of information was quite good and had the potential to help adults who struggle with depression and rumination but expressed concern that poor aesthetics would interfere with users’ desire to continue using the app. To address these comments, we hired a graphic designer and redesigned each screen to improve visual appeal. We also removed time gating from the app based on participant feedback and findings from related research. These changes helped elevate RuminAid and informed its initial software build for a pilot trial that focused on evaluating its feasibility and acceptability.

    JMIR Form Res 2022;6(12):e40045

    doi:10.2196/40045

    Keywords

    depression; rumination; mobile health; mHealth; evidence-based treatment; focus group; mental health; mobile app; mobile phone 


    Background

    Depression is the leading cause of global disability [1] and poses a significant public health burden [2-7]. Following the COVID-19 outbreak, approximately one-third of Americans reported clinically significant symptoms of depression [8], highlighting the substantial need for effective treatment. Evidence-based treatments (EBTs) for depression exist; however, access to EBTs remains limited. Less than 35% of individuals with depression receive “minimally adequate” treatment [9-11], and the ratio is even lower for ethnic minority groups [9,10,12,13]. “Stay-at-home” orders during the peak of the COVID-19 pandemic disrupted traditional health care delivery, and our already-overburdened health care system struggled to meet patient needs [14]. Consequently, the pandemic not only increased the prevalence of depression but also reduced treatment accessibility.

    Smartphone-based mobile health (mHealth) apps are well suited to address treatment barriers given the relative accessibility of smartphones. In the United States, 85% of adults own a smartphone, and 91% of users report that their smartphone is within arm’s reach 24 hours per day [15]. Consequently, mHealth apps can be accessed at any time, allowing for real-time intervention. In addition, 15% of Americans reported that their only internet-accessible device was their smartphone, and marginalized communities reported higher proportions of smartphone-only internet access [16]. Furthermore, emerging mHealth interventions—particularly self-guided interventions—alleviate the burden on the mental health care system by providing contactless, automated interventions and facilitating self-management.

    Unfortunately, many mHealth apps purported to treat depression are not informed by evidence-based care. Specifically, only 10% of publicly available depression apps use evidence-based principles of cognitive behavioral therapy or behavioral activation (BA) [17]. Of those that are empirically informed (eg, MoodTools [18]), few have been empirically evaluated. A systematic review found that only 2% of publicly available mental health apps were empirically supported [19]. Thus, it is unclear to what extent specific mHealth apps are efficacious or effective treatments for depression.

    A minority of depression apps have been informed by EBTs and subjected to at least an initial empirical evaluation. For example, Moodivate is a BA self-help app involving psychoeducation, identification of personal values, activity scheduling, and daily mood ratings [20]. A pilot trial found initial evidence of its feasibility and efficacy [21], and a large-scale randomized clinical trial is underway [22]. Nevertheless, too few app-based interventions are subjected to rigorous empirical testing before they reach the market.

    In addition, existing evidence-informed mHealth apps, such as Moodivate, target depressive symptoms by delivering an app version of a traditional face-to-face psychotherapy protocol using a varied set of nonoverlapping skills (eg, values clarification, cognitive restructuring, and activity scheduling). These tend to be presented in a nonstepwise fashion, with all psychoeducational material available at the outset in a handout format. This approach is inconsistent with how consumers use apps: in frequent, short bursts [23]. Treatment engagement may suffer if mHealth interventions require lengthy engagement, are overly didactic, or lack a clear sequential structure. Therefore, we argue that it might be most useful for an mHealth app to target a specific psychological process using a limited set of scaffolded skills [23]. Each targeted skill builds on the skills previously developed as the user progresses through the app content. The intervention would include minimal, targeted psychoeducation and narrowly focus on brief, sequential skill-oriented tasks that are “gamified.”

    To date, no empirically supported mHealth apps have focused on rumination, despite it being a potentially promising treatment target. According to response styles theory [24], rumination is a pattern of behaviors and cognitions that focus attention on depressive symptoms, including their causes, consequences, and implications [25]. Rumination causes excessive focus on negative emotional states, inhibits mood-enhancing behaviors, and exacerbates and prolongs depression [24,25]. Although various theories of rumination conceptualize its content, antecedents, and functions somewhat differently [26,27], most converge on key features: rumination is a specific, depression-related form of repetitive negative thinking that occurs in response to a triggering event, and it is experienced as distressing and difficult to control [28-30].

    Overwhelming evidence demonstrates a strong relationship between rumination and depression. Rumination prospectively predicts higher levels of depressive symptoms over time [31-33], is a trait of vulnerability to depression [34-37], contributes to the maintenance of depressive episodes [24,25], is a precursor to the onset of clinical depression [32,35,37,38], and is a risk factor for relapse and recurrence [39-42]. Experimental studies have causally linked rumination to the maintenance of depression [25,43,44] and have demonstrated its co-occurrence with other forms of psychopathology [45-52]. Rumination functions as an experiential avoidance strategy (ie, avoidance of uncomfortable private experiences [53]) wherein ruminators distract themselves from emotionally arousing material (eg, sadness) through repetitive thinking, which contributes to negative sequelae [54].

    Several EBTs explicitly target rumination, including BA, mindfulness-based cognitive therapy, and rumination-focused cognitive behavioral therapy (RFCBT). Although these interventions are well supported by research [55-57], they are neither widely disseminated nor easily accessible. Considering that depression is so pervasive and problematic, rumination is strongly implicated in depression, and effective treatments for rumination exist but are not widely disseminated, rumination is a promising target for mHealth intervention. To address the need for an easily accessible evidence-based intervention targeting rumination, we designed RuminAid—a new mHealth app.

    Treatment Development and Description

    Overview

    RuminAid skills were drawn from well-supported EBTs for depression and rumination (eg, BA, mindfulness-based cognitive therapy, and RFCBT) and mHealth research, which highlighted the importance of balancing EBT components with simplicity and user friendliness by distilling EBTs to key components. We identified five essential components: (1) brief psychoeducation, (2) recognizing ruminative episodes, (3) alternative behaviors to replace rumination, (4) counteracting rumination-related attentional deficits, and (5) gamification.

    RuminAid integrates these elements across 5 brief lessons presented to users as sequential “quests,” with users completing 1 quest per day. Each quest includes brief psychoeducation and gamified elements. Quests 1 to 2 focus on identifying rumination. Quests 3 to 4 teach users to use alternative behaviors to combat rumination. Quest 5 teaches users mindfulness and behavioral skills to counteract the deleterious effects of rumination on attention.

    Brief Psychoeducation

    Brief psychoeducation is integrated into each quest (Multimedia Appendix 1). This material identifies rumination as a treatment target, normalizes rumination as an experience, addresses harmful meta-cognitive beliefs, and orients users to new skills. As overemphasis on didactic content is inconsistent with typical smartphone use [23], psychoeducation is not the primary focus. Instead, psychoeducation is intended to facilitate skills acquisition and is limited to a few minutes per quest, at the most.

    Identifying Rumination

    RuminAid teaches users to discriminate between rumination and nonpathological processes (eg, problem-solving and introspection) and familiarizes users with rumination warning signs (Multimedia Appendix 2). For example, RuminAid users explicitly label and log periods of rumination in real time, tracking associated content, triggers, and internal contexts (eg, thoughts and emotions). The “Two-Minute Rule for Recognizing Rumination” involves engaging in current patterns of thinking for 2 minutes and then answering specific questions to determine if the user is ruminating [58].

    To accommodate the idiographic nature of rumination, users store their ruminative content, triggers, and contexts in a personalized, editable list of rumination “red flags” (ie, signs they are ruminating). RuminAid also includes a list of “common red flags” that users can save on their personalized list. When logging rumination, items from the personalized lists of users are available for tracking via a drop-down menu. Moreover, users can use text entries to enter novel red flags. As users familiarize themselves with their personal red flags, they learn to quickly identify and label periods of rumination, allowing for rapid application of later therapeutic skills.

    Alternative Behaviors

    RuminAid is behaviorally oriented, providing users with specific alternative behaviors to replace unhelpful rumination habits [59,60] and counteract its avoidance function [61]. RuminAid includes 2 forms of alternative behaviors: self-soothing and approach-oriented behaviors (Multimedia Appendix 3). Both forms of alternative behaviors can be conceptualized as BA strategies, wherein self-soothing behaviors aim to increase pleasure and improve mood, whereas approach behaviors aim to increase mastery and promote problem-solving over avoidance.

    Self-soothing behaviors provide distressed individuals with a sense of calmness or pleasure and are promoted in EBTs (eg, BA [57] and Dialectical Behavior Therapy [62,63]). In quest 3, users are instructed to engage in self-soothing whenever they ruminate and are automatically prompted to do so every time rumination is logged. Self-soothing helps disrupt the ruminative cycle and replace the rumination-avoidance association with a rumination-action association. Self-soothing orients users to use “rumination as a call to action,” preparing them for the greater challenge of implementing approach-oriented behaviors. Practicing self-soothing (rather than avoidance) introduces opportunities for positive reinforcement, potentially improving mood. Users are briefed on the distinction between avoidance and self-soothing behaviors and how to discriminate between them (Multimedia Appendix 4). Specifically, RuminAid emphasizes observing the functional consequences of a behavior on one’s mood and thoughts. If a given behavior results in improvements in mood and disrupts the ruminative thought cycle, it can be used again as an effective self-soothing behavior. In contrast, if a given behavior results in emotional numbness and temporary distraction from ruminative thoughts, the behavior is not an effective self-soothing behavior. Users are also taught that the same behavior (eg, watching a comedy show) might function as an effective self-soothing behavior for one person but as an avoidance behavior for another person. Users are encouraged to try new self-soothing behaviors to explore which options work best for them. They are able to edit their list of self-soothing behaviors at any time.

    The second category of alternative behaviors is approach-oriented behaviors. Introduced in quest 4, approach-oriented behaviors are aimed at directly counteracting avoidance. Specifically, RuminAid helps users to create a new, adaptive habitual response to rumination and its triggers, a strategy drawn from EBTs (eg, BA and RFCBT). Users are taught to identify what they are avoiding during rumination by using their rumination topics. They are then instructed to generate and engage in alternative approach-oriented behaviors. These behaviors focus on addressing avoidance head-on and are framed as “facing your fears” (akin to exposure) or “doing the opposite” of the avoidance impulse (akin to opposite action in dialectical behavior therapy). Examples are provided to users to demonstrate these concepts (Multimedia Appendix 5). In addition, users are provided with tools to identify when self-soothing or approach-oriented behavior techniques may be more helpful in a given context.

    Counteracting Rumination-Related Attentionionl Deficits

    Rumination has negative effects on attention and concentration, particularly attention-switching [64-69], that is, the ability to flexibly attend to and adjust behavior in accordance with changes in task goals [70]. Resource allocation theory [71-73] suggests that depression-related thoughts consume cognitive resources, making it difficult for ruminators to attend to task-relevant processes [74]. Because of this, ruminators may not be fully attentive to their experiences while executing alternative behaviors, which could reduce purported positive effects. RuminAid teaches users to use mindfulness (ie, purposeful, present-focused, and nonjudgmental awareness of internal experiences and external environment; Multimedia Appendix 6) to counteract these deficits during quest 5. This approach has effectively reduced rumination in other interventions [75-77]. RuminAid users are taught to choose active rather than passive alternative behaviors and to enhance present-moment awareness using the 5 senses. RuminAid also includes formal mindfulness exercises that can be used as alternative behaviors.

    Gamification

    RuminAid rewards treatment engagement through gamification (Multimedia Appendix 7). Gamification enhances user experience in mHealth apps by integrating gaming elements (eg, completing a quest map) into the intervention [19]. Although gamification research is limited, initial evidence has suggested that gamified elements reduce attrition and increase engagement by creating enjoyable, engaging, and reinforcing experiences [19,78,79]. Pleasurable gaming experiences trigger dopamine and endorphin release [80]. By triggering this response, game-like experiences in digital interventions may reinforce engagement [81]. A meta-analysis found moderate effect sizes for the effectiveness of gamified digital depression interventions [81].

    RuminAid’s quest structure and “map” gamifies psychoeducation and assignments. Progressing in a given quest is rewarded by earning a corresponding star on the map, unlocking new app features and map stages, and prompting celebratory messages of encouragement. These elements should facilitate more active and enjoyable user experiences and reinforce engagement.

    Goal of This Study

    An overwhelming number of mHealth apps claim to treat depression, but a minority are empirically supported [19]. To address this issue directly, we have and will continue to incorporate scientific inquiry into RuminAid from development to dissemination in an iterative, data-driven process. This study aimed to estimate the initial acceptability of RuminAid by evaluating its user-perceived quality and identifying the potential modifications required. We distributed a storyboard presentation of the initial version of RuminAid to potential consumers and collected quantitative and qualitative feedback via individual surveys and focus group interviews. This feedback was used to facilitate improvements in RuminAid before testing its feasibility, acceptability, and effectiveness.


    Participants

    We recruited participants from the University at Buffalo Psychology 101 courses. Individuals were eligible to participate if they were native English speakers and adults (age ≥ 18 years). Participants were oversampled for moderate, moderately severe, or severe depression, but individuals with lower depression scores were also allowed to participate. A total of 22 individuals completed all the required surveys, viewed the complete storyboard presentation, passed the attention check items, and attended a focus group interview. We conducted a total of 7 focus group sessions with groups ranging in size from 1 to 7 participants (mean 3.14, SD 2.04; median 3).

    Our sample comprised 77% (17/22) men and 23% (5/22) women. The mean age of the participants was approximately 19 (mean 18.86, SD 0.83) years. The sample was primarily heterosexual (19/22, 86%), and 14% (3/22) of the participants were bisexual. None of the participants endorsed any other sexual orientation. The sample consisted mainly of first- and second-year students: 68% (15/22) were first-year students, 23% (5/22) were second-year students, and 9% (2/22) were third-year students. In terms of marital status, most (20/22, 91%) participants were single and a minority (2/22, 9%) were married or partnered. In terms of religious background, 46% (10/22) identified as Catholic, 9% (2/22) as Protestant, 5% (1/22) as Muslim, and 9% (2/22) as some other religion (Christian nondenominational, n=1; spiritual, n=1), whereas 32% (7/22) of the participants reported that they did not identify with a religion. Participants were allowed to select all ethnic and racial identities with which they identified. The ethnic and racial makeup of the sample was as follows: 50% (11/22) White, 36% (8/22) Black, 18% (4/22) Latin American, and 9% (2/22) Asian American and Pacific Islander.

    Regarding depressive symptoms, 41% (9/22) of the participants reported minimal symptoms of depression based on the Patient Health Questionnaire (PHQ; PHQ-8), 41% (9/22) reported mild depression, 9% (2/22) reported moderate depression, and 9% (2/22) reported moderately severe depression. None of the participants reported experiencing severe depression. The participants had an average PHQ-8 score of 6.55 (SD 5.12) and a median score of 5, which indicated that, overall, the participants experienced mild depression.

    Measures

    PHQ-8 Measure

    The PHQ-8 [82-84] is an 8-item self-report measure of depressive symptoms and severity. Items are rated on a 4-point scale, with higher scores indicating more severe depression. The PHQ-8 omits the self-harm item from the PHQ-9 and is often used in research settings where interventions for suicidality or self-injury are difficult to coordinate [83]. The PHQ-8 has demonstrated reliability and validity as a measure of depression severity [83,84]. The PHQ-8 was used to oversample for moderate or worse depression (PHQ-8 score ≥10) at screening and readministered at baseline. Scores at readministration were used for all analyses.

    Mobile Application Rating Scale: User Version

    The Mobile Application Rating Scale–user version (uMARS [85]) is a 26-item measure of user-rated app quality consisting of the following scales: (1) engagement (degree of fun, interestingness, customizability, interactivity, whether it has prompts such as sending alerts, reminders, etc), (2) functionality (app functioning, ease of use, navigation, flow logic, and gestural design), (3) aesthetics (graphic design, visual appeal, color scheme, and stylistic consistency), (4) information (contains high-quality information from a credible source), (5) app quality (average mean score of the 4 preceding scales), (6) subjective app quality (overall like or dislike), and (7) perceived impact (whether this app will help the target population with the target problem). Subjective app quality and perceived impact scales can be reported as individual items or mean scores. For our purposes, we used mean scores. To identify qualitative descriptors for each scale (1=inadequate, 2=poor, 3=acceptable, 4=good, and 5=excellent), the average mean scores were rounded to the nearest whole number [86]. The uMARS has demonstrated good test-retest reliability and excellent internal consistency [85].

    Focus Group Interview

    The semistructured focus group interview was designed specifically for this study to collect qualitative feedback through a standardized set of questions. Initial questions asked participants to elaborate on uMARS ratings and identify potential improvements for items rated <4 by at least one focus group member. Additional questions included, “what app features did you find most helpful?” and “based on your experience trying out RuminAid, do you think this app would help you to identify rumination in your day-to-day life?” The interviews lasted for 1 hour and were moderated by the first author through Zoom videoconferencing (Zoom Video Communications).

    Qualitative content analysis [87] was used to systematically identify and describe themes within the participants’ focus group responses. Main categories were generated in a content-driven manner (ie, using theoretical models to derive categories [87]) derived from uMARS scales—engagement, functionality, aesthetics, information, subjective quality, and perceived impact; we added an “other” category for feedback that fell outside of these domains. Subcategories within these domains were generated in a data-driven manner (ie, using the data collected to derive categories [87]). We used subsummation (ie, an iterative process whereby relevant concepts were identified, compared with existing categories, added to existing categories if appropriate, or used to define new categories [87]) to add data-driven subcategories until saturation (ie, the inability to find additional concepts [87]) was achieved. For example, we identified that participants tended to comment on the entertainment value of the app, whether entertainment would impact engagement, and ideas they had about how a social media component could be added to the app to increase engagement. We created 3 corresponding data-driven subcategories within the engagement domain: “Not Entertaining,” “Entertainment not Important,” and “Social Media.”

    Transcriptions of the recorded interviews were coded by 2 trained research assistants for the presence or absence of positive and negative statements on each content domain. “Positive statements” were operationalized as comments that were positive in nature, such as liking elements of the app, suggesting that existing elements be retained, recommendations to further capitalize on well-liked components, or explicit agreement with another participant’s positive statement. “Negative statements” were operationalized as comments that were negative in nature, such as suggesting modifications to a disliked element, recommending the removal of elements, or explicit agreement with another participant’s negative statement. Within a given domain, positive and negative statements were not mutually exclusive, that is, an individual participant could have made both positive and negative statements. Responses were coded according to the presence or absence of statements related to each subcategory.

    Procedures

    Participants were screened through mass testing using the University at Buffalo’s Sona Systems to determine whether they met the eligibility criteria. Eligible participants signed up for this study on the University at Buffalo’s Sona Systems and attended a web-based focus group session. The sessions were conducted on a web-based videoconferencing platform (Zoom). Upon enrollment, the participants completed a demographics questionnaire and the PHQ-8 via Qualtrics. Next, participants were provided with a link to view the RuminAid storyboard using Panopto, which is a lecture recording and streaming platform. Participants’ progress in viewing and listening to the presentation was tracked using Panopto, which allowed study staff to view the percentage of the presentation the participants had completed. The participants had to complete 100% of the storyboard to proceed with the study. After viewing the storyboard presentation for 2 hours, 2 minutes, and 53 seconds, participants completed the follow-up Qualtrics survey, which consisted of the uMARS and attention-check questions that the participants needed to answer correctly to participate in the focus groups. The correct answers to the attention-check items were embedded within the audio of the storyboard presentation, with clear instructions to make a note of the information for the attention-check items. This ensured that the attention-check items would be easy to answer correctly for participants who attended to the presentation and quite difficult for those who did not. Next, participants attended a scheduled focus group session and provided feedback in a discussion-based format. Focus group sessions were video- and audio-recorded and then transcribed and coded.

    Ethical Considerations

    The University at Buffalo Institutional Review Board deemed this project exempt from review; all documents used were reviewed and approved by the institutional review board. Before participating, participants were informed that this research was being conducted to investigate the user-perceived quality of a new mHealth smartphone app called RuminAid and that their feedback would be used to improve the quality of the app. We told participants that their responses would be deidentified and stored on a password-protected server. We did not collect identifying information (eg, name, email, and phone number), but participants could provide their email address to opt in to receive a free download of RuminAid once it reached the marketplace; this was entirely optional. Similarly, participants could opt out of their feedback being anonymously quoted in publications about RuminAid. Participants were told that they had the right to end the study at any time and were compensated with credits that partially fulfilled the course research requirement.


    Quantitative Feedback

    In our sample, engagement (mean 3.58, SD 0.65), functionality (mean 3.86, SD 0.64), information (mean 4.07, SD 0.46), app quality (mean 3.63, SD 0.47), and perceived impact (mean 3.63, SD 0.47) were rated as “good.” In contrast, aesthetics (mean 3.02, SD 0.66) and subjective app quality (mean 2.89, SD 0.64) were rated as “acceptable.”

    Qualitative Feedback

    Engagement

    During focus group sessions, 50% (11/22) of the participants made positive statements about engagement (ie, degree of fun, interestingness, customizability, interactivity, whether it has prompts such as sending alerts, reminders, etc) and 86% (19/22) made negative statements about engagement. This suggested potentially mixed feelings about how engaging RuminAid was for the participants. Within this domain, 64% (14/22) of the participants reported that they did not find the app entertaining. However, 36% (8/22) of the participants felt that the entertainment value of the app was unimportant, given that its purpose was related to mental health, not entertainment. In addition, 18% (4/22) of the participants suggested adding a social media element to RuminAid to improve engagement.

    Functionality

    In terms of app functionality (ie, app functioning, ease of use, navigation, flow logic, and gestural design), 55% (12/22) of the participants made positive statements and 41% (9/22) made negative statements. This suggested that there were mixed feelings about app functionality, although most participants identified positive aspects of app functionality. Specifically, 27% (6/22) of the participants noted that RuminAid might be difficult to navigate. In contrast, 50% (11/22) of the participants reported that the flow logic of the app (ie, how the content progresses from one screen to the next) made sense.

    Aesthetics

    Regarding aesthetics (ie, graphic design, visual appeal, color scheme, and stylistic consistency), 9% (2/22) of the participants made positive statements, whereas 96% (21/22) of the participants made negative statements. This suggested that app aesthetics was a major concern for the participants. Specifically, 91% (20/22) of the participants described the color scheme of RuminAid as boring, “drab,” “depressing,” or dull. In addition, 18% (4/22) of the participants expressed that this issue would interfere with app use and decrease the likelihood of initial or continued app use, although 5% (1/22) of the participants stated that the color scheme would not interfere. In terms of graphics, 45% (10/22) of the participants expressed that these appeared amateur and 59% (13/22) stated that the graphics appeared outdated.

    Information

    Participants also commented on the information included in RuminAid (ie, containing high-quality information from a credible source): 77% (17/22) of the participants made positive statements and 32% (7/22) made negative statements. This indicated that, overall, information appeared to be a relative strength of the app. More specifically, 73% (16/22) of the participants stated that the amount of detail included in the information was appropriate and to their liking. A minority (4/22, 18%) of the participants found that there was excessive detail. None of the participants reported insufficient detail. Furthermore, 59% (13/22) of the participants reported learning something new from RuminAid. In terms of credibility, 18% (4/22) of the participants explicitly reported feeling that information came from a credible source; however, 23% (5/22) of the participants felt that credibility could be improved if the sources were cited within the app.

    Subjective Quality

    The subjective quality of the app (ie, overall like or dislike of the app) was overwhelmingly positive: 96% (21/22) of the participants made positive statements about the subjective quality of RuminAid. None of the participants made negative statements regarding the subjective app quality. This suggested that the participants felt positive about the app as a whole.

    Perceived Impact

    Participants endorsed overwhelmingly positive beliefs about the perceived impact of RuminAid: 96% (21/22) of the participants indicated that they believed RuminAid would help depressed ruminators with their depression and rumination. None of the participants made negative statements regarding the perceived impact of RuminAid. This indicated that the participants saw the app as potentially effective and helpful for depression and rumination.

    Other

    Approximately 18% (4/22) of the participants made positive statements and 36% (8/22) made negative statements that did not fit within the aforementioned domains. Specifically, 46% (10/22) of the participants reported that integration of measurement-based care features would improve the app, and 23% (5/22) of the participants raised concerns that “time gating” (ie, when users are prevented from accessing new app content until a certain amount of time has passed) quests with a 1-week delay between quests (as originally planned) would negatively impact app quality, frustrate users, and decrease engagement.

    Example quotes from participants and the interrater reliability for each domain and subcategory mentioned above are included in Tables 1 and 2. Examples of modifications based on focus group feedback can be seen in Figures 1-3.

    Table 1. Domains of qualitative focus group feedbacka.
    DomainDescriptionQuotes from participantsCohen κb
    Engagement

    		PositiveApp is fun, interesting, customizable, or interactive and has prompts (eg, sends alerts, messages, reminders, or feedback or enables sharing)
    • “I feel like [engagement] is kind of the whole point. And I think that it does it pretty well.”
      • ‎
    • “I really like games. So, going through the levels—I really liked that and how it was kind of an incentive to keep going.”
      • ‎
    		0.91

    		NegativeApp is not fun, interesting, customizable, or interactive or does not have prompts (eg, sends alerts, messages, reminders, or feedback or enables sharing)
    • “I would say it could be a little more interactive, because the steps are just so repetitive and it’s kind of the same thing going through each task.”
      • ‎
    		1
    Functionality

    		PositiveApp functioning well; easy to learn; or good navigation, flow logic, or gestural design
    • “I thought the format where everything is laid out, like, the layout is good.”
      • ‎
    • “My favorite part of the app personally was the quest system put in place.”
      • ‎
    		0.47

    		NegativeApp not functioning well; not easy to learn; or poor navigation, flow logic, and gestural design
    • “[You should] make the flow a little smoother.”
      • ‎
    		0.38
    Aesthetics

    		PositiveGraphic design, overall visual appeal, color scheme, and stylistic consistency are good or appealing
    • “The brain graphic...was pretty good with the pink on blue brain contrast.”
      • ‎
    		0.8

    		NegativeGraphic design, overall visual appeal, color scheme, and stylistic consistency are not good or unappealing
    • “I feel like a complete artistic overhaul of the app needs to be done.”
      • ‎
    		0.38
    Information

    		PositiveApp contains high-quality information (eg, text, feedback, measures, or references) from a credible source
    • “It had a lot of solid information that would be useful to someone struggling with depression.”
      • ‎
    		1

    		NegativeApp does not contain high-quality information (eg, text, feedback, measures, and references) or lacks credible source
    • “Some of the information seems sort of redundant.”
      • ‎
    		0.51
    Subjective quality

    		PositiveOverall positive impression or liked the app
    • “Everything that was presented was presented clearly. So, I was able to retain it better, and actually learn about it...I also like some of the smaller, finer details...certain ways of rewarding you for staying on the app.”
      • ‎
    		1

    		NegativeOverall negative impression or did not like the app
    • No participants made negative comments about subjective quality.
      • ‎
    		1
    Perceived impact

    		PositiveThis app will help the target population (ie, depressed ruminators) with the problem (ie, depression and rumination)
    • “I think it would help because it does have those red flag areas, you make it customizable for you...I think that it would definitely be helpful for people.”
      • ‎
    • “If I were struggling with rumination and depression, it would be eye-opening in a way...If you didn’t realize you were doing that, or how it affected you, you’re going to get a new perspective to see why it’s affecting you and how you can stop ruminating.”
      • ‎
    		1

    		NegativeThis app will not help the target population (ie, depressed ruminators) with the problem (ie, depression and rumination)
    • No participants made negative comments about perceived impact.
      • ‎
    		1
    Other

    		PositiveAdditional themes that emerged but did not fit into the above categories; positive valence to this feedback
    • “When I first started looking at it, I thought it was going to be too short...I like how it’s a process, because at first I almost wrote it off. I felt like it was kind of just like, ‘this is a quick fix’ type thing. But a lot of mental health things aren’t a quick fix. So, I like how it was a process that you go through and you can plan out ahead of time and that’s how you can really make a change with it is by making it a process.”
      • ‎
    		0.31

    		NegativeAdditional themes that emerged but did not fit into the above categories; negative valence to this feedback
    • “I would say maybe like further on down the road, make sure you keep updating it, because if someone uses it frequently, they could very easily go through all quests very quick. And then after that, there’s really no use for it.”
      • ‎
    		0.36

    aDescriptions and examples of categories identified in qualitative focus group feedback.

    bCohen κ represents an estimate of interrater reliability for qualitative items.

    Table 2. Subcategories of qualitative focus group feedbacka.
    SubcategoriesDescriptionQuotes from participantsCohen κb
    Engagement

    		Not entertainingThe app was not entertaining
    • “I saw a lot of stick figure-like images. And if they were just replaced with humans or something doing the same thing, it would have been more entertaining.”
      • ‎
    		0.8

    		Entertainment not importantThe entertainment value is not relevant to this app, is not an important factor, or is not the purpose of the app and would not impact the likelihood of using it
    • “I don’t really feel like it’s intended to be entertaining. It’s meant to help somebody.”
      • ‎
    • “I wouldn’t, at least for me, you know, as somebody that would be downloading it, I don’t consider the entertainment aspect necessarily a priority.”
      • ‎
    		0.56

    		Social mediaAdding a social media component to the app would make it more entertaining or engaging
    • “I would [add] a feature where you can connect with other users and maybe talk about with another actual person, like what’s going on, and they can give you like feedback or something like that.”
      • ‎
    		1
    Functionality

    		Difficult navigationThe app appears difficult to navigate
    • “I felt like...it jumped around a little bit. [I suggest] making the flow of it better.”
      • ‎
    		0.38

    		Logical flowThe app flow makes sense
    • “I liked how it was kind of structured so that you aren’t just having a bunch of information dumped on you, it’s kind of separated into these five larger segments, that you slowly make your way through with demonstrations that you do yourself.”
      • ‎
    		0.54
    Aesthetics

    		Boring color schemeThe color scheme was boring, drab, “depressing,” etc
    • “Other mental health apps that I’ve seen typically have a little bit better contrast, just things to make the other menu items pop out a little bit more.”
      • ‎
    • “The color could use a little updating.”
      • ‎
    		0.32

    		Color scheme interferesThe color scheme would interfere with app use or the likelihood of using the app
    • “Sometimes people’s emotions can be affected by the colors they see and all that. So just brighten it up.”
      • ‎
    		0.54

    		Color scheme does not interfereThe color scheme would not interfere with app use or the likelihood of using the app
    • “When you need the help, you need the help. And I don’t think making a whole bunch of pretty colors and all that is really going to change what the app is doing.”
      • ‎
    		1

    		Amateur graphicsGraphics looked amateur
    • “People will trust it more if it’s looks more professional.”
      • ‎
    • “In terms of just the buttons, specifically, the little arrow icons, took up half the screen, almost. And it just kind of looked like out of place and too big. And proportions like that matter if you’re going to have a clear and concise experience with the app.”
      • ‎
    		0.67

    		Outdated graphicsGraphics looked outdated
    • “It just looked a little outdated, especially the pictures. Like it’s not current.”
      • ‎
    		0.35
    Information

    		Good detailLiked the level of detail; found it appropriate
    • “Yeah, I think I think it does a really good job and gives very detailed information.”
      • ‎
    		0.89

    		Insufficient detailInsufficient detail; too little detail
    • No participants made comments about insufficient detail.
      • ‎
    		1

    		Excessive detailFelt overwhelmed by information; too much detail
    • “It’s just like, a lot of information, I guess, and a lot of things to do. I think, personally, I would get a little confused on the app.”
      • ‎
    		1

    		LearnedLearned new things
    • “And I wasn’t really like educated on the subject before. So that’s something I did like.”
      • ‎
    • “I learned a lot.”
      • ‎
    		0.81

    		Credible informationInformation seemed credible
    • “I don’t have any reason to believe that it’s not a credible source.”
      • ‎
    		0.62

    		Credible source neededIncluding information such as references, expert videos, or other ways to make the credibility explicit would be useful
    • “I feel like there could be some kind of link to information that has credited sources, and not just put those facts up with no way to immediately check the background of it.”
      • ‎
    		0.51
    Other

    		Measurement-based careMeasurement-based care would improve the app (eg, tracking ruminative episodes over time)
    • “I think the tracking is a good idea, especially to see how much you’ve progressed since starting the app.”
      • ‎
    		0.06

    		Time gatingRestricting people’s access to parts of the quest based on time (ie, 1 week before you can move on to the next quest) might be frustrating to users
    • “People do not like time-gated content, especially artificially time gating.”
      • ‎
    		0.64

    aDescriptions and examples of subcategories identified in qualitative focus group feedback.

    bCohen κ represents an estimate of interrater reliability for qualitative items.

    ‎
    Figure 1. Home screen redesign. RuminAid home screen before user-centered redesign (left) and after user-centered redesign (right).
    View this figure
    ‎
    Figure 2. Ask an expert screen redesign. RuminAid’s first set of “Ask an Expert” screens before user-centered redesign (left) and after user-centered redesign (right).
    View this figure
    ‎
    Figure 3. Quest launch screen redesign. Quest 4 launch screen before user-centered redesign (left) and after user-centered redesign (right).
    View this figure

    Principal Findings

    This study used a mixed methods approach and aimed to estimate the initial acceptability of RuminAid by evaluating its user-perceived quality and identifying potential modifications needed. The results of this study suggested that, overall, RuminAid was perceived as “acceptable” to “good” by the focus group participants. In terms of strengths, participants highlighted the information and perceived impact of RuminAid: focus group members reported that the quality of information was quite good and had the potential to help adults who struggle with depression and rumination. In contrast, focus group members expressed concern that poor aesthetics could interfere with users’ desire to continue using the app. Indeed, focus group members highlighted RuminAid’s aesthetics as its primary weakness.

    In particular, participants felt that the visual elements of the app—especially the color scheme and graphics—appeared amateur, outdated, and dull. Specific suggestions for improvement included a brighter color scheme, modern aesthetics, and professionally designed graphics. To address these concerns, we hired a graphic designer to redesign all RuminAid screens. The graphic designer was instructed to revise the app screens to look more professional, modern, and colorful, based on participant recommendations. Comparisons of the RuminAid app screens before and after focus group testing are displayed in Figures 1-3 to demonstrate how the data collected in this study directly informed the modifications.

    In addition to redesigning the app screens so that they were more aesthetically pleasing, we modified the RuminAid timeline. Although only a minority of focus group participants commented on the “time gating” involved in RuminAid (ie, restricting users from moving from one quest to the next for a period of 1 week), this feedback highlighted potential frustration and loss of interest that might result from this restriction. This is reflected in the literature, which indicates that people use phone apps sporadically [23,88-90], and in lay commentary, which suggested that people dislike apps and games with time-gated content [91]. In addition, attrition tends to be high in mHealth interventions [92-94]. Thus, to prevent frustration and attrition, we reduced RuminAid from a 5-week intervention restricting users to completing 1 quest per week to a 5-day intervention with instructions (but no time gates) to complete 1 quest per day. Notably, we retained the calendar feature for scheduling quests but modified the instructions to fit the revised timeline.

    Limitations

    It is possible that demand characteristics and group factors may have played a role in participants reporting that they liked the app in a focus group setting led by a researcher [95]. During the interviews, positive statements about the subjective app quality were made by all but 1 participant. In contrast, the subjective app quality scores obtained via uMARS suggested less satisfaction. However, the uMARS subjective quality scale includes items such as, “would you pay for this app?” The sample of focus group participants consisted entirely of college students, who may be unlikely to pay for any app. More broadly, monetary commitment may not indicate an individual’s true feelings regarding app quality. For example, some individuals may never feel comfortable paying for a smartphone app, regardless of what the app is. Notably, the app quality domain, based on mean scores of engagement, information, functionality, and aesthetics, was more consistent with the feedback provided during focus group interviews. Nevertheless, it is possible that demand characteristics and group factors may have played a role in participants reporting that they liked the app in a focus group setting led by a researcher. In the future, it might be helpful to have the research assistant running the focus group explicitly state that they were not involved in app development. Furthermore, it may be more effective to collect supplemental qualitative feedback via one-on-one interviews to help reduce feedback biases owing to group factors, such as groupthink or reluctance to dissent [96].

    Another potential limitation of our study was the use of a sample of college students. We intentionally selected a sample of college students because this population tends to be young and highly digitally literate. As such, college students often have extensive experience using a variety of high-quality smartphone apps with which they could meaningfully compare RuminAid to provide detailed qualitative feedback informed by the current standards of app quality. However, the use of a college student sample may have potentially limited the amount of critical feedback related to app functionality, particularly flow logic. For example, older adults with less digital literacy may have identified more potential difficulties with app navigation and functionality. As we plan for the next stages of this research, it is important to seek feedback from stakeholders of all ages to identify a broader spectrum of potential modifications before making RuminAid available on the public marketplace.

    Finally, participants interacted with a story board rather than a beta version of the app itself, and there were limitations to seeking feedback on an app that participants did not have the opportunity to use. Without having first-hand experience of using RuminAid, the participants may have relied on personal biases, expectations, and perceptions to provide feedback on the app’s usability.

    Future Directions

    In the next phase of this research, we are most interested in evaluating the core therapeutic content contained within RuminAid as an intervention. As such, we will conduct a pilot trial of RuminAid as an intervention for depression and rumination, which will allow us to examine the app’s feasibility and acceptability and provide initial estimates of its effectiveness among a community sample of adults with depression and rumination. Subsequently, we may conduct a case series study of a beta version of RuminAid. To do so, we may have a community sample of adults with depression install RuminAid and use it for a week before interviewing them individually about their experience. The rich qualitative data collected in such a study would inform any additional modifications made to RuminAid before it is made available to the public.

    Once the feasibility, acceptability, and effectiveness of the basic RuminAid approach are established with support for its core therapeutic content, additional features could be incorporated. For example, a substantial percentage of the participants stated that integration of measurement-based care features would be favorable. Measurement-based care has been shown to improve outcomes of treatment for depression [97,98]. Therefore, a longer-term future direction would be to develop measurement-based care features and evaluate whether outcomes are enhanced. Likewise, several participants mentioned wanting a social media component to help them feel more engaged. The incorporation of social media has been shown to improve app engagement [98] and may help to improve long-term app use [99]; therefore, we may also develop and include social features in future iterations of RuminAid.

    Conclusions

    RuminAid was intended to address the need for accessible EBT for depression by targeting a key risk factor and maintenance factor—rumination. Although the overwhelming majority of mHealth apps are neither empirically tested nor empirically informed, RuminAid incorporates recent research findings and elements of EBTs for rumination and depression. Should research support its feasibility, acceptability, and effectiveness, RuminAid stands to increase access to treatment for rumination, depression, and their negative sequelae. Importantly, this intervention involved no therapist contact, making it inexpensive and timely, given the recent COVID-19 pandemic and the need for remote service delivery options. The self-guided nature of RuminAid may also minimize stigma-related concerns and "better engage individuals who have limited access to or interest in traditional face-to-face interventions.

    In summary, the overall quality and treatment approach used in RuminAid was acceptable to potential users. The feedback obtained in this study directly informed the modifications to RuminAid, and the usability, acceptability, and feasibility of this modified version will be tested in a pilot trial. In terms of its potential impact, RuminAid has broad applications. It is a contactless intervention, based on EBTs, and could reduce the burden on the mental health care system by offsetting care to automated service delivery. If effective, RuminAid could be implemented and tested in a variety of settings such as primary care, behavioral health integrated care, stepped care facilities, or remote care clinics or offered to individuals on clinic waitlists. Potential future augmentations to RuminAid could include peer coaching, therapist coaching, or group facilitation. Consequently, pending empirical evaluation, RuminAid may be an accessible and effective intervention for depression and rumination with positive public health consequences, and its development process could serve as a road map for developing evidence-based mHealth apps and empirically supported mHealth apps.

    Acknowledgments

    The authors would like to thank Steven Redler, Brendon Jones, and Daniel Lesniak for their contributions to the development of RuminAid, without whom this project would not have been possible. In addition, the authors thank Ian Lechevet for his dedicated contributions as a research assistant on this project.

    Data Availability

    All quantitative data sets generated and analyzed during this study are available at Open Science Framework [100]. Full access to the qualitative data collected during focus group feedback sessions is available from the corresponding author upon reasonable request.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Psychoeducational content screens. These screens from quests 1, 2, and 5 display psychoeducational content to users to enhance their understanding before skills application. These images are screenshots of RuminAid after the user-centered redesign, which was informed by the results of this study.

    PNG File , 272 KB
    ‎
    Multimedia Appendix 2

    Identification and labeling of rumination screens. These screens display the in-app use of skills related to identifying and labeling rumination. These images are screenshots of RuminAid after the user-centered redesign, which was informed by the results of this study.

    PNG File , 239 KB
    ‎
    Multimedia Appendix 3

    Alternative behavior screens. These screens from quests 3 and 4 display psychoeducation about alternative behaviors as well as in-app skills use. These images are screenshots of RuminAid after the user-centered redesign, which was informed by the results of this study.

    PNG File , 236 KB
    ‎
    Multimedia Appendix 4

    Self-soothing behavior screens. These screens from quest 3 demonstrate how users learn to discriminate between self-soothing and avoidance behaviors. These images are screenshots of RuminAid after the user-centered redesign, which was informed by the results of this study.

    PNG File , 319 KB
    ‎
    Multimedia Appendix 5

    Approach behavior screens. These screens from quest 4 display examples of identification of avoidance (left) and generating approach behaviors (right). These images are screenshots of RuminAid after the user-centered redesign, which was informed by the results of this study.

    PNG File , 148 KB
    ‎
    Multimedia Appendix 6

    Mindfulness screens. These screens from quest 5 display psychoeducation, in-app skills use, and tips for mindfulness practice. These images are screenshots of RuminAid after the user-centered redesign, which was informed by the results of this study.

    PNG File , 245 KB
    ‎
    Multimedia Appendix 7

    Gamified elements screens. These screens display gamified elements of RuminAid, including the quest 1 map in progress and at completion. These images are screenshots of RuminAid after the user-centered redesign, which was informed by the results of this study.

    PNG File , 219 KB
    1. Depression. World Health Organization. 2021 Sep 13.   URL: https://www.who.int/news-room/fact-sheets/detail/depression [accessed 2022-09-16]
    2. Kessler RC, Berglund P, Demler O, Jin R, Koretz D, Merikangas KR, National Comorbidity Survey Replication. The epidemiology of major depressive disorder: results from the National Comorbidity Survey Replication (NCS-R). JAMA 2003 Jun 18;289(23):3095-3105. [CrossRef] [Medline]
    3. McLaughlin KA. The public health impact of major depression: a call for interdisciplinary prevention efforts. Prev Sci 2011 Dec 6;12(4):361-371 [FREE Full text] [CrossRef] [Medline]
    4. McKenna MT, Michaud CM, Murray CJ, Marks JS. Assessing the burden of disease in the United States using disability-adjusted life years. Am J Prev Med 2005 Jun;28(5):415-423. [CrossRef] [Medline]
    5. Walker ER, McGee RE, Druss BG. Mortality in mental disorders and global disease burden implications: a systematic review and meta-analysis. JAMA Psychiatry 2015 Apr;72(4):334-341 [FREE Full text] [CrossRef] [Medline]
    6. Angst J, Angst F, Stassen HH. Suicide risk in patients with major depressive disorder. J Clin Psychiatry 1999;60 Suppl 2:57-62; discussion 75. [Medline]
    7. Lépine JP, Briley M. The increasing burden of depression. Neuropsychiatr Dis Treat 2011;7(Suppl 1):3-7 [FREE Full text] [CrossRef] [Medline]
    8. Anxiety and Depression. Centers for Disease Control and Prevention.   URL: https://www.cdc.gov/nchs/covid19/pulse/mental-health.htm [accessed 2020-10-28]
    9. González HM, Vega WA, Williams DR, Tarraf W, West BT, Neighbors HW. Depression care in the United States: too little for too few. Arch Gen Psychiatry 2010 Jan 01;67(1):37-46 [FREE Full text] [CrossRef] [Medline]
    10. Hahm HC, Cook BL, Ault-Brutus A, Alegría M. Intersection of race-ethnicity and gender in depression care: screening, access, and minimally adequate treatment. Psychiatr Serv 2015 Mar 01;66(3):258-264 [FREE Full text] [CrossRef] [Medline]
    11. Thornicroft G, Chatterji S, Evans-Lacko S, Gruber M, Sampson N, Aguilar-Gaxiola S, et al. Undertreatment of people with major depressive disorder in 21 countries. Br J Psychiatry 2017 Feb;210(2):119-124 [FREE Full text] [CrossRef] [Medline]
    12. Alegría M, Chatterji P, Wells K, Cao Z, Chen C, Takeuchi D, et al. Disparity in depression treatment among racial and ethnic minority populations in the United States. Psychiatr Serv 2008 Nov;59(11):1264-1272 [FREE Full text] [CrossRef] [Medline]
    13. Bailey R, Patel M, Barker N, Ali S, Jabeen S. Major depressive disorder in the African American population. J National Med Assoc 2011 Jul;103(7):548-559 [FREE Full text] [CrossRef]
    14. Bojdani E, Rajagopalan A, Chen A, Gearin P, Olcott W, Shankar V, et al. COVID-19 pandemic: impact on psychiatric care in the United States. Psychiatry Res 2020 Jul;289:113069 [FREE Full text] [CrossRef] [Medline]
    15. Conner C. Fifty essential mobile marketing facts. Forbes. 2013 Nov 12.   URL: https://www.forbes.com/sites/cherylsnappconner/2013/11/12/fifty-essential-mobile-marketing-facts/ [accessed 2021-01-03]
    16. Mobile Fact Sheet. Pew Research Center. 2021 Apr 7.   URL: https://www.pewresearch.org/internet/fact-sheet/mobile/ [accessed 2022-04-28]
    17. Huguet A, Rao S, McGrath PJ, Wozney L, Wheaton M, Conrod J, et al. A systematic review of cognitive behavioral therapy and behavioral activation apps for depression. PLoS One 2016;11(5):e0154248 [FREE Full text] [CrossRef] [Medline]
    18. MoodTools Review. One Mind PsyberGuide.   URL: https://onemindpsyberguide.org/apps/moodtools/ [accessed 2022-04-28]
    19. Fleming TM, Bavin L, Stasiak K, Hermansson-Webb E, Merry SN, Cheek C, et al. Serious games and gamification for mental health: current status and promising directions. Front Psychiatry 2016;7:215 [FREE Full text] [CrossRef] [Medline]
    20. Dahne J, Lejuez CW, Kustanowitz J, Felton JW, Diaz VA, Player MS, et al. Moodivate: a self-help behavioral activation mobile app for utilization in primary care-Development and clinical considerations. Int J Psychiatry Med 2017 Mar;52(2):160-175 [FREE Full text] [CrossRef] [Medline]
    21. Dahne J, Lejuez C, Diaz VA, Player MS, Kustanowitz J, Felton JW, et al. Pilot randomized trial of a self-help behavioral activation mobile app for utilization in primary care. Behav Ther 2019 Jul;50(4):817-827 [FREE Full text] [CrossRef] [Medline]
    22. Dahne J. Evaluation of a mobile therapy for elevated depressive symptoms. ClinicalTrials.gov.   URL: https://clinicaltrials.gov/ct2/show/NCT04463914 [accessed 2022-04-27]
    23. Mohr DC, Tomasino KN, Lattie EG, Palac HL, Kwasny MJ, Weingardt K, et al. Intellicare: an eclectic, skills-based app suite for the treatment of depression and anxiety. J Med Internet Res 2017 Jan 05;19(1):e10 [FREE Full text] [CrossRef] [Medline]
    24. Nolen-Hoeksema S, Wisco BE, Lyubomirsky S. Rethinking rumination. Perspect Psychol Sci 2008 Sep;3(5):400-424. [CrossRef] [Medline]
    25. Nolen-Hoeksema S. Responses to depression and their effects on the duration of depressive episodes. J Abnormal Psychol 1991 Nov;100(4):569-582. [CrossRef]
    26. Smith JM, Alloy LB. A roadmap to rumination: a review of the definition, assessment, and conceptualization of this multifaceted construct. Clin Psychol Rev 2009 Mar;29(2):116-128 [FREE Full text] [CrossRef] [Medline]
    27. Smith ER, Henry S. An in-group becomes part of the self: response time evidence. Pers Soc Psychol Bull 2016 Jul 02;22(6):635-642. [CrossRef]
    28. Martin L, Tesser A. Some ruminative thoughts. In: Ruminative Thoughts Advances in Social Cognition, Volume IX. New York: Psychology Press; 1996.
    29. Morrison R, O'Connor RC. A systematic review of the relationship between rumination and suicidality. Suicide Life Threat Behav 2008 Oct;38(5):523-538. [CrossRef] [Medline]
    30. Watkins E. Rumination-Focused Cognitive-Behavioral Therapy for Depression. New York: Guilford Publications; 2018.
    31. Just N, Alloy LB. The response styles theory of depression: tests and an extension of the theory. J Abnormal Psychol 1997 May;106(2):221-229. [CrossRef]
    32. Nolan S, Roberts J, Gotlib I. Neuroticism and ruminative response style as predictors of change in depressive symptomatology. Cognit Ther Res 1998;22(5):445-455. [CrossRef]
    33. Treynor W, Gonzalez R, Nolen-Hoeksema S. Rumination reconsidered: a psychometric analysis. Cognit Ther Res 2003;27:247-259. [CrossRef]
    34. Nolen-Hoeksema S. The response styles theory. In: Depressive Rumination: Nature, Theory and Treatment. Hoboken, New Jersey: Wiley; 2003.
    35. Roberts J, Gilboa E, Gotlib I. Ruminative response style and vulnerability to episodes of dysphoria: gender, neuroticism, and episode duration. Cognit Ther Res 1998;22:401-423. [CrossRef]
    36. Spasojević J, Alloy L, Abramson L, Maccoon D, Robinson M. Reactive rumination: outcomes, mechanisms, and developmental antecedents. In: Depressive Rumination: Nature, Theory and Treatment. Hoboken, New Jersey, United States: Wiley; 2003.
    37. Martell C, Addis M, Jacobson N. Depression in Context: Strategies for Guided Action. US: W.W. Norton; 2001.
    38. De Raedt R, Koster EH. Understanding vulnerability for depression from a cognitive neuroscience perspective: a reappraisal of attentional factors and a new conceptual framework. Cogn Affect Behav Neurosci 2010 Mar;10(1):50-70. [CrossRef] [Medline]
    39. Buckman J, Underwood A, Clarke K, Saunders R, Hollon S, Fearon P, et al. Risk factors for relapse and recurrence of depression in adults and how they operate: a four-phase systematic review and meta-synthesis. Clin Psychol Rev 2018 Aug;64:13-38 [FREE Full text] [CrossRef] [Medline]
    40. Hick SF, Chan L. Mindfulness-based cognitive therapy for depression: effectiveness and limitations. Social Work Mental Health 2010 Apr 13;8(3):225-237. [CrossRef]
    41. Michalak J, Hölz A, Teismann T. Rumination as a predictor of relapse in mindfulness-based cognitive therapy for depression. Psychol Psychother 2011 Jun;84(2):230-236. [CrossRef] [Medline]
    42. Spinhoven P, Drost J, de Rooij M, van Hemert AM, Penninx BW. Is experiential avoidance a mediating, moderating, independent, overlapping, or proxy risk factor in the onset, relapse and maintenance of depressive disorders? Cognit Ther Res 2016 Dec 30;40(2):150-163 [FREE Full text] [CrossRef] [Medline]
    43. Watkins ER. Constructive and unconstructive repetitive thought. Psychol Bull 2008 Mar;134(2):163-206 [FREE Full text] [CrossRef] [Medline]
    44. Watkins ER. Depressive rumination: investigating mechanisms to improve cognitive behavioural treatments. Cogn Behav Ther 2009 Jan;38 Suppl 1(sup1):8-14 [FREE Full text] [CrossRef] [Medline]
    45. Joormann J, Dkane M, Gotlib I. Adaptive and maladaptive components of rumination? Diagnostic specificity and relation to depressive biases. Behav Ther 2006 Sep;37(3):269-280 [FREE Full text] [CrossRef] [Medline]
    46. Watkins ER. Depressive rumination and co-morbidity: evidence for brooding as a transdiagnostic process. J Ration Emot Cogn Behav Ther 2009 Sep 7;27(3):160-175 [FREE Full text] [CrossRef] [Medline]
    47. Wong Q, Moulds M. Impact of rumination versus distraction on anxiety and maladaptive self-beliefs in socially anxious individuals. Behav Res Ther 2009 Oct;47(10):861-867 [FREE Full text] [CrossRef] [Medline]
    48. Jenness JL, Jager-Hyman S, Heleniak C, Beck AT, Sheridan MA, McLaughlin KA. Catastrophizing, rumination, and reappraisal prospectively predict adolescent PTSD symptom onset following a terrorist attack. Depress Anxiety 2016 Nov 24;33(11):1039-1047 [FREE Full text] [CrossRef] [Medline]
    49. Naumann E, Tuschen-Caffier B, Voderholzer U, Caffier D, Svaldi J. Rumination but not distraction increases eating-related symptoms in anorexia and bulimia nervosa. J Abnorm Psychol 2015 May;124(2):412-420. [CrossRef] [Medline]
    50. Startup H, Lavender A, Oldershaw A, Stott R, Tchanturia K, Treasure J, et al. Worry and rumination in anorexia nervosa. Behav Cogn Psychother 2012 Oct 29;41(3):301-316. [CrossRef]
    51. Nolen-Hoeksema S, Harrell ZA. Rumination, depression, and alcohol use: tests of gender differences. J Cogn Psychother 2002 Oct 01;16(4):391-403. [CrossRef]
    52. Selby EA, Joiner TE. Cascades of emotion: the emergence of borderline personality disorder from emotional and behavioral dysregulation. Rev Gen Psychol 2009 Sep 01;13(3):219-229 [FREE Full text] [CrossRef] [Medline]
    53. Hayes SC, Wilson KG, Gifford EV, Follette VM, Strosahl K. Experimental avoidance and behavioral disorders: a functional dimensional approach to diagnosis and treatment. J Consult Clin Psychol 1996 Dec;64(6):1152-1168. [CrossRef] [Medline]
    54. Giorgio JM, Sanflippo J, Kleiman E, Reilly D, Bender RE, Wagner CA, et al. An experiential avoidance conceptualization of depressive rumination: three tests of the model. Behav Res Ther 2010 Oct;48(10):1021-1031 [FREE Full text] [CrossRef] [Medline]
    55. Cook L, Watkins E. Guided, internet-based, rumination-focused cognitive behavioural therapy (i-RFCBT) versus a no-intervention control to prevent depression in high-ruminating young adults, along with an adjunct assessment of the feasibility of unguided i-RFCBT, in the REducing Stress and Preventing Depression trial (RESPOND): study protocol for a phase III randomised controlled trial. Trials 2016 Jan 4;17(1). [CrossRef]
    56. Hvenegaard M, Watkins ER, Poulsen S, Rosenberg NK, Gondan M, Grafton B, et al. Rumination-focused cognitive behaviour therapy vs. cognitive behaviour therapy for depression: study protocol for a randomised controlled superiority trial. Trials 2015 Aug 11;16(1):344 [FREE Full text] [CrossRef] [Medline]
    57. Perestelo-Perez L, Barraca J, Peñate W, Rivero-Santana A, Alvarez-Perez Y. Mindfulness-based interventions for the treatment of depressive rumination: systematic review and meta-analysis. Int J Clin Health Psychol 2017;17(3):282-295 [FREE Full text] [CrossRef] [Medline]
    58. Addis M, Martell C. Overcoming Depression One Step at a Time: The New Behavioral Activation Approach to Getting Your Life Back. Oakland, California, United States: New Harbinger Publications; 2004.
    59. Hertel P, Mor N, Ferrari C, Hunt O, Agrawal N. Looking on the dark side: rumination and cognitive-bias modification. Clin Psychol Sci 2014 Apr 16;2(6):714-726. [CrossRef]
    60. Watkins ER, Nolen-Hoeksema S. A habit-goal framework of depressive rumination. J Abnorm Psychol 2014 Feb;123(1):24-34. [CrossRef] [Medline]
    61. Thomas J, Raynor M, Ribott D. Depressive rumination and experiential avoidance: a task based exploration. Personal Ment Health 2015 Feb;9(1):58-65 [FREE Full text] [CrossRef] [Medline]
    62. Linehan M. DBT Skills Training Manual, Second Edition. New York, United States: Guilford Publications; 2014.
    63. McKay M, Wood J, Brantley J. The Dialectical Behavior Therapy Skills Workbook: Practical DBT Exercises for Learning Mindfulness, Interpersonal Effectiveness, Emotion Regulation, and Distress Tolerance. Oakland, California, United States: New Harbinger Publications; 2019.
    64. Chuen Yee Lo B, Lau S, Cheung S, Allen NB. The impact of rumination on internal attention switching. Cogn Emot 2012 Feb;26(2):209-223. [CrossRef] [Medline]
    65. Davis R, Nolen-Hoeksema S. Cognitive inflexibility among ruminators and nonruminators. Cognit Ther Res 2000;24(6):699-711.
    66. Koster EH, De Lissnyder E, De Raedt R. Rumination is characterized by valence-specific impairments in switching of attention. Acta Psychol (Amst) 2013 Nov;144(3):563-570. [CrossRef] [Medline]
    67. Philippot P, Brutoux F. Induced rumination dampens executive processes in dysphoric young adults. J Behav Ther Exp Psychiatry 2008 Sep;39(3):219-227 [FREE Full text] [CrossRef] [Medline]
    68. Whitmer AJ, Gotlib IH. Switching and backward inhibition in major depressive disorder: the role of rumination. J Abnorm Psychol 2012 Aug;121(3):570-578. [CrossRef] [Medline]
    69. Wilkinson PO, Goodyer IM. Attention difficulties and mood-related ruminative response style in adolescents with unipolar depression. J Child Psychol Psychiatry 2006 Dec 07;47(12):1284-1291. [CrossRef] [Medline]
    70. Hanania R, Smith L. Selective attention and attention switching: towards a unified developmental approach. Dev Sci 2010 Jul;13(4):622-635 [FREE Full text] [CrossRef] [Medline]
    71. Ellis H, Ashbrook P. Resource allocation model of the effects of depressed mood states on memory. BibSonomy. 1988.   URL: https://www.bibsonomy.org/publication/22058cc28abec9a4c57dcc07a46e391c/brian.mingus [accessed 2022-11-29]
    72. Ellis H, Ashbrook P. The "state" of mood and memory research: a selective review. J Social Behav Personal 1989;4(2):1-21.
    73. Ellis HC, Thomas RL, McFarland AD, Lane JW. Emotional mood states and retrieval in episodic memory. J Experiment Psychol Learn Memory Cognit 1985;11(2):363-370. [CrossRef]
    74. Rosenfeld EA, Kennedy K, Farchione TJ, Roberts JE. Cleansing the attentional palate: a preliminary test of a novel approach to facilitate disengagement from rumination. J Cogn Psychother 2019 May 01;33(2):128-139. [CrossRef] [Medline]
    75. Deyo M, Wilson K, Ong J, Koopman C. Mindfulness and rumination: does mindfulness training lead to reductions in the ruminative thinking associated with depression? Explore (NY) 2009;5(5):265-271 [FREE Full text] [CrossRef] [Medline]
    76. Hawley LL, Schwartz D, Bieling PJ, Irving J, Corcoran K, Farb NA, et al. Mindfulness practice, rumination and clinical outcome in mindfulness-based treatment. Cogn Ther Res 2013 Oct 8;38(1):1-9. [CrossRef]
    77. Sipe WE, Eisendrath SJ. Mindfulness-based cognitive therapy: theory and practice. Can J Psychiatry 2012 Feb;57(2):63-69. [CrossRef] [Medline]
    78. Fleming T, Dixon R, Frampton C, Merry S. A pragmatic randomized controlled trial of computerized CBT (SPARX) for symptoms of depression among adolescents excluded from mainstream education. Behav Cogn Psychother 2012 Oct;40(5):529-541. [CrossRef] [Medline]
    79. Batterham PJ, Neil AL, Bennett K, Griffiths KM, Christensen H. Predictors of adherence among community users of a cognitive behavior therapy website. Patient Prefer Adherence 2008 Feb 02;2:97-105 [FREE Full text] [Medline]
    80. Koepp MJ, Gunn RN, Lawrence AD, Cunningham VJ, Dagher A, Jones T, et al. Evidence for striatal dopamine release during a video game. Nature 1998 May 21;393(6682):266-268. [CrossRef] [Medline]
    81. Li J, Theng Y, Foo S. Game-based digital interventions for depression therapy: a systematic review and meta-analysis. Cyberpsychol Behav Soc Netw 2014 Aug;17(8):519-527 [FREE Full text] [CrossRef] [Medline]
    82. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001 Sep;16(9):606-613 [FREE Full text] [CrossRef] [Medline]
    83. Kroenke K, Strine TW, Spitzer RL, Williams JB, Berry JT, Mokdad AH. The PHQ-8 as a measure of current depression in the general population. J Affect Disord 2009 Apr;114(1-3):163-173. [CrossRef] [Medline]
    84. Kroenke K, Spitzer R. The PHQ-9: a new depression diagnostic and severity measure. Psychiatric Annals 2002 Sep;32(9):509-515. [CrossRef]
    85. Stoyanov SR, Hides L, Kavanagh DJ, Wilson H. Development and validation of the User Version of the Mobile Application Rating Scale (uMARS). JMIR Mhealth Uhealth 2016 Jun 10;4(2):e72 [FREE Full text] [CrossRef] [Medline]
    86. LeBeau K, Huey LG, Hart M. Assessing the quality of mobile apps used by occupational therapists: evaluation using the user version of the mobile application rating scale. JMIR Mhealth Uhealth 2019 May 01;7(5):e13019 [FREE Full text] [CrossRef] [Medline]
    87. Schreier M. The SAGE Handbook of Qualitative Data Analysis. Thousand Oaks, California, United States: SAGE Publications; 2014.
    88. Oulasvirta A, Tamminen S, Roto V, Kuorelahti J. Interaction in 4-second bursts: the fragmented nature of attentional resources in mobile HCI. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2005 Presented at: CHI05: CHI 2005 Conference on Human Factors in Computing Systems; Apr 2 - 7, 2005; Portland Oregon USA. [CrossRef]
    89. Smith AC, Fowler LA, Graham AK, Jaworski BK, Firebaugh M, Monterubio GE, et al. Digital overload among college students: implications for mental health app use. Social Sci 2021 Jul 21;10(8):279. [CrossRef]
    90. Baumel A, Muench F, Edan S, Kane JM. Objective user engagement with mental health apps: systematic search and panel-based usage analysis. J Med Internet Res 2019 Sep 25;21(9):e14567 [FREE Full text] [CrossRef] [Medline]
    91. Walker N. Time Gating in Video Games. YouTube. 2018.   URL: https://www.youtube.com/watch?v=b1phy6TpgXk [accessed 2022-11-15]
    92. Meyer T, Miller M, Metzger R, Borkovec T. Development and validation of the penn state worry questionnaire. Behav Res Ther 1990;28(6):487-495 [FREE Full text] [CrossRef]
    93. Grand View Research. MarketResearch.com.   URL: https://www.marketresearch.com/Grand-View-Research-v4060/ [accessed 2022-09-28]
    94. Mobile app retention rate: what’s a good retention rate? Upland.   URL: https://uplandsoftware.com/localytics/resources/blog/mobile-apps-whats-a-good-retention-rate/ [accessed 2022-09-28]
    95. Wooffitt R. Communication and laboratory performance in parapsychology experiments: demand characteristics and the social organization of interaction. Br J Soc Psychol 2007 Sep;46(Pt 3):477-498. [CrossRef] [Medline]
    96. Brew K, Clark T, Feingold-Link J, Barth H. Do demand characteristics contribute to minimal ingroup preferences? J Exp Child Psychol 2021 Apr;204:105043. [CrossRef] [Medline]
    97. Guo T, Xiang Y, Xiao L, Hu C, Chiu HF, Ungvari GS, et al. Measurement-based care versus standard care for major depression: a randomized controlled trial with blind raters. Am J Psychiatry 2015 Oct;172(10):1004-1013. [CrossRef] [Medline]
    98. Alqahtani F, Winn A, Orji R. Co-designing a mobile app to improve mental health and well-being: focus group study. JMIR Form Res 2021 Feb 26;5(2):e18172 [FREE Full text] [CrossRef] [Medline]
    99. Kaveladze BT, Wasil AR, Bunyi JB, Ramirez V, Schueller SM. User experience, engagement, and popularity in mental health apps: secondary analysis of app analytics and expert app reviews. JMIR Hum Factors 2022 Jan 31;9(1):e30766 [FREE Full text] [CrossRef] [Medline]
    100. Lyman C, Rosenfeld E, Roberts JE. RuminAid: A mHealth App for Depressive Rumination. Open Science Framework. 2022 Sep 18.   URL: https://osf.io/wjct4/ [accessed 2022-12-02]

    ‎
    BA: behavioral activation
    EBT: evidence-based treatment
    mHealth: mobile health
    PHQ: Patient Health Questionnaire
    RFCBT: rumination-focused cognitive behavioral therapy
    uMARS: Mobile Application Rating Scale–user version

    Edited by A Mavragani; submitted 03.06.22; peer-reviewed by K Stawarz, P Anderson; comments to author 09.09.22; revised version received 05.10.22; accepted 16.10.22; published 13.12.22

    Copyright

    ©Eve A Rosenfeld, Cassondra Lyman, John E Roberts. Originally published in JMIR Formative Research (https://formative.jmir.org), 13.12.2022.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.