This paper is in the following e-collection/theme issue:

Early Results from COVID-19 Studies (299) JMIR Theme Issue: COVID-19 Special Issue (2452) Use and User Demographics of mHealth (336) Formative Evaluation of Digital Health Interventions (2423) Methods and Feasibility Studies (380) Digital Mental Health Interventions, e-Mental Health and Cyberpsychology (1518) Users' and Patients' Needs for Mental Health Services (404)

Published on in Vol 6, No 6 (2022): June

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/38162, first published .
Digital Screening and Automated Resource Identification System to Address COVID-19–Related Behavioral Health Disparities: Feasibility Study

Digital Screening and Automated Resource Identification System to Address COVID-19–Related Behavioral Health Disparities: Feasibility Study

Digital Screening and Automated Resource Identification System to Address COVID-19–Related Behavioral Health Disparities: Feasibility Study

Authors of this article:

Colleen Stiles-Shields1 Author Orcid Image ;   Kathryn R Batts2 Author Orcid Image ;   Karen M Reyes1 Author Orcid Image ;   Joseph Archer1, 3 Author Orcid Image ;   Sharad Crosby1, 4 Author Orcid Image ;   Janel M Draxler1 Author Orcid Image ;   Nia Lennan1 Author Orcid Image ;   Philip Held1 Author Orcid Image
Article Authors Cited by (2) Tweetations (14) Metrics

    Original Paper

    1Section of Community Behavioral Health, Department of Psychiatry and Behavioral Sciences, Rush University Medical Center, Chicago, IL, United States

    2Community Health Research Division, RTI International, Research Triangle Park, NC, United States

    3University of Wisconsin School of Medicine and Public Health, Madison, WI, United States

    4Institute for Translational Medicine, University of Chicago, Chicago, IL, United States

    Corresponding Author:

    Colleen Stiles-Shields, PhD

    Section of Community Behavioral Health

    Department of Psychiatry and Behavioral Sciences

    Rush University Medical Center

    1645 W Jackson Blvd

    Suite 302

    Chicago, IL, 60612

    United States

    Phone: 1 312 942 9998

    Email: colleen_stiles-shields@rush.edu


    Background: Digital mental health (DMH) tools use technology (eg, websites and mobile apps) to conveniently deliver mental health resources to users in real time, reducing access barriers. Underserved communities facing health care provider shortages and limited mental health resources may benefit from DMH tools, as these tools can help improve access to resources.

    Objective: This study described the development and feasibility evaluation of the Emotional Needs Evaluation and Resource Guide for You (ENERGY) System, a DMH tool to meet the mental health and resource needs of youth and their families developed in the context of the COVID-19 pandemic. The ENERGY System offers a brief assessment of resource needs; problem-solving capabilities; and symptoms of depression, anxiety, trauma, and alcohol and substance use followed by automated, personalized feedback based on the participant’s responses.

    Methods: Individuals aged ≥15 years were recruited through community partners, community events, targeted electronic health record messages, and social media. Participants completed screening questions to establish eligibility, entered demographic information, and completed the ENERGY System assessment. Based on the participant’s responses, the ENERGY System immediately delivered digital resources tailored to their identified areas of need (eg, relaxation). A subset of participants also voluntarily completed the following: COVID-19 Exposure and Family Impact Survey (CEFIS) or COVID-19 Exposure and Family Impact Survey Adolescent and Young Adult Version (CEFIS-AYA); resource needs assessment; and feedback on their experience using the ENERGY System. If resource needs (eg, housing and food insecurity) were endorsed, lists of local resources were provided.

    Results: A total of 212 individuals accessed the ENERGY System link, of which 96 (45.3%) completed the screening tool and 86 (40.6%) received resources. Participant responses on the mental health screening questions triggered on average 2.04 (SD 1.94) intervention domains. Behavioral Activation/Increasing Activities was the most frequently launched intervention domain (56%, 54/96), and domains related to alcohol or substance use were the least frequent (4%, 4/96). The most frequently requested support areas were finances (33%, 32/96), transportation (26%, 25/96), and food (24%, 23/96). The CEFIS and CEFIS-AYA indicated higher than average impacts from the pandemic (ie, average scores >2.5). Participants were satisfied with the ENERGY System overall (65%, 39/60) as well as the length of time it took to answer the questions (90%, 54/60), which they found easy to answer (87%, 52/60).

    Conclusions: This study provided initial support for the feasibility of the ENERGY System, a DMH tool capable of screening for resource and mental health needs and providing automated, personalized, and free resources and techniques to meet the identified needs. Future studies should seek direct feedback from community members to further improve the ENERGY System and its dissemination to encourage use.

    JMIR Form Res 2022;6(6):e38162

    doi:10.2196/38162

    Keywords

    digital mental health; underserved; health disparities; COVID-19; screening; referral; mental health; digital health; feasibility study; mobile app; mHealth; mobile health; emotional need; digital health tool; health resource; health care cost 


    Background

    Prior to the COVID-19 pandemic, nearly half of all adolescents and a quarter of all adults in the United States met the criteria for at least one mental health disorder [1,2]. The effects of the COVID-19 pandemic have exacerbated mental health concerns, particularly for adolescents and young adults [3-5]. Further, structural barriers that existed prior to the COVID-19 pandemic primed specific communities to disproportionately experience negative effects from the pandemic [6,7]. However, the majority of these individuals have not received mental health care [8,9]. Mental health stigma, time limitations, and the identification of mental health problems were among the primary barriers to accessing care [10-20]. Additionally, the demand for behavioral health care frequently exceeds clinic capacity, particularly in rural areas [21]. Given the lack of accessible mental health services and the increased need for such resources, it is imperative to identify novel means for providing behavioral health assessment and care, particularly to members of vulnerable communities.

    Digital mental health (DMH) tools are one way to enhance the accessibility of evidence-based interventions and resources [22]. DMH refers to the use of technology platforms, such as websites (ie, eHealth), mobile apps (ie, mobile health), and electronic devices (eg, Fitbit and Apple Watch), to deliver behavioral health assessments and interventions. The use of DMH has expanded rapidly during the COVID-19 pandemic [23], and there are multiple reasons it represents a viable means to provide continued behavioral health resources, particularly for communities disproportionately facing access barriers. First, DMH tools can assess and offer resources for use in real time and real-world environments without requiring individuals to travel to mental health care settings and allow for asynchronous, socially distanced care [24]. Further, DMH can also link to local or web-based resources for basic needs (eg, housing support); facilitating access to resources that support basic physiological needs is consistent with tools aimed at supporting higher level needs, such as mental health [25]. Second, mobile device access is ubiquitous [26], and the use of DMH therefore increases the likelihood of engaging individuals through technology that they are already using (eg, smartphones and tablets). Finally, youths are more likely to seek and access emotional support through technology compared to in-person care [26,27], and the pandemic has increased technology use and ownership across all ages, including older adults [28]. This makes DMH a delivery mechanism that harnesses an environment in which those at the highest risk for pandemic-related mental health concerns are more likely to feel comfortable. Importantly, DMH tools developed prior to and during the COVID-19 pandemic have the potential to extend in utility beyond the current pandemic to potential future crises [29].

    The digital marketplace for DMH tools can be overwhelming with app stores featuring over 10,000 publicly available mobile health apps. The majority of available DMH tools are either not based on evidence-based interventions or have not been rigorously evaluated [28]. In addition to the overwhelming amount of available DMH tools, most require users to have already identified the problems for which they are seeking help, which can be difficult for some [30]. Thus, to make DMH more accessible to vulnerable populations, potential users must be able to identify (1) what problems they are having (eg, anxiety and depression, etc) and (2) which tools are appropriate and efficacious for their needs. Moreover, it is important that interventions be transparent and specifically targeted toward vulnerable groups and adapted to their needs.

    Objective

    To address these challenges, we developed a DMH tool, the Emotional Needs Evaluation and Resource Guide for You (ENERGY) System, to briefly assess mental health and other resource needs and provide automated, personalized feedback based on the identified needs. The objective of this study was to develop the tool and assess its feasibility (ie, engagement and satisfaction) to guide future developments. The ENERGY System was originally designed in the context of the COVID-19 pandemic for adolescents and young adults (AYAs) and their families residing in the West and South Side communities of Chicago but was ultimately deployed throughout the city and surrounding neighborhoods of Chicago, Illinois. These communities have faced stark mental and behavioral health disparities, both prior to and during the pandemic [7,31-33].


    Participants

    Participants were recruited from the Chicagoland area (ie, the city of Chicago, Illinois, and its surrounding suburbs and towns). Recruitment was conducted via (1) the distribution of flyers that contained the study link and QR code to the Rush University Education and Career Hub and local community partners and at community events (eg, Easter egg hunt and vaccine clinics); (2) the posting of flyers and short videos on social media platforms (eg, departmental Twitter account); and (3) targeted messaging to electronic health records (eg, the Epic MyChart app) of patients receiving services from 3 school-based health centers located in Chicago’s West Side communities. The initial deployment of the ENERGY System in March 2021 limited participation to people aged 15-25 years or caregivers of children aged <18 years. However, due to community needs and recognition that families with complex needs extend beyond these subgroups, the inclusion criteria were expanded at the end of May 2021 to include anyone aged ≥15 years.

    Ethics Approval

    The study procedures were approved by the Rush University Medical Center Institutional Review Board (20092408).

    Enrollment Procedure

    Waivers of the documentation of informed consent and parental permission as well as an alteration of consent were obtained. Interested individuals accessed the system via a REDCap [34] link provided on recruitment materials (eg, physical flyers, social media posts, and email blurbs); the REDCap link opened a page on the participant’s personal device’s web browser. REDCap was used to increase accessibility, as users would not be required to download anything to their device to use the ENERGY System. After clicking the link, participants were asked to enter their age and whether they are a parent of a child aged <18 years. Eligible persons were directed to a brief form listing all the required elements of consent. Participants were asked to click “I agree” to continue to the ENERGY System. Those who did not agree to the elements of consent were exited from the system.

    ENERGY System

    System Overview

    The ENERGY System is a brief, DMH-screening, and automated resource identification system. The ENERGY System was developed to provide individuals with an automated tool that can identify various emotional and mental health concerns and provide an automated list of appropriate self-help and other resources to address these identified concerns. Automating the screening and resource identification process was intended to improve the accessibility of such services, which are usually performed by mental health or social service providers and thus have limited scalability. Although the ENERGY System was administered via REDCap [34] for this study, it can also be transported to other platforms.

    ENERGY System Development

    The ENERGY System was designed to minimize the burden related to the assessment of current needs and concerns and maximize the feasibility. To achieve this, the mental health screening portion of the ENERGY System combines 16 items drawn from existing, validated scales assessing problem-solving issues, depression and anxiety symptoms, as well as alcohol and substance use into a single survey. The items used in the ENERGY System were derived from prior research on the development of brief assessments of each of these constructs. Specifically, the items used in the ENERGY System were drawn from the 7-item Generalized Anxiety Disorder Assessment (GAD-7) [35], 5-item PTSD Checklist for DSM-5 (PCL-5) [36], 9-item Patient Health Questionnaire (PHQ-9) [37], 3-item Alcohol Use Disorder Identification Test-Consumption (AUDIT-C) [38], and Drug Abuse Screening Test (DAST-10) [39]. In a separate research study (Christ et al, unpublished data, 2022), our team used Item Response Theory on several of these short forms and the full scales from which they were derived to identify specific items that best represent the underlying latent constructs. For each brief scale derived based on Item Response Theory, the associations with the respective full scales as well as related constructs were assessed. Furthermore, our team evaluated the predictive validity of these brief scales by examining how changes in the brief scales predicted changes in the related constructs over the course of treatment and how this compared to the full scales (Christ et al, unpublished data, 2022). For the ENERGY System, we only retained items/brief scales that explained the underlying constructs and were able to adequately capture changes in these constructs following an intervention. This step was important to ensure that the ENERGY System would be able to accurately reflect changes in the underlying constructs when administered repeatedly. The resulting 16-item ENERGY System uses 2 items from the PCL-5, 4 items from the PHQ-9, 3 items from the GAD-7, 3 items from the AUDIT-C, 3 items from the DAST-10, and 1 author-developed item asking about participants’ ability to handle problems.

    The ENERGY System screening items were then mapped onto 6 different intervention domains. Anxiety-related symptoms were mapped to Relaxation strategies. Depressive symptoms that were primarily associated with the lack of activity were mapped to Behavioral Activation strategies. Anxiety and depressive symptoms that were primarily associated with thinking processes were mapped to Cognitive Restructuring strategies. Problem-solving was mapped to Specific, Measurable, Achievable, Relevant, and Time-bound goals and other Problem-solving strategies. Alcohol and substance use were mapped to Alcohol and Substance Use Management and Harm Reduction strategies. These domains were created to identify specific intervention targets that could be more easily matched to specific evidence-based cognitive behavioral skills. For example, although anxiety is a single construct, different interventions may be more effective depending on the symptom presentation. An individual with anxiety symptoms of feeling on edge or having difficulty relaxing may benefit more from exercises focused on activating the parasympathetic nervous system (eg, diaphragmatic breathing, progressive muscle relaxation, and mindfulness), whereas an individual who has difficulty managing their worried thoughts may benefit from cognitive restructuring. Expert-reviewed resources, including MindTools [40] and PsyberGuide [41], were used to create a repository of DMH tools that have been shown to be beneficial for depression, anxiety, alcohol use, substance use, and problem-solving within personalized resources (eg, COVID Coach App [42]).

    In addition to assessing specific emotional and mental health concerns, a goal of the ENERGY System was to capture resource needs and provide immediate, relevant information about readily accessible resources specific to the location in which the ENERGY System was first tested (ie, the West and South Side communities of Chicago). Namely, participants were directed to indicate whether they needed assistance with any of the following domains: housing, finances, parenting, childcare, food, transportation, medical care, pet care, elderly care, and any other miscellaneous areas of concern. Participants clicked a box next to any domain for which they wanted to receive assistance. These domains were selected because deficits in these areas have been associated with stress and negative impacts on individuals and their families in the short and long term [43]. Local community resources were identified through various methods. A web-based search for local Chicago resources (eg, lists of local food banks and rent/mortgage assistance programs) was conducted, and recommendations for COVID-19 testing sites and community safety hotlines were included to meet the needs of Chicago residents. Additionally, the team incorporated pertinent resources from a pre-existing list of community resources created by a team member for use during clinical care. Community resources were also identified through organizational LISTSERV software applications, colleagues, and a list of resources sent by a local congressman to his constituents. All community resources were checked for accuracy to ensure that they were still available as advertised prior to publishing them into recommendations for study participants.

    ENERGY System Mental Health and Resource Feedback

    Immediately upon the completion of the screening questions, the ENERGY System automatically scored the assessments to identify the domains that could benefit from intervention. Positive endorsement of an intervention domain was defined as a severity rating of at least 50% of the total possible score for the domain. For each positive domain, the ENERGY System automatically provided the individual with personalized feedback (eg, “Based on your responses, you may benefit from learning some relaxation strategies”) and community-based and digital resources tailored to participant responses. Resources were presented in a person-focused rather than clinical manner (eg, using terms such as “sadness” rather than “depression”) to reduce stigma.

    The emotional and mental health feedback involved specific cognitive behavioral techniques that have been identified in prior research to be particularly effective for the respective concerns. For example, individuals who screened positive for sadness were introduced to behavioral activation techniques (referred to as “Increasing Activities”). Those who screened positive for symptoms of anxiety were provided tips and resources around progressive muscle relaxation, self-soothing, and meditation (referred to as “Relaxation”). All techniques were framed in a self-guided format by cognitive-behaviorally trained psychologists and described so that they could be easily followed by users without requiring coaching from a professional. In addition to detailed instructions, links to publicly available resources explaining these techniques (eg, YouTube) were provided to present individuals with added options for learning the different skills.

    Feedback about other available resources was presented in a similar manner. Requested resources were presented as brief excerpts detailing the types of services provided and linking individuals to the agencies’ websites where more details could be found. Each resource opened to a new tab so that individuals could keep their list of recommended resources open in their browser for as long as necessary. Individuals also had the option to enter their email address to have their list automatically emailed to them. The resources feedback page also contained the link to the ENERGY System for the individual to visit again at any time or to share with others.

    Measures

    Demographic Characteristics

    Participants were asked to provide demographic characteristics, including age, gender, ethnicity, race, sexual orientation, the highest level of education, current student/employment status, family’s total annual income before taxes, zip code, and parental status.

    ENERGY System

    As described above, the ENERGY System assessment consisted of 16 items drawn from previously validated symptom scales. The reliabilities of the items in each of the intervention domains were adequate to good (Relaxation: Cronbach α=.77; Behavioral Activation/Increasing Activities: Cronbach α=.83; and Cognitive Restructuring [Low Mood]: Cronbach α=.86). Cronbach α was not calculated for Problem-solving as this was assessed via a single item. Cronbach α’s were also not calculated for Alcohol and Substance Use Management as items on the underlying scales were only triggered after the initial question about any alcohol or substance use was positively endorsed. Thus, participants had a widely varying number of responses to these items.

    ENERGY System Satisfaction

    Immediately after reviewing their recommended resources, participants were given the opportunity to provide overall feedback about the ENERGY System. Elicited feedback involved overall satisfaction and satisfaction with the ease of answering the questions, amount of time it took to complete, and recommended resources. Participants were also asked how likely they were to use the resources in the future and recommend the ENERGY System to others. Finally, participants were asked whether they would be interested in receiving professional support for resources or emotional needs, such as via connection to a therapist or primary care provider, if this were to be offered in the future.

    COVID-19 Exposure and Family Impact

    The COVID-19 Exposure and Family Impact Survey (CEFIS) [44] or the COVID-19 Exposure and Family Impact Survey Adolescent and Young Adult Version (CEFIS-AYA) [45] were administered, depending on age (CEFIS for all adults aged ≥30 years and CEFIS-AYA for AYAs aged 15-29 years). The CEFIS and CEFIS-AYA assess the impact of the COVID-19 pandemic on adolescents, young adults, and families. The CEFIS yields (1) an Exposure score (scores range from 0-25, with higher scores indicating greater exposure); (2) an Impact score (scores averaged from a 4-point Likert scale with scores >2.5 considered positively valenced, meaning more impact, and scores <2.5 considered negatively valenced, meaning less impact); (3) a Distress score (scores range from 1-10, with higher scores indicating greater distress); and (4) an open-ended question to promote the sharing of details not covered. The CEFIS-AYA differs from the CEFIS by having 3 additional Exposure items, 6 additional Impact items, and a single item for the Distress scale to reflect the experience of AYAs compared to that of a family. The CEFIS demonstrated acceptable reliability for the current sample (αs>.73). However, due to the possibility that adult respondents completed the CEFIS without having children, the CEFIS Distress was reported at the item level (personal and child, separately) as opposed to the scale level (personal and child combined). The CEFIS-AYA demonstrated acceptable reliability across subscales (αs>.69).

    Data Analysis

    Descriptive analyses were used to (1) characterize the sample in terms of demographic characteristics and reported resource needs; (2) determine use rates; (3) identify the total provided mental health resources; and (4) assess user satisfaction with the ENERGY System.


    Participants

    Table 1 depicts the demographic characteristics of the sample. Of the 110 participants who provided demographic data, they were primarily AYAs (68.2%, n=75), cisgender female (81.3%, n=91), heterosexual or straight (67.6%, n=75), non-Hispanic/Latinx (67.3%, n=74), and Black or African American (40.9%, n=45) or White (40%, n=44). A total of 103 (93.6%) participants provided their zip codes. Of these 103 participants, the majority (95.1%, n=98) lived in Illinois, with 80 (77.7%) residing within the city limits of Chicago. The remaining 5 (4.9%) were from California, Indiana, New York, Texas, and Wisconsin.

    Table 1. Participant characteristics.
    CharacteristicAdolescents and young adultsa (n=75)Adultsa (n=35)All participants (N=110)
    Age (years), mean (SD; range)18.95 (3.18; 15-25)43.89 (13.94; 26-71)26.88 (14.27; 15-71)
    Gender, n (%)

    		Cisgender female68 (90.7)22 (62.9)90 (81.8)

    		Cisgender male4 (5.3)11 (31.4)15 (13.6)

    		Transgender female0 (0)1 (2.9)1 (0.9)

    		Transgender male1 (1.3)1 (2.9)2 (1.8)

    		Nonbinary1 (1.3)0 (0)1 (0.9)

    		Prefer not to answer1 (1.3)0 (0)1 (0.9)
    Ethnicity, n (%)

    		Hispanic/Latinx28 (37.3)7 (20)35 (31.8)

    		Non-Hispanic/Latinx47 (62.7)27 (77.1)74 (67.3)

    		Prefer not to answer0 (0)1 (2.9)1 (0.9)
    Raceb, n (%)

    		American Indian or Alaskan Native1 (1.3)1 (2.9)2 (1.8)

    		Asian5 (6.7)2 (5.7)7 (6.4)

    		Black or African American38 (50.7)7 (20)45 (40.9)

    		White20 (26.7)24 (68.6)44 (40)

    		Other5 (6.7)1 (2.9)6 (5.5)

    		Prefer not to answer10 (13.3)1 (2.9)11 (10)
    Sexual orientation, n (%)

    		Heterosexual or straight51 (68)24 (68.6)75 (68.2)

    		Gay or lesbian3 (4)5 (14.3)8 (7.3)

    		Bisexual or pansexual18 (24)4 (11.4)22 (20)

    		Asexual0 (0)1 (2.9)1 (0.9)

    		Other1 (1.3)0 (0)1 (0.9)

    		Prefer not to answer2 (2.7)1 (2.9)3 (2.7)
    Highest level of education, n (%)

    		Finished grade school5 (6.7)0 (0)5 (4.5)

    		Some high school29 (38.7)0 (0)29 (26.4)

    		Finished high school9 (12)0 (0)9 (8.2)

    		Business or technical school0 (0)1 (2.9)1 (0.9)

    		Some college21 (28)5 (14.3)26 (23.6)

    		Finished college8 (10.7)8 (22.9)16 (14.5)

    		Some graduate or professional school1 (1.3)6 (17.1)7 (6.4)

    		Finished graduate or professional school2 (2.7)15 (42.9)17 (15.5)

    aAdolescents and young adults were aged 15-25 years and adults were aged ≥26 years.

    bThe total number of reported racial identities is 115 due to participants identifying with more than one racial category.

    ENERGY System Use Data

    Between August 2020 and December 2021, the ENERGY System link was accessed 212 times. Attrition occurred with each advancement of the system, with 84.4% (n=179) completing the screening questions, 51.9% (n=110) completing the demographic questions, 45.3% (n=96) completing the resource needs and mental health screening, 40.6% (n=86) completing the review of their personalized resources, and 28.3% (n=60) providing feedback about the ENERGY System. On average, participants completed their interactions with the system in 12 (SD 8.33) minutes (6 outliers were removed from the average calculation as it was believed that they completed their interactions across multiple visits [ie, >1 hour], with a range from 1 hour and 20 minutes to 23 hours and 29 minutes). Figure 1 depicts the flow chart for the ENERGY System interactions.

    ‎
    Figure 1. Flow chart of participants through ENERGY System use. ENERGY: Emotional Needs Evaluation and Resource Guide for You.
    View this figure

    Mental Health Intervention Domains

    A total of 96 participants (45.3%, N=212) completed the mental health screening questions. Participant responses on the mental health screening questions triggered on average 2.04 (SD 1.94) intervention domains. Of the 96 responses, Behavioral Activation/Increasing Activities was the most launched intervention domain (56%, n=54), followed by Relaxation (44%, n=42), Problem-solving (41%, n=39), and Cognitive Restructuring (Low Mood; 32%, n=31). Alcohol Use and Substance Use were the least frequently launched intervention domains, launching only 4 (4%) times each.

    Resource Needs

    A total of 96 participants (45.3%, N=212) completed resource needs questions, with half (50%, n=48) responding in a way that indicated that they did not need assistance with any resources. As such, the average number of resource categories provided to participants was 0.69 (SD1.44; range 0-7). Of the 96 responses, the endorsed resource needs included financial support (33%, n=32), transportation (26%, n=25), food (24%, n=23), housing (17%, n=16), medical care (14%, n=13), parenting (12%, n=12), pet care (11%, n=11), childcare (10%, n=10), and resources for caretakers of older adults (5%, n=5).

    COVID-19 Exposure and Impact

    A total of 64 AYA participants and 24 adult participants completed some portion of the CEFIS-AYA and CEFIS, respectively. To limit participant burden, Exposure subscale items were offered optionally and completed by 16 (25%, N=64) AYAs and 17 (71%, N=24) adults. The AYAs and adults had an average Exposure rating of 13.13 (SD 4.00; range 6-20) and 8.65 (SD 3.02; range 3-16), respectively. Both the AYA and adult Impact subscale scores were positively valenced (more impact), with AYAs having an average Impact score of 2.82 (SD 0.65; range 1.21-4.00) and adults having an average Impact score of 3.00 (SD 0.52; range 2.10-3.78). AYAs endorsed an average personal Distress rating of 5.78 (SD 2.44; range 1-10). Adults reported their personal Distress on average as 6.46 (SD 2.41; range 2-10) and their children’s Distress was rated on average as 4.64 (SD 3.01; range 1-10).

    ENERGY System Satisfaction

    A total of 60 (28.3%, N=212) participants provided feedback about the ENERGY System following use. Of the 60 participants, the majority agreed or strongly agreed that they were satisfied with the system overall (65%, n=39), ease of answering questions (87%, n=52), and time it took to answer questions (90%, n=54). The majority (75%, n=45) also agreed or strongly agreed that the ENERGY System asked about all of their needs. Finally, the majority (78%, n=47) somewhat agreed, agreed, or strongly agreed that they were likely to use the system’s recommended resources.


    Principal Findings

    This study aimed to develop and assess the feasibility of a DMH tool that (1) screened for resource and mental health needs and (2) provided automated, personalized, and free resources and techniques to meet the identified needs. Dissemination efforts focused on institutional and community partner collaborations to focus on use by residents of communities facing disproportionate behavioral health disparities. Over 200 community members, largely within the Chicagoland area, demonstrated initial interest in the ENERGY System. However, attrition occurred with each progressing stage of the system (ie, screener, demographic questions, resource needs and mental health screening assessment, review of personalized resources, and system feedback). Half of all individuals who completed the screening questions endorsed not having resource needs. On average, individuals received resources for 2 mental health symptoms and lists based on their responses. As is common for subjective evaluations of DMH, even in the face of relatively low real-world engagement [46], users who provided feedback were primarily satisfied with the system.

    The current sample endorsed higher than average impacts from the COVID-19 pandemic, with AYAs reporting higher exposure to pandemic experiences than their adult counterparts [44]. These experiences have occurred within the context of historical and current behavioral health disparities and broader community hardships [32]. As such, the COVID-19 pandemic has exacerbated the already existent need for scalable, accessible, and affordable DMH, particularly for vulnerable communities [47]. Although multiple studies of pandemic-specific DMH interventions have occurred [48], the ENERGY System is unique in its intended use (ie, single or repeated use) and potential for extended utility beyond the pandemic (eg, compliant with but not centered upon social distancing practices). However, to be useful for communities moving forward, multiple limitations will need to be addressed.

    Accessibility

    The ENERGY System was intended to serve those facing the greatest hardships and disparities from the pandemic. However, the majority of sample members denied requiring resource needs, and about 40% of the adults who used the system had a graduate or professional degree. These findings likely indicate that our recruitment efforts did not reach many of the residents requiring this additional support. Further, slightly over 200 initial clicks to the system suggest that the deployment had a low reach (eg, Chicago's Lower West Side community has a population of about 12,000 people per square mile) [49]. Therefore, increasing accessibility is an important factor for future deployments of the ENERGY System and similar DMH tools. The ENERGY System was designed for an immediate need and in the context of social distancing mandates. However, best practices would include the incorporation of human-centered design [50-52] and community-based participatory research methodologies [53] to ensure that likely end users are active collaborators—those with agency—throughout the design and deployment phases. Doing so better ensures that DMH is accessible to and appropriate for the intended users [22,54]. For example, nearly one-third of the sample identified as Hispanic or Latinx, but the tool was only available in the English language. The use of human-centered design practices with community members would provide ongoing assessment of likely end users’ native languages and language preferences for DMH, promoting the likelihood of increasing accessibility to nonnative English speakers.

    Engagement

    Less than half of the sample members completed the resource needs and mental health screening questions. The inclusion of additional questions for the purposes of research (eg, CEFIS and satisfaction questions) likely contributed to some of the attrition that occurred with the ENERGY System in this study. We note that engagement with DMH in real-world settings is generally low [55], with top barriers including (1) stigma, (2) problem recognition, and (3) knowledge of treatment options [10-12]. The ENERGY System is designed to address these concerns (ie, anonymous participation on a personal device to address stigma; feedback about mental health symptoms to address problem recognition; and free resources provided to address knowledge of treatment options). However, engagement with the system was low. These findings again support the need to involve representative potential users throughout the design process. It is unsurprising that community members systemically experiencing disproportionate disparities may not engage well with DMH in real-world settings, as DMH has historically been developed without their input regarding their lived experiences and needs [56]. A focus on engagement strategies with systemically excluded and marginalized community members must also be a focus of future design and research.

    Limitations

    This study should be considered in light of its specific limitations. First, social distancing mandates limited recruitment efforts. Namely, typical recruitment methods of community partners often involve in-person communication (even in the case of sharing a website link or QR code), which was limited due to the pandemic. A large portion of flier dissemination occurred through mass emails; this methodology likely resulted in study materials being overshadowed by other web-based obligations [57,58]. Second, remote learning or e-learning limited the ability of recruitment partners in school settings to engage and explain the study to younger potential participants. Third, the ENERGY System was only provided in the English language. Roughly 16% of Chicago residents do not speak English as a primary language, with Spanish being the most frequently spoken language among this group [59]. This barrier left several Chicago families unable to complete the screener. Future resource tools similar to the ENERGY System should create multiple translations, based on formative assessments of preferred language, for optimal use among diverse communities. Finally, the ENERGY System was disseminated through the Rush University Medical Center clinics and community partnerships. Although some users may trust DMH when overseen by a university setting due to institutional review board oversight, others are less inclined to trust university and larger health systems–sponsored DMH due to historical injustices [60]. Future research should collaborate with community members to assess how the role of academic institution involvement may promote or hinder DMH engagement.

    Conclusions

    This study provided initial support for the feasibility of the ENERGY System, a DMH tool capable of screening for resource and mental health needs and providing automated, personalized, and free resources and techniques to meet the identified needs. Future research should involve collaboration with community members and collect more detailed formative and summative data from representative end users of the ENERGY System. As such, better attempts to meet the needs of users with intersectional identities and varying mental health and resource needs may result in improved engagement [46]. Additionally, larger scale research is needed to determine the feasibility of the ENERGY System beyond the areas targeted in this study. Moreover, longitudinal studies are needed to determine whether individuals use the resources and mental health skills they were provided and whether the use of skills and resources improves their reported symptoms over time.

    Acknowledgments

    This research was supported in part by an R3 funding mechanism supported by the Rush University Medical Center and RTI International. CSS was also supported by fellowships from the National Institute of Mental Health (K08 MH125069) and the Cohn Family Foundation. PH was supported by a career development award from the National Center for Advancing Translational Sciences (KL2-TR002387) and received additional grant from the Wounded Warrior Project. We thank the Rush Community-Based Practices, comprising the school-based health centers and the Adolescent Family Center; Rush Education and Career Hub; and our West Side community partners, without whom this research would not have been possible.

    Conflicts of Interest

    None declared.

    1. Merikangas KR, He JP, Burstein M, Swanson SA, Avenevoli S, Cui L, et al. Lifetime prevalence of mental disorders in U.S. adolescents: results from the National Comorbidity Survey Replication--Adolescent Supplement (NCS-A). J Am Acad Child Adolesc Psychiatry 2010 Oct;49(10):980-989. [CrossRef] [Medline]
    2. Key substance use and mental health indicators in the United States: results from the 2020 National Survey on Drug Use and Health. Substance Abuse and Mental Health Services Administration. 2021 Oct.   URL: https:/​/www.​samhsa.gov/​data/​sites/​default/​files/​reports/​rpt35325/​NSDUHFFRPDFWHTMLFiles2020/​2020NSDUHFFR1PDFW102121.​pdf [accessed 2022-06-14]
    3. Magson NR, Freeman JYA, Rapee RM, Richardson CE, Oar EL, Fardouly J. Risk and protective factors for prospective changes in adolescent mental health during the COVID-19 pandemic. J Youth Adolesc 2021 Jan;50(1):44-57. [CrossRef] [Medline]
    4. Czeisler, Lane RI, Wiley JF, Czeisler CA, Howard ME, Rajaratnam SMW. Follow-up survey of US adult reports of mental health, substance use, and suicidal ideation during the COVID-19 pandemic, September 2020. JAMA Netw Open 2021 Feb 01;4(2):e2037665 [FREE Full text] [CrossRef] [Medline]
    5. Chadi N, Spinoso-Di Piano C, Osmanlliu E, Gravel J, Drouin O. Mental health-related emergency department visits in adolescents before and during the COVID-19 pandemic: a multicentric retrospective study. J Adolesc Health 2021 Nov;69(5):847-850 [FREE Full text] [CrossRef] [Medline]
    6. Dorn AV, Cooney RE, Sabin ML. COVID-19 exacerbating inequalities in the US. Lancet 2020 Apr 18;395(10232):1243-1244. [CrossRef] [Medline]
    7. Pierce JB, Harrington K, McCabe ME, Petito LC, Kershaw KN, Pool LR, et al. Racial/ethnic minority and neighborhood disadvantage leads to disproportionate mortality burden and years of potential life lost due to COVID-19 in Chicago, Illinois. Health Place 2021 Mar;68:102540 [FREE Full text] [CrossRef] [Medline]
    8. Walker ER, Cummings JR, Hockenberry JM, Druss BG. Insurance status, use of mental health services, and unmet need for mental health care in the United States. Psychiatr Serv 2015 Jun;66(6):578-584. [CrossRef] [Medline]
    9. Merikangas KR, He J, Burstein M, Swendsen J, Avenevoli S, Case B, et al. Service utilization for lifetime mental disorders in U.S. adolescents: results of the National Comorbidity Survey-Adolescent Supplement (NCS-A). J Am Acad Child Adolesc Psychiatry 2011 Jan;50(1):32-45. [CrossRef] [Medline]
    10. Radez J, Reardon T, Creswell C, Lawrence PJ, Evdoka-Burton G, Waite P. Why do children and adolescents (not) seek and access professional help for their mental health problems? a systematic review of quantitative and qualitative studies. Eur Child Adolesc Psychiatry 2021 Feb 21;30(2):183-211 [FREE Full text] [CrossRef] [Medline]
    11. Gulliver A, Griffiths KM, Christensen H. Perceived barriers and facilitators to mental health help-seeking in young people: a systematic review. BMC Psychiatry 2010 Dec 30;10:113 [FREE Full text] [CrossRef] [Medline]
    12. Wisdom JP, Clarke GN, Green CA. What teens want: barriers to seeking care for depression. Adm Policy Ment Health 2006 Mar;33(2):133-145. [CrossRef] [Medline]
    13. Reardon T, Harvey K, Baranowska M, O'Brien D, Smith L, Creswell C. What do parents perceive are the barriers and facilitators to accessing psychological treatment for mental health problems in children and adolescents? a systematic review of qualitative and quantitative studies. Eur Child Adolesc Psychiatry 2017 Jun 4;26(6):623-647. [CrossRef] [Medline]
    14. Hyde PS, Substance Abuse and Mental Health Services Administration. Report to Congress on the nation's substance abuse and mental health workforce issues. California Institute for Behavioral Health Solutions. 2013 Jan 24.   URL: http://www.cimh.org/sites/main/files/file-attachments/samhsa_bhwork_0.pdf [accessed 2022-06-14]
    15. Salloum A, Johnco C, Lewin AB, McBride NM, Storch EA. Barriers to access and participation in community mental health treatment for anxious children. J Affect Disord 2016 May 15;196:54-61. [CrossRef] [Medline]
    16. Douthit N, Kiv S, Dwolatzky T, Biswas S. Exposing some important barriers to health care access in the rural USA. Public Health 2015 Jun;129(6):611-620. [CrossRef] [Medline]
    17. Allen EM, Call KT, Beebe TJ, McAlpine DD, Johnson PJ. Barriers to care and health care utilization among the publicly insured. Med Care 2017 Mar;55(3):207-214. [CrossRef] [Medline]
    18. Hacker K, Anies M, Folb BL, Zallman L. Barriers to health care for undocumented immigrants: a literature review. Risk Manag Healthc Policy 2015;8:175-183 [FREE Full text] [CrossRef] [Medline]
    19. Stevens J, Kelleher KJ, Ward-Estes J, Hayes J. Perceived barriers to treatment and psychotherapy attendance in child community mental health centers. Community Ment Health J 2006 Oct;42(5):449-458. [CrossRef] [Medline]
    20. de Haan AM, Boon AE, de Jong JTVM, Hoeve M, Vermeiren RRJM. A meta-analytic review on treatment dropout in child and adolescent outpatient mental health care. Clin Psychol Rev 2013 Jul;33(5):698-711. [CrossRef] [Medline]
    21. Andrilla CHA, Patterson DG, Garberson LA, Coulthard C, Larson EH. Geographic variation in the supply of selected behavioral health providers. Am J Prev Med 2018 Jun;54(6 Suppl 3):S199-S207 [FREE Full text] [CrossRef] [Medline]
    22. Lattie EG, Stiles-Shields C, Graham AK. An overview of and recommendations for more accessible digital mental health services. Nat Rev Psychol 2022 Jan 26;1(2):87-100. [CrossRef]
    23. Clipper B. The influence of the COVID-19 pandemic on technology: adoption in health care. Nurse Lead 2020 Oct;18(5):500-503. [CrossRef] [Medline]
    24. Torous J, Jän Myrick K, Rauseo-Ricupero N, Firth J. Digital mental health and COVID-19: using technology today to accelerate the curve on access and quality tomorrow. JMIR Ment Health 2020 Mar 26;7(3):e18848 [FREE Full text] [CrossRef] [Medline]
    25. SAMHSA's working definition of recovery. Substance Abuse and Mental Health Services Administration. 2012.   URL: https://store.samhsa.gov/sites/default/files/d7/priv/pep12-recdef.pdf [accessed 2022-06-14]
    26. Anderson M, Jiang J. Teens' social media habits and experiences. Pew Research Center. 2018 Nov 28.   URL: https://www.pewresearch.org/internet/2018/11/28/teens-social-media-habits-and-experiences/ [accessed 2022-06-14]
    27. Anderson M, Jiang J. Pew Research Center. 2018 May 31.   URL: https://www.pewinternet.org/2018/05/31/teens-social-media-technology-2018/ [accessed 2022-06-14]
    28. Faverio M. Share of those 65 and older who are tech users has grown in the past decade. Pew Research Center. 2022 Jan 13.   URL: https:/​/www.​pewresearch.org/​fact-tank/​2022/​01/​13/​share-of-those-65-and-older-who-are-tech-users-has-grown-in-the-past-decade/​ [accessed 2022-02-08]
    29. Stiles-Shields C, Plevinsky JM, Psihogios AM, Holmbeck GN. Considerations and future directions for conducting clinical research with pediatric populations during the COVID-19 pandemic. J Pediatr Psychol 2020 Aug 01;45(7):720-724. [CrossRef] [Medline]
    30. Evans-Lacko S, Stolzenburg S, Gronholm PC, Ribeiro W, York-Smith M, Schomerus G. Psychometric validation of the self-identification of having a mental illness (SELF-I) scale and the relationship with stigma and help-seeking among young people. Soc Psychiatry Psychiatr Epidemiol 2019 Jan 4;54(1):59-67 [FREE Full text] [CrossRef] [Medline]
    31. Hunt BR, Tran G, Whitman S. Life expectancy varies in local communities in Chicago: racial and spatial disparities and correlates. J Racial Ethn Health Disparities 2015 Dec;2(4):425-433. [CrossRef] [Medline]
    32. Lange-Maia BS, De Maio F, Avery EF, Lynch EB, Laflamme EM, Ansell DA, et al. Association of community-level inequities and premature mortality: Chicago, 2011-2015. J Epidemiol Community Health 2018 Dec;72(12):1099-1103. [CrossRef] [Medline]
    33. Dircksen JC, Prachand NG, Adams D, Bocskay K, Brown J, Cibulskis A, et al. Healthy Chicago 2.0: partnering to improve health equity. City of Chicago. 2016 Mar.   URL: https://www.chicago.gov/content/dam/city/depts/cdph/CDPH/HC2.0Plan_3252016.pdf [accessed 2022-06-14]
    34. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009 Apr;42(2):377-381 [FREE Full text] [CrossRef] [Medline]
    35. Spitzer RL, Kroenke K, Williams JBW, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med 2006 May 22;166(10):1092-1097. [CrossRef] [Medline]
    36. Weathers FW, Litz BT, Keane TM, Palmieri PA, Marx BP, Schnurr PP. The PTSD checklist for DSM-5 (PCL-5). National Center for PTSD. 2013.   URL: https://www.ptsd.va.gov/professional/assessment/adult-sr/ptsd-checklist.asp [accessed 2022-06-14]
    37. Kroenke K, Spitzer RL, Williams JBW. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001 Sep;16(9):606-613 [FREE Full text] [CrossRef] [Medline]
    38. Bradley KA, DeBenedetti AF, Volk RJ, Williams EC, Frank D, Kivlahan DR. AUDIT-C as a brief screen for alcohol misuse in primary care. Alcohol Clin Exp Res 2007 Jul;31(7):1208-1217. [CrossRef] [Medline]
    39. Villalobos-Gallegos L, Pérez-López A, Mendoza-Hassey R, Graue-Moreno J, Marín-Navarrete R. Psychometric and diagnostic properties of the Drug Abuse Screening Test (DAST): comparing the DAST-20 vs. the DAST-10. Salud Ment 2015 Apr 14;38(2):89-94. [CrossRef]
    40. MindTools.io – find top mental health apps based on expert reviews. MindTools.   URL: https://mindtools.io/ [accessed 2022-02-09]
    41. Neary M, Schueller SM. State of the field of mental health apps. Cogn Behav Pract 2018 Nov;25(4):531-537. [CrossRef] [Medline]
    42. Mobile apps: COVID Coach. PTSD: National center for PTSD.   URL: https://www.ptsd.va.gov/appvid/mobile/COVID_coach_app.asp [accessed 2022-05-31]
    43. Francis L, DePriest K, Wilson M, Gross D. Child poverty, toxic stress, and social determinants of health: screening and care coordination. Online J Issues Nurs 2018 Sep;23(3):2. [CrossRef] [Medline]
    44. Kazak AE, Alderfer M, Enlow PT, Lewis AM, Vega G, Barakat L, et al. COVID-19 exposure and family impact scales: factor structure and initial psychometrics. J Pediatr Psychol 2021 Jun 03;46(5):504-513. [CrossRef] [Medline]
    45. Schwartz LA, Lewis AM, Alderfer MA, Vega G, Barakat LP, King-Dowling S, et al. COVID-19 exposure and family impact scales for adolescents and young adults. J Pediatr Psychol 2022 Jun 07;47(6):631-640. [CrossRef] [Medline]
    46. Ng MM, Firth J, Minen M, Torous J. User engagement in mental health apps: a review of measurement, reporting, and validity. Psychiatr Serv 2019 Jul 01;70(7):538-544. [CrossRef] [Medline]
    47. Figueroa CA, Aguilera A. The need for a mental health technology revolution in the COVID-19 pandemic. Front Psychiatry 2020;11:523 [FREE Full text] [CrossRef] [Medline]
    48. Aguilera A, Hernandez-Ramos R, Haro-Ramos AY, Boone CE, Luo TC, Xu J, et al. A text messaging intervention (StayWell at Home) to counteract depression and anxiety during COVID-19 social distancing: pre-post study. JMIR Ment Health 2021 Nov 01;8(11):e25298 [FREE Full text] [CrossRef] [Medline]
    49. Maciag M. Population density for U.S. cities statistics. Governing. 2013 Oct 01.   URL: http://www.governing.com/gov-data/population-density-land-area-cities-map.html [accessed 2022-01-07]
    50. ISO 9241-210:2019 ergonomics of human-system interaction — part 210: human-centred design for interactive systems. International Organization for Standardization. 2019 Jul.   URL: https://www.iso.org/standard/77520.html [accessed 2022-06-14]
    51. IDEO. The Field Guide to Human-Centered Design. San Francisco, CA: IDEO; 2015.
    52. An introduction to design thinking: process guide. Institute of Design at Stanford. 2010.   URL: https://web.stanford.edu/~mshanks/MichaelShanks/files/509554.pdf [accessed 2022-06-14]
    53. Collins SE, Clifasefi SL, Stanton J, The Leap Advisory Board, Straits KJE, Gil-Kashiwabara E, et al. Community-based participatory research (CBPR): towards equitable involvement of community in psychology research. Am Psychol 2018 Oct;73(7):884-898. [CrossRef] [Medline]
    54. Stiles-Shields C, Potthoff LM, Bounds DT, Burns MTS, Draxler JM, Otwell CH, et al. Harnessing phones to target pediatric populations with socially complex needs: systematic review. JMIR Pediatr Parent 2020 Aug 26;3(2):e19269 [FREE Full text] [CrossRef] [Medline]
    55. Torous J, Nicholas J, Larsen ME, Firth J, Christensen H. Clinical review of user engagement with mental health smartphone apps: evidence, theory and improvements. Evid Based Ment Health 2018 Aug;21(3):116-119. [CrossRef] [Medline]
    56. Hill C, Molitor M, Ortiz C. Racism and inequity are products of design. they can be redesigned. Medium. 2016 Nov 15.   URL: https:/​/medium.​com/​@equityXdesign/​racism-and-inequity-are-products-of-design-they-can-be-redesigned-12188363cc6a [accessed 2022-06-14]
    57. Hayes S, Priestley JL, Ishmakhametov N, Ray HE. “I’m not working from home, I’m living at work”: perceived stress and work-related burnout before and during COVID-19. PsyArXiv Preprint posted online on July 8, 2020. [CrossRef]
    58. Morgantini LA, Naha U, Wang H, Francavilla S, Acar, Flores JM, et al. Factors contributing to healthcare professional burnout during the COVID-19 pandemic: a rapid turnaround global survey. PLoS One 2020;15(9):e0238217 [FREE Full text] [CrossRef] [Medline]
    59. Language access. City of Chicago.   URL: https://www.chicago.gov/city/en/depts/mayor/supp_info/office-of-new-americans/language-access.html [accessed 2022-01-07]
    60. Lattie EG, Stiles-Shields C, Graham AK. An overview of and recommendations for more accessible digital mental health services. Nat Rev Psychol 2022 Jan 26;1(2):87-100. [CrossRef]

    ‎
    AUDIT-C: 3-item Alcohol Use Disorder Identification Test-Consumption
    AYA: adolescent and young adult
    CEFIS: COVID-19 Exposure and Family Impact Survey
    CEFIS-AYA: COVID-19 Exposure and Family Impact Survey Adolescent and Young Adult Version
    DAST-10: Drug Abuse Screening Test
    DMH: digital mental health
    ENERGY: Emotional Needs Evaluation and Resource Guide for You
    GAD-7: 7-item Generalized Anxiety Disorder Assessment
    PCL-5: 5-item PTSD Checklist for DSM-5
    PHQ-9: 9-item Patient Health Questionnaire

    Edited by A Mavragani; submitted 21.03.22; peer-reviewed by L Francis, A Haro-Ramos; comments to author 23.05.22; revised version received 03.06.22; accepted 05.06.22; published 22.06.22

    Copyright

    ©Colleen Stiles-Shields, Kathryn R Batts, Karen M Reyes, Joseph Archer, Sharad Crosby, Janel M Draxler, Nia Lennan, Philip Held. Originally published in JMIR Formative Research (https://formative.jmir.org), 22.06.2022.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.