All procedures were approved by the Institutional Review Board at Lurie Children’s Hospital and the University of Chicago (2018-2189).

The 2017 rate (27.9 per 100,000 population) of HIV infection diagnoses in Chicago is approximately 2.5 times higher than the national rate, and the prevalence rate for Chicago (827.9 per 100,000 population) is nearly 3 times the national rate [27]. Community areas on the West and South sides of Chicago represent a concentrated HIV epidemic, with HIV prevalence consistently over 5% [3,27]. The study recruitment site was a Planned Parenthood Illinois clinic that is located centrally within the city and serves predominantly patients of color.

A community advisory board (CAB) of 9 young Black women was formed to guide and tailor content of the app prototype. CAB participants were recruited through partner agencies, email listservs, and flyers posted throughout the community. Recruitment flyers included information about the project, participation expectations, and learning opportunities (ie, the app design process). Our goal was to have a group of 8-10 young women who met the inclusion criteria for the larger pilot study to participate in the CAB. CAB members were provided with a US $50 incentive for each 2-hour meeting they attended.

Guided by user-centered design and the information systems research frameworks, the study team conducted relevance, rigor, and design cycles to refine and adapt an existing prototype app called miPrEP [28]. The miPrEP app was developed to increase PrEP knowledge and engagement among young Black men who have sex with men [29]. The CAB conducted 4 design sprints over a 3-month period; sessions were guided by generating app content and approving mock-ups (ie, iterative design process). In the rigor cycle, the CAB members met and reviewed the available components of the miPrEP app. The miPrEP app contained basic PrEP information, as well as a video about how PrEP works in the body. The study team identified and presented additional health-based apps to the CAB for review. Existing content and desired features of an adapted sexual health app were discussed and ranked. Finally, features and content were identified for inclusion and tailoring for the In the Loop prototype. In the design cycle, a low fidelity prototype was developed. Paper and pencil mock-ups were used to design the visual assets, video content, and text materials. The prototype of In the Loop was created using paper models to represent each intervention component and a Google slide deck to demonstrate app navigation and flow. All elements were iteratively revised with the CAB until there was consensus on the content, quantities, and the form of the app.

Potential participants were recruited from a Planned Parenthood clinic waiting room and, if interested and eligible, they completed informed consent and the study procedures. Participants were eligible if they (1) reported being cisgender women; (2) identified as African American or Black; (3) were English speakers; (4) were aged 18-24 years (inclusive); (5) reported vaginal or anal sex with a male partner within the past 6 months; (6) were seeking testing or treatment for a sexually transmitted infection, or seeking contraception or abortion services; (7) self-reported being HIV-negative; (8) were not currently pregnant; and (9) were neither currently taking PrEP as HIV prevention nor intending to initiate PrEP at the current clinic appointment.

A pre- and postdesign assessment was used in this pilot study to evaluate the impact of the app on PrEP knowledge immediately after using the app. PrEP knowledge was measured via 7 true or false items that were selected based on use in previous studies with cisgender women [14,30]. Other study aims were to assess usability, including basic feasibility and acceptability of the app prototype among young Black women. Usability was assessed through open-ended questions that asked participants their opinion about the app (eg, “what did you like about using In the Loop while waiting for your appointment?” and “what information would you add to In the Loop app?”) and ways to improve the app (eg, text or graphic display, navigation, and content). Acceptability of In the Loop was evaluated using a series of 9 items derived from an abbreviated acceptability rating scale used in prior research [31]. Participants used a 5-point Likert scale (1=strongly disagree and 5=strongly agree) to rate the extent to which they agreed with each acceptability statement (eg, “I found the information in the app easy to understand and comprehend”). The measure was administered at the immediate postintervention assessment. Overall interest in using mHealth for sexual health was measured via a 7-item scale based on previous research with adolescents (eg, “how likely are you to download a sexual health mobile phone app?”) [32-34]. Finally, participants were asked about preferences toward emerging modalities of PrEP, specifically their interest in a vaginal ring, implant, or injection, or PrEP combined with birth control [35].

Descriptive statistics (eg, mean, SD, and percentage values) were used to describe the sample, and Wilcoxon matched-pairs signed-ranks test was used to detect changes in PrEP knowledge before and immediately after using the app [36]. Open-ended responses were recorded verbatim and were thematically coded by the first author (AKJ), following a standardized procedure [37]. Each open-ended response was read twice, themes were then created based on response content, and each response was coded into one of 4 main themes. If the response contained more than one theme, it was coded as having both themes.

The In the Loop prototype was evaluated from July 2019 through August 2019. A total of 50 sexually active, young Black women aged 18-24 (mean 21, SD 1.9) years were enrolled in the study. Initially, a total of 139 individuals were screened, with 62 being eligible; of those eligible, 50 (81%) enrolled and completed the study visit (Figure 1).

Of the 50 participants, 23 (46%) reported being current students, with 21 (42%) participants reporting their highest level of education as some college education, 19 (38%) as high school graduate, and 9 (18%) of them reported having completed college (Table 1). The majority of participants (n=37, 74%) were employed either full- or part-time. To characterize potential social determinants of health that impact HIV transmission, participants reported whether they had ever received government assistance (n=19, 38%) and whether they had ever been homeless (n=11, 22%).

A total of 44 (88%) participants reported identifying as straight or heterosexual, and 6 (12%) reported their sexual orientation as bisexual. In terms of sexual behavior, the majority (n=42, 84%) reported having 1 sex partner in the past 3 months. Inconsistent condom use was common, with 43 (86%) participants reporting sometimes or never using condoms during vaginal sex; 9 (18%) participants reported having sex with a person whose HIV status they did not know; 6 (12%) of them had a recent STI diagnosis, and 24 (48%) had experienced an unplanned pregnancy.

Prior to using the app, 30 (60%) participants reported having “never heard of PrEP” and 44 (88%) considered themselves as “very unlikely or unlikely” at risk for HIV based on their current sexual behaviors. Analysis comparing scores before and immediately after using the app revealed a significant increase in PrEP content knowledge scores on a 7-item true or false scale. Total mean scores improved from 2.8 (SD 1.2) prior to using the app to 4.9 (SD 1.4) immediately after using the app (z=–6.04, P<.001). In item-by-item analysis, 4 of 7 items showed statistically significant improvements from before to immediately after app use (Table 2). After using the app, 13 (26%) participants reported they were “very likely” to start PrEP in the next 3 months, and 46 (92%) participants agreed with the statement “PrEP is a good option for HIV prevention for young women like me.”

In the immediate postsurvey, participants were asked whether they were interested in other PrEP modalities (Figure 2), including vaginal ring, implant, or injection, or PrEP combined with birth control. The most frequently endorsed mode was a dual method option with PrEP and birth control combined; 39 (78%) participants endorsed this option.

When participants were asked about interest levels in using technology as part of sexual health care, 22 (44%) indicated they were very interested, and 21 (42%) reported being somewhat interested. A total of 31 (63%) indicated they were very interested in using a mobile app for sexual health education, 10 (21%) indicated they were somewhat interested, and 8 (16%) indicated no interest. Just over half (n=26, 52%) of the sample reported being very interested in receiving sexual health reminders via email (n=29, 57%) or text message (n=25, 51%). Finally, only 17 (35%) participants reported being very interested in learning about sexual health topics via internet games.

Overall, participants considered the In the Loop app feasible and acceptable to use while waiting for a family planning visit. More specifically, 46 (92%) participants agreed that they would recommend In the Loop to friends to learn more about PrEP. Participants also reported that they liked using the app—rated 4.6 on a scale from 1 (strongly disagree) to 5 (strongly agree). Participants rated the overall quality of the app 4.3 on a scale from 1 (very poor) to 5 (very good); 40 (80%) participants agreed that the app was easy to use, and 48 (96%) agreed that they found the information in the app easy to understand. Finally, 40 (80%) agreed that they had enjoyed using the app while waiting for their family planning visit.

Participants were also asked to provide open-ended feedback on improvements that could be made to the app, which were summarized into four different themes to inform future refinement: increasing interactive components, refining app navigation, refining app aesthetics, and increasing sexual health and HIV prevention information (Table 3). Finally, participants responded to an open-ended question providing insight to their favorite parts of the app, including liking the fact that the main character is a young Black woman (“I liked that it was relatable. I liked that there was a character that looked like me” [participant ID, ie, PID 66]), that it did not require a lot of time to use (“It was quick. Wasn’t long. Wasn’t boring” [PID 15]), and that it was private or confidential (“Private on the iPad” [PID 50]).

Two functional bugs were identified in testing the app with users. The first bug caused a video to stall if the navigate back button was pushed; the second bug caused an incorrect follow-up prompt to be displayed. Both bugs, once identified, were remedied.

This study demonstrates that using an app in a family planning clinic waiting room is feasible and acceptable to young Black women. As family planning clinics are primary sources of health care for young Black women, integrating PrEP services into their practices is a critical component of comprehensive reproductive health care. Further, providing high-quality, confidential information via an mHealth app while waiting for clinic visits makes the most of a current missed opportunity. The In the Loop app is an innovative strategy to engage young Black women in HIV prevention.

In the Loop followed key tenets of developing effective mHealth interventions, including incorporating ease of modification and tailoring to the target population. Our team, in collaboration with young Black women, modified the content of an existing PrEP knowledge app and tailored it to be specific to the needs of the target population. Through user testing, we identified and addressed 2 bugs in the app. Finally, we gathered feedback on further refinements such as increasing the interactive components and refining the app navigation. The majority of participants were interested in using technology as part of sexual health care, with an app being the most popular mode of technology selected.

Cordova et al [38] designed and evaluated an mHealth intervention—Storytelling 4 Empowerment—to provide HIV, STI, and drug abuse prevention information to adolescents in primary care waiting rooms. In the acceptability pilot, participants reported approval of using down time while waiting for clinic visits; however, they also highlighted the need for privacy in shared spaces. Participants recommended privacy covers over tablets. In our pilot study, participants noted that the app felt private and did not raise concerns over privacy of participating in an mHealth app in a family planning clinic waiting room; nevertheless, it should be considered best practice to use privacy screens on tablets as recommended by Cordova et al [38].

In a systematic review, Chávez et al [39] highlighted the fact that apps tailored to adolescents’ specific race, ethnicity, and gender identity demonstrated stronger effects on health behavior compared to apps that were not tailored. Results from our community-engaged app development demonstrated high levels of app acceptability among the target population. Not only is the tailoring of mHealth important to success, but also the inclusion of the target population in mHealth development is critical for ensuring salience and acceptability.

In a 2020 systematic review of mHealth strategies to promote PrEP uptake, only 1 app was identified that was tailored specifically for young women; it was designed for those aged 11-14 years in Western Kenya [40,41]. No apps were identified that were tailored for young Black women in the United States. The results of the review highlight the need for mHealth strategies designed specifically to meet the needs of young Black women. We believe that our app will help to fill this gap.

Results of this pilot study must be interpreted in light of several limitations. First, the app was tailored exclusively for young Black women in a single urban setting, and thus our results cannot be generalized to other populations of young Black women. Further, the intervention was tailored for young Black women who have sex with men; further research is needed to tailor the intervention for young Black women who have sex with women or transgender and gender-diverse individuals. Second, the high positive rating of the app may be biased due to social desirability. We attempted to mitigate the effect of this bias by using computer-assisted questionnaires. Finally, although In the Loop was found to be acceptable and feasible, and to have an immediate impact on PrEP knowledge, future studies are needed to determine the sustained effect of the app on PrEP knowledge and to determine whether the app can improve PrEP uptake. Our results are limited, as they reflect prescores and immediate postscores without randomization; future studies should use randomization and assess durable effects of the intervention. Despite these limitations, this formative research may help guide future design and implementation of mHealth interventions in family planning settings, optimizing their chances for success.

Overall, our findings suggest that young Black women waiting for family planning visits found In the Loop to be feasible and acceptable. Further, this study demonstrates the value of engaging young Black women in the design process. As family planning clinics are primary sources of health care access for young women, they provide an ideal setting to integrate PrEP information and care into existing clinic practices. mHealth approaches can be scaled rapidly with fidelity. Future studies should evaluate the efficacy and durability of the In the Loop app in improving PrEP knowledge and uptake among PrEP-eligible young Black women.