From July 2019 to May 2020, we conducted a 6-month single-arm pilot study to evaluate the feasibility and acceptability of an mHealth app, named WYZ (pronounced “wise”), to address barriers to engagement in HIV care among individuals aged 18 to 29 years living with HIV in the San Francisco Bay Area [18]. WYZ was designed and developed using a human-centered design (HCD) approach [21-24]; formative research with youth and young adults living with HIV [18,20,25-27]; the information, motivation, and behavioral skills (IMB) [28-30] framework; and mHealth designers and developers from the University of California, San Francisco (UCSF) School of Medicine Technology team (SOM Tech). HCD focuses on creating approaches and delivering solutions to problems based on efforts to understand the specific needs and perspectives of the users. Therefore, HCD seeks to gain insights into the needs of the beneficiaries of an innovation, and creates approaches and delivers solutions to meet their needs.

Details of WYZ design and development, as well as the pilot study protocol, have previously been published [18]. In short, WYZ contains three main features, My Health, My Team, and My Community. My Health allows users to keep track of their ART medication information, visualize their adherence and laboratory data, and understand their health; My Team provides community resources and facilitates communication with health care team members; and My Community allows for social support from peers through anonymous and moderated discussion forums and allows users to stay up-to-date on health-related news. These features were developed with guidance from youth and young adults living with HIV and further refined through focus groups with youth and young adults living with HIV and iterative field testing with our Youth Advisory Panel (YAP), and were chosen to address specific barriers to ART adherence and engagement in HIV care (eg, social isolation and lack of community support).

WYZ design, development, and technological support were provided by UCSF’s SOM Tech. To ensure Health Insurance Portability and Accountability Act (HIPAA) compliance, we used Salesforce as the backend service and for storing sensitive data in a secure cloud-based database. Data about app usage were collected using Flurry (a mobile analytics tool) and Salesforce analytics. To enhance the security and privacy of WYZ, we used a two-step authentication process for downloading, password protection (with each log in), aliases, deletion of all communications over 30 days old, and remote revocation of app access in case of theft, loss, or misuse.

Participants were recruited using various strategies, including flyers at clinics and community-based organizations, emails to clinicians at clinics serving youth and young adults living with HIV, peer referral, and contacting prior study participants who had consented to being notified of future research. Information about the study was also disseminated through the YAP.

Individuals aged 18 to 29 years living with HIV, who lived or received medical care in the San Francisco Bay Area, spoke English, and had access to an Android or iOS smartphone, were included. Those with any evidence of severe cognitive impairment or active psychosis that impeded their ability to provide informed consent were excluded. To confirm an individual’s age and HIV serostatus, the potential participant text messaged a photo identification showing their date of birth and either a clinician’s letter of HIV diagnosis, a copy of laboratory test results (for HIV antibody or HIV viral load), or their ART medication vial. These photos were sent via text message to an encrypted and secure study phone for verification by study staff.

All study activities, including recruitment, screening, enrollment, study assessments, provision of incentives, and exit interviews, were conducted remotely using text message, telephone, email, and videoconference. Participants received a check-in at weeks 1, 2, and 4, followed by monthly check-ins, and up to US $215 for completion of all study activities. All procedures were reviewed and approved by the UCSF Institutional Review Board with a requirement for electronic consent. At baseline and 6 months, participants completed study assessments using a Qualtrics survey.

Descriptive statistics of the baseline demographics of WYZ study participants were calculated. Next, we examined descriptive statistics for feasibility metrics and compared them to predefined benchmarks. We then calculated frequencies for all of our acceptability metrics measured at the exit survey. Lastly, frequencies for HIV and psychosocial outcomes were calculated at baseline and the exit survey (6 months). For these data evaluated at both baseline and 6 months, we compared data from those who were retained in the study until 6 months and the entire group to examine divergent results. Given that this was a pilot study with limited statistical power, based on guidance from the National Institutes of Health and literature regarding wide confidence intervals and instability of effect sizes from pilot studies, tests of statistical significance and the efficacy of the intervention to compare HIV clinical outcomes preintervention and postintervention were not evaluated [41-44]. All analyses were completed using SAS 9.2 (SAS Institute).

Demographics of the study participants are presented in Table 1. At baseline, there were 79 participants (mean age 26.9 years, SD 2.9 years). Among the 79 participants, 69 (87%) identified as male, 48 (61%) identified as gay, 33 (43%) identified as Latino, and 16 (21%) identified as Black. Although nearly 57% (45/79) of participants were working, financial insecurity was relatively common, as 62% (49/79) of participants noted “barely getting by” or “not getting by” on the money they have.

Feasibility metrics are presented in Table 2. Of the 92 individuals who were screened, 84 were eligible. Of these 84 individuals, 79 (94%) consented to participate in the study, and 69 (87%) of those who enrolled completed the exit survey at 6 months. All predefined benchmarks were met (Table 2), except for mean minutes in the app per week (benchmark=15 min/week, actual=8.7 min/week). The mean number of logins per week was 5.3 (SD 5.6). In My Health, ART adherence tracking was conducted in 57.9% (5442/9393) of the inquiries. Moreover, the mean number of postings of chat topics on the My Community chat per person per week was 4.8 (range, 1-42), and the number of reported app crashes was less than once per week (0.24).

Acceptability metrics are presented in Table 3. Among the 69 participants who completed the study, 77% (n=53) rated their overall experience with the app as excellent to very good, 91% (n= 63) reported the app to be extremely to somewhat easy to download and install, and 96% (n=66) reported that WYZ setup was extremely to somewhat easy. All participants reported being extremely to somewhat comfortable with the security, privacy, and anonymity of WYZ. Moreover, approximately 83% (n=57) stated that they would be extremely to somewhat likely to continue to use WYZ and 94% (n=64) were extremely to somewhat likely to participate in a similar study in the future. Furthermore, 86% (n=59) of participants rated their overall experience with participation in the WYZ study as excellent to very good and 90% (n=62) reported excellent to very good experience with participating in a completely remotely conducted study. The mean SUS score was 75.6, which is considered to be well above average.

HIV and psychosocial metrics are presented in Table 4. At baseline and 6 months, 9% (7/79) and 4% (3/69) of participants, respectively, reported a detectable HIV viral load. During this time, self-reported ART adherence was unchanged. From baseline to 6 months, participants reporting mild depressive symptoms decreased by 9% (30/79, 38% to 19/66, 29%). Moreover, the mean social isolation score decreased by 12.1 points. Overall, we did not note divergent patterns with regard to the HIV and psychosocial metrics between baseline data from the entire sample (N=79) and those who were retained until 6 months (N=69).

The use of WYZ was highly feasible and acceptable among youth and young adults living with HIV in the San Francisco Bay Area. We met predefined benchmarks for recruitment, mean logins per week, tracking ART adherence, posting chat topics, and app crashes reported. The ease of app download, installation, and setup, and the overall comfort with security, privacy, and anonymity were highly rated. Additionally, participants reported high satisfaction for a research project that was remotely conducted. These findings demonstrate high potential for uptake and app functionality, indicating a promising role for WYZ as an intervention for engagement in HIV care and ART adherence among youth and young adults living with HIV.

Participants used the app for shorter timeframes than were predefined; however, our predefined benchmark may have been an overestimate. Additionally, due to the ability to log ART adherence using out-of-app notifications, some of the interactions with WYZ were not captured in the analytical tools used. In the next phase of this study, we will ask participants to further elaborate about app use during exit qualitative interviews.

Small changes in self-reported HIV and psychosocial metrics from baseline to 6 months highlight the limitations of pilot studies, in which examination of the intervention’s “preliminary impact” is not meaningful due to wide confidence intervals [41-44]. However, we noted improvements in the social isolation score, which, along with the high level of activity in the My Community Chat section, underscore the importance of this feature and deserve further evaluation in future research.

There are currently few mHealth apps in the early stages of development and pilot testing for enhanced engagement in HIV care, ART adherence, and communication with health care teams for people living with HIV [14-17]. The limitations of some of these mHealth apps include lack of specification of a theoretical framework, limited feasibility and acceptability metrics with no predefined benchmarks, small sample size (N<30), wide age range (≥18 years), and availability for either iOS or Android (not both). We have previously summarized these studies [18]. In developing and pilot testing WYZ, we have addressed these limitations.

In this pilot study, we were able to recruit a diverse group of participants with regard to race/ethnicity; however, participants were mainly gay cis-gender men. The other limitations of our study include a single-arm design (ie, no control group) and a relatively small convenience sample of participants from the San Francisco Bay Area who had access to a smartphone and most of who had an undetectable HIV viral load; therefore, study findings may not be generalizable to other populations. The loss to follow-up was approximately 13%, which is lower than estimates among youth and young adults living with HIV in the HIV Research Network (20%) [45] and in other studies in this population (up to 55%) [46]. We believe that the relatively low loss to follow-up may have been due to the fact that this research was conducted completely remotely, which allowed for flexibility for participation. Since the completion of this pilot study, we have resolved all minor bugs and smartphone compatibility challenges. Additionally, we are updating My Health for those who may use long-acting injectables in the near future and the My Team resources section based on user geolocation.

Youth and young adults living with HIV represent a population that is disproportionately impacted by HIV and requires tailored youth-friendly interventions. There is a dearth of technology-based interventions that address the changing needs of youth and young adults living with HIV. In future research, we will examine the efficacy and effectiveness of WYZ in improving engagement in HIV care and ART adherence among a larger sample of youth and young adults living with HIV taking into account findings from this study. Given the speed of technological advancement and the need for evidence-based solutions for improved HIV health outcomes among youth and young adults living with HIV, we believe that more funding should be allocated to technology-based interventions to move the National Institutes of Health’s Behavioral and Social Sciences Research agenda forward.