A Mobile Health App (WYZ) for Engagement in Care and Antiretroviral Therapy Adherence Among Youth and Young Adults Living With HIV: Single-Arm Pilot Intervention Study

Background Youth are globally recognized as being vulnerable to HIV. Younger age has been correlated with worse health outcomes. Mobile health (mHealth) interventions have the potential to interact with youth where they are, using a device they already access. Objective Using predefined benchmarks, we sought to evaluate the feasibility and acceptability of WYZ, an mHealth app, for improved engagement in care and antiretroviral therapy (ART) adherence among youth and young adults living with HIV. WYZ was designed and developed with input from youth and young adults living with HIV using a human-centered design approach and was based on the information, motivation, and behavioral skills framework to address common barriers to care and ART adherence among youth and young adults living with HIV. Methods We recruited youth and young adults living with HIV (18-29 years old) from the San Francisco Bay Area to take part in a 6-month pilot trial. Their participation included completing baseline and exit surveys, and participating in seven phone check-ins about their use of WYZ. Results Youth and young adults living with HIV (N=79) reported high levels of feasibility and acceptability with WYZ use. We met predefined benchmarks for recruitment (79/84, 94%), mean logins per week (5.3), tracking ART adherence (5442/9393, 57.9%), posting chat topics per week (4.8), and app crashes reported per week (0.24). The ease of app download, install, and setup, and comfort with security, privacy, and anonymity were highly rated (all over 91%). Additionally, participants reported high satisfaction for a research project that was remotely conducted. Participants used the app for shorter timeframes compared to the predefined benchmark. Conclusions We noted high feasibility and acceptability with WYZ. Further research to examine the efficacy of WYZ will enable youth and young adults living with HIV and their providers to make informed decisions when using, recommending, and prescribing the app for improved engagement in HIV care and ART adherence. Trial Registration ClinicalTrials.gov NCT03587857; https://clinicaltrials.gov/ct2/show/NCT03587857


Introduction
In the United States, youth and young adults carry a significant burden of HIV. Youth and young adults living with HIV experience disparities at all steps of the HIV care continuum, including higher HIV incidence, lower linkage and retention in care, suboptimal antiretroviral therapy (ART) adherence, and lower virologic suppression [1][2][3][4][5][6]. The consequences of continued disparities include poor health outcomes, development and transmission of drug-resistant viruses, a future generation of adults who are more susceptible to developing AIDS, and further widening of these health disparities. Youth and young adults living with HIV experience many individual, structural (eg, transition to adult health care, inexperience with medical systems, and lack of insurance), social (eg, poverty, unstable housing, food insecurity, social isolation, and stigma), and biological (cognitive developmental stages) challenges that impact their abilities to access and adhere to oral ART [2,3,[7][8][9]. However, there are few effective and tailored interventions that address ART adherence and engagement in HIV care for youth and young adults living with HIV.
In the United States, over 96% of youth and young adults living with HIV own smartphones [10], over two-thirds have downloaded mobile health (mHealth) apps [11], and over 90% are social media users [12]. The nearly ubiquitous access to and use of smartphones represents a powerful platform for the delivery of mHealth interventions to this population. Additionally, given the reduction in transportation costs, time constraints, potential stigma associated with participation in in-person HIV research [13], and missing data, mHealth technology can surmount common barriers, increasing the reach and generalizability of findings. Several mHealth apps are in various stages of development for people living with HIV [14][15][16][17], as we have previously summarized [18]. However, despite technology-based behavioral interventions showing promise in older adults living with HIV [19], few interventions have shown efficacy in addressing the unique aspects of youth developmental phases, youth culture, and gravitation of youth toward the use of technology [20]. In this study, we pilot tested an mHealth app to address barriers to engagement in care among youth and young adults living with HIV.

Study Design and Sample
From July 2019 to May 2020, we conducted a 6-month single-arm pilot study to evaluate the feasibility and acceptability of an mHealth app, named WYZ (pronounced "wise"), to address barriers to engagement in HIV care among individuals aged 18 to 29 years living with HIV in the San Francisco Bay Area [18]. WYZ was designed and developed using a human-centered design (HCD) approach [21][22][23][24]; formative research with youth and young adults living with HIV [18,20,[25][26][27]; the information, motivation, and behavioral skills (IMB) [28][29][30] framework; and mHealth designers and developers from the University of California, San Francisco (UCSF) School of Medicine Technology team (SOM Tech). HCD focuses on creating approaches and delivering solutions to problems based on efforts to understand the specific needs and perspectives of the users. Therefore, HCD seeks to gain insights into the needs of the beneficiaries of an innovation, and creates approaches and delivers solutions to meet their needs.
Details of WYZ design and development, as well as the pilot study protocol, have previously been published [18]. In short, WYZ contains three main features, My Health, My Team, and My Community. My Health allows users to keep track of their ART medication information, visualize their adherence and laboratory data, and understand their health; My Team provides community resources and facilitates communication with health care team members; and My Community allows for social support from peers through anonymous and moderated discussion forums and allows users to stay up-to-date on health-related news. These features were developed with guidance from youth and young adults living with HIV and further refined through focus groups with youth and young adults living with HIV and iterative field testing with our Youth Advisory Panel (YAP), and were chosen to address specific barriers to ART adherence and engagement in HIV care (eg, social isolation and lack of community support).
WYZ design, development, and technological support were provided by UCSF's SOM Tech. To ensure Health Insurance Portability and Accountability Act (HIPAA) compliance, we used Salesforce as the backend service and for storing sensitive data in a secure cloud-based database. Data about app usage were collected using Flurry (a mobile analytics tool) and Salesforce analytics. To enhance the security and privacy of WYZ, we used a two-step authentication process for downloading, password protection (with each log in), aliases, deletion of all communications over 30 days old, and remote revocation of app access in case of theft, loss, or misuse.
Participants were recruited using various strategies, including flyers at clinics and community-based organizations, emails to clinicians at clinics serving youth and young adults living with HIV, peer referral, and contacting prior study participants who had consented to being notified of future research. Information about the study was also disseminated through the YAP.
Individuals aged 18 to 29 years living with HIV, who lived or received medical care in the San Francisco Bay Area, spoke English, and had access to an Android or iOS smartphone, were included. Those with any evidence of severe cognitive impairment or active psychosis that impeded their ability to provide informed consent were excluded. To confirm an individual's age and HIV serostatus, the potential participant text messaged a photo identification showing their date of birth and either a clinician's letter of HIV diagnosis, a copy of laboratory test results (for HIV antibody or HIV viral load), or their ART medication vial. These photos were sent via text message to an encrypted and secure study phone for verification by study staff.
All study activities, including recruitment, screening, enrollment, study assessments, provision of incentives, and exit interviews, were conducted remotely using text message, telephone, email, and videoconference. Participants received a check-in at weeks 1, 2, and 4, followed by monthly check-ins, and up to US $215 for completion of all study activities. All procedures were reviewed and approved by the UCSF Institutional Review Board with a requirement for electronic consent. At baseline and 6 months, participants completed study assessments using a Qualtrics survey.

Demographics
Demographic data, including date of birth, sex at birth, sexual identity, race/ethnicity, perceived financial security, and work status (full time, part time, or not working), were collected.

Feasibility Metrics
Feasibility metrics were collected using Flurry and Salesforce analytics. Metrics were based on predefined thresholds [18], including how many people were recruited for the study, mean logins to the app, mean minutes in the app, and use of specific features in the app.

Acceptability Metrics
Acceptability metrics were collected using a Qualtrics survey administered during the last study visit at 6 months. The survey included questions related to satisfaction with WYZ, ease of WYZ use, and satisfaction with the study. Additionally, we asked participants about WYZ acceptability using the System Usability Scale (SUS), with scores ranging from 0 to 100 and scores >68 being considered above average [31,32]. A threshold of 70% or greater satisfaction on all questions was used to determine acceptability.

Data Analysis
Descriptive statistics of the baseline demographics of WYZ study participants were calculated. Next, we examined descriptive statistics for feasibility metrics and compared them to predefined benchmarks. We then calculated frequencies for all of our acceptability metrics measured at the exit survey. Lastly, frequencies for HIV and psychosocial outcomes were calculated at baseline and the exit survey (6 months). For these data evaluated at both baseline and 6 months, we compared data from those who were retained in the study until 6 months and the entire group to examine divergent results. Given that this was a pilot study with limited statistical power, based on guidance from the National Institutes of Health and literature regarding wide confidence intervals and instability of effect sizes from pilot studies, tests of statistical significance and the efficacy of the intervention to compare HIV clinical outcomes preintervention and postintervention were not evaluated [41][42][43][44]. All analyses were completed using SAS 9.2 (SAS Institute).
Feasibility metrics are presented in Table 2. Of the 92 individuals who were screened, 84 were eligible. Of these 84 individuals, 79 (94%) consented to participate in the study, and 69 (87%) of those who enrolled completed the exit survey at 6 months. All predefined benchmarks were met (Table 2), except for mean minutes in the app per week (benchmark=15 min/week, actual=8.7 min/week). The mean number of logins per week was 5.3 (SD 5.6). In My Health, ART adherence tracking was conducted in 57.9% (5442/9393) of the inquiries. Moreover, the mean number of postings of chat topics on the My Community chat per person per week was 4.8 (range, , and the number of reported app crashes was less than once per week (0.24).  Acceptability metrics are presented in Table 3. Among the 69 participants who completed the study, 77% (n=53) rated their overall experience with the app as excellent to very good, 91% (n= 63) reported the app to be extremely to somewhat easy to download and install, and 96% (n=66) reported that WYZ setup was extremely to somewhat easy. All participants reported being extremely to somewhat comfortable with the security, privacy, and anonymity of WYZ. Moreover, approximately 83% (n=57) stated that they would be extremely to somewhat likely to continue to use WYZ and 94% (n=64) were extremely to somewhat likely to participate in a similar study in the future. Furthermore, 86% (n=59) of participants rated their overall experience with participation in the WYZ study as excellent to very good and 90% (n=62) reported excellent to very good experience with participating in a completely remotely conducted study. The mean SUS score was 75.6, which is considered to be well above average.
HIV and psychosocial metrics are presented in Table 4. At baseline and 6 months, 9% (7/79) and 4% (3/69) of participants, respectively, reported a detectable HIV viral load. During this time, self-reported ART adherence was unchanged. From baseline to 6 months, participants reporting mild depressive symptoms decreased by 9% (30/79, 38% to 19/66, 29%). Moreover, the mean social isolation score decreased by 12.1 points. Overall, we did not note divergent patterns with regard to the HIV and psychosocial metrics between baseline data from the entire sample (N=79) and those who were retained until 6 months (N=69).

Principal Findings
The use of WYZ was highly feasible and acceptable among youth and young adults living with HIV in the San Francisco Bay Area. We met predefined benchmarks for recruitment, mean logins per week, tracking ART adherence, posting chat topics, and app crashes reported. The ease of app download, installation, and setup, and the overall comfort with security, privacy, and anonymity were highly rated. Additionally, participants reported high satisfaction for a research project that was remotely conducted. These findings demonstrate high potential for uptake and app functionality, indicating a promising role for WYZ as an intervention for engagement in HIV care and ART adherence among youth and young adults living with HIV.
Participants used the app for shorter timeframes than were predefined; however, our predefined benchmark may have been an overestimate. Additionally, due to the ability to log ART adherence using out-of-app notifications, some of the interactions with WYZ were not captured in the analytical tools used. In the next phase of this study, we will ask participants to further elaborate about app use during exit qualitative interviews.
Small changes in self-reported HIV and psychosocial metrics from baseline to 6 months highlight the limitations of pilot studies, in which examination of the intervention's "preliminary impact" is not meaningful due to wide confidence intervals [41][42][43][44]. However, we noted improvements in the social isolation score, which, along with the high level of activity in the My Community Chat section, underscore the importance of this feature and deserve further evaluation in future research.
There are currently few mHealth apps in the early stages of development and pilot testing for enhanced engagement in HIV care, ART adherence, and communication with health care teams for people living with HIV [14][15][16][17]. The limitations of some of these mHealth apps include lack of specification of a theoretical framework, limited feasibility and acceptability metrics with no predefined benchmarks, small sample size (N<30), wide age range (≥18 years), and availability for either iOS or Android (not both). We have previously summarized these studies [18]. In developing and pilot testing WYZ, we have addressed these limitations.
In this pilot study, we were able to recruit a diverse group of participants with regard to race/ethnicity; however, participants were mainly gay cis-gender men. The other limitations of our study include a single-arm design (ie, no control group) and a relatively small convenience sample of participants from the San Francisco Bay Area who had access to a smartphone and most of who had an undetectable HIV viral load; therefore, study findings may not be generalizable to other populations. The loss to follow-up was approximately 13%, which is lower than estimates among youth and young adults living with HIV in the HIV Research Network (20%) [45] and in other studies in this population (up to 55%) [46]. We believe that the relatively low loss to follow-up may have been due to the fact that this research was conducted completely remotely, which allowed for flexibility for participation. Since the completion of this pilot study, we have resolved all minor bugs and smartphone compatibility challenges. Additionally, we are updating My Health for those who may use long-acting injectables in the near future and the My Team resources section based on user geolocation.

Conclusion
Youth and young adults living with HIV represent a population that is disproportionately impacted by HIV and requires tailored youth-friendly interventions. There is a dearth of technology-based interventions that address the changing needs of youth and young adults living with HIV. In future research, we will examine the efficacy and effectiveness of WYZ in improving engagement in HIV care and ART adherence among a larger sample of youth and young adults living with HIV taking into account findings from this study. Given the speed of technological advancement and the need for evidence-based solutions for improved HIV health outcomes among youth and young adults living with HIV, we believe that more funding should be allocated to technology-based interventions to move the National Institutes of Health's Behavioral and Social Sciences Research agenda forward.