The Innovative Tools to Expand HIVST collaboration between the Nigerian Institute of Medical Research, Saint Louis University, New York University, and the University of North Carolina at Chapel Hill, known locally as 4 Youth By Youth (4YBY), engaged a team of developers at the Co-Creation Hub to develop the 4YBY photo verification mHealth app. The 4YBY photo verification mHealth app aims to promote HIVST by guiding users (youth) through the proper use of the HIVST kit and linking them to care and anonymous counseling. The mobile app was built with React Native to support the Android operating system from version 4.4 (Kitkat) upward. The development and design process were based on the Human-Centered Design framework. The design team worked together to solve 2 main challenges in creating the app: (1) how might we connect young people to health care and counseling when self-testing for HIV? (2) How can data on HIVST be collected and analyzed?

First, interviews were held with project key stakeholders, including potential users, with the goal to understand the facilitators and constraints or major concerns with HIVST. Second, there were immersion and shadowing activities with potential users in which designers sought to understand and gain real-world experience with HIVST to understand end users’ perception and preferences of HIVST. The goal of this process is to identify and provide value to young people who will use the product. Third, user interface prototyping and content creation was done. The design team worked with researchers from the Nigerian Institute of Medical Research to create content for the different sections of the mobile app. Given that the vast majority of people perceive images faster than words, we used illustrations supported by text to communicate important concepts throughout the design process.

Following the design phase, the front end and back end of the app were developed. A front-end section of the app, consisting of a visual representation of the app and a back-end section, also known as the server side of an app, including communication between the database and the app, were developed. After the development of the first version of the app, alpha and beta testing was performed. α testing was performed to identify bugs before releasing the app to end users, whereas beta testing was performed by potential end users of the app (young people aged 14-24 years) to determine the usability of the app.

The key features of the app include the following:

Other features were as follows:

A qualitative research design approach was used to evaluate the usability of the 4YBY mobile phone app by answering the following questions: To what extent do users achieve the desired goal of using the app (effectiveness)? Does the effort put into the use of the app allow the performance of a task in a timely manner (efficiency)? What is the perceived user’s experience of the app? Participants used a think-aloud approach while performing usability tasks in three rounds of usability testing and subsequently completed semistructured one-on-one interviews.

Participants were recruited through advertisements and word of mouth at university campuses and youth centers in Yaba, Lagos, Nigeria. Participants’ recruitment for the study was guided by previous mHealth app studies that showed that preliminary usability of apps can be detected by a sample of 5 to 10 individuals [18,19]. Inclusion criteria included being aged between 14 and 24 years, fluency in English, the ability to read, owning an Android electronic device, and the ability to provide informed consent for the study. Data collection was performed by researchers who were part of the study team with requisite training in good clinical practice and ethical conduct of research.

During each of the beta tests (Multimedia Appendix 1), participants performed HIVST using the OraQuick by Orasure testing kit. Tests were conducted by each participant swabbing their gum and then placing it in a processing fluid and waiting for 20 minutes for the result. Each participant had a prototype of the 4YBY app installed on their Android app and subsequently carried out the HIVST by following the directive provided by the app. The app also engaged the participants with basic information on HIV during the waiting time before the test completion, and they also participated in a HIV risk self-assessment test.

Following informed consent, participants were involved in usability testing via the think-aloud approach [20]. The think-aloud approach required users to continuously verbalize their thoughts about their underlying thinking behind their interactions while using the mobile app by freely expressing what they were doing and why and stating when they encountered any problem in the process. This provided an opportunity for participants to test the app’s functionalities and provide opinions, comments, and concerns to improve on the app. They also discussed how they felt about the use of the app, which was audiotaped.

Immediately following usability testing, participants were asked a series of questions to assess their experience with the usability of the app (Multimedia Appendix 2). The objective of the phase was to obtain the participants’ immediate interpretation of the app design and functionality and facilitate the elaboration of usability issues and increasing insight and design suggestions [21]. A total of 3 beta tests using this methodology were performed over 3 months. After each of the beta tests, the design team used the user experience feedback to improve the development of the app by correcting all glitches noted and modifying as necessary to meet users.

The demographic characteristics of the users were reported as descriptive statistics. Each interview was audio recorded, subsequently transcribed verbatim, and manually coded. Field notes were also included in the analysis. Inductive thematic analysis, which describes the process of coding data without trying to fit the data into a pre-existing coding frame or the researcher’s analytic preconceptions, was used for this study. This process ensured that thematic analysis was completely data driven [22]. The Braun and Clark [22,23] guidelines were used by 2 research team members with experience in qualitative research, which included the following steps: (1) repeated reading of the transcripts to become familiar with the data; (2) generating initial codes relevant to the research questions (effectiveness, efficiency, and usefulness); (3) organizing the codes; (4) arranging the subthemes into overarching themes; and (5) defining and naming themes. Disagreements around themes were resolved by an in-person research team discussion until consensus was reached, and a final theme was agreed upon.

Ethical approval for the Innovative Tools to Expand HIVST study was obtained from the institutional review board of the Nigerian Institute of Medical Research and Saint Louis University. Informed consent was obtained from each participant before the commencement of the usability beta testing study.

A total of 19 youth participated in the usability analysis of the mHealth app for HIVST (Figure 4). The mean age of the participants was 22 years (SD 2.4), and the age range was 14-25 years. There were 12 females and 7 males and 68% (13/19) of the participants were students. The majority (18/19, 95%) of the participants completed all the usability tasks on the first attempt. On average, the usability test lasted between 25 and 30 minutes. All the participants successfully uploaded a photo of their test results using the app. Figure 4 illustrates participants’ progression through each of the study phases.

Thematic analysis of the user comments from the think-aloud interviews from the 3 beta usability tests yielded findings on how the app was used, including the relation to test outcomes (ie, positive, negative, and invalid) and strengths and weaknesses of the prototype app and suggested improvements. The following four themes were identified (Table 1): (1) the design of the app, (2) the ease of use of the app, (3) user privacy reinforcement, and (4) linkage to care.

Participants frequently commented on the need for the app to have a more user-friendly design, with interactive images, an appealing color scheme, and features that are easily accessible. The participants involved in the first beta testing felt that a mobile app intended for use by youth should be colorful and attractive to ensure adequate coverage and acceptability among this age group. They further explained that the likelihood of a young person downloading the app from the Google Play store depends largely on how they perceived the app:

During the second and third beta tests, participants also wanted an improvement in the appearance of the logo of the app and its background theme. Other concerns raised during the second beta test are the need to improve the resolution of the video that describes the process of HIVST in the app:

A participant in the third beta test also expressed a preference that the phrases used in the app should be the type that will easily catch the attention of Nigerian youths who are the end users of the app. He would prefer developers to use some locally trending phrases on social media:

During the first beta test, some participants noted that it would be desirable to have the app work in the background while he could be engaged in the use of the mobile phone for other things. The feature was corrected before the third beta testing, and participants at the third beta testing were impressed with this feature:

The ease of use of the app assesses the user experience of the app without direct guidance from the developers. Participants in each of the beta tests overall found the app easy to use and did not report that anything went wrong during the use of the app. Some of the other observations made on the ease of use include the need to adjust the app directive on how to set up the test and if the app could be used offline without internet access:

HIV stigma is a key pertinent issue with HIV testing. To ensure that all users can test anonymously, aliases, rather than real names, were used. The developer also implemented an auto-logout feature for further security. However, in think-aloud exercises, one of the major concerns shared among participants was the issue of privacy. The participants would not want their personal information to be shared in any form, and the app ensured anonymity to address this. The participants during the 3 phases of the beta test noted that the app further reinforced the practice of privacy in getting tested for HIV:

One of the features of the 4YBY photo verification app is that the participants can be linked to care after conducting an HIV test. This is usually done by the user calling a counselor to discuss the HIV test result or requesting a call from a counselor by clicking on a radio button in the app. The users were pleased that this feature was included in the app, and they would either call or request a call to be linked to care after performing the HIV self-test:

We developed a photo verification mHealth app to promote HIVST among young people in Nigeria. To our knowledge, this is one of the first mHealth apps that has been developed to promote HIVST and linkage to care in sub-Saharan Africa [24]. Our usability findings add evidence to the limited data on young people’s experiences and perspectives on mHealth apps in sub-Saharan Africa. We identified 4 main themes (eg, app design, ease of use, user privacy reinforcement, and linkage to care) that may influence the adoption and continued use of the apps for HIVST.

This study corroborates the findings of limited studies in sub-Saharan Africa, including how mHealth apps for HIVST may overcome the experience of stigma and structural barriers of clinic-based testing by providing privacy and convenience [25]. Even in middle- to high-income countries, with a substantial body of research on mHealth apps and HIVST, findings suggest that the design of these apps, along with their ease of use, privacy, and linkage to services, provides an opportunity to disseminate information about the accuracy, safety, and appropriate use of this technology to promote HIVST [26].

Our iterative cocreation process led to the development of an mHealth HIVST app suited to the needs of young people in Nigeria. In addition, the iterative process and feedback from beta testing led to the modification of the app to address issues raised during the development phase, including the desire to have an app with minimal internet data demand, features to encourage users’ anonymity, and step-by-step video description of the process of conducting the HIVST. Previous studies on mHealth app development have documented the importance of involving end users in the development of mHealth apps [27]. Through their own experiences and by showing participants how to use our app, the majority found the app to be informative, accessible, and engaging. Think-aloud activities identified not only facilitating features that may potentially enhance uptake but also factors that may hinder its use.

Consistent with other mHealth app literature [28,29], one primary recommendation was for the app to reinforce privacy concerns. This is particularly salient for young people, given their limited access to youth-friendly health centers and barriers associated with navigating and appearing at clinics for HIV prevention services that mostly cater to adults [30,31]. Protecting the privacy of young people accessing the app will not only maintain youth support but also enhance trust. Providing linkage to additional services via the app also provides an opportunity to further enhance youth engagement with HIV prevention, treatment, and care.

Participants noted that the app’s linkage platform can enable HIV services to reach young people at risk for or living with HIV, who may not otherwise use health care services. In addition, to the linkage to services, participants reported that the use of multimedia text, short videos, and graphics within the app may not only help youth to remain HIV-uninfected but also enhance their ability to stay healthy and thrive if they are already living with HIV. Overall, these findings provide insight into young people’s concerns and strategies to enhance the uptake of our mHealth app for HIVST promotion.

This study had some limitations. First, this was a small sample of youth participants recruited from Lagos, Nigeria, thereby making it difficult to reach generalizations from this data source. Second, access to smartphones and mHealth apps may have influenced social desirability bias with the decision to participate in the study. In addition, the flexibility of thematic analysis can lead to inconsistency and a lack of coherence with theme development. To address authors’ bias, the coding of the transcript was independently performed by 2 researchers before theme development. Finally, given that this is a usability study, it is not possible to conclude the potential efficacy of the app in promoting HIVST uptake among young people in Nigeria.

The results of this study have implications for interventions targeting HIVST uptake among young people in Nigeria. It also has important implications for using technology to disseminate HIV prevention information to youth populations in Nigeria. The 2019 Nigerian operational guidelines for HIVST seek to increase individuals’ autonomy, decentralize services, and create demand for HIV testing, especially for underserved populations, including young people. Given the new operational guidelines that seek to decentralize HIV testing, verification of self-reported outcomes for HIVST is of utmost importance. Bearing in mind that there are no available photo verification modalities for HIVST for young people in Nigeria, this study provides evidence on the usability of the 4YBY mHealth photo verification app and its potential to promote uptake of HIV testing among young people. These findings lay the foundation for future studies to assess the feasibility, reach, and effectiveness of the mHealth app to promote the uptake of HIV testing and other preventive services among young people in Nigeria. Through this formative work, we also obtained contextual recommendations from the participants to improve the design of the app and reinforce privacy to enhance acceptability among young people in Nigeria. The feedback has practical implications for mHealth technologies to seek end users’ feedback to ensure acceptability and appropriateness among target populations. The 4YBY mHealth app may provide a model for supporting young people to access other sexual and reproductive health services, given the app’s built-in function to link users to available youth-friendly health facilities. The findings have implications for practice in Nigeria and can inform the integration of mHealth technologies within youth-friendly health facilities to promote the uptake of preventive sexual health services.

This study demonstrated the usability of an mHealth app to promote HIVST uptake among young people in Nigeria. Findings from this study illustrated that the design and ease of use of the app, along with privacy and linkage to additional services, may enhance the reach and acceptability of HIVST services among young people who are traditionally underserved in traditional HIV research conducted not only in Nigeria but also in sub-Saharan Africa. Future clinical trials are needed to determine the efficacy of using an mHealth app to verify HIVST results among young people.