Background: Mobile health (mHealth) interventions among adolescents and young adults (AYAs) are increasingly available in African low- and middle-income countries (LMICs). For example, the unstructured supplementary service data (USSD) could be used to verify HIV self-testing (HIVST) among AYAs with poor bandwidth.
Objective: The aim of this study is to describe the creation of an USSD platform and determine its feasibility and usability to promote the verification of HIVST results among AYAs in Nigeria.
Methods: We developed and evaluated a USSD platform to verify HIVST results using a user-centered approach. The USSD platform guided AYAs in performing HIVST, interpreting the result, and providing linkage to care after the test. Following the usability assessment, the USSD platform was piloted. We used a mixed methods study to assess the platform’s usability through a process of quantitative heuristic assessment, a qualitative think-aloud method, and an exit interview. Descriptive statistics of quantitative data and inductive thematic analysis of qualitative variables were organized.
Results: A total of 19 AYAs participated in the usability test, with a median age of 19 (IQR 16-23) years. There were 11 females, 8 males, and 0 nonbinary individuals. All individuals were out-of-school AYAs. Seven of the 10 Nielsen usability heuristics assessed yielded positive results. The participants found the USSD platform easy to use, preferred the simplicity of the system, felt no need for a major improvement in the design of the platform, and were happy the system provided linkage to care following the interpretation of the HIVST results. The pilot field test of the platform enrolled 164 out-of-school AYAs, mostly young girls and women (101, 61.6%). The mean age was 17.5 (SD 3.18) years, and 92.1% (151/164) of the participants reported that they were heterosexual, while 7.9% (13/164) reported that they were gay. All the participants in the pilot study were able to conduct HIVST, interpret their results, and use the linkage to care feature of the USSD platform without any challenge. A total of 7.9% (13/164) of the AYAs had positive HIV results (reactive to the OraQuick kit).
Conclusions: This study demonstrated the usability and feasibility of using a USSD system as an alternative to mobile phone apps to verify HIVST results among Nigerian youth without smartphone access. Therefore, the use of a USSD platform has implications for the verification of HIVST in areas with low internet bandwidth. Further pragmatic trials are needed to scale up this approach.
Mobile health (mHealth) interventions are increasingly available in African low- and middle-income countries (LMICs) . Some of its uses were documented in the promotion of maternal health, antenatal clinic attendance, HIV testing [ , ], and sexual and reproductive health (SRH) among adolescents and young adults (AYAs) in Africa [ ]. The increase in its use is partly because AYAs may have more education, better bandwidth, and better access to cell phones compared to 10 years ago [ ]. The use of mHealth in this population has increased the access to SRH services for AYAs, reduced the incidence of unsafe abortions, promoted contraception, and enhanced HIV self-testing (HIVST) [ - ]. And it is increasingly becoming the medium for addressing the SRH challenge among AYAs, some of which include the high prevalence of sexually transmitted infections (STIs) and HIV [ - ]. mHealth intervention has become common in Nigeria because the high HIV burden among AYAs requires innovative youth-centered implementation strategies. For example, implementation strategies to promote HIV self-testing have significantly improved access to HIVST services in Nigeria [ - ].
A major challenge with the scale-up of the HIVST services among AYAs in African LMICs is how to verify the results for individuals who self-test . A recent systematic review documented various means of verifying HIVST results, including provision of supervision by health care workers, return of used test kits, and electronic transmission of photographs [ ]. It then follows that the use of mobile phones for the verification of HIVST results, a form of electronic transmission, remains a feasible and acceptable intervention in this population.
However, a significant proportion of Nigerian AYAs have analog phones and low bandwidth . Therefore, mHealth strategies are needed to promote HIVST among AYAs without smartphone access. The unstructured supplementary service data (USSD) platform represents a viable alternative to verify HIVST uptake in this population. The USSD is a protocol used to send text messages, similar to SMS. USSD uses character codes available on the mobile phone and creates a real time communication session between the phone and another device, usually a network or server. With USSD, users can interact directly from their mobile phone by selecting from various menus [ ]. Unlike an SMS, during a USSD session, a USSD message establishes a connection in real time. This means that USSD allows 2-way information communication as long as the communication line remains open. It could provide step-by-step instructions to execute a task and collect some basic information stored on the backend [ ].
The existing mHealth literature focuses on high-income countries and centers on the perspective of people with higher electronic device bandwidth . Additionally, the USSD platform has been used in Africa to address some SRH issues, but to the best of our knowledge, no studies have used USSD to verify HIVST results, thereby promoting its uptake. There is a need to deliver USSD intervention to verify HIVST results among AYAs who use analog phones with limited bandwidth.
This study, therefore, described the development of a USSD platform and determined the feasibility and usability of this platform to verify HIVST results among AYAs in Nigeria.
We designed this study as a parallel mixed methods investigation.
Development of the Unstructured Supplementary Service Data Platform
The innovative tools to expand HIVST (I-TEST) project is a National Institutes of Health (NIH)–sponsored research collaboration between the Nigerian Institute of Medical Research (NIMR), Saint Louis University, New York University, and the University of North Carolina at Chapel Hill . To meet the challenge of using mHealth to promote HIVST among young people in Nigeria who do not have access to smart mobile phones, we engaged a team of developers to develop a USSD platform with the aim of verifying HIVST uptake by guiding users (youths) through the proper use of the HIVST kit and linking them to care and anonymous counseling.
First, the participant enters a numeric code (eg, *347*4929#) on their mobile phone. The network provider then validates the code, obtained from Africa’s Talking Ltd, a Kenya-based communication company that organizes USSD code service for a fee. After validation of the code, the process is repeated in reverse after contact with the third-party web-based app, with the response going back to the USSD gateway, which displays the response’s content on the screen of the user’s mobile phone ().
The USSD code requires the user to enter key information required for the research, like a unique ID, the current state of residence, the current local government area of residence, age, sexual orientation, gender, and finally, the number of lines that appear on the test kit after the HIVST process has been completed. The USSD platform also provided an opportunity for posttest counseling by collecting information on participants’ mobile numbers ().
On completing the test, the participant is sent an SMS notifying him or her of the test result. The information collected from the participant is saved into a database. Also, an administrative portal at the backend helps manage all records of participants involved in the research (). The records on this admin portal can be exported.
This study took place in Lagos State, Nigeria, which is a state located in the South-West geopolitical zone of Nigeria with an estimated 24.6 million inhabitants as of 2015. Lagos State is a major financial center in Nigeria. It has 20 local government areas and 37 local council development areas.
A mixed methods approach was used for the usability study to provide contextualized insights into the qualitative data and a quantitative heuristic evaluation of the user’s experience to provide a concurrent assessment of the usability of the USSD platform to verify HIVST results.
A Think-Aloud and an Exit Interview
During the user testing phase, a qualitative evaluation of user experience was done using the think-aloud method, and an exit interview was done for participants to document further their experience of the USSD platform for the HIVST.
Through the think-aloud method, participants performed HIVST, verbalized their experience (thinking aloud), and said whatever came into their minds when completing the task [, ]. After completing the task, they were invited to participate in an exit interview with the research assistants for the study to document their experience further.
Study population and participants’ recruitment: the user testing was conducted among out-of-school adolescents and AYAs in Lagos State, Nigeria. The I-TEST research team from NIMR worked with adolescent peer educators working with an indigenous nongovernmental organization in Lagos to recruit the study participants. A total of 19 AYAs took part in the user testing phase. Eligibility and inclusion criteria included being aged 14-24 years old, owning a mobile phone, and being able to read. AYAs who do not have mobile phones, are currently enrolled in an educational institution, and cannot read and write were excluded from this study. AYAs were provided with oral HIV self-test kits (Orasure) and performed HIVST using USSD guidance. Oral HIVST has been shown to be effective in earlier studies [, ].
Usability Heuristic Evaluation
Heuristic evaluation is the process of evaluating a mobile technology, app, or system to determine issues with its use and discover flaws. It is traditionally carried out by experts who can quickly and efficiently evaluate systems. A major setback with the assessment method is that usability experts are usually not the system’s end users. Hence, the heuristic evaluation mostly ignores the end users’ point of view on the system or app. Previous research has underlined the importance of obtaining end user feedback in the process [, ]. In this study, we adapted Nelson’s usability heuristics to understand the usability of the USSD platform. This included how efficiently users can navigate the USSD system, perform an HIVST kit, interpret their results, and access posttest referral services. The advantage of the heuristic evaluation of the user’s experience lies in its ability to efficiently identify usability issues in an inexpensive way as multiple evaluations are done independently at the same time [ ]. Specifically, users answered 21 questions through a simplified dichotomous scale that addresses various aspects of Nelson’s 10-item heuristics.
Data Collection for Usability Testing
Participants for the usability test were invited to an easily accessible place in the community where informed consent was obtained before the testing. The think-aloud approach  was used for the usability testing, which required users to continuously talk about their thoughts as they interact with the USSD platform by freely expressing what they were doing and why and stating when they encountered any problems in the process. This process was followed by the administration of the heuristic questionnaires, which the interviewer administered through an exit interview. An audio recording of the exit interview was done for further analysis.
The pilot study of the USSD platform in verifying HIVST was done after the usability assessment to generate preliminary real-life effectiveness data on using the platform before deployment for large-scale use. During this process, we evaluated data collected from participants who used the USSD platform in an informal settlement in Lagos, Nigeria.
Study Population and Participants Recruitment
The pilot testing was conducted among out-of-school and in-school AYAs in Lagos State, Nigeria.
The I-TEST research team from NIMR worked with an indigenous nongovernmental organization in Lagos, with existing collaboration with community mobilizers with previous experience in SRH research among adolescents in the community. A total of 164 AYAs were recruited during the pilot phase from the urban inner-city of Yaba in Lagos State. Inclusion criteria were being aged between 14 and 24 years, owning a mobile phone, and having the ability to read. Typically, AYAs in the community were invited to the designated areas in the community where comprehensive SRH services were provided, including counseling. They were then invited to perform HIVST using the USSD platform.
Data Collection for Pilot Testing
Before the pilot testing exercise, a community mobilization exercise was carried out. This involved proper community entry with the community development association (CDA) chairman and other community stakeholders who helped secure a space in the community for the field test. Participants were instructed on using the USSD code through an “information education and communication” (IEC) material developed by the NIMR I-TEST team. Each was provided with the OraQuick kit. Immediately after completing the test, participants were asked about their experience using the USSD platform.
The participants’ demographic characteristics (user testing and pilot study) were evaluated and reported as descriptive statistics. The recorded interview with the participants was transcribed verbatim, and inductive thematic analysis was done by members of the research team with experience in qualitative research. The inductive thematic analysis process ensured that the thematic analysis was completely data driven. The process involved repeated reading of the transcripts to become familiar with the data, generating initial codes relevant to the research questions, organizing the codes generated, arranging the subthemes into overarching themes, and defining and naming themes. A descriptive analysis of the findings from the heuristic questionnaire and the pilot study was also done. Variables were reported as proportions with bivariate analysis of the association between some variables through SPSS (version 26; IBM Corp).
The institutional review board of the Nigerian Institute of Medical Research (IRB-18-028) and Saint Louis University (approval number 31457) gave ethical approval for the Innovative Tools to Expand HIVST study, and informed consent was obtained from each participant before the commencement of the usability beta testing study.
A total of 19 AYAs participated in the usability study. The median age was 19 (IQR 16-23) years. There were 11 females, 8 males, and 0 nonbinary individuals. They were all out-of-school AYAs.
A total of 7 of Nielsen’s 10 usability heuristics yielded positive results. The visibility of the system status of the USSD platform showed that the user could tell the state of the platform and the alternatives for action (19/19), and the response times were appropriate to the user’s cognitive processing (18/19). The assessment of the match between the USSD platform and the real world showed that menu choices were ordered in the most logical way, given the user, the item names, and the task variables (18/19), and questions were stated in clear, simple language for the question-and-answer interfaces (19/19). Assessment of consistency and standards of the platform revealed that the field labels are consistent from 1 data entry screen to another (19/19), and commands are used the same way and mean the same thing in all parts of the system (16/19). Also, on whether the platform helps users to recognize, diagnose, and recover from errors, all the participants reported that messages place users in control of the system, and the prompts were brief and unambiguous. Similarly, the assessment of recognition and recall revealed that there was a good color and brightness contrast between image and background colors (19/19), and there was an obvious visual distinction made between the “choose one” menu and the “choose many” menus (18/19). On the aesthetic and minimalist design of the USSD platform, pop-up or pull-down menus within data entry fields do have many, but well-defined, entry options (19/19), and the field labels are brief, familiar, and descriptive (100%), while in the help and documentation assessment, all the users reported that instructions follow the sequence of user actions and the information on the platform is relevant ().
|Checklist||Yes, n (%)||No, n (%)|
|Visibility of system status|
|Does the system provide visibility—that is, by looking, can the user tell the state of the system and the alternatives for action?||19 (100)||0 (0)|
|Are response times appropriate to the user’s cognitive processing?||18 (94.7)||1 (5.3)|
|Match between system and the real world|
|Are menu choices ordered in the most logical way, given the user, the item names, and the task variables?||18 (94.7)||1 (5.3)|
|For question-and-answer interfaces, are questions stated in clear, simple language?||19 (100)||0 (0)|
|User control and freedom|
|Can users easily reverse their actions?||5 (26.3)||14 (73.7)|
|If the system uses a question-and-answer interface, can users go back to previous questions or skip forward to later questions?||12 (63.2)||7 (36.8)|
|If users can go back to a previous menu, can they change their earlier menu choice?||11 (57.9)||8 (42.1)|
|Consistency and standards|
|Are field labels consistent from 1 data entry screen to another?||19 (100)||0 (0)|
|Are commands used the same way, and do they mean the same thing, in all parts of the system?||16 (84.2)||3 (15.8)|
|Help users recognize, diagnose, and recover from errors|
|Do messages place users in control of the system?||19 (100)||0 (0)|
|Are prompts brief and unambiguous?||19 (100)||0 (0)|
|Does the system prevent users from making errors whenever possible?||13 (68.4)||6 (31.6)|
|Does the system warn users if they are about to make a potentially serious error?||12 (63.2)||7 (36.8)|
|Recognition rather than recall|
|Is there good color and brightness contrast between the image and background colors?||19 (100)||0 (0)|
|Is there an obvious visual distinction made between “choose one” and “choose many” menus?||18 (94.7)||1 (5.3)|
|Aestheticand minimalist design|
|Are there pop-up or pull-down menus within data entry fields that have many, but well-defined, entry options?||16 (84.2)||3 (15.8)|
|Are field labels brief, familiar, and descriptive?||19 (100)||0 (0)|
|Help and documentation|
|Do the instructions follow the sequence of user actions?||19 (100)||0 (0)|
|Is the information relevant?||19 (100)||0 (0)|
|Does the system or device give the option to return to factory settings?||10 (52.6)||9 (47.4)|
|Can the user take their own decisions? (Personalization)||16 (84.2)||3 (15.8)|
However, 3 of Nielsen's usability heuristics yielded negative results. These were user control and freedom, error prevention, and autonomy. The assessment of the user’s control and freedom revealed that the majority of the participants felt that users could not easily reverse their actions (14/19), and 7 out of 19 felt that if the platform uses a question-and-answer interface, users cannot go back to previous questions or skip forward to later questions, and 8/19 felt that users cannot change their earlier menu choice if they can go back to a previous menu. Concerning error prevention, 6 out of 19 of the participants believe that the platform does not prevent users from making errors whenever possible, and 7 out of 19 felt that the platform does warn users if they are about to make a potentially serious error. While the assessment of user autonomy showed that 9 out of 19 participants believed that the platform does not give the option to return to factory settings.
Review and recommendations from the heuristic evaluation: 1 of the limitations of a USSD platform is the difficulty or challenge of reversing actions after they have been completed or when an error occurs while using the platform without an opportunity to go back to the beginning. It was recommended that the question prompts on the platform be simplified to reduce this challenge. Also, to address the concern about users' autonomy, it was recommended that a unique ID be assigned to the user instead of a name. These changes were incorporated into the pilot study.
A qualitative assessment of the user experience was done through exit interviews. A total of 13 youths out of the 19 who participated in the usability testing agreed to participate in the exit interview.
Ease of Use
The participants expressed various views on the ease of use of the USSD platform for verifying HIVST results and linkage to care. Most participants believed that the USSD platform was easy to use without any significant challenge in guiding users toward performing HIVST, interpreting the result, and getting access to a trained counselor about their test results. However, some participants had some challenges in using the platform because of connectivity issues arising from their mobile network operator.
It was easy to understand, has simple language, and it is efficient to say a few
it was cool, matches the trend of the social media age. This experience means a lot. However, a few participants did not find it that easy to use mainly because of mobile network issues which were due to their GSM service provider.
Personal Preference About the Unstructured Supplementary Service Data Platform
When asked about their personal preference regarding the platform, about half of the respondents did not have any particular preference for the USSD. However, a few participants submitted that they were impressed by the simplicity of the question prompts on the USSD platform. Another participant stated that she prefers the fact that it is easily accessible and a good alternative to going to the hospital for HIV tests, as it promotes privacy when performing HIVST. One participant, however, had a concern about the fact that the USSD platform asks for some personal or sensitive information.
It was asking for basic information and I wondered the need for that
It is easily accessible, rather than going to the hospital. Especially the youths who spend most of their time with their phones.
The Aspect of the Unstructured Supplementary Service Data Platform That Needs Improvement
Some of the participants noted that the USSD platform would not work when the required code is dialed if there is no phone call credit on some of the mobile network operators’ platforms in Nigeria. However, it appears that this challenge is outside the immediate control of the study investigator. Otherwise, the majority of the participants felt there was no serious need for a significant improvement to the USSD platform in its current state.
Referral After Test
All the participants reported that they are happy with the feature of the USSD platform that provides linkage to experienced AYA counselors through phone calls after performing and interpreting their HIVST results, and all unanimously submitted that they would recommend this USSD code or platform to their friends who would like to do HIV self-testing and who do not have a smartphone.
Pilot Field Test
Pilot field testing of the USSD platform was done among 164 young persons. Females were in the majority (101/164, 61.6%), and the mean age of the participants was 17.5 (SD 3.18) years. Most participants were out-of-school AYAs involved in various unskilled and semiskilled work, trading, and nonprofessional work in the communities. Evaluation of their sexual orientation revealed that 92.1% (151/164) of the participants reported that they were heterosexual, while 7.9% (13/164) of them reported that they were gay (). All the participants in the pilot study were able to conduct HIVST, interpret their results, and use the linkage to care feature of the USSD platform without any challenge. A total of 7.9% (13/164) of youths had reactive HIV results (reactive to the OraQuick kit), and 92.1% (151/164) of them were nonreactive. However, among the 13 persons who were reactive and came for a confirmatory test at NIMR, none were confirmed HIV positive.
|Variable||Frequency (N=164), n (%)|
|Same-sex attracted||13 (7.9)|
|HIV test result|
|Age (years), mean (SD)||17.5 (3.18)|
There was no statistically significant association between the gender of the participants and the HIV-positive result done with the test kit (χ21=0.4; P=.76). Also, older participants had positive results using the test kit with a mean age of 19 (SD 3.2) years compared to those who were negative with a mean age of 17 (SD 3.2) years. This difference in age and the HIV-positive test result was not statistically significant (t1=1.60; P=.11). However, a statistically significant association existed between participants’ sexual orientation and HIV-positive test results using the OraQuick test kits. A total of 38% of persons who identified as gay were reactive using the test kit when compared to 5.3% among those who were heterosexual (χ21=18; P<0.001;).
|Variables||Nonreactive, n (%)||Reactive, n (%)||Chi-square (df)||t test (df)||P value|
|Male||92 (91.1)||9 (8.9)|
|Female||59 (93.7)||4 (6.3)|
|Sexual orientation||18 (1)||N/A||.001|
|Heterosexual||143 (94.7)||8 (5.3)|
|Same-sex attracted||8 (61.5)||5 (38.5)|
|Age (years), mean (SD)||17.38 (3.16)||18.85 (3.21)||N/A||1.60 (1)||0.11|
aN/A: Not applicable.
This study reported on the development, usability, and field pilot of a USSD system to verify HIV self-testing results among AYAs in Nigeria. The study demonstrated the benefits of involving AYAs in the development and usability testing of an intervention aimed at addressing a health challenge peculiar to them. It extends the existing literature [, , ] on the feasibility of the use of mHealth among AYAs in Africa. From the study, we were able to demonstrate that the usability of the USSD platform as an alternative platform to the use of mobile apps in the verification of HIV self-testing among young people is good and well accepted.
The young persons who participated in the exit interviews reported that the platform was easy to use and had no significant challenges except for the occasional inability to access the USSD service on their phones, mainly due to mobile network service interruptions. This was said to occur when the user does not have call credit on their mobile phone. This is similar to a challenge with mobile network operators reported by a South African study that submitted that the USSD is prone to network and user timeouts .
One of the important features of the USSD platform in interpreting the result of HIVST and promoting linkage to care is that users of the platform could indicate how they would prefer to be contacted after performing HIVST. This could be either to understand the test result or to learn more about HIV prevention, in which case they could either call a mobile phone number or request a call from a trained AYA counselor.
The participants in the heuristic evaluation of the USSD platform reported 3 issues regarding the ease of reversing an action or making corrections if an error was committed when using the platform and the perceived challenge of the user’s autonomy. Though modifications were made to reduce these challenges in the final product, it is noteworthy that these were some of the challenges associated with using the USSD platform for health interventions, as reported by previous studies . However, the benefits of improved health and access provided by the USSD platform were thought to outweigh these challenges [ ].
We believe usability assessment using heuristic evaluations among potential end users is unique to this study because the traditional approach for heuristic evaluation uses few content experts, with its attendant shortcomings. However, as previously documented by Silverbratt et al , heuristic evaluations involving end users have a higher frequency and relevance value, and the potential of real-life assessment of a new app toward improved acceptability of the same [ ].
The proportion of participants in the pilot study who tested positive after HIVST was about 7.9% (13/164), and none were confirmed to be living with HIV infection. This finding was not surprising because a previous study documented the sensitivity of the Oraquick HIVST kit to be between 92% and 93%, even though its specificity is as high as 99.9% [, ]. Expectedly, most of the participants in the pilot study requested a phone call from the study team through the USSD platform after their HIVST was completed. All participants with reactive HIV test results were contacted by the AYA counselor and encouraged to do confirmatory HIV testing.
Furthermore, given that all categories of young people could benefit from the use of the USSD platform for HIVST because it is cheaper, and does not require the use of internet data, and can be used on all mobile phones (whether they are smartphones or not), we chose the population of out-of-school young persons who were not likely to have a smartphone in the inner-city of an urban area of Lagos State. Successful piloting of the USSD platform among this population suggests that the USSD platform would be useful in promoting HIVST among many Nigerian AYAs. Also, the iterative design that used input from end users in the design increased the acceptability of the platform in promoting HIVST.
Even though the piloting was done in an urban population among out-of-school young persons, caution needs to be expressed in generalizing the findings to the population of AYAs in Nigeria. Also, though the exit interviews were conducted with measures to ensure privacy, removing the potential for social desirability bias is difficult because the young people might want to say what they consider appropriate about using a new mobile phone feature.
This study demonstrated the usability and feasibility of the USSD system as an alternative to mobile phone apps for the verification of HIVST results among the population of Nigerian AYAs without access to smartphones. Hence, the scale-up of access to creative and youth-friendly mHealth interventions is urgently needed, especially among the AYA population, who would benefit from the USSD for HIVST services. Additionally, further research using implementation research methods is necessary to further document this evidence-based intervention’s adoption, feasibility, and sustainability in resource-limited settings.
The authors would like to thank the Nigerian Institute of Medical Research and the youth peer educators involved in study recruitment. The research reported in this publication was supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development (award UG3HD096929). KT is supported by the National Institute of Allergy and Infectious Diseases (NIAID) (grant number: T32AI007001). The funders had no role in the design of the study, the collection, analysis, and interpretation of data, or the writing of the manuscript.
The data sets generated during and analyzed during this study are available from the corresponding author on reasonable request.
DAO, JI, JT, and OE conceived the ideas for the study. DAO and JT drafted the manuscript. DAO, JOO, and TG performed data acquisition and analysis. JO, UN, AZM, TG, COU, WT, and II reviewed the drafts and contributed substantially to the preparation of this manuscript. All authors have read and approved the manuscript.
Conflicts of Interest
Unstructured supplementary service data (USSD) instruction flow for HIV self-testing (HIVST).PNG File , 60 KB
Administrative dashboard of the unstructured supplementary service data (USSD) app.PNG File , 66 KB
- Akter S, Ray P. mHealth — an ultimate platform to serve the unserved. Yearb Med Inform 2010;19(01):94-100 [https://www.thieme-connect.com/products/ejournals/html/10.1055/s-0038-1638697] [Medline]
- Ippoliti NB, L'Engle K. Meet us on the phone: mobile phone programs for adolescent sexual and reproductive health in low-to-middle income countries. Reprod Health 2017;14(1):11 [https://reproductive-health-journal.biomedcentral.com/articles/10.1186/s12978-016-0276-z] [CrossRef] [Medline]
- Inhae C, Jiwon K. Effects of mHealth intervention on sexual and reproductive health in emerging adulthood: a systematic review and meta-analysis of randomized controlled trials. Int J Nurs Stud 2021;119:103949 [https://www.sciencedirect.com/science/article/abs/pii/S0020748921000924?via%3Dihub] [CrossRef] [Medline]
- Sahu M, Gandhi S, Sharma MK. Mobile phone addiction among children and adolescents: a systematic review. J Addict Nurs 2019;30(4):261-268 [CrossRef] [Medline]
- Gerdts C, Hudaya I, Belusa E. Mobile health innovations for improved access to safe abortion in Indonesia. Contraception 2014;90(3):299 [CrossRef]
- Smith C, Ngo TD, Gold J, Edwards P, Vannak U, Sokhey L, et al. Effect of a mobile phone-based intervention on post-abortion contraception: a randomized controlled trial in Cambodia. Bull World Health Organ 2015;93(12):842-850A [https://europepmc.org/abstract/MED/26668436] [CrossRef] [Medline]
- Lopez AD, Murray CC. The global burden of disease, 1990-2020. Nat Med 1998;4(11):1241-1243 [CrossRef] [Medline]
- Torrone EA, Morrison CS, Chen PL, Kwok C, Francis SC, Hayes RJ, STIMA Working Group. Prevalence of sexually transmitted infections and bacterial vaginosis among women in sub-Saharan Africa: an individual participant data meta-analysis of 18 HIV prevention studies. PLoS Med 2018;15(2):e1002511 [https://boris.unibe.ch/id/eprint/112279] [CrossRef] [Medline]
- Hokororo A, Kihunrwa A, Hoekstra P, Kalluvya SE, Changalucha JM, Fitzgerald DW, et al. High prevalence of sexually transmitted infections in pregnant adolescent girls in Tanzania: a multi-community cross-sectional study. Sex Transm Infect 2015;91(7):473-478 [https://europepmc.org/abstract/MED/25834122] [CrossRef] [Medline]
- 2020 global AIDS update—seizing the moment—tackling entrenched inequalities to end epidemics. Joint United Nations Programme on HIV/AIDS (UNAIDS). 2020. URL: https://www.unaids.org/en/resources/documents/2020/global-aids-report?_ga=2.27859967.251568576.1690863514-1813473850.1690863514&_gl=1*1jresw3*_ga*MTgxMzQ3Mzg1MC4xNjkwODYzNTE0*_ga_T7FBEZEXNC*MTY5MDg2MzUxNC4xLjEuMTY5MDg2NTA2Ni40NC4wLjA [accessed 2023-08-01]
- Federal Ministry of Health, Nigeria. Nigeria HIV/AIDS Indicator and Impact Survey (NAIIS). Technical Report. Abuja, Nigeria: Federal Ministry of Health, Nigeria; 2019 Oct. URL: https://www.naiis.ng/resource/factsheet/NAIIS%20PA%20NATIONAL%20FACTSHEET%20FINAL.pdf [accessed 2023-08-01]
- Iwelunmor J, Ezechi O, Obiezu-Umeh C, Gbaja-Biamila T, Nwaozuru U, Oladele D, et al. The 4 youth by youth HIV self-testing crowdsourcing contest: a qualitative evaluation. PLoS One 2020;15(5):e0233698 [https://dx.plos.org/10.1371/journal.pone.0233698] [CrossRef] [Medline]
- Obiezu-Umeh C, Gbajabiamila T, Ezechi O, Nwaozuru U, Ong JJ, Idigbe I, et al. Young people's preferences for HIV self-testing services in Nigeria: a qualitative analysis. BMC Public Health 2021;21(1):67 [https://bmcpublichealth.biomedcentral.com/articles/10.1186/s12889-020-10072-1] [CrossRef] [Medline]
- Oladele D, Iwelunmor J, Gbajabiamila T, Obiezu-Umeh C, Okwuzu JO, Nwaozuru U, et al. The 4 youth by youth mhealth photo verification app for HIV self-testing in Nigeria: qualitative analysis of user experiences. JMIR Form Res 2021;5(11):e25824 [https://formative.jmir.org/2021/11/e25824/] [CrossRef] [Medline]
- Mngadi KT, Maharaj B, Duki Y, Grove D, Andriesen J. Using mobile technology (pMOTAR) to assess reactogenicity: protocol for a pilot randomized controlled trial. JMIR Res Protoc 2018;7(10):e175 [https://www.researchprotocols.org/2018/10/e175/] [CrossRef] [Medline]
- Macharia P, Pérez-Navarro A, Sambai B, Inwani I, Kinuthia J, Nduati R, et al. An unstructured supplementary service data-based mHealth app providing on-demand sexual reproductive health information for adolescents in Kibra, Kenya: randomized controlled trial. JMIR Mhealth Uhealth 2022;10(4):e31233 [https://mhealth.jmir.org/2022/4/e31233/] [CrossRef] [Medline]
- Perrier T, DeRenzi B, Anderson R. USSD: the third universal app. 2015 Presented at: ACM DEV '15: Annual Symposium on Computing for Development; December 1 - 2, 2015; London United Kingdom [CrossRef]
- Tucker JD, Iwelunmor J, Abrams E, Donenberg G, Wilson EC, Blachman-Demner D, et al. Accelerating adolescent HIV research in low-income and middle-income countries: evidence from a research consortium. AIDS 2021;35(15):2503-2511 [https://europepmc.org/abstract/MED/34870930] [CrossRef] [Medline]
- Kientz JA, Choe EK, Birch B, Maharaj R, Fonville A, Glasson C, et al. Heuristic evaluation of persuasive health technologies. 2010 Presented at: IHI '10: ACM International Health Informatics Symposium; November 11 - 12, 2010; Arlington Virginia USA [CrossRef]
- Charters E. The use of think-aloud methods in qualitative research an introduction to think-aloud methods. Brock Educ J 2003;12(2):68-82 [https://journals.library.brocku.ca/brocked/index.php/home/article/view/38] [CrossRef]
- Lif P, Hermelin J, Bengtsson K, Svensson J. Heuristic evaluation conducted by end users. 2019 Presented at: International Conference Interfaces and Human Computer Interaction; July 16-18, 2019; Porto p. 65-72 URL: https://www.iadisportal.org/digital-library/heuristic-evaluation-conducted-by-end-users [CrossRef]
- Wong AMK, Chang HJ, Tsai TH, Tseng KC. A user-based heuristic evaluation of an intelligent healthcare system. : IEEE; 2013 Presented at: 2013 IEEE Third International Conference on Consumer Electronics ¿ Berlin (ICCE-Berlin); 09-11 September 2013; Berlin, Germany [CrossRef]
- Beatty AL, Magnusson SL, Fortney JC, Sayre GG, Whooley MA. VA FitHeart, a mobile app for cardiac rehabilitation: usability study. JMIR Hum Factors 2018;5(1):e3 [https://humanfactors.jmir.org/2018/1/e3/] [CrossRef] [Medline]
- Haug S, Castro RP, Wenger A, Schaub MP. A mobile phone-based life-skills training program for substance use prevention among adolescents: cluster-randomized controlled trial. JMIR Mhealth Uhealth 2021;9(7):e26951 [https://mhealth.jmir.org/2021/7/e26951/] [CrossRef] [Medline]
- Barron P, Peter J, LeFevre AE, Sebidi J, Bekker M, Allen R, et al. Mobile health messaging service and helpdesk for South African mothers (MomConnect): history, successes and challenges. BMJ Glob Health 2018;3(Suppl 2):e000559 [https://gh.bmj.com/lookup/pmidlookup?view=long&pmid=29713503] [CrossRef] [Medline]
- Barjis J, Kolfschoten G, Maritz J. A sustainable and affordable support system for rural healthcare delivery. Decis Support Syst 2013;56:223-233 [https://www.sciencedirect.com/science/article/abs/pii/S0167923613001760] [CrossRef]
- Silverbratt M. Employing a comparative evaluation of heuristic evaluations with end-users and usability experts as evaluators. Digitala Vetenskapliga Arkivet. 2022. URL: https://www.diva-portal.org/smash/record.jsf?pid=diva2%3A1676977&dswid=-1570 [accessed 2023-08-01]
- Nkenfou CN, Kembou JE, Temgoua ES, Djikeng A, Mouafo LM, Ndzie EN, et al. Evaluation of OraQuick® HIV-1/2 as oral rapid test. Afr J Infect Dis 2013;7(2):27-30 [https://europepmc.org/abstract/MED/28451078] [CrossRef] [Medline]
- Zachary D, Mwenge L, Muyoyeta M, Shanaube K, Schaap A, Bond V, et al. Field comparison of OraQuick advance rapid HIV-1/2 antibody test and two blood-based rapid HIV antibody tests in Zambia. BMC Infect Dis 2012;12(1):1-7 [https://bmcinfectdis.biomedcentral.com/articles/10.1186/1471-2334-12-183] [CrossRef] [Medline]
|AYA: adolescents and young adult|
|CDA: community development association|
|HIVST: HIV self-testing|
|IEC: information education and communication|
|I-TEST: innovative tools to expand HIVST|
|LMIC: low- and middle-income country|
|mHealth: mobile health|
|NIH: National Institutes of Health|
|NIMR: Nigerian Institute of Medical Research|
|SRH: sexual and reproductive health|
|STI: sexually transmitted infection|
|USSD: unstructured supplementary service data|
Edited by A Mavragani; submitted 17.11.22; peer-reviewed by C Li, A Olufemi; comments to author 30.05.23; revised version received 10.07.23; accepted 17.07.23; published 25.09.23Copyright
©David Ayoola Oladele, Juliet Iwelunmor, Titilola Gbajabiamila, Chisom Obiezu-Umeh, Jane Ogoamaka Okwuzu, Ucheoma Nwaozuru, Adesola Zaidat Musa, Kadija Tahlil, Ifeoma Idigbe, Jason Ong, Weiming Tang, Joseph Tucker, Oliver Ezechi. Originally published in JMIR Formative Research (https://formative.jmir.org), 25.09.2023.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.