We conducted a 3-month pilot RCT with 40 participants from October 2023 to June 2024. Participants in both arms received STI counseling and condoms (baseline) and completed a survey (baseline and 3-month follow-up). Participants who were randomized to the intervention arm (mLab+ app) were given app access and took an HIV and syphilis test while using the app at both their baseline and 3-month follow-up appointments. This paper only reports the findings of the intervention arm.

We used a multimodal recruitment strategy. We recruited through sexual networking apps (eg, Grindr and Scruff) and through community partners. Online advertisements included images of diverse MSM and were targeted to MSM through dating apps. A public link to a screener form was embedded within the advertisement image so that participants could fill out a brief survey to determine eligibility prior to being contacted by the research team. Regarding recruitment through community partners, research staff visited several clinics, organizations, gay bars, and shops throughout New York City to post physical flyers. Physical flyers included the study team’s phone number and a QR code that directed interested participants to the public screener link. The eligibility criteria can be found in Textbox 1. Eligible participants came to the Columbia University Irving Medical Center Nurse Practitioner Group (NPG) clinic for their study visit. Because all study procedures took place in person, participants were all from the New York City metropolitan area. They had to be able to travel to the study office for visits to be enrolled in the study.

All study activities were approved by the Columbia University Institutional Review Board under approval AAAU3380 prior to the recruitment or enrollment of study participants. This study was registered on ClinicalTrials.gov (NCT06059443). Study participants signed electronic informed consent forms through REDCap (Research Electronic Data Capture; Vanderbilt University) that stated the procedures for both the intervention and control arms of the study. The consent form outlined analysis procedures following the study’s completion, including the individuals and agencies who had permission to look through research records, the deidentification of all pieces of information that may identify them, and the use of their deidentified data for future research purposes. The consent form also included a statement of confidentiality. This detailed authorization for Columbia University Irving Medical Center, the New York-Presbyterian Hospital, and the principal investigator and her staff to use and disclose protected health information (PHI) in connection with the research study; an explanation of what PHI consists of; and for what purposes the PHI may be used or disclosed. Furthermore, it stated that we obtained a certificate of confidentiality from the National Institutes of Health. Participants were compensated with US $40 for their baseline appointment and US $60 for their 3-month follow-up appointment.

Of a total of 707 people who filled out an eligibility screener, 122 (17.3%) were screened as eligible, and 40 (5.7%) were enrolled and randomized to the control (n=20, 50%) or intervention (n=20, 50%) arms (Figure 2).

In total, 95% (19/20) of the participants who used the mLab+ app identified as male, and 5% (1/20) identified as nonbinary. Age ranged from 21 to 35 years, with an average of 28 (SD 3.94) years. Participants reported their race as Asian (5/20, 25%), Black (6/20, 30%), White (8/20, 40%), and other (1/20, 5%) and their ethnicity as not Hispanic or Latino (14/20, 70%), Dominican (2/20, 10%), Mexican (1/20, 5%), Puerto Rican (2/20, 10%), and another Hispanic identity (1/20, 5%).

Most participants (16/20, 80%) were able to complete the test on their own at both time points. Table 1 shows the number of participants who completed testing independently (ie, “yes”) and with the help of a clinician (ie, “no”). Table 2 shows the test results for both HIV and syphilis at each time point.

Paradata were collected during baseline and follow-up visits. The average duration of a session, from after authentication until log-out or abandonment, was 30 minutes and 33 seconds (SD 21 minutes and 40 seconds; median 36 minutes and 2 seconds; IQR 0 minutes 42 seconds-44 minutes and 15 seconds). Apart from the 27% (13/48) of sessions that were 5 minutes or less, the distribution of session durations was approximately normal (Figure 3). The distribution of duration viewing each screen (Figure 4) shows that users spent the longest time viewing testing screens (ie, timer screens, the initial testing screen, the test guided walkthroughs, test results, and picture and result upload). Auxiliary screens (ie, privacy policy and help) were typically only briefly visited. Notably, the 2 timer screens (“Timer: running buffer” and “Timer: sample”) both had very narrow distributions, indicating that time spent viewing the timer screens was brief, and the automated timers functioned as expected, moving participants along the testing workflow at the completion of each timer.

As test taking was the primary objective of app use, it is informative to look at paradata related to the testing pathway. The workflow conversion funnel (Table 3) shows the percentage of time during which a screen was visited in an individual session. These data indicate that 77% (37/48) of sessions resulted in a user visiting the screen where a test could be initiated (“Testing”) and 71% (34/48) of sessions started a test. There was no drop-off in conversion until the “Preview picture” screen, which 67% (32/48) of the sessions reached. This conversion remained constant using the DPP Micro Reader and only decreased to 63% (30/48) of sessions that ended with a test result.

Of the sessions that resulted in a completed test, the distribution of the time it took from log-in to obtaining the test results (Figure 5) was approximately normal (mean 39 minutes and 6 seconds, SD 10 minutes and 1 second; median 38 minutes and 14 seconds).

The mLab+ app was rated by participants in the intervention arm using the Health-ITUES [29] and PSSUQ [32] (Table 4). The overall mean score on the Health-ITUES was 3.62 (SD 1.07; range 1-5), with higher scores indicating greater perceived usefulness. The overall mean score on the PSSUQ was 2.65 (SD 1.06; range 1-4.19), with higher scores indicating lower perceived usefulness. The scores on both measures indicate medium to high usability.

During the recruitment and enrollment period, 707 people were screened for eligibility, with 82.7% (585/707) being screened as ineligible and 17.3% (122/707) being screened as eligible. Having had an HIV test within the previous 3 months (n=327, 46.8%) and an HIV diagnosis (n=152, 21.5%) were the most frequently cited reasons for ineligibility. Exclusion criteria were not mutually exclusive. In addition, 7% (6/82) of the people who were screened as eligible did not show up to their study visit, 9% (7/82) of the participants showed up to their appointment but were found to be ineligible and not enrolled, and others (69/82, 84%) never scheduled a baseline appointment. In total, 50% (20/40) of the enrolled participants were randomized to the control arm and, thus, did not use the mLab+ app. During the study’s 6-month recruitment and enrollment period, an average of 5 participants (SD 4.57) were enrolled per month.

Of the 20 participants who enrolled in the study and used the mLab+ app, 16 (80%) completed their 3-month follow-up study assessment. One participant had a known HIV diagnosis before enrolling in the study and was administratively withdrawn. Another participant voluntarily withdrew from the study. The remaining 2/4 (50%) were lost to follow-up.

The findings of this study focused on the feasibility and usability of the mLab+ app. Most, but not all, participants (16/20, 80%) were able to use the test on their own without the assistance of the clinician. This points to some of the challenges for consumers using these tests at home without the supervision of a clinician. Notably, resistance to finger stick blood collection is often a barrier to self-testing [33]. Nonetheless, past work has similarly demonstrated difficulties among people using at-home testing, such as the OraQuick HIV test, which requires fewer steps than the DPP HIV-Syphilis test and has already been FDA approved for at-home use [9]. Future studies should collect data on participant self-testing history, including experience with and acceptance of at-home tests, which may highlight additional barriers associated with successful completion of a self-test through the mLab+ app.

The overall usability of the app was medium to high, with lower scores on the usefulness subscale. Given that the mLab+ app was originally designed for in-home use, these scores reflect the use of this app in a clinical setting, which was not a part of its intended design. Furthermore, the usefulness of the app after the study visit was limited given that the app was designed to support using the HIV and syphilis test at home and participants did not have access to the test in their homes.

The recruitment and retention rate were acceptable within the context of this small sample size and would warrant further consideration in a fully powered trial. Retention would likely be improved in a future trial if research visits took place remotely [7,34], as is evidenced by the original mLab study, which largely took place over Zoom and resulted in a retention rate of 89.5% at the 6-month follow-up and 84.6% at the 12-month follow-up; this trial was much larger (N=525) and longer (12-month study period) than the mLab+ app study, demonstrating the potential for improved retention in a remote setting [7,34]. Given the FDA restrictions at the time of this study’s start, we were not able to conduct the mLab+ app study remotely. Furthermore, to improve linkage to care among HIV-positive participants, future studies should consider integrating nearby resources, including addresses and phone numbers, into the app, as well as next steps to facilitate making a physician’s appointment, as demonstrated in the SMARTtest mobile health study for HIV and syphilis testing [35]. This feature would be helpful for participants who refused on-site testing the day they received their reactive result.

Finally, the paradata highlight key aspects of user behavior and the effectiveness of the testing workflow. Users predominantly spent time on testing-related screens, such as timers, guided walkthroughs, and result screens, reflecting the primary purpose of the app. Auxiliary screens such as the privacy policy and help were visited briefly, which is expected given the scope of the study; these screens, particularly the help screen, would be expected to have more use in a larger trial or in an environment where study team members were not immediately available. The consistency in time spent on both timer screens suggests that the automated workflows function as intended, guiding users smoothly through the testing process by closely following the manufacturer’s guidelines. Importantly, the analysis of the workflow conversion funnel indicates minimal drop-off once a test was initiated. This, particularly when combined with the usability results, suggests that the app effectively supported users in completing the testing process. Furthermore, the time to complete the tests aligns well with the expected duration, although slight deviations may occur due to session resets when users resume testing after logging out. Overall, the findings underscore the usability and robustness of the app in facilitating the testing workflow and its suitability for use in a larger study.

This study was designed as a pilot RCT, but due to the limitations of conducting the study in a clinical setting, we are only reporting on the intervention arm, who had access to the mLab+ app and the combination test for HIV and syphilis. Although participants were randomized into intervention versus control arms, we did not include control arm data in this paper. As originally designed, study participants would use the mLab+ app and the combination test for HIV and syphilis for 3 months in a pilot RCT. The original conceptualization of the study design was to assess testing uptake and referral to services in the participants who were given access to the mLab+ app and HIV and syphilis test as compared to a control arm, which would only receive information on HIV and syphilis testing services. The FDA denied the request for study participants to use the combination test for HIV and syphilis in their homes, so the study design was modified, and participants were only able to use the test in a clinical setting under the supervision of a nurse practitioner. Future studies should consider the feasibility and acceptability of the mLab+ app outside of the clinical setting.

Findings from this study support the use of the mLab+ app as a tool for assisting consumers in self-testing for HIV and syphilis. This is especially important in the context of the growing HIV and syphilis epidemic among MSM in the United States. Nonetheless, the limitations of the study design warrant further examination outside of the clinic setting to better understand the utility of these tools for improving consumer health outcomes.