Korean American women (11.9 per 100,000) are disproportionately affected by cervical cancer compared with the overall population of women in the United States (7.2 per 100,000) [1]. Korean American women have been consistently identified as the least likely subgroup to receive cervical cancer screenings [2,3]. Despite growing evidence showing the benefits of human papillomavirus (HPV) vaccines in preventing cervical cancers, there are few population-based data on HPV vaccination behavior and intervention studies to promote HPV vaccination behavior for young Korean Americans [4,5].

Mobile health (mHealth) and Web-based research methods are increasingly common as a tool for research and research settings [6]. In particular, online outreach is the most effective recruitment method for young, bilingual Korean Americans [7]. Despite this significant potential, there are minimal HPV intervention studies that discuss issues specific to mHealth and Web-based studies of particular populations.

To fill this gap, this study aimed to (1) assess the feasibility of research procedures (recruitment, eligibility, randomization, and retention) to inform a future randomized controlled trial (RCT) of an mHealth and Web-based HPV prevention program targeting young Korean Americans and (2) discuss the challenges and lessons learned from an mHealth and Web-based study.

This was a pilot RCT that consisted of multiple stages: (1) a qualitative study [8] including an HPV storytelling intervention [9], (2) assessment of the acceptability of the intervention [9], (3) assessment of the feasibility of mHealth and Web-based research procedures, and (4) evaluation of the preliminary effectiveness of the intervention [4]. This study reports on the feasibility of research procedures in light of a 2-month follow-up.

The study protocol was designed to maintain data and research integrity (Figure 1). We used personally identifiable information (eg, name, email address, phone number, and internet protocol [IP] address) to detect multiple attempts or duplication in the online survey. The protocol was explained in the consent form provided. The institutional review board at the University of Massachusetts Boston approved the research procedures of this study.

From September 2016 to October 2016, we recruited participants via social networking sites (eg, Facebook and KakaoTalk), Korean community websites, and word-of-mouth sources, including Korean church communities and Korean student associations in the northeast. Recruitment materials focused on the following theme: “I Want to Know More about the HPV Vaccine .” Potential subjects were instructed to visit the study website.

We anticipated that some respondents would be ineligible for the study. Therefore, we used a 4-question online survey to ensure that respondents were female Korean American college students (aged 18-26 years) from the northeastern United States who had not yet been vaccinated against HPV. Eligible participants received a link to our mHealth and Web-based study platform where we collected data, disseminated the intervention, performed randomization, and accommodated mobile devices and Web services.

At the end of the baseline survey, participants were randomly assigned to either the experimental group (a storytelling video intervention) or the comparison group (a nonnarrative, written statement about HPV and HPV vaccine) using the randomizer tool in Qualtrics survey software that offers the mobile and Web-based platform, until each group contained 60 participants.

Furthermore, 2 months after the intervention, participants received a follow-up survey via email with a passcode. In total, two reminders about the follow-up survey were sent via email. Each individual received a US $20 electronic gift (e-gift) card at the end of the postintervention survey and was entered into a drawing for an e-gift card at the end of the follow-up survey.

The feasibility and validity of mHealth and Web-based research procedures were measured as follows: (1) recruitment rate was measured for the percentage of participants who signed the informed consent forms out of the total number of eligible participants, (2) eligibility rate was measured by dividing the number of total entries in the eligibility screening survey by the number of individuals who met the inclusion criteria, (3) randomization was measured by whether we were able to randomize eligible participants evenly, and (4) retention rate was measured by the 2-month follow-up participation. In addition, we identified the recruitment source by asking how participants had heard of the study.

Figure 2 shows the flow of participants throughout the pilot RCT.

Between September 2016 and October 2016, the recruitment target of 60 participants in each group was achieved. Out of 225 potentially eligible participants, 204 individuals responded to how they heard about the study. The most commonly reported recruitment methods were through friends 46.1% (94/204) and social media 33.8% (68/204). The recruitment rate was 95.6% (111/122).

The eligibility rate was 54.2% (122/225) after excluding ineligible participants (n=60), multiple attempts at entries (n=6), and questionable entries (n=37; Figure 2).

Randomization between groups was effectively demonstrated through the Qualtrics software. However, after ineligible and incomplete cases were excluded, an uneven number of participants remained in each group: 56 in the experimental group and 55 in the comparison group.

In total, 83.7% (87/104) of participants participated in the 2-month follow-up study: 45 in the intervention group and 42 in the comparison group.

This study assessed the feasibility of using an mHealth and Web-based HPV intervention experimental study with young Korean Americans to inform a future RCT. Overall, these findings demonstrate that the mHealth and Web-based experimental study is feasible if software protection and a systemic approach to eligibility screening are used. This study demonstrated a high recruitment rate (95.6%, 111/122) and a moderate eligibility rate (54.2%, 122/225). Although our eligibility rate was moderate because of restrictive inclusion criteria, we believe that our recruitment strategies filled the gaps of previous studies’ concerns in recruiting Korean Americans [10,11] by achieving the target sample size in 2 months of recruitment activities. Our approach suggests a way to overcome this challenge. Randomization was evenly distributed between the groups, although ineligible and incomplete cases led to an uneven number of participants in each group in the dataset. In this study, the retention rate was high (83.7%, 87/104) at the 2-month follow-up. In future RCT research, encouraging reminders are recommended as a method to improve HPV vaccine participation and compliance and increase engagement, minimize attrition, and maximize retention.

Several challenges in the process provide important lessons for future mHealth and Web-based experimental studies. One challenge was the prevention and detection of multiple attempts at entries and potential misrepresentation, which could threaten the validity of the study. We addressed this challenge with software protection, such as Prevent Indexing and Prevent Ballot Box Stuffing (which prevents participants from taking a survey multiple times by using browser cookies), as well as a systemic approach for eligibility screening to prevent and detect multiple attempts at entries and identify potential misrepresentation.

Despite these precautions, we found a considerable number of invalid entries. These entries had similar or identical IP addresses, and contact information and IP addresses were from outside the northeastern United States. These activities may be explained by our use of monetary incentives, which can not only enhance recruitment and retention in mHealth and Web-based studies [12] but also encourage ineligible applicants and multiple attempts by eligible individuals. Studies using electronic health have discussed similar challenges [13,14] and suggested that a systematic approach for eligibility screening may help obtain valid data [13-15]. In addition, we believe it is critical to cross-check the respondent’s information (eg, email address, phone number, and demographic information) with the information collected during the eligibility screening and after the data collection. We found 6 cases of multiple attempts during the eligibility screening and additional 5 cases of multiple attempts via the screening of IP addresses and contact information after the postintervention survey when we reviewed their contact information to send incentives.

Ultimately, it may be necessary to use an email validation feature to ensure that respondents are entering a valid email address. For example, researchers can set a condition to include edu in the first email field and a logic condition to require that verification email to be equal to the first email. In this way, researchers may decrease invalid or multiple entries and confirm the verification of respondents’ student status regardless of recruitment methods. Moreover, researchers can block questionable IP addresses or contact information using a survey software.

To protect the integrity of mHealth and Web-based research data, researchers and institutional review boards must be mindful about possible misrepresentation activities and how or what to take action on when these activities are suspected or detected. In future research studies, we suggest researchers develop a data and safety monitoring plan regarding how to maintain research integrity if an invalid activity is suspected or detected.

To our knowledge, this is the first study to examine the feasibility of an mHealth and Web-based experimental study to promote HPV vaccination among young Korean Americans who are at a high risk for HPV infections and HPV-attributable cancers. These findings may not be generalizable to other populations.

An mHealth and Web-based HPV prevention experimental study is an efficient and flexible way to perform research with female Korean American college students. However, given the increased possibility of invalid and multiple entries in mHealth and Web-based studies, strategies for detection and prevention are critical. Findings from this study demonstrated sufficient feasibility in terms of research procedures—including recruitment, data collection, randomization, and retention—to justify a full-scale RCT. The lessons learned from this study provide key insights into other future mHealth and Web-based experimental studies.