The technology-based training tool entailed close collaboration between the academic team who created PHAT Life and the industry team who developed the tool. The final prototype uses dynamic multimedia presentations and an interactive self-paced course to teach group facilitator skills and curricular material. Development of the tool followed nine steps. First, the industry team created an outline of the PHAT Life in-person training for translation into the technology-based tool. Using this outline, trained PHAT Life facilitators and research staff produced text to guide video vignettes of intervention activities and demonstrate facilitator skills. A two-person production team traveled to the research site and filmed two expert PHAT Life facilitators delivering the curriculum to 5 mock participants. The videos were then edited to create 25 clips that were subsequently inserted into the training tool as examples of how to deliver the intervention. For the prototype, the team built an introduction and two of the most challenging PHAT Life sessions. This resulted in five distinct modules (see Table 1).

Next, text was created for each of the five modules based on the PHAT Life curriculum and training materials. The text also described the video clips, pointed out facilitator skills, and brought attention to facilitation strategies. Each web page was reviewed by several team members for ease of access, comprehension, and usability. Once the text was finalized, a team member with voice recording experience created audio narrations to accompany each page and accommodate literacy concerns (see Figure 1).

The final prototype of the PHAT Life technology-based training tool was available for both iOS and Android operating systems. The five modules were completed sequentially, but a menu-driven option allowed participants to review previously completed material (see Figure 2). Audio narration allowed participants to listen to the written text, and each page was supplemented by a video demonstration or relevant photograph. Games and questions were interspersed throughout the training to increase user engagement (see Figure 3). At the end of the module, participants completed quizzes to evaluate knowledge learned. The training tool was designed so participants could upload video role plays facilitating curriculum activities. Expert trainers who viewed the videos could insert time-stamped comments to alert participants to segments associated with the feedback (see Figure 4). The tool provided a “Resource” tab with supplemental PHAT Life materials and links to research articles (see Figure 5), and messaging between the trainers and participants was enabled.

Participants were 12 staff members aged 26-54 (mean 36.67, SD 9.45) years recruited from organizations that provide detention alternative services to justice-involved youth. Staff were eligible for the pilot study if they provided group-based services to youth. Table 2 provides demographic information about the sample. A total of 67% (n=8) of participants reported prior experience providing youth with sexual health information. One participant left their job and withdrew from the study, but the remainder (n=11) provided written and verbal feedback about the technology-based training platform. We deliberately enrolled 3 staff with previous in-person PHAT Life training experience to explore the perceived advantages and disadvantages of the different approaches.

Participants were recruited from two sites likely to deliver PHAT Life. Eligible staff received study information via email, and interested individuals were given a link to the consent form and baseline survey. Participants were instructed to log on to the training tool and complete modules 1 to 5 (in order) at their own pace. Following modules 4 and 5, participants uploaded or conducted a live demonstration of two PHAT Life activities: anatomy and STI information. Participants could send messages to the PHAT Life trainers within the training tool. Expert trainers reviewed the role plays and quizzes, and assigned fidelity scores. All participants received live supervision where expert trainers provided feedback about the role plays and quiz responses, answered questions, and solicited comments about the intervention and training tool. Participants completed a posttraining survey evaluating HIV and STI knowledge, confidence delivering sex education, and perceptions about the tool (adoption, implementation, acceptability, appropriateness, sustainability, and satisfaction). They received a US $150 electronic gift card as compensation. All procedures were approved by the Oregon Research Institute Institutional Review Board.

Participants completed surveys at baseline and posttraining via Qualtrics.

The pilot test occurred over 13.1 weeks. Participants spent an average of 4.3 (SD 2.2) hours to complete the training. All participants completed the five modules, two role plays, and one supervision session. Most participants preferred to use their computer to complete the training (n=7, 64%), but 27% (n=3) used a combination of the computer and smartphone app. Only 9% (n=1) used the browser on their phone exclusively. Participants spent 70% (33.17 hours) of their time using the tool during the day (8 AM to 5 PM), 20% (9.54 hours) after the workday (5 PM to 9 PM), 9% (4.61 hours) in the evening (9 PM to 12 AM), and 1% (0.43 hours) at night (12 AM to 8 AM). In total, participants exchanged 67 messages with expert trainers within the tool, primarily focused on scheduling supervision sessions.

Participant STI knowledge significantly increased from baseline to posttraining (t₁₀=3.07; P=.01; see Figure 6). Participants also showed slight improvements in HIV knowledge (t₁₀=1.70; P=.12), although this did not reach statistical significance. The difference in the paired t test for HIV knowledge resulted in a Cohen d of 0.63, which is considered a moderate to large effect size. Quizzes at the end of each module indicated that most participants learned the material. A quiz score of 80% was considered passing, and 80% (n=9) of participants passed modules 1, 3, and 4; all participants passed module 2; and 90% (n=10) passed module 5. At baseline, participants reported high confidence in providing sexual education to youth (mean 5.14, SD 0.67), but this diminished following the training (mean 4.74, SD 0.68; t₁₀=2.51; P=.03).

On average, participants strongly agreed that PHAT Life could be adopted (mean 4.26, SD 0.42) and implemented (mean 9.07, SD 0.87) in their setting to meet the needs of the youth served. They rated the training as very acceptable (mean 3.74, SD 0.32) and appropriate (mean 3.80, SD 0.32), and highly sustainable in their settings (mean 6.02, SD 0.94). They described the videos (mean 4.82, SD 0.41), manuals (mean 4.73, SD 0.47), role play activities (mean 4.64, SD 0.67), and supervision (mean 4.73, SD 0.47) as very useful. Few technical issues were reported, and there was general agreement that the tool was easy to use and navigate. Some participants expressed difficulty uploading videos of the role plays and preferred live demonstrations. Of the 3 participants who previously received the in-person PHAT Life training, 2 rated the in-person and technology-based training equally and 1 preferred the in-person training to be able to ask questions in the moment. The mean rating for satisfaction posttraining (mean 4.42, SD 0.36) indicated high satisfaction, and this was supported by participants’ open-ended comments:

Fidelity to the curriculum was excellent (mean 3.39, SD 0.62) with high ratings for adherence (mean 3.71, SD 0.78) and competence (mean 3.38, SD 0.87). Supervision sessions lasted 20 to 60 minutes. Participants were asked on the postsurvey how often they preferred individual supervision to inform the training model; 55% (n=6) said once a year or every 6 months, and 36% (n=4) endorsed more frequent supervision. Only 1 participant said they did not need ongoing supervision.

This study describes the development and pilot testing of a prototype technology-based training tool for PHAT Life, a comprehensive health promotion and sex education EBP for justice-involved youth [21]. Creation of the training tool required strong collaboration between research and industry teams, and findings support the use of online training as a strategy to address a critical implementation barrier to the uptake of EBPs for justice-involved youth. Results demonstrated feasibility and warrant completion of the PHAT Life technology-based training tool.

Consistent with the hypothesis, participant knowledge of HIV and STI increased as reflected in their high scores on module quizzes and pre/posttraining measures. In addition, participants performed the two session activities with high fidelity. Surprisingly, HIV knowledge did not significantly increase from pre- to posttraining, but a trend emerged. Three possible reasons may explain the lack of significant increase in HIV knowledge. First, it is possible that role playing teaching the STI activity for evaluation by the expert trainer facilitated learning about STIs more so than HIV. Indeed, evidence supports that teaching improves knowledge [51]. Second, it is also possible that the two sessions in the prototype lacked sufficient information to significantly improve HIV knowledge among participants. Lastly, since participants were highly knowledgeable about HIV at pretest (ie, they answered 10 of 12 questions correctly), there was little room for improvement. Overall, results of this pilot test suggest that technology-based training may be sufficient to learn new information and achieve intervention fidelity.

Contrary to expectations, sex education confidence significantly decreased from pre- to posttraining. It is possible that participants overestimated their knowledge about sexual health at baseline and once exposed to the training material became aware of their knowledge deficits. For example, one participant noted, “I didn’t know this before,” when reflecting on topics related to male and female anatomy and STIs. The reduced confidence may be temporary, however; once participants deliver the intervention, their confidence and self-efficacy will likely improve [52].

Consistent with expectations, participants rated the technology-based training tool positively across all implementation outcomes. This is highly encouraging because in-person training can be a significant barrier to uptake, adoption, and dissemination of EBPs [53,54]. Moreover, other advantages of a technology-based training tool exist that could diminish implementation barriers raised by CFIR [32]. First, it eliminates variability across trainings since the tool is the same for all learners (ie, intervention characteristics) [32]. Second, a technology-based tool can be accessible at any time of the day (or night) to accommodate the learner and the setting’s scheduling needs (ie, characteristics of individuals and inner setting) [32]. Third, it does not require group-based meetings that could interfere with organizational operations by pulling staff from their typical job responsibilities (ie, inner setting). Finally, a technology-based tool can be accessed on different platforms (phone app, computer) from which learners can choose depending on what is most comfortable and accessible (ie, characteristics of individuals). A technology-based tool can reduce inner setting barriers by limiting interference with daily work responsibilities and by providing opportunities to quickly train new staff.

It is important to note that the results of this study are preliminary, and the sample size was small. Once the tool is fully built, it will be important to evaluate the same participant and implementation outcomes with a larger sample and to diversify the learner pool to include potential facilitators not included here. Fidelity was achieved in this pilot study, but only for two curricular activities. The next step will be to examine fidelity to the full training. Most participants used a computer to complete the training, and this limited feedback about the tool’s utility using the phone app. Increasing app use will offer much-needed information to refine this training approach. Two-thirds of participants reported prior experience providing sexual health information to youth. The broad nature of the question (ie, have you had experience providing sexual health information to youth?) limits our understanding of the kind and type of prior experience. It is possible that inexperienced participants would respond differently to the training. As a next step, once the platform is fully developed, increasing the sample size, implementing a rigorous randomization schedule, and widening the sample to include individuals without prior experience in sexual health will inform generalizability.

Several lessons from the pilot study will guide adaptations as we complete the training tool. Participants request embedded links to connect them with additional information about related topics (eg, sexual trauma) and skill development. In the final tool, we will include supplementary resources and links to key topics. We observed that some participants skipped the videos, and these omissions appeared to diminish knowledge acquisition. We will require all participants to watch the videos before moving on to the next module. Some learners reported missing peer-to-peer interactions during training. For the final tool, we will create a forum for peer-to-peer support and communication.

The COVID-19 pandemic has normalized the use of technology for many activities, including health care delivery, workplace meetings, educational conferences, and university teaching. This pilot study provides strong preliminary evidence that a technology-based tool can effectively train individuals to deliver an EBP with fidelity, and that this implementation strategy is satisfying, appropriate, and sustainable with justice-involved settings. If replicated with a larger sample, this approach addresses a key barrier to adoption and implementation of EBPs.