Exploring Feature Priorities and User Needs in Developing Virtual Study Assistants

Introduction

With the proliferation of generative artificial intelligence (GenAI) tools in recent years, many research teams are exploring their potential applications in the health care field, including medical education, information provision, and disease diagnosis [1,2]. For example, GenAI tools have been used to support health care research training [3]. One underexplored application of GenAI is an AI-based virtual study assistant (VSA), which we define as a conversational and agentic technology capable of supporting participant-facing tasks in clinical research, such as screening, providing information, and facilitating consent. The purposes of this formative research were to (1) explore health science researchers’ perspectives on the development of an AI-based VSA and (2) identify potential features and their priorities for future AI-based VSA prototype development.

Methods

Study Design

Participants were recruited from the University of Washington (UW) and consisted of research investigators and study staff with at least 2 years of experience in human subjects research. A snowball approach was used for recruitment. The individuals first completed a questionnaire (Multimedia Appendix 1) collecting information on their research experience and prior experience with GenAI. One-hour focus groups were conducted to identify a list of potential features for AI-based VSA guided by semistructured questions (Multimedia Appendix 2) about key features and participants’ perceptions related to their use. In the final stage, participants completed follow-up surveys (Multimedia Appendix 3) to assess feature acceptability and preference across studies of varying risk levels.

Quantitative data were analyzed using Microsoft Excel. Descriptive statistics, including frequency and mean (SD), were calculated. Thematic analysis was used to identify potential features in the qualitative interview data [4]. Two researchers coded the transcripts and identified themes. The prioritization of preferred features was evaluated using Borda Count, a ranking-based scoring method.

Ethical Considerations

This study was deemed exempt by the University of Washington Institutional Review Board (#21197) since it involved only minimal risk interviews and surveys. Informed consent was obtained from all participants, and data were recorded without identifiable information. All participants received a $10 gift card for completing the survey and a $25 gift card for participating in the focus group.

Results

A total of 14 respondents completed the pre–focus group survey. Among them, 10 took part in focus groups (n=5 per group). The others could not attend due to scheduling conflicts. Following the focus groups, a follow-up survey was distributed to all pre–focus group survey respondents, and 11 completed it. Focus group participants mostly included faculty members (n=6) or student research assistants (n=3). Participant demographics and work experience are presented in Table 1.

Eight potential features for the AI-based VSA were identified from focus group responses, presented in order of most acceptable to least: (1) translating study documents into other languages, (2) contacting a potential participant to gauge their interest, (3) asking and answering questions about eligibility for the study, (4) scheduling participant interactions, (5) describing the study to participants, (6) answering participant questions about the consent form or study participation, (7) verifying participant understanding of consent, and (8) verifying eligibility. Most features were generally considered acceptable. However, those related to answering participant questions about the consent form or study participation, as well as consent verification, received lower levels of acceptance, particularly in studies involving more than minimal risk (Figure 1).

‎

Discussion

Our findings reveal promising directions for the development of AI-based VSA for human subjects research. Among the potential features, translating study documents into other languages was rated as the most preferred. Features related to participant outreach and eligibility-related questions also ranked relatively high, indicating preferences for tools that reduce administrative burden and improve participant communication. One previous study [5] explored the use of GenAI for translating medical or public health–related documents in the health care sector. Specifically, postediting machine translation was on average 14% faster than translating from the beginning. Only a small proportion of outputs (11%-16%) required no human edits. Approximately half of the words needed to be edited [5]. These findings revealed both the potential and the limitations of such tools in this domain. Future research regarding VSA development could explore the linguistic and cultural considerations. There is limited research on using GenAI for participant outreach and scheduling; however, if such features were to be designed, considerations regarding data privacy and management would be necessary. Furthermore, these features entail agentic abilities beyond conversation, such as cross-checking participant and staff schedules and creating calendar appointments, which may involve greater potential for error than conversation alone. Some specific features such as consent verification or addressing questions regarding the consent form may require careful implementation and oversight, given the crucial ethical implications of informed consent. These options generated the highest rates of “not acceptable” responses, particularly among respondents conducting research in the “more than minimal risk” category.

This study identified potential features and offered preliminary observations regarding the development of a prototype. Limitations include the small, single institution sample, which may reduce the generalizability of findings, and the administration of a follow-up survey to all participants, that is, those who attended focus groups and those who did not, as responses from attendees may be biased by having attended a focus group. More formative work may be required to further validate and refine these findings, including engaging diverse potential participant populations and conducting iterative prototype development and usability assessments.