Food insecurity in the United States has been studied in many contexts; more recently, there has been a focus on how it affects college students due to recognition of its prevalence in this population. Those enrolled in college are more likely to be food insecure than the average US household, with some studies finding rates of food insecurity as high as 30%, in comparison to the national average of 10.5% [1]. Additionally, because the cost of attending college has increased dramatically and the minimum wage has not risen in concordance with the increased cost of living, working to pay for college and to live is now not feasible for many students [2].

Food insecurity has been associated with poor academic performance, in addition to poorer psychological and physical health [1,3]. There are many coinciding problems that typically accompany food insecurity as well, such as housing insecurity, less access to health care, and other socioeconomic difficulties [3-5]. The use of food assistance programs by college students is less than the nonstudent population due to multiple factors, including lack of self-perceived risk of food insecurity, lack of accommodations and convenience of certain programs, and systemic regulations that impede some college students from accessing federal nutrition assistance programs [2,4,5]. However, there has been more discourse in the political and public health spheres surrounding food insecurity, which may increase students’ access to federal assistance. For example, a law was recently passed in New Jersey that has increased the minimum monthly Supplemental Nutrition Assistance Program (SNAP) benefit from US $50 to US $95, which has addressed the gap that would have been left behind when the pandemic-era benefits were discontinued [6].

The COVID-19 pandemic has affected college students in different ways. Shifting to remote classes and closing campuses resulted in many students changing their residence [1]. However, the more impactful change was the closure of businesses or the reduction of working hours. The COVID-19 pandemic caused 19.7% of college students to experience a reduction in employment after the start of the pandemic, and 18-to-24–year olds had the highest percentage of unemployment among working-age people after the start of COVID-19 [4,7]. These effects were deleterious for those who were already at risk for food insecurity or experiencing food insecurity; yet, only about 4% of eligible college students use nutrition assistance programs [7]. Mitigating factors for students who were at risk of being food insecure or were food insecure before the pandemic include increased unemployment benefits, federal aid available to students through the CARES Act, and moving back in with parents or family [1]. Students who did not relocate for college and were responsible for providing financial support for their family before and during the COVID-19 pandemic were more likely to experience worsened food security [8].

Current enrollment in SNAP among eligible college students is estimated to be around 30%, providing assistance to 2.26 million students out of 7.3 million who are eligible and in need of food assistance [9,10]. This percentage of eligible students using SNAP has decreased dramatically since the 1990s [2]. Barriers that may explain this reduction in enrollment include a lack of knowledge about requirements and the application process, unnecessarily intricate bureaucracy around the initial application and reapplication of benefits, and strict rules that specifically target college-aged students [2]. The perceived insufficient need is another barrier to applying for students; believing they are ineligible or that they are able to get enough food through other means can prevent them from trying to apply at all. There is also significant stigma surrounding food assistance programs, not limited to SNAP [11]. Many eligible people who are not enrolled in social assistance programs discuss the shame that surrounds experiencing poverty and needing federal or state-funded assistance, in addition to the sense of pride and need to be self-reliant being damaged by enrolling in government-funded programs [11].

Facilitators to college student enrollment in SNAP largely revolve around increased support available on and near campus for learning about the application process and providing tools to support the application process. Essentially, providing support and information about the program and its application process and increasing awareness of eligibility requirements to combat perceived insufficient need, alongside destigmatizing the use of government assistance programs, should help increase SNAP enrollment among eligible college students [12]. Other screening tools for New York residents have been developed both by independent organizations—such as the Community Service Society of New York’s screener and Hunger Solutions NY [13] and by public offices, such as NYC’s ACCESS tool [13-15]. However, these tools do not target college students specifically and often do not offer information about student-specific eligibility requirements, highlighting a gap in available resources for a population that experiences food insecurity at higher-than-average rates.

Guided by formative research and user-centered design approaches, an educational web-based tool prototype that provides information about the SNAP application process and a brief SNAP eligibility screener was developed. The objective of this study was to evaluate the perceived usability and acceptability of this web-based tool among urban college students.

This mixed methods study was conducted from September to December 2022 at a 4-year public college in New York City. Two sessions of usability testing were conducted using a standardized think-aloud method. Both qualitative and quantitative data were collected.

The protocols of this study were reviewed by the City University of New York Human Research Protection Program. The study was determined to not be defined as research, per institutional guidelines. Therefore, formal institutional review board approval of its procedures was not needed (protocol 2022-0508). However, oral consent was still obtained from the participants prior to the initiation of any study procedures. Participants also had the ability to opt out of the study at any point in time. Compensation was provided for study participation. Specifically, participants received up to US $30 in Amazon gift cards (US $10 and US $20 for round 1 and round 2, respectively) upon completion of the usability testing sessions. All analyses were conducted with deidentified data.

English-speaking college students aged 18 to 25 years were recruited to participate in 2 rounds of usability testing to provide feedback and identify problems to help inform the development of the web-based tool. Participants were recruited via flyers placed in communal spaces of campus buildings and emails that were sent to students through departmental email lists, which contained a digital copy of the flyer and sign-up link. The flyers contained general study information, including study purpose, compensation, location, eligibility, and contact information, and a QR code linking to the screening questionnaire. Prospective participants were able to access a self-administered screening questionnaire to screen for eligibility. Inclusion criteria also included students who were at risk for food insecurity or who answered yes to not having enough money for food within the past 30 days. A total of 50 potential student participants completed the screening survey, and 33 (66%) were found to be eligible. However, only 12 agreed to participate in usability testing sessions, which is an expected sample size for such types of studies [16-21]. The remaining 21 students became uncontactable, declined to participate in the study, or failed to schedule a usability testing session.

The development of this screening tool prototype was guided by constructs from the social cognitive theory and the theory of planned behavior. The social cognitive theory characterizes learning as a process that is affected by dynamic interactions between the individual, environment, and behavior [22]. The theory of planned behavior posits that intention to perform a behavior is influenced by attitude toward the behavior, normative and subjective beliefs, and perceived behavioral control [23]. The contents of the web-based tool were developed to address the constructs of behavioral capability, outcome expectations, attitudes, subjective norms, perceived control, and self-efficacy. The focus of the tool was to provide information to clarify the intricacies of the SNAP application process and empower college students to submit applications so that more eligible students would receive and use SNAP benefits.

The design of this web-based tool was grounded in user-centered design approaches to create a convenient, appealing, and straightforward website to promote potential engagement [24,25]. Development began with defining the target audience and constructing prototypical profiles of potential college student users, which generated ideas on what expectations users would have when using an informative SNAP website. The research team also analyzed other similar websites, including government-affiliated SNAP websites and other SNAP eligibility screening tools to identify gaps in service to urban college students. This formative research found that while existing tools provided varying levels of detail about SNAP for the general adult population, there was a lack of easily accessible information specific to the college student population. Furthermore, during usability testing, participants were encouraged to provide solutions to perceived usability issues, facilitating user input to inform the continual development of the tool.

SNAP For-U was developed as an educational web-based tool, offering detailed information about the SNAP application process and a quick SNAP eligibility screener (Multimedia Appendix 1). The website contained 3 text-based informative content pages, in addition to the more visual-focused home page (Figure 1). The home page included colorful imagery, positive messaging, facts, and testimonials from college students. These were included to address the stigma surrounding SNAP use by showing perspectives that oppose normative beliefs on self-reliance and to show how SNAP can have a positive impact on a recipient’s livelihood. Other pages contained more detailed text with referrals to resources to assist with applying for SNAP and information to questions a college student may have when applying, which aimed to provide instruction and guidance. An additional page was added to provide other food assistance resources for users who choose not to apply or are found to be ineligible for SNAP. These pages objectively lay out information about SNAP to avoid associations of shame and other negative perceptions. The SNAP screening tool consisted of questions relating to eligibility criteria assessed on the SNAP application, including questions specifically pertaining to the additional eligibility restrictions students often face. The results page presents an estimation of eligibility and monthly benefit amount. This screening component aims to enhance self-efficacy by serving as a preview to the SNAP application and improve outcome expectations as receiving a favorable outcome on the results page could encourage students to engage in the perceived riskier and more time-consuming process of submitting a SNAP application. The highlighted eligibility factors and application guidance also convey the aspects of the SNAP application that are within or outside of the applicant’s control to increase agency and manage expectations and perceived behavioral control.

A total of 12 students participated in usability testing sessions. Round 1 consisted of 8 in-person sessions and 4 remote sessions via Zoom. Participants were asked to maintain the same modality for round 2 sessions, but 1 participant switched from in-person to remote due to a scheduling conflict. Demographic characteristics of the study sample are summarized in Table 1.

The mean age of participants was 21.8 (SD 2.8) years, and almost all of the participants (11/12, 92%) identified as female. Participants were from a number of cultural backgrounds, with the majority (10/12, 83%) being of an ethnicity other than White. Furthermore, in addition to English, the majority of participants (8/12, 67%) spoke other languages. Half of the participants reported annual household incomes within the range of US $30,000 to US $49,999. Of 12 participants, only 3 (25%) participants attested to currently receiving SNAP benefits. However, 8 (67%) participants answered yes to having ever received SNAP benefits. In comparison, the majority of participants (9/12, 75%) scored low or very low for food security according to the 6-Item Food Security Scale [38].

All participants used smartphones regularly to access the internet, and nearly all (10/12, 83%) also used a desktop computer or laptop. During usability testing, 5 participants chose to test the web-based tool using an iPad, 4 used a laptop, and 3 used their smartphone. Participants used the same type of device for both round 1 and round 2.

Overall, SUS scores in both round 1 and round 2 (median 91.25, IQR 82.50-95; median 92.50, IQR 81.25-96.09, respectively) indicate high usability. When compared to acceptability ranges as presented by Bangor et al [39] (below 50 is unacceptable, 50 to 70 is marginally acceptable, and over 70 is acceptable), the overall SUS scores in both rounds indicate that this tool is acceptable in usability. In comparison to a school grading scale, the scores from both rounds can be interpreted as a B for scores between 80 and 89 and an A for scores 90 and 100 [39]. Participants’ responses to their likelihood to recommend the tool were found to be high in both round 1 and round 2 (mean 8.83, SD 2.1; mean 9.33, SD 1.4, respectively; P=.25), indicating good acceptability of the tool.

The objective of this study was to evaluate the perceived usability and acceptability of an informative web-based tool prototype to encourage SNAP enrollment among urban college students. Through 2 rounds of usability testing, the evaluation of the tool generated overall positive user experiences and further considerations to modify and add to the tool to improve usability and increase potential engagement and impact.

Determining effective modalities to address food insecurity in younger disadvantaged populations, such as college students, continues to be a public health concern. Literature indicates that younger populations increasingly look toward the internet as a trusted resource for health information compared to traditional media [40,41]. However, common difficulties associated with accessing health-related information include determining reliability and the ability to find information of interest [42]. Past studies have also identified that key barriers to college students accessing food assistance programs, such as food pantries or governmental benefit programs, include social stigma, unclear program information, and difficult application processes or other administrative burdens [2,12,43,44]. This study applied a college student–centric approach to create an informative web-based tool, which could alleviate obstacles that food-insecure students face when applying for SNAP. The features and design of SNAP For-U reflect previous findings in which students have suggested the need for greater awareness, positive messaging, and access to information, along with preferences for resources that appear credible, contain quality information, and are easy to use [42,43,45]. Creating a web-based tool that fulfills the information-seeking criteria sought by young adult audiences can increase engagement, leading to increased self-efficacy in facilitating learning and modulating behavior intentions.

In both rounds, we observed high usability and acceptability through user interactions, verbal feedback, and the SUS questionnaire. Participants consistently expressed approval for the concise wording and organization of the tool’s informational sections. The inclusion of multiple methods to access the screening tool throughout the website was effective in directing participants to the screening tool, the main component within the overall tool. Although it is suggested that redundancy in website design can negatively increase cognitive load for users, participants expressed that some redundancy provided the benefits of better visibility for important features and key information [46,47]. This complements previous research that found that redundant user interface design improves speed and accuracy in using the product for new users with moderate to high familiarity with technology [48]. A few participants compared the visual and informational presentation of this study’s tool to government-affiliated websites or other similar websites, conveying that the tool’s streamlined format with attractive imagery was preferable, whereas government-affiliated websites can sometimes feel “overwhelming.” This is consistent with findings from Lee et al [49], where the textual presentation of information on government websites was associated with higher information overload, which had a negative relationship with perceived usefulness [49]. Furthermore, evidence also suggests a direct relationship exists between perceived usefulness or usability and trust in government websites [50,51].

While some participants felt that the tool offered sufficient information in round 1, others recommended adding more information to provide a complete guide to the process. Most participants also experienced difficulties with navigation and difficulties with the screening tool. Adding an application guide, improving navigation features, and updating the screening tool with clarified wording and an added calculator tool aimed to address these issues. In round 2, all participants were able to navigate through the website with ease, with only a few participants providing additional minor considerations to further improve the clarity of navigation. While a discrete “home page” button was not added for round 2, probing revealed that most participants felt that it is common for website logos to also function as a home page button. However, participants still preferred having an additional button stating “home” or “home page” to eliminate the need to guess. After testing in round 2, when asked, most users expressed that the tool was sufficient in providing a thorough and accurate overview of the SNAP application process. However, some participants mentioned that presenting key information, such as eligibility criteria, in multiple locations would provide better visibility and emphasize the most important information about SNAP. The questions in the screening tool were noted to be comparable to those of the actual SNAP applications. Moreover, the resulting estimated benefit amount was perceived as particularly encouraging as it provided a quantified expectation. Usability testing underscored the importance of providing adequate instruction to decrease the effort needed to learn and effectively use the web-based tool. Participants often preferred specific and detailed but concisely worded information instead of generalized information to better address the varying knowledge levels between different users. Appealing visual design was also an important aspect that mediated the perception of credibility, usability, and usefulness. As usability scores were high to begin with, differences between round 1 and round 2 usability scores were not statistically significant. However, participants’ feedback on the revisions further reaffirmed positive perceptions of the tool as appealing, easy to use, and effectively providing support for the SNAP application process.

Overall findings reinforce the value of usability testing with representative end users. Despite tool development being informed by user-centered design principles and iterative testing within the research team, study participants highlighted several issues and presented various future considerations that did not emerge during internal testing. Of note, sections of the SNAP screening tool had to be revised in consideration of the perceived difficulty of questions and the limited baseline SNAP knowledge of potential end users. Through a combination of qualitative and quantitative methodology to conduct usability testing, a number of usability issues along with potential user-suggested solutions were able to be discovered.

Although this mixed methods approach to usability testing produced extensive and valuable insights from student testers to inform necessary tool improvements, this study is not without limitations. While the study sample size is typical for usability testing, our results may not be generalizable to the overall target audience of the tool and almost all participants self-identified as female [16-21]. This may have led to the underrepresentation of nuances in web-related user experiences from perspectives of other genders.

A common limitation of think-aloud procedures is the risk of social acceptability bias due to the presence of 2 researchers during usability testing. Participants may have felt compelled to verbalize positive feedback or may have interacted with the tool differently than in a natural setting. Another consideration is that participants may have perceived the round 2 prototype as more usable or easier to learn due to their previous experience during round 1. However, maintaining the same participants for both rounds was necessary to make direct comparisons of both iterations of the tool. Finally, the mixed modalities of participation may have contributed to varying amounts and qualities of information captured in each usability testing session. In-person sessions allowed for better documentation of a participant’s body language. Conversely, remote sessions enabled researchers to observe the entirety of a participant’s interactions with the tool through screen sharing.

The findings from this study will inform further development of this tool. However, the tool’s usability and acceptability should also be tested with other cohorts of college students who may be at increased food insecurity risk and pilot-tested to determine potential effectiveness. Future expansions of this tool may include considerations for language and other accessibility features, additional connected support resources, and modifications for other at-risk populations or public assistance programs, such as young pregnant women and Special Supplemental Nutrition Program for Women, Infants, and Children. Our study also provides broader guidance for web-based tool development of other similar public health-orientated interventions for young and technologically proficient audiences.

Usability testing is vital to the development of web-based tools to determine the usefulness of a tool and to gain insights to further enhance its effectiveness, usability, and engagement among its end users. The researchers engaged representative student participants using a standard combined think-aloud approach, gaining immediate impressions during the testing of the tool and retrospective feedback contextualized by the users’ overall experiences. These perspectives will help to refine the tool in its next iterations. Overall usability findings suggest that this web-based tool was highly usable and acceptable among urban public college students and thus could be an effective and appealing approach as a support tool to introduce college students to the SNAP application process. This tool could eventually be disseminated across various urban college campuses and adapted for other localities or nonurban regions to encourage SNAP use among college students.