This paper is in the following e-collection/theme issue:

Early Results from COVID-19 Studies (299) Registered Report (560) JMIR Theme Issue: COVID-19 Special Issue (2452) Outbreak and Pandemic Preparedness and Management (1727) COVID-19 Vaccination (146) Vaccination Sentiment and Anti-Vaccination Infodemiology (58) Vaccination and Immunization in the Digital Age (374) Vulnerable Populations in Health Research (99)

Published on in Vol 6, No 8 (2022): August

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/38351, first published .
Knowledge, Attitude, Practices, and Vaccine Hesitancy Among the Latinx Community in Southern California Early in the COVID-19 Pandemic: Cross-sectional Survey

Knowledge, Attitude, Practices, and Vaccine Hesitancy Among the Latinx Community in Southern California Early in the COVID-19 Pandemic: Cross-sectional Survey

Knowledge, Attitude, Practices, and Vaccine Hesitancy Among the Latinx Community in Southern California Early in the COVID-19 Pandemic: Cross-sectional Survey

Authors of this article:

Shivani N Mehta1, 2 Author Orcid Image ;   Zoe C Burger1 Author Orcid Image ;   Stephanie A Meyers-Pantele3 Author Orcid Image ;   Richard S Garfein4 Author Orcid Image ;   Dayanna O Ortiz1 Author Orcid Image ;   Pavan K Mudhar2, 5 Author Orcid Image ;   Smit B Kothari2, 6 Author Orcid Image ;   Jigna Kothari2 Author Orcid Image ;   Meena Meka2 Author Orcid Image ;   Timothy Rodwell1 Author Orcid Image
Article Authors Cited by (4) Tweetations (2) Metrics

    Original Paper

    1San Diego School Of Medicine, University of California, La Jolla, CA, United States

    2Amistad Medical Clinic, Santa Ana, CA, United States

    3Department of Psychology, San Diego State University, San Diego, CA, United States

    4Herbert Wertheim School of Public Health, University of California, San Diego, San Diego, CA, United States

    5Department of Audiology, Arizona School of Health Sciences, Mesa, AZ, United States

    6University of Rochester, Rochester, NY, United States

    Corresponding Author:

    Shivani N Mehta, BA

    San Diego School Of Medicine

    University of California

    9500 Gilman Dr #0039

    La Jolla, CA, 92093

    United States

    Phone: 1 7146790080

    Email: s1mehta@ucsd.edu


    Background: The Latinx population in the United States has experienced high rates of infection, hospitalization, and death since the beginning of the COVID-19 pandemic. There is little data on the knowledge, attitude, and practices (KAP) specifically in Latinx communities in the United States.

    Objective: We aimed to assess COVID-19 KAP and vaccine hesitancy among a Latinx cohort in the early stages of the COVID-19 pandemic (from July 2020 to October 2020), at a unique time when a vaccine was not available. 

    Methods: Participants aged ≥18 years were recruited at a primary care clinic in Southern California and asked to self-report sociodemographic characteristics, KAP, and vaccine hesitancy. A subset of the participants answered the vaccine hesitancy assessment as it was added after the start of data collection. KAP items were summed to create composite scores, with higher scores reflecting increased COVID-19 knowledge, positive attitudes toward the COVID-19 pandemic, and disease prevention practices. Bivariate and multivariable regression models were fitted to test associations between sociodemographic characteristics and KAP scores. For our analysis, we only included patients who self-identified as Latinx.

    Results: Our final data set included 265 participants. The participants had a mean age of 49 (IQR 38.5-59) years, and 72.1% (n=191) were female, 77% (n=204) had at most a high school degree, 34.7% (n=92) had an annual income <US $25,000, and 11.7% (n=31) had previously tested positive for COVID-19. We found high knowledge regarding transmission and spread; moderate knowledge regarding symptoms awareness; overall negative attitudes, which included high pessimism in government public health efforts and high amounts of fear, anxiety, and frustration due to COVID-19 pandemic; and moderate participation in preventive practices. A college education was positively associated with a higher knowledge score than those without a college education (β=0.14, 95% CI 0.01-1.60; P=.04) when adjusted for covariates. Male gender had a positive association with COVID-19 attitude scores compared to female gender (β=1.61, 95% CI 0.50-2.72; P=.05), and male gender was negatively associated with the COVID-19 practices score compared to female gender (β=–0.16, 95% CI –0.56 to –0.06; P=.03), when both were adjusted for covariates. Among a subset of 203 patients, 26.6% (n=54) indicated that if the vaccine was available, they would not take a COVID-19 vaccine, and 18.7% (n=38) were unsure.

    Conclusions: Good knowledge and preventative practices in the population may have reflected effective public health messaging and the implementation of public health laws during the first wave of the pandemic; however, the overall fear and anxiety may have reflected the negative impact that the pandemic had on vulnerable populations such as the Latinx community. Although our data are a reflection of a previous time in the pandemic, we believe it captures a critical time that can be used to provide unique insights regarding potential avenues to better protect the Latinx communities against future vaccine-resistant COVID-19 strains.

    International Registered Report Identifier (IRRID): RR2-10.2196/25265

    JMIR Form Res 2022;6(8):e38351

    doi:10.2196/38351

    Keywords

    COVID-19; knowledge; attitude; practices; KAP survey; vaccine hesitancy; Latinx; Latinx cohort; minority population; primary care; sociodemographic characteristic; public health; vulnerable population; epidemiology 


    In early 2020, the global community was overwhelmed by the COVID-19 pandemic caused by the SARS-CoV-2 virus, leading to quickly rising incidence and death rates with no available vaccine. By December 2020, COVID-19 had infected almost 20 million people and caused more than 300,000 deaths in the United States [1]. As cases of COVID-19 climbed across the United States throughout 2020, racial/ethnic minority populations were more likely to be infected, experience complications, and die from COVID-19 than the general population during a time when the vaccine was not available [2]. Although Latinx populations made up only 38.9% of the total population in California, they accounted for almost 60% of COVID-19 cases and approximately 50% of COVID-19–related deaths during the early pandemic in 2020 [3].

    In 2021, the Moderna and Pfizer vaccines were made accessible to thousands of Latinx individuals in California [4], especially due to outreach and advocacy by organizations such as the Latin Community Foundation, Hispanic Federation, and the Orange County Public Health Department [5-7]. Despite this important milestone in the pandemic, the Latinx, Black, and American Indian or Alaska Native communities had similarly slow rates of vaccine uptake throughout 2020 and 2021 compared to Asian and White populations. As expected, Black and Latinx communities reported increased vaccine hesitancy compared to White and Asian populations in the United States [8,9].

    The Latinx community remained one of the most vulnerable groups in terms of infection and death rates compared to the rest of the state population in the emergence of the new Omicron variant [10].

    As we enter the third year of the COVID-19 pandemic, we still have not fully investigated the risk factors that predisposed the Latinx population in California to significantly higher incidence rates in the first year of the pandemic compared to other communities. In addition to the greater lack of access to care in the Latinx community, there are many other structural disparities that contribute to decreased health equity and increased risk behaviors in this population [11-14]. The Knowledge, Attitude, and Practices (KAP) theory is a health behavior theory based on the belief that knowledge is the foundation of behavior change and helps form attitudes and practices that drive behavior change [15,16]. Numerous global studies from the beginning of the pandemic showed that a lack of COVID-19 knowledge, negative attitudes toward the pandemic, and the lack of disease preventative practice were more common in communities with high disease rates [14,17-21]. Further, vaccine hesitancy has been a persisting issue for the last few decades during the presence of numerous pandemics, especially within minority populations [22,23]. In the beginning of the COVID-19 pandemic, few studies measured the vaccine hesitancy of a potential COVID-19 vaccine before the vaccine was made available [24,25]. It is possible that if this data was measured early on, there would be more information to assist public health programs to combat vaccine misinformation and mistrust at the start of the vaccine release to the public [26]. There have been a small number of other studies that have explored the differences in KAP between different ethnic groups and genders in the United States, but none have specifically focused on the Latinx population.

    In this study, we sought to investigate COVID-19 KAP and vaccine hesitancy among a Latinx population in Southern California. Specifically, we aimed to capture information on KAP and vaccine hesitancy from a critical time early in the COVID-19 pandemic when the vaccine was not available. The findings of this work provide unique insights regarding vulnerabilities and potential avenues for better protecting Latinx communities in future pandemics and against new vaccine-resistant COVID-19 strains, such as the currently circulating Omicron strain.


    Study Design

    We conducted a cross-sectional study from July to October 2020 in Southern California, during the “stay at home” order period that was implemented by the state government of California. We carried out our surveys at 2 primary health care centers in Santa Ana and Anaheim, California. Using consecutive sampling methods, we predominately recruited and surveyed patients presenting for telehealth appointments but also recruited patients presenting for in-person appointments at the clinics. Patients were eligible for study participation if they were (1) aged ≥18 years and (2) an enrolled patient of Amistad Medical Clinic Santa Ana or Amistad Medical Clinic Anaheim. For this analysis, we only included participants who self-identified as Latinx. Our study design, which included in-person and telemedical recruitment, and data collection methods have been previously described [27].

    Survey Validation

    Our KAP survey was adapted from the World Health Organization’s (WHO) Guide to Developing Knowledge, Attitude, and Practice Surveys [15], in addition to the few existing global COVID-19 KAP studies from the time period [28-30]. The most current COVID-19 symptoms included in the knowledge survey were identified using Centers for Disease Control and Prevention symptom reports that were the most current at the time of data collection. No standardized questionnaire for COVID-19 KAP had been created at the time of the study due to the early nature of the pandemic. Therefore, we pretested our KAP survey among 10 participants before implementing it.

    Questions about vaccine hesitancy were added to the survey after the start of data collection. The vaccine hesitancy questions were adapted from the Vaccine Hesitancy Scale by WHO’s Strategic Advisory Group of Experts on Immunization [31]. The survey was pretested among 10 participants before including it in our data collection.

    Measures

    Sociodemographic Characteristics

    We asked participants to self-report sociodemographic characteristics including age (in years); gender (male vs female); education level completed (elementary school, middle school, high school, college degree, or graduate degree); annual income (less than US $25,000, US $25,000 to US $49,999, US $50,000 to US $99,999, and US $100,000 or more); marital status (single/never married, married, divorced, separated, or widowed); ethnicity (Latinx vs non-Latinx); and employment status (currently employed, recently unemployed or on temporary leave due to COVID-19, unemployed before COVID-19, retired, or a student). We also asked participants to self-report whether they had ever tested positive for COVID-19 (yes vs no).

    COVID-19 KAP Survey

    The COVID-19 KAP survey was comprised of 3 parts: (1) COVID-19 knowledge, (2) COVID-19 attitudes, and (3) COVID-19 practices. Knowledge included questions about disease transmission and spread through 12 dichotomous (yes vs no) questions and 1 open-ended question. It also measured knowledge of the most common COVID-19 symptoms through a multiple mark–style question in which participants were able to endorse 1 or more symptoms. All items were then summed to create a composite knowledge score ranging from 0 to 12, with higher scores indicating greater COVID-19 knowledge. Correct answers were based on current COVID-19 data at the time that was disseminated by the WHO to the general public [32].

    COVID-19 attitudes were assessed through 5 items regarding confidence in the government control of the pandemic and emotions about the pandemic. All items were measured on a 5-point Likert-type scale ranging from 1 (strongly disagree) to 5 (strongly agree) and were summed to create a composite attitude score, ranging from 5 to 25, with lower scores indicating more negative attitudes. COVID-19 practices consisted of 2 items that assessed participants’ engagement in behaviors that reduce COVID-19 transmission. Both items were measured using a 5-point Likert-type scale ranging from 1 (never) to 5 (always) and were summed to create a composite practices score, ranging from 2 to 10, with higher scores indicating greater engagement in COVID-19 preventative behaviors.

    COVID-19 Vaccine Hesitancy

    Additionally, a question regarding vaccine hesitancy was added to the study after the start of data collection, and a subset of participants provided information on their willingness to take a potential COVID-19 vaccine through their responses (yes, maybe, or no) to the following question, “If a COVID-19 vaccine was available, would you get one?” Those who indicated “no” or “maybe” to this question were then asked to report their reason for their vaccine hesitancy. Response options included “I don’t think it would work,” “I’m concerned about the possible side effects,” “I’m worried about the cost,” “I don’t think it’s necessary,” and “Other people will get it so I won’t need to.”

    Statistical Analyses

    We performed descriptive analysis for participant sociodemographic characteristics and COVID-19 vaccine hesitancy. Bivariate linear regression models were fitted to test associations between sociodemographic characteristics and COVID-19 KAP scores. Multivariable linear regression analyses were then conducted to further test the associations between sociodemographic characteristics and COVID-19 KAP scores while controlling for covariates found to be associated with the variables of interest in the bivariate analyses (P<.05). As determined a priori, variables associated with COVID-19 KAP in the bivariate analysis were retained for inclusion in the multivariable analysis. In addition, other sociodemographic variables previously found to be associated with COVID-19 KAP in the literature (ie, age, education, gender, income, gender, and marital status) were also included as covariates in the multivariable models [33-35]. All analyses were conducted in SPSS statistical software (version 9.8.0; IBM Corp).

    Ethical Approval

    The study was exempted from review by the University of California, San Diego Institutional Review Board on June 22, 2020. Patients’ names or other identifying information were not collected. Participants were informed of the confidential nature of the study and were allowed to terminate the survey at any time.


    Demographics

    A total of 323 participants completed the COVID-19 KAP survey. Our final data set included 265 participants after 58 individuals who did not self-identify as Latinx were excluded from analysis. Among the 265 participants, the mean age was 49.3 (IQR 38.5-59) years, and 191 (72.1%) identified as female, 139 (52.5%) were married, 109 (41.1%) indicated they had a middle school education or below, 89 (33.6%) had lost their job or were on temporary leave due to COVID-19, 92 (34.7%) had an annual income less than US $25,000, and 31 (11.7%) reported a current or previous positive test for COVID-19 (Table 1).

    Table 1. Sociodemographic and clinical characteristics among Latinx clinic attendees in Southern California (N=265).
    Variable, categoryParticipant
    Gender, n (%)

    		Female191 (72.1)

    		Male74 (27.9)
    Age (year), mean (IQR)49.3 (38.5-59)
    Marital status, n (%)

    		Single/never married84 (31.7)

    		Married139 (52.5)

    		Divorced10 (3.8)

    		Separated19 (7.2)

    		Widowed11 (4.2)

    		Refused to answer2 (0.8)
    Highest educational level, n (%)

    		Middle school or below109 (41.1)

    		High school95 (35.8)

    		College degree36 (13.6)

    		Graduate school4 (1.5)

    		Refused to answer21 (7.9)
    Employment status, n (%)

    		Currently employed59 (22.2)

    		Lost job or on temporary leave due to COVID-1989 (33.6)

    		Unemployed currently and before COVID-1974 (27.9)

    		Retired40 (15.1)

    		Student2 (0.8)

    		Refused to answer1 (0.4)
    Annual household income (US $), n (%)

    		<25,00092 (34.7)

    		25,000-49,99930 (11.3)

    		50,000-99,99918 (6.8)

    		≥100,0007 (2.6)

    		Refused to answer118 (44.5)
    Previously tested positive for COVID-19, n (%)

    		Yes31 (11.7)

    		No234 (88.7)

    COVID-19 Knowledge

    Of the total 265 participants, 254 (95.8%) correctly knew that COVID-19 is spread when an infected person coughs, sneezes, or speaks; 251 (94.7%) correctly knew that COVID-19 is spread through touching contaminated surfaces and then touching the eyes or mouth; and 243 (91.7%) correctly knew to stand 6 feet or more away from another person to be safe from COVID-19 (Table S1 in Multimedia Appendix 1). However, 212 (80%) participants incorrectly believed that only older individuals (aged >65 years) are susceptible to COVID-19. Additionally, 196 (74%) participants correctly knew that individuals with pre-existing medical conditions are more susceptible to COVID-19, and 224 (84.5%) correctly identified that asymptomatic individuals with COVID-19 can still infect someone else. When asking participants for common symptoms of COVID-19, 171 (64.5%) correctly identified fever, 111 (41.9%) correctly identified cough, 109 (41.1%) correctly identified headache, 100 (37.7%) correctly identified shortness of breath, 91 (34.3%) correctly identified body aches, 46 (17.4%) correctly identified loss of taste, and 34 (13.6%) correctly identified loss of smell.

    In the multivariate analysis, when adjusting for age, gender, annual household income, and education, having a college degree was significantly associated with a higher knowledge score than those who completed middle school or below (β=0.14, 95% CI 0.01-1.60; P=.04; Table 2).

    Table 2. Bivariate and multivariable analysis of factors associated with COVID-19 knowledge among Latinx individuals in Southern California (N=265).
    FactorBivariate analysis, β (95% CI)P valueMultivariate analysis, aβ (95% CI)P value
    Age–0.19 (–0.04 to –0.01).002–0.01 (–0.03 to 0.01).26
    Gender

    		Female (refa)—b———

    		Male–0.04 (–0.34 to 0.72).300.06 (–0.53 to 0.56).95
    Marital status

    		Married (ref)————

    		Single0.01 (–0.51 to 0.55).93——

    		Divorced0.09 (–0.24 to 1.63).14——

    		Widowed–0.10 (–2.21 to 0.18).09——

    		Separated0.01 (–1.18 to 1.32).91——
    Annual household income (US $)

    		<25,000 (ref)————

    		25,000-49,9990.07 (–0.28 to 1.19).230.01 (–0.69 to 0.83).87

    		50,000-99,9990.14 (0.17-2.03).020.09 (–0.28 to 1.69).19

    		≥100,0000.13 (0.15-3.06).030.12 (–0.14 to 3.09).08
    Education

    		Middle school or below (ref)————

    		High school0.26 (0.23-1.23).0050.11 (–0.13 to 1.01).25

    		College degree0.36 (0.47-1.89).0010.14 (0.01-1.60).04

    		Graduate degree0.96 (0.46-4.24).020.12 (–0.05 to 3.92).07
    Employment status

    		Lost job or on temporary leave due to COVID-19 (ref)————

    		Currently employed0.08 (–0.24 to 1.03).22—.24

    		Unemployed currently and before COVID-19–0.01 (–0.63 to 0.55).89—.57

    		Retired–0.18 (–1.68 to –0.24).009—.53

    		Student–0.06 (–4.13 to 1.27).30—.19
    Previously tested positive for COVID-19

    		No (ref)————

    		Yes0.29 (–0.56 to 0.90).69——

    aRef: reference.

    bNot available (item was not accessed).

    COVID-19 Attitudes

    Among the 265 total participant, 104 (39.2%) strongly disagreed or disagreed that the government would help stop the spread of the virus; 198 (74.7%) strongly agreed or agreed that they felt nervous about how the COVID-19 pandemic would impact their future; 120 (45.3%) strongly agreed or agreed that they had felt angry or frustrated because of the COVID-19 pandemic; and 147 (55.5%) strongly agreed or agreed that they had felt scared to leave their home because of COVID-19. However, a majority 72.5% (n=192) of the participants strongly agreed or agreed that they had felt hopeful about the future (Table S2 in Multimedia Appendix 1).

    In the multivariate analysis, when adjusting for marital status and education, male gender had a positive association with COVID-19 attitude scores compared to female gender (β=1.61, 95% CI 0.50-2.72; P=.005; Table 3).

    Table 3. Bivariate and multivariable analysis of factors associated with COVID-19 attitudes among Latinx individuals in Southern California (N=265).
    FactorBivariate analysis, β (95% CI)P valueMultivariate analysis, aβ (95% CI)P value
    Age0.06 (–0.02 to 0.07).370.00 (–0.04 to 0.04).96
    Gender

    		Female (refa)—b———

    		Male1.65 (0.58-2.71).0031.61 (0.50-2.72).005
    Marital status

    		Married (ref)————

    		Single–0.15 (–2.33 to –0.18).04–0.13 (–2.31 to 0.08).07

    		Divorced0.56 (–2.74 to 1.04).14–0.03 (–2.45 to 1.44).61

    		Widowed0.07 (–4.35 to 0.71).09–0.07 (–3.99 to 1.09).26

    		Separated–0.09 (–3.95 to 1.11).91–0.07 (–4.00 to 1.06).25
    Annual household income (US $)

    		<25,000 (ref)————

    		25,000-49,999–0.01 (–1.68 to 1.38).85——

    		50,000-99,9990.02 (–1.68 to 2.17).81——

    		≥100,000–0.01 (–3.09 to 2.93).96——
    Education

    		Middle school or below (ref)————

    		High school–0.08 (–1.74 to 0.38).21–0.06 (–1.65 to 0.73).45

    		College degree–0.01 (–1.55 to 1.42).930.00 (–1.64 to 1.68).98

    		Graduate degree0.01 (–3.85 to 4.07).96–0.02 (–4.51 to 3.47).80
    Employment status

    		Lost job or on temporary leave due to COVID-19 (ref)————

    		Currently employed–0.09 (–1.99 to 0.46).96——

    		Unemployed currently and before COVID-19–0.01 (–1.34 to 1.28).22——

    		Retired0.12 (–0.21 to 2.74).09——

    		Student0.48 (–7.73 to 3.55).44——
    Previously tested positive for COVID-19

    		No (ref)————

    		Yes–0.05 (–2.06 to 0.88).43——

    aRef: reference.

    bNot available (item was not accessed).

    COVID-19 Preventative Practices

    Among the 265 participants, when asked about preventative practices to stay safe from COVID-19 transmission, 161 (60.8%) indicated that they washed their hands most of the time when entering their homes, whereas 92 (34.7%) indicated that they never engage in this practice. Interestingly, 248 (93.6%) indicated that they always wear a mask outside of their home, whereas none indicated that they never wear a mask outside their home (Table S3 in Multimedia Appendix 1).

    In the multivariable regression analysis, controlling for marital status, annual household income, and education, male gender was negatively associated with the COVID-19 practices score compared to female gender (β=–0.16, 95% CI –0.56 to –0.06; P=.03; Table 4).

    Table 4. Bivariate and multivariable analysis of factors associated with COVID-19 attitudes among Latinx individuals in Southern California (N=265).
    FactorBivariate analysis, β (95% CI)P valueMultivariate analysis, aβ (95% CI)P value
    Age0.04 (–0.01 to 0.01).54——
    Gender

    		Female (refa)—b———

    		Male–0.65 (–0.56 to –0.08).008–0.16 (–0.56 to –0.06).03
    Marital status

    		Married (ref)————

    		Single0.05 (–0.16 to 0.33).490.05 (–0.17 to 0.34).50

    		Divorced0.07 (–0.18 to 0.67).260.05 (–0.28 to 0.60).47

    		Widowed0.05 (–0.34 to 0.75).460.04 (–0.38 to 0.71).56

    		Separated0.02 (–0.47 to 0.66).740.03 (–0.46 to 0.71).67
    Annual household income (US $)

    		<25,000 (ref)————

    		25,000-49,9990.05 (–0.20 to 0.48).430.04 (–0.24 to 0.46).54

    		50,000-99,999–0.08 (–0.69 to 0.16).22–0.04 (–0.60 to 0.33).57

    		≥100,000–0.01 (–0.72 to 0.62).88–0.02 (–0.80 to 0.60).77
    Education

    		Middle school or below (ref)————

    		High school–0.01 (0.25-0.22).88–0.01 (–1.65 to 0.73).89

    		College degree–0.11 (–0.36 to 0.30).87–0.00 (–1.64 to 1.68).90

    		Graduate degree–0.00 (–0.90 to 0.86).960.01 (–4.51 to 3.47).84
    Employment status

    		Lost job or on temporary leave due to COVID-19 (ref)————

    		Currently employed0.07 (–0.15 to 0.44).32——

    		Unemployed currently and before COVID-190.05 (–0.18 to 0.36).52——

    		Retired0.12 (–0.04 to 0.62).09——

    		Student–0.07 (–1.40 to 1.08).80——
    Previously tested positive for COVID-19

    		No (ref)————

    		Yes0.05 (–0.19 to 0.46).43——

    aRef: reference.

    bNot available (item was not accessed).

    COVID-19 Vaccine Hesitancy

    We asked a subset of 203 participants about their views on a potential COVID-19 vaccine before the COVID-19 vaccine was made available. Within this subset (N=203), 111 (54.7%) indicated that they would take a vaccine if it were available, but 54 (26.6%) indicated that they would not take a vaccine, and 38 (18.7%) indicated that they were unsure (ie, answered “maybe”). Of those who indicated they would not take a vaccine (n=54), 23 (43%) indicated that they were concerned with possible side effects, 18 (33%) indicated that they didn’t think it was necessary, 8 (15%) indicated that they didn’t think it will work, and 1 (2%) indicated that they are worried about the cost and financial burden. Of those who indicated they were unsure (n=38), 27 (71%) indicated that they are concerned about possible side effects, 3 (8%) indicated that they didn’t think it will work, 2 (5%) indicated that they didn’t think it’s necessary, and none indicated that they are worried about cost and financial burden (Table 5).

    Table 5. COVID-19 vaccine hesitancy before vaccine availability (N=203).
    View on vaccine, reasons for hesitancy Participants, n (%)
    Willing to take the vaccine (n=111)111 (100)
    Unsure about taking the vaccine (n=38)

    		Don’t think it’s necessary2 (5)

    		Don’t think it will work3 (8)

    		Concerned about the possible side effects 27 (71)

    		Worried about the cost and financial burden0 (0)

    		Refused to answer6 (16)
    Not willing to take the vaccine (n=54)

    		Don’t think it’s necessary18 (33)

    		Don’t think it will work8 (15)

    		Concerned about the possible side effects 23 (43)

    		Worried about the cost and financial burden1 (2)

    		Refused to answer4 (7)

    Principal Findings

    Since the start of the COVID-19 pandemic in Southern California, the Latinx population has been disproportionately affected by disease incidence, morbidity, and mortality. Our data demonstrate that this population had good knowledge of COVID-19 transmission and spread; moderate knowledge of COVID-19 symptoms; overall negative attitudes including high pessimism in government public health efforts and high amounts fear, anxiety, and frustration due to COVID-19 pandemic; but high optimism about the future. Our data also show moderate participation in preventative practices and moderate vaccine hesitancy within a Southern California Latinx community during the first wave of the pandemic before the vaccine was available. In addition, our data show a significant association between college education and a higher COVID-19 knowledge score; male gender and a more optimistic attitude; and male gender and a lower practices score. We believe that observational data from this period are critical to identifying how a pandemic can initially affect vulnerable populations’ understanding of disease, since knowledge helps form attitudes and practices that drive behavior change [36].

    Comparison to Prior Works

    Participants were particularly aware of the common physical symptoms of COVID-19 illness, such as cough, headache, and fever. Relatively few participants, however, identified loss of taste and smell as common COVID-19 symptoms. The lack of knowledge around these very specific symptoms may have been due to the unusual nature of the symptoms and the public health messaging at the time that was emphasizing “coughing” as an identifiable symptom to be aware of [37,38]. When participants were asked about their knowledge regarding COVID-19 transmission and susceptibility, most knew that transmission could happen when an infected person coughs, sneezes, or speaks and that the disease could be spread by touching contaminated surfaces and then touching the eyes or mouth. Most also knew to stand 6 feet or more away from others to prevent disease transmission. However, our data show that over half of our participants felt that only older people were susceptible to COVID-19 infection.

    Our findings also demonstrate that younger age was associated with lower knowledge scores, a finding supported by Alsan et al [39], which reported that people aged <55 years were less likely to know how the disease is spread and the symptoms of COVID-19. Higher education and income were associated with higher knowledge scores, which is supported by global COVID-19 studies from 2020 that showed higher education and income allowed for greater knowledge regarding the disease [34,40,41].

    When participants were asked about their COVID-19 attitudes, only one-third agreed or strongly agreed that the government would stop the spread of the virus, whereas more than half disagreed or strongly disagreed. Focus groups among Latinx farmworkers in the central valley reported similar themes of mistrust in government institutions regarding COVID-19 vaccinations and testing [42]. This finding is in-line with the Latinx community’s growing mistrust in the American government’s medical and public health policies [43].

    The participants also expressed feelings of fear, anxiety, and frustration. Widespread fear, anxiety, and depression related to COVID-19 has been well-documented in the United States, and recent research has shown that these feelings have been magnified in Latinx populations where the loss of employment has been widespread [44-46]. In our study, nearly one-third of the participants reported COVID-19–related “temporary leave” or lost jobs during the pandemic. This substantial change in employment status may have also contributed to the participants’ reports of feeling angry or frustrated due to COVID-19. However, despite the negative attitudes toward COVID-19, over half of the participants indicated that they were still hopeful about the future. A positive outlook aligns with research that shows that specific Latinx populations have high resilience during stressful situations and carry optimism for the future despite dissatisfaction with the present and the multiple obstacles and stressors they face [47].

    Our data show that men had more COVID-19–related positive attitudes about government response and the pandemic’s effect on their lives compared to women. Our data differ from other European- and South American–based studies that have shown no significant association between gender and COVID-19–related positive attitudes in the Latinx community [48,49]. However, our results could be explained by women having reportedly higher levels of fear and stress and negative perceptions of government actions than men during the pandemic [50]. Multiple studies conducted globally and in the United States have indicated that women have been carrying high burdens of care (eg, caring for and teaching children in the absence of in-person learning environments) and emotional labor [51-53]. Consequently, this gendered division of labor could potentially have contributed to the gender differences in positive attitudes found in our study.

    Additionally, when asked about COVID-19 preventative practices, almost all participants indicated that they always wore a mask when they went outside of their home. This strong knowledge base and the corresponding safety practices regarding COVID-19 may indicate that there was effective public health messaging regarding transmission and preventative practices that had penetrated the community. In addition, local laws possibly contributed to higher compliance with COVID-19 preventative practices, such as wearing masks [54]. Our data indicate that men had decreased preventative practices compared to women. This aligns with a study by Alsharawy et al [50] indicating that women engaged in more preventative practices during the pandemic and were more risk averse.

    Our data captured vaccine attitudes and acceptability between July and October 2020—a period before COVID-19 vaccines were approved for essential workers in December 2020. The most common reasons given for an expressed reluctance to accept vaccination in our study were concern about possible side effects, thinking it was not necessary, and thinking it would not work. After the vaccine was released, studies documented mass vaccine hesitancy in Latinx and Black communities across the United States who had historically faced government oppression and had mistrust in the intentions of public health medical interventions [55,56]. Our findings have built upon these studies and provided further evidence of relatively high vaccine hesitancy in marginalized populations, and we suggest that in future pandemics, more focused effort should be expended on both vaccine education and interventions that are targeted toward combatting medical mistrust in marginalized communities prior to vaccine distribution.

    Limitations

    Our study had several limitations. First, due to the cross-sectional nature of the study, we are unable to report whether the KAP was casually associated with the covariates examined. In addition, we had a limited set of questions due to time constraints in the clinical setting. Lastly, the reliance on self-reporting may have introduced response bias due to participants’ fear of judgement from health care providers regarding COVID-19 practices. To mitigate these challenges, we made sure that our survey administrators were not health care providers, were well-trained, and approached surveys in a nonjudgmental manner and that the participants were informed of the confidential nature of the study and allowed to terminate the survey at any time. Additionally, our vaccine acceptance questions were asked before a vaccine was approved, hence we recognize that these findings may not reflect current attitudes toward vaccines for our population. Additionally, the cross-sectional nature of our study did not allow us to track data over time but rather enabled us to capture data at one point in time. We acknowledge that the situation with COVID-19 has changed rapidly, and therefore, KAP in populations can change over time.

    We believe, however, that our data capture important information related to symptom awareness, knowledge, attitudes, and practice, in addition to vaccine hesitancy during a critical time in the history of the COVID-19 pandemic; this data might provide important insights into the future implementation of critical public health interventions and policies serving Latinx populations.

    Conclusion

    We found high knowledge regarding transmission and spread; moderate knowledge regarding symptoms awareness; overall negative attitudes which included high pessimism in government public health efforts and high amounts of fear, anxiety, and frustration due to the COVID-19 pandemic; and moderate participation in preventive practices. We also found moderate vaccine hesitancy in the Latinx community in Southern California during the first wave of the pandemic. Good knowledge and preventative practices in the population may be a reflection of effective public health messaging and the implementation of public health laws during the first wave of the pandemic; however, the overall fear and anxiety may have been a reflection of the negative impact that the pandemic had on vulnerable populations such as the Latinx community. We believe it would be useful to examine KAP in larger populations during this time period and potentially use the data to inform public health approaches in Latinx communities amid the potential arrival of new pandemics.

    Acknowledgments

    The authors would like to thank all of the participants for their willingness to participate and all study staff for their support. SAMP was supported by National Institute of Drug Abuse grants R01 DA039950 and T32 DA023356.

    Authors' Contributions

    SNM led the project coordination, data analysis, and writing of the manuscript. SAMP and RSG also led the data analysis. ZCB, SNM, and TR led the proposal and protocol development. DOO, PKM, SBK, JK, and MM contributed to the clinical and logistical aspects of protocol development. All authors have approved the final manuscript and agreed to publication.

    Conflicts of Interest

    TR is a University of California, San Diego coinventor of a COVID-19 rapid test—patent pending. All other authors declared no additional conflicts of interest.

    ‎
    Multimedia Appendix 1

    Additional knowledge, attitude, and practices survey results.

    DOCX File , 17 KB
    1. COVID data tracker. Centers for Disease Control and Prevention.   URL: https://covid.cdc.gov/covid-data-tracker [accessed 2020-03-28]
    2. Gold JA, Rossen LM, Ahmad FB, Sutton P, Li Z, Salvatore PP, et al. Race, ethnicity, and age trends in persons who died from COVID-19 - United States, May-August 2020. MMWR Morb Mortal Wkly Rep 2020 Oct 23;69(42):1517-1521 [FREE Full text] [CrossRef] [Medline]
    3. Despres C. Update: coronavirus case rates and death rates for Latinos in the United States. Salud America. 2022 May 20.   URL: https://salud-america.org/coronavirus-case-rates-and-death-rates-for-latinos-in-the-united-states/ [accessed 2022-07-06]
    4. Kriss JL, Reynolds LE, Wang A, Stokley S, Cole MM, Harris LQ, CDC COVID-19 Vaccine Task Force. COVID-19 vaccine second-dose completion and interval between first and second doses among vaccinated persons - United States, December 14, 2020-February 14, 2021. MMWR Morb Mortal Wkly Rep 2021 Mar 19;70(11):389-395 [FREE Full text] [CrossRef] [Medline]
    5. Hispanic Federation launches COVID-19 VIDA initiative in partnership with 14 Latino community health clinics. Hispanic Federation.   URL: https:/​/www.​hispanicfederation.org/​media/​press_releases/​hispanic_federation_launches_covid-19_vida_initiative_in_partnership_with_14_latino_community_health_clinics/​ [accessed 2022-07-06]
    6. COVID-19 prevention and vaccination for Latinos. Latino Community Foundation.   URL: https://latinocf.org/covid-19-vaccine-outreach/ [accessed 2022-07-06]
    7. Washburn KJ, LeBrón AMW, Reyes AS, Becerra I, Bracho A, Ahn E, et al. Orange County, California COVID-19 vaccine equity best practices checklist: a community-centered call to action for equitable vaccination practices. Health Equity 2022 Jan 01;6(1):3-12 [FREE Full text] [CrossRef] [Medline]
    8. Nguyen LH, Joshi AD, Drew DA, Merino J, Ma W, Lo C, COPE Consortium. Self-reported COVID-19 vaccine hesitancy and uptake among participants from different racial and ethnic groups in the United States and United Kingdom. Nat Commun 2022 Feb 01;13(1):636 [FREE Full text] [CrossRef] [Medline]
    9. Centers for Disease Control and Prevention. 2021.   URL: https://covid.cdc.gov/covid-data-tracker/#health-care-personnel [accessed 2022-07-06]
    10. Wang L, Berger NA, Kaelber D, Davis PB, Volkow ND, Xu R. COVID infection rates, clinical outcomes, and racial/ethnic and gender disparities before and after Omicron emerged in the US. medRxiv Preprint posted online on February 22, 2022. [FREE Full text] [CrossRef] [Medline]
    11. Turner-Musa J, Ajayi O, Kemp L. Examining social determinants of health, stigma, and COVID-19 disparities. Healthcare (Basel) 2020 Jun 12;8(2):168 [FREE Full text] [CrossRef] [Medline]
    12. Ruprecht MM, Wang X, Johnson AK, Xu J, Felt D, Ihenacho S, et al. Evidence of social and structural COVID-19 disparities by sexual orientation, gender identity, and race/ethnicity in an urban environment. J Urban Health 2021 Feb;98(1):27-40 [FREE Full text] [CrossRef] [Medline]
    13. Harkness A, Gattamorta K, Estrada Y, Jimenez D, Kanamori M, Prado G, et al. Latinx health disparities research during COVID-19: challenges and innovations. Ann Behav Med 2020 Aug 08;54(8):544-547 [FREE Full text] [CrossRef] [Medline]
    14. Reiter PL, Katz ML. Racial/Ethnic differences in knowledge, attitudes, and beliefs about COVID-19 among adults in the United States. Front Public Health 2021 May 11;9:653498 [FREE Full text] [CrossRef] [Medline]
    15. Advocacy, communication and social mobilization for TB control: a guide to developing knowledge, attitude and practice surveys. World Health Organization. 2008.   URL: http://apps.who.int/iris/bitstream/handle/10665/43790/9789241596176_eng.pdf?sequence=1 [accessed 2022-07-06]
    16. Andrade C, Menon V, Ameen S, Kumar Praharaj S. Designing and conducting knowledge, attitude, and practice surveys in psychiatry: practical guidance. Indian J Psychol Med 2020 Sep 27;42(5):478-481 [FREE Full text] [CrossRef] [Medline]
    17. Tomar BS, Singh P, Nathiya D, Suman S, Raj P, Tripathi S, et al. Indian community's knowledge, attitude, and practice toward COVID-19. Indian J Soc Psychiatry 2021;37(1):48-56 [FREE Full text]
    18. Xu Y, Lin G, Spada C, Zhao H, Wang S, Chen X, et al. Public knowledge, attitudes, and practices behaviors towards coronavirus disease 2019 (COVID-19) during a national epidemic-China. Front Public Health 2021 Mar 19;9:638430 [FREE Full text] [CrossRef] [Medline]
    19. Sarria-Guzmán Y, Fusaro C, Bernal JE, Mosso-González C, González-Jiménez FE, Serrano-Silva N. Knowledge, attitude and practices (KAP) towards COVID-19 pandemic in America: a preliminary systematic review. J Infect Dev Ctries 2021 Jan 31;15(1):9-21 [FREE Full text] [CrossRef] [Medline]
    20. An L, Bacon E, Hawley S, Yang P, Russell D, Huffman S, et al. Relationship between coronavirus-related eHealth literacy and COVID-19 knowledge, attitudes, and practices among US adults: web-based survey study. J Med Internet Res 2021 Mar 29;23(3):e25042 [FREE Full text] [CrossRef] [Medline]
    21. Lau L, Hung N, Go D, Ferma J, Choi M, Dodd W, et al. Knowledge, attitudes and practices of COVID-19 among income-poor households in the Philippines: a cross-sectional study. J Glob Health 2020 Jun;10(1):011007 [FREE Full text] [CrossRef] [Medline]
    22. Kumar D, Chandra R, Mathur M, Samdariya S, Kapoor N. Vaccine hesitancy: understanding better to address better. Isr J Health Policy Res 2016 Feb 1;5(1):2 [FREE Full text] [CrossRef] [Medline]
    23. Daley MF, Narwaney KJ, Shoup JA, Wagner NM, Glanz JM. Addressing parents' vaccine concerns: a randomized trial of a social media intervention. Am J Prev Med 2018 Jul;55(1):44-54 [FREE Full text] [CrossRef] [Medline]
    24. Odone A, Bucci D, Croci R, Riccò M, Affanni P, Signorelli C. Vaccine hesitancy in COVID-19 times. an update from Italy before flu season starts. Acta Biomed 2020 Sep 07;91(3):e2020031 [FREE Full text] [CrossRef] [Medline]
    25. Khan YH, Mallhi TH, Alotaibi NH, Alzarea AI, Alanazi AS, Tanveer N, et al. Threat of COVID-19 vaccine hesitancy in Pakistan: the need for measures to neutralize misleading narratives. Am J Trop Med Hyg 2020 Aug;103(2):603-604 [FREE Full text] [CrossRef] [Medline]
    26. Dror AA, Eisenbach N, Taiber S, Morozov NG, Mizrachi M, Zigron A, et al. Vaccine hesitancy: the next challenge in the fight against COVID-19. Eur J Epidemiol 2020 Aug 12;35(8):775-779 [FREE Full text] [CrossRef] [Medline]
    27. Burger ZC, Mehta SN, Ortiz D, Sor S, Kothari J, Lam Y, et al. Assessing COVID-19-related knowledge, attitudes, and practices among Hispanic primary care patients: protocol for a cross-sectional survey study. JMIR Res Protoc 2021 Jan 27;10(1):e25265 [FREE Full text] [CrossRef] [Medline]
    28. Yang K, Liu H, Ma L, Wang S, Tian Y, Zhang F, et al. Knowledge, attitude and practice of residents in the prevention and control of COVID-19: an online questionnaire survey. J Adv Nurs 2021 Apr 14;77(4):1839-1855 [FREE Full text] [CrossRef] [Medline]
    29. Ferdous MZ, Islam MS, Sikder MT, Mosaddek ASM, Zegarra-Valdivia JA, Gozal D. Knowledge, attitude, and practice regarding COVID-19 outbreak in Bangladesh: an online-based cross-sectional study. PLoS One 2020 Oct 9;15(10):e0239254 [FREE Full text] [CrossRef] [Medline]
    30. Peng Y, Pei C, Zheng Y, Wang J, Zhang K, Zheng Z, et al. A cross-sectional survey of knowledge, attitude and practice associated with COVID-19 among undergraduate students in China. BMC Public Health 2020 Aug 26;20(1):1292. [CrossRef] [Medline]
    31. Domek GJ, O'Leary ST, Bull S, Bronsert M, Contreras-Roldan IL, Bolaños Ventura GA, et al. Measuring vaccine hesitancy: field testing the WHO SAGE Working Group on Vaccine Hesitancy survey tool in Guatemala. Vaccine 2018 Aug 23;36(35):5273-5281. [CrossRef] [Medline]
    32. Modes of transmission of virus causing COVID-19: implications for IPC precaution recommendations. World Health Organization. 2020 Mar 29.   URL: https:/​/www.​who.int/​news-room/​commentaries/​detail/​modes-of-transmission-of-virus-causing-covid-19-implications-for-ipc-precaution-recommendations [accessed 2022-07-06]
    33. Molla K, Abegaz SB. Community knowledge, attitude and practices to SARS-CoV-2 disease 2019 (COVID-19): a cross-sectional study in Woldia town, Northeast Ethiopia. PLoS One 2021 Apr 20;16(4):e0250465 [FREE Full text] [CrossRef] [Medline]
    34. Lee M, Kang BA, You M. Knowledge, attitudes, and practices (KAP) toward COVID-19: a cross-sectional study in South Korea. BMC Public Health 2021 Feb 05;21(1):295 [FREE Full text] [CrossRef] [Medline]
    35. Papageorge NW, Zahn MV, Belot M, van den Broek-Altenburg E, Choi S, Jamison JC, et al. Socio-demographic factors associated with self-protecting behavior during the COVID-19 pandemic. J Popul Econ 2021 Jan 14;34(2):691-738 [FREE Full text] [CrossRef] [Medline]
    36. Heimlich JE, Ardoin NM. Understanding behavior to understand behavior change: a literature review. Environ Educ Res 2008 Jul 07;14(3):215-237. [CrossRef]
    37. Kazeminia M, Jalali R, Vaisi-Raygani A. Fever and cough are two important factors in identifying patients with the COVID-19: a meta-analysis. J Mill Med 2020;22(2):193-202 [FREE Full text]
    38. Agrawal A, Bhardwaj R. Reducing chances of COVID-19 infection by a cough cloud in a closed space. Phys Fluids (1994) 2020 Oct 01;32(10):101704 [FREE Full text] [CrossRef] [Medline]
    39. Alsan M, Stantcheva S, Yang D, Cutler D. Disparities in coronavirus 2019 reported incidence, knowledge, and behavior among US adults. JAMA Netw Open 2020 Jun 01;3(6):e2012403 [FREE Full text] [CrossRef] [Medline]
    40. Rugarabamu S, Ibrahim M, Byanaku A. Knowledge, attitudes, and practices (KAP) towards COVID-19: a quick online cross-sectional survey among Tanzanian residents. medRxiv Preprint posted online on December 10, 2021. [CrossRef]
    41. Isah MB, Abdulsalam M, Bello A, Ibrahim MI, Usman A, Nasir A, et al. Coronavirus Disease 2019 (COVID-19): Knowledge, attitudes, practices (KAP) and misconceptions in the general population of Katsina State, Nigeria. medRxiv Preprint posted online on June 14, 2020. [CrossRef]
    42. Gehlbach D, Vázquez E, Ortiz G, Li E, Sánchez CB, Rodríguez S, et al. COVID-19 testing and vaccine hesitancy in Latinx farm-working communities in the Eastern Coachella Valley. Res Sq 2021 Jun 25:2021 [FREE Full text] [CrossRef] [Medline]
    43. Cruz Nichols V, LeBrón AMW, Pedraza FI. Spillover effects: immigrant policing and government skepticism in matters of health for Latinos. Public Admin Rev 2018 Feb 21;78(3):432-443. [CrossRef]
    44. Cervantes L, Martin M, Frank MG, Farfan JF, Kearns M, Rubio LA, et al. Experiences of Latinx individuals hospitalized for COVID-19: a qualitative study. JAMA Netw Open 2021 Mar 01;4(3):e210684 [FREE Full text] [CrossRef] [Medline]
    45. Maurizio R, Bertranou F. The labor market in Latin America at the time of the COVID-19 pandemic: impacts, responses and perspectives. Gac Méd Caracas 2021;128(Supl 2):S156-S171. [CrossRef]
    46. Page KR, Flores-Miller A. Lessons we've learned - COVID-19 and the undocumented Latinx community. N Engl J Med 2021 Jan 07;384(1):5-7. [CrossRef] [Medline]
    47. O’Neal CR, Espino MM, Goldthrite A, Morin MF, Weston L, Hernandez P, et al. Grit under duress: stress, strengths, and academic success among non-citizen and citizen Latina/o first-generation college students. Hisp J Behav Sci 2016 Jul 28;38(4):446-466. [CrossRef]
    48. Zegarra-Valdivia J, Vilca BNC, Guerrero RJA. Knowledge, attitudes, and perception susceptibility towards the COVID-19 pandemic in Latin American region. PsyArXiv Preprint posted online on June 07, 2020. [CrossRef]
    49. Aguilar-Latorre A, Asensio-Martínez Á, García-Sanz O, Oliván-Blázquez B. Knowledge, attitudes, risk perceptions, and practices of Spanish adolescents toward the COVID-19 pandemic: validation and results of the Spanish version of the questionnaire. Front Psychol 2021 Jan 4;12:804531 [FREE Full text] [CrossRef] [Medline]
    50. Alsharawy A, Spoon R, Smith A, Ball S. Gender differences in fear and risk perception during the COVID-19 pandemic. Front Psychol 2021 Aug 5;12:689467 [FREE Full text] [CrossRef] [Medline]
    51. Sriharan A, Ratnapalan S, Tricco AC, Lupea D, Ayala AP, Pang H, et al. Occupational stress, burnout, and depression in women in healthcare during COVID-19 pandemic: rapid scoping review. Front Glob Womens Health 2020 Nov 26;1:596690 [FREE Full text] [CrossRef] [Medline]
    52. Alon T, Doepke M, Olmstead-Rumsey J, Tertilt M. This time it's different: the role of women's employment in a pandemic recession. National Bureau of Economic Research. 2020 Aug.   URL: https://www.nber.org/system/files/working_papers/w27660/w27660.pdf [accessed 2022-07-06]
    53. Aldossari M, Chaudhry S. Women and burnout in the context of a pandemic. Gender Work Organ 2020 Oct 30;28(2):826-834. [CrossRef]
    54. Joo H, Miller GF, Sunshine G, Gakh M, Pike J, Havers FP, et al. Decline in COVID-19 hospitalization growth rates associated with statewide mask mandates - 10 states, March-October 2020. MMWR Morb Mortal Wkly Rep 2021 Feb 12;70(6):212-216 [FREE Full text] [CrossRef] [Medline]
    55. Manning KD. More than medical mistrust. Lancet 2020 Nov 07;396(10261):1481-1482. [CrossRef] [Medline]
    56. Jaiswal J, Halkitis PN. Towards a more inclusive and dynamic understanding of medical mistrust informed by science. Behav Med 2019 Jul 25;45(2):79-85 [FREE Full text] [CrossRef] [Medline]

    ‎
    KAP: knowledge, attitude, and practices
    WHO: World Health Organization

    Edited by A Mavragani; submitted 29.03.22; peer-reviewed by I Melyani P., D Madhusudhan, H Utunen; comments to author 04.05.22; revised version received 03.06.22; accepted 06.06.22; published 04.08.22

    Copyright

    ©Shivani N Mehta, Zoe C Burger, Stephanie A Meyers-Pantele, Richard S Garfein, Dayanna O Ortiz, Pavan K Mudhar, Smit B Kothari, Jigna Kothari, Meena Meka, Timothy Rodwell. Originally published in JMIR Formative Research (https://formative.jmir.org), 04.08.2022.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.