Background: eHealth literacy is the ability to seek, obtain, and decipher online health information (OHI) for health and disease management. Rapid developments in eHealth (eg, health care services and online information) place increased demands on patients to have high eHealth literacy levels. Yet, greater emphasis on eHealth may disproportionately affect groups with limited eHealth literacy. Cultural background, language, and eHealth literacy are influential considerations affecting health care and information access, health care use, and successful eHealth resource use, and they may influence OHI seeking for behavioral change toward cancer prevention.
Objective: This study aimed to characterize the extent of OHI seeking and eHealth literacy among Spanish-dominant (SD) Latino adults aged 50 to 75 years. Further, we aimed to examine potential associations between sociodemographic characteristics, Preventive Health Model (PHM) constructs, OHI-seeking behaviors, and eHealth literacy, separately.
Methods: Participants (N=76) self-identified as Latino, were enrolled in a colorectal cancer (CRC) screening intervention, were aged 50 to 75 years, were at average risk for CRC, were not up to date with CRC screening, and preferred receiving health information in Spanish. We describe participants’ sociodemographic characteristics, PHM constructs, OHI-seeking behaviors, and eHealth literacy—among those seeking OHI—assessed at enrollment. Descriptive analyses were first performed for all variables. Next, primary univariate logistic analyses explored possible associations with OHI seeking. Finally, using data from those seeking OHI, exploratory univariate analyses sought possible associations with eHealth literacy.
Results: A majority (51/76, 67%) of the participants were female, 62% (47/76) reported not having graduated high school, and 41% (31/76) reported being unemployed or having an annual income of less than US $10,000. Additionally, 75% (57/76) of the participants reported not having health insurance. In total, 71% (54/76) of the participants reported not having sought OHI for themselves or others. Univariate logistic regression suggested that higher educational attainment was significantly associated with an increased likelihood of having sought OHI (odds ratio 17.4, 95% CI 2.0-150.7; P=.009). Among those seeking OHI (22/76, 29%), 27% (6/22) were at risk of having low eHealth literacy based on an eHealth Literacy Scale score of less than 26. Among OHI seekers (22/76, 29%), an examination of associations found that higher eHealth literacy was associated with greater self-efficacy for screening with the fecal immunochemical test (β=1.20, 95% CI 0.14-2.26; P=.02).
Conclusions: Most SD Latino participants had not sought OHI for themselves or others (eg, family or friends), thus potentially limiting access to beneficial online resources. Preliminary findings convey that higher eHealth literacy occurs among those with higher self-efficacy for CRC screening. Findings inform areas of focus for future larger-scale investigations, including further exploration of reasons for not seeking OHI among SD Latino adults and an in-depth look at eHealth literacy and cancer screening behaviors.
Trial Registration: ClinicalTrials.gov NCT03078361; https://clinicaltrials.gov/ct2/show/NCT03078361
Today, an ever-increasing quantity of health information can be accessed online [, ]. Accordingly, online health information (OHI) is now becoming commonplace in health care interactions and health education for patients and caregivers [ - ]. Such developments place an increased demand on patients to be able to seek, obtain, and decipher OHI for health and disease management [ - ]. eHealth tools provide little value if the intended users require added training and skills to effectively engage these resources. These skills are termed “eHealth literacy” and comprise a multifaceted dynamic construct, including previous and current technology use, demographic and cognitive status, and health-related quality of life (HRQOL) [ , ].
A growing evidence base now attests to the promise of eHealth for promoting positive health behavior change, self-efficacy, and knowledge acquisition [- ].
In recognition of the importance of health information access, the US Department of Health and Human Services set objectives in 2010 for health communication (HC) and health information technology (HIT) as part of the Healthy People initiative. Today, Healthy People 2030 offers HC and HIT objectives aimed at enhancing the use of OHI in public health, including the following: (1) “Increase the proportion of people who can view, download, and send their electronic health information”  and (2) “Increase the proportion of people who say their online medical record is easy to understand” [ ].
Access to OHI is not uniformly distributed throughout the population, which may exacerbate disparities in health and health care [, ].
Among Latino adults, the fastest growing demographic group in the United States, 72% overall are using the internet . Yet, among Spanish-dominant (SD) Latino adults aged 50 to 64 years, internet use drops to 67%, and only 42% of those aged 65 years and older use the internet [ ]. Since disparities exists, it is necessary to understand the current OHI and eHealth literacy levels of SD Latino adults aged 50 to 75 years, yet there is insufficient knowledge in this area.
Further, among Latino adults, cancer continues to be the leading cause of death, accounting for 21% of overall deaths [, , ]. Specifically, colorectal cancer (CRC) is the third leading cause of cancer deaths among Latino adults [ ].
The Preventive Health Model (PHM) is a conceptual framework that aims to explain how health beliefs are related to CRC screening . Specifically, among the CRC screening literature, 26 items measure seven PHM constructs. The PHM constructs include salience and coherence, perceived susceptibility, self-efficacy, response efficacy, cancer worry, social influence, and religious beliefs [ ]. These are assessed using a 5-point Likert scale, ranging from 1 (strongly disagree) to 5 (strongly agree) [ , - ]. Reliability and validity for these subscales have been demonstrated previously [ , - ]. The salience and coherence subscale measures one’s belief that CRC screening is important and makes sense in one’s life. The perceived susceptibility subscale assesses one’s perceived risk of being diagnosed with CRC. The self-efficacy subscale measures the belief that one could complete the steps necessary for fecal immunochemical test (FIT) collection. The response efficacy subscale measures the belief that CRC screening is beneficial for early detection and prevention of CRC. The cancer worry subscale measures the degree to which one is worried about having an abnormal CRC screening result. The social influence subscale assesses the perception that important others (eg, family members, friends, and one’s health care provider) would want the individual to complete CRC screening. The religious beliefs subscale assesses the degree to which one relies on one’s religious beliefs to make health decisions. Thus, in relating a need to understand current OHI and eHealth literacy levels of SD Latino adults aged 50 to 75 years, it is also of interest to explore how these may relate to PHM constructs in this understudied population. Prior studies suggest that sociodemographic and PHM constructs, such as higher social influence and religious beliefs, were associated with lower health literacy among English-preferring individuals [ , ]. By contrast, lower cancer worry and lower religious beliefs [ ] were associated with adequate health literacy to understand written health information, and higher educational attainment was significantly associated with adequate health literacy in completing health forms among SD Latino adults aged 50 to 75 years [ ]. However, less is known about how sociodemographic variables and PHM constructs might influence OHI seeking or eHealth literacy among SD Latino adults.
Thus, the aims of this study were to (1) describe the prevalence of OHI seeking and eHealth literacy and (2) examine preliminary associations between sociodemographic characteristics, PHM constructs, OHI seeking, and eHealth literacy, separately, among SD Latino adults.
To achieve our study aims, we posed the following four research questions:
- Research question 1 (RQ1): To what extent do SD Latino adults use the internet to locate OHI?
- Research question 2 (RQ2): How are self-reported levels of eHealth literacy described among SD Latino adults who seek OHI?
- Research question 3 (RQ3): What sociodemographic characteristics and PHM constructs are associated with OHI seeking by SD Latino adults?
- Research question 4 (RQ4): What sociodemographic characteristics and PHM constructs are preliminarily associated with eHealth literacy among SD Latino adults who seek OHI?
Taken together, this study examines OHI and eHealth literacy among a diverse understudied population (ie, SD Latino adults). Further, the study provides a preliminary look at possible associations among sociodemographic characteristics, PHM constructs, OHI seeking, and eHealth literacy through an innovative examination within the literature.
Data for this report were collected as part of a larger pilot randomized controlled trial—Latino Colorectal Cancer Awareness, Research, Education, and Screening (Latino CARES)—that promoted CRC screening by providing education and a FIT. The pilot study aimed to evaluate the feasibility and impact of a Spanish-language, low-literacy, culturally targeted intervention (ie, a photonovella and DVD) plus FIT compared with a standard Spanish-language booklet developed by the Centers for Disease Control and Prevention plus FIT. Of note, recruitment took place at two participating clinic sites that are part of a Federally Qualified Health Center (FQHC) in Southwest Florida. The FQHC sites are centrally located in agricultural farmworker communities and annually serve a large number (ie, approximately 5000) of medically underserved patients aged 50 to 75 years, a majority of whom are of Latino origin from diverse nationalities and include farmworker populations . Detailed methods are provided in prior publications that communicate the results of the main outcomes of the Latino CARES study [ - ].
Participant consent was obtained prior to baseline interview and randomization. Eligible participants (1) were receiving care at two participating FQHC clinic locations; (2) were between the ages of 50 and 75 years; (3) self-identified as Latino; (4) were able to read, speak, and understand Spanish; (5) preferred to receive health information in Spanish; (6) were currently not up to date per CRC screening guidelines (ie, had never screened or previously screened but were now overdue); and (7) were at average risk for CRC.
The University of South Florida Institutional Review Board approved the study (approval No. MCC-17665) prior to participant enrollment. Study procedures were conducted in accordance with the Declaration of Helsinki. During enrollment, participants provided informed consent to be included in the study.
Study assessments consisted of validated measures that were translated into Spanish and refined for cultural relevance using the following established procedures. Our study team included three bilingual (ie, fluent in English and Spanish) researchers and a bilingual (ie, fluent in English and Spanish) community advisory board. Applying the Brislin method , measures were first translated into Spanish by a bilingual study coordinator and then back-translated by a second bilingual study team member. Any discrepancies were arbitrated by a third bilingual study team member. Items were pretested among community members using learner verification methodology [ , ]; they were further refined in consultation with the community advisory board for cultural relevance and comprehension. Baseline items were administered by bilingual (ie, English and Spanish) study coordinators at the time of interview. To minimize literacy issues, all questions were read aloud for all participants.
Online Health Information Seeking
A single item, gathered from the Pew Research Center  and used in previous OHI behavioral studies among Latino adults [ , ], assessed OHI-seeking usage: “Have you previously personally searched for health information on the Internet/Online for yourself or for others? For example, [have you] sought on Google/Yahoo information on high blood pressure, healthy recipes, or efficient exercises?” Response options were yes or no.
eHealth literacy was assessed only among participants who reported engaging in OHI seeking. Thus, the number of participants for whom eHealth literacy was assessed was lower than that for those responding to the other measures. The Spanish-translated eHealth Literacy Scale (eHEALS) [, ] was administered to assess eHealth literacy, including knowledge, comfort, and perceived skills at finding, evaluating, and applying OHI to health problems. The eHEALS [ ] comprises eight items scored on a 5-point Likert scale, ranging from 1 (strongly disagree) to 5 (strongly agree), and aims to reflect the individuals’ own perceptions of their knowledge and skills at using eHealth information [ , ]. The final result is the sum of all items and ranges from 8 to 40, with higher scores reflecting a higher level of eHealth literacy. The validity and reliability of the eHEALS has been demonstrated across various health conditions [ , ], ages [ - ], and languages [ ], including Spanish [ ]. Following other studies with similar target populations, the cutoff for high eHealth literacy was set at 26 [ , , - ]. Thus, in maintaining consistency in terminology with the literature, this study defined high self-perceived eHealth literacy as an eHEALS score equal to or greater than 26 out of 40, and low self-perceived eHealth literacy was defined as an eHEALS score of less than 26 [ , , - ].
Sociodemographic variables assessed included age, gender, race and ethnicity, marital status, education, health insurance status, employment status, and income. Additionally, as aggregated data in prior literature masked substantial heterogeneity within the Latino population, we assessed parental foreign-born status and participant foreign-born status; if foreign-born, the country of origin and years lived in the United States were also assessed .
Preventive Health Model Variables
Seven constructs of the PHM were assessed in this study using 26 total items referenced from prior CRC studies [, , , , , - ]. For each item assessing PHM constructs, response options were based on a 5-point Likert scale, ranging from 1 (strongly disagree) to 5 (strongly agree). Four items assessed salience and coherence, or the perception that performing a health behavior is consistent with their beliefs about how to protect and maintain health [ - , , , , ]. Three items assessed perceived susceptibility, or one’s perceived personal risk for developing CRC or colon polyps [ , , , ]. Two items assessed cancer worry, or one’s concern that completing CRC screening will reveal a health concern [ , , , , , , ].
Two items measured response efficacy, or the belief that adopting a behavior will be effective in reducing disease threat [, , , , ]. Four items measured social influence, or the influence of family members and doctors or health professionals on an individual’s willingness to comply with CRC screening [ , , , , , ]. Five items assessed religiosity, or the extent to which religious beliefs might influence medical decision–making, such as CRC screening [ , , , ]. Six items measured self-efficacy for screening using FIT, or attitudes and confidence toward completing FIT testing [ - , , , , ]. For each respective construct, the corresponding items’ scores reported by participants were added together; construct total scores were used for analyses.
Sociodemographic characteristics and PHM constructs were summarized using descriptive statistics. The assessment details for each research question are discussed in the following four sections.
Research Question 1
RQ1 is as follows: To what extent do SD Latino adults use the internet to locate OHI? OHI seeking was characterized using descriptive statistics.
Research Question 2
RQ2 is as follows: How are self-reported levels of eHealth literacy described among SD Latino adults who seek OHI? eHealth literacy was characterized using descriptive statistics.
Research Question 3
RQ3 is as follows: What sociodemographic characteristics and PHM constructs are associated with OHI seeking by SD Latino adults? Univariate logistic regression analyses were conducted to examine potential sociodemographic and PHM constructs associated with OHI seeking. Data from all study participants (N=76) were available for this exploration. For these analyses, we coded responses to the item “Personally looked online for health information for self or others” as a binary outcome. Gender, insurance status, employment status, marriage status, age, annual income, educational attainment, perceived salience, perceived susceptibility, response efficacy, cancer worry, social influence, religious beliefs, and self-efficacy for screening with FIT were treated as independent variables, and whether participants sought OHI for themselves or others—yes or no response—was treated as the dependent variable.
Research Question 4
RQ4 is as follows: What sociodemographic characteristics and PHM constructs are preliminarily associated with eHealth literacy among SD Latino adults who seek OHI? Univariate analyses were conducted leveraging linear regressions to examine potential sociodemographic characteristics and PHM constructs associated with eHealth literacy (ie, the eHEALS score). The eHealth literacy outcome score was treated as a continuous outcome, ranging from 8 to 40. For this exploration, data were available only from those seeking OHI (22/76, 29%). Gender, insurance status, employment status, marriage status, age, annual income, educational attainment, perceived salience, perceived susceptibility, response efficacy, cancer worry, social influence, religious beliefs, and self-efficacy for screening with FIT were treated as independent variables, and the eHEALS score was treated as the dependent variable. Due to the sample size limitation, this exploration was underpowered; hence, the goal was to provide reasonably reliable estimates to guide the design of a future, larger, appropriately powered study.
Analyses were conducted using SAS (version 9.4 [TS1M6]; SAS Institute Inc). A P value of less than .05 was considered statistically significant. Of note, analyses are exploratory and not for definitive inferential interpretations.
Sociodemographic characteristics are described inand in the main reported outcomes of Gwede et al [ ]. In total, 67% (51/76) of participants were female. The mean age of the participants was 57.2 (SD 6.0) years (range 50-74). In total, 62% (47/76) of participants reported not completing high school. In addition, 41% (31/76) of participants reported being unemployed and having an annual income of less than US $10,000. Further, 75% (57/76) of participants lacked health insurance. In total, 93% (71/76) of participants reported being born outside of the United States. Among those born outside of the United States, a majority (49/71, 69%) reported Mexico as their country of birth. Further, among those born outside of the United States, the mean length of time reported living in the United States was 23.4 (SD 10.9) years (range 2-56).
|Variables||Full sample (N=76)||OHIa seekers (n=22)|
|Age (years), mean (SD), range||57.2 (6.0), 50-74||56.6 (5.8), 50-69|
|Years in the United States, mean (SD), range||23.4 (10.9), 2-56||19.4 (11.2), 2-56|
|Gender, n (%)|
|Male||25 (33)||3 (4)|
|Female||51 (67)||19 (25)|
|Race, n (%)|
|White||23 (30)||8 (11)|
|Black||1 (1)||1 (1)|
|Other or more than one race||52 (68)||13 (17)|
|Marital status, n (%)|
|Married or partnered||53 (70)||14 (18)|
|Divorced or separated||10 (13)||5 (7)|
|Widowed||6 (8)||3 (4)|
|Never married or single||7 (9)||0 (0)|
|Employment status, n (%)|
|Employed or self-employed||40 (53)||16 (21)|
|Unemployed||31 (41)||6 (8)|
|Student||1 (1)||0 (0)|
|Retired||3 (4)||0 (0)|
|Education, n (%)|
|Less than high school||47 (62)||3 (4)|
|High school graduate||13 (17)||6 (8)|
|Some college or technical school||7 (9)||5 (7)|
|College graduate||7 (9)||6 (8)|
|Graduate or professional (postcollege)||2 (3)||2 (3)|
|Annual income (US $), n (%)|
|<10,000||31 (41)||8 (11)|
|10,000-25,000||29 (38)||10 (13)|
|25,001-35,000||8 (11)||2 (3)|
|35,001-75,000||2 (3)||1 (1)|
|Don’t know or prefer not to answer||6 (8)||1 (1)|
|Insurance status, n (%)|
|No insurance||57 (75)||14 (18)|
|Medicaid or Medicare||4 (5)||0 (0)|
|County health insurance||8 (11)||3 (4)|
|Private health insurance||7 (9)||5 (7)|
|Country of birth, n (%)|
|United States||5 (7)||2 (3)|
|Other||71 (94)||20 (26)|
|Self-reported country of birth (if that reported was other than the United States), n (%)|
|Mexico||49 (64)||7 (9)|
|Colombia||6 (8)||4 (5)|
|Puerto Rico||5 (7)||2 (3)|
|Costa Rica||3 (4)||2 (3)|
|Dominican Republic||2 (3)||0 (0)|
|Peru||2 (3)||2 (3)|
|Chile||1 (1)||1 (1)|
|Cuba||1 (1)||1 (1)|
|El Salvador||1 (1)||0 (0)|
|Venezuela||1 (1)||1 (1)|
|Parents born outside the United States, including Puerto Rico, n (%)|
|Yes||73 (96)||21 (28)|
|No||3 (4)||1 (1)|
|Personally sought OHI, n (%)|
|Yes||22 (29)||22 (29)|
|No||54 (71)||0 (0)|
|eHEALSb score, mean (SD), range||N/Ac||29.73 (6.6), 15-40|
aOHI: online health information.
beHEALS: eHealth Literacy Scale; the total score ranged from 8 to 40; 22 out of 76 participants completed the scale.
cN/A: not applicable; this was not calculated for the full sample, since not all participants completed the scale.
Extent to Which SD Latino Adults Use the Internet to Locate OHI (RQ1)
All participants (N=76) responded to the item assessing personal OHI seeking for themselves or others. In total, 71% (54/76) of participants reported not having personally sought OHI for themselves or others (), whereas the remaining 29% (22/76) of participants reported having personally looked for OHI information for themselves or others.
Variability in Self-reported Levels of eHealth Literacy Among SD Latino Adults (RQ2)
Assessment of eHealth literacy using the eHEALS was only completed for those who answered yes to having sought OHI for themselves or others (22/76, 29%). Of those individuals, the mean eHEALS score was 29.7 (SD 6.6; range 15-40). In total, 27% (6/22) of those seeking OHI had an eHEALS score of less than 26, indicating that these participants were in the low–eHealth literacy category.
Frequency of responses to the eight-item eHEALS—note that only 22 participants completed this scale, as they sought OHI—is reported in. The item with the greatest degree of agreement was “I know how to use the health resources I find on the internet to help me,” with 91% (20/22) of eHEALS respondents self-reporting either mildly agree or strongly agree. The item with the least amount of agreement was “I feel confident in using information from the internet to make health decisions,” with 45% (10/22) of eHEALS respondents self-reporting being uncertain, mildly disagree, or strongly disagree.
Sociodemographics and PHM Constructs Associated With OHI Seeking by SD Latino Adults (RQ3)
Univariate logistic regression analyses on data from 76 participants were performed to examine preliminary associations between sociodemographic variables, PHM constructs, and the likelihood of seeking OHI. Only educational attainment was found to be significantly associated with OHI seeking (). Higher educational attainment was significantly associated with an increased likelihood of having sought OHI (odds ratio 17.4, 95% CI 2.0-150.7; P=.009). Employment status approached significance (P=.07). None of the PHM health beliefs were significantly associated with OHI. The logistic regression model was statistically significant (χ214=60.3, P<.001). The model explained 81.9% (Nagelkerke R2) of the variance in OHI and correctly classified 90.0% of cases. Sensitivity was 85.7%, specificity was 91.8%, positive predictive value was 81.8%, and negative predictive value was 93.8%.
|Covariate||Respondents (N=76), n (%)||Odds ratio (95% CI)||P valuea|
|Female||51 (67)||170.05 (0.37 to >999.99)||.10|
|Yes||19 (25)||59.19 (0.42 to >999.99)||.11|
|Yes||40 (53)||>999.99 (0.57 to >999.99)||.07|
|Yes||53 (70)||62.01 (0.20 to >999.99)||.16|
|Age||76 (100)||0.93 (0.64 to 1.35)||.68|
|Annual income||70 (92)||2.03 (0.50 to 8.29)||.32|
|Educational attainment||76 (100)||17.40 (2.01 to 150.72)||.009|
|Perceived salience||76 (100)||1.84 (0.45 to 7.44)||.39|
|Perceived susceptibility||76 (100)||0.99 (0.53 to 1.85)||.98|
|Response efficacy||76 (100)||2.33 (0.32 to 17.10)||.41|
|Cancer worry||76 (100)||1.18 (0.70 to 2.02)||.53|
|Social influence||76 (100)||1.07 (0.60 to 1.90)||.83|
|Religious beliefs||76 (100)||0.95 (0.74 to 1.22)||.68|
|Self-efficacy for screening with FITc||76 (100)||0.96 (0.43 to 2.14)||.91|
aP<.05 indicates statistical significance.
bN/A: not applicable.
cFIT: fecal immunochemical test.
Sociodemographics and PHM Constructs Associated With the eHealth Literacy of SD Latino Adults (RQ4)
Univariate analyses using linear regressions were completed to examine preliminary associations between eHEALS scores, sociodemographic variables, and PHM constructs. Higher self-efficacy for screening with FIT was significantly associated with higher eHEALS scores (ie, eHealth literacy; β=1.20, 95% CI 0.14-2.26; P=.02).reports the regression coefficients and standard errors. The best-fit model was significantly associated with eHEALS score (F1,20=5.53, P=.02; adjusted R2=0.18).
|Factors associated with the eHEALSa||β coefficient (SE)||Standardized β||t test (df=1)||P valueb|
|Annual income||–0.60 (5.04)||–0.08||–0.12||.91|
|Educational attainment||–0.19 (2.99)||–0.04||–0.06||.95|
|Perceived salience||–0.57 (2.88)||–0.11||–0.20||.84|
|Perceived susceptibility||–0.82 (1.18)||–0.32||–0.69||.51|
|Response efficacy||–0.46 (2.36)||–0.08||–0.19||.85|
|Cancer worry||0.09 (1.66)||0.04||0.06||.96|
|Social influence||–0.10 (1.30)||–0.05||–0.08||.94|
|Religious beliefs||–0.33 (1.00)||–0.25||–0.34||.75|
|Self-efficacy for screening with FITc||1.58 (1.17)||0.64||1.35||.22|
|Self-efficacy for screening with FIT||1.20 (0.51)||0.47||2.35||.02|
aeHEALS: eHealth Literacy Score.
bP<.05 indicates statistical significance.
cFIT: fecal immunochemical test.
OHI has become more routine in health care interactions and health education for patients and caregivers [- ]. Yet, greater emphasis on eHealth may disproportionately affect groups with limited eHealth literacy. Healthy People 2030 continues the national effort initiated in 2010 in recognition of the importance of HC and HIT to support better communication, care, and outcomes toward achieving health equity [ ]. There is a dearth of information available in the literature that communicates specifically on factors (eg, sociodemographic characteristics and health beliefs) associated with OHI seeking and eHealth literacy among SD Latino adults.
In this study, nearly three-quarters (54/76, 71%) of participants reported not having personally sought OHI for themselves or others. Among those for whom eHealth literacy was assessed (22/76, 29%), nearly one-third (6/22, 27%) had an eHEALS score of less than 26, indicating self-perceived low eHealth literacy. In this study, preliminary findings suggest that higher educational attainment was associated with an increased likelihood of having sought OHI. Although further research to confirm directionality is necessary, this association between higher educational attainment and seeking OHI among SD Latino adults is consistent with prior findings on OHI behaviors conducted among English-speaking populations [, ]. Specifically, Jacobs et al [ ] highlight that a heavy reliance on e-technologies for disseminating health information may increase the likelihood of further perpetuating health disparities; they suggest a need for interventions and efforts focused on developing training and services to boost internet self-efficacy tailored to patients’ learning styles and their cultural and demographic characteristics to reduce this digital disparity.
This study was conducted among SD Latino adults of whom the majority were born outside of the United States, which is acutely different from studies conducted among participants preferring the English language [, ]. Our efforts are preliminary and, thus, additional research is needed to explore the relationships between educational attainment and OHI seeking among various groups who prefer non-English languages and groups newly arrived in the United States. Of interest, in our exploration of factors associated with the act of OHI seeking, there was a high proportion of individuals who had not sought OHI (54/76, 71%). This lack of OHI seeking may disproportionately affect SD Latino adults, as reliance on eHealth resources continues to gain emphasis in the United States [ - ] and globally [ ]. Indeed, our efforts are cautiously interpreted, yet our data signal a timely opportunity to examine the reasons preventing OHI-seeking behaviors among SD Latino adults aged 50 to 75 years, as well as the potential for intervention research. Thus, in recognizing this distinction, considerations for culture and language are a fertile area of future research examining reasons for the lack of OHI seeking (eg, learning styles and preferences, availability and access to Spanish-language computer resources, knowledge, and technological skills, such as search strategies, among others). In considering culture, previous studies exploring OHI behaviors suggested that younger, English-dominant Latino and Hispanic generations may be OHI brokers for older generations of SD Latino adults [ , , ]. Therefore, whether OHI seeking could be facilitated by intergenerational co-learning approaches is an avenue for future exploration. In a study examining digital health disparities among ethnically diverse older adults who prefer the English language [ ], researchers suggested important steps to close this digital divide, including cultural adaptation based on preferences for receiving health information. Research has found that personal instruction through the process of internet use and assistance with using new digital devices helps older adults to adopt a daily habit of internet use [ ]. However, whether these outcomes may be replicated among SD Latino adults aged 50 to 75 years remains to be examined. Further still, the literature communicates that smartphone ownership offers no statistically significant difference relevant to race or ethnicity [ ] and is the primary source of internet use among all Latino adults [ ]. Thus, considerations for examining the potential of leveraging smartphones for interventions on promoting OHI seeking among SD Latino adults is worth noting. Moreover, in addition to reaching underserved populations in health centers [ ], community libraries offer another potential avenue for engaging with diverse community members [ ].
In our study, exploratory findings suggest that higher eHealth literacy was preliminarily associated with higher self-efficacy for FIT screening (β=1.20, 95% CI 0.14-2.26; P=.02) among SD Latino adults. This preliminary finding suggests that further investigation is warranted. Future in-depth examinations are necessary to confirm directionality of association. Prior to this study, the literature did not communicate specifically regarding findings that have examined eHealth literacy and self-efficacy for CRC screening. Efforts by Park et al  conveyed an association between higher levels of eHealth literacy and greater confidence in seeking online cancer information. Yet, the study was completed among English-language participants; importantly, the “confidence” examined by Park et al [ ] differs from our use of “self-efficacy” that we occupy via the PHM.
This preliminary study communicates promising results suggesting that increased eHealth literacy may be associated with increased self-efficacy for CRC screening with FIT. While this study’s results must be replicated in a larger sample, these findings are encouraging and set an important small guiding step in suggesting that eHealth literacy skills development might benefit self-efficacy of CRC screening. Future appropriately powered research is necessary to examine whether eHealth literacy training and building self-efficacy for CRC screening may ultimately impact CRC screening uptake.
Limitations and Strengths
This study has several limitations and strengths to acknowledge. First, the study was conducted in the context of a small pilot trial, underpowered for inferential analyses. Thus, our findings should be interpreted cautiously and call for further study in larger samples of SD Latino individuals. Second, this study did not assess access to the internet nor barriers to OHI seeking, limiting potential findings. Third, Latino ethnicity is shared among many Americans from various backgrounds and, most certainly, does not comprise a homogenous group. Characteristics of Latino adults vary by region. In this study, participants communicated their Latino ethnicity, and most were of Mexican heritage, thus potentially limiting generalizability. Yet, the sample representing 10 different Latin American countries offers preliminary evidence of feasibility for recruiting Latino adults from diverse backgrounds into future studies.
There are several implications to consider and acknowledge. First, to reach the Healthy People 2030 HC and HIT objectives, it is necessary to expand research efforts to highlight and address barriers and leverage facilitators to OHI behaviors; it is also necessary to increase access to the internet. Second, future intervention research to promote eHealth literacy should examine and consider how cultural background and language affect access, health care use, successful use of eHealth resources, and behavior change. Finally, this study provided preliminary findings that suggested a need for pursuing further research on promoting OHI-seeking behaviors and eHealth literacy training that may impact CRC self-efficacy and, ultimately, CRC screening behaviors. Nevertheless, larger research studies are needed to corroborate these findings before clear implications for practice can be reached.
This preliminary study adds to an extremely small evidence base and is the first to communicate findings on the assessment and analyses of OHI seeking and eHealth literacy in the context of the PHM among SD Latino adults. A high proportion of SD Latino participants in our study have not sought OHI for themselves or others, thus limiting their access to beneficial resources. In light of the growing use and reliance on technologies in health care, factors preventing OHI seeking remain to be further examined. In addition, there is a need for resources to improve eHealth literacy among SD Latino adults. Our study communicates preliminary evidence that higher eHealth literacy is occurring among those with higher self-efficacy for CRC screening. Appropriately powered research in the future is warranted to further examine this preliminary finding. Additionally, the next logical step that future research should examine is whether eHealth literacy training and building self-efficacy for CRC screening could increase the uptake of CRC screening.
Funding for the project was provided by the Florida Department of Health–Bankhead Coley Cancer Research Program (4BB09; principal investigators [PIs]: CKG and CDM). The data collection efforts of EAC and SMC were supported by a grant from the National Cancer Institute (NCI) while they were postdoctoral fellows (grant R25CA090314; PIs: PB Jacobsen and TH Brandon). Support was provided by the Center for Biostatistics Collaboration and Data Service at the University of Texas Health Science Center at Houston. The publication of the research was supported, in part, by the Winship Cancer Institute of Emory University and the National Institutes of Health, NCI (award P30CA138292). The funders had no role in the study design; the collection, analysis, and interpretation of the data; the writing of the manuscript; or the decision to submit the manuscript for publication.
The authors communicate a special thank-you to Dr Colleen M McBride (Professor, Department of Behavioral, Social, and Health Education Sciences, Rollins School of Public Health, Emory University) for the valuable and much appreciated review of earlier drafts of this article.
Conflicts of Interest
HM is the cofounder of SmartCT Studio LLC, which is not related to or involved in this published work.
- Prestin A, Vieux SN, Chou WS. Is online health activity alive and well or flatlining? Findings from 10 years of the health information national trends survey. J Health Commun 2015;20(7):790-798. [CrossRef] [Medline]
- Percheski C, Hargittai E. Health information-seeking in the digital age. J Am Coll Health 2011;59(5):379-386. [CrossRef] [Medline]
- Cutrona SL, Mazor KM, Vieux SN, Luger TM, Volkman JE, Finney Rutten LJ. Health information-seeking on behalf of others: Characteristics of "surrogate seekers". J Cancer Educ 2015 Mar;30(1):12-19 [FREE Full text] [CrossRef] [Medline]
- Chavarria EA, Chaney EH, Stellefson ML, Chaney JD, Chavarria N, Dodd VJ. Types and factors associated with online health information seeking among college men in Latino fraternities: A qualitative study. Am J Mens Health 2017 Nov;11(6):1692-1702 [FREE Full text] [CrossRef] [Medline]
- Bundorf MK, Wagner TH, Singer SJ, Baker LC. Who searches the internet for health information? Health Serv Res 2006 Jun;41(3 Pt 1):819-836 [FREE Full text] [CrossRef] [Medline]
- Rice RE. Influences, usage, and outcomes of internet health information searching: Multivariate results from the Pew surveys. Int J Med Inform 2006 Jan;75(1):8-28. [CrossRef] [Medline]
- Arora NK, Hesse BW, Rimer BK, Viswanath K, Clayman ML, Croyle RT. Frustrated and confused: The American public rates its cancer-related information-seeking experiences. J Gen Intern Med 2008 Mar;23(3):223-228 [FREE Full text] [CrossRef] [Medline]
- Fox S, Duggan M. Health Online 2013. Washington, DC: Pew Research Center’s Internet & American Life Project; 2013 Jan 15. URL: https://www.pewresearch.org/internet/2013/01/15/health-online-2013/ [accessed 2022-08-23]
- Norman CD, Skinner HA. eHEALS: The eHealth Literacy Scale. J Med Internet Res 2006 Nov 14;8(4):e27 [FREE Full text] [CrossRef] [Medline]
- Neter E, Brainin E. eHealth literacy: Extending the digital divide to the realm of health information. J Med Internet Res 2012 Jan 27;14(1):e19 [FREE Full text] [CrossRef] [Medline]
- Sanchez MA, Rabin BA, Gaglio B, Henton M, Elzarrad MK, Purcell P, et al. A systematic review of eHealth cancer prevention and control interventions: New technology, same methods and designs? Transl Behav Med 2013 Dec;3(4):392-401 [FREE Full text] [CrossRef] [Medline]
- Slev VN, Mistiaen P, Pasman HRW, Verdonck-de Leeuw IM, van Uden-Kraan CF, Francke AL. Effects of eHealth for patients and informal caregivers confronted with cancer: A meta-review. Int J Med Inform 2016 Mar;87:54-67. [CrossRef] [Medline]
- Horvath KJ, Bauermeister JA. eHealth literacy and intervention tailoring impacts the acceptability of a HIV/STI testing intervention and sexual decision making among young gay and bisexual men. AIDS Educ Prev 2017 Feb;29(1):14-23 [FREE Full text] [CrossRef] [Medline]
- Increase the proportion of people who can view, download, and send their electronic health information — HC/HIT‑D09. US Department of Health and Human Services, Healthy People 2030. 2020. URL: https://tinyurl.com/3t9e6az2 [accessed 2022-08-23]
- Increase the proportion of people who say their online medical record is easy to understand — HC/HIT‑D10. US Department of Health and Human Services, Healthy People 2030. 2020. URL: https://health.gov/healthypeople/objectives-and-data/browse-objectives/health-it/increase-proportion-people-who-say-their-online-medical-record-easy-understand-hchit-d10 [accessed 2022-08-23]
- McCloud RF, Okechukwu CA, Sorensen G, Viswanath K. Beyond access: Barriers to internet health information seeking among the urban poor. J Am Med Inform Assoc 2016 Nov;23(6):1053-1059 [FREE Full text] [CrossRef] [Medline]
- Nguyen A, Mosadeghi S, Almario CV. Persistent digital divide in access to and use of the internet as a resource for health information: Results from a California population-based study. Int J Med Inform 2017 Jul;103:49-54. [CrossRef] [Medline]
- Goldberg R, Robinson A, Carlson E. Digital divide is shrinking for America’s Hispanic population, NTIA data show. National Telecommunications and Information Administration. 2019 Oct 28. URL: https://www.ntia.gov/blog/2019/digital-divide-shrinking-america-s-hispanic-population-ntia-data-show [accessed 2022-08-23]
- Anna A, López G, Lopez MH. Digital Divide Narrows for Latinos as More Spanish Speakers and Immigrants Go Online. Washington, DC: Pew Research Center; 2016 Jul. URL: https://www.pewresearch.org/hispanic/wp-content/uploads/sites/5/2016/07/PH_2016.07.21_Broadbank_Final.pdf [accessed 2022-08-23]
- Lee YJ, Boden-Albala B, Larson E, Wilcox A, Bakken S. Online health information seeking behaviors of Hispanics in New York City: A community-based cross-sectional study. J Med Internet Res 2014 Jul 22;16(7):e176 [FREE Full text] [CrossRef] [Medline]
- Cancer Facts & Figures for Hispanics/Latinos 2018-2020. Atlanta, GA: American Cancer Society; 2018. URL: https://tinyurl.com/5949prpv [accessed 2022-08-23]
- Myers RE, Ross E, Jepson C, Wolf T, Balshem A, Millner L, et al. Modeling adherence to colorectal cancer screening. Prev Med 1994 Mar;23(2):142-151. [CrossRef] [Medline]
- Christy SM, Sutton SK, Gwede CK, Chavarria EA, Davis SN, Abdulla R, et al. Examining the durability of colorectal cancer screening awareness and health beliefs among medically underserved patients: Baseline to 12 months post-intervention. J Cancer Educ 2017 Nov 25;34(2):297-303. [CrossRef]
- Tiro JA. Factorial validity and invariance of a survey measuring psychosocial correlates of colorectal cancer screening among African Americans and Caucasians. Cancer Epidemiol Biomarkers Prev 2005 Dec 01;14(12):2855-2861. [CrossRef]
- Vernon SW, Myers RE, Tilley BC, Li S. Factors associated with perceived risk in automotive employees at increased risk of colorectal cancer. Cancer Epidemiol Biomarkers Prev 2001 Jan;10(1):35-43. [Medline]
- Myers RE, Sifri R, Hyslop T, Rosenthal M, Vernon SW, Cocroft J, et al. A randomized controlled trial of the impact of targeted and tailored interventions on colorectal cancer screening. Cancer 2007 Nov 01;110(9):2083-2091 [FREE Full text] [CrossRef] [Medline]
- Davis SN, Wischhusen JW, Sutton SK, Christy SM, Chavarria EA, Sutter ME, et al. Demographic and psychosocial factors associated with limited health literacy in a community-based sample of older Black Americans. Patient Educ Couns 2020 Feb;103(2):385-391 [FREE Full text] [CrossRef] [Medline]
- Christy SM, Gwede CK, Sutton SK, Chavarria E, Davis SN, Abdulla R, et al. Health literacy among medically underserved: The role of demographic factors, social influence, and religious beliefs. J Health Commun 2017 Nov;22(11):923-931 [FREE Full text] [CrossRef] [Medline]
- Christy S, Cousin L, Sutton S. Characterizing health literacy among Spanish language-preferring Latinos ages 50-75. Nurs Res 2021;70(5):344-353. [CrossRef]
- Gwede C, Sutton S, Chavarria E, Gutierrez L, Abdulla R, Christy SM, et al. A culturally and linguistically salient pilot intervention to promote colorectal cancer screening among Latinos receiving care in a Federally Qualified Health Center. Health Educ Res 2019 Jun 01;34(3):310-320 [FREE Full text] [CrossRef] [Medline]
- Aguado Loi CX, Martinez Tyson D, Chavarria EA, Gutierrez L, Klasko L, Davis S, et al. 'Simple and easy:' Providers' and Latinos' perceptions of the fecal immunochemical test (FIT) for colorectal cancer screening. Ethn Health 2020 Feb;25(2):206-221 [FREE Full text] [CrossRef] [Medline]
- Brislin RW. Back-translation for cross-cultural research. J Cross Cult Psychol 2016 Jul 27;1(3):185-216. [CrossRef]
- Doak CC, Doak LG, Root JH. Learner verification and revision of materials. In: Teaching Patients With Low Literacy Skills. 2nd edition. Philadelphia, PA: J. B. Lippincott Company; 1996:167-188.
- Chavarria EA, Christy SM, Simmons VN, Vadaparampil ST, Gwede CK, Meade CD. Learner verification: A methodology to create suitable education materials. Health Lit Res Pract 2021 Jan 11;5(1):e49-e59 [FREE Full text] [CrossRef] [Medline]
- Chavarria EA, Chaney BH, Stellefson ML, Chaney JD, Chavarria NE, Dodd VJ. Predictors of online health information-seeking behaviors among college Latino men. Health Behav Policy Rev 2017 Jul 01;4(4):367-379. [CrossRef]
- Paramio Pérez G, Almagro BJ, Hernando Gómez Á, Aguaded Gómez JI. [Validation of the eHealth Literacy Scale (eHEALS) in Spanish university students] [Article in Spanish]. Rev Esp Salud Publica 2015;89(3):329-338 [FREE Full text] [CrossRef] [Medline]
- Collins SA, Currie LM, Bakken S, Vawdrey DK, Stone PW. Health literacy screening instruments for eHealth applications: A systematic review. J Biomed Inform 2012 Jun;45(3):598-607 [FREE Full text] [CrossRef] [Medline]
- Milne RA, Puts MTE, Papadakos J, Le LW, Milne VC, Hope AJ, et al. Predictors of high eHealth literacy in primary lung cancer survivors. J Cancer Educ 2015 Dec;30(4):685-692. [CrossRef] [Medline]
- Robinson C, Graham J. Perceived internet health literacy of HIV-positive people through the provision of a computer and internet health education intervention. Health Info Libr J 2010 Dec;27(4):295-303 [FREE Full text] [CrossRef] [Medline]
- Xie B. Experimenting on the impact of learning methods and information presentation channels on older adults' e-health literacy. J Am Soc Inf Sci Technol 2011 Jun 06;62(9):1797-1807. [CrossRef]
- Chung S, Nahm E. Testing reliability and validity of the eHealth Literacy Scale (eHEALS) for older adults recruited online. Comput Inform Nurs 2015 Apr;33(4):150-156 [FREE Full text] [CrossRef] [Medline]
- Nguyen J, Moorhouse M, Curbow B, Christie J, Walsh-Childers K, Islam S. Construct validity of the eHealth Literacy Scale (eHEALS) among two adult populations: A Rasch analysis. JMIR Public Health Surveill 2016 May 20;2(1):e24 [FREE Full text] [CrossRef] [Medline]
- De Caro W, Corvo E, Marucci A, Mitello L, Lancia L, Sansoni J. eHealth Literacy Scale: A nursing analysis and Italian validation. Stud Health Technol Inform 2016;225:949. [Medline]
- Richtering SS, Hyun K, Neubeck L, Coorey G, Chalmers J, Usherwood T, et al. eHealth literacy: Predictors in a population with moderate-to-high cardiovascular risk. JMIR Hum Factors 2017 Jan 27;4(1):e4 [FREE Full text] [CrossRef] [Medline]
- Tennant B, Stellefson M, Dodd V, Chaney B, Chaney D, Paige S, et al. eHealth literacy and Web 2.0 health information seeking behaviors among baby boomers and older adults. J Med Internet Res 2015 Mar 17;17(3):e70 [FREE Full text] [CrossRef] [Medline]
- Choi NG, Dinitto DM. The digital divide among low-income homebound older adults: Internet use patterns, eHealth literacy, and attitudes toward computer/internet use. J Med Internet Res 2013 May 02;15(5):e93 [FREE Full text] [CrossRef] [Medline]
- van der Vaart R, van Deursen AJ, Drossaert CH, Taal E, van Dijk JA, van de Laar MA. Does the eHealth Literacy Scale (eHEALS) measure what it intends to measure? Validation of a Dutch version of the eHEALS in two adult populations. J Med Internet Res 2011 Nov 09;13(4):e86 [FREE Full text] [CrossRef] [Medline]
- Mitsutake S, Shibata A, Ishii K, Okazaki K, Oka K. [Developing Japanese version of the eHealth Literacy Scale (eHEALS)] [Article in Japanese]. Nihon Koshu Eisei Zasshi 2011 May;58(5):361-371. [Medline]
- McQueen A, Tiro JA, Vernon SW. Construct validity and invariance of four factors associated with colorectal cancer screening across gender, race, and prior screening. Cancer Epidemiol Biomarkers Prev 2008 Sep 02;17(9):2231-2237. [CrossRef]
- Vernon SW, Myers RE, Tilley BC. Development and validation of an instrument to measure factors related to colorectal cancer screening adherence. Cancer Epidemiol Biomarkers Prev 1997 Oct;6(10):825-832. [Medline]
- Davis SN, Christy SM, Chavarria EA, Abdulla R, Sutton SK, Schmidt AR, et al. A randomized controlled trial of a multicomponent, targeted, low-literacy educational intervention compared with a nontargeted intervention to boost colorectal cancer screening with fecal immunochemical testing in community clinics. Cancer 2017 Apr 15;123(8):1390-1400 [FREE Full text] [CrossRef] [Medline]
- Gwede CK, William CM, Thomas KB, Tarver WL, Quinn GP, Vadaparampil ST, et al. Exploring disparities and variability in perceptions and self-reported colorectal cancer screening among three ethnic subgroups of US Blacks. Oncol Nurs Forum 2010 Aug 26;37(5):581-591. [CrossRef]
- Christy SM, Schmidt A, Wang HL. Understanding cancer worry among patients in a community clinic-based colorectal cancer screening intervention study. Nurs Res 2018;67(4):275-285. [CrossRef]
- Christy SM, Davis SN, Williams KR, Zhao X, Govindaraju SK, Quinn GP, et al. A community-based trial of educational interventions with fecal immunochemical tests for colorectal cancer screening uptake among Blacks in community settings. Cancer 2016 Nov 15;122(21):3288-3296 [FREE Full text] [CrossRef] [Medline]
- Flight IH, Wilson CJ, McGillivray J, Myers RE. Cross-cultural validation of the preventive health model for colorectal cancer screening: An Australian study. Health Educ Behav 2010 Oct;37(5):724-736. [CrossRef] [Medline]
- Patient Engagement Playbook. Washington, DC: US Department of Health and Human Services, Office of the National Coordinator for Health Information Technology URL: https://www.healthit.gov/playbook/pe/ [accessed 2022-08-23]
- Lee HY, Jin SW, Henning-Smith C, Lee J, Lee J. Role of health literacy in health-related information-seeking behavior online: Cross-sectional study. J Med Internet Res 2021 Jan 27;23(1):e14088 [FREE Full text] [CrossRef] [Medline]
- Jacobs W, Amuta AO, Jeon KC. Health information seeking in the digital age: An analysis of health information seeking behavior among US adults. Cogent Soc Sci 2017 Mar 13;3(1):1302785 [FREE Full text] [CrossRef]
- Global Diffusion of eHealth: Making Universal Health Coverage Achievable. Report of the Third Global Survey on eHealth. Geneva, Switzerland: World Health Organization; 2016. URL: http://apps.who.int/iris/bitstream/handle/10665/252529/9789241511780-eng.pdf [accessed 2022-08-23]
- Gonzalez C, Bollinger B, Yip J, Pina L, Roldan W, Nieto Ruiz C. Intergenerational online health information searching and brokering: Framing health literacy as a family asset. Health Commun 2022 Apr;37(4):438-449. [CrossRef] [Medline]
- Tappen RM, Cooley ME, Luckmann R, Panday S. Digital health information disparities in older adults: A mixed methods study. J Racial Ethn Health Disparities 2022 Feb;9(1):82-92 [FREE Full text] [CrossRef] [Medline]
- Smith A. Older Adults and Technology Use. Washington, DC: Pew Research Center; 2014 Apr 03. URL: https://www.pewresearch.org/internet/2014/04/03/older-adults-and-technology-use/ [accessed 2022-08-23]
- Atske S, Perrin A. Home broadband adoption, computer ownership vary by race, ethnicity in the US. Pew Research Center. 2021 Jul 16. URL: https://www.pewresearch.org/fact-tank/2021/07/16/home-broadband-adoption-computer-ownership-vary-by-race-ethnicity-in-the-u-s/ [accessed 2022-08-23]
- Davis S, Govindaraju S, Jackson B. Recruitment techniques and strategies in a community-based colorectal cancer screening study of men and women of African ancestry. Nurs Res 2018;67(3):212-221. [CrossRef]
- Whitney W, Keselman A, Humphreys B. Libraries and librarians: Key partners for progress in health literacy research and practice. Inf Serv Use 2017 Mar 07;37(1):85-100. [CrossRef]
- Park H, Moon M, Baeg JH. Association of eHealth literacy with cancer information seeking and prior experience with cancer screening. Comput Inform Nurs 2014 Sep;32(9):458-463. [CrossRef] [Medline]
|CRC: colorectal cancer|
|eHEALS: eHealth Literacy Scale|
|FIT: fecal immunochemical test|
|FQHC: Federally Qualified Health Center|
|HC: health communication|
|HIT: health information technology|
|HRQOL: health-related quality of life|
|Latino CARES: Latino Colorectal Cancer Awareness, Research, Education, and Screening|
|NCI: National Cancer Institute|
|OHI: online health information|
|PHM: Preventive Health Model|
|PI: principal investigator|
|RQ1: research question 1|
|RQ2: research question 2|
|RQ3: research question 3|
|RQ4: research question 4|
Edited by A Mavragani; submitted 03.03.22; peer-reviewed by S Song, H Liu; comments to author 25.04.22; revised version received 12.07.22; accepted 09.08.22; published 13.10.22Copyright
©Enmanuel A Chavarria, Shannon M Christy, Han Feng, Hongyu Miao, Rania Abdulla, Liliana Gutierrez, Diana Lopez, Julian Sanchez, Clement K Gwede, Cathy D Meade. Originally published in JMIR Formative Research (https://formative.jmir.org), 13.10.2022.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.