JFR JMIR Form Res JMIR Formative Research 2561-326X JMIR Publications Toronto, Canada v8i1e52189 38662429 10.2196/52189 Original Paper Original Paper Assessing Electronic Health Literacy in Individuals With the Post–COVID-19 Condition Using the German Revised eHealth Literacy Scale: Validation Study Mavragani Amaryllis Graf Joachim Bacon Simon Lavoie Kim Marsall Matthias Dr 1
Institute for Patient Safety University Hospital Bonn Venusberg-Campus 1 Bonn, 53127 Germany 49 228 287 ext 11595 matthias.marsall@ukbonn.de
https://orcid.org/0000-0001-9342-3798 Dinse Hannah Dr Med 2 3 https://orcid.org/0000-0002-8855-2699 Schröder Julia 2 https://orcid.org/0000-0002-7185-9280 Skoda Eva-Maria Prof Dr Med 2 3 https://orcid.org/0000-0002-4667-5877 Teufel Martin Prof Dr Med 2 3 https://orcid.org/0000-0003-2120-1840 Bäuerle Alexander Dr 2 3 https://orcid.org/0000-0003-1488-8592 Institute for Patient Safety University Hospital Bonn Bonn Germany Clinic for Psychosomatic Medicine and Psychotherapy LVR–University Hospital Essen University of Duisburg-Essen Essen Germany Center for Translational Neuro- and Behavioral Sciences University of Duisburg-Essen Essen Germany Corresponding Author: Matthias Marsall matthias.marsall@ukbonn.de 2024 25 4 2024 8 e52189 28 8 2023 28 12 2023 12 1 2024 5 2 2024 ©Matthias Marsall, Hannah Dinse, Julia Schröder, Eva-Maria Skoda, Martin Teufel, Alexander Bäuerle. Originally published in JMIR Formative Research (https://formative.jmir.org), 25.04.2024. 2024

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.

 Background

The eHealth Literacy Scale (eHEALS) is a widely used instrument for measuring eHealth literacy (eHL). However, little is known so far about whether the instrument is valid for the assessment of eHL in persons who are affected by the post–COVID-19 condition. This is particularly important as people with the post–COVID-19 condition are frequently affected by false information from the internet.

 Objective

The objective of our study was to evaluate the validity and reliability of the German Revised eHealth Literacy Scale (GR-eHEALS) in individuals with the post–COVID-19 condition.

 Methods

A cross-sectional study was conducted from January to May 2022. The self-assessment survey consisted of the GR-eHEALS, health status– and internet use–related variables, sociodemographic data, and (post)–COVID-19–related medical data. Confirmatory factor analysis (CFA), correlational analyses, and tests of measurement invariance were deployed.

 Results

In total, 330 participants were included in the statistical analyses. CFA revealed that the 2-factor model reached an excellent model fit (comparative fit index=1.00, Tucker–Lewis index=0.99, root mean square error of approximation=0.036, standardized root mean square residual=0.038). Convergent validity was confirmed by significant positive correlations between eHL and knowledge of internet-based health promotion programs, experience in using these programs, and the duration of private internet use. In addition, a significantly negative relationship of eHL with internet anxiety supported convergent validity. Further, significant relationships of eHL with mental health status and internal health locus of control confirmed the criterion validity of the instrument. However, relationships of eHL with physical health status and quality of life could not be confirmed. The 2-factor model was fully measurement invariant regarding gender. Regarding age and educational level, partial measurement invariance was confirmed. The subscales as well as the overall GR-eHEALS reached good-to-excellent reliability (Cronbach α≥.86).

 Conclusions

The GR-eHEALS is a reliable and largely valid instrument for assessing eHL in individuals with the post–COVID-19 condition. Measurement invariance regarding gender was fully confirmed and allows the interpretation of group differences. Regarding age and educational level, group differences should be interpreted with caution. Given the high likelihood that individuals with the post–COVID-19 condition will be confronted with misinformation on the Internet, eHL is a core competency that is highly relevant in this context, in both research and clinical practice. Therefore, future research should also explore alternative instruments to capture eHL to overcome shortcomings in the validity of the GR-eHEALS.

 eHealth literacy eHEALS factor analysis measurement invariance psychometric properties infodemic Introduction

In the current phase of the COVID-19 pandemic, it has become apparent that infection with SARS-CoV-2 can be associated with the experience of prolonged physical and cognitive impairments. According to the clinical case definition by the World Health Organization, the occurrence of specific delayed symptoms about 12 weeks after recovery from COVID-19 infection is referred to as the “post–COVID-19 condition” and is estimated to affect about 10%-35% of people with infection [1,2]. Individuals with this condition regularly report physical and psychological symptoms, such as fatigue, muscular weakness, and sleep difficulties, as well as anxiety and depression [3]. Current data from Germany demonstrate that employees on sick leave due to the post–COVID-19 condition were unable to work for an average of 90 days. Moreover, people who needed inpatient treatment more than 7 days due to the post–COVID-19 condition were unable to work for an average of 168 days [4]. Data from Sweden likewise confirmed high incidences of about 13% of individuals with the post–COVID-19 condition resulting in long sick leave periods [5]. Nevertheless, studies have also reported that employees, especially health care workers, return to work despite ongoing symptoms due to the post–COVID-19 condition, which could be associated with negative long-term health effects [5,6]. These findings imply that the post–COVID-19 condition is a major public health issue with a significant impact on entire societies and public health systems. However, the etiology of the post–COVID-19 condition is not conclusively understood, and psychosocial factors may have a significant impact on physical and cognitive impairments experienced after COVID-19 infection [7,8].

In addition to the illness-related burden of the post–COVID-19 condition, individuals affected are faced with a high amount of false or confusing information, mostly spread via social media [9]. This comprises all topics around the COVID-19 pandemic, including symptoms, medication, treatments, and vaccination, as well as denial of the existence of the virus and the disease at all [10-12]. In this context, it has become apparent that individuals with COVID-19–related symptoms are susceptible to encountering misinformation, referred to as the COVID-19 “infodemic” [10,13-16]. This infodemic arose right after the beginning of the worldwide pandemic, and an analysis to investigate the proportion of false information revealed that more than 80% of the information spread via social media was presumably wrong [17]. Education about the existence of COVID-19–related misinformation and the labeling of misinformation may be relevant factors in maintaining a functioning health care system [10,18].

The fact that individuals are confronted with false or dubious health information from the internet is not a new phenomenon [19]. The concept of the competence in dealing with health information from the internet—eHealth Literacy (eHL)—was introduced back in 2006 by Norman and Skinner [20]. The concept and its measurement were based on the assumption that individuals may have competencies to seek, find, understand, and evaluate health information from the internet and to distinguish valid and reliable health information from dubious or wrong information [20]. Namely, eHL comprises peoples’ skills, knowledge, and competencies required to navigate, evaluate, and effectively use health information from digital sources, such as websites, apps, and online communication platforms, to make informed decisions about health-related issues and engage in self-care, health promotion, and disease prevention activities [20-22]. The measurement instrument developed by the same researchers, the eHealth Literacy Scale (eHEALS) [23], was translated into several languages and validated in many countries worldwide [24-33]. Over the years of research regarding eHL, studies have shown that higher eHL is associated with health-related outcomes, such as better health behaviors and health cognitions in older adults [34,35], more regular sports exercises in students [36], better health behaviors regarding physical exercise and eating behavior in adults [21], and higher physical and mental health in patients after percutaneous coronary intervention [37]. Moreover, studies have shown that individuals with higher eHL show higher adherence to prevention behavior guidelines in the context of the COVID-19 pandemic [38,39]. Vaccination is a critical factor in reducing the infection rates and severity of COVID-19 symptoms and therefore the burden on the public health system [40]. Consequently, initiatives to improve eHL could increase acceptance of vaccinations as part of a human-centered strategy to reduce the burden on the health care system [41].

In summary, humans benefit from high eHL, which empowers them to effectively use health information from the internet to cope with physical or psychological impairments. Due to the high density of information, and especially due to the high proportion of false information regarding COVID-19, people with COVID-19 or the post–COVID-19 condition have particularly high requirements for well-developed eHL. Because when people are exposed to the fact that around 80% of health information is not evidence based, it is even more difficult to identify the correct and evidence-based information from the internet and apply it to their own health situation [17]. In addition, the use of social media for obtaining health information has changed significantly during the COVID-19 pandemic [42,43]. More precisely, people obtain health information more frequently from social media sources rather than from homepages. This has fundamentally changed the way health information is obtained, which raises the question of whether eHEALS is still adequate to capture the concept of digital health literacy in the changed information era. In addition, criticism regarding the measurement is also arising with regard to the consistency of health-related outcomes as well as the actuality of eHEALS in an increasingly networked, digital environment [44-48]. Nevertheless, eHEALS is, to date, the most widely used instrument to assess eHL [44].

Therefore, a validation of eHEALS is needed to investigate the applicability of this assessment in terms of validity and reliability in a changed environment for people with the post–COVID-19 condition. In this study, we focused on the application of eHEALS in measuring eHL in German-speaking individuals with the post–COVID-19 condition. Specifically, this study aimed to fill the following research gaps:

First, we evaluated the construct validity and reliability of the German version of eHEALS, the German Revised eHealth Literacy Scale (GR-eHEALS) [49], in a sample of individuals with the post–COVID-19 condition (aim 1).

Second, we investigated the convergent validity of the GR-eHEALS (aim 2).

Third, we examined the criterion validity of the GR-eHEALS (aim 3).

Fourth, we tested the equivalence of the measurement properties of the GR-eHEALS regarding sociodemographic variables of individuals with the post–COVID-19 condition as part of the construct validity (aim 4).

 Methods Ethical Considerations

Before taking part in the survey, all participants received study information. Electronic informed consent was digitally obtained at the beginning of the survey from all participants. There was no compensation for participation in the study. Due to the anonymous study design, no individual information of participants is reported in this paper. The study was executed in accordance with the Declaration of Helsinki and was approved by the Ethics Committee of the Medical Faculty of the University of Duisburg-Essen (19-89-47-BO).

 Study Design and Participants

A cross-sectional online survey study was conducted. Data were collected via the online survey system Unipark (Tivian XI GmbH). Participation in the survey was anonymous, voluntary, and without monetary compensation. The participants of this study were recruited between January and May 2022. Flyer and information materials were laid out in different hospitals (eg, University Hospital Essen) and rehabilitation clinics (eg, MEDIAN clinics) in North Rhine-Westphalia, Germany. Furthermore, self-help communities on different social media platforms (eg, Facebook) were contacted, and online flyers were distributed. To participate in this study, the following inclusion criteria were applied: age≥18 years, internet access, a good command of the German language, confirmed COVID-19 infection in the past, and reporting as currently having post–COVID-19 symptoms. COVID-19 infection was assessed via a self-report of the date of a positive detection of SARS-CoV-2 with a nasopharyngeal swab. Symptoms related to the post–COVID-19 condition were assessed according to the clinical case definition by the World Health Organization [2]. For this purpose, participants were asked about the presence of the following symptoms: sore throat, cough, shortness of breath, headache/pain in the limbs, body temperature above 38°C, olfactory or gustatory disturbances, diarrhea, and other symptoms that could be entered via a text field. Multimedia Appendix 1 shows how frequently each of these symptoms was reported.

Participants took on average 13 (SD 5.43) minutes to complete the survey. Participants were able to stop the survey at any time. 554 people have participated in the study. According to the inclusion criteria, we excluded n=196 (35.4%) participants who reported that they currently have no post–COVID-19 symptoms. Further, we excluded participants with implausible responses regarding age (age<18 years: n=3, 0.5%; age>100: n=1, 0.2%) and outliers regarding the survey completion time (n=24, 4.3%). The final sample consisted of 330 participants, reflecting a completion rate of 59.6%.

 Measurements

All data were collected using a self-report questionnaire. The following assessments were used in the survey: eHL, constructs to examine convergent validity, constructs to examine criterion validity, and sociodemographic and COVID-19–related variables.

eHealth Literacy

The 8-item GR-eHEALS [23,49] was applied. This instrument reaches good psychometric properties regarding its validity and reliability as a 2-factor model covering the competences information seeking (IS) and information appraisal (IA) [49]. The response scale ranged from 1 for “strongly disagree” to 5 for “strongly agree,” with higher values corresponding to higher eHL.

 Constructs to Examine Convergent Validity

We assessed the knowledge of internet-based health promotion programs via 3 self-developed items using a 5-point Likert scale (1 for “strongly disagree” to 5 for “strongly agree”). Higher scores indicated higher knowledge of internet-based health promotion programs. The items covered knowledge of the contents of such programs (“I can certainly imagine something under that.”), how they work (“I know how such programs work.”), and how to find them (“I know how to find such programs.”). Cronbach α was .88. The instrument has been previously used [50,51].

One self-developed item (“Have you already had experience with internet-based health promotion programs?”) was used to assess experience in using internet-based health promotion programs on a 3-point Likert scale (1 for “already used such programs,” 2 for “not used but aware of the possibilities of such programs,” and 3 for “not aware of the possibilities of such programs”). This item was inverted, so higher values represented more experience with internet-based health promotion programs. The assessment has been used in previous studies [50,51].

We obtained information about the duration of daily private internet use through a self-developed single item: “How long do you use the internet for private purposes every day?”. The response scale ranged from 1 for “not at all” to 5 for “more than 5 hours.” Previous studies have used this item [50,51].

The construct of internet anxiety was measured on a 5-point Likert scale (1 for “does not apply” to 5 for “applies”) using 3 self-developed items (“I have concerns about using the internet,” “I am afraid that I could make an irrevocable mistake when using the internet,” and “The internet is something that worries me.”). Higher scores represented higher internet anxiety. Cronbach α was .82. These items were applied previously [49-51].

We expected significant positive relationships between eHL and knowledge of internet-based health promotion programs, experience in using internet-based health promotion programs, and duration of daily private internet use. Further, a significant negative relationship between eHL and internet anxiety was expected.

 Constructs to Examine Criterion Validity

We assessed the internal health locus of control with 3 items on a 5-point Likert scale (1 for “strongly disagree” to 5 for “strongly agree”) via an adapted German version of the Multidimensional Health Locus of Control Scale [52]. Higher scores reflected a higher internal health locus of control. Cronbach α was .76.

Moreover, we measured the physical health status (“On a scale of 0 to 10, how do you rate your physical health [eg, no physical limitations, pain]?”), mental health status (“On a scale of 0 to 10, how do you rate your mental health [eg, no feelings of anxiety, depression]?”), and overall quality of life (“On a scale of 0 to 10, how would you rate your current quality of life?”) with self-developed single items on 11-point Likert scales. These items have been used in previous studies [49,53].

We expected significant positive relationships between eHL and all criterion validity constructs.

 Sociodemographic and COVID-19–Related Variables

We assessed participants’ main sociodemographic variables: age (as an exact number), gender (female, male, diverse), and marital status (married, living in a relationship, single, divorced/widowed, other). Further, participants’ educational degree (no school degree, secondary school certificate [Hauptschule], secondary school certificate [Mittlere Reife], university entrance qualification, university degree, academic degree, other, not specified), current employment status (attending school/study, employed [part-time, full-time], sick leave, retirement/pension, not employed/other), and population of the community they live in (big city, medium city, small city, rural village) were assessed.

Regarding COVID-19, we asked participants for the date of the positive test result of SARS-CoV-2 with a nasopharyngeal swab, as well as symptoms related to the post–COVID-19 condition (inclusion criteria). In addition, we gathered information about whether participants required treatment in a hospital (yes, no) and, in addition, intensive care due to their COVID-19 infection (yes, no). Further, we asked participants for a self-assessment of the severity of their COVID-19 symptoms (1 for “no symptoms” to 4 for “severe symptoms”) as well as their current physical capacity (1 for “still significantly limited” to 3 for “good”).

There was no “I don't know” or similar option available for any of the items. The instruments and items used are provided in Multimedia Appendix 2.

 Statistical Analysis

We conducted all statistical analyses using R (R Foundation for Statistical Computing) and RStudio (Posit) [54,55] extended with several packages. We report the distribution of sociodemographic as well as COVID-19–related characteristics of the study sample. Further, descriptive statistics were used to determine the item statistics of the GR-eHEALS items.

Next, we followed analytical steps to examine the validity and reliability of the GR-eHEALS according to the study aims: First, we conducted confirmatory factor analysis (CFA) to evaluate the factorial structure and construct validity. In the first model, we tested the 1-factorial structure of all 8 items loading on a single factor, as proposed by the authors of the original instrument [23]. In the second model, we considered the 2-factorial structure with the 2 intercorrelated factors IS and IA, as validated in the GR-eHEALS [49]. For model evaluation, we considered the recommendations of Hu and Bentler [56] and defined a good model fit when reaching a comparative fit index (CFI) and a Tucker–Lewis index (TLI) of about 0.95. Further, we determined that the root mean square error of approximation (RMSEA) should be <0.06, while the standardized root mean square residual (SRMR) should be ≤0.08. For model estimation, we used the robust version of the means-adjusted unweighted least squares (ULSM) estimator as the items were measured on ordinal-scaled levels [57]. We reported Cronbach α to evaluate the reliability of the scales.

Second, regarding aims 2 and 3, we conducted Spearman rank correlation analyses to examine convergent and criterion validity. All analyses considered a significance level of P=.05, and missing values were treated via listwise deletion.

Lastly, tests of measurement invariance were deployed to assess measurement equivalence of the instrument regarding age, gender, and educational level of participants (aim 4). Measurement invariance is a statistical requirement for the correct interpretation of group differences measured by an instrument. Otherwise, if an instrument is not measurement invariant, mean differences found between different groups differences should not be interpreted, as it may be that these differences arise because the instrument measures differently in different groups [58,59]. To allow the interpretation of mean differences, a scalar level of invariance should be achieved [57]. Therefore, we performed measurement invariance tests for the gradually more restrictive levels of configural invariance, metric invariance, and, finally, scalar invariance [60]. As there is no consensus regarding cutoff criteria for the evaluation of model fits within measurement invariance testing [61], we considered 2 common approaches: First, we compared the change in the CFI between the more restrictive model and the less restrictive model, which should be ≤0.01 [62]. Second, we set the configural level of invariance as the baseline model that should meet the criteria of good model fit, as reported earlier. Subsequently, we performed χ² difference tests to compare the model of metric invariance against the model of metric invariance and then the model of scalar invariance against the model of metric invariance [61]. Significant χ² difference tests would reflect a significant change in the model fit, implying substantial deterioration of the model.

 Results Study Sample Description

The study participants’ mean age was 36.6 (SD 12.9, min.=18, max.=70 years). An overview of the participants’ sociodemographic background is reported in Table 1. Most participants did not need hospital or intensive care treatment but experienced mild-to-severe symptoms due to COVID-19 infection. Most participants also reported limited physical capacities. Table 2 presents COVID-19–related information of the study participants. On average, participants reported to have experienced 2.7 (range 1-7) of the defined post–COVID-19 symptoms.

Sociodemographic description of study participants (N=329a).

Sociodemographic characteristics Participants, n (%)
Gender
Female 264 (80.0)
Male 63 (19.1)
Diverse 2 (0.6)
Marital status
Married 106 (32.1)
Living in a relationship 107 (32.4)
Single 100 (30.3)
Divorced/widowed 11 (3.3)
Other 5 (1.5)
Educational degreeb
Secondary school diploma 77 (23.3)
University entrance qualification 97 (29.4)
University degree 148 (44.8)
Other/not specified 7 (2.1)
Employment status
Attending school/study 81 (24.5)
Employed (part-time, full-time) 118 (35.8)
Sick leave 91 (27.6)
Retirement/pension 5 (1.5)
Not employed/other 34 (10.3)
Community size
Big city (>100,000 inhabitants) 142 (43.0)
Medium city (>20,000 inhabitants) 79 (23.9)
Small city (>5000 inhabitants) 52 (15.8)
Rural village (<5000 inhabitants) 56 (17.0)

aMissing: n=1 (0.3%).

bStatements were summarized: secondary school certificate (Hauptschule) and secondary school certificate (Mittlere Reife) as secondary school diploma, and university degree and academic degree as university degree.

Post–COVID-19–related medical data of study participants (N=330).

Post–COVID-19–related information Participants, n (%)
Required treatment in hospital
Yes 18 (5.5)
No 312 (94.5)
Required intensive care
Yes 7 (2.1)
No 323 (97.9)
Severity of COVID-19 symptoms
No symptoms 16 (4.8)
Mild symptoms 137 (41.5)
Moderate symptoms 156 (47.3)
Severe symptoms 21 (6.4)
Current physical capacity
Good 58 (17.6)
Average 106 (32.1)
Still significantly limited 166 (50.3)
Psychometric Properties of the German Revised eHealth Literacy Scale

Table 3 shows the item statistics of the 8 GR-eHEALS items. The exact item wordings of the GR-eHEALS are presented in the original study [49].

Descriptive item statistics of the GR-eHEALSa items including means (SDs), skewness, kurtosis, and distribution of responses for each item (N=330).

Item Mean (SD) Skew Kurtosis Response distribution (%)
1 (strongly disagree) 2 (disagree) 3 (neutral) 4 (agree) 5 (strongly agree)
GR-eHEALS1 3.93 (0.91) –0.97 0.82 2 8 12 52 26
GR-eHEALS2 4.05 (0.82) –1.05 1.53 1 5 10 55 28
GR-eHEALS3 3.70 (0.97) –0.60 –0.20 2 12 20 47 19
GR-eHEALS4 3.80 (0.93) –0.70 –0.01 1 11 16 51 21
GR-eHEALS5 3.81 (0.84) –0.75 0.62 1 7 19 56 17
GR-eHEALS6 4.03 (0.90) –1.06 1.04 1 7 10 51 31
GR-eHEALS7 3.91 (1.01) –1.00 0.69 3 7 15 45 30
GR-eHEALS8 3.80 (0.96) –0.78 0.28 2 9 18 48 22

aGR-eHEALS: German Revised eHealth Literacy Scale.

All items had a median of 4 and showed slightly negative skewness, indicating that participants experienced high levels of eHL.

To examine the factorial structure of the GR-eHEALS, 2 rounds of CFA were performed. The results are presented in Table 4.

Results of CFAa of the GR-eHEALSb.

Model χ² (df) CFIc TLId RMSEAe SRMRf
1-factor modelg 218.14 (20) 0.98 0.97 0.088 0.079
2-factor modelh 59.96 (19) 1.00 0.99 0.036 0.038

aCFA: confirmatory factor analysis.

bGR-eHEALS: German Revised eHealth Literacy Scale.

cCFI: comparative fit index.

dTLI: Tucker–Lewis index.

eRMSEA: root mean square error of approximation.

fSRMR: standardized root mean square residual.

gModel with all 8 items loading on 1 common factor.

hModel with 2 subscales, “information seeking” (items 1-4) and “information appraisal” (items 5-8).

The 2-factor model revealed an excellent model fit, whereas the 1-factor model slightly did not meet the recommendations regarding the RMSEA and SRMR. In addition, the χ² difference test, which compared the 2 models based on the nonrobust estimator, confirmed significant model improvement for the 2-factor model over the 1-factor model (χ²1=33.18, P<.001). Consequently, the 2-factor model with the 2 interrelated factors IS and IA was used for subsequent analyses. All items reached high factor loadings (≥.73) and the 2 factors were significantly correlated, with r=0.74 (P<.001). The item factor loadings are presented in Multimedia Appendix 3. IS (mean 3.87, SD 0.80) and IA (mean 3.89, SD 0.78) reached high reliability (Cronbach α=.90 and .86, respectively).

Due to the high correlation of the subscales, the overall GR-eHEALS with all 8 items was also reported for the investigation of convergent and criterion validity. The overall GR-eHEALS (mean 3.88, SD 0.72) achieved an excellent reliability of 0.91. Therefore, the construct validity and reliability of the GR-eHEALS was confirmed.

 Convergent and Criterion Validity of the GR-eHEALS in Individuals With the Post–COVID-19 Condition

To evaluate the convergent validity and criterion validity of the GR-eHEALS, Spearman rank correlation analyses were performed. The results are presented in Table 5.

Spearman rank correlations of the GR-eHEALSa subscales and the overall GR-eHEALS with convergent and criterion validity variables.

Validity and variables ISb subscale IAc subscale Overall scale
Correlation coefficient, r P value Correlation coefficient, r P value Correlation coefficient, r P value
Convergent validity
Knowledge of internet-based health promotion programs 0.45 <.001 0.31 <.001 0.41 <.001
Experience in using internet-based health promotion programs 0.28 <.001 0.17 <.01 0.25 <.001
Duration of daily private internet use 0.15 <.01 0.16 <.01 0.16 <.01
Internet anxiety –0.20 <.001 –0.30 <.001 –0.28 <.001
Criterion validity
Physical health 0.04 .47 –0.01 .86 0.02 .71
Mental health 0.14 <.05 0.11 <.05 0.14 <.05
Quality of life 0.12 <.05 0.06 .29 0.10 .06
Internal health locus of control 0.19 <.001 0.18 <.001 0.20 <.001

aGR-eHEALS: German Revised eHealth Literacy Scale.

bIS: information seeking.

cIA: information appraisal.

IS and IA were both significantly positively associated with knowledge of and experience in using internet-based health promotion programs as well as with the duration of daily private internet use. Moreover, both factors were significantly negatively related to internet anxiety. The results were supported by similar relationships regarding the overall GR-eHEALS. These results confirm the convergent validity of the GR-eHEALS regarding the study aim 2.

Although criterion validity was supported by significantly positively relations of the IS and IA with mental health and the internal health locus of control, quality of life was only related to IS, and physical health was not related to any of the 2 factors. The overall GR-eHEALS was similarly significantly related to mental health status and the internal health locus of control but not to physical health and quality of life. Consequently, results regarding aim 3 were inconsistent.

To further understand these results, we conducted correlation analyses between all criterion validity scales. The results revealed that lower physical health is strongly associated with lower mental health (r=0.40, P<.001), a lower quality of life (r=0.69, P<.001), and a lower internal health locus of control (r=0.31, P<.001). The correlation table is presented in Multimedia Appendix 4.

IS and IA were both not associated with any of the post–COVID-19–related characteristics (need for intensive care was not tested due to the small subgroup sample size).

 Measurement Invariance of the GR-eHEALS in Individuals With the Post–COVID-19 Condition

For testing measurement invariance regarding gender, we excluded participants who indicated their gender as diverse in order to have sufficient group sample sizes for this analysis. Regarding age, we performed a median split to divide the sample at the median into 2 groups (median age 34 years). Regarding educational level, we divided the sample into 2 groups: group 1 consisted of participants with any school certificate, and group 2 consisted of participants holding a university degree. Participants who indicated other educational levels than these were excluded from this analysis. Multimedia Appendix 5 summarizes the results of the tests of measurement invariance of the GR-eHEALS regarding gender, age, and educational level.

All changes in the CFI were below 0.01, indicating measurement invariance for gender, age, and educational level. Nevertheless, χ² difference tests showed that the configural level of invariance for all 3 sociodemographic variables reached good model fit indices, implying that this level of invariance could be confirmed regarding all 3 sociodemographic variables. Regarding gender, subsequent χ² difference tests were not significant for metric and scalar levels of invariance, which is evidence for the assumption of measurement invariance of the GR-eHEALS. With respect to age and educational level, the χ² difference tests were not significant for the metric levels of measurement invariance but were significant for the scalar levels. This indicated that only partial measurement invariance of the GR-eHEALS regarding age and educational level could be confirmed. Hence, results regarding aim 4 were inconsistent.

 Discussion Principal Findings

This is the first study to evaluate the validity and reliability of the internationally most common instrument for measuring eHL in individuals with the post–COVID-19 condition. Thus, there is a gap in the knowledge of whether this instrument is still appropriate in measuring eHL in an era of an increasing number of individuals with the post–COVID-19 condition. Our study addresses this research gap and provides the first evidence by examining the measurement properties of the German version of eHEALS in individuals with the post–COVID-19 condition. The CFA results underpin the assumed factorial structure of the GR-eHEALS [49], supporting the construct validity of the instrument (aim 1). The reliability of the GR-eHEALS could be confirmed with high coefficients for both factors, IS and IA, as well as for the overall scale.

The subsequently performed correlation analyses with 4 convergent validity scales showed highly consistent results and confirmed the convergent validity of the GR-eHEALS (aim 2).

Concerning the criterion validity of the GR-eHEALS (aim 3), results did not fully meet our assumptions: physical health was not related to the GR-eHEALS, and the overall quality of life was only related to IS. However, our results are not the first that showed no relation between eHL and the physical health status [35]. In addition, this result could be explained by the fact that the study sample consisted of individuals with an impaired physical health status due to the inclusion criteria. Many of the study participants still experienced limitations in their physical capacity at the time of completing the questionnaire. Further, as shown in Multimedia Appendix 4, quality of life and physical health were strongly interrelated. That could explain why quality of life also did not confirm the expected relation to IA. In summary, most individuals in the study sample still had symptoms of the post–COVID-19 condition, which may not be affected by their eHL. It is possible that people who currently have diseases with physical health limitations cannot compensate for those limitations by high levels of eHL. Nevertheless, we found significant relations between the GR-eHEALS and mental health as well as the internal health locus of control. A high internal health locus of control reflects individuals’ confidence to influence their own health, which is associated with higher mental health status [63], better health behaviors [64], higher levels of physical activity [65], lower perceived stress [66], and higher medication adherence [67]. The fact that the internal health locus of control was strongly associated with eHL implies that this competence may play a central role when individuals are facing ambiguous information from the internet and still experience themselves as competent in handling that information and their health.

Regarding aim 4, the results of the measurement invariance tests of the GR-eHEALS were partly confirmed. Measurement invariance is important to assume that assessments measure the same underlying constructs consistently over different groups and populations [68]. If an assessment is not measurement invariant, differences between groups may be due to a bias in the assessment tool rather than true differences in the construct, which could lead to inaccurate interpretation of study results and potentially misleading recommendations for public health [69]. Measurement invariance is also an important prerequisite for the generalizability of study results [70]. This is particularly important for public health research, since study results are used to inform public policies and for the development of interventions, such as for health promotion [71,72]. Our results indicate that there are no substantial differences in the measurement model of the GR-eHEALS regarding gender. However, regarding age and educational level, measurement invariance was only partly confirmed but missed achieving scalar invariance. These results are important as studies have revealed that COVID-19 infection and its consequences are determined by gender, age, and educational level [73-77]. With the confirmation of measurement invariance of the GR-eHEALS regarding gender, we provide a valid instrument to assess and interpret differences in gender. With regard to age and educational level, group differences may only be interpreted under consideration of the partly confirmed invariance.

Summarizing, our results underpin the general construct validity, convergent validity, and reliability of the GR-eHEALS in individuals with the post–COVID-19 condition. The criterion validity of the instrument could mainly be confirmed. By partly confirming the measurement invariance, we showed that the GR-eHEALS is a valid instrument to assess and interpret eHL but is limited in terms of the interpretation of differences in age and educational level.

The overall GR-eHEALS showed consistent results in terms of convergent and criterion validity. Provided the focus is on a general examination of eHL, the overall scale can therefore be used appropriately. Nevertheless, depending on the research question, the subscales offer an opportunity to consider the competence domains of eHL in a differentiated manner.

The mean values and factor loadings of the items as well as the interrelation of the GR-eHEALS subscales were similar of those reported in the initial validation of the GR-eHEALS [49]. This is an indication that the GR-eHEALS performs and can be used similarly across different studies and in different populations. Nevertheless, the measurement invariance of the instrument across different populations should be investigated in future studies before comparing the results of different study groups. However, as the factorial structure of eHEALS is not consistent in different languages [23,78,79], further studies are needed to verify the validity of the instrument in the population with the post–COVID-19 condition.

It is important to note that the GR-eHEALS is based on the original eHEALS, which was published back in 2006 [23]. Now, the (digital) world has changed significantly. New information sources have emerged, and social media platforms have become a considerable source for health information gathering [80]. This is particularly relevant because the COVID-19 pandemic has contributed to the fact that people no longer simply obtain health information from the Internet but increasingly use social media (eg, Instagram, YouTube, Twitter) and follow individual influencers to acquire health information [81]. Therefore, the appropriateness of the use of eHEALS should be considered critically and subject to future research. Newer instruments for measuring eHL have appeared claiming to assess this construct more comprehensively and adapt to new environmental conditions [45]. Future studies should therefore establish comparative analyses of the psychometric properties of different instruments, especially in consideration of the social as well as digital environments individuals are facing in the postpandemic world. However, the study results indicate largely the validity and reliability of the GR-eHEALS to assess eHL in people after COVID-19 infection. As eHEALS is still the most widely used instrument worldwide to assess eHL [44], it offers the possibility of comparing results between different countries and populations. eHEALS thus remains an important instrument for the assessment of eHL.

 Limitations

The study sample covers a wide range of sociodemographic characteristics. Nevertheless, female participants were overrepresented in the sample, and most participants were middle aged, resulting in a small number of individuals who were already retired. This could represent a bias as young people interact with digital media differently than older people [82]. In addition, a high proportion of individuals with a university degree was present in the sample. Our survey was only accessible online, so the sample may be biased toward individuals with higher affinity to the use of digital media. Moreover, responses to the survey may have been biased by participants’ technology skills. These circumstances may limit the generalizability of our results. Exclusively self-report instruments were used, which may have introduced a response bias. As we also recorded via self-report whether a confirmed COVID-19 infection and post–COVID-19 symptoms were present, the inclusion criteria for this study were particularly affected by this restriction. Moreover, this cross-sectional study does not allow for the interpretation of causal relationships. Therefore, all interrelations found only reflect relationships without implying causal directions. The validation methods applied were based on the assumptions of classical test theory, and future studies should consider alternative paradigms, such as item response theory. Further, even if already used in other studies, the instruments used to test convergent and criterion validity were mostly not validated instruments. This could limit the external validity of the results.

 Conclusion

The validity and reliability of the GR-eHEALS could mainly be confirmed in a sample of individuals with the post–COVID-19 condition. This is the first study to examine eHL in this population, which is particularly vulnerable to misinformation. By partly confirming the measurement invariance of the instrument, we provided evidence that the GR-eHEALS is an important instrument in public health research due to its ability to be interpretable regarding differences in gender and partly regarding age and educational level. However, future research should also explore alternative instruments to capture eHL and thus consider changing behaviors of disseminating and obtaining digital health information, for example, through social media.

Number of participants reporting each post–COVID-19 symptom.

Study survey.

Item factor loadings of the German Revised eHealth Literacy Scale (GR-eHEALS).

Correlation table of criterion validity scales.

Results of tests of measurement invariance of the German Revised eHealth Literacy Scale (GR-eHEALS) regarding gender, age, and educational level.

 Abbreviations CFA

confirmatory factor analysis

 CFI

comparative fit index

 eHL

eHealth Literacy

 eHEALS

eHealth Literacy Scale

 GR-eHEALS

German Revised eHealth Literacy Scale

 IA

information appraisal

 IS

information seeking

 RMSEA

root mean square error of approximation

 SRMR

standardized root mean square residual

 TLI

Tucker–Lewis index

 ULSM

unweighted least squares estimator

This work was supported by the Open Access Publication Fund of the University of Bonn.

 Data Availability

The data sets used and analyzed during this current study are available from the corresponding author upon reasonable request.

AB, HD, and JS conceptualized the study. Project administration and data collection were performed by HD, JS, and AB. MM conducted statistical analyses and interpretation of results. MM and AB wrote the original draft of the manuscript. AB, EMS, and MT supervised the study and contributed to the study design, data collection, and critical revision of the manuscript. All authors have reviewed and approved the final manuscript.

None declared.

 Pavli A Theodoridou M Maltezou HC Post-COVID syndrome: incidence, clinical spectrum, and challenges for primary healthcare professionals Arch Med Res 2021 08 52 6 575 581 10.1016/j.arcmed.2021.03.010 33962805 S0188-4409(21)00081-3 PMC8093949 Soriano JB Murthy S Marshall JC Relan P Diaz JV WHO Clinical Case Definition Working Group on Post-COVID-19 Condition A clinical case definition of post-COVID-19 condition by a Delphi consensus Lancet Infect Dis 2022 04 22 4 e102 e107 10.1016/S1473-3099(21)00703-9 34951953 S1473-3099(21)00703-9 PMC8691845 Nalbandian A Sehgal K Gupta A Madhavan MV McGroder C Stevens JS Cook JR Nordvig AS Shalev D Sehrawat TS Ahluwalia N Bikdeli B Dietz D Der-Nigoghossian C Liyanage-Don N Rosner GF Bernstein EJ Mohan S Beckley AA Seres DS Choueiri TK Uriel N Ausiello JC Accili D Freedberg DE Baldwin M Schwartz A Brodie D Garcia CK Elkind MSV Connors JM Bilezikian JP Landry DW Wan EY Post-acute COVID-19 syndrome Nat Med 2021 04 27 4 601 615 10.1038/s41591-021-01283-z 33753937 10.1038/s41591-021-01283-z PMC8893149 TK-Gesundheitsreport 2022: Krankenstand durch Long-COVID Die Techniker - Presse & Politik 2022-07-12 https://www.tk.de/presse/themen/praevention/gesundheitsstudien/gesundheitsreport-2022-long-covid-krankenstand-2130826 Westerlind E Palstam A Sunnerhagen KS Persson HC Patterns and predictors of sick leave after Covid-19 and long Covid in a national Swedish cohort BMC Public Health 2021 05 31 21 1 1023 10.1186/s12889-021-11013-2 34059034 10.1186/s12889-021-11013-2 PMC8164957 Gaber T Ashish A Unsworth A Persistent post-covid symptoms in healthcare workers Occup Med (Lond) 2021 06 16 71 3 144 146 10.1093/occmed/kqab043 33830208 6217385 PMC8083525 Matta J Wiernik E Robineau O Carrat F Touvier M Severi G de Lamballerie X Blanché H Deleuze J Gouraud C Hoertel N Ranque B Goldberg M Zins M Lemogne C Santé‚ Pratiques‚ Relations et Inégalités Sociales en Population Générale Pendant la Crise COVID-19–Sérologie (SAPRIS-SERO) Study Group Association of self-reported COVID-19 infection and SARS-CoV-2 serology test results with persistent physical symptoms among French adults during the COVID-19 pandemic JAMA Intern Med 2022 01 01 182 1 19 25 10.1001/jamainternmed.2021.6454 34747982 2785832 PMC8576624 Staudt A Jörres RA Hinterberger T Lehnen N Loew T Budweiser S Associations of post-acute COVID syndrome with physiological and clinical measures 10 months after hospitalization in patients of the first wave Eur J Intern Med 2022 01 95 50 60 10.1016/j.ejim.2021.10.031 34838369 S0953-6205(21)00370-8 PMC8612867 Cinelli M Quattrociocchi W Galeazzi A Valensise CM Brugnoli E Schmidt AL Zola P Zollo F Scala A The COVID-19 social media infodemic Sci Rep 2020 10 06 10 1 16598 10.1038/s41598-020-73510-5 33024152 10.1038/s41598-020-73510-5 PMC7538912 Zhang S Pian W Ma F Ni Z Liu Y Characterizing the COVID-19 infodemic on Chinese social media: exploratory study JMIR Public Health Surveill 2021 02 05 7 2 e26090 10.2196/26090 33460391 v7i2e26090 PMC7869922 van der Linden S Roozenbeek J Compton J Inoculating against fake news about COVID-19 Front Psychol 2020 11 566790 10.3389/fpsyg.2020.566790 33192844 PMC7644779 Charquero-Ballester M Walter JG Nissen IA Bechmann A Different types of COVID-19 misinformation have different emotional valence on Twitter Big Data Soc 2021 09 22 8 2 205395172110412 10.1177/20539517211041279 Rovetta A Bhagavathula AS Global infodemiology of COVID-19: analysis of Google web searches and Instagram hashtags J Med Internet Res 2020 08 25 22 8 e20673 10.2196/20673 32748790 v22i8e20673 PMC7458585 Rathore FA Farooq F Information overload and infodemic in the COVID-19 pandemic J Pak Med Assoc 2020 05 70(Suppl 3) 5 S162 S165 10.5455/JPMA.38 32515403 The Lancet Infectious Diseases The COVID-19 infodemic Lancet Infect Dis 2020 08 20 8 875 10.1016/S1473-3099(20)30565-X 32687807 S1473-3099(20)30565-X PMC7367666 Eysenbach G How to fight an infodemic: the four pillars of infodemic management J Med Internet Res 2020 06 29 22 6 e21820 10.2196/21820 32589589 v22i6e21820 PMC7332253 Islam MS Sarkar T Khan SH Mostofa Kamal A Hasan SMM Kabir A Yeasmin D Islam MA Amin Chowdhury KI Anwar KS Chughtai AA Seale H COVID-19-related infodemic and its impact on public health: a global social media analysis Am J Trop Med Hyg 2020 10 103 4 1621 1629 10.4269/ajtmh.20-0812 32783794 PMC7543839 Mheidly N Fares J Leveraging media and health communication strategies to overcome the COVID-19 infodemic J Public Health Policy 2020 12 41 4 410 420 10.1057/s41271-020-00247-w 32826935 10.1057/s41271-020-00247-w PMC7441141 Wang Y McKee M Torbica A Stuckler D Systematic literature review on the spread of health-related misinformation on social media Soc Sci Med 2019 11 240 112552 10.1016/j.socscimed.2019.112552 31561111 S0277-9536(19)30546-5 PMC7117034 Norman CD Skinner HA eHealth literacy: essential skills for consumer health in a networked world J Med Internet Res 2006 06 16 8 2 e9 10.2196/jmir.8.2.e9 16867972 v8i2e9 PMC1550701 Mitsutake S Shibata A Ishii K Oka K Associations of eHealth literacy with health behavior among adult internet users J Med Internet Res 2016 07 18 18 7 e192 10.2196/jmir.5413 27432783 v18i7e192 PMC4969548 Kim K Shin S Kim S Lee E The relation between eHealth literacy and health-related behaviors: systematic review and meta-analysis J Med Internet Res 2023 01 30 25 e40778 10.2196/40778 36716080 v25i1e40778 PMC9926349 Norman CD Skinner HA eHEALS: the eHealth Literacy Scale J Med Internet Res 2006 11 14 8 4 e27 10.2196/jmir.8.4.e27 17213046 v8i4e27 PMC1794004 Rathnayake S Liyanage IP Cross-cultural adaptation and psychometric properties of the Sinhala version of electronic health literacy scale: a cross-sectional validation study PLoS One 2022 17 4 e0266515 10.1371/journal.pone.0266515 35395027 PONE-D-21-33240 PMC8993018 Shiferaw KB Validation of the Ethiopian version of eHealth Literacy Scale (ET-eHEALS) in a population with chronic disease Risk Manag Healthc Policy 2020 13 465 471 10.2147/RMHP.S240829 32547277 240829 PMC7247716 Wangdahl JM Dahlberg K Jaensson M Nilsson U Psychometric validation of Swedish and Arabic versions of two health literacy questionnaires, eHEALS and HLS-EU-Q16, for use in a Swedish context: a study protocol BMJ Open 2019 09 17 9 9 e029668 10.1136/bmjopen-2019-029668 Del Giudice P Bravo G Poletto M De Odorico A Conte A Brunelli L Arnoldo L Brusaferro S Correlation between eHealth literacy and health literacy using the eHealth Literacy Scale and real-life experiences in the health sector as a proxy measure of functional health literacy: cross-sectional web-based survey J Med Internet Res 2018 10 31 20 10 e281 10.2196/jmir.9401 30381283 v20i10e281 PMC6246968 Paramio Pérez G Almagro BJ Hernando Gómez Á Aguaded Gómez JI [Validation of the eHealth Literacy Scale (eHEALS) in Spanish university students] Rev Esp Salud Publica 2015 89 3 329 338 10.4321/S1135-57272015000300010 26388346 S1135-57272015000300010 van der Vaart R van Deursen AJ Drossaert CH Taal E van Dijk JA van de Laar MA Does the eHealth Literacy Scale (eHEALS) measure what it intends to measure? Validation of a Dutch version of the eHEALS in two adult populations J Med Internet Res 2011 11 09 13 4 e86 10.2196/jmir.1840 22071338 v13i4e86 PMC3222202 Chang A Schulz PJ The measurements and an elaborated understanding of Chinese eHealth literacy (C-eHEALS) in chronic patients in China Int J Environ Res Public Health 2018 07 23 15 7 1553 10.3390/ijerph15071553 30041420 ijerph15071553 PMC6069069 Gazibara T Cakic J Cakic M Pekmezovic T Grgurevic A eHealth and adolescents in Serbia: psychometric properties of eHeals questionnaire and contributing factors to better online health literacy Health Promot Int 2019 08 01 34 4 770 778 10.1093/heapro/day028 29800141 5004395 Chung S Park BK Nahm E The Korean eHealth Literacy Scale (K-eHEALS): reliability and validity testing in younger adults recruited online J Med Internet Res 2018 04 20 20 4 e138 10.2196/jmir.8759 29678800 v20i4e138 PMC5935806 Wijaya MC Kloping YP Validity and reliability testing of the Indonesian version of the eHealth Literacy Scale during the COVID-19 pandemic Health Inform J 2021 27 1 1460458220975466 10.1177/1460458220975466 33446030 Cui G Li S Yin Y Chen L Li J Liang F Liu X Chen L The relationship among social capital, eHealth literacy and health behaviours in Chinese elderly people: a cross-sectional study BMC Public Health 2021 01 06 21 1 45 10.1186/s12889-020-10037-4 33407275 10.1186/s12889-020-10037-4 PMC7789238 Xie L Zhang S Xin M Zhu M Lu W Mo PK Electronic health literacy and health-related outcomes among older adults: a systematic review Prev Med 2022 04 157 106997 10.1016/j.ypmed.2022.106997 35189203 S0091-7435(22)00045-7 Tsukahara S Yamaguchi S Igarashi F Uruma R Ikuina N Iwakura K Koizumi K Sato Y Association of eHealth literacy with lifestyle behaviors in university students: questionnaire-based cross-sectional study J Med Internet Res 2020 06 24 22 6 e18155 10.2196/18155 32579126 v22i6e18155 PMC7381004 Brørs G Wentzel-Larsen T Dalen H Hansen TB Norman CD Wahl A Norekvål TM CONCARD Investigators Psychometric properties of the Norwegian version of the electronic Health Literacy Scale (eHEALS) among patients after percutaneous coronary intervention: cross-sectional validation study J Med Internet Res 2020 07 28 22 7 e17312 10.2196/17312 32720900 v22i7e17312 PMC7420812 Guo Z Zhao SZ Guo N Wu Y Weng X Wong JY Lam TH Wang MP Socioeconomic disparities in eHealth literacy and preventive behaviors during the COVID-19 pandemic in Hong Kong: cross-sectional study J Med Internet Res 2021 04 14 23 4 e24577 10.2196/24577 33784240 v23i4e24577 PMC8048711 Li S Cui G Kaminga AC Cheng S Xu H Associations between health literacy, eHealth literacy, and COVID-19–related health behaviors among Chinese college students: cross-sectional online study J Med Internet Res 2021 05 06 23 5 e25600 10.2196/25600 33822734 v23i5e25600 PMC8104003 Moghadas S Vilches T Zhang K Wells C Shoukat A Singer B Meyers L Neuzil K Langley J Fitzpatrick M Galvani A The impact of vaccination on coronavirus disease 2019 (COVID-19) outbreaks in the United States Clin Infect Dis 2021 12 16 73 12 2257 2264 10.1093/cid/ciab079 33515252 6124429 PMC7929033 Nath R Imtiaz A Nath SD Hasan E Role of vaccine hesitancy, eHealth literacy, and vaccine literacy in young adults’ COVID-19 vaccine uptake intention in a lower-middle-income country Vaccines (Basel) 2021 11 29 9 12 1405 10.3390/vaccines9121405 34960151 vaccines9121405 PMC8704098 Rosen AO Holmes AL Balluerka N Hidalgo MD Gorostiaga A Gómez-Benito J Huedo-Medina TB Is social media a new type of social support? Social media use in Spain during the COVID-19 pandemic: a mixed methods study Int J Environ Res Public Health 2022 03 26 19 7 3952 10.3390/ijerph19073952 35409634 ijerph19073952 PMC8997662 Van Aelst P Toth F Castro L Štětka V Vreese CD Aalberg T Cardenal AS Corbu N Esser F Hopmann DN Koc-Michalska K Matthes J Schemer C Sheafer T Splendore S Stanyer J Stępińska A Strömbäck J Theocharis Y Does a a crisis change news habits? A comparative study of the effects of COVID-19 on news media use in 17 European countries Digital Journalism 2021 07 21 9 9 1208 1238 10.1080/21670811.2021.1943481 Lee J Lee E Chae D eHealth literacy instruments: systematic review of measurement properties J Med Internet Res 2021 11 15 23 11 e30644 10.2196/30644 34779781 v23i11e30644 PMC8663713 Paige SR Stellefson M Krieger JL Miller MD Cheong J Anderson-Lewis C Transactional eHealth literacy: developing and testing a multi-dimensional instrument J Health Commun 2019 24 10 737 748 10.1080/10810730.2019.1666940 31583963 PMC7366705 Karnoe A Kayser L How is eHealth literacy measured and what do the measurements tell us? A systematic review Knowl Manag E-Learn Int J 2015 12 15 7 4 576 600 10.34105/j.kmel.2015.07.038 Neter E Brainin E Association between health literacy, eHealth literacy, and health outcomes among patients with long-term conditions: a systematic review Eur Psychol 2019 01 24 1 68 81 10.1027/1016-9040/a000350 Tariq A Khan SR Basharat A Internet use, eHealth literacy, and dietary supplement use among young adults in Pakistan: cross-sectional study J Med Internet Res 2020 06 10 22 6 e17014 10.2196/17014 32519974 v22i6e17014 PMC7315369 Marsall M Engelmann G Skoda E Teufel M Bäuerle A Measuring electronic health literacy: development, validation, and test of measurement invariance of a revised German version of the eHealth Literacy Scale J Med Internet Res 2022 02 02 24 2 e28252 10.2196/28252 35107437 v24i2e28252 PMC8851340 Rentrop V Damerau M Schweda A Steinbach J Schüren LC Niedergethmann M Skoda E Teufel M Bäuerle A Predicting acceptance of e–mental health interventions in patients with obesity by using an extended unified theory of acceptance model: cross-sectional study JMIR Form Res 2022 03 17 6 3 e31229 10.2196/31229 35297769 v6i3e31229 PMC8972105 Bäuerle A Frewer A Rentrop V Schüren LC Niedergethmann M Lortz J Skoda E Teufel M Determinants of acceptance of weight management applications in overweight and obese individuals: using an extended unified theory of acceptance and use of technology model Nutrients 2022 05 08 14 9 1968 10.3390/nu14091968 35565935 nu14091968 PMC9101973 Hasenbring M Schüffel W Zur Adaptivität von Kontrollüberzeugungen — Empirische Befunde bei Patienten mit Krebserkrankungen, lumbalem Bandscheibenvorfall und chronischen Schmerzsyndromen Sich Gesund Fühlen Im Jahre 2000 1988 Berlin, Heidelberg Springer 222 230 Engelmann G Marsall M Skoda E Knoll-Pientka N Bäuerle L Stroebele-Benschop N Teufel M Bäuerle A Development and validation of the General Dietary Behavior Inventory (GDBI) in scope of international nutrition guidelines Nutrients 2021 04 17 13 4 1328 10.3390/nu13041328 33920515 nu13041328 PMC8073993 R: a language and environment for statistical computing R Core Team 2024-04-11 https://www.r-project.org/ RSTUDIO IDE: the most trusted IDE for open source data science Posit 2024-04-11 https://posit.co/products/open-source/rstudio/ Hu L Bentler PM Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives Struct Equ Model 1999 01 6 1 1 55 10.1080/10705519909540118 Sellbom M Tellegen A Factor analysis in psychological assessment research: common pitfalls and recommendations Psychol Assess 2019 12 31 12 1428 1441 10.1037/pas0000623 31120298 2019-28376-001 Van De Schoot R Schmidt P De Beuckelaer A Lek K Zondervan-Zwijnenburg M Editorial: measurement invariance Front Psychol 2015 07 28 6 1064 10.3389/fpsyg.2015.01064 26283995 PMC4516821 Hirschfeld G Von BR Improving multiple-group confirmatory factor analysis in R–a tutorial in measurement invariance with continuous and ordinal indicators Pract Assess Res Eval 2014 19 1 7 10.4135/9781071938959.n5 Dimitrov DM Testing for factorial invariance in the context of construct validation Meas Eval Counsel Dev 2017 03 13 43 2 121 149 10.1177/0748175610373459 Svetina D Rutkowski L Rutkowski D Multiple-group invariance with categorical outcomes using updated guidelines: an illustration using M plus and the lavaan/semTools packages Struct Equ Model 2019 04 29 27 1 111 130 10.1080/10705511.2019.1602776 Putnick DL Bornstein MH Measurement invariance conventions and reporting: the state of the art and future directions for psychological research Dev Rev 2016 09 41 71 90 10.1016/j.dr.2016.06.004 27942093 PMC5145197 Wilski M Brola W Tomczak M Health locus of control and mental health in patients with multiple sclerosis: mediating effect of coping strategies Res Nurs Health 2019 08 07 42 4 296 305 10.1002/nur.21955 31173383 Milaniak I Dębska G Król B Wierzbicki K Przybyłowski P Health locus of control and health behaviors in organ transplant recipients: a multicenter study Transplant Proc 2022 05 54 4 995 1001 10.1016/j.transproceed.2022.02.058 35660277 S0041-1345(22)00215-9 Mercer DA Ditto B Lavoie KL Campbell T Arsenault A Bacon SL Health locus of control is associated with physical activity and other health behaviors in cardiac patients J Cardiopulm Rehabil Prev 2018 11 38 6 394 399 10.1097/HCR.0000000000000350 30106790 Ganjoo M Farhadi A Baghbani R Daneshi S Nemati R Association between health locus of control and perceived stress in college student during the COVID-19 outbreak: a cross-sectional study in Iran BMC Psychiatry 2021 10 26 21 1 529 10.1186/s12888-021-03543-1 34702220 10.1186/s12888-021-03543-1 PMC8547728 Náfrádi L Nakamoto K Schulz PJ Is patient empowerment the key to promote adherence? A systematic review of the relationship between self-efficacy, health locus of control and medication adherence PLoS One 2017 10 12 10 e0186458 10.1371/journal.pone.0186458 29040335 PONE-D-16-38852 PMC5645121 Milfont TL Fischer R Testing measurement invariance across groups: applications in cross-cultural research Int J Psychol Res 2010 06 30 3 1 111 130 10.21500/20112084.857 Davidov E Meuleman B Cieciuch J Schmidt P Billiet J Measurement equivalence in cross-national research Annu Rev Sociol 2014 07 30 40 1 55 75 10.1146/annurev-soc-071913-043137 Vandenberg RJ Lance CE A review and synthesis of the measurement invariance literature: suggestions, practices, and recommendations for organizational research Org Res Methods 2016 06 29 3 1 4 70 10.1177/109442810031002 Dobrow MJ Miller FA Frank C Brown AD Understanding relevance of health research: considerations in the context of research impact assessment Health Res Policy Syst 2017 04 17 15 1 31 10.1186/s12961-017-0188-6 28412937 10.1186/s12961-017-0188-6 PMC5392970 Weise A Büchter R Pieper D Mathes T Assessing context suitability (generalizability, external validity, applicability or transferability) of findings in evidence syntheses in healthcare—an integrative review of methodological guidance Res Synth Methods 2020 11 29 11 6 760 779 10.1002/jrsm.1453 32920989 Khanijahani A Iezadi S Gholipour K Azami-Aghdash S Naghibi D A systematic review of racial/ethnic and socioeconomic disparities in COVID-19 Int J Equity Health 2021 11 24 20 1 248 10.1186/s12939-021-01582-4 34819081 10.1186/s12939-021-01582-4 PMC8611382 De Negri F Galiezz R Miranda P Koeller P Zucoloto G Costa J Farias CM Travassos GH Medronho RA Socioeconomic factors and the probability of death by COVID-19 in Brazil J Public Health (Oxf) 2021 09 22 43 3 493 498 10.1093/pubmed/fdaa279 33501982 6120274 PMC7928781 Hawkins RB Charles EJ Mehaffey JH Socio-economic status and COVID-19-related cases and fatalities Public Health 2020 12 189 129 134 10.1016/j.puhe.2020.09.016 33227595 S0033-3506(20)30435-2 PMC7568122 Jin J Bai P He W Wu F Liu X Han D Liu S Yang J Gender differences in patients with COVID-19: focus on severity and mortality Front Public Health 2020 8 152 10.3389/fpubh.2020.00152 32411652 PMC7201103 Sykes DL Holdsworth L Jawad N Gunasekera P Morice AH Crooks MG Post-COVID-19 symptom burden: what is long-COVID and how should we manage it? Lung 2021 04 199 2 113 119 10.1007/s00408-021-00423-z 33569660 10.1007/s00408-021-00423-z PMC7875681 Ma Z Wu M The psychometric properties of the Chinese eHealth Literacy Scale (C-eHEALS) in a Chinese rural population: cross-sectional validation study J Med Internet Res 2019 10 22 21 10 e15720 10.2196/15720 31642811 v21i10e15720 PMC6914234 Dale JG Lüthi A Fundingsland Skaraas B Rundereim T Dale B Testing measurement properties of the Norwegian version of electronic Health Literacy Scale (eHEALS) in a group of day surgery patients J Multidiscip Healthc 2020 13 241 247 10.2147/JMDH.S242985 32210568 242985 PMC7069557 Niu Z Willoughby J Zhou R Associations of health literacy, social media use, and self-efficacy with health information–seeking intentions among social media users in China: cross-sectional survey J Med Internet Res 2021 02 25 23 2 e19134 10.2196/19134 33629955 v23i2e19134 PMC7952238 Lim MSC Molenaar A Brennan L Reid M McCaffrey T Young adults’ use of different social media platforms for health information: insights from web-based conversations J Med Internet Res 2022 01 18 24 1 e23656 10.2196/23656 35040796 v24i1e23656 PMC8808344 Yates S Kirby J Lockley E Digital media use: differences and inequalities in relation to class and age Sociol Res Online 2015 11 30 20 4 71 91 10.5153/sro.3751