This paper is in the following e-collection/theme issue:

Development and Evaluation of Research Methods, Instruments and Tools (511) eHealth Literacy / Digital Literacy (396) Research Instruments, Questionnaires, and Tools (589) Development of Instruments and Surveys (86) Infodemiology and Infoveillance (1369)

Published on in Vol 8 (2024)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/52189, first published .
Assessing Electronic Health Literacy in Individuals With the Post–COVID-19 Condition Using the German Revised eHealth Literacy Scale: Validation Study

Assessing Electronic Health Literacy in Individuals With the Post–COVID-19 Condition Using the German Revised eHealth Literacy Scale: Validation Study

Assessing Electronic Health Literacy in Individuals With the Post–COVID-19 Condition Using the German Revised eHealth Literacy Scale: Validation Study

Authors of this article:

Matthias Marsall1 Author Orcid Image ;   Hannah Dinse2, 3 Author Orcid Image ;   Julia Schröder2 Author Orcid Image ;   Eva-Maria Skoda2, 3 Author Orcid Image ;   Martin Teufel2, 3 Author Orcid Image ;   Alexander Bäuerle2, 3 Author Orcid Image
Article Authors Cited by Tweetations (3) Metrics

    Original Paper

    1Institute for Patient Safety, University Hospital Bonn, Bonn, Germany

    2Clinic for Psychosomatic Medicine and Psychotherapy, LVR–University Hospital Essen, University of Duisburg-Essen, Essen, Germany

    3Center for Translational Neuro- and Behavioral Sciences, University of Duisburg-Essen, Essen, Germany

    Corresponding Author:

    Matthias Marsall, Dr

    Institute for Patient Safety

    University Hospital Bonn

    Venusberg-Campus 1

    Bonn, 53127

    Germany

    Phone: 49 228 287 ext 11595

    Email: matthias.marsall@ukbonn.de


    Background: The eHealth Literacy Scale (eHEALS) is a widely used instrument for measuring eHealth literacy (eHL). However, little is known so far about whether the instrument is valid for the assessment of eHL in persons who are affected by the post–COVID-19 condition. This is particularly important as people with the post–COVID-19 condition are frequently affected by false information from the internet.

    Objective: The objective of our study was to evaluate the validity and reliability of the German Revised eHealth Literacy Scale (GR-eHEALS) in individuals with the post–COVID-19 condition.

    Methods: A cross-sectional study was conducted from January to May 2022. The self-assessment survey consisted of the GR-eHEALS, health status– and internet use–related variables, sociodemographic data, and (post)–COVID-19–related medical data. Confirmatory factor analysis (CFA), correlational analyses, and tests of measurement invariance were deployed.

    Results: In total, 330 participants were included in the statistical analyses. CFA revealed that the 2-factor model reached an excellent model fit (comparative fit index=1.00, Tucker–Lewis index=0.99, root mean square error of approximation=0.036, standardized root mean square residual=0.038). Convergent validity was confirmed by significant positive correlations between eHL and knowledge of internet-based health promotion programs, experience in using these programs, and the duration of private internet use. In addition, a significantly negative relationship of eHL with internet anxiety supported convergent validity. Further, significant relationships of eHL with mental health status and internal health locus of control confirmed the criterion validity of the instrument. However, relationships of eHL with physical health status and quality of life could not be confirmed. The 2-factor model was fully measurement invariant regarding gender. Regarding age and educational level, partial measurement invariance was confirmed. The subscales as well as the overall GR-eHEALS reached good-to-excellent reliability (Cronbach α≥.86).

    Conclusions: The GR-eHEALS is a reliable and largely valid instrument for assessing eHL in individuals with the post–COVID-19 condition. Measurement invariance regarding gender was fully confirmed and allows the interpretation of group differences. Regarding age and educational level, group differences should be interpreted with caution. Given the high likelihood that individuals with the post–COVID-19 condition will be confronted with misinformation on the Internet, eHL is a core competency that is highly relevant in this context, in both research and clinical practice. Therefore, future research should also explore alternative instruments to capture eHL to overcome shortcomings in the validity of the GR-eHEALS.

    JMIR Form Res 2024;8:e52189

    doi:10.2196/52189

    Keywords

    eHealth literacy; eHEALS; factor analysis; measurement invariance; psychometric properties; infodemic 


    In the current phase of the COVID-19 pandemic, it has become apparent that infection with SARS-CoV-2 can be associated with the experience of prolonged physical and cognitive impairments. According to the clinical case definition by the World Health Organization, the occurrence of specific delayed symptoms about 12 weeks after recovery from COVID-19 infection is referred to as the “post–COVID-19 condition” and is estimated to affect about 10%-35% of people with infection [1,2]. Individuals with this condition regularly report physical and psychological symptoms, such as fatigue, muscular weakness, and sleep difficulties, as well as anxiety and depression [3]. Current data from Germany demonstrate that employees on sick leave due to the post–COVID-19 condition were unable to work for an average of 90 days. Moreover, people who needed inpatient treatment more than 7 days due to the post–COVID-19 condition were unable to work for an average of 168 days [4]. Data from Sweden likewise confirmed high incidences of about 13% of individuals with the post–COVID-19 condition resulting in long sick leave periods [5]. Nevertheless, studies have also reported that employees, especially health care workers, return to work despite ongoing symptoms due to the post–COVID-19 condition, which could be associated with negative long-term health effects [5,6]. These findings imply that the post–COVID-19 condition is a major public health issue with a significant impact on entire societies and public health systems. However, the etiology of the post–COVID-19 condition is not conclusively understood, and psychosocial factors may have a significant impact on physical and cognitive impairments experienced after COVID-19 infection [7,8].

    In addition to the illness-related burden of the post–COVID-19 condition, individuals affected are faced with a high amount of false or confusing information, mostly spread via social media [9]. This comprises all topics around the COVID-19 pandemic, including symptoms, medication, treatments, and vaccination, as well as denial of the existence of the virus and the disease at all [10-12]. In this context, it has become apparent that individuals with COVID-19–related symptoms are susceptible to encountering misinformation, referred to as the COVID-19 “infodemic” [10,13-16]. This infodemic arose right after the beginning of the worldwide pandemic, and an analysis to investigate the proportion of false information revealed that more than 80% of the information spread via social media was presumably wrong [17]. Education about the existence of COVID-19–related misinformation and the labeling of misinformation may be relevant factors in maintaining a functioning health care system [10,18].

    The fact that individuals are confronted with false or dubious health information from the internet is not a new phenomenon [19]. The concept of the competence in dealing with health information from the internet—eHealth Literacy (eHL)—was introduced back in 2006 by Norman and Skinner [20]. The concept and its measurement were based on the assumption that individuals may have competencies to seek, find, understand, and evaluate health information from the internet and to distinguish valid and reliable health information from dubious or wrong information [20]. Namely, eHL comprises peoples’ skills, knowledge, and competencies required to navigate, evaluate, and effectively use health information from digital sources, such as websites, apps, and online communication platforms, to make informed decisions about health-related issues and engage in self-care, health promotion, and disease prevention activities [20-22]. The measurement instrument developed by the same researchers, the eHealth Literacy Scale (eHEALS) [23], was translated into several languages and validated in many countries worldwide [24-33]. Over the years of research regarding eHL, studies have shown that higher eHL is associated with health-related outcomes, such as better health behaviors and health cognitions in older adults [34,35], more regular sports exercises in students [36], better health behaviors regarding physical exercise and eating behavior in adults [21], and higher physical and mental health in patients after percutaneous coronary intervention [37]. Moreover, studies have shown that individuals with higher eHL show higher adherence to prevention behavior guidelines in the context of the COVID-19 pandemic [38,39]. Vaccination is a critical factor in reducing the infection rates and severity of COVID-19 symptoms and therefore the burden on the public health system [40]. Consequently, initiatives to improve eHL could increase acceptance of vaccinations as part of a human-centered strategy to reduce the burden on the health care system [41].

    In summary, humans benefit from high eHL, which empowers them to effectively use health information from the internet to cope with physical or psychological impairments. Due to the high density of information, and especially due to the high proportion of false information regarding COVID-19, people with COVID-19 or the post–COVID-19 condition have particularly high requirements for well-developed eHL. Because when people are exposed to the fact that around 80% of health information is not evidence based, it is even more difficult to identify the correct and evidence-based information from the internet and apply it to their own health situation [17]. In addition, the use of social media for obtaining health information has changed significantly during the COVID-19 pandemic [42,43]. More precisely, people obtain health information more frequently from social media sources rather than from homepages. This has fundamentally changed the way health information is obtained, which raises the question of whether eHEALS is still adequate to capture the concept of digital health literacy in the changed information era. In addition, criticism regarding the measurement is also arising with regard to the consistency of health-related outcomes as well as the actuality of eHEALS in an increasingly networked, digital environment [44-48]. Nevertheless, eHEALS is, to date, the most widely used instrument to assess eHL [44].

    Therefore, a validation of eHEALS is needed to investigate the applicability of this assessment in terms of validity and reliability in a changed environment for people with the post–COVID-19 condition. In this study, we focused on the application of eHEALS in measuring eHL in German-speaking individuals with the post–COVID-19 condition. Specifically, this study aimed to fill the following research gaps:

    • First, we evaluated the construct validity and reliability of the German version of eHEALS, the German Revised eHealth Literacy Scale (GR-eHEALS) [49], in a sample of individuals with the post–COVID-19 condition (aim 1).
    • Second, we investigated the convergent validity of the GR-eHEALS (aim 2).
    • Third, we examined the criterion validity of the GR-eHEALS (aim 3).
    • Fourth, we tested the equivalence of the measurement properties of the GR-eHEALS regarding sociodemographic variables of individuals with the post–COVID-19 condition as part of the construct validity (aim 4).

    Ethical Considerations

    Before taking part in the survey, all participants received study information. Electronic informed consent was digitally obtained at the beginning of the survey from all participants. There was no compensation for participation in the study. Due to the anonymous study design, no individual information of participants is reported in this paper. The study was executed in accordance with the Declaration of Helsinki and was approved by the Ethics Committee of the Medical Faculty of the University of Duisburg-Essen (19-89-47-BO).

    Study Design and Participants

    A cross-sectional online survey study was conducted. Data were collected via the online survey system Unipark (Tivian XI GmbH). Participation in the survey was anonymous, voluntary, and without monetary compensation. The participants of this study were recruited between January and May 2022. Flyer and information materials were laid out in different hospitals (eg, University Hospital Essen) and rehabilitation clinics (eg, MEDIAN clinics) in North Rhine-Westphalia, Germany. Furthermore, self-help communities on different social media platforms (eg, Facebook) were contacted, and online flyers were distributed. To participate in this study, the following inclusion criteria were applied: age≥18 years, internet access, a good command of the German language, confirmed COVID-19 infection in the past, and reporting as currently having post–COVID-19 symptoms. COVID-19 infection was assessed via a self-report of the date of a positive detection of SARS-CoV-2 with a nasopharyngeal swab. Symptoms related to the post–COVID-19 condition were assessed according to the clinical case definition by the World Health Organization [2]. For this purpose, participants were asked about the presence of the following symptoms: sore throat, cough, shortness of breath, headache/pain in the limbs, body temperature above 38°C, olfactory or gustatory disturbances, diarrhea, and other symptoms that could be entered via a text field. Multimedia Appendix 1 shows how frequently each of these symptoms was reported.

    Participants took on average 13 (SD 5.43) minutes to complete the survey. Participants were able to stop the survey at any time. 554 people have participated in the study. According to the inclusion criteria, we excluded n=196 (35.4%) participants who reported that they currently have no post–COVID-19 symptoms. Further, we excluded participants with implausible responses regarding age (age<18 years: n=3, 0.5%; age>100: n=1, 0.2%) and outliers regarding the survey completion time (n=24, 4.3%). The final sample consisted of 330 participants, reflecting a completion rate of 59.6%.

    Measurements

    All data were collected using a self-report questionnaire. The following assessments were used in the survey: eHL, constructs to examine convergent validity, constructs to examine criterion validity, and sociodemographic and COVID-19–related variables.

    eHealth Literacy

    The 8-item GR-eHEALS [23,49] was applied. This instrument reaches good psychometric properties regarding its validity and reliability as a 2-factor model covering the competences information seeking (IS) and information appraisal (IA) [49]. The response scale ranged from 1 for “strongly disagree” to 5 for “strongly agree,” with higher values corresponding to higher eHL.

    Constructs to Examine Convergent Validity

    We assessed the knowledge of internet-based health promotion programs via 3 self-developed items using a 5-point Likert scale (1 for “strongly disagree” to 5 for “strongly agree”). Higher scores indicated higher knowledge of internet-based health promotion programs. The items covered knowledge of the contents of such programs (“I can certainly imagine something under that.”), how they work (“I know how such programs work.”), and how to find them (“I know how to find such programs.”). Cronbach α was .88. The instrument has been previously used [50,51].

    One self-developed item (“Have you already had experience with internet-based health promotion programs?”) was used to assess experience in using internet-based health promotion programs on a 3-point Likert scale (1 for “already used such programs,” 2 for “not used but aware of the possibilities of such programs,” and 3 for “not aware of the possibilities of such programs”). This item was inverted, so higher values represented more experience with internet-based health promotion programs. The assessment has been used in previous studies [50,51].

    We obtained information about the duration of daily private internet use through a self-developed single item: “How long do you use the internet for private purposes every day?”. The response scale ranged from 1 for “not at all” to 5 for “more than 5 hours.” Previous studies have used this item [50,51].

    The construct of internet anxiety was measured on a 5-point Likert scale (1 for “does not apply” to 5 for “applies”) using 3 self-developed items (“I have concerns about using the internet,” “I am afraid that I could make an irrevocable mistake when using the internet,” and “The internet is something that worries me.”). Higher scores represented higher internet anxiety. Cronbach α was .82. These items were applied previously [49-51].

    We expected significant positive relationships between eHL and knowledge of internet-based health promotion programs, experience in using internet-based health promotion programs, and duration of daily private internet use. Further, a significant negative relationship between eHL and internet anxiety was expected.

    Constructs to Examine Criterion Validity

    We assessed the internal health locus of control with 3 items on a 5-point Likert scale (1 for “strongly disagree” to 5 for “strongly agree”) via an adapted German version of the Multidimensional Health Locus of Control Scale [52]. Higher scores reflected a higher internal health locus of control. Cronbach α was .76.

    Moreover, we measured the physical health status (“On a scale of 0 to 10, how do you rate your physical health [eg, no physical limitations, pain]?”), mental health status (“On a scale of 0 to 10, how do you rate your mental health [eg, no feelings of anxiety, depression]?”), and overall quality of life (“On a scale of 0 to 10, how would you rate your current quality of life?”) with self-developed single items on 11-point Likert scales. These items have been used in previous studies [49,53].

    We expected significant positive relationships between eHL and all criterion validity constructs.

    Sociodemographic and COVID-19–Related Variables

    We assessed participants’ main sociodemographic variables: age (as an exact number), gender (female, male, diverse), and marital status (married, living in a relationship, single, divorced/widowed, other). Further, participants’ educational degree (no school degree, secondary school certificate [Hauptschule], secondary school certificate [Mittlere Reife], university entrance qualification, university degree, academic degree, other, not specified), current employment status (attending school/study, employed [part-time, full-time], sick leave, retirement/pension, not employed/other), and population of the community they live in (big city, medium city, small city, rural village) were assessed.

    Regarding COVID-19, we asked participants for the date of the positive test result of SARS-CoV-2 with a nasopharyngeal swab, as well as symptoms related to the post–COVID-19 condition (inclusion criteria). In addition, we gathered information about whether participants required treatment in a hospital (yes, no) and, in addition, intensive care due to their COVID-19 infection (yes, no). Further, we asked participants for a self-assessment of the severity of their COVID-19 symptoms (1 for “no symptoms” to 4 for “severe symptoms”) as well as their current physical capacity (1 for “still significantly limited” to 3 for “good”).

    There was no “I don't know” or similar option available for any of the items. The instruments and items used are provided in Multimedia Appendix 2.

    Statistical Analysis

    We conducted all statistical analyses using R (R Foundation for Statistical Computing) and RStudio (Posit) [54,55] extended with several packages. We report the distribution of sociodemographic as well as COVID-19–related characteristics of the study sample. Further, descriptive statistics were used to determine the item statistics of the GR-eHEALS items.

    Next, we followed analytical steps to examine the validity and reliability of the GR-eHEALS according to the study aims: First, we conducted confirmatory factor analysis (CFA) to evaluate the factorial structure and construct validity. In the first model, we tested the 1-factorial structure of all 8 items loading on a single factor, as proposed by the authors of the original instrument [23]. In the second model, we considered the 2-factorial structure with the 2 intercorrelated factors IS and IA, as validated in the GR-eHEALS [49]. For model evaluation, we considered the recommendations of Hu and Bentler [56] and defined a good model fit when reaching a comparative fit index (CFI) and a Tucker–Lewis index (TLI) of about 0.95. Further, we determined that the root mean square error of approximation (RMSEA) should be <0.06, while the standardized root mean square residual (SRMR) should be ≤0.08. For model estimation, we used the robust version of the means-adjusted unweighted least squares (ULSM) estimator as the items were measured on ordinal-scaled levels [57]. We reported Cronbach α to evaluate the reliability of the scales.

    Second, regarding aims 2 and 3, we conducted Spearman rank correlation analyses to examine convergent and criterion validity. All analyses considered a significance level of P=.05, and missing values were treated via listwise deletion.

    Lastly, tests of measurement invariance were deployed to assess measurement equivalence of the instrument regarding age, gender, and educational level of participants (aim 4). Measurement invariance is a statistical requirement for the correct interpretation of group differences measured by an instrument. Otherwise, if an instrument is not measurement invariant, mean differences found between different groups differences should not be interpreted, as it may be that these differences arise because the instrument measures differently in different groups [58,59]. To allow the interpretation of mean differences, a scalar level of invariance should be achieved [57]. Therefore, we performed measurement invariance tests for the gradually more restrictive levels of configural invariance, metric invariance, and, finally, scalar invariance [60]. As there is no consensus regarding cutoff criteria for the evaluation of model fits within measurement invariance testing [61], we considered 2 common approaches: First, we compared the change in the CFI between the more restrictive model and the less restrictive model, which should be ≤0.01 [62]. Second, we set the configural level of invariance as the baseline model that should meet the criteria of good model fit, as reported earlier. Subsequently, we performed χ² difference tests to compare the model of metric invariance against the model of metric invariance and then the model of scalar invariance against the model of metric invariance [61]. Significant χ² difference tests would reflect a significant change in the model fit, implying substantial deterioration of the model.


    Study Sample Description

    The study participants’ mean age was 36.6 (SD 12.9, min.=18, max.=70 years). An overview of the participants’ sociodemographic background is reported in Table 1. Most participants did not need hospital or intensive care treatment but experienced mild-to-severe symptoms due to COVID-19 infection. Most participants also reported limited physical capacities. Table 2 presents COVID-19–related information of the study participants. On average, participants reported to have experienced 2.7 (range 1-7) of the defined post–COVID-19 symptoms.

    Table 1. Sociodemographic description of study participants (N=329a).
    Sociodemographic characteristicsParticipants, n (%)
    Gender

    		Female264 (80.0)

    		Male63 (19.1)

    		Diverse2 (0.6)
    Marital status

    		Married106 (32.1)

    		Living in a relationship107 (32.4)

    		Single100 (30.3)

    		Divorced/widowed11 (3.3)

    		Other5 (1.5)
    Educational degreeb

    		Secondary school diploma77 (23.3)

    		University entrance qualification97 (29.4)

    		University degree148 (44.8)

    		Other/not specified7 (2.1)
    Employment status

    		Attending school/study81 (24.5)

    		Employed (part-time, full-time)118 (35.8)

    		Sick leave91 (27.6)

    		Retirement/pension5 (1.5)

    		Not employed/other34 (10.3)
    Community size

    		Big city (>100,000 inhabitants)142 (43.0)

    		Medium city (>20,000 inhabitants)79 (23.9)

    		Small city (>5000 inhabitants)52 (15.8)

    		Rural village (<5000 inhabitants)56 (17.0)

    aMissing: n=1 (0.3%).

    bStatements were summarized: secondary school certificate (Hauptschule) and secondary school certificate (Mittlere Reife) as secondary school diploma, and university degree and academic degree as university degree.

    Table 2. Post–COVID-19–related medical data of study participants (N=330).
    Post–COVID-19–related informationParticipants, n (%)
    Required treatment in hospital

    		Yes18 (5.5)

    		No312 (94.5)
    Required intensive care

    		Yes7 (2.1)

    		No323 (97.9)
    Severity of COVID-19 symptoms

    		No symptoms16 (4.8)

    		Mild symptoms137 (41.5)

    		Moderate symptoms156 (47.3)

    		Severe symptoms21 (6.4)
    Current physical capacity

    		Good58 (17.6)

    		Average106 (32.1)

    		Still significantly limited166 (50.3)

    Psychometric Properties of the German Revised eHealth Literacy Scale

    Table 3 shows the item statistics of the 8 GR-eHEALS items. The exact item wordings of the GR-eHEALS are presented in the original study [49].

    Table 3. Descriptive item statistics of the GR-eHEALSa items including means (SDs), skewness, kurtosis, and distribution of responses for each item (N=330).
    ItemMean (SD)SkewKurtosisResponse distribution (%)
    1 (strongly disagree)2 (disagree)3 (neutral)4 (agree)5 (strongly agree)
    GR-eHEALS13.93 (0.91)–0.970.8228125226
    GR-eHEALS24.05 (0.82)–1.051.5315105528
    GR-eHEALS33.70 (0.97)–0.60–0.20212204719
    GR-eHEALS43.80 (0.93)–0.70–0.01111165121
    GR-eHEALS53.81 (0.84)–0.750.6217195617
    GR-eHEALS64.03 (0.90)–1.061.0417105131
    GR-eHEALS73.91 (1.01)–1.000.6937154530
    GR-eHEALS83.80 (0.96)–0.780.2829184822

    aGR-eHEALS: German Revised eHealth Literacy Scale.

    All items had a median of 4 and showed slightly negative skewness, indicating that participants experienced high levels of eHL.

    To examine the factorial structure of the GR-eHEALS, 2 rounds of CFA were performed. The results are presented in Table 4.

    Table 4. Results of CFAa of the GR-eHEALSb.
    Modelχ² (df)CFIcTLIdRMSEAeSRMRf
    1-factor modelg218.14 (20)0.980.970.0880.079
    2-factor modelh59.96 (19)1.000.990.0360.038

    aCFA: confirmatory factor analysis.

    bGR-eHEALS: German Revised eHealth Literacy Scale.

    cCFI: comparative fit index.

    dTLI: Tucker–Lewis index.

    eRMSEA: root mean square error of approximation.

    fSRMR: standardized root mean square residual.

    gModel with all 8 items loading on 1 common factor.

    hModel with 2 subscales, “information seeking” (items 1-4) and “information appraisal” (items 5-8).

    The 2-factor model revealed an excellent model fit, whereas the 1-factor model slightly did not meet the recommendations regarding the RMSEA and SRMR. In addition, the χ² difference test, which compared the 2 models based on the nonrobust estimator, confirmed significant model improvement for the 2-factor model over the 1-factor model (χ²1=33.18, P<.001). Consequently, the 2-factor model with the 2 interrelated factors IS and IA was used for subsequent analyses. All items reached high factor loadings (≥.73) and the 2 factors were significantly correlated, with r=0.74 (P<.001). The item factor loadings are presented in Multimedia Appendix 3. IS (mean 3.87, SD 0.80) and IA (mean 3.89, SD 0.78) reached high reliability (Cronbach α=.90 and .86, respectively).

    Due to the high correlation of the subscales, the overall GR-eHEALS with all 8 items was also reported for the investigation of convergent and criterion validity. The overall GR-eHEALS (mean 3.88, SD 0.72) achieved an excellent reliability of 0.91. Therefore, the construct validity and reliability of the GR-eHEALS was confirmed.

    Convergent and Criterion Validity of the GR-eHEALS in Individuals With the Post–COVID-19 Condition

    To evaluate the convergent validity and criterion validity of the GR-eHEALS, Spearman rank correlation analyses were performed. The results are presented in Table 5.

    Table 5. Spearman rank correlations of the GR-eHEALSa subscales and the overall GR-eHEALS with convergent and criterion validity variables.
    Validity and variablesISb subscaleIAc subscaleOverall scale

    		Correlation coefficient, rP valueCorrelation coefficient, rP valueCorrelation coefficient, rP value
    Convergent validity

    		Knowledge of internet-based health promotion programs0.45<.0010.31<.0010.41<.001

    		Experience in using internet-based health promotion programs0.28<.0010.17<.010.25<.001

    		Duration of daily private internet use0.15<.010.16<.010.16<.01

    		Internet anxiety–0.20<.001–0.30<.001–0.28<.001
    Criterion validity

    		Physical health0.04.47–0.01.860.02.71

    		Mental health0.14<.050.11<.050.14<.05

    		Quality of life0.12<.050.06.290.10.06

    		Internal health locus of control0.19<.0010.18<.0010.20 <.001

    aGR-eHEALS: German Revised eHealth Literacy Scale.

    bIS: information seeking.

    cIA: information appraisal.

    IS and IA were both significantly positively associated with knowledge of and experience in using internet-based health promotion programs as well as with the duration of daily private internet use. Moreover, both factors were significantly negatively related to internet anxiety. The results were supported by similar relationships regarding the overall GR-eHEALS. These results confirm the convergent validity of the GR-eHEALS regarding the study aim 2.

    Although criterion validity was supported by significantly positively relations of the IS and IA with mental health and the internal health locus of control, quality of life was only related to IS, and physical health was not related to any of the 2 factors. The overall GR-eHEALS was similarly significantly related to mental health status and the internal health locus of control but not to physical health and quality of life. Consequently, results regarding aim 3 were inconsistent.

    To further understand these results, we conducted correlation analyses between all criterion validity scales. The results revealed that lower physical health is strongly associated with lower mental health (r=0.40, P<.001), a lower quality of life (r=0.69, P<.001), and a lower internal health locus of control (r=0.31, P<.001). The correlation table is presented in Multimedia Appendix 4.

    IS and IA were both not associated with any of the post–COVID-19–related characteristics (need for intensive care was not tested due to the small subgroup sample size).

    Measurement Invariance of the GR-eHEALS in Individuals With the Post–COVID-19 Condition

    For testing measurement invariance regarding gender, we excluded participants who indicated their gender as diverse in order to have sufficient group sample sizes for this analysis. Regarding age, we performed a median split to divide the sample at the median into 2 groups (median age 34 years). Regarding educational level, we divided the sample into 2 groups: group 1 consisted of participants with any school certificate, and group 2 consisted of participants holding a university degree. Participants who indicated other educational levels than these were excluded from this analysis. Multimedia Appendix 5 summarizes the results of the tests of measurement invariance of the GR-eHEALS regarding gender, age, and educational level.

    All changes in the CFI were below 0.01, indicating measurement invariance for gender, age, and educational level. Nevertheless, χ² difference tests showed that the configural level of invariance for all 3 sociodemographic variables reached good model fit indices, implying that this level of invariance could be confirmed regarding all 3 sociodemographic variables. Regarding gender, subsequent χ² difference tests were not significant for metric and scalar levels of invariance, which is evidence for the assumption of measurement invariance of the GR-eHEALS. With respect to age and educational level, the χ² difference tests were not significant for the metric levels of measurement invariance but were significant for the scalar levels. This indicated that only partial measurement invariance of the GR-eHEALS regarding age and educational level could be confirmed. Hence, results regarding aim 4 were inconsistent.


    Principal Findings

    This is the first study to evaluate the validity and reliability of the internationally most common instrument for measuring eHL in individuals with the post–COVID-19 condition. Thus, there is a gap in the knowledge of whether this instrument is still appropriate in measuring eHL in an era of an increasing number of individuals with the post–COVID-19 condition. Our study addresses this research gap and provides the first evidence by examining the measurement properties of the German version of eHEALS in individuals with the post–COVID-19 condition. The CFA results underpin the assumed factorial structure of the GR-eHEALS [49], supporting the construct validity of the instrument (aim 1). The reliability of the GR-eHEALS could be confirmed with high coefficients for both factors, IS and IA, as well as for the overall scale.

    The subsequently performed correlation analyses with 4 convergent validity scales showed highly consistent results and confirmed the convergent validity of the GR-eHEALS (aim 2).

    Concerning the criterion validity of the GR-eHEALS (aim 3), results did not fully meet our assumptions: physical health was not related to the GR-eHEALS, and the overall quality of life was only related to IS. However, our results are not the first that showed no relation between eHL and the physical health status [35]. In addition, this result could be explained by the fact that the study sample consisted of individuals with an impaired physical health status due to the inclusion criteria. Many of the study participants still experienced limitations in their physical capacity at the time of completing the questionnaire. Further, as shown in Multimedia Appendix 4, quality of life and physical health were strongly interrelated. That could explain why quality of life also did not confirm the expected relation to IA. In summary, most individuals in the study sample still had symptoms of the post–COVID-19 condition, which may not be affected by their eHL. It is possible that people who currently have diseases with physical health limitations cannot compensate for those limitations by high levels of eHL. Nevertheless, we found significant relations between the GR-eHEALS and mental health as well as the internal health locus of control. A high internal health locus of control reflects individuals’ confidence to influence their own health, which is associated with higher mental health status [63], better health behaviors [64], higher levels of physical activity [65], lower perceived stress [66], and higher medication adherence [67]. The fact that the internal health locus of control was strongly associated with eHL implies that this competence may play a central role when individuals are facing ambiguous information from the internet and still experience themselves as competent in handling that information and their health.

    Regarding aim 4, the results of the measurement invariance tests of the GR-eHEALS were partly confirmed. Measurement invariance is important to assume that assessments measure the same underlying constructs consistently over different groups and populations [68]. If an assessment is not measurement invariant, differences between groups may be due to a bias in the assessment tool rather than true differences in the construct, which could lead to inaccurate interpretation of study results and potentially misleading recommendations for public health [69]. Measurement invariance is also an important prerequisite for the generalizability of study results [70]. This is particularly important for public health research, since study results are used to inform public policies and for the development of interventions, such as for health promotion [71,72]. Our results indicate that there are no substantial differences in the measurement model of the GR-eHEALS regarding gender. However, regarding age and educational level, measurement invariance was only partly confirmed but missed achieving scalar invariance. These results are important as studies have revealed that COVID-19 infection and its consequences are determined by gender, age, and educational level [73-77]. With the confirmation of measurement invariance of the GR-eHEALS regarding gender, we provide a valid instrument to assess and interpret differences in gender. With regard to age and educational level, group differences may only be interpreted under consideration of the partly confirmed invariance.

    Summarizing, our results underpin the general construct validity, convergent validity, and reliability of the GR-eHEALS in individuals with the post–COVID-19 condition. The criterion validity of the instrument could mainly be confirmed. By partly confirming the measurement invariance, we showed that the GR-eHEALS is a valid instrument to assess and interpret eHL but is limited in terms of the interpretation of differences in age and educational level.

    The overall GR-eHEALS showed consistent results in terms of convergent and criterion validity. Provided the focus is on a general examination of eHL, the overall scale can therefore be used appropriately. Nevertheless, depending on the research question, the subscales offer an opportunity to consider the competence domains of eHL in a differentiated manner.

    The mean values and factor loadings of the items as well as the interrelation of the GR-eHEALS subscales were similar of those reported in the initial validation of the GR-eHEALS [49]. This is an indication that the GR-eHEALS performs and can be used similarly across different studies and in different populations. Nevertheless, the measurement invariance of the instrument across different populations should be investigated in future studies before comparing the results of different study groups. However, as the factorial structure of eHEALS is not consistent in different languages [23,78,79], further studies are needed to verify the validity of the instrument in the population with the post–COVID-19 condition.

    It is important to note that the GR-eHEALS is based on the original eHEALS, which was published back in 2006 [23]. Now, the (digital) world has changed significantly. New information sources have emerged, and social media platforms have become a considerable source for health information gathering [80]. This is particularly relevant because the COVID-19 pandemic has contributed to the fact that people no longer simply obtain health information from the Internet but increasingly use social media (eg, Instagram, YouTube, Twitter) and follow individual influencers to acquire health information [81]. Therefore, the appropriateness of the use of eHEALS should be considered critically and subject to future research. Newer instruments for measuring eHL have appeared claiming to assess this construct more comprehensively and adapt to new environmental conditions [45]. Future studies should therefore establish comparative analyses of the psychometric properties of different instruments, especially in consideration of the social as well as digital environments individuals are facing in the postpandemic world. However, the study results indicate largely the validity and reliability of the GR-eHEALS to assess eHL in people after COVID-19 infection. As eHEALS is still the most widely used instrument worldwide to assess eHL [44], it offers the possibility of comparing results between different countries and populations. eHEALS thus remains an important instrument for the assessment of eHL.

    Limitations

    The study sample covers a wide range of sociodemographic characteristics. Nevertheless, female participants were overrepresented in the sample, and most participants were middle aged, resulting in a small number of individuals who were already retired. This could represent a bias as young people interact with digital media differently than older people [82]. In addition, a high proportion of individuals with a university degree was present in the sample. Our survey was only accessible online, so the sample may be biased toward individuals with higher affinity to the use of digital media. Moreover, responses to the survey may have been biased by participants’ technology skills. These circumstances may limit the generalizability of our results. Exclusively self-report instruments were used, which may have introduced a response bias. As we also recorded via self-report whether a confirmed COVID-19 infection and post–COVID-19 symptoms were present, the inclusion criteria for this study were particularly affected by this restriction. Moreover, this cross-sectional study does not allow for the interpretation of causal relationships. Therefore, all interrelations found only reflect relationships without implying causal directions. The validation methods applied were based on the assumptions of classical test theory, and future studies should consider alternative paradigms, such as item response theory. Further, even if already used in other studies, the instruments used to test convergent and criterion validity were mostly not validated instruments. This could limit the external validity of the results.

    Conclusion

    The validity and reliability of the GR-eHEALS could mainly be confirmed in a sample of individuals with the post–COVID-19 condition. This is the first study to examine eHL in this population, which is particularly vulnerable to misinformation. By partly confirming the measurement invariance of the instrument, we provided evidence that the GR-eHEALS is an important instrument in public health research due to its ability to be interpretable regarding differences in gender and partly regarding age and educational level. However, future research should also explore alternative instruments to capture eHL and thus consider changing behaviors of disseminating and obtaining digital health information, for example, through social media.

    Acknowledgments

    This work was supported by the Open Access Publication Fund of the University of Bonn.

    Data Availability

    The data sets used and analyzed during this current study are available from the corresponding author upon reasonable request.

    Authors' Contributions

    AB, HD, and JS conceptualized the study. Project administration and data collection were performed by HD, JS, and AB. MM conducted statistical analyses and interpretation of results. MM and AB wrote the original draft of the manuscript. AB, EMS, and MT supervised the study and contributed to the study design, data collection, and critical revision of the manuscript. All authors have reviewed and approved the final manuscript.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Number of participants reporting each post–COVID-19 symptom.

    DOCX File , 19 KB
    Multimedia Appendix 2

    Study survey.

    DOCX File , 28 KB
    Multimedia Appendix 3

    Item factor loadings of the German Revised eHealth Literacy Scale (GR-eHEALS).

    DOCX File , 19 KB
    Multimedia Appendix 4

    Correlation table of criterion validity scales.

    DOCX File , 21 KB
    Multimedia Appendix 5

    Results of tests of measurement invariance of the German Revised eHealth Literacy Scale (GR-eHEALS) regarding gender, age, and educational level.

    DOCX File , 22 KB
    1. Pavli A, Theodoridou M, Maltezou HC. Post-COVID syndrome: incidence, clinical spectrum, and challenges for primary healthcare professionals. Arch Med Res. Aug 2021;52(6):575-581. [FREE Full text] [CrossRef] [Medline]
    2. Soriano JB, Murthy S, Marshall JC, Relan P, Diaz JV, WHO Clinical Case Definition Working Group on Post-COVID-19 Condition. A clinical case definition of post-COVID-19 condition by a Delphi consensus. Lancet Infect Dis. Apr 2022;22(4):e102-e107. [FREE Full text] [CrossRef] [Medline]
    3. Nalbandian A, Sehgal K, Gupta A, Madhavan MV, McGroder C, Stevens JS, et al. Post-acute COVID-19 syndrome. Nat Med. Apr 2021;27(4):601-615. [FREE Full text] [CrossRef] [Medline]
    4. TK-Gesundheitsreport 2022: Krankenstand durch Long-COVID. Die Techniker - Presse & Politik. URL: https:/​/www.​tk.de/​presse/​themen/​praevention/​gesundheitsstudien/​gesundheitsreport-2022-long-covid-krankenstand-2130826 [accessed 2022-07-12]
    5. Westerlind E, Palstam A, Sunnerhagen KS, Persson HC. Patterns and predictors of sick leave after Covid-19 and long Covid in a national Swedish cohort. BMC Public Health. May 31, 2021;21(1):1023. [FREE Full text] [CrossRef] [Medline]
    6. Gaber T, Ashish A, Unsworth A. Persistent post-covid symptoms in healthcare workers. Occup Med (Lond). Jun 16, 2021;71(3):144-146. [FREE Full text] [CrossRef] [Medline]
    7. Matta J, Wiernik E, Robineau O, Carrat F, Touvier M, Severi G, et al. Santé‚ Pratiques‚ Relations et Inégalités Sociales en Population Générale Pendant la Crise COVID-19–Sérologie (SAPRIS-SERO) Study Group. Association of self-reported COVID-19 infection and SARS-CoV-2 serology test results with persistent physical symptoms among French adults during the COVID-19 pandemic. JAMA Intern Med. Jan 01, 2022;182(1):19-25. [FREE Full text] [CrossRef] [Medline]
    8. Staudt A, Jörres RA, Hinterberger T, Lehnen N, Loew T, Budweiser S. Associations of post-acute COVID syndrome with physiological and clinical measures 10 months after hospitalization in patients of the first wave. Eur J Intern Med. Jan 2022;95:50-60. [FREE Full text] [CrossRef] [Medline]
    9. Cinelli M, Quattrociocchi W, Galeazzi A, Valensise CM, Brugnoli E, Schmidt AL, et al. The COVID-19 social media infodemic. Sci Rep. Oct 06, 2020;10(1):16598. [FREE Full text] [CrossRef] [Medline]
    10. Zhang S, Pian W, Ma F, Ni Z, Liu Y. Characterizing the COVID-19 infodemic on Chinese social media: exploratory study. JMIR Public Health Surveill. Feb 05, 2021;7(2):e26090. [FREE Full text] [CrossRef] [Medline]
    11. van der Linden S, Roozenbeek J, Compton J. Inoculating against fake news about COVID-19. Front Psychol. 2020;11:566790. [FREE Full text] [CrossRef] [Medline]
    12. Charquero-Ballester M, Walter JG, Nissen IA, Bechmann A. Different types of COVID-19 misinformation have different emotional valence on Twitter. Big Data Soc. Sep 22, 2021;8(2):205395172110412. [CrossRef]
    13. Rovetta A, Bhagavathula AS. Global infodemiology of COVID-19: analysis of Google web searches and Instagram hashtags. J Med Internet Res. Aug 25, 2020;22(8):e20673. [FREE Full text] [CrossRef] [Medline]
    14. Rathore FA, Farooq F. Information overload and infodemic in the COVID-19 pandemic. J Pak Med Assoc. May 2020;70(Suppl 3)(5):S162-S165. [FREE Full text] [CrossRef] [Medline]
    15. The Lancet Infectious Diseases. The COVID-19 infodemic. Lancet Infect Dis. Aug 2020;20(8):875. [FREE Full text] [CrossRef] [Medline]
    16. Eysenbach G. How to fight an infodemic: the four pillars of infodemic management. J Med Internet Res. Jun 29, 2020;22(6):e21820. [FREE Full text] [CrossRef] [Medline]
    17. Islam MS, Sarkar T, Khan SH, Mostofa Kamal A, Hasan SMM, Kabir A, et al. COVID-19-related infodemic and its impact on public health: a global social media analysis. Am J Trop Med Hyg. Oct 2020;103(4):1621-1629. [FREE Full text] [CrossRef] [Medline]
    18. Mheidly N, Fares J. Leveraging media and health communication strategies to overcome the COVID-19 infodemic. J Public Health Policy. Dec 2020;41(4):410-420. [FREE Full text] [CrossRef] [Medline]
    19. Wang Y, McKee M, Torbica A, Stuckler D. Systematic literature review on the spread of health-related misinformation on social media. Soc Sci Med. Nov 2019;240:112552. [FREE Full text] [CrossRef] [Medline]
    20. Norman CD, Skinner HA. eHealth literacy: essential skills for consumer health in a networked world. J Med Internet Res. Jun 16, 2006;8(2):e9. [FREE Full text] [CrossRef] [Medline]
    21. Mitsutake S, Shibata A, Ishii K, Oka K. Associations of eHealth literacy with health behavior among adult internet users. J Med Internet Res. Jul 18, 2016;18(7):e192. [FREE Full text] [CrossRef] [Medline]
    22. Kim K, Shin S, Kim S, Lee E. The relation between eHealth literacy and health-related behaviors: systematic review and meta-analysis. J Med Internet Res. Jan 30, 2023;25:e40778. [FREE Full text] [CrossRef] [Medline]
    23. Norman CD, Skinner HA. eHEALS: the eHealth Literacy Scale. J Med Internet Res. Nov 14, 2006;8(4):e27. [FREE Full text] [CrossRef] [Medline]
    24. Rathnayake S, Liyanage IP. Cross-cultural adaptation and psychometric properties of the Sinhala version of electronic health literacy scale: a cross-sectional validation study. PLoS One. 2022;17(4):e0266515. [FREE Full text] [CrossRef] [Medline]
    25. Shiferaw KB. Validation of the Ethiopian version of eHealth Literacy Scale (ET-eHEALS) in a population with chronic disease. Risk Manag Healthc Policy. 2020;13:465-471. [FREE Full text] [CrossRef] [Medline]
    26. Wangdahl JM, Dahlberg K, Jaensson M, Nilsson U. Psychometric validation of Swedish and Arabic versions of two health literacy questionnaires, eHEALS and HLS-EU-Q16, for use in a Swedish context: a study protocol. BMJ Open. Sep 17, 2019;9(9):e029668. [CrossRef]
    27. Del Giudice P, Bravo G, Poletto M, De Odorico A, Conte A, Brunelli L, et al. Correlation between eHealth literacy and health literacy using the eHealth Literacy Scale and real-life experiences in the health sector as a proxy measure of functional health literacy: cross-sectional web-based survey. J Med Internet Res. Oct 31, 2018;20(10):e281. [FREE Full text] [CrossRef] [Medline]
    28. Paramio Pérez G, Almagro BJ, Hernando Gómez Á, Aguaded Gómez JI. [Validation of the eHealth Literacy Scale (eHEALS) in Spanish university students]. Rev Esp Salud Publica. 2015;89(3):329-338. [FREE Full text] [CrossRef] [Medline]
    29. van der Vaart R, van Deursen AJ, Drossaert CH, Taal E, van Dijk JA, van de Laar MA. Does the eHealth Literacy Scale (eHEALS) measure what it intends to measure? Validation of a Dutch version of the eHEALS in two adult populations. J Med Internet Res. Nov 09, 2011;13(4):e86. [FREE Full text] [CrossRef] [Medline]
    30. Chang A, Schulz PJ. The measurements and an elaborated understanding of Chinese eHealth literacy (C-eHEALS) in chronic patients in China. Int J Environ Res Public Health. Jul 23, 2018;15(7):1553. [FREE Full text] [CrossRef] [Medline]
    31. Gazibara T, Cakic J, Cakic M, Pekmezovic T, Grgurevic A. eHealth and adolescents in Serbia: psychometric properties of eHeals questionnaire and contributing factors to better online health literacy. Health Promot Int. Aug 01, 2019;34(4):770-778. [CrossRef] [Medline]
    32. Chung S, Park BK, Nahm E. The Korean eHealth Literacy Scale (K-eHEALS): reliability and validity testing in younger adults recruited online. J Med Internet Res. Apr 20, 2018;20(4):e138. [FREE Full text] [CrossRef] [Medline]
    33. Wijaya MC, Kloping YP. Validity and reliability testing of the Indonesian version of the eHealth Literacy Scale during the COVID-19 pandemic. Health Inform J. 2021;27(1):1460458220975466. [FREE Full text] [CrossRef] [Medline]
    34. Cui G, Li S, Yin Y, Chen L, Li J, Liang F, et al. The relationship among social capital, eHealth literacy and health behaviours in Chinese elderly people: a cross-sectional study. BMC Public Health. Jan 06, 2021;21(1):45. [FREE Full text] [CrossRef] [Medline]
    35. Xie L, Zhang S, Xin M, Zhu M, Lu W, Mo PK. Electronic health literacy and health-related outcomes among older adults: a systematic review. Prev Med. Apr 2022;157:106997. [CrossRef] [Medline]
    36. Tsukahara S, Yamaguchi S, Igarashi F, Uruma R, Ikuina N, Iwakura K, et al. Association of eHealth literacy with lifestyle behaviors in university students: questionnaire-based cross-sectional study. J Med Internet Res. Jun 24, 2020;22(6):e18155. [FREE Full text] [CrossRef] [Medline]
    37. Brørs G, Wentzel-Larsen T, Dalen H, Hansen TB, Norman CD, Wahl A, et al. CONCARD Investigators. Psychometric properties of the Norwegian version of the electronic Health Literacy Scale (eHEALS) among patients after percutaneous coronary intervention: cross-sectional validation study. J Med Internet Res. Jul 28, 2020;22(7):e17312. [FREE Full text] [CrossRef] [Medline]
    38. Guo Z, Zhao SZ, Guo N, Wu Y, Weng X, Wong JY, et al. Socioeconomic disparities in eHealth literacy and preventive behaviors during the COVID-19 pandemic in Hong Kong: cross-sectional study. J Med Internet Res. Apr 14, 2021;23(4):e24577. [FREE Full text] [CrossRef] [Medline]
    39. Li S, Cui G, Kaminga AC, Cheng S, Xu H. Associations between health literacy, eHealth literacy, and COVID-19–related health behaviors among Chinese college students: cross-sectional online study. J Med Internet Res. May 06, 2021;23(5):e25600. [FREE Full text] [CrossRef] [Medline]
    40. Moghadas S, Vilches T, Zhang K, Wells C, Shoukat A, Singer B, et al. The impact of vaccination on coronavirus disease 2019 (COVID-19) outbreaks in the United States. Clin Infect Dis. Dec 16, 2021;73(12):2257-2264. [FREE Full text] [CrossRef] [Medline]
    41. Nath R, Imtiaz A, Nath SD, Hasan E. Role of vaccine hesitancy, eHealth literacy, and vaccine literacy in young adults’ COVID-19 vaccine uptake intention in a lower-middle-income country. Vaccines (Basel). Nov 29, 2021;9(12):1405. [FREE Full text] [CrossRef] [Medline]
    42. Rosen AO, Holmes AL, Balluerka N, Hidalgo MD, Gorostiaga A, Gómez-Benito J, et al. Is social media a new type of social support? Social media use in Spain during the COVID-19 pandemic: a mixed methods study. Int J Environ Res Public Health. Mar 26, 2022;19(7):3952. [FREE Full text] [CrossRef] [Medline]
    43. Van Aelst P, Toth F, Castro L, Štětka V, Vreese CD, Aalberg T, et al. Does a a crisis change news habits? A comparative study of the effects of COVID-19 on news media use in 17 European countries. Digital Journalism. Jul 21, 2021;9(9):1208-1238. [CrossRef]
    44. Lee J, Lee E, Chae D. eHealth literacy instruments: systematic review of measurement properties. J Med Internet Res. Nov 15, 2021;23(11):e30644. [FREE Full text] [CrossRef] [Medline]
    45. Paige SR, Stellefson M, Krieger JL, Miller MD, Cheong J, Anderson-Lewis C. Transactional eHealth literacy: developing and testing a multi-dimensional instrument. J Health Commun. 2019;24(10):737-748. [FREE Full text] [CrossRef] [Medline]
    46. Karnoe A, Kayser L. How is eHealth literacy measured and what do the measurements tell us? A systematic review. Knowl Manag E-Learn Int J. Dec 15, 2015;7(4):576-600. [CrossRef]
    47. Neter E, Brainin E. Association between health literacy, eHealth literacy, and health outcomes among patients with long-term conditions: a systematic review. Eur Psychol. Jan 2019;24(1):68-81. [CrossRef]
    48. Tariq A, Khan SR, Basharat A. Internet use, eHealth literacy, and dietary supplement use among young adults in Pakistan: cross-sectional study. J Med Internet Res. Jun 10, 2020;22(6):e17014. [FREE Full text] [CrossRef] [Medline]
    49. Marsall M, Engelmann G, Skoda E, Teufel M, Bäuerle A. Measuring electronic health literacy: development, validation, and test of measurement invariance of a revised German version of the eHealth Literacy Scale. J Med Internet Res. Feb 02, 2022;24(2):e28252. [FREE Full text] [CrossRef] [Medline]
    50. Rentrop V, Damerau M, Schweda A, Steinbach J, Schüren LC, Niedergethmann M, et al. Predicting acceptance of e–mental health interventions in patients with obesity by using an extended unified theory of acceptance model: cross-sectional study. JMIR Form Res. Mar 17, 2022;6(3):e31229. [FREE Full text] [CrossRef] [Medline]
    51. Bäuerle A, Frewer A, Rentrop V, Schüren LC, Niedergethmann M, Lortz J, et al. Determinants of acceptance of weight management applications in overweight and obese individuals: using an extended unified theory of acceptance and use of technology model. Nutrients. May 08, 2022;14(9):1968. [FREE Full text] [CrossRef] [Medline]
    52. Hasenbring M. Zur Adaptivität von Kontrollüberzeugungen — Empirische Befunde bei Patienten mit Krebserkrankungen, lumbalem Bandscheibenvorfall und chronischen Schmerzsyndromen. In: Schüffel W, editor. Sich Gesund Fühlen Im Jahre 2000. Berlin, Heidelberg. Springer; 1988;222-230.
    53. Engelmann G, Marsall M, Skoda E, Knoll-Pientka N, Bäuerle L, Stroebele-Benschop N, et al. Development and validation of the General Dietary Behavior Inventory (GDBI) in scope of international nutrition guidelines. Nutrients. Apr 17, 2021;13(4):1328. [FREE Full text] [CrossRef] [Medline]
    54. R: a language and environment for statistical computing. R Core Team. URL: https://www.r-project.org/ [accessed 2024-04-11]
    55. RSTUDIO IDE: the most trusted IDE for open source data science. Posit. URL: https://posit.co/products/open-source/rstudio/ [accessed 2024-04-11]
    56. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model. Jan 1999;6(1):1-55. [CrossRef]
    57. Sellbom M, Tellegen A. Factor analysis in psychological assessment research: common pitfalls and recommendations. Psychol Assess. Dec 2019;31(12):1428-1441. [CrossRef] [Medline]
    58. Van De Schoot R, Schmidt P, De Beuckelaer A, Lek K, Zondervan-Zwijnenburg M. Editorial: measurement invariance. Front Psychol. Jul 28, 2015;6:1064. [FREE Full text] [CrossRef] [Medline]
    59. Hirschfeld G, Von BR. Improving multiple-group confirmatory factor analysis in R–a tutorial in measurement invariance with continuous and ordinal indicators. Pract Assess Res Eval. 2014;19(1):7. [CrossRef]
    60. Dimitrov DM. Testing for factorial invariance in the context of construct validation. Meas Eval Counsel Dev. Mar 13, 2017;43(2):121-149. [CrossRef]
    61. Svetina D, Rutkowski L, Rutkowski D. Multiple-group invariance with categorical outcomes using updated guidelines: an illustration using M plus and the lavaan/semTools packages. Struct Equ Model. Apr 29, 2019;27(1):111-130. [CrossRef]
    62. Putnick DL, Bornstein MH. Measurement invariance conventions and reporting: the state of the art and future directions for psychological research. Dev Rev. Sep 2016;41:71-90. [FREE Full text] [CrossRef] [Medline]
    63. Wilski M, Brola W, Tomczak M. Health locus of control and mental health in patients with multiple sclerosis: mediating effect of coping strategies. Res Nurs Health. Aug 07, 2019;42(4):296-305. [CrossRef] [Medline]
    64. Milaniak I, Dębska G, Król B, Wierzbicki K, Przybyłowski P. Health locus of control and health behaviors in organ transplant recipients: a multicenter study. Transplant Proc. May 2022;54(4):995-1001. [CrossRef] [Medline]
    65. Mercer DA, Ditto B, Lavoie KL, Campbell T, Arsenault A, Bacon SL. Health locus of control is associated with physical activity and other health behaviors in cardiac patients. J Cardiopulm Rehabil Prev. Nov 2018;38(6):394-399. [CrossRef] [Medline]
    66. Ganjoo M, Farhadi A, Baghbani R, Daneshi S, Nemati R. Association between health locus of control and perceived stress in college student during the COVID-19 outbreak: a cross-sectional study in Iran. BMC Psychiatry. Oct 26, 2021;21(1):529. [FREE Full text] [CrossRef] [Medline]
    67. Náfrádi L, Nakamoto K, Schulz PJ. Is patient empowerment the key to promote adherence? A systematic review of the relationship between self-efficacy, health locus of control and medication adherence. PLoS One. Oct 2017;12(10):e0186458. [FREE Full text] [CrossRef] [Medline]
    68. Milfont TL, Fischer R. Testing measurement invariance across groups: applications in cross-cultural research. Int J Psychol Res. Jun 30, 2010;3(1):111-130. [CrossRef]
    69. Davidov E, Meuleman B, Cieciuch J, Schmidt P, Billiet J. Measurement equivalence in cross-national research. Annu Rev Sociol. Jul 30, 2014;40(1):55-75. [CrossRef]
    70. Vandenberg RJ, Lance CE. A review and synthesis of the measurement invariance literature: suggestions, practices, and recommendations for organizational research. Org Res Methods. Jun 29, 2016;3(1):4-70. [CrossRef]
    71. Dobrow MJ, Miller FA, Frank C, Brown AD. Understanding relevance of health research: considerations in the context of research impact assessment. Health Res Policy Syst. Apr 17, 2017;15(1):31. [FREE Full text] [CrossRef] [Medline]
    72. Weise A, Büchter R, Pieper D, Mathes T. Assessing context suitability (generalizability, external validity, applicability or transferability) of findings in evidence syntheses in healthcare—an integrative review of methodological guidance. Res Synth Methods. Nov 29, 2020;11(6):760-779. [CrossRef] [Medline]
    73. Khanijahani A, Iezadi S, Gholipour K, Azami-Aghdash S, Naghibi D. A systematic review of racial/ethnic and socioeconomic disparities in COVID-19. Int J Equity Health. Nov 24, 2021;20(1):248. [FREE Full text] [CrossRef] [Medline]
    74. De Negri F, Galiezz R, Miranda P, Koeller P, Zucoloto G, Costa J, et al. Socioeconomic factors and the probability of death by COVID-19 in Brazil. J Public Health (Oxf). Sep 22, 2021;43(3):493-498. [FREE Full text] [CrossRef] [Medline]
    75. Hawkins RB, Charles EJ, Mehaffey JH. Socio-economic status and COVID-19-related cases and fatalities. Public Health. Dec 2020;189:129-134. [FREE Full text] [CrossRef] [Medline]
    76. Jin J, Bai P, He W, Wu F, Liu X, Han D, et al. Gender differences in patients with COVID-19: focus on severity and mortality. Front Public Health. 2020;8:152. [FREE Full text] [CrossRef] [Medline]
    77. Sykes DL, Holdsworth L, Jawad N, Gunasekera P, Morice AH, Crooks MG. Post-COVID-19 symptom burden: what is long-COVID and how should we manage it? Lung. Apr 2021;199(2):113-119. [FREE Full text] [CrossRef] [Medline]
    78. Ma Z, Wu M. The psychometric properties of the Chinese eHealth Literacy Scale (C-eHEALS) in a Chinese rural population: cross-sectional validation study. J Med Internet Res. Oct 22, 2019;21(10):e15720. [FREE Full text] [CrossRef] [Medline]
    79. Dale JG, Lüthi A, Fundingsland Skaraas B, Rundereim T, Dale B. Testing measurement properties of the Norwegian version of electronic Health Literacy Scale (eHEALS) in a group of day surgery patients. J Multidiscip Healthc. 2020;13:241-247. [FREE Full text] [CrossRef] [Medline]
    80. Niu Z, Willoughby J, Zhou R. Associations of health literacy, social media use, and self-efficacy with health information–seeking intentions among social media users in China: cross-sectional survey. J Med Internet Res. Feb 25, 2021;23(2):e19134. [FREE Full text] [CrossRef] [Medline]
    81. Lim MSC, Molenaar A, Brennan L, Reid M, McCaffrey T. Young adults’ use of different social media platforms for health information: insights from web-based conversations. J Med Internet Res. Jan 18, 2022;24(1):e23656. [FREE Full text] [CrossRef] [Medline]
    82. Yates S, Kirby J, Lockley E. Digital media use: differences and inequalities in relation to class and age. Sociol Res Online. Nov 30, 2015;20(4):71-91. [CrossRef]

    ‎
    CFA: confirmatory factor analysis
    CFI: comparative fit index
    eHL: eHealth Literacy
    eHEALS: eHealth Literacy Scale
    GR-eHEALS: German Revised eHealth Literacy Scale
    IA: information appraisal
    IS: information seeking
    RMSEA: root mean square error of approximation
    SRMR: standardized root mean square residual
    TLI: Tucker–Lewis index
    ULSM: unweighted least squares estimator

    Edited by A Mavragani; submitted 28.08.23; peer-reviewed by J Graf, S Bacon, K Lavoie; comments to author 28.12.23; revised version received 12.01.24; accepted 05.02.24; published 25.04.24.

    Copyright

    ©Matthias Marsall, Hannah Dinse, Julia Schröder, Eva-Maria Skoda, Martin Teufel, Alexander Bäuerle. Originally published in JMIR Formative Research (https://formative.jmir.org), 25.04.2024.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.