<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Publishing DTD v2.0 20040830//EN" "http://dtd.nlm.nih.gov/publishing/2.0/journalpublishing.dtd">
<?covid-19-tdm?>
<article xmlns:xlink="http://www.w3.org/1999/xlink" article-type="research-article" dtd-version="2.0">
  <front>
    <journal-meta>
      <journal-id journal-id-type="publisher-id">JFR</journal-id>
      <journal-id journal-id-type="nlm-ta">JMIR Form Res</journal-id>
      <journal-title>JMIR Formative Research</journal-title>
      <issn pub-type="epub">2561-326X</issn>
      <publisher>
        <publisher-name>JMIR Publications</publisher-name>
        <publisher-loc>Toronto, Canada</publisher-loc>
      </publisher>
    </journal-meta>
    <article-meta>
      <article-id pub-id-type="publisher-id">v6i12e40825</article-id>
      <article-id pub-id-type="pmid">36446048</article-id>
      <article-id pub-id-type="doi">10.2196/40825</article-id>
      <article-categories>
        <subj-group subj-group-type="heading">
          <subject>Original Paper</subject>
        </subj-group>
        <subj-group subj-group-type="article-type">
          <subject>Original Paper</subject>
        </subj-group>
      </article-categories>
      <title-group>
        <article-title>State-Level COVID-19 Symptom Searches and Case Data: Quantitative Analysis of Political Affiliation as a Predictor for Lag Time Using Google Trends and Centers for Disease Control and Prevention Data</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="editor">
          <name>
            <surname>Mavragani</surname>
            <given-names>Amaryllis</given-names>
          </name>
        </contrib>
      </contrib-group>
      <contrib-group>
        <contrib contrib-type="reviewer">
          <name>
            <surname>Kumar</surname>
            <given-names>Jishnu Krishna</given-names>
          </name>
        </contrib>
        <contrib contrib-type="reviewer">
          <name>
            <surname>Hu</surname>
            <given-names>Yaolin</given-names>
          </name>
        </contrib>
      </contrib-group>
      <contrib-group>
        <contrib id="contrib1" contrib-type="author" corresp="yes">
          <name name-style="western">
            <surname>Turvy</surname>
            <given-names>Alex</given-names>
          </name>
          <degrees>BA, MS</degrees>
          <xref rid="aff1" ref-type="aff">1</xref>
          <address>
            <institution>City, Culture, and Community</institution>
            <institution>Department of Sociology</institution>
            <institution>Tulane University</institution>
            <addr-line>6823 St Charles Ave</addr-line>
            <addr-line>New Orleans, LA, 70118</addr-line>
            <country>United States</country>
            <phone>1 504 865 5231</phone>
            <email>aturvy@tulane.edu</email>
          </address>
          <ext-link ext-link-type="orcid">https://orcid.org/0000-0002-5610-7515</ext-link>
        </contrib>
      </contrib-group>
      <aff id="aff1">
        <label>1</label>
        <institution>City, Culture, and Community</institution>
        <institution>Department of Sociology</institution>
        <institution>Tulane University</institution>
        <addr-line>New Orleans, LA</addr-line>
        <country>United States</country>
      </aff>
      <author-notes>
        <corresp>Corresponding Author: Alex Turvy <email>aturvy@tulane.edu</email></corresp>
      </author-notes>
      <pub-date pub-type="collection">
        <month>12</month>
        <year>2022</year>
      </pub-date>
      <pub-date pub-type="epub">
        <day>23</day>
        <month>12</month>
        <year>2022</year>
      </pub-date>
      <volume>6</volume>
      <issue>12</issue>
      <elocation-id>e40825</elocation-id>
      <history>
        <date date-type="received">
          <day>6</day>
          <month>7</month>
          <year>2022</year>
        </date>
        <date date-type="rev-request">
          <day>6</day>
          <month>10</month>
          <year>2022</year>
        </date>
        <date date-type="rev-recd">
          <day>24</day>
          <month>10</month>
          <year>2022</year>
        </date>
        <date date-type="accepted">
          <day>8</day>
          <month>11</month>
          <year>2022</year>
        </date>
      </history>
      <copyright-statement>©Alex Turvy. Originally published in JMIR Formative Research (https://formative.jmir.org), 23.12.2022.</copyright-statement>
      <copyright-year>2022</copyright-year>
      <license license-type="open-access" xlink:href="https://creativecommons.org/licenses/by/4.0/">
        <p>This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.</p>
      </license>
      <self-uri xlink:href="https://formative.jmir.org/2022/12/e40825" xlink:type="simple"/>
      <abstract>
        <sec sec-type="background">
          <title>Background</title>
          <p>Across each state, the emergence of the COVID-19 pandemic in the United States was marked by policies and rhetoric that often corresponded to the political party in power. These diverging responses have sparked broad ongoing discussion about how the political leadership of a state may affect not only the COVID-19 case numbers in a given state but also the subjective individual experience of the pandemic.</p>
        </sec>
        <sec sec-type="objective">
          <title>Objective</title>
          <p>This study leverages state-level data from Google Search Trends and Centers for Disease Control and Prevention (CDC) daily case data to investigate the temporal relationship between increases in relative search volume for COVID-19 symptoms and corresponding increases in case data. I aimed to identify whether there are state-level differences in patterns of lag time across each of the 4 spikes in the data (RQ1) and whether the political climate in a given state is associated with these differences (RQ2).</p>
        </sec>
        <sec sec-type="methods">
          <title>Methods</title>
          <p>Using publicly available data from Google Trends and the CDC, linear mixed modeling was utilized to account for random state-level intercepts. Lag time was operationalized as number of days between a peak (a sustained increase before a sustained decline) in symptom search data and a corresponding spike in case data and was calculated manually for each of the 4 spikes in individual states. Google offers a data set that tracks the relative search incidence of more than 400 potential COVID-19 symptoms, which is normalized on a 0-100 scale. I used the CDC’s definition of the 11 most common COVID-19 symptoms and created a single construct variable that operationalizes symptom searches. To measure political climate, I considered the proportion of 2020 Trump popular votes in a state as well as a dummy variable for the political party that controls the governorship and a continuous variable measuring proportional party control of federal Congressional representatives.</p>
        </sec>
        <sec sec-type="results">
          <title>Results</title>
          <p>The strongest overall fit was for a linear mixed model that included proportion of 2020 Trump votes as the predictive variable of interest and included controls for mean daily cases and deaths as well as population. Additional political climate variables were discarded for lack of model fit. Findings indicated evidence that there are statistically significant differences in lag time by state but that no individual variable measuring political climate was a statistically significant predictor of these differences.</p>
        </sec>
        <sec sec-type="conclusions">
          <title>Conclusions</title>
          <p>Given that there will likely be future pandemics within this political climate, it is important to understand how political leadership affects perceptions of and corresponding responses to public health crises. Although this study did not fully model this relationship, I believe that future research can build on the state-level differences that I identified by approaching the analysis with a different theoretical model, method for calculating lag time, or level of geographic modeling.</p>
        </sec>
      </abstract>
      <kwd-group>
        <kwd>COVID-19</kwd>
        <kwd>search trends</kwd>
        <kwd>prediction</kwd>
        <kwd>case</kwd>
        <kwd>political</kwd>
        <kwd>symptom</kwd>
        <kwd>pandemic</kwd>
        <kwd>data</kwd>
        <kwd>google</kwd>
        <kwd>disease</kwd>
        <kwd>prevention</kwd>
        <kwd>model</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec sec-type="introduction">
      <title>Introduction</title>
      <sec>
        <title>Background</title>
        <p>Incidence of COVID-19 cases in the United States has widely varied across states and across time, as have state-level policies and some of the rhetoric heard in response. There has been ongoing investigation about how mitigation measure mandates such as mask wearing [<xref ref-type="bibr" rid="ref1">1</xref>,<xref ref-type="bibr" rid="ref2">2</xref>], social distancing [<xref ref-type="bibr" rid="ref3">3</xref>], and vaccines [<xref ref-type="bibr" rid="ref4">4</xref>] affect uptake of these measures as well as how they are associated with actual case numbers. One existing study focused on the political dimensions of mandates and cases [<xref ref-type="bibr" rid="ref5">5</xref>] by trying to understand the broader social forces that are associated with the response to the pandemic and to mandates, but it is challenging to observe and understand the informal behaviors and tacit unreported beliefs that drive state-level differences in case numbers and response to the pandemic. There has been significant ongoing debate regarding theories for why individual behavioral responses to the pandemic have diverged so significantly over time, and many of these theories have involved analysis of political and administrative messaging. This research enters this conversation by exploring the intersection of political affiliation and perceptions of the pandemic. Increases in search traffic seem to reflect individual concerns about the pandemic as well as information seeking for individuals who are experiencing and observing symptoms. This study aimed to specifically analyze the effects that political affiliation has on this dynamic; there has been plenty of concern about the danger of public health becoming more politicized, and this research intended to add granularity to our understanding.</p>
        <p>Both anecdotally and in popular media, there is discussion about how trends in COVID-19–related Google searches might be associated with ongoing COVID-19 case numbers as reported to the Centers for Disease Control and Prevention (CDC) [<xref ref-type="bibr" rid="ref6">6</xref>]. Using Google Trends search data about COVID-19–related symptoms along with CDC data concerning state-level case numbers, I investigated 2 questions:</p>
        <list list-type="order">
          <list-item>
            <p>RQ1: Are there state-level differences in the lag time between spikes in searches for COVID-19 symptoms and spikes in reported COVID-19 cases?</p>
          </list-item>
          <list-item>
            <p>RQ2: If these state-level differences do exist, do covariates related to political leadership contribute to state-level variance in lag time?</p>
          </list-item>
        </list>
        <p>I hypothesized that state-level political outcomes, as a marker of the dominant or collective political identification of a state’s voters, offer a route to investigating how social behavior via self-identified group affiliation explains differences in the temporal relationship between spikes in COVID-19–related searches and later spikes in total confirmed COVID-19 cases. I expected to find that political variables marking Republican identification are associated with a decrease in lag time, given what we know about the differences in compliance with mitigation measures and vaccine uptake and what this suggests about broader COVID-19 risk beliefs and self-surveillance of symptoms. This lag time relationship may give us insight into how people think about COVID-19: Are they proactive about watching for and managing symptoms, or do they only start noticing symptoms once cases begin to increase and spike?</p>
      </sec>
      <sec>
        <title>Literature Review</title>
        <sec>
          <title>Theoretical Approach</title>
          <p>Social cognitive theory (SCT) [<xref ref-type="bibr" rid="ref7">7</xref>] frames learning and behavior socially, noting that there is a reciprocal relationship between an individual, their environment, and their behavior—while emphasizing the specifically social nature of this triad. That is, people tend to learn through observing the actions of those in their environment along with their own experiences. The essential lens to understand SCT in this context is how it focuses specifically on personal but environmentally contextualized agency. The social identity approach (SIA) by Abrams and Hogg [<xref ref-type="bibr" rid="ref8">8</xref>] is a complementary perspective, adding that not only is learning and behavior social but also that knowledge of being in social groups affects how people attach emotion and value to certain behaviors and circumstances. They also emphasize the influence of one’s own in-groups and out-groups as part of this individual/group relationship.</p>
          <p>In terms of compliance with health behaviors, the research has settled around 3 major factors that tend to drive an individual’s level of compliance: perceived risk to oneself, belief in behavior effectiveness, and observed risk to others. During the H1N1 influenza pandemic, a review of 26 studies found consistently strong associations between an individual’s perceived susceptibility to the virus and increased compliance with recommended behaviors; this effect was strengthened when perceived severity of infection increased and was consistent across many countries and cultures [<xref ref-type="bibr" rid="ref9">9</xref>]. Perceived risk to self is affected by factors such as perceived personal vulnerability [<xref ref-type="bibr" rid="ref10">10</xref>], level of cultural individualism [<xref ref-type="bibr" rid="ref11">11</xref>], fear [<xref ref-type="bibr" rid="ref12">12</xref>,<xref ref-type="bibr" rid="ref13">13</xref>], and anger [<xref ref-type="bibr" rid="ref14">14</xref>].</p>
          <p>Belief in the effectiveness of health behaviors is driven by a number of affective and epistemological factors as well; laypeople tend to create their own justifications for health behaviors by pulling from not only both establishment and nonestablishment sources [<xref ref-type="bibr" rid="ref15">15</xref>] but also their preferred mass media sources [<xref ref-type="bibr" rid="ref16">16</xref>]. Although this belief is reduced in all groups when they perceive recommendations to predominantly be moralistic [<xref ref-type="bibr" rid="ref17">17</xref>], general trust in government is strongly associated with affecting perceived risk in complicated and sometimes counterintuitive ways [<xref ref-type="bibr" rid="ref10">10</xref>,<xref ref-type="bibr" rid="ref17">17</xref>,<xref ref-type="bibr" rid="ref18">18</xref>]. Finally, observed risk to specific others and a more generalized community tends to be positively affected by a general sense of conscientiousness [<xref ref-type="bibr" rid="ref12">12</xref>]. This is true when individuals feel an ethical responsibility to their community [<xref ref-type="bibr" rid="ref19">19</xref>] but is also true on a more individual level when individuals experience the vulnerability of their close ties [<xref ref-type="bibr" rid="ref20">20</xref>] and so avoid the perception that health concerns are “overhyped” [<xref ref-type="bibr" rid="ref21">21</xref>].</p>
          <p>In this paper, I do not explore the details of how particular political ideologies specifically affect compliance behaviors. Instead, the context described in the previous paragraphs serves to highlight the power that political identities and communication have on individual compliance. From both Democratic and Republican politicians, we frequently hear justifications for COVID-19 behaviors that speak directly to these 3 factors. These justifications are often oriented around personal ethical responsibility, the meaning of a sense of community, and the epistemological justifications for agency recommendations; each of these threads is an important part of building the set of beliefs that ultimately drive behaviors. More recent research extends these claims further, demonstrating that political affiliation is associated with particular pandemic responses that cannot be explained solely via these 3 factors [<xref ref-type="bibr" rid="ref5">5</xref>], highlighting how at least part of an individual’s response is related to affinity-related dogma or that operational constraints and diverging priorities tend to be privileged over more objective risk assessments [<xref ref-type="bibr" rid="ref22">22</xref>].</p>
        </sec>
        <sec>
          <title>Using Search Data in Public Health Research</title>
          <p>Public health professionals and researchers have broadly been discussing the value of search data for both detection and surveillance for quite some time, initially highlighting its value in a landmark paper that advocated for its use but cautioned that it should be predominantly used in areas with widespread internet access [<xref ref-type="bibr" rid="ref23">23</xref>]. Given that American internet usage is now frequent and widespread across most settings, researchers have been able to turn their attention specifically to its use in early detection of emergent diseases [<xref ref-type="bibr" rid="ref24">24</xref>].</p>
          <p>There is also evidence that search volume is effective for ongoing monitoring and surveillance, both for active and predictive surveillance [<xref ref-type="bibr" rid="ref25">25</xref>], and for passive or retrospective surveillance that aims to understand how factors such as the media affect the relationship between search interest and cases [<xref ref-type="bibr" rid="ref26">26</xref>].</p>
          <p>Search data have been used as a lens specifically for understanding COVID-19 data, but this research has had different areas of focus: searches as predictive of local metropolitan-level data [<xref ref-type="bibr" rid="ref27">27</xref>,<xref ref-type="bibr" rid="ref28">28</xref>], impacts on mental health [<xref ref-type="bibr" rid="ref29">29</xref>], and more rare symptoms (anosmia and ageusia) as ineffective predictors of case incidence [<xref ref-type="bibr" rid="ref30">30</xref>]. Eysenbach [<xref ref-type="bibr" rid="ref31">31</xref>] took a somewhat similar approach to my own but concerning flu symptoms and incidence instead of COVID-19, finding strong correlations between clicks on sponsored flu-related links and flu diagnoses 7 days later.</p>
          <p>There are ongoing challenges to using search data as well as other novel data streams (NDS) such as social media posts; although there is some evidence that they can help to retroactively explore associations and wield predictive power, there are still unresolved issues of how to assess reliability and validity of these data [<xref ref-type="bibr" rid="ref32">32</xref>]. Some challenges such as lack of transparency and reproducibility [<xref ref-type="bibr" rid="ref33">33</xref>] can be resolved by establishing accepted best practices such as sharing specific search strings and Boolean operators, but others are more related to complex sociological and psychological phenomena such as a panic-induced search increase that will likely prove to be much more difficult to solve [<xref ref-type="bibr" rid="ref34">34</xref>]. Given the established value and unresolved challenges of using NDS such as search traffic, the use of Google Trends data should be seen as a supplemental tool for public health researchers along with more traditional and localized practices instead of as a substitute [<xref ref-type="bibr" rid="ref35">35</xref>].</p>
          <p>This paper addressed these methodological limitations by using search data indirectly; although this means that some of these concerns become endogenous to the modeling, it is beneficial insofar as this captures these complex dynamics within the lag time variable. Instead of relying on search trends data as an accurate predictive or surveillance tool, I used it to highlight areas of difference across regions and explore the reasons for those differences.</p>
        </sec>
      </sec>
    </sec>
    <sec sec-type="methods">
      <title>Methods</title>
      <p>Although there is discussion in popular media about using trends data in a predictive way and some push toward this in technical methods literature [<xref ref-type="bibr" rid="ref36">36</xref>], there is not yet evidence that search data are defensible for use in a predictive way about future health trends [<xref ref-type="bibr" rid="ref30">30</xref>] given that access to real-time or otherwise timely raw data is not possible. Instead, trends data are most useful for monitoring and evaluating relationships between events in the past, especially as one predictive element within a larger model [<xref ref-type="bibr" rid="ref37">37</xref>,<xref ref-type="bibr" rid="ref38">38</xref>].</p>
      <p>For the purposes of this study, I used the publicly available COVID-19 Search Trends data set, which tracks the incidence of more than 400 symptoms associated at various levels with COVID-19. Typically, Google does not allow for large-scale downloads of granular daily search trends data except through use of their API. However, the company made this COVID-19–specific search data available specifically for researchers and journalists; the data include both daily data as well as state-level geographic data. The data set allows for what Google calls “metro areas,” but these do not include shapefiles that could be used to match the search data with CDC data via geographic information system (GIS) software. Like all Google Search Trends data, it is normalized on a 0-100 scale, contextualized within the geography and time range in question, and based on a particular search string’s incidence in proportion to all searches in that same geography and time frame. This study used daily trends data from each state for the time period from March 11, 2020, through April 4, 2022, and was retrieved on April 15, 2022, via Google’s internally hosted GitHub.</p>
      <p>The beginning of the study period is the day on which COVID-19 was declared a global pandemic by the World Health Organization, and the end of the period is the final day of trends data from Google’s data set; so, the total number of days in the study period is 762. Although not every region experienced their first case by the beginning of the trends data time period, I was specifically interested in the lag time between increases in searches and increases in cases. Given that there was already widespread discussion of COVID-19 in popular media and so this is broadly reflected in the search data, search volume was already increasing across all regions by March 11, 2020, and this allowed me to examine state-level differences in when cases began to increase.</p>
      <p>The CDC makes daily data detailing new cases, new hospitalizations, and new deaths associated with COVID-19 available to the public. These data are available to the county level, but this study exclusively used state-level data in order to look at these variables alongside the state-level search trends data. These data are raw numbers, but population is controlled for in regression modeling to account for this. There are likely data gaps related to submission logistics and other issues; additionally, the data had some level of daily noise within the data as states catch up with missed submissions and correct submissions from previous days, but this has been accounted for in the process of examining the data closely in the calculations for lag time as the dependent variable.</p>
      <p>The analytic strategy used a linear mixed model with fixed effects for all included predictors and controls and random intercepts for each state to investigate the state-level differences named in the research questions. Additionally, I included a random effect for the political predictor nested within state clusters, recognizing how SCT indicates that behavior is affected in an ongoing reciprocal way by environment—here, the state and its political climate are considered as that environment.</p>
      <p>I called the key outcome variable “lag time,” and it measures the amount of time in days between a spike in COVID-19 symptom searches and a (typically) later corresponding spike in reported COVID-19 cases. This variable was calculated manually using the raw data and plots as a guide. Each state had 4 identifiable case peaks of varying magnitudes. After marking these and accounting for any noise or reporting gaps in the data, I turned to the search data to identify whether there was a corresponding spike in symptom searches that preceded the case spike. In nearly all cases, there was an associated spike, and this was measured in number of days.</p>
      <p>Political variables under consideration included the proportion of Trump popular votes within a state in the 2020 election, a dummy variable indicating whether a Republican holds the Governor office, and a variable measuring the proportion of a state’s federal representatives in the House of Representatives and Senate that is Republican. The latter 2 variables did not lead to a strong model fit in any case and were discarded, so the Trump proportion variable remained as the main predictor in this model. Controls for mean daily cases, mean daily deaths, and state population were also included in the model. All predictors and controls were normalized using <italic>z</italic> scores.</p>
      <p>Within the search data, I identified the 11 most common symptoms of COVID-19 as reported by the CDC and created a construct to represent the collective incidence of these search terms. These symptoms included headache, nasal congestion, rhinorrhea, fever, sore throat, nausea, anosmia, ageusia, fatigue, and diarrhea. This symptom construct has a Cronbach alpha score of .812. An alpha value greater than .8 generally indicates a strong level of construct reliability. Reliability analysis showed that the alpha value would not be improved by removing any variable from the construct.</p>
      <p>Descriptive analysis for daily new case and death data by state, initial bivariate linear regression modeling, calculations for construct reliability, lag time calculations, mixed modeling, and model comparisons were all completed in R. The primary packages used were lubridate for parsing date variables, plotly for examining ggplot2 results in more detail, lme4 and lmerTest for fitting linear mixed models, sjPlot for plotting data to test model assumptions, stargazer for table and figure creation, and all of the packages within the “tidyverse” (primarily dplyr and ggplot2) for cleaning, organizing, and preparing data for analysis and presentation.</p>
    </sec>
    <sec sec-type="results">
      <title>Results</title>
      <p>Descriptive statistics for all variables included in the final model are included in <xref ref-type="table" rid="table1">Table 1</xref>. These data are the raw numbers, but predictors and controls were standardized for analysis to control for the vastly different scales for many of the variables. Given the extremely large volume of daily case and search data, this is not included in this table.</p>
      <p>Assumptions for linear mixed model regression were checked, confirming that there was a linear relationship between the predictor and outcome variable and that the residuals were independent, uncorrelated, and normally distributed. The residual plot for homoscedasticity of residuals is in <xref rid="figure1" ref-type="fig">Figure 1</xref>.</p>
      <p>A linear mixed model was fitted using the proportion of 2020 Trump votes within a state as the primary predictive variable. This variable was also nested within state-level clusters to allow its effect to vary within each state. Models using the proposed Governor and Congressional proportion variables were discarded due to comparatively poor model fit statistics. Akaike information criterion (AIC) and Bayesian information criterion (BIC) scores for discarded models were in the range of 600 to 800 points higher, indicating poorer fit.</p>
      <table-wrap position="float" id="table1">
        <label>Table 1</label>
        <caption>
          <p>Descriptive statistics.</p>
        </caption>
        <table width="1000" cellpadding="5" cellspacing="0" border="1" rules="groups" frame="hsides">
          <col width="50"/>
          <col width="50"/>
          <col width="50"/>
          <col width="50"/>
          <col width="50"/>
          <col width="0"/>
          <col width="50"/>
          <col width="50"/>
          <col width="50"/>
          <col width="50"/>
          <col width="0"/>
          <col width="90"/>
          <col width="0"/>
          <col width="80"/>
          <col width="0"/>
          <col width="100"/>
          <col width="0"/>
          <col width="90"/>
          <col width="0"/>
          <col width="110"/>
          <col width="0"/>
          <col width="80"/>
          <thead>
            <tr valign="top">
              <td>State</td>
              <td colspan="5">Case peaks</td>
              <td colspan="5">Search peaks</td>
              <td colspan="2">Proportion of Trump votes</td>
              <td colspan="2">GOP governor</td>
              <td colspan="2">GOP proportion of Congress</td>
              <td colspan="2">Population</td>
              <td colspan="2">Daily cases, mean</td>
              <td>Daily deaths, mean</td>
            </tr>
            <tr valign="top">
              <td>
                <break/>
              </td>
              <td>1</td>
              <td>2</td>
              <td>3</td>
              <td>4</td>
              <td colspan="2">1</td>
              <td>2</td>
              <td>3</td>
              <td>4</td>
              <td colspan="2">
                <break/>
              </td>
              <td colspan="2">
                <break/>
              </td>
              <td colspan="2">
                <break/>
              </td>
              <td colspan="2">
                <break/>
              </td>
              <td colspan="2">
                <break/>
              </td>
              <td colspan="2">
                <break/>
              </td>
            </tr>
          </thead>
          <tbody>
            <tr valign="top">
              <td>AL</td>
              <td>185</td>
              <td>349</td>
              <td>584</td>
              <td>728</td>
              <td colspan="2">157</td>
              <td>342</td>
              <td>581</td>
              <td>707</td>
              <td colspan="2">0.62</td>
              <td colspan="2">1</td>
              <td colspan="2">0.83</td>
              <td colspan="2">5024803</td>
              <td colspan="2">1700.48</td>
              <td colspan="2">25.48</td>
            </tr>
            <tr valign="top">
              <td>AK</td>
              <td>187</td>
              <td>319</td>
              <td>612</td>
              <td>729</td>
              <td colspan="2">176</td>
              <td>273</td>
              <td>589</td>
              <td>714</td>
              <td colspan="2">0.53</td>
              <td colspan="2">1</td>
              <td colspan="2">0.83</td>
              <td colspan="2">732441</td>
              <td colspan="2">314.21</td>
              <td colspan="2">1.38</td>
            </tr>
            <tr valign="top">
              <td>AZ</td>
              <td>162</td>
              <td>348</td>
              <td>585</td>
              <td>732</td>
              <td colspan="2">156</td>
              <td>342</td>
              <td>579</td>
              <td>718</td>
              <td colspan="2">0.49</td>
              <td colspan="2">1</td>
              <td colspan="2">0.45</td>
              <td colspan="2">7177986</td>
              <td colspan="2">2639.60</td>
              <td colspan="2">39.21</td>
            </tr>
            <tr valign="top">
              <td>AR</td>
              <td>199</td>
              <td>346</td>
              <td>576</td>
              <td>729</td>
              <td colspan="2">156</td>
              <td>342</td>
              <td>496</td>
              <td>718</td>
              <td colspan="2">0.62</td>
              <td colspan="2">1</td>
              <td colspan="2">0.92</td>
              <td colspan="2">3012232</td>
              <td colspan="2">1092.82</td>
              <td colspan="2">14.46</td>
            </tr>
            <tr valign="top">
              <td>CA</td>
              <td>183</td>
              <td>354</td>
              <td>589</td>
              <td>727</td>
              <td colspan="2">169</td>
              <td>344</td>
              <td>463</td>
              <td>714</td>
              <td colspan="2">0.34</td>
              <td colspan="2">0</td>
              <td colspan="2">0.16</td>
              <td colspan="2">39499738</td>
              <td colspan="2">11713.8</td>
              <td colspan="2">116.06</td>
            </tr>
            <tr valign="top">
              <td>CO</td>
              <td>189</td>
              <td>297</td>
              <td>457</td>
              <td>717</td>
              <td colspan="2">181</td>
              <td>302</td>
              <td>461</td>
              <td>708</td>
              <td colspan="2">0.42</td>
              <td colspan="2">0</td>
              <td colspan="2">0.39</td>
              <td colspan="2">5784308</td>
              <td colspan="2">1775.10</td>
              <td colspan="2">15.75</td>
            </tr>
            <tr valign="top">
              <td>CT</td>
              <td>92</td>
              <td>317</td>
              <td>605</td>
              <td>720</td>
              <td colspan="2">52</td>
              <td>294</td>
              <td>567</td>
              <td>707</td>
              <td colspan="2">0.39</td>
              <td colspan="2">0</td>
              <td colspan="2">0.00</td>
              <td colspan="2">3600260</td>
              <td colspan="2">969.89</td>
              <td colspan="2">14.17</td>
            </tr>
            <tr valign="top">
              <td>DE</td>
              <td>104</td>
              <td>354</td>
              <td>598</td>
              <td>718</td>
              <td colspan="2">57</td>
              <td>322</td>
              <td>567</td>
              <td>708</td>
              <td colspan="2">0.40</td>
              <td colspan="2">0</td>
              <td colspan="2">0.00</td>
              <td colspan="2">991886</td>
              <td colspan="2">341.14</td>
              <td colspan="2">3.76</td>
            </tr>
            <tr valign="top">
              <td>FL</td>
              <td>172</td>
              <td>345</td>
              <td>583</td>
              <td>718</td>
              <td colspan="2">157</td>
              <td>342</td>
              <td>567</td>
              <td>708</td>
              <td colspan="2">0.51</td>
              <td colspan="2">1</td>
              <td colspan="2">0.58</td>
              <td colspan="2">21569932</td>
              <td colspan="2">7696.68</td>
              <td colspan="2">96.48</td>
            </tr>
            <tr valign="top">
              <td>GA</td>
              <td>185</td>
              <td>353</td>
              <td>591</td>
              <td>713</td>
              <td colspan="2">158</td>
              <td>343</td>
              <td>579</td>
              <td>707</td>
              <td colspan="2">0.49</td>
              <td colspan="2">1</td>
              <td colspan="2">0.61</td>
              <td colspan="2">10725800</td>
              <td colspan="2">3628.51</td>
              <td colspan="2">48.40</td>
            </tr>
            <tr valign="top">
              <td>HI</td>
              <td>204</td>
              <td>287</td>
              <td>586</td>
              <td>707</td>
              <td colspan="2">174</td>
              <td>273</td>
              <td>537</td>
              <td>701</td>
              <td colspan="2">0.34</td>
              <td colspan="2">0</td>
              <td colspan="2">0.00</td>
              <td colspan="2">1451911</td>
              <td colspan="2">308.27</td>
              <td colspan="2">1.80</td>
            </tr>
            <tr valign="top">
              <td>ID</td>
              <td>176</td>
              <td>323</td>
              <td>631</td>
              <td>729</td>
              <td colspan="2">119</td>
              <td>300</td>
              <td>592</td>
              <td>727</td>
              <td colspan="2">0.64</td>
              <td colspan="2">1</td>
              <td colspan="2">0.75</td>
              <td colspan="2">1847772</td>
              <td colspan="2">583.05</td>
              <td colspan="2">6.43</td>
            </tr>
            <tr valign="top">
              <td>IL</td>
              <td>101</td>
              <td>297</td>
              <td>444</td>
              <td>720</td>
              <td colspan="2">57</td>
              <td>293</td>
              <td>462</td>
              <td>708</td>
              <td colspan="2">0.41</td>
              <td colspan="2">0</td>
              <td colspan="2">0.30</td>
              <td colspan="2">12785245</td>
              <td colspan="2">4045.41</td>
              <td colspan="2">49.53</td>
            </tr>
            <tr valign="top">
              <td>IN</td>
              <td>97</td>
              <td>317</td>
              <td>598</td>
              <td>720</td>
              <td colspan="2">57</td>
              <td>293</td>
              <td>579</td>
              <td>708</td>
              <td colspan="2">0.57</td>
              <td colspan="2">1</td>
              <td colspan="2">0.82</td>
              <td colspan="2">6785644</td>
              <td colspan="2">2218.97</td>
              <td colspan="2">30.79</td>
            </tr>
            <tr valign="top">
              <td>IA</td>
              <td>102</td>
              <td>297</td>
              <td>602</td>
              <td>724</td>
              <td colspan="2">92</td>
              <td>293</td>
              <td>601</td>
              <td>714</td>
              <td colspan="2">0.53</td>
              <td colspan="2">1</td>
              <td colspan="2">0.67</td>
              <td colspan="2">3188669</td>
              <td colspan="2">990.51</td>
              <td colspan="2">12.42</td>
            </tr>
            <tr valign="top">
              <td>KS</td>
              <td>100</td>
              <td>307</td>
              <td>597</td>
              <td>729</td>
              <td colspan="2">89</td>
              <td>301</td>
              <td>518</td>
              <td>714</td>
              <td colspan="2">0.56</td>
              <td colspan="2">0</td>
              <td colspan="2">0.83</td>
              <td colspan="2">2935880</td>
              <td colspan="2">1010.45</td>
              <td colspan="2">11.14</td>
            </tr>
            <tr valign="top">
              <td>KY</td>
              <td>101</td>
              <td>351</td>
              <td>590</td>
              <td>731</td>
              <td colspan="2">54</td>
              <td>344</td>
              <td>579</td>
              <td>707</td>
              <td colspan="2">0.62</td>
              <td colspan="2">0</td>
              <td colspan="2">0.88</td>
              <td colspan="2">3503958</td>
              <td colspan="2">1532.39</td>
              <td colspan="2">22.52</td>
            </tr>
            <tr valign="top">
              <td>LA</td>
              <td>173</td>
              <td>351</td>
              <td>590</td>
              <td>713</td>
              <td colspan="2">157</td>
              <td>342</td>
              <td>554</td>
              <td>707</td>
              <td colspan="2">0.58</td>
              <td colspan="2">0</td>
              <td colspan="2">0.82</td>
              <td colspan="2">4651203</td>
              <td colspan="2">1532.39</td>
              <td colspan="2">22.52</td>
            </tr>
            <tr valign="top">
              <td>ME</td>
              <td>119</td>
              <td>352</td>
              <td>450</td>
              <td>722</td>
              <td colspan="2">107</td>
              <td>293</td>
              <td>448</td>
              <td>707</td>
              <td colspan="2">0.44</td>
              <td colspan="2">0</td>
              <td colspan="2">0.12</td>
              <td colspan="2">1362280</td>
              <td colspan="2">312.94</td>
              <td colspan="2">2.98</td>
            </tr>
            <tr valign="top">
              <td>MD</td>
              <td>119</td>
              <td>318</td>
              <td>613</td>
              <td>719</td>
              <td colspan="2">111</td>
              <td>317</td>
              <td>615</td>
              <td>707</td>
              <td colspan="2">0.32</td>
              <td colspan="2">1</td>
              <td colspan="2">0.10</td>
              <td colspan="2">6172679</td>
              <td colspan="2">1334.10</td>
              <td colspan="2">18.86</td>
            </tr>
            <tr valign="top">
              <td>MA</td>
              <td>94</td>
              <td>3353</td>
              <td>603</td>
              <td>720</td>
              <td colspan="2">53</td>
              <td>350</td>
              <td>567</td>
              <td>708</td>
              <td colspan="2">0.32</td>
              <td colspan="2">1</td>
              <td colspan="2">0.05</td>
              <td colspan="2">7022220</td>
              <td colspan="2">2246.86</td>
              <td colspan="2">31.38</td>
            </tr>
            <tr valign="top">
              <td>MI</td>
              <td>115</td>
              <td>294</td>
              <td>440</td>
              <td>728</td>
              <td colspan="2">111</td>
              <td>293</td>
              <td>442</td>
              <td>708</td>
              <td colspan="2">0.48</td>
              <td colspan="2">0</td>
              <td colspan="2">0.44</td>
              <td colspan="2">10067664</td>
              <td colspan="2">3134.87</td>
              <td colspan="2">39.68</td>
            </tr>
            <tr valign="top">
              <td>MN</td>
              <td>107</td>
              <td>298</td>
              <td>441</td>
              <td>729</td>
              <td colspan="2">105</td>
              <td>300</td>
              <td>462</td>
              <td>714</td>
              <td colspan="2">0.45</td>
              <td colspan="2">0</td>
              <td colspan="2">0.35</td>
              <td colspan="2">5707165</td>
              <td colspan="2">1869.44</td>
              <td colspan="2">16.37</td>
            </tr>
            <tr valign="top">
              <td>MS</td>
              <td>181</td>
              <td>353</td>
              <td>576</td>
              <td>728</td>
              <td colspan="2">157</td>
              <td>342</td>
              <td>578</td>
              <td>707</td>
              <td colspan="2">0.58</td>
              <td colspan="2">1</td>
              <td colspan="2">0.83</td>
              <td colspan="2">2956870</td>
              <td colspan="2">1029.39</td>
              <td colspan="2">14.62</td>
            </tr>
            <tr valign="top">
              <td>MI</td>
              <td>98</td>
              <td>293</td>
              <td>560</td>
              <td>713</td>
              <td colspan="2">55</td>
              <td>259</td>
              <td>567</td>
              <td>708</td>
              <td colspan="2">0.57</td>
              <td colspan="2">1</td>
              <td colspan="2">0.80</td>
              <td colspan="2">6154481</td>
              <td colspan="2">1841.18</td>
              <td colspan="2">22.75</td>
            </tr>
            <tr valign="top">
              <td>MT</td>
              <td>107</td>
              <td>298</td>
              <td>643</td>
              <td>734</td>
              <td colspan="2">57</td>
              <td>257</td>
              <td>641</td>
              <td>719</td>
              <td colspan="2">0.57</td>
              <td colspan="2">1</td>
              <td colspan="2">0.67</td>
              <td colspan="2">1086193</td>
              <td colspan="2">355.38</td>
              <td colspan="2">4.24</td>
            </tr>
            <tr valign="top">
              <td>NE</td>
              <td>107</td>
              <td>302</td>
              <td>603</td>
              <td>727</td>
              <td colspan="2">57</td>
              <td>301</td>
              <td>574</td>
              <td>714</td>
              <td colspan="2">0.58</td>
              <td colspan="2">1</td>
              <td colspan="2">0.80</td>
              <td colspan="2">1961455</td>
              <td colspan="2">598.27</td>
              <td colspan="2">4.97</td>
            </tr>
            <tr valign="top">
              <td>NV</td>
              <td>177</td>
              <td>352</td>
              <td>598</td>
              <td>728</td>
              <td colspan="2">169</td>
              <td>343</td>
              <td>587</td>
              <td>714</td>
              <td colspan="2">0.48</td>
              <td colspan="2">0</td>
              <td colspan="2">0.33</td>
              <td colspan="2">3114071</td>
              <td colspan="2">895.19</td>
              <td colspan="2">13.34</td>
            </tr>
            <tr valign="top">
              <td>NH</td>
              <td>101</td>
              <td>348</td>
              <td>444</td>
              <td>728</td>
              <td colspan="2">57</td>
              <td>259</td>
              <td>400</td>
              <td>714</td>
              <td colspan="2">0.45</td>
              <td colspan="2">1</td>
              <td colspan="2">0.00</td>
              <td colspan="2">1377848</td>
              <td colspan="2">399.08</td>
              <td colspan="2">3.23</td>
            </tr>
            <tr valign="top">
              <td>NJ</td>
              <td>73</td>
              <td>354</td>
              <td>604</td>
              <td>717</td>
              <td colspan="2">57</td>
              <td>350</td>
              <td>567</td>
              <td>707</td>
              <td colspan="2">0.41</td>
              <td colspan="2">0</td>
              <td colspan="2">0.07</td>
              <td colspan="2">9279743</td>
              <td colspan="2">2919.76</td>
              <td colspan="2">41.26</td>
            </tr>
            <tr valign="top">
              <td>NM</td>
              <td>188</td>
              <td>303</td>
              <td>569</td>
              <td>728</td>
              <td colspan="2">174</td>
              <td>302</td>
              <td>567</td>
              <td>707</td>
              <td colspan="2">0.43</td>
              <td colspan="2">0</td>
              <td colspan="2">0.09</td>
              <td colspan="2">3117566</td>
              <td colspan="2">680.66</td>
              <td colspan="2">9.39</td>
            </tr>
            <tr valign="top">
              <td>NY</td>
              <td>78</td>
              <td>353</td>
              <td>605</td>
              <td>717</td>
              <td colspan="2">57</td>
              <td>350</td>
              <td>567</td>
              <td>707</td>
              <td colspan="2">0.37</td>
              <td colspan="2">0</td>
              <td colspan="2">0.28</td>
              <td colspan="2">21054933</td>
              <td colspan="2">3541.09</td>
              <td colspan="2">36.21</td>
            </tr>
            <tr valign="top">
              <td>NC</td>
              <td>184</td>
              <td>346</td>
              <td>590</td>
              <td>723</td>
              <td colspan="2">158</td>
              <td>342</td>
              <td>596</td>
              <td>708</td>
              <td colspan="2">0.50</td>
              <td colspan="2">0</td>
              <td colspan="2">0.72</td>
              <td colspan="2">10457177</td>
              <td colspan="2">3452.16</td>
              <td colspan="2">30.48</td>
            </tr>
            <tr valign="top">
              <td>ND</td>
              <td>121</td>
              <td>297</td>
              <td>440</td>
              <td>729</td>
              <td colspan="2">113</td>
              <td>295</td>
              <td>449</td>
              <td>707</td>
              <td colspan="2">0.65</td>
              <td colspan="2">1</td>
              <td colspan="2">0.67</td>
              <td colspan="2">778962</td>
              <td colspan="2">314.86</td>
              <td colspan="2">2.95</td>
            </tr>
            <tr valign="top">
              <td>OH</td>
              <td>89</td>
              <td>322</td>
              <td>598</td>
              <td>725</td>
              <td colspan="2">57</td>
              <td>302</td>
              <td>595</td>
              <td>708</td>
              <td colspan="2">0.53</td>
              <td colspan="2">1</td>
              <td colspan="2">0.71</td>
              <td colspan="2">11790587</td>
              <td colspan="2">3502.58</td>
              <td colspan="2">45.09</td>
            </tr>
            <tr valign="top">
              <td>OK</td>
              <td>168</td>
              <td>343</td>
              <td>582</td>
              <td>727</td>
              <td colspan="2">158</td>
              <td>343</td>
              <td>550</td>
              <td>718</td>
              <td colspan="2">0.65</td>
              <td colspan="2">1</td>
              <td colspan="2">0.93</td>
              <td colspan="2">3962031</td>
              <td colspan="2">1358.26</td>
              <td colspan="2">17.05</td>
            </tr>
            <tr valign="top">
              <td>OR</td>
              <td>180</td>
              <td>318</td>
              <td>584</td>
              <td>728</td>
              <td colspan="2">128</td>
              <td>300</td>
              <td>567</td>
              <td>718</td>
              <td colspan="2">0.40</td>
              <td colspan="2">0</td>
              <td colspan="2">0.14</td>
              <td colspan="2">4241544</td>
              <td colspan="2">927.88</td>
              <td colspan="2">9.70</td>
            </tr>
            <tr valign="top">
              <td>PA</td>
              <td>79</td>
              <td>325</td>
              <td>448</td>
              <td>718</td>
              <td colspan="2">57</td>
              <td>314</td>
              <td>462</td>
              <td>708</td>
              <td colspan="2">0.49</td>
              <td colspan="2">0</td>
              <td colspan="2">0.50</td>
              <td colspan="2">12989625</td>
              <td colspan="2">3635.72</td>
              <td colspan="2">58.22</td>
            </tr>
            <tr valign="top">
              <td>RI</td>
              <td>108</td>
              <td>317</td>
              <td>602</td>
              <td>716</td>
              <td colspan="2">106</td>
              <td>258</td>
              <td>568</td>
              <td>709</td>
              <td colspan="2">0.39</td>
              <td colspan="2">0</td>
              <td colspan="2">0.00</td>
              <td colspan="2">1096229</td>
              <td colspan="2">454.25</td>
              <td colspan="2">4.62</td>
            </tr>
            <tr valign="top">
              <td>SC</td>
              <td>181</td>
              <td>353</td>
              <td>584</td>
              <td>732</td>
              <td colspan="2">158</td>
              <td>342</td>
              <td>587</td>
              <td>708</td>
              <td colspan="2">0.55</td>
              <td colspan="2">1</td>
              <td colspan="2">0.83</td>
              <td colspan="2">5130829</td>
              <td colspan="2">1927.38</td>
              <td colspan="2">23.20</td>
            </tr>
            <tr valign="top">
              <td>SD</td>
              <td>109</td>
              <td>296</td>
              <td>679</td>
              <td>729</td>
              <td colspan="2">57</td>
              <td>301</td>
              <td>576</td>
              <td>707</td>
              <td colspan="2">0.62</td>
              <td colspan="2">1</td>
              <td colspan="2">1.00</td>
              <td colspan="2">887099</td>
              <td colspan="2">310.05</td>
              <td colspan="2">3.79</td>
            </tr>
            <tr valign="top">
              <td>TN</td>
              <td>192</td>
              <td>330</td>
              <td>599</td>
              <td>730</td>
              <td colspan="2">158</td>
              <td>343</td>
              <td>582</td>
              <td>707</td>
              <td colspan="2">0.61</td>
              <td colspan="2">1</td>
              <td colspan="2">0.82</td>
              <td colspan="2">6920119</td>
              <td colspan="2">2633.41</td>
              <td colspan="2">30.82</td>
            </tr>
            <tr valign="top">
              <td>TX</td>
              <td>178</td>
              <td>350</td>
              <td>596</td>
              <td>722</td>
              <td colspan="2">158</td>
              <td>343</td>
              <td>587</td>
              <td>707</td>
              <td colspan="2">0.52</td>
              <td colspan="2">1</td>
              <td colspan="2">0.66</td>
              <td colspan="2">29217653</td>
              <td colspan="2">8525.78</td>
              <td colspan="2">113.08</td>
            </tr>
            <tr valign="top">
              <td>UT</td>
              <td>177</td>
              <td>398</td>
              <td>664</td>
              <td>728</td>
              <td colspan="2">126</td>
              <td>300</td>
              <td>596</td>
              <td>714</td>
              <td colspan="2">0.58</td>
              <td colspan="2">1</td>
              <td colspan="2">0.92</td>
              <td colspan="2">3281685</td>
              <td colspan="2">1217.06</td>
              <td colspan="2">6.19</td>
            </tr>
            <tr valign="top">
              <td>VT</td>
              <td>74</td>
              <td>436</td>
              <td>604</td>
              <td>728</td>
              <td colspan="2">57</td>
              <td>460</td>
              <td>561</td>
              <td>715</td>
              <td colspan="2">0.30</td>
              <td colspan="2">1</td>
              <td colspan="2">0.00</td>
              <td colspan="2">642495</td>
              <td colspan="2">144.45</td>
              <td colspan="2">0.78</td>
            </tr>
            <tr valign="top">
              <td>VA</td>
              <td>125</td>
              <td>361</td>
              <td>597</td>
              <td>720</td>
              <td colspan="2">111</td>
              <td>342</td>
              <td>601</td>
              <td>708</td>
              <td colspan="2">0.44</td>
              <td colspan="2">1</td>
              <td colspan="2">0.31</td>
              <td colspan="2">8632044</td>
              <td colspan="2">2202.54</td>
              <td colspan="2">25.94</td>
            </tr>
            <tr valign="top">
              <td>WA</td>
              <td>167</td>
              <td>322</td>
              <td>561</td>
              <td>737</td>
              <td colspan="2">113</td>
              <td>301</td>
              <td>567</td>
              <td>715</td>
              <td colspan="2">0.39</td>
              <td colspan="2">0</td>
              <td colspan="2">0.25</td>
              <td colspan="2">7718785</td>
              <td colspan="2">1920.45</td>
              <td colspan="2">16.46</td>
            </tr>
            <tr valign="top">
              <td>WV</td>
              <td>183</td>
              <td>346</td>
              <td>612</td>
              <td>713</td>
              <td colspan="2">159</td>
              <td>265</td>
              <td>553</td>
              <td>707</td>
              <td colspan="2">0.69</td>
              <td colspan="2">1</td>
              <td colspan="2">0.80</td>
              <td colspan="2">1789798</td>
              <td colspan="2">653.50</td>
              <td colspan="2">8.81</td>
            </tr>
            <tr valign="top">
              <td>WI</td>
              <td>143</td>
              <td>297</td>
              <td>623</td>
              <td>727</td>
              <td colspan="2">111</td>
              <td>293</td>
              <td>615</td>
              <td>707</td>
              <td colspan="2">0.49</td>
              <td colspan="2">0</td>
              <td colspan="2">0.60</td>
              <td colspan="2">5892323</td>
              <td colspan="2">2085.48</td>
              <td colspan="2">18.84</td>
            </tr>
            <tr valign="top">
              <td>WY</td>
              <td>189</td>
              <td>301</td>
              <td>589</td>
              <td>748</td>
              <td colspan="2">159</td>
              <td>301</td>
              <td>603</td>
              <td>722</td>
              <td colspan="2">0.69</td>
              <td colspan="2">1</td>
              <td colspan="2">1.00</td>
              <td colspan="2">577267</td>
              <td colspan="2">204.82</td>
              <td colspan="2">2.35</td>
            </tr>
          </tbody>
        </table>
      </table-wrap>
      <fig id="figure1" position="float">
        <label>Figure 1</label>
        <caption>
          <p>Homoscedasticity (constant variance of residuals). Note: amount and distance of points scattered above and below the line is equal.</p>
        </caption>
        <graphic xlink:href="formative_v6i12e40825_fig1.png" alt-version="no" mimetype="image" position="float" xlink:type="simple"/>
      </fig>
      <p>The model results are in <xref ref-type="table" rid="table2">Table 2</xref>; note that the scales displayed for predictors and controls are standardized, but lag time remains measured in days. I used 762 observations for 50 states to calculate 4 intervals of lag time, which led to a sample for the model of 50 states, or units of analysis with 4 repeated measurements per state. The model is a significant improvement over the null model with a single predictor (Trump proportion), no controls, and random intercepts for states. The AIC and BIC scores for the null model were 1776.04 and 1785.93, respectively—the selected model’s scores were 588.85 and 618.535, respectively, and so were a significant improvement in terms of model fit.</p>
      <p>However, none of the predictors within the final model were statistically significant, even those that were considered but excluded from the model (Governor party and Congressional delegation parties) because of worse overall model fit. The only significant independent variable in the selected model was mean daily deaths, which had a small negative relationship with lag time, at <italic>P</italic>&#60;.10.</p>
      <p>Random elements in the model were individual state-level intercepts and Trump proportion nested within state as a random effect. There was sufficient variance (σ<sup>2</sup><sub>state</sub>=0.099 and σ<sup>2</sup><sub>trump</sub>=0.049) in this random portion of the model to justify their inclusion. The random intercepts for state clusters captured a portion of and thereby reduced the fixed effect residuals, but the model itself lacked predictive power. Thus, the model’s fixed effect error term still captured a relatively high amount of the variance in the data. As <xref rid="figure2" ref-type="fig">Figure 2</xref> shows, there was a range of positive and negative values for each effect, but the relatively wide confidence intervals (95% CI) were another artifact of the relatively low predictive power for this model.</p>
      <table-wrap position="float" id="table2">
        <label>Table 2</label>
        <caption>
          <p>Linear mixed model results (dependent variable is lag time in days).</p>
        </caption>
        <table width="1000" cellpadding="5" cellspacing="0" border="1" rules="groups" frame="hsides">
          <col width="30"/>
          <col width="370"/>
          <col width="200"/>
          <col width="400"/>
          <thead>
            <tr valign="top">
              <td colspan="2">Variable</td>
              <td>Statistic</td>
              <td><italic>P</italic> value</td>
            </tr>
          </thead>
          <tbody>
            <tr valign="top">
              <td colspan="2">Proportion of Trump 2020 votes, β (SE)</td>
              <td>0.063 (0.091)</td>
              <td>.49</td>
            </tr>
            <tr valign="top">
              <td colspan="2">Mean daily cases, β (SE)</td>
              <td>0.461 (0.569)</td>
              <td>.42</td>
            </tr>
            <tr valign="top">
              <td colspan="2">Mean daily deaths, β (SE)</td>
              <td>–0.670 (0.356)</td>
              <td>.07</td>
            </tr>
            <tr valign="top">
              <td colspan="2">Population, β (SE)</td>
              <td>0.168 (0.430)</td>
              <td>.70</td>
            </tr>
            <tr valign="top">
              <td colspan="2">Constant, β (SE)</td>
              <td>–0.012 (0.076)</td>
              <td>.88</td>
            </tr>
            <tr valign="top">
              <td colspan="4">
                <bold>Overall model</bold>
              </td>
            </tr>
            <tr valign="top">
              <td>
                <break/>
              </td>
              <td>Observations</td>
              <td>200</td>
              <td>N/A<sup>a</sup></td>
            </tr>
            <tr valign="top">
              <td>
                <break/>
              </td>
              <td>Log likelihood</td>
              <td>–285.425</td>
              <td>N/A</td>
            </tr>
            <tr valign="top">
              <td>
                <break/>
              </td>
              <td>Akaike information criterion</td>
              <td>588.850</td>
              <td>N/A</td>
            </tr>
            <tr valign="top">
              <td>.</td>
              <td>Bayesian information criterion</td>
              <td>618.535</td>
              <td>N/A</td>
            </tr>
          </tbody>
        </table>
        <table-wrap-foot>
          <fn id="table2fn1">
            <p><sup>a</sup>N/A: not applicable.</p>
          </fn>
        </table-wrap-foot>
      </table-wrap>
      <fig id="figure2" position="float">
        <label>Figure 2</label>
        <caption>
          <p>Random effect estimates (intercept and 95% CI) in lag time by state; red: negative; blue: positive.</p>
        </caption>
        <graphic xlink:href="formative_v6i12e40825_fig2.png" alt-version="no" mimetype="image" position="float" xlink:type="simple"/>
      </fig>
    </sec>
    <sec sec-type="discussion">
      <title>Discussion</title>
      <sec>
        <title>Principal Findings</title>
        <p>I have identified evidence to support a positive answer for the first research question but not the second. Results suggest that yes, there are meaningful state-level differences in the lag time between spikes in COVID-19–related search traffic and spikes in COVID-19 cases. However, my hypothesis that political covariates would contribute to a portion of this variation in a statistically significant way was not supported. My findings indicate that including proportion of 2020 Trump voters in the final linear mixed model leads to a model fit that is overall much stronger than a null model with control variables only, but this political affiliation variable is not itself statistically significant or appropriate for predictive inference.</p>
        <p>Although this is a partially negative finding, it challenges further work to explore how environmental factors and social group forces (both structural and interpersonal) may cause diverging responses to shared societal crises. The statistically significant differences in lag time across states demonstrate that there are meaningful differences across states that are at least partially causing these changes. I propose 3 potential causes for the mismatch between my theorized model and the results that also suggest directions for future research: level of geographic modeling, approach to calculating lag, and theoretical model mismatch.</p>
      </sec>
      <sec>
        <title>Level of Geographic Modeling</title>
        <p>My approach considered state as the geographic unit primarily for logistical reasons. State is an easily available unit in both the trends and cases data, meaning that they can be reliably matched for analysis. However, in doing so, I lost the opportunity for more localized nuance in my political covariates: Even using local city elections as a proxy for political identification might lead to stronger model fit and predictive power, but it may also be possible to use smaller blocks such as census tracts to map the trends data onto already narrowly geotagged CDC case data. However, the Google search data are limited by the lack of GIS shapefiles as well as the lack of city or tract-level data for areas outside of its large metro areas. An analysis with more granular geographic modeling would require the theoretical model to be reconsidered, given that it would exclude data from smaller cities and rural areas. If these localized variables were included in a hypothetical model, it would also likely be wise to include controls such as income and educational attainment, assuming that these are available for the census tracts or areas in question. On a smaller scale, these controls would likely contribute more significantly to a model than the same statistics at a broad state level.</p>
      </sec>
      <sec>
        <title>Approach to Calculating Lag</title>
        <p>Although I believe that the method I used to calculate lag is defensible, it is possible that another approach may uncover a significant relationship that is not present here. Pelat et al [<xref ref-type="bibr" rid="ref39">39</xref>] provided one option within the same domain, describing a method that calculates correlations between increases in searches and incidence of a disease at predefined intervals (eg, 1 week, 1 month). These correlations were stored and used for further regression analysis along with selected predictors.</p>
        <p>Effenberger et al [<xref ref-type="bibr" rid="ref40">40</xref>] took another approach that accounts for lag specifically via time lag correlational analysis. Instead of calculating lag as a repeated measure in longitudinal analysis, they instead mapped multiple models as a network, examining how associations changed at predefined time intervals. This would require a significant reconfiguring of how the research questions were operationalized but may uncover relationships that were not established here.</p>
      </sec>
      <sec>
        <title>Theoretical Model Mismatch</title>
        <p>In short, my proposed theoretical model was that, per Bandura’s SCT [<xref ref-type="bibr" rid="ref7">7</xref>], individuals act as part of a constantly changing and reciprocal triad that is bound by personal factors, environmental influences, and past behavior. As part of this triad, the SIA also tells us that individuals sort themselves into groups by categorizing others, giving meaning to those categories, and then self-sorting into one of these groups. Given how politicized the ongoing cultural response to COVID-19 and mitigation measures has been, I suspected that the competing political understandings of the nature of the pandemic and appropriate reaction would be part of these social processes. Eventually, I hypothesized that this dynamic would affect how people managed their own symptom surveillance and perceived risk, meaning that we would discover differences in lag time between search incidence and case incidence. This theoretical model was not supported by the data here.</p>
        <p>It is possible that another theoretical model, operationalized with a different set of predictors, would generate a significant statistical model to explain lag time variation. Barber and Pope [<xref ref-type="bibr" rid="ref41">41</xref>] approached political identity by investigating how one’s political party identification is associated with his or her individual political ideology; one element from their model that is missing here is the influence of fellow members of a political group. They also define the concepts of “party loyalist” and “policy loyalist” in the context of Trump’s election, which represented an opportune time for investigating this relationship, as ideology and party often diverged. It may be possible to capture these concepts in a more localized model as described in the previous paragraphs.</p>
        <p>Another potential theoretical model comes partially from Agadjanian and Lacy [<xref ref-type="bibr" rid="ref42">42</xref>], who investigated how individual political leaders have a more significant influence on public opinion and behavior than party or ideology. This suggests that, if we could capture political leader characteristics and ideology with some level of granularity, this could be folded into the proposed SCT/SIA framework. As an example, coding leader rhetoric (whether manually or via sentiment analysis) would be possible under both a localized (city or tract-specific) or state-level model.</p>
        <p>Notably, a limitation of my approach to geographic modeling is that it does not allow for investigation into whether state-level data are reliably correlated with local provincial data; future research could address this limitation via a study that specifically considers larger cities in comparison with their states or even extend this via propensity score matching at the metro and state levels. Further, the way that I chose to operationalize political identity and affiliation is suboptimal because of the existing geographic constraint. In taking an approach that allows for more geographic granularity, future studies could also model for more granular political variables such as voting by census tract, city council representatives, and mayors.</p>
        <p>Although there may be structural, demographic, and geographic factors that contribute to these differences, I believe that the effects of political affiliation and its rippling effects are also closely tied to these significant differences in lag time across states. In the context of not only increasing political polarization and opportunities for political speech but also changing laws and norms around American federalism that may give states more control over shared social functions, it is more important than ever to understand how political identity and political communication affect the physical well-being of a state’s residents.</p>
      </sec>
    </sec>
  </body>
  <back>
    <app-group/>
    <glossary>
      <title>Abbreviations</title>
      <def-list>
        <def-item>
          <term id="abb1">AIC</term>
          <def>
            <p>Akaike information criterion</p>
          </def>
        </def-item>
        <def-item>
          <term id="abb2">BIC</term>
          <def>
            <p>Bayesian information criterion</p>
          </def>
        </def-item>
        <def-item>
          <term id="abb3">CDC</term>
          <def>
            <p>Centers for Disease Control and Prevention</p>
          </def>
        </def-item>
        <def-item>
          <term id="abb4">GIS</term>
          <def>
            <p>geographic information system</p>
          </def>
        </def-item>
        <def-item>
          <term id="abb5">NDS</term>
          <def>
            <p>novel data streams</p>
          </def>
        </def-item>
        <def-item>
          <term id="abb6">SCT</term>
          <def>
            <p>social cognitive theory</p>
          </def>
        </def-item>
        <def-item>
          <term id="abb7">SIA</term>
          <def>
            <p>social identity approach</p>
          </def>
        </def-item>
      </def-list>
    </glossary>
    <fn-group>
      <fn fn-type="conflict">
        <p>None declared.</p>
      </fn>
    </fn-group>
    <ref-list>
      <ref id="ref1">
        <label>1</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Lyu</surname>
              <given-names>W</given-names>
            </name>
            <name name-style="western">
              <surname>Wehby</surname>
              <given-names>GL</given-names>
            </name>
          </person-group>
          <article-title>Community use of face masks and COVID-19: evidence from a natural experiment of state mandates in the US</article-title>
          <source>Health Aff (Millwood)</source>
          <year>2020</year>
          <month>08</month>
          <volume>39</volume>
          <issue>8</issue>
          <fpage>1419</fpage>
          <lpage>1425</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://doi.org/10.1377/hlthaff.2020.00818"/>
          </comment>
          <pub-id pub-id-type="doi">10.1377/hlthaff.2020.00818</pub-id>
          <pub-id pub-id-type="medline">32543923</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref2">
        <label>2</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <collab>Fischer</collab>
            <name name-style="western">
              <surname>Adrien</surname>
              <given-names>N</given-names>
            </name>
            <name name-style="western">
              <surname>Silguero</surname>
              <given-names>JJ</given-names>
            </name>
            <name name-style="western">
              <surname>Hopper</surname>
              <given-names>JJ</given-names>
            </name>
            <name name-style="western">
              <surname>Chowdhury</surname>
              <given-names>AI</given-names>
            </name>
            <name name-style="western">
              <surname>Werler</surname>
              <given-names>MM</given-names>
            </name>
          </person-group>
          <article-title>Mask adherence and rate of COVID-19 across the United States</article-title>
          <source>PLoS One</source>
          <year>2021</year>
          <volume>16</volume>
          <issue>4</issue>
          <fpage>e0249891</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://dx.plos.org/10.1371/journal.pone.0249891"/>
          </comment>
          <pub-id pub-id-type="doi">10.1371/journal.pone.0249891</pub-id>
          <pub-id pub-id-type="medline">33852626</pub-id>
          <pub-id pub-id-type="pii">PONE-D-21-01163</pub-id>
          <pub-id pub-id-type="pmcid">PMC8046247</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref3">
        <label>3</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Thunström</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Newbold</surname>
              <given-names>SC</given-names>
            </name>
            <name name-style="western">
              <surname>Finnoff</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Ashworth</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Shogren</surname>
              <given-names>JF</given-names>
            </name>
          </person-group>
          <article-title>The benefits and costs of using social distancing to flatten the curve for COVID-19</article-title>
          <source>J. Benefit Cost Anal</source>
          <year>2020</year>
          <month>05</month>
          <day>21</day>
          <volume>11</volume>
          <issue>2</issue>
          <fpage>179</fpage>
          <lpage>195</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://doi.org/10.1017/bca.2020.12"/>
          </comment>
          <pub-id pub-id-type="doi">10.1017/bca.2020.12</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref4">
        <label>4</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Lin</surname>
              <given-names>C</given-names>
            </name>
            <name name-style="western">
              <surname>Tu</surname>
              <given-names>P</given-names>
            </name>
            <name name-style="western">
              <surname>Beitsch</surname>
              <given-names>LM</given-names>
            </name>
          </person-group>
          <article-title>Confidence and receptivity for COVID-19 vaccines: a rapid systematic review</article-title>
          <source>Vaccines (Basel)</source>
          <year>2020</year>
          <month>12</month>
          <day>30</day>
          <volume>9</volume>
          <issue>1</issue>
          <fpage>A</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://europepmc.org/abstract/MED/33396832"/>
          </comment>
          <pub-id pub-id-type="doi">10.3390/vaccines9010016</pub-id>
          <pub-id pub-id-type="medline">33396832</pub-id>
          <pub-id pub-id-type="pii">vaccines9010016</pub-id>
          <pub-id pub-id-type="pmcid">PMC7823859</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref5">
        <label>5</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Cowan</surname>
              <given-names>SK</given-names>
            </name>
            <name name-style="western">
              <surname>Mark</surname>
              <given-names>N</given-names>
            </name>
            <name name-style="western">
              <surname>Reich</surname>
              <given-names>JA</given-names>
            </name>
          </person-group>
          <article-title>COVID-19 vaccine hesitancy is the new terrain for political division among Americans</article-title>
          <source>Socius</source>
          <year>2021</year>
          <month>06</month>
          <day>17</day>
          <volume>7</volume>
          <fpage>237802312110236</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://doi.org/10.1177/23780231211023657"/>
          </comment>
          <pub-id pub-id-type="doi">10.1177/23780231211023657</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref6">
        <label>6</label>
        <nlm-citation citation-type="web">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Stephens-Davidowitz</surname>
              <given-names>S</given-names>
            </name>
          </person-group>
          <article-title>Google Searches Can Help Us Find Emerging Covid-19 Outbreaks</article-title>
          <source>New York Times</source>
          <year>2020</year>
          <month>04</month>
          <day>05</day>
          <access-date>2022-12-07</access-date>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://www.nytimes.com/2020/04/05/opinion/coronavirus-google-searches.html">https://www.nytimes.com/2020/04/05/opinion/coronavirus-google-searches.html</ext-link>
          </comment>
        </nlm-citation>
      </ref>
      <ref id="ref7">
        <label>7</label>
        <nlm-citation citation-type="book">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Bandura</surname>
              <given-names>A</given-names>
            </name>
          </person-group>
          <source>Social Foundations of Thought and Action: A Social Cognitive Theory</source>
          <year>1986</year>
          <publisher-loc>Englewood Cliffs, NJ</publisher-loc>
          <publisher-name>Prentice Hall</publisher-name>
        </nlm-citation>
      </ref>
      <ref id="ref8">
        <label>8</label>
        <nlm-citation citation-type="book">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Abrams</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Hogg</surname>
              <given-names>MA</given-names>
            </name>
          </person-group>
          <source>Social identity theory: Constructive and critical advances</source>
          <year>1990</year>
          <publisher-loc>New York, NY</publisher-loc>
          <publisher-name>Springer-Verlag Publishing</publisher-name>
        </nlm-citation>
      </ref>
      <ref id="ref9">
        <label>9</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Bish</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Michie</surname>
              <given-names>S</given-names>
            </name>
          </person-group>
          <article-title>Demographic and attitudinal determinants of protective behaviours during a pandemic: a review</article-title>
          <source>Br J Health Psychol</source>
          <year>2010</year>
          <month>11</month>
          <volume>15</volume>
          <issue>Pt 4</issue>
          <fpage>797</fpage>
          <lpage>824</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://europepmc.org/abstract/MED/20109274"/>
          </comment>
          <pub-id pub-id-type="doi">10.1348/135910710X485826</pub-id>
          <pub-id pub-id-type="medline">20109274</pub-id>
          <pub-id pub-id-type="pii">bjhp807</pub-id>
          <pub-id pub-id-type="pmcid">PMC7185452</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref10">
        <label>10</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Clark</surname>
              <given-names>C</given-names>
            </name>
            <name name-style="western">
              <surname>Davila</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Regis</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Kraus</surname>
              <given-names>S</given-names>
            </name>
          </person-group>
          <article-title>Predictors of COVID-19 voluntary compliance behaviors: An international investigation</article-title>
          <source>Glob Transit</source>
          <year>2020</year>
          <volume>2</volume>
          <fpage>76</fpage>
          <lpage>82</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://linkinghub.elsevier.com/retrieve/pii/S2589-7918(20)30009-8"/>
          </comment>
          <pub-id pub-id-type="doi">10.1016/j.glt.2020.06.003</pub-id>
          <pub-id pub-id-type="medline">32835202</pub-id>
          <pub-id pub-id-type="pii">S2589-7918(20)30009-8</pub-id>
          <pub-id pub-id-type="pmcid">PMC7318969</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref11">
        <label>11</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Cho</surname>
              <given-names>H</given-names>
            </name>
            <name name-style="western">
              <surname>Lee</surname>
              <given-names>JS</given-names>
            </name>
          </person-group>
          <article-title>The influence of self-efficacy, subjective norms, and risk perception on behavioral intentions related to the H1N1 flu pandemic: A comparison between Korea and the US</article-title>
          <source>Asian Journal of Social Psychology</source>
          <year>2015</year>
          <month>06</month>
          <day>17</day>
          <volume>18</volume>
          <issue>4</issue>
          <fpage>311</fpage>
          <lpage>324</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://doi.org/10.1111/ajsp.12104"/>
          </comment>
          <pub-id pub-id-type="doi">10.1111/ajsp.12104</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref12">
        <label>12</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Brouard</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Vasilopoulos</surname>
              <given-names>P</given-names>
            </name>
            <name name-style="western">
              <surname>Becher</surname>
              <given-names>M</given-names>
            </name>
          </person-group>
          <article-title>Sociodemographic and psychological correlates of compliance with the COVID-19 public health measures in France</article-title>
          <source>Can J Pol Sci</source>
          <year>2020</year>
          <month>04</month>
          <day>23</day>
          <volume>53</volume>
          <issue>2</issue>
          <fpage>253</fpage>
          <lpage>258</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://doi.org/10.1017/S0008423920000335"/>
          </comment>
          <pub-id pub-id-type="doi">10.1017/s0008423920000335</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref13">
        <label>13</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Harper</surname>
              <given-names>CA</given-names>
            </name>
            <name name-style="western">
              <surname>Satchell</surname>
              <given-names>LP</given-names>
            </name>
            <name name-style="western">
              <surname>Fido</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Latzman</surname>
              <given-names>RD</given-names>
            </name>
          </person-group>
          <article-title>Functional fear predicts public health compliance in the COVID-19 pandemic</article-title>
          <source>Int J Ment Health Addict</source>
          <year>2021</year>
          <volume>19</volume>
          <issue>5</issue>
          <fpage>1875</fpage>
          <lpage>1888</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://europepmc.org/abstract/MED/32346359"/>
          </comment>
          <pub-id pub-id-type="doi">10.1007/s11469-020-00281-5</pub-id>
          <pub-id pub-id-type="medline">32346359</pub-id>
          <pub-id pub-id-type="pii">281</pub-id>
          <pub-id pub-id-type="pmcid">PMC7185265</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref14">
        <label>14</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Lerner</surname>
              <given-names>JS</given-names>
            </name>
            <name name-style="western">
              <surname>Keltner</surname>
              <given-names>D</given-names>
            </name>
          </person-group>
          <article-title>Fear, anger, and risk</article-title>
          <source>Journal of Personality and Social Psychology</source>
          <year>2001</year>
          <volume>81</volume>
          <issue>1</issue>
          <fpage>146</fpage>
          <lpage>159</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://doi.org/10.1037/0022-3514.81.1.146"/>
          </comment>
          <pub-id pub-id-type="doi">10.1037/0022-3514.81.1.146</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref15">
        <label>15</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Hughner</surname>
              <given-names>RS</given-names>
            </name>
            <name name-style="western">
              <surname>Kleine</surname>
              <given-names>SS</given-names>
            </name>
          </person-group>
          <article-title>Views of health in the lay sector: a compilation and review of how individuals think about health</article-title>
          <source>Health (London)</source>
          <year>2004</year>
          <month>10</month>
          <volume>8</volume>
          <issue>4</issue>
          <fpage>395</fpage>
          <lpage>422</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://doi.org/10.1177/1363459304045696"/>
          </comment>
          <pub-id pub-id-type="doi">10.1177/1363459304045696</pub-id>
          <pub-id pub-id-type="medline">15358896</pub-id>
          <pub-id pub-id-type="pii">8/4/395</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref16">
        <label>16</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Becker</surname>
              <given-names>MH</given-names>
            </name>
            <name name-style="western">
              <surname>Maiman</surname>
              <given-names>LA</given-names>
            </name>
          </person-group>
          <article-title>Sociobehavioral determinants of compliance with health and medical care recommendations</article-title>
          <source>Med Care</source>
          <year>1975</year>
          <month>01</month>
          <volume>13</volume>
          <issue>1</issue>
          <fpage>10</fpage>
          <lpage>24</lpage>
          <pub-id pub-id-type="doi">10.1097/00005650-197501000-00002</pub-id>
          <pub-id pub-id-type="medline">1089182</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref17">
        <label>17</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Nivette</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Ribeaud</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Murray</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Steinhoff</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Bechtiger</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Hepp</surname>
              <given-names>U</given-names>
            </name>
            <name name-style="western">
              <surname>Shanahan</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Eisner</surname>
              <given-names>M</given-names>
            </name>
          </person-group>
          <article-title>Non-compliance with COVID-19-related public health measures among young adults in Switzerland: Insights from a longitudinal cohort study</article-title>
          <source>Soc Sci Med</source>
          <year>2021</year>
          <month>01</month>
          <volume>268</volume>
          <fpage>113370</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://linkinghub.elsevier.com/retrieve/pii/S0277-9536(20)30589-X"/>
          </comment>
          <pub-id pub-id-type="doi">10.1016/j.socscimed.2020.113370</pub-id>
          <pub-id pub-id-type="medline">32980677</pub-id>
          <pub-id pub-id-type="pii">S0277-9536(20)30589-X</pub-id>
          <pub-id pub-id-type="pmcid">PMC7493799</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref18">
        <label>18</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Barari</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Caria</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Davola</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Falco</surname>
              <given-names>O</given-names>
            </name>
            <name name-style="western">
              <surname>Fetzer</surname>
              <given-names>T</given-names>
            </name>
            <name name-style="western">
              <surname>Fiorin</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Hensel</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Ivchenko</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Jachimowicz</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>King</surname>
              <given-names>G</given-names>
            </name>
            <name name-style="western">
              <surname>Kraft-Todd</surname>
              <given-names>G</given-names>
            </name>
            <name name-style="western">
              <surname>Ledda</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>MacLennan</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Mutoi</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Pagani</surname>
              <given-names>C</given-names>
            </name>
            <name name-style="western">
              <surname>Reutskaja</surname>
              <given-names>E</given-names>
            </name>
            <name name-style="western">
              <surname>Roth</surname>
              <given-names>C</given-names>
            </name>
            <name name-style="western">
              <surname>Raimondi Slepoi</surname>
              <given-names>F</given-names>
            </name>
          </person-group>
          <article-title>Evaluating COVID-19 public health messaging in Italy: self-reported compliance and growing mental health concerns</article-title>
          <source>medRxiv.</source>
          <comment>Preprint posted online April 5 2020. <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://www.medrxiv.org/content/10.1101/2020.03.27.20042820v2"/>
          </comment>
          <pub-id pub-id-type="doi">10.1101/2020.03.27.20042820</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref19">
        <label>19</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Callahan</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Jennings</surname>
              <given-names>B</given-names>
            </name>
          </person-group>
          <article-title>Ethics and public health: forging a strong relationship</article-title>
          <source>Am J Public Health</source>
          <year>2002</year>
          <month>02</month>
          <volume>92</volume>
          <issue>2</issue>
          <fpage>169</fpage>
          <lpage>76</lpage>
          <pub-id pub-id-type="doi">10.2105/ajph.92.2.169</pub-id>
          <pub-id pub-id-type="medline">11818284</pub-id>
          <pub-id pub-id-type="pmcid">PMC1447035</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref20">
        <label>20</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Baum</surname>
              <given-names>NM</given-names>
            </name>
            <name name-style="western">
              <surname>Jacobson</surname>
              <given-names>PD</given-names>
            </name>
            <name name-style="western">
              <surname>Goold</surname>
              <given-names>SD</given-names>
            </name>
          </person-group>
          <article-title>"Listen to the people": public deliberation about social distancing measures in a pandemic</article-title>
          <source>Am J Bioeth</source>
          <year>2009</year>
          <month>11</month>
          <day>04</day>
          <volume>9</volume>
          <issue>11</issue>
          <fpage>4</fpage>
          <lpage>14</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://doi.org/10.1080/15265160903197531"/>
          </comment>
          <pub-id pub-id-type="doi">10.1080/15265160903197531</pub-id>
          <pub-id pub-id-type="medline">19882444</pub-id>
          <pub-id pub-id-type="pii">916397252</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref21">
        <label>21</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Rubin</surname>
              <given-names>G</given-names>
            </name>
            <name name-style="western">
              <surname>Amlôt</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Page</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Wessely</surname>
              <given-names>S</given-names>
            </name>
          </person-group>
          <article-title>Public perceptions, anxiety, and behaviour change in relation to the swine flu outbreak: cross sectional telephone survey</article-title>
          <source>BMJ</source>
          <year>2009</year>
          <month>07</month>
          <day>02</day>
          <volume>339</volume>
          <fpage>b2651</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://europepmc.org/abstract/MED/19574308"/>
          </comment>
          <pub-id pub-id-type="doi">10.1136/bmj.b2651</pub-id>
          <pub-id pub-id-type="medline">19574308</pub-id>
          <pub-id pub-id-type="pii">bmj.b2651</pub-id>
          <pub-id pub-id-type="pmcid">PMC2714687</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref22">
        <label>22</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Callaway</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Runge</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Mullen</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Rentz</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Staley</surname>
              <given-names>K</given-names>
            </name>
            <name name-style="western">
              <surname>Stanford</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Watson</surname>
              <given-names>C</given-names>
            </name>
          </person-group>
          <article-title>Risk and the Republican National Convention: application of the novel COVID-19 operational risk assessment</article-title>
          <source>Disaster Med Public Health Prep</source>
          <year>2022</year>
          <month>08</month>
          <volume>16</volume>
          <issue>4</issue>
          <fpage>1612</fpage>
          <lpage>1617</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://europepmc.org/abstract/MED/33762039"/>
          </comment>
          <pub-id pub-id-type="doi">10.1017/dmp.2020.499</pub-id>
          <pub-id pub-id-type="medline">33762039</pub-id>
          <pub-id pub-id-type="pii">S1935789320004991</pub-id>
          <pub-id pub-id-type="pmcid">PMC8007956</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref23">
        <label>23</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Ginsberg</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Mohebbi</surname>
              <given-names>MH</given-names>
            </name>
            <name name-style="western">
              <surname>Patel</surname>
              <given-names>RS</given-names>
            </name>
            <name name-style="western">
              <surname>Brammer</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Smolinski</surname>
              <given-names>MS</given-names>
            </name>
            <name name-style="western">
              <surname>Brilliant</surname>
              <given-names>L</given-names>
            </name>
          </person-group>
          <article-title>Detecting influenza epidemics using search engine query data</article-title>
          <source>Nature</source>
          <year>2009</year>
          <month>02</month>
          <day>19</day>
          <volume>457</volume>
          <issue>7232</issue>
          <fpage>1012</fpage>
          <lpage>4</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://doi.org/10.1038/nature07634"/>
          </comment>
          <pub-id pub-id-type="doi">10.1038/nature07634</pub-id>
          <pub-id pub-id-type="medline">19020500</pub-id>
          <pub-id pub-id-type="pii">nature07634</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref24">
        <label>24</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Brownstein</surname>
              <given-names>JS</given-names>
            </name>
            <name name-style="western">
              <surname>Freifeld</surname>
              <given-names>CC</given-names>
            </name>
            <name name-style="western">
              <surname>Madoff</surname>
              <given-names>LC</given-names>
            </name>
          </person-group>
          <article-title>Digital disease detection — harnessing the web for public health surveillance</article-title>
          <source>N Engl J Med</source>
          <year>2009</year>
          <month>05</month>
          <day>21</day>
          <volume>360</volume>
          <issue>21</issue>
          <fpage>2153</fpage>
          <lpage>2157</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://doi.org/10.1056/NEJMp0900702"/>
          </comment>
          <pub-id pub-id-type="doi">10.1056/nejmp0900702</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref25">
        <label>25</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Polgreen</surname>
              <given-names>P</given-names>
            </name>
            <name name-style="western">
              <surname>Chen</surname>
              <given-names>Y</given-names>
            </name>
            <name name-style="western">
              <surname>Pennock</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Nelson</surname>
              <given-names>F</given-names>
            </name>
          </person-group>
          <article-title>Using internet searches for influenza surveillance</article-title>
          <source>Clin Infect Dis</source>
          <year>2008</year>
          <month>12</month>
          <day>01</day>
          <volume>47</volume>
          <issue>11</issue>
          <fpage>1443</fpage>
          <lpage>8</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://doi.org/10.1086/593098"/>
          </comment>
          <pub-id pub-id-type="doi">10.1086/593098</pub-id>
          <pub-id pub-id-type="medline">18954267</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref26">
        <label>26</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Cooper</surname>
              <given-names>CP</given-names>
            </name>
            <name name-style="western">
              <surname>Mallon</surname>
              <given-names>KP</given-names>
            </name>
            <name name-style="western">
              <surname>Leadbetter</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Pollack</surname>
              <given-names>LA</given-names>
            </name>
            <name name-style="western">
              <surname>Peipins</surname>
              <given-names>LA</given-names>
            </name>
          </person-group>
          <article-title>Cancer Internet search activity on a major search engine, United States 2001-2003</article-title>
          <source>J Med Internet Res</source>
          <year>2005</year>
          <month>07</month>
          <day>01</day>
          <volume>7</volume>
          <issue>3</issue>
          <fpage>e36</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://www.jmir.org/2005/3/e36/"/>
          </comment>
          <pub-id pub-id-type="doi">10.2196/jmir.7.3.e36</pub-id>
          <pub-id pub-id-type="medline">15998627</pub-id>
          <pub-id pub-id-type="pii">v7i3e36</pub-id>
          <pub-id pub-id-type="pmcid">PMC1550657</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref27">
        <label>27</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Jimenez</surname>
              <given-names>AJ</given-names>
            </name>
            <name name-style="western">
              <surname>Estevez-Reboredo</surname>
              <given-names>RM</given-names>
            </name>
            <name name-style="western">
              <surname>Santed</surname>
              <given-names>MA</given-names>
            </name>
            <name name-style="western">
              <surname>Ramos</surname>
              <given-names>V</given-names>
            </name>
          </person-group>
          <article-title>COVID-19 symptom-related Google searches and local COVID-19 incidence in Spain: correlational study</article-title>
          <source>J Med Internet Res</source>
          <year>2020</year>
          <month>12</month>
          <day>18</day>
          <volume>22</volume>
          <issue>12</issue>
          <fpage>e23518</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://www.jmir.org/2020/12/e23518/"/>
          </comment>
          <pub-id pub-id-type="doi">10.2196/23518</pub-id>
          <pub-id pub-id-type="medline">33156803</pub-id>
          <pub-id pub-id-type="pii">v22i12e23518</pub-id>
          <pub-id pub-id-type="pmcid">PMC7757783</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref28">
        <label>28</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Rabiolo</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Alladio</surname>
              <given-names>E</given-names>
            </name>
            <name name-style="western">
              <surname>Morales</surname>
              <given-names>E</given-names>
            </name>
            <name name-style="western">
              <surname>McNaught</surname>
              <given-names>AI</given-names>
            </name>
            <name name-style="western">
              <surname>Bandello</surname>
              <given-names>F</given-names>
            </name>
            <name name-style="western">
              <surname>Afifi</surname>
              <given-names>AA</given-names>
            </name>
            <name name-style="western">
              <surname>Marchese</surname>
              <given-names>A</given-names>
            </name>
          </person-group>
          <article-title>Forecasting the COVID-19 epidemic by integrating symptom search behavior into predictive models: infoveillance study</article-title>
          <source>J Med Internet Res</source>
          <year>2021</year>
          <month>08</month>
          <day>11</day>
          <volume>23</volume>
          <issue>8</issue>
          <fpage>e28876</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="http://hdl.handle.net/2318/1834032"/>
          </comment>
          <pub-id pub-id-type="doi">10.2196/28876</pub-id>
          <pub-id pub-id-type="medline">34156966</pub-id>
          <pub-id pub-id-type="pii">v23i8e28876</pub-id>
          <pub-id pub-id-type="pmcid">PMC8360333</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref29">
        <label>29</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Hoerger</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Alonzi</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Perry</surname>
              <given-names>LM</given-names>
            </name>
            <name name-style="western">
              <surname>Voss</surname>
              <given-names>HM</given-names>
            </name>
            <name name-style="western">
              <surname>Easwar</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Gerhart</surname>
              <given-names>JI</given-names>
            </name>
          </person-group>
          <article-title>Impact of the COVID-19 pandemic on mental health: Real-time surveillance using Google Trends</article-title>
          <source>Psychol Trauma</source>
          <year>2020</year>
          <month>09</month>
          <volume>12</volume>
          <issue>6</issue>
          <fpage>567</fpage>
          <lpage>568</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://doi.org/10.1037/tra0000872"/>
          </comment>
          <pub-id pub-id-type="doi">10.1037/tra0000872</pub-id>
          <pub-id pub-id-type="medline">32790441</pub-id>
          <pub-id pub-id-type="pii">2020-59192-001</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref30">
        <label>30</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Asseo</surname>
              <given-names>K</given-names>
            </name>
            <name name-style="western">
              <surname>Fierro</surname>
              <given-names>F</given-names>
            </name>
            <name name-style="western">
              <surname>Slavutsky</surname>
              <given-names>Y</given-names>
            </name>
            <name name-style="western">
              <surname>Frasnelli</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Niv</surname>
              <given-names>MY</given-names>
            </name>
          </person-group>
          <article-title>Tracking COVID-19 using taste and smell loss Google searches is not a reliable strategy</article-title>
          <source>Sci Rep</source>
          <year>2020</year>
          <month>11</month>
          <day>25</day>
          <volume>10</volume>
          <issue>1</issue>
          <fpage>20527</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://doi.org/10.1038/s41598-020-77316-3"/>
          </comment>
          <pub-id pub-id-type="doi">10.1038/s41598-020-77316-3</pub-id>
          <pub-id pub-id-type="medline">33239650</pub-id>
          <pub-id pub-id-type="pii">10.1038/s41598-020-77316-3</pub-id>
          <pub-id pub-id-type="pmcid">PMC7689487</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref31">
        <label>31</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Eysenbach</surname>
              <given-names>G</given-names>
            </name>
          </person-group>
          <article-title>Infodemiology: tracking flu-related searches on the web for syndromic surveillance</article-title>
          <source>AMIA Annu Symp Proc</source>
          <year>2006</year>
          <volume>2006</volume>
          <fpage>244</fpage>
          <lpage>8</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://europepmc.org/abstract/MED/17238340"/>
          </comment>
          <pub-id pub-id-type="medline">17238340</pub-id>
          <pub-id pub-id-type="pii">86095</pub-id>
          <pub-id pub-id-type="pmcid">PMC1839505</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref32">
        <label>32</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Althouse</surname>
              <given-names>BM</given-names>
            </name>
            <name name-style="western">
              <surname>Scarpino</surname>
              <given-names>SV</given-names>
            </name>
            <name name-style="western">
              <surname>Meyers</surname>
              <given-names>LA</given-names>
            </name>
            <name name-style="western">
              <surname>Ayers</surname>
              <given-names>JW</given-names>
            </name>
            <name name-style="western">
              <surname>Bargsten</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Baumbach</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Brownstein</surname>
              <given-names>JS</given-names>
            </name>
            <name name-style="western">
              <surname>Castro</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Clapham</surname>
              <given-names>H</given-names>
            </name>
            <name name-style="western">
              <surname>Cummings</surname>
              <given-names>DA</given-names>
            </name>
            <name name-style="western">
              <surname>Del Valle</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Eubank</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Fairchild</surname>
              <given-names>G</given-names>
            </name>
            <name name-style="western">
              <surname>Finelli</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Generous</surname>
              <given-names>N</given-names>
            </name>
            <name name-style="western">
              <surname>George</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Harper</surname>
              <given-names>DR</given-names>
            </name>
            <name name-style="western">
              <surname>Hébert-Dufresne</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Johansson</surname>
              <given-names>MA</given-names>
            </name>
            <name name-style="western">
              <surname>Konty</surname>
              <given-names>K</given-names>
            </name>
            <name name-style="western">
              <surname>Lipsitch</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Milinovich</surname>
              <given-names>G</given-names>
            </name>
            <name name-style="western">
              <surname>Miller</surname>
              <given-names>JD</given-names>
            </name>
            <name name-style="western">
              <surname>Nsoesie</surname>
              <given-names>EO</given-names>
            </name>
            <name name-style="western">
              <surname>Olson</surname>
              <given-names>DR</given-names>
            </name>
            <name name-style="western">
              <surname>Paul</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Polgreen</surname>
              <given-names>PM</given-names>
            </name>
            <name name-style="western">
              <surname>Priedhorsky</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Read</surname>
              <given-names>JM</given-names>
            </name>
            <name name-style="western">
              <surname>Rodríguez-Barraquer</surname>
              <given-names>I</given-names>
            </name>
            <name name-style="western">
              <surname>Smith</surname>
              <given-names>DJ</given-names>
            </name>
            <name name-style="western">
              <surname>Stefansen</surname>
              <given-names>C</given-names>
            </name>
            <name name-style="western">
              <surname>Swerdlow</surname>
              <given-names>DL</given-names>
            </name>
            <name name-style="western">
              <surname>Thompson</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Vespignani</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Wesolowski</surname>
              <given-names>A</given-names>
            </name>
          </person-group>
          <article-title>Enhancing disease surveillance with novel data streams: challenges and opportunities</article-title>
          <source>EPJ Data Sci</source>
          <year>2015</year>
          <month>10</month>
          <day>16</day>
          <volume>4</volume>
          <issue>1</issue>
          <fpage>1</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://europepmc.org/abstract/MED/27990325"/>
          </comment>
          <pub-id pub-id-type="doi">10.1140/epjds/s13688-015-0054-0</pub-id>
          <pub-id pub-id-type="medline">27990325</pub-id>
          <pub-id pub-id-type="pii">17</pub-id>
          <pub-id pub-id-type="pmcid">PMC5156315</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref33">
        <label>33</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Nuti</surname>
              <given-names>SV</given-names>
            </name>
            <name name-style="western">
              <surname>Wayda</surname>
              <given-names>B</given-names>
            </name>
            <name name-style="western">
              <surname>Ranasinghe</surname>
              <given-names>I</given-names>
            </name>
            <name name-style="western">
              <surname>Wang</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Dreyer</surname>
              <given-names>RP</given-names>
            </name>
            <name name-style="western">
              <surname>Chen</surname>
              <given-names>SI</given-names>
            </name>
            <name name-style="western">
              <surname>Murugiah</surname>
              <given-names>K</given-names>
            </name>
          </person-group>
          <article-title>The use of Google Trends in health care research: a systematic review</article-title>
          <source>PLoS One</source>
          <year>2014</year>
          <month>10</month>
          <day>22</day>
          <volume>9</volume>
          <issue>10</issue>
          <fpage>e109583</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://dx.plos.org/10.1371/journal.pone.0109583"/>
          </comment>
          <pub-id pub-id-type="doi">10.1371/journal.pone.0109583</pub-id>
          <pub-id pub-id-type="medline">25337815</pub-id>
          <pub-id pub-id-type="pii">PONE-D-14-22976</pub-id>
          <pub-id pub-id-type="pmcid">PMC4215636</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref34">
        <label>34</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Chan</surname>
              <given-names>EH</given-names>
            </name>
            <name name-style="western">
              <surname>Sahai</surname>
              <given-names>V</given-names>
            </name>
            <name name-style="western">
              <surname>Conrad</surname>
              <given-names>C</given-names>
            </name>
            <name name-style="western">
              <surname>Brownstein</surname>
              <given-names>JS</given-names>
            </name>
          </person-group>
          <article-title>Using web search query data to monitor dengue epidemics: a new model for neglected tropical disease surveillance</article-title>
          <source>PLoS Negl Trop Dis</source>
          <year>2011</year>
          <month>05</month>
          <day>31</day>
          <volume>5</volume>
          <issue>5</issue>
          <fpage>e1206</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://dx.plos.org/10.1371/journal.pntd.0001206"/>
          </comment>
          <pub-id pub-id-type="doi">10.1371/journal.pntd.0001206</pub-id>
          <pub-id pub-id-type="medline">21647308</pub-id>
          <pub-id pub-id-type="pii">PNTD-D-11-00327</pub-id>
          <pub-id pub-id-type="pmcid">PMC3104029</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref35">
        <label>35</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Olson</surname>
              <given-names>DR</given-names>
            </name>
            <name name-style="western">
              <surname>Konty</surname>
              <given-names>KJ</given-names>
            </name>
            <name name-style="western">
              <surname>Paladini</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Viboud</surname>
              <given-names>C</given-names>
            </name>
            <name name-style="western">
              <surname>Simonsen</surname>
              <given-names>L</given-names>
            </name>
          </person-group>
          <article-title>Reassessing Google Flu Trends data for detection of seasonal and pandemic influenza: a comparative epidemiological study at three geographic scales</article-title>
          <source>PLoS Comput Biol</source>
          <year>2013</year>
          <month>10</month>
          <day>17</day>
          <volume>9</volume>
          <issue>10</issue>
          <fpage>e1003256</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://dx.plos.org/10.1371/journal.pcbi.1003256"/>
          </comment>
          <pub-id pub-id-type="doi">10.1371/journal.pcbi.1003256</pub-id>
          <pub-id pub-id-type="medline">24146603</pub-id>
          <pub-id pub-id-type="pii">PCOMPBIOL-D-13-00957</pub-id>
          <pub-id pub-id-type="pmcid">PMC3798275</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref36">
        <label>36</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Jun</surname>
              <given-names>SP</given-names>
            </name>
            <name name-style="western">
              <surname>Yoo</surname>
              <given-names>HS</given-names>
            </name>
            <name name-style="western">
              <surname>Choi</surname>
              <given-names>S</given-names>
            </name>
          </person-group>
          <article-title>Ten years of research change using Google Trends: From the perspective of big data utilizations and applications</article-title>
          <source>Technological Forecasting and Social Change</source>
          <year>2018</year>
          <month>05</month>
          <volume>130</volume>
          <fpage>69</fpage>
          <lpage>87</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://doi.org/10.1016/j.techfore.2017.11.009"/>
          </comment>
          <pub-id pub-id-type="doi">10.1016/j.techfore.2017.11.009</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref37">
        <label>37</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Choi</surname>
              <given-names>HY</given-names>
            </name>
            <name name-style="western">
              <surname>Varian</surname>
              <given-names>H</given-names>
            </name>
          </person-group>
          <article-title>Predicting the present with Google Trends</article-title>
          <source>Economic Record</source>
          <year>2012</year>
          <volume>88</volume>
          <issue>S1</issue>
          <fpage>2</fpage>
          <lpage>9</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://doi.org/10.1111/j.1475-4932.2012.00809.x"/>
          </comment>
          <pub-id pub-id-type="doi">10.1111/j.1475-4932.2012.00809.x</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref38">
        <label>38</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Mavragani</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Ochoa</surname>
              <given-names>G</given-names>
            </name>
          </person-group>
          <article-title>Google Trends in infodemiology and infoveillance: methodology framework</article-title>
          <source>JMIR Public Health Surveill</source>
          <year>2019</year>
          <month>05</month>
          <day>29</day>
          <volume>5</volume>
          <issue>2</issue>
          <fpage>e13439</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://publichealth.jmir.org/2019/2/e13439/"/>
          </comment>
          <pub-id pub-id-type="doi">10.2196/13439</pub-id>
          <pub-id pub-id-type="medline">31144671</pub-id>
          <pub-id pub-id-type="pii">v5i2e13439</pub-id>
          <pub-id pub-id-type="pmcid">PMC6660120</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref39">
        <label>39</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Pelat</surname>
              <given-names>C</given-names>
            </name>
            <name name-style="western">
              <surname>Turbelin</surname>
              <given-names>C</given-names>
            </name>
            <name name-style="western">
              <surname>Bar-Hen</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Flahault</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Valleron</surname>
              <given-names>AJ</given-names>
            </name>
          </person-group>
          <article-title>More diseases tracked by using Google Trends</article-title>
          <source>Emerg Infect Dis</source>
          <year>2009</year>
          <month>08</month>
          <volume>15</volume>
          <issue>8</issue>
          <fpage>1327</fpage>
          <lpage>8</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://europepmc.org/abstract/MED/19751610"/>
          </comment>
          <pub-id pub-id-type="doi">10.3201/eid1508.090299</pub-id>
          <pub-id pub-id-type="medline">19751610</pub-id>
          <pub-id pub-id-type="pmcid">PMC2815981</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref40">
        <label>40</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Effenberger</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Kronbichler</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Shin</surname>
              <given-names>JI</given-names>
            </name>
            <name name-style="western">
              <surname>Mayer</surname>
              <given-names>G</given-names>
            </name>
            <name name-style="western">
              <surname>Tilg</surname>
              <given-names>H</given-names>
            </name>
            <name name-style="western">
              <surname>Perco</surname>
              <given-names>P</given-names>
            </name>
          </person-group>
          <article-title>Association of the COVID-19 pandemic with internet search volumes: a Google Trends analysis</article-title>
          <source>Int J Infect Dis</source>
          <year>2020</year>
          <month>06</month>
          <volume>95</volume>
          <fpage>192</fpage>
          <lpage>197</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://linkinghub.elsevier.com/retrieve/pii/S1201-9712(20)30249-6"/>
          </comment>
          <pub-id pub-id-type="doi">10.1016/j.ijid.2020.04.033</pub-id>
          <pub-id pub-id-type="medline">32305520</pub-id>
          <pub-id pub-id-type="pii">S1201-9712(20)30249-6</pub-id>
          <pub-id pub-id-type="pmcid">PMC7162745</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref41">
        <label>41</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Barber</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Pope</surname>
              <given-names>JC</given-names>
            </name>
          </person-group>
          <article-title>Does party trump ideology? Disentangling party and ideology in America</article-title>
          <source>Am Polit Sci Rev</source>
          <year>2018</year>
          <month>12</month>
          <day>18</day>
          <volume>113</volume>
          <issue>1</issue>
          <fpage>38</fpage>
          <lpage>54</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://doi.org/10.1017/S0003055418000795"/>
          </comment>
          <pub-id pub-id-type="doi">10.1017/s0003055418000795</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref42">
        <label>42</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Agadjanian</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Lacy</surname>
              <given-names>D</given-names>
            </name>
          </person-group>
          <article-title>Changing votes, changing identities? Racial fluidity and vote switching in the 2012–2016 US Presidential Elections</article-title>
          <source>Public Opinion Quarterly</source>
          <year>2021</year>
          <volume>85</volume>
          <issue>3</issue>
          <fpage>737</fpage>
          <lpage>752</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://academic.oup.com/poq/article-abstract/85/3/737/6414558"/>
          </comment>
          <pub-id pub-id-type="doi">10.1093/poq/nfab045</pub-id>
        </nlm-citation>
      </ref>
    </ref-list>
  </back>
</article>
