This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.
Crowdsourcing is a useful way to rapidly collect information on COVID-19 symptoms. However, there are potential biases and data quality issues given the population that chooses to participate in crowdsourcing activities and the common strategies used to screen participants based on their previous experience.
The study aimed to (1) build a pipeline to enable data quality and population representation checks in a pilot setting prior to deploying a final survey to a crowdsourcing platform, (2) assess COVID-19 symptomology among survey respondents who report a previous positive COVID-19 result, and (3) assess associations of symptomology groups and underlying chronic conditions with adverse outcomes due to COVID-19.
We developed a web-based survey and hosted it on the Amazon Mechanical Turk (MTurk) crowdsourcing platform. We conducted a pilot study from August 5, 2020, to August 14, 2020, to refine the filtering criteria according to our needs before finalizing the pipeline. The final survey was posted from late August to December 31, 2020. Hierarchical cluster analyses were performed to identify COVID-19 symptomology groups, and logistic regression analyses were performed for hospitalization and mechanical ventilation outcomes. Finally, we performed a validation of study outcomes by comparing our findings to those reported in previous systematic reviews.
The crowdsourcing pipeline facilitated piloting our survey study and revising the filtering criteria to target specific MTurk experience levels and to include a second attention check. We collected data from 1254 COVID-19–positive survey participants and identified the following 6 symptomology groups: abdominal and bladder pain (Group 1); flu-like symptoms (loss of smell/taste/appetite; Group 2); hoarseness and sputum production (Group 3); joint aches and stomach cramps (Group 4); eye or skin dryness and vomiting (Group 5); and no symptoms (Group 6). The risk factors for adverse COVID-19 outcomes differed for different symptomology groups. The only risk factor that remained significant across 4 symptomology groups was influenza vaccine in the previous year (Group 1: odds ratio [OR] 6.22, 95% CI 2.32-17.92; Group 2: OR 2.35, 95% CI 1.74-3.18; Group 3: OR 3.7, 95% CI 1.32-10.98; Group 4: OR 4.44, 95% CI 1.53-14.49). Our findings regarding the symptoms of abdominal pain, cough, fever, fatigue, shortness of breath, and vomiting as risk factors for COVID-19 adverse outcomes were concordant with the findings of other researchers. Some high-risk symptoms found in our study, including bladder pain, dry eyes or skin, and loss of appetite, were reported less frequently by other researchers and were not considered previously in relation to COVID-19 adverse outcomes.
We demonstrated that a crowdsourced approach was effective for collecting data to assess symptomology associated with COVID-19. Such a strategy may facilitate efficient assessments in a dynamic intersection between emerging infectious diseases, and societal and environmental changes.
COVID-19 represents a global public health concern [
To understand and predict the adverse health outcomes in patients affected by COVID-19, many scientific efforts studying sociodemographic, clinical, and symptomatic risk factors are underway. Findings from those efforts, however, are not all consistent, with conflicting evidence on the risk factors associated with adverse COVID-19 outcomes [
A crowdsourcing model is a useful way to rapidly collect information in the context of the COVID-19 pandemic [
To eliminate substandard crowd data submissions, we used a “crowdsourcing via a broker” strategy with broker services that allowed for filtering participants and their responses, and testing data quality before finalizing the crowdsourcing data collection strategy. We chose to use the Amazon Mechanical Turk (MTurk) crowdsourcing platform that provides filtering mechanisms via setting qualifications [
In this paper, we describe (1) a pipeline to enable data quality and population representation checks in a pilot setting prior to deploying the final survey to MTurk workers, (2) an assessment of COVID-19 symptomology among MTurk worker survey respondents who reported a previous positive COVID-19 result, and (3) an assessment of the associations of symptomology groups and underling chronic conditions with adverse outcomes due to COVID-19.
This was a cross-sectional study. We developed 2 web-based surveys using Qualtrics. One survey was for individuals (ie, individual survey) who indicated a self-reported positive test for COVID-19, and another survey was for individuals whose relatives (ie, family survey), living in the same house, tested positive for COVID-19. We hosted both surveys on MTurk between August and December 2020. To improve the quality of data collection through MTurk and to make our study sample more representative of the target population, we followed the best practices suggested by Young et al [
A few restrictions were implemented to exclude certain survey responses from the final data analysis. First, only those participants who provided an existing COVID-19 test type (nasal/throat/blood/sputum) answer in response to our screening question could continue with the survey. Second, participants could fill the survey only once for themselves (individual survey) and for 1 family member (family survey). Third, a quality control question was included during the questionnaire, which stated, “Do not answer this question (Please click NEXT to go to the next question).” If the question was answered, the survey responses were excluded from the data analyses. Fourth, in the family survey, we asked the participants about their confidence level in their responses regarding their family member. Responses with low reported confidence were excluded from the final analyses.
This study was judged as imposing only minimal risks on participants and was determined to be exempt research by the Johns Hopkins University (IRB00248053) Institutional Review Board.
The web-based surveys had 5 blocks as follows (
Survey measures were from the Johns Hopkins University COVID-19 community response survey guidance toolkit that draws from multiple sources [
The inclusion criteria for this study were individuals living in the United States, adults (aged 18 years or older), and MTurk workers with a self-reported positive COVID-19 result. For the family survey, the participants could complete the survey for 1 family member, even if the family member, who lived in the same household, was below 18 years old. Thus, the target population of this study was COVID-19 patients living in the United States and having sufficient skills to use the MTurk platform. The participants were compensated according to a standard minimum wage and our estimate of completion time (about 5-10 min).
Before posting the final survey to MTurk, we conducted a pilot from August 5, 2020, to August 14, 2020, to assess the quality of responses among workers with different levels of experience. The pilot analysis stratified the worker sample into the following 3 experience groups: those who previously completed 100-499 HITs, 500-999 HITs, and 1000+ HITs. First, a worker would complete a qualification test asking them to verify that they or a family member tested positive for COVID-19. If qualified, the worker could then start the MTurk HIT that included a link to the 26-question web-based Qualtrics survey. The first part of the survey was a screening question (age ≥18 years) and comprehension check. In response to the comprehension check, if a worker selected an invalid COVID-19 test type (eg, urine test), they could not continue the survey.
For those passing the screening test, responses were labeled as “high quality” according to the following criteria: sufficient time taken (threshold of more than 60 s); matching codes and IDs between Qualtrics and MTurk; each code being associated with only 1 worker; and worker had not taken the survey previously (ie, nonduplicate response). A worker’s response was included in the “high-quality” group if they passed all of these criteria.
Separately, we assessed “nonduplicate responses.” A nonduplicate response indicates that the respondent completed the survey only once. This criterion was considered under the assumption that workers who attempted to complete the survey multiple times to receive more compensation did not read through survey instructions carefully, and thus, they may provide lower quality responses than those who attempted to complete the survey once.
The general characteristics of age, sex, race, education, and income were extracted and compared among experience groups. Chi-square analysis was conducted to evaluate if there was a significant difference between experience groups in the number of high-quality and nonduplicate responses. Findings from this analysis were used to refine our filtering criteria in the final crowdsourcing pipeline.
We assessed the following 2 primary adverse outcomes related to COVID-19: hospital admission due to COVID-19 and use of mechanical ventilation during admission.
We used descriptive statistics to characterize the total cohort of participants. Bivariate analyses, using Pearson χ2 tests, were performed to assess differences in participant characteristics between those hospitalized and those not hospitalized, and between those who needed mechanical ventilation during admission and those who did not need mechanical ventilation. We then fitted multivariate logistic regression models to identify the association of COVID-19 symptoms with hospitalization and mechanical ventilation due to COVID-19, adjusted for sociodemographic characteristics and comorbid conditions. Thereafter, hierarchical cluster analysis was conducted to search for patterns based on COVID-19 symptoms. The similarity measure was cosine similarity, and the linkage method was
To validate our findings, we performed a comparison with existing systematic review or meta-analysis papers that assessed symptoms as risk factors for COVID-19 adverse outcomes. Articles for which the analyses occurred prior to our data collection were selected for comparison.
For each article and this study, individual symptoms were checked for being reported as (1) a significant risk factor for an adverse outcome (“yes”) and (2) a nonsignificant risk factor for an adverse outcome (“no”). We also noted if a symptom was not assessed (“NA”). When synthesizing findings across studies, if we found a statistically significant association between an adverse outcome and a symptom that was not studied by others, we labeled it “New.” If there was agreement between this study and at least one other study in identifying a symptom as a risk factor (significant or nonsignificant), we labeled it “1.” Symptoms we did not assess were labeled “NA.”
Pilot survey data were collected from 259 respondents who passed both the qualification test and the screening questions, and of these, 147 (56.8%) were considered to have “high quality” responses. For the experience groups 100-499, 500-999, and 1000+ HITs, the proportions of high-quality responses were 58% (48/83), 43% (41/95), and 72% (58/81), respectively (
Two modifications were made to our crowdsourcing pipeline following the pilot. First, we included only workers with 500+ prior HITs in our final filtering criteria. Given the differences in nonduplicate responses between groups, we reasoned that for tasks requiring a higher cognitive ability, workers with 500+ HITs may provide more high-quality responses than those with 100-499 HITs. Second, we added an attention check question to the Qualtrics survey (ie, “don’t answer this question”).
Comparison of the approval rates and percentage of quality responses out of approved responses between the different experience groups.
Variable | Experience group | |||
|
100-499 HITsa , n/N (%) | 500-999 HITs, n/N (%) | 1000+ HITs, n/N (%) |
|
High-quality responses | 48/83 (58) | 41/95 (43) | 58/81 (72) | 0.135 |
High-quality and nonduplicate responses | 10/48 (21) | 41/41 (100) | 49/58 (85) | <0.001 |
aHIT: human intelligence task.
After implementing our final crowdsourcing pipeline, we collected data from 930 individual surveys and 1243 family surveys; however, data from 410 individual surveys and 496 family surveys were excluded (late August to December 31, 2020). The reasons for exclusion were completion of the survey previously, noncompletion of the survey, initial screening failure for age or comprehension check, and attention check failure (
Regarding family survey respondents, 68.3% (501/734) provided answers about a first-degree family member, 25.7% (189/734) provided answers about a second-degree family member, and only 6.0% (44/734) provided answers about a third-degree relative. There were no statistically significant differences in characteristics or outcomes between the individual respondents and the persons the respondents completed the family survey for, except for age (
Study Inclusion and Exclusion of Amazon Mechanical Turk Worker Responses.
Over 90% (1159/1254, 92.4%) of the participants were up to 65 years old, and only 1.2% (15/1254) were less than 18 years old. Moreover, 52.0% (652/1254) were male, 81.2% (1018/1254) were white, 79.5% (997/1254) were not Hispanic or Latino, 68.4% (858/1254) had a bachelor’s degree or any postgraduate degree, 14.4% (180/1254) had yearly income of US $75,000 or more, 39.6% (496/1254) were smokers, and 46.8% (587/1254) had an influenza vaccine in the last season (
Overall, 47.6% (597/1254) of participants were hospitalized due to COVID-19. Bivariate analysis showed statistically significant differences between hospitalized and nonhospitalized COVID-19 participants for most demographic factors, except gender (
From the logistic regression analysis of the total study population (
Overall, 66.8% (399/597) of hospitalized participants were connected to a mechanical ventilator (31.8% of all participants). There were 11 hospitalized participants from the family survey whose mechanical ventilation use was unknown to the survey respondents, and these participants were not included in the subsequent mechanical ventilation analysis. Smoking every day (OR 3.51, 95% CI 1.45-9.1), influenza vaccine in the last season (OR 3.65, 95% CI 2.29-5.89), loss of appetite (OR 2.07, 95% CI 1.09-4.02), tiredness and fatigue (OR 2.36, 95% CI 1.04-5.44), and vomiting (OR 2.68, 95% CI 1.3-5.71) were significantly associated with higher risk for mechanical ventilation (
We identified the following 6 symptomology groups using hierarchical cluster analysis (
The flu-like symptoms group (Group 2) mostly represented the general study population. The abdominal and bladder pain group (Group 1) and the skin or eye dryness group (Group 5) had the highest hospitalization frequencies (153/227, 67.4% and 134/196, 68.4%, respectively). Both groups were characterized by a lower chance of high income (19/227, 8.4% and 21/196, 10.7%, respectively), more smoking (121/227, 53.3% and 102/196, 52.0%, respectively), and more influenza vaccinations (144/227, 63.4% and 102/196, 52.0%, respectively). The group with abdominal and bladder pain symptoms (Group 1) had higher proportions of Hispanic participants (82/227, 36.1%), asthma patients (64/227, 28.2%), alcohol disorder patients (64/227, 28.2%), and anemia patients (54/227, 23.8%). The group with skin or eye dryness (Group 5) had higher proportions of patients with depression (64/196, 32.7%), diabetes (28/196, 14.3%), weight loss (27/196, 13.8%), and ulcers (25/196, 12.8%). The group with joint aches and stomach cramps (Group 4) had lower proportions of hospitalization (65/158, 41.1%) and mechanical ventilation (31/158, 47.7%). Compared with the general study population, the asymptomatic group (Group 6) was younger (age 18-44 years; 70/85, 82.4%), had more males (54/85, 63.5%), had less white participants (60/85, 70.6%), had less Hispanic participants (7/85, 8.2%), had more participants with a high income (18/85, 21.2%), had less smokers (26/85, 30.6%), had less influenza vaccinations reported (30/85, 35.3%), had a higher proportion of participants with no chronic conditions (45/85, 52.9%), and had a very low risk for hospitalization (12/85, 14.1%).
Dendrogram for COVID-19 symptom clusters.
Descriptive characteristics of COVID-19 symptomology groups.
Characteristic | Group 1 (abdominal and bladder pain) (N=227), n (%) | Group 2 (flu-like symptoms) (N=1139), n (%) | Group 3 (hoarseness, sputum production) (N=144), n (%) | Group 4 (joint aches, stomach cramps) (N=158), n (%) | Group 5 (skin or eye dryness) (N=196), n (%) | Group 6 (no symptoms) (N=85), n (%) | |
|
153 (67.4) | 621 (54.5) | 79 (54.9) | 65 (41.1) | 134 (68.4) | 12 (14.1) | |
|
Mechanical ventilation (among hospitalized patients) | 112 (73.2) | 366 (58.9) | 46 (58.2) | 31 (47.7) | 86 (64.2) | 10 (83.3) |
|
|
|
|
|
|
|
|
|
Male gender | 107 (47.1) | 588 (51.6) | 64 (44.4) | 87 (55.1) | 91 (46.4) | 54 (63.5) |
|
Age 18-44 years | 148 (65.2) | 727 (63.8) | 82 (56.9) | 94 (59.5) | 133 (67.9) | 70 (82.4) |
|
Age ≥45 years | 76 (33.5) | 379 (33.3) | 54 (37.5) | 58 (36.7) | 57 (29.1) | 15 (17.6) |
|
White race | 186 (81.9) | 929 (81.6) | 118 (81.9) | 134 (84.8) | 168 (85.7) | 60 (70.6) |
|
Hispanic or Latino ethnicity | 82 (36.1) | 230 (20.2) | 23 (16.0) | 27 (17.1) | 43 (21.9) | 7 (8.2) |
|
US $75,000 or more yearly income | 19 (8.4) | 162 (14.2) | 20 (13.9) | 28 (17.7) | 21 (10.7) | 18 (21.2) |
|
Smoking | 121 (53.3) | 450 (39.5) | 53 (36.8) | 45 (28.5) | 102 (52.0) | 26 (30.6) |
|
Flu vaccination | 144 (63.4) | 533 (46.8) | 62 (43.1) | 69 (43.7) | 102 (52.0) | 30 (35.3) |
|
|||||||
|
Depression | 38 (16.7) | 285 (25.0) | 35 (24.3) | 37 (23.4) | 64 (32.7) | 12 (14.1) |
|
Obesity | 32 (14.1) | 165 (14.5) | 35 (24.3) | 37 (23.4) | 29 (14.8) | 6 (7.1) |
|
Asthma | 64 (28.2) | 156 (13.7) | 29 (20.1) | 19 (12.0) | 29 (14.8) | 6 (7.1) |
|
Alcohol or substance use disorder | 64 (28.2) | 128 (11.2) | 19 (13.2) | 17 (10.8) | 25 (12.8) | 7 (8.2) |
|
Diabetes, uncomplicated | 12 (5.3) | 116 (10.2) | 15 (10.4) | 19 (12.0) | 28 (14.3) | 1 (1.2) |
|
Mental illness | 36 (15.9) | 98 (8.6) | 23 (16.0) | 22 (13.9) | 17 (8.7) | 4 (4.7) |
|
Migraines | 23 (10.1) | 81 (7.1) | 15 (10.4) | 23 (14.6) | 15 (7.7) | 4 (4.7) |
|
Weight loss | 20 (8.8) | 76 (6.7) | 14 (9.7) | 15 (9.5) | 27 (13.8) | 3 (3.5) |
|
Anemia | 54 (23.8) | 70 (6.1) | 12 (8.3) | 13 (8.2) | 22 (11.2) | 3 (3.5) |
|
High cholesterol | 7 (3.1) | 69 (6.1) | 14 (9.7) | 20 (12.7) | 9 (4.6) | 3 (3.5) |
|
Ulcer | 10 (4.4) | 68 (6.0) | 5 (3.5) | 11 (7.0) | 25 (12.8) | 3 (3.5) |
|
No chronic condition | 34 (15.0) | 264 (23.2) | 23 (16.0) | 31 (19.6) | 29 (14.8) | 45 (52.9) |
Our findings from the logistic regression models, using symptomology groups as risk factors for adverse COVID-19 outcomes and adjusted for all sociodemographic characteristics and comorbid conditions, showed the following 3 groups associated with hospitalization: abdominal and bladder pain group (Group 1; OR 1.5, 95% CI 1.01-2.34); flu-like symptoms group (Group 2; OR 3.33, 95% CI 1.97-5.79); and skin or eye dryness group (Group 5; OR 1.63, 95% CI 1.07-2.52). No symptomology group was associated with a high risk for mechanical ventilation (
Associations between COVID-19 symptomology groups and adverse COVID-19 outcomes.
Symptomology groupa | Hospitalizationb | Mechanical ventilationb | ||||||
ORc | 95% CI | OR | 95% CI |
|
||||
Abdominal and bladder pain group (Group 1) | 1.54 | 1.01-2.34 | .04 | 1.12 | 0.66-1.92 | .68 |
|
|
Flu-like symptoms group (Group 2) | 3.33 | 1.97-5.79 | <.001 | 0.17 | 0.04-0.54 | .01 |
|
|
Hoarseness and sputum production group (Group 3) | 1.51 | 0.92-2.48 | .10 | 1.09 | 0.57-2.11 | .80 |
|
|
Joint aches and stomach cramps group (Group 4) | 0.56 | 0.35-0.88 | .01 | 0.54 | 0.27-1.07 | .08 |
|
|
Skin or eye dryness group (Group 5) | 1.63 | 1.07-2.52 | .02 | 1.25 | 0.76-2.05 | .39 |
|
aGroup 6 (no symptoms) is excluded.
bMultivariate logistic models adjusted for sociodemographic characteristics and comorbid conditions.
cOR: odds ratio.
Finally, we developed 5 logistic regression models for symptomology groups to compare the risk factors for COVID-19 hospitalization among those groups (asymptomatic participants were excluded from this analysis). The results of those models are presented as a forest plot of significant variables in at least one symptomology group (
Risk factors for hospitalization among individuals in different symptomology groups.
A comparison of our findings with those of other studies can be found in
We found agreement between this study and previous studies for 18 symptoms, 6 of which were associated with adverse outcomes (abdominal pain, cough, dyspnea/shortness of breath, fever, fatigue, and vomiting). In addition, we assessed 14 symptoms that were not previously studied by others, 6 of which were associated with adverse outcomes (bladder pain, dry eyes, dry skin, loss of appetite, seizure, and skin rash).
Our results identified individual symptoms and behaviors associated with COVID-19 adverse outcomes. Among these, some were well-known and some were new. We also identified 6 symptomology groups, with 3 groups showing statistically significant associations with COVID-19 outcomes. Furthermore, the findings of this work increase our understanding of the MTurk population and show that with precautionary measures, high-quality data can be obtained.
Well-known single COVID-19 symptoms identified (ie, abdominal pain, cough, fever, and shortness of breath) were associated with hospitalization [
Our analysis of chronic conditions and associations with COVID-19 adverse outcomes showed that patients with preexisting asthma, diabetes, depression, and bladder problems were at high risk for hospitalization, similar to the findings in previous studies. Although previous studies have shown an increased risk of severe COVID-19 among people with obesity [
When studying behaviors influencing adverse COVID-19 outcomes, like previous studies, we found that smoking increased the risk of severe COVID‐19 outcomes [
In addition to studying individual symptoms and behaviors, this study identified 6 COVID-19 symptomology groups by cluster analysis and assessed their associations with adverse outcomes of the disease. Three symptomology groups (flu-like symptoms, abdominal and bladder pain symptoms, and eye and skin dryness symptoms) were highly associated with a high risk for hospitalization. While the characteristics of respondents in the flu-like symptoms group were similar to the characteristics of the general population, the abdominal and bladder pain group included survey respondents who had lower income, and were more likely to have smoked and to be influenza vaccinated. They also tended to have chronic conditions, such as asthma and anemia, and alcohol disorder. The survey respondents in the eye and skin dryness group were generally older and had a greater possibility of being white. They were also more likely to have smoked and to be influenza vaccinated. This group also had a very high percentage of survey respondents with depression, diabetes, and ulcers.
Characterizing patients according to clusters using artificial intelligence devices and machine learning is a pioneering method in a variety of infectious and noninfectious diseases. The use of scientific methods to identify clusters of patients with similar characteristics and specific disease risks might improve awareness of heterogeneity in symptomology, and may enable targeted interventions to reduce disease severity. Other studies of COVID-19 disease trajectories have been able to identify vulnerable population clusters that could benefit from specific health resources, and have provided insights for public health targets for managing the pandemic [
The percentage of those connected to a mechanical ventilator among hospitalized patients may seem high in our study (61.8%); however, the management of patients hospitalized with COVID-19 has changed considerably over the course of the pandemic. More than half of the study population had been hospitalized, and two-thirds of them were on ventilators. Since the survey was conducted in the first months of the COVID-19 pandemic, many people who got sick with COVID-19 were hospitalized and then connected to a mechanical ventilator. Over time, fewer people with COVID-19 were hospitalized, and among those who were hospitalized, only patients with more severe disease were put on ventilators. Other studies have also shown a high percentage (68%) of ventilator use among hospitalized COVID patients [
This work also showed that with precautionary measures to ensure high-quality data collection, a crowdsourcing model can be used to collect data to characterize symptomology for COVID-19 diagnosis and prognosis. There are many studies assessing health data on MTurk as a source of high-quality and rapidly collected data, and it has demonstrated good reliability [
A major limitation of this study was the self-reported data, which can be less reliable than physiological assessments. Our crowdsourced approach, however, allowed for reaching many participants, which helped mitigate the noise, and the fast data collection process was helpful during this pandemic. In addition, during this pandemic, many risk factors of COVID-19 were discovered through social media and other self-reported surveys [
Our work demonstrated that a crowdsourced approach was effective for collecting data to assess the symptomology associated with COVID-19. Conducting a pilot study to assess data quality and population representation facilitated refining the filtering criteria for our final data collection strategy. We validated our approach by comparing the findings from assessing individual symptoms associated with COVID-19 to those identified by others and found highly concordant results. In our assessment of symptomology groups, we discovered that the bladder pain and skin or eye dryness groups had a high risk of COVID-19 hospitalization. Given these findings, we believe that a crowdsourcing strategy, such as the one proposed here, should be considered by others for quick and cost-effective assessments in a rapidly changing spectrum of infectious diseases, and societal and environmental factors.
Individual survey of COVID-19 symptoms.
Family survey of COVID-19 symptoms.
Comparison of demographic characteristics across all experience groups among approved Amazon Mechanical Turk workers.
Descriptive characteristics of participants in the individual and family surveys.
Demographic characteristics of the study participants.
Chronic condition characteristics of the study participants.
COVID-19 symptom characteristics of the study participants.
Associations between symptoms and adverse COVID-19 outcomes adjusted for sociodemographic factors and chronic conditions (multivariate logistic regression).
Comparison of our findings with those of systematic review and meta-analysis studies regarding the association between COVID-19 symptoms and adverse outcomes.
human intelligence task
Amazon Mechanical Turk
odds ratio
We would like to thank Dr Dina Demner-Fushman (National Library of Medicine, National Institutes of Health) for her helpful feedback. We would also like to thank the Johns Hopkins COVID-19 Research Response Program Community Research Working Group for providing access to the toolkit used by this research team to create a survey that includes harmonized data elements. This group is led by Dr Shruti Mehta (Public Health), Dr Jason Farley (Nursing), and Dr Jacky Jennings (Medicine). This work was supported in part by a Microsoft Investigator Fellowship awarded to COT.
None declared.