This paper is in the following e-collection/theme issue:

Early Results from COVID-19 Studies (281) JMIR Theme Issue: COVID-19 Special Issue (2394) JMH Theme Issue: COVID-19 and Mental Health: Impact and Interventions (374)

Published on in Vol 6, No 8 (2022): August

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/36444, first published .
The Prevalence of Psychotic Symptoms, Violent Ideation, and Disruptive Behavior in a Population With SARS-CoV-2 Infection: Preliminary Study

The Prevalence of Psychotic Symptoms, Violent Ideation, and Disruptive Behavior in a Population With SARS-CoV-2 Infection: Preliminary Study

The Prevalence of Psychotic Symptoms, Violent Ideation, and Disruptive Behavior in a Population With SARS-CoV-2 Infection: Preliminary Study

Authors of this article:

Sumra Bari1 Author Orcid Image ;   Nicole L Vike1 Author Orcid Image ;   Khrystyna Stetsiv1 Author Orcid Image ;   Sean Woodward1 Author Orcid Image ;   Shamal Lalvani2 Author Orcid Image ;   Leandros Stefanopoulos3 Author Orcid Image ;   Byoung Woo Kim1 Author Orcid Image ;   Nicos Maglaveras3 Author Orcid Image ;   Hans C Breiter4 Author Orcid Image ;   Aggelos K Katsaggelos2 Author Orcid Image
Article Authors Cited by (4) Tweetations (18) Metrics

    Original Paper

    1Department of Psychiatry and Behavioral Sciences, Northwestern University, Chicago, IL, United States

    2Department of Electrical and Computer Engineering, Northwestern University, Evanston, IL, United States

    3Laboratory of Medical Informatics, Aristotle University of Thessaloniki, Thessaloniki, Greece

    4Laboratory of Neuroimaging and Genetics, Division of Psychiatric Neuroscience, Massachusetts General Hospital, Charlestown, MA, United States

    *these authors contributed equally

    Corresponding Author:

    Hans C Breiter, MD

    Laboratory of Neuroimaging and Genetics

    Division of Psychiatric Neuroscience, Massachusetts General Hospital

    120 2nd Avenue

    Charlestown, MA, 02129-2060

    United States

    Phone: 1 312 503 0527

    Email: hbreiter@mgh.harvard.edu


    Background: The COVID-19 disease results from infection by the SARS-CoV-2 virus to produce a range of mild to severe physical, neurological, and mental health symptoms. The COVID-19 pandemic has indirectly caused significant emotional distress, triggering the emergence of mental health symptoms in individuals who were not previously affected or exacerbating symptoms in those with existing mental health conditions. Emotional distress and certain mental health conditions can lead to violent ideation and disruptive behavior, including aggression, threatening acts, deliberate harm toward other people or animals, and inattention to or noncompliance with education or workplace rules. Of the many mental health conditions that can be associated with violent ideation and disruptive behavior, psychosis can evidence greater vulnerability to unpredictable changes and being at a greater risk for them. Individuals with psychosis can also be more susceptible to contracting COVID-19 disease.

    Objective: This study aimed to investigate whether violent ideation, disruptive behavior, or psychotic symptoms were more prevalent in a population with COVID-19 and did not precede the pandemic.

    Methods: In this preliminary study, we analyzed questionnaire responses from a population sample (N=366), received between the end of February 2021 and the start of March 2021 (1 year into the COVID-19 pandemic), regarding COVID-19 illness, violent ideation, disruptive behavior, and psychotic symptoms. Using the Wilcoxon rank sum test followed by multiple comparisons correction, we compared the self-reported frequency of these variables for 3 time windows related to the past 1 month, past 1 month to 1 year, and >1 year ago among the distributions of people who answered whether they tested positive or were diagnosed with COVID-19 by a clinician. We also used multivariable logistic regression with iterative resampling to investigate the relationship between these variables occurring >1 year ago (ie, before the pandemic) and the likelihood of contracting COVID-19.

    Results: We observed a significantly higher frequency of self-reported violent ideation, disruptive behavior, and psychotic symptoms, for all 3 time windows of people who tested positive or were diagnosed with COVID-19 by a clinician. Using multivariable logistic regression, we observed 72% to 94% model accuracy for an increased incidence of COVID-19 in participants who reported violent ideation, disruptive behavior, or psychotic symptoms >1 year ago.

    Conclusions: This preliminary study found that people who reported a test or clinician diagnosis of COVID-19 also reported higher frequencies of violent ideation, disruptive behavior, or psychotic symptoms across multiple time windows, indicating that they were not likely to be the result of COVID-19. In parallel, participants who reported these behaviors >1 year ago (ie, before the pandemic) were more likely to be diagnosed with COVID-19, suggesting that violent ideation, disruptive behavior, in addition to psychotic symptoms, were associated with COVID-19 with an approximately 70% to 90% likelihood.

    JMIR Form Res 2022;6(8):e36444

    doi:10.2196/36444

    Keywords

    COVID-19; paranoia; delusions; disruptive behavior; violent ideation; psychotic symptoms; pandemic; mental health; distress; stress; psychological health; psychosis; risk; machine learning 


    Background

    The COVID-19 disease results from infection by the single-stranded RNA virus SARS-CoV-2. Physically, COVID-19 can produce a range of mild symptoms, including fever, cough, shortness of breath, fatigue, muscle aches, headache, and new loss of taste or smell [1], to severe symptoms requiring ventilation in an intensive care unit because of respiratory failure, septic shock, and multiple organ dysfunction [1]. In the context of mental function, COVID-19 has been linked to significant cognitive and attention deficits, brain fog, anxiety, depression, and sleep problems [2,3]—all of which can affect mental health and well-being.

    The COVID-19 pandemic has burdened the mental health of both those infected with SARS-CoV-2 and those who have been living through the pandemic without infection. The pandemic has indirectly caused worldwide emotional distress that has triggered the development of mental conditions in persons not previously affected and exacerbated symptoms in those with existing conditions [4-7]. Numerous COVID-19 research reviews have reported the adverse psychological effects brought on by pandemic-related stressors, including posttraumatic stress symptoms, confusion, and anger [8]. The management of behavioral symptoms in patients with COVID-19 (eg, agitation) has become a unique challenge for health care workers in emergency departments [9]. COVID-19 can be particularly distressing to individuals with pre-existing mental health conditions such as autism spectrum disorders and other neurodevelopmental disorders, leading to more intense and frequent behavior problems, including disruptive behavior [10] or increased externalizing and aggressive behavior [11].

    Many mental health conditions can lead to violent ideation and disruptive behavior (VIDB), which is defined by aggression, threatening acts, or deliberate harm toward other people or animals, as well as inattention and noncompliance in education or workplace settings [12,13]. For instance, people with substance use disorders [14,15], mania [16,17], psychosis [18,19], and personality disorders [20,21] can all evidence VIDB or be at an increased risk for adverse psychosocial outcomes during pandemic-related periods of stress. Individuals with mild to severe symptoms of psychosis (eg, delusions, paranoia, and hallucinations) have been reported to be susceptible to pandemic-related emotional distress [18,19]. The 12-month prevalence of psychosis is 3.89 to 4.03 per 1000 individuals, and the median lifetime prevalence is 7.49 per 1000 individuals [22]. Prior research has shown that people with psychosis are less likely to take precautionary measures such as receiving vaccination or isolating during the influenza pandemic [23], thereby increasing their potential risk for COVID-19 infection. As with autism spectrum disorder, individuals with psychosis can show greater vulnerability to unpredictable changes, such as COVID-19, and are thus at greater risk for VIDB [24,25].

    Objective

    In this preliminary study, we assessed potential relationships among COVID-19 infection, VIDB, and symptoms related to psychosis, which may increase the risk of infection and VIDB. Psychotic symptoms and VIDB increase the potential for engagement in riskier-than-average behaviors, potentially including nonadherence to COVID-19 precautions. Individuals exhibiting VIDB and psychotic symptoms could thus be at a greater risk of contracting and spreading the virus. Although these associations are not directional, any findings between VIDB and psychotic symptoms with COVID-19 would permit hypothesis framing for two scenarios: (1) COVID-19 infection increases the likelihood of psychosis and VIDB or (2) psychotic symptoms and VIDB increase the possibility of COVID-19 infections. Other viral infections, including influenza and HIV, have led to similar hypotheses [26-34]. To do this, we analyzed questionnaire responses regarding COVID-19 history along with historical questions regarding VIDB and psychotic symptoms over 3 time windows in a small but representative internet sample that followed the US Census (ie, 300<n<500). The questionnaire was distributed between the end of February 2021 and the start of March 2021 and was timed to overlap with the onset of the COVID-19 pandemic 1 year earlier. We assessed the frequency of self-reported VIDB and symptoms of psychosis from the past 1 month, 1 month to 1 year ago, and >1 year ago among participants with and without self-reported COVID-19. Given the existing case reports of an increased incidence of VIDB and psychotic symptoms during the pandemic, we hypothesized that participants with COVID-19 would exhibit an increase in VIDB and psychotic symptoms. We also investigated the relationship between these behaviors occurring >1 year ago and the likelihood of contracting SARS-CoV-2 with the hypothesis that people with these behaviors were more likely to experience COVID-19 infection. We used multivariable logistic regression (MVLR) with iterative resampling to investigate how accurately VIDB and psychotic symptoms from before the pandemic discriminated between participants with and without self-reported COVID-19.

    Altogether, the results of this study show that individuals with VIDB and psychotic symptoms have an increased risk for SARS-CoV-2, although long-term longitudinal data will be needed to assess causal relationships.


    Participant Recruitment

    Study participants were recruited by Gold Research Inc from multiple vendors. Gold Research vendors recruit the emails of willing participants in multiple ways. Some are recruited by invitation only from customer databases of large companies in revenue-sharing agreements, some are recruited from social media, some through direct mail, and others sign up voluntarily to participate in research studies in lieu of monetary or other incentives such as coupons for everyday household purchases. During recruitment, all survey respondents also go through a double opt-in process to indicate the types of research studies they would like to participate in, in addition to providing their profiles on different demographic attributes such as age, race, and sex. This information is then used to reflect representation against US Census metrics. In this process, respondents are also asked multiple test questions to screen out those providing random and illogical responses or showing flatline or speeder behavior. In addition to having cohort demographics balanced to meet the demographic criteria established by the US Census, Gold Research also oversampled 15% (7500/50,000) of the sample for mental health conditions. Gold Research reported that >50,000 respondents were contacted for questionnaire completion. They estimated that of the 50,000 participants, >37,500 (75%) either did not respond or said no. Of the remaining 25% (12,500/50,000) of participants who clicked on the survey link, >50% did not complete the questionnaire. Of the ≥6000 participants who completed the survey, those who did not clear the data integrity assessments were omitted to achieve the final number of completed surveys. Participants meeting quality assurance procedures (including completion of the survey) were studied, with a limit of 500-520 participants. All participants provided informed consent following oversight from the Northwestern University Institutional Review Board, and a double opt-in methodology was used for consenting (shown later in the following section).

    This study assessed multiple mental health conditions, including psychotic symptoms, violent ideation, and disruptive behavior. For this paper, we focused only on psychotic symptoms, violent ideation, and disruptive behavior. All participants provided informed consent following oversight from the Northwestern University Institutional Review Board, which reviewed and approved the project procedures, including the use of a double opt-in methodology for consenting. Participants were guaranteed anonymity and confidentiality, and the researchers did not possess any protected health information. Questionnaire responses were collected between the end of February 2021 and the beginning of March 2021, approximately 1 year after the official pandemic declaration in the United States (March 11, 2020) [35] (Figure 1). Data integrity assessment (see the Data Quality Assurance section) reduced the final sample size to 366 participants.

    Participants reported their age, sex, ethnicity, handedness, annual household income, employment status, level of education, and years of schooling. A total of 506 participants (mean age 47, SD 15 years) completed the study by Gold Research Inc. After quality assurance, participants were found to be 57.9% (212/366) female, 68% (249/366) White, 82% (300/366) right-handed, 42.6% (156/366) employed full-time, and 28.7% (105/366) with some college education (mean years of schooling 13, SD 5 years), approximating the national averages for these measures. A complete summary of the demographic variables is provided in Table 1.

    Participants were asked questions related to COVID-19, as shown in Textbox 1. They were asked to report whether they had ever had a positive COVID-19 test (yes or no; test+), whether they were ever diagnosed with COVID-19 by a medical professional (yes or no; diagnosis), and whether a family member or close friend had experienced serious symptoms or died of COVID-19 (yes or no; family). Of the 366 participants, 36 (9.8%) reported a positive COVID-19 test+, and 34 (9.3%) reported a COVID-19 diagnosis; 26 (7.1%) participants answered yes to both the COVID-19 test+ and diagnosis. Of the 366 participants, 95 (26%) reported that a family member or close friend had serious symptoms or had died of COVID-19.

    ‎
    Figure 1. Timeline for survey data collection.
    View this figure
    Table 1. Summary statistics of the demographic variables reported by the participants, including age, gender, ethnicity, handedness, annual household income, employment status, level of education, and years of schooling (N=366).
    VariableValues
    Age (years)

    		Participants, n (%)366 (100)

    		Value, mean (SD; range)46.67 (15.40; 18-70)
    Years of schooling

    		Participants, n (%)366 (100)

    		Value, mean (SD; range)13.29 (5.04; 1-30)
    Sex, n (%)

    		Male153 (41.8)

    		Female212 (57.9)

    		Other1 (0.3)
    Ethnicity, n (%)

    		White249 (68)

    		African American48 (13.1)

    		Hispanic29 (7.9)

    		Asian American or Pacific Islander13 (3.6)

    		Native American or Alaskan Native18 (4.9)

    		Mixed race3 (0.8)

    		Other3 (0.8)

    		Prefer not to answer3 (0.8)
    Handedness, n (%)

    		Right300 (82)

    		Left52 (14.2)

    		Both14 (3.8)
    Level of education, n (%)

    		Some high school6 (1.6)

    		High school graduate95 (26)

    		Some college105 (28.7)

    		Bachelor degree83 (22.7)

    		Some graduate school12 (3.3)

    		Graduate degree27 (7.4)

    		Postgraduate or doctorate38 (10.4)
    Employment status, n (%)

    		Unemployed58 (15.8)

    		Full-time156 (42.6)

    		Part-time41 (11.2)

    		Self-employed24 (6.6)

    		>1 job3 (0.8)

    		Retired61 (16.7)

    		Other23 (6.3)
    Annual household income (US $), n (%)

    		<25,00086 (23.5)

    		25,000-50,00092 (25.1)

    		50,000-75,00070 (19.1)

    		75,000-100,00047 (12.8)

    		100,000-150,00037 (10.1)

    		150,000-300,00026 (7.1)

    		>300,0008 (2.2)
    COVID-19–related questions from the survey and their abbreviations used in this study.

    Abbreviations used and COVID-19 questions (yes or no)

    • Test+
      • Have you ever tested positive for COVID-19?
    • Diagnosis
      • Have you ever been diagnosed with COVID-19 by a medical clinician?
    • Family
      • Has anyone in your family or group of friends had serious symptoms or died of COVID-19?
    Textbox 1. COVID-19–related questions from the survey and their abbreviations used in this study.

    Power analysis for a 2-sample Wilcoxon rank sum test revealed an estimated power of >0.80 for the experimental conditions based on test+ (sample size for test+ was 36 and sample size for no test+ was 330; α=.05) and an estimated power >0.9 for experimental conditions based on diagnosis (sample size for diagnosis was 34 and sample size for no diagnosis was 332; α=.05). This power suggests that we had statistically adequate sample sizes for the analysis that follows.

    For initial recruitment, participants received the following communication:

    Gold Research Inc., a national market research firm and its client, Northwestern University, request your participation in this study of emotional health. We will be evaluating how different emotions and experiences are connected and may relate to our emotional health. The information you provide will be kept confidential, coded to be anonymous so it cannot be connected back to you and will be used only for research purposes. Researchers will not be able to contact you or restudy you after this survey. We will not share your information with any other third party. We will also not use your information to identify you individually or use your responses to market or sell other services or products to you. As part of this effort, you will not be asked to provide any personal identifiers such as your name, email, phone number, address or social media handles. A unique identifier will be generated for you and each survey participant to enhance privacy. As part of the survey process, we will be able to tell if you completed the survey, but we will not be able to tell which answers were yours. For this study, we are going to ask you some questions about yourself and how much you like or dislike a set of pictures. You may discontinue this study at any time. We appreciate your help with this study, given the serious challenges facing many people regarding emotional health at this time. We thank you in advance.
    1. Accept
    2. Decline

    If participants responded with “Accept,” they were sent a further communication with the following:

    Thank you for participating in our survey. All responses during this survey are anonymous and confidential. We will be able to tell if you completed the survey, but we will not be able to tell which answers were yours. In this study, we aim to understand how different emotions and experiences relate to visual processing.
    We are going to:
    *Ask you some questions about yourself
    *Have you rate how much you like or dislike a set of pictures
    For this study, your identity is protected and your answers are anonymous and confidential. Press “Next” to proceed.

    The survey would then begin if participants pressed “Next.”

    Ethics Approval

    Participants were offered notice that Gold Research administered an emotional health questionnaire on behalf of Northwestern University, with the following phrasing: “We will be evaluating how different emotions and experiences are connected and may relate to our emotional health.” The complete text related to the solicitation, study description, and opt-in procedures can be found in the Methods under Participant Recruitment. All participants provided informed consent following oversight from the Northwestern University Institutional Review Board (approval number STU00213665), which reviewed and approved the project proposal. Participants were guaranteed anonymity and confidentiality, and the researchers did not possess any protected health information.

    Survey Questions and Scoring

    The survey comprised several blocks of questions from existing questionnaires on depression, anxiety, suicidality, addiction, psychosis, VIDB, and COVID-19 infection, in addition to demographic and historical diagnoses of mental health disorders. A picture-rating task was also administered; however, it is not the focus of this study. This study assessed the relationships among psychotic symptoms, VIDB, and COVID-19 infection. Questions about VIDB and 4 positive psychotic symptoms were taken from the behavioral neurology screening questions in the Massachusetts General Hospital Subjective Question (MGH SQ) Screener from the Phenotype Genotype Project in Addiction and Mood Disorder [36]. The MGH SQ has been used in several studies [37-43]. Of the 13 questions used here, 4 were related to positive psychotic symptoms (auditory and visual hallucinations, paranoia, and delusions), which had originally been adapted to the MGH SQ from a clinical textbook on emergency psychiatry (Textbox 2, Psych1-Psych4) [44]. A total of 9 other questions were related to VIDB (eg, wanting to hurt others, prior attempts to hurt others or animals, wanting to start fires, being disruptive, or breaking rules) and were similarly adapted to the MGH SQ using the same clinical textbook on emergency psychiatry (Textbox 2, Disruption1-Disruption9) [44]. Participants rated the questions based on how often they experienced these behaviors as follows: (1) past ≥1 year (long term), (2) past 1 month to 1 year (medium term), and (3) past 1 month (short term; Figure 1 provides the timeline) on a 1 to 7 Likert scale (1=never; 2-3=rarely; 4-5=sometimes; 6=often; 7=always; Figure 2). The data were collected regarding psychotic symptoms and VIDB from the past 1 month (short term) and between 1 month and 1 year (medium term) to assess and relate these symptoms and behaviors during the recent past to COVID-19 infection. We chose a 1-month cutoff to obtain the most recent history, similar to what is done for timeline follow-back methods with substance use or menstrual cycle assessments. The data from >1 year ago (long term) were collected to assess these behaviors before the pandemic and to determine whether a history of psychosis and VIDB before the pandemic influenced a person’s likelihood of COVID-19 infection. This would provide insight into the participants’ current state and how COVID-19 infection or the pandemic, in general, has affected the general population. All 13 questions are listed in Textbox 2 as they appeared in the survey.

    The 13 questions from the larger survey related to psychosis and violent ideation and disruptive behavior that was used in this study for 3 time windows (long term, medium term, and short term) for which participants answered each question.

    Questions and time windows

    • Psych1-LT
      • Hallucinations/Hearing voices others cannot past ≥1 year
    • Psych1-MT
      • Hallucinations/Hearing voices others cannot past 1 month to 1 year
    • Psych1-ST
      • Hallucinations/Hearing voices others cannot past 1 month
    • Psych2-LT
      • Hallucinations/Seeing things others cannot see past ≥1 year
    • Psych 2-MT
      • Hallucinations/Seeing things others cannot see past 1 month to 1 year
    • Psych2-ST
      • Hallucinations/Seeing things others cannot see past 1 month
    • Psych3-LT
      • Worries that others are out to get you or to get people close to you (these might resemble paranoia) past ≥1 year
    • Psych 3-MT
      • Worries that others are out to get you or to get people close to you (these might resemble paranoia) past 1 month to 1 year
    • Psych3-ST
      • Worries that others are out to get you or to get people close to you (these might resemble paranoia) past 1 month
    • Psych4-LT
      • Having one or more unique beliefs or impressions that are strong despite being contradicted by others (these might resemble delusions) past ≥1 year
    • Psych 4-MT
      • Having one or more unique belief or impression that is strong despite being contradicted by others (these might resemble delusions) past 1 month to 1 year
    • Psych4-ST
      • Having one or more unique belief or impression that is strong despite being contradicted by others (these might resemble delusions) past 1 month
    • Disruption1-LT
      • Wanting to hurt others past ≥1 year
    • Disruption 1-MT
      • Wanting to hurt others past 1 month to 1 year
    • Disruption1-ST
      • Wanting to hurt others past 1 month
    • Disruption2-LT
      • Prior attempts at hurting others past ≥1 year
    • Disruption 2-MT
      • Prior attempts at hurting others past 1 year to 1 month
    • Disruption2-ST
      • Prior attempts at hurting others past 1 month
    • Disruption3-LT
      • Having a plan for not hurting others when these feelings arise past ≥1 year
    • Disruption3-MT
      • Having a plan for not hurting others when these feelings arise past 1 year to 1 month
    • Disruption3-ST
      • Having a plan for not hurting others when these feelings arise past 1 month
    • Disruption4-LT
      • Prior attempts at hurting insects or small animals past ≥1 year
    • Disruption4-MT
      • Prior attempts at hurting insects or small animals past 1 year to 1 month
    • Disruption4-ST
      • Prior attempts at hurting insects or small animals past 1 month
    • Disruption5-LT
      • Intrusive thoughts that lead you to repetitive actions past ≥1 year
    • Disruption5-MT
      • Intrusive thoughts that lead you to repetitive actions past 1 year to 1 month
    • Disruption5-ST
      • Intrusive thoughts that lead you to repetitive actions past 1 month
    • Disruption6-LT
      • Desire to start fires past ≥1 year
    • Disruption6-MT
      • Desire to start fires past 1 year to 1 month
    • Disruption6-ST
      • Desire to start fires past 1 month
    • Disruption7-LT
      • Being disruptive in a social environment (eg, at school or elsewhere) past ≥1 year
    • Disruption7-MT
      • Being disruptive in a social environment (eg, at school or elsewhere) past 1 year to 1 month
    • Disruption7-ST
      • Being disruptive in a social environment (eg, at school or elsewhere) past 1 month
    • Disruption8-LT
      • Attention problems past ≥1 year
    • Disruption8-MT
      • Attention problems past 1 year to 1 month
    • Disruption8-ST
      • Attention problems past 1 month
    • Disruption9-LT
      • Breaking rules at school or elsewhere past ≥1 year
    • Disruption9-MT
      • Breaking rules at school or elsewhere past 1 year to 1 month
    • Disruption9-ST
      • Breaking rules at school or elsewhere past 1 month
    Textbox 2. The 13 questions from the larger survey related to psychosis and violent ideation and disruptive behavior that was used in this study for 3 time windows (long term, medium term, and short term) for which participants answered each question.
    ‎
    Figure 2. The Likert scale on which participants rated the questions.
    View this figure

    Data Quality Assurance

    Data quality used 4 exclusion criteria: (1) participants with the same responses throughout any section of the questionnaire (eg, “1” for all questions), (2) participants indicating they had been diagnosed by a clinician with ≥10 illnesses, (3) participants with minimal variance in a picture-rating task (all pictures were rated the same or varied only by 1 point; data not described here), (4) participants reporting inconsistent education level and years of education and participants who completed the questionnaire in <500 seconds. From these procedures, 366 participants were cleared for statistical analysis.

    Statistical Analyses

    Analysis of Demographics and Survey Questions by Self-reported COVID-19 Infection

    Demographic variables (Table 1) and survey question scores for all questions were divided into 2 distributions based on the yes and no responses from participants for each of the COVID-19 questions: test+, diagnosis, and family. Distributions of the demographic variables and survey question scores were assessed for differences using the Wilcoxon rank sum test [45]. Significant categorical demographic variables were further assessed for distribution equality using the Kolmogorov-Smirnov test (α=.05) [46]. The resulting P values for the questions Psych1-LT, Psych1-MT, Psych1-ST, and Psych2-LT to Psych4-ST were corrected for multiple comparisons using the Benjamini-Hochberg procedure [47] (reported as q[FDR]) for each COVID-19 question. The same procedure was repeated for questions Disruption1-LT to Disruption9-ST. The normalized test statistic from the Wilcoxon rank sum test and q(FDR) are reported. Box plots were generated to display the yes or no distribution for all survey questions and are presented in the Multimedia Appendix 1.

    MVLR and Iterative Resampling: Using Demographics and Survey Questions to Model COVID-19 Likelihood

    MVLR was performed with the goal of modeling COVID-19 likelihood. The aim was to determine how psychotic symptoms and VIDB from before the pandemic made people more vulnerable to COVID-19 infection during the pandemic. The set of past ≥1 year (long term) survey responses for the psychotic symptoms and VIDB, which significantly differed (q[FDR]) between those responding yes/no to test+, diagnosis, and family questions for COVID-19, were used as independent variables. The demographic variables with significant differences across yes or no responses for COVID-19 test+, diagnosis, or family (q[FDR]<0.05; Wilcoxon rank sum test) were used as covariates in the MVLR analyses. The COVID-19 question responses (yes=1 and no=0) were the binary dependent variables. However, the percentage of participants with a positive self-reported COVID-19 test+ or diagnosis (yes=1; 9% to 10%) was much lower than the percentage of those without COVID-19 test+ or diagnosis (no=0; approximately 90%), implying a class imbalance that could lead to model overfitting. To avoid overfitting the MVLR model to the majority class, data in the majority class (ie, participants in the COVID-19 test+ no and diagnosis no groups) were randomly downsampled to match the sample size of those self-reporting COVID-19 (yes=0; 36/366, 9.8% for test+ model, and 34/366, 9.3% for diagnosis model). Downsampling was iterated 1000 times, and MVLR was run at each iteration for the downsampled data to obtain the model accuracy, root mean square error (RMSE), and mean absolute error (MAE) of the model. The model accuracy was computed by comparing the number of times the model correctly determined the binary outcome divided by the size of the downsampled data. The average accuracy, SD of the accuracy, average RMSE, and average MAE across all iterations are reported.


    Overview

    The demographic variables and the frequency of self-reported symptoms of psychosis and VIDB from the 3 time intervals were assessed for participants with and without self-reported COVID-19. For this assessment, we used a Wilcoxon rank sum test followed by multiple comparisons correction. The relationship between these behaviors occurring >1 year ago and the likelihood of contracting COVID-19 was investigated using MVLR with iterative resampling. Namely, we asked whether symptoms from >1 year ago could predict COVID-19 infection.

    Demographic Variables and Survey Question Responses Varied by Self-reported COVID-19

    In summary, of the 366 participants, 36 (9.8%) reported a positive COVID-19 test+, and 34 (9.3%) reported a COVID-19 diagnosis; 26 (7.1%) participants answered yes to both the COVID-19 test+ and diagnosis. Of the 366 participants, 95 (26%) reported that a family member or close friend had serious COVID-19–related symptoms or died of COVID-19. We found that participants who self-reported yes to contracting COVID-19 were middle-aged and belonged to the higher-income and higher education groups. Participants who self-reported yes to contracting COVID-19 also reported a higher frequency of psychotic symptoms and VIDB across all 3 time windows of the past 1 month, 1 month to 1 year ago, >1 year ago.

    Specifically, age and income significantly varied by test+, and age, income, and education level significantly varied by diagnosis (all q[FDR]<0.05; Wilcoxon rank sum test; Table 2). Participants who responded yes to test+ and diagnosis were, on average, younger than those who responded no (Table 3; Figure 3A and 3B). Specifically, middle-aged adults more frequently reported yes to test+ (IQR 25-47, median 37 years) and diagnosis (IQR 25-45, median 37.5 years) than those responding no (IQR 31-59, median 45). Participants responding no to test+ or diagnosis showed a distribution with a higher percentage of low-income levels than those with COVID-19 (P=.004; P=.001; Kolmogorov-Smirnov test; Figure 3C and 3D). Participants responding yes to test+ or diagnosis exhibited a bimodal distribution of education level, whereas the distribution for NO responses was skewed left (P=.04; P=.01; Kolmogorov-Smirnov test; Figure 3E and 3F) indicating a higher percentage of low education levels. None of the demographic variables exhibited significant differences based on family COVID-19 questions.

    Furthermore, the survey questions for which responses significantly varied (q[FDR]<0.05; Wilcoxon rank sum test) by COVID-19 status (test+, diagnosis, and family) have q(FDR) highlighted in Tables 4 and 5. All 4 questions related to psychotic symptoms showed significant differences in medians across the yes and no distributions with respect to test+ and diagnosis for COVID-19 across the three time windows: (1) past ≥1 year (long term), (2) past 1 month to 1 year (medium term), and (3) past 1 month (short term; Table 4). All 9 questions related to VIDB also showed significantly different medians across the yes and no distributions with respect to test+ and diagnosis for COVID-19 across the 3 time windows (Table 5). There were no significant response differences across the 13 psychotic symptoms and VIDB questions relative to the COVID-19 family questions. Box plots based on test+ and diagnosis for yes distribution had higher scores than for those who answered no for all survey questions, as presented in the Multimedia Appendix 1.

    Table 2. List of the demographic variables that significantly differed by COVID-19 questions (Wilcoxon rank sum test)a.
    DemographicCOVID-19 questionsP valueq(FDR)
    AgeTest+b.0060.024b
    AgeDiagnosedb.0040.02b
    EmploymentDiagnosed.040.20
    IncomeTest+b.0040.02 b
    IncomeDiagnosedb<.001<0.001b
    Education levelTest+.020.08
    Education levelDiagnosedb.0090.04b

    aUncorrected P values and q(FDR) are reported.

    bDemographic variables with q(FDR) <.05.

    Table 3. List of demographic variables that significantly differ by Kolmogorov-Smirnov test (P<.05).
    DemographicCOVID-19 questionsP value (yes>no)P value (no>yes)
    IncomeTest+.004>.99
    IncomeDiagnosis.001>.99
    Education levelTest+.64.04
    Education levelDiagnosis.48.01
    ‎
    Figure 3. Box plots for distributions of age for participants who answered yes or no for (A) COVID-19 test+ and (B) COVID-19 diagnosis. Histograms for distributions of annual household income (US dollars) for participants who answered yes or no for (C) COVID-19 test+ and (D) COVID-19 diagnosis. Histograms for distributions of education for participants who answered yes or no for (E) COVID-19 test+ and (F) COVID-19 diagnosis.
    View this figure
    Table 4. Survey questions related to psychosis assessed by COVID-19 status (test+, diagnosis, and family) using the Wilcoxon rank sum test.
    QuestionsCOVID-19 questions

    		Test+DiagnosisFamily

    		Test statisticq(FDR)Test statisticq(FDR)Test statisticq(FDR)
    Hallucinations/Hearing voices others cannot

    		Past ≥1 year3.506.21×10−44.782.33×10−60.420.95

    		Past 1 month to 1 year3.516.21×10−44.931.21×10−60.060.95

    		Past 1 month4.171.42×10−45.567.88×10−80.080.95
    Hallucinations/Seeing things others cannot see

    		Past 1 ≥year3.615.30×10−44.941.21×10−60.110.95

    		Past 1 month to 1 year4.101.42×10−45.921.90×10−80.240.95

    		Past 1 month4.071.42×10−45.961.90×10−80.260.95
    Worries that others are out to get you, or to get people close to you (these might resemble paranoia)

    		Past ≥1 year3.823.25×10−43.841.24×10−41.090.95

    		Past 1 month to 1 year3.291.08×10−34.272.34×10−50.380.95

    		Past 1 month2.785.51×10−33.881.14×10−40.180.95
    Having one or more unique beliefs or impressions that are strong despite being contradicted by others (these might resemble delusions)

    		Past ≥1 year3.446.95×10−45.391.44×10−70.440.95

    		Past 1 month to 1 year3.665.05×10−45.508.99×10−80.670.95

    		Past 1 month4.161.42×10−45.861.90×10−80.460.95
    Table 5. Survey questions related to violent ideation and suicidal behavior assessed by COVID-19 status (test+, diagnosis, and family) using the Wilcoxon rank sum test.
    QuestionsCOVID-19 questions

    		Test+DiagnosisFamily

    		Test statisticq(FDR)Test statisticq(FDR)Test statisticq(FDR)
    Wanting to hurt others

    		Past ≥1 year4.131.62×10−45.333.88×10−70.610.72

    		Past 1 month to 1 year4.341.42×10−45.353.88×10−71.070.67

    		Past 1 month3.803.19×10−44.773.09×10−60.550.72
    Prior attempts at hurting others

    		Past 1 ≥year4.181.58×10−44.901.83×10−60.340.83

    		Past 1 month to 1 year3.773.19×10−44.981.39×10−60.810.67

    		Past 1 month4.012.33×10−45.353.88×10−70.650.72
    Having a plan for not hurting others when these feelings arise

    		Past ≥1 year2.212.84×10−24.262.48×10−51.160.67

    		Past 1 month to 1 year2.766.43×10−34.104.24×10−51.460.55

    		Past 1 month2.756.45×10−34.203.01×10−51.530.55
    Prior attempts at hurting insects or small animals

    		Past ≥1 year3.753.24×10−45.333.88×10−70.580.72

    		Past 1 month to 1 year3.873.15×10−45.473.88×10−70.870.67

    		Past 1 month3.853.15×10−45.127.32×10−71.430.55
    Intrusive thoughts that lead you to repetitive actions

    		Past ≥1 year3.409.65×10−44.693.80×10−60.260.86

    		Past 1 month to 1 year3.311.27×10−34.538.00×10−60.140.92

    		Past 1 month3.823.19×10−45.157.01×10−70.080.93

    		Desire to start fires






    		Past ≥1 year4.281.42×10−45.186.98×10−70.840.67

    		Past 1 month to 1 year4.501.42×10−45.573.88×10−70.830.67

    		Past 1 month4.251.42×10−44.173.29×10−50.630.72
    Being disruptive in a social environment (eg, at school or elsewhere)

    		Past ≥1 year3.221.56×10−34.361.69×10−51.190.67

    		Past 1 month to 1 year3.873.15×10−45.176.98×10−71.720.55

    		Past 1 month3.773.19×10−44.733.53×10−61.780.55
    Attention problem

    		Past ≥1 year3.281.34×10−34.723.61×10−61.400.55

    		Past 1 month to 1 year2.874.83×10−34.213.00×10−51.600.55

    		Past 1 month2.182.92×10−23.761.70×10−41.400.55
    Breaking rules at school or elsewhere

    		Past ≥1 year3.458.36×10−44.822.53×10−61.030.67

    		Past 1 month to 1 year3.634.84×10−44.971.39×10−60.890.67

    		Past 1 month3.565.84×10−45.373.88×10−70.420.79

    MVLR: Using Demographics and Survey Questions to Model COVID-19 Likelihood

    In brief, using MVLR with iterative resampling, we observed a model accuracy of 72% to 94% for an increased incidence of COVID-19 in participants with psychotic symptoms or VIDB >1 year ago (ie, before the pandemic).

    Specifically, MVLR with iterative resampling focused on the responses from the time window of past ≥1 year (long term) for the 13 survey questions that significantly varied by COVID-19 status (Table 4 and Table 5). Responses to these 13 questions were used as primary predictors to model COVID-19 test+ and diagnosis using MVLR. Given that no significant differences were observed in the survey question responses when split by responses to the COVID-19 family question, no attempt was made to model COVID-19 family using MVLR. The significant demographic variables of age, income, and education level were used as covariates in the model. To avoid overfitting of the model to the majority class (ie, participants in COVID-19 test+ no and diagnosis no groups), the majority class was iteratively downsampled 1000 times, and the MVLR model was run each time. The average accuracy, SD of the accuracy, average RMSE, and average MAE across all iterations are reported. The null model with only the survey questions regarding psychosis with responses from >1 year ago (ie, Psych1-LT, Psych2-LT, Psych3-LT, and Psych4-LT) had an average accuracy of 72.08% for test+ and 76.75% for diagnosis. The model with 4 survey questions as predictors along with demographic variables as covariates had an average accuracy of 85.45% for model test+ and 89.43% for model diagnosis (Table 6). Similarly, the null model for 9 survey questions regarding VIDB with responses from more than a year ago (Disruption1-LT, Disruption2-LT..., Disruption9-LT) had average accuracies of 81.32% and 87.53% for test+ and diagnosis, respectively. The 9 survey questions, along with covariates, had an average accuracy of 90.35% for the model test+ and 94.39% for the model diagnosis (Table 6). Additional measures, including the SD of the accuracy, average RMSE, and average MAE for these models are also reported in Table 6.

    Table 6. Multivariable logistic regression resultsa.
    COVID-19 question (dependent variable)Independent variablesCovariatesAccuracy, mean (SD)RMSEb, mean (SD)MAEc, mean (SD)
    Test+Psych1-LT, Psych2-LT, Psych3-LT, and Psych4-LT—d72.08 (3.80)0.53 (0.04)0.28 (0.04)
    Test+Psych1-LT, Psych2-LT, Psych3-LT, and Psych4-LT
    • Age
      • ‎
    • Income
      • ‎
    • Education level
      • ‎
    		85.45 (4.60)0.38 (0.06)0.15 (0.04)
    DiagnosisPsych1-LT, Psych2-LT, Psych3-LT, and Psych4-LT—76.75 (3.67)0.48 (0.04)0.23 (0.04)
    DiagnosisPsych1-LT, Psych2-LT, Psych3-LT, and Psych4-LT
    • Age
      • ‎
    • Income
      • ‎
    • Education level
      • ‎
    		89.43 (4.44)0.32 (0.08)0.11 (0.04)
    Test+Disruption1-LT, Disruption2-LT, Disruption3-LT, Disruption4-LT, Disruption5-LT, Disruption6-LT, Disruption7-LT, Disruption8-LT, and Disruption9-LT—81.32 (1.09)0.43 (0.01)0.19 (0.01)
    Test+Disruption1-LT, Disruption2-LT, Disruption3-LT, Disruption4-LT, Disruption5-LT, Disruption6-LT, Disruption7-LT, Disruption8-LT, and Disruption9-LT
    • Age
      • ‎
    • Income
      • ‎
    • Education level
      • ‎
    		90.35 (3.20)0.31 (0.05)0.10 (0.03)
    DiagnosisDisruption1-LT, Disruption2-LT, Disruption3-LT, Disruption4-LT, Disruption5-LT, Disruption6-LT, Disruption7-LT, Disruption8-LT, and Disruption9-LT—87.53 (1.15)0.35 (0.02)0.12 (0.01)
    DiagnosisDisruption1-LT, Disruption2-LT, Disruption3-LT, Disruption4-LT, Disruption5-LT, Disruption6-LT, Disruption7-LT, Disruption8-LT, and Disruption9-LT
    • Age
      • ‎
    • Income
      • ‎
    • Education level
      • ‎
    		94.39 (3.51)0.22 (0.09)0.06 (0.03)

    aThe table lists the dependent variable, independent variables, and covariates for each multivariable logistic regression model and reports the average and SD of accuracy, RMSE, and MAE.

    bRMSE: root mean square error.

    cMAE: mean absolute error.

    dNo covariates were included in the model.


    Principal Findings

    In a small population sample, we assessed the frequency of self-reported VIDB and psychotic symptoms among people with or proximate to someone with a COVID-19 diagnosis and further tested the potential for discriminating who might become infected by COVID-19 based on a preceding history of these behaviors. For all 3 time windows, we found that participants who answered yes to having COVID-19 (test+ and diagnosis) experienced VIDB and psychotic symptoms more frequently than those who answered no. No significant differences were observed between the yes or no distributions for those who had a close friend or family contract COVID-19 (ie, family). Using MVLR with iterative resampling to avoid model overfitting, we also found that participants who experienced psychotic symptoms and VIDB more frequently before the COVID-19 pandemic were more likely to subsequently contract COVID-19, suggesting that individuals experiencing psychotic symptoms and VIDB may (1) have comorbid conditions that increase their susceptibility to COVID-19 infection and (2) engage in riskier-than-average behaviors, potentially including nonadherence to COVID-19 precautions and engaging with other people.

    The 3 demographic variables (age, income, and educational level) were significantly different when divided into yes or no distributions for COVID-19. Regarding age, COVID-19 was more prevalent in middle-aged individuals, with yes distributions for both COVID-19 infection questions having a median age of 37 years. These results contrast with some reports of COVID-19 incidence, where it was reported that older adults were more likely to be hospitalized because of COVID-19 [48], but support the results of other studies, in which younger to middle-aged people were more likely to contract COVID-19 [49]. It should be noted that this study only asked about the diagnosis or testing positive for COVID-19, which is different from hospitalization.

    Annual household income for participants who answered yes to COVID-19 questions was more equally distributed across all income levels, with a peak at higher-income households (>US $75,000). In comparison, the income distribution of participants who answered no was skewed, with a higher concentration of participants in lower-income households. However, the participants who answered yes were evenly distributed across all income levels. This demonstrates that people who answered no for COVID-19 test+ and diagnosis were predominately from lower-income households and that people who answered yes to test+ and diagnosis were evenly distributed across all household income levels. This suggests that COVID-19 occurred in people with a wide range of income levels and did not preferentially occur in those living in lower- or higher-income households. These findings are in line with those reported in the study by Alsan et al [49] but are in contrast with those in the study by Baena-Diéz et al [50]. This study [50] found that people belonging to lower-income level households had higher incidence rates of contracting COVID-19 infection in Barcelona, Spain. Education had bimodal distributions for participants who answered yes to COVID-19 questions, with a larger peak at higher education levels. For participants who responded no, the distribution was skewed to the left, suggesting that education level did not predispose individuals to COVID-19.

    Responses to survey questions about VIDB and psychotic symptoms exhibited higher scores for participants who answered yes for COVID-19 than for people who responded no. This observation showed similar statistical effects regardless of the time window assessed (ie, in the past 1 month, 1 month to 1 year ago, and >1 year ago; Tables 4 and 5). Given these findings for all time windows in this specific study, including those before the pandemic, our observation of a higher frequency of psychotic episodes is less likely to be a direct consequence of COVID-19, which has been linked to cognitive and attention deficits, brain fog, and anxiety [2]. For the same reason, our findings are less likely to be because of higher doses of steroid treatment, which is known to complicate or trigger psychosis [51]. It should be noted that there have been multiple reports of subacute onset of psychotic episodes in healthy individuals with no prior history of mental illness after treatment for COVID-19 [52-55]. These psychotic episodes also led to aggressive, violent, and disruptive behavior and a preponderance of not following rules [53,55]. The psychological impacts of quarantine, temporary loss of employment, lack of livelihood, and financial insecurity have also been shown to trigger violent and disruptive behavior [8,56]. Owing to feelings of frustration and agitation associated with COVID-19, pandemic-related stressors may manifest as aggression and violence toward household members [57-59]. Fear surrounding the COVID-19 pandemic has also been associated with aggressive behavior on the web [60]. Although this is consistent with our results where participants who contracted SARS-CoV-2 reported higher incidences of psychotic symptoms and VIDB in the past 1 month and 1 month to 1 year, our observation of these symptoms, ideation, and behavior before the pandemic for these participants suggests that COVID-19 infection was not causal in the presentation of VIDB or psychotic symptoms. Our study did not find associations that were directional, namely, whether COVID-19 infection increases the likelihood of psychosis and VIDB or whether psychotic symptoms and VIDB increase the possibility of COVID-19 infection. Other viral infections, including influenza and HIV, have led to similar hypotheses [26-34]. Recently, a study by Douaud et al [61] observed a greater reduction in overall brain size, reduction in gray matter thickness and tissue contrast in the orbitofrontal cortex and parahippocampal gyrus, and changes in markers of tissue damage in regions functionally connected to the primary olfactory cortex in people who tested positive for infection with SARS-CoV-2. Similar brain regions are known to be affected in the same way by psychosis and VIDB [62-66].

    To assess whether self-reported VIDB or psychotic symptoms that occurred >1 year ago (ie, before the pandemic) modeled that individuals subsequently had COVID-19 infection, MVLR was performed with iterative downsampling of the majority class to reduce the possibility of model overfitting (Table 6). The demographic variables of age, income, and education level were used as covariates. The full model was not confounded by covariates as the highest accuracy without covariates (ie, null model) was 87.5%, whereas the highest accuracy for the full model was 94.4%. Across all the models, the accuracy of modeling COVID-19 by diagnosis was higher than that for test+. This might be because of the likelihood of false positives from COVID-19 testing and the fact that COVID-19 testing is an inherent component of clinical diagnosis, making clinical diagnosis more informed. There was also a lag at the beginning of the pandemic before testing was widely available. Overall, the highest accuracy for modeling COVID-19 infection from psychotic symptoms from >1 year ago was 89.4%, and for VIDB, it was 94.4%. These results may reflect comorbid conditions that increase susceptibility to COVID-19 infection or reflect engagement in riskier-than-average behaviors, potentially including nonadherence to COVID-19 precautions, as has been observed with other viral infections [30-32]. Conversely, there is another hypothesis that people with mental health conditions become more antisocial and are thus more distant from other people, potentially resulting in a reduced viral spread. Further work is needed to determine whether psychotic symptoms or VIDB increase the risk for, and the spread of, COVID-19.

    This study has several limitations. First, this study involved a small sample size of participants responding yes to test+ (36/366, 9.8%) and diagnosis (34/366, 9.3%) in our cohort. Although the percentage of studied participants infected with COVID-19 herein was consistent with the population estimates of COVID-19 in the United States at the time of data collection, future work needs to assess larger population samples. Another limitation is the bias arising from the false positives of self-reported COVID-19 test positives. It should be noted that the survey did not collect data when the respondents were infected with COVID-19 in the previous year; this is a limitation in estimating the precise time frame for an increase in behavioral symptoms associated with COVID-19. Recall bias is a well-known caveat for any study that uses survey methods [67]. Furthermore, it should also be noted that MVLR results do not represent true predictions, which would require adequately sized training and test sets, along with cross-validation of prediction outcomes. Future work will need to use larger sample sizes and perform better predictive analyses with more sophisticated machine learning algorithms. These caveats aside, it must be noted that this study used iterative resampling to overcome a major confounder that is common in current machine learning papers with class imbalance in smaller data sets—model overfitting—supporting the generalizability of these results.

    Conclusions

    This preliminary study examined a population sample collected at the end of February 2021 and the beginning of March 2021 and found that people who reported a test and clinician diagnosis of COVID-19 also reported higher frequencies of VIDB or psychotic symptoms across multiple time windows (the past 1 month, 1 month to 1 year ago, and >1 year ago), including those before the pandemic started. We also found that participants who experienced VIDB or psychotic symptoms >1 year ago (ie, before the pandemic) were more likely to contract SARS-CoV-2 or be diagnosed with COVID-19. A greater understanding of how COVID-19 may trigger these mental health issues is needed, given other literature on viral, carcinogenic, and toxic effects in changing mental status, including triggering psychotic symptoms and disruptive behavior [26-29,33,34,68-71]. It is also of utmost importance to understand which vulnerable populations are at a greater risk of contracting and spreading the virus to curb the viral spread, which includes vulnerable populations related to age, sex, and household income.

    Acknowledgments

    Funding for this work was provided by the Office of Naval Research (award N00014-21-1-2216) and the Warren Wright Adolescent Center at Northwestern University, Feinberg School of Medicine.

    Authors' Contributions

    SB, NLV, NM, AKK, and HCB took part in the study concept and design and acquisition of original data. SB performed the coding of statistical tools (with guidance from NLV, AKK, and HCB). SB and NLV performed the analysis of data (with guidance from HCB and AKK). Interpretation of data was conducted by SB, NLV, AKK, and HCB (with input from KS, SW, SL, LS, B-WK, and NM). SB performed the statistical assessment (with input from NLV, AKK, and HCB), authored the original draft (with input from HCB), and generated figures. All authors were involved in the revision of the manuscript for content. All authors approved the final version of this manuscript for submission. Coauthorship is as follows: SB and NLV are joint first authors; KS, SW, LS, LS, BWK, and NM are joint second authors; and HCB and AKK are cosenior authors.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Survey questions and box plots of survey questions for yes or no distributions based on COVID-19 questions.

    DOCX File , 21265 KB
    1. Hu B, Guo H, Zhou P, Shi ZL. Characteristics of SARS-CoV-2 and COVID-19. Nat Rev Microbiol 2021 Mar;19(3):141-154 [FREE Full text] [CrossRef] [Medline]
    2. Boldrini M, Canoll PD, Klein RS. How COVID-19 affects the brain. JAMA Psychiatry 2021 Jun 01;78(6):682-683. [CrossRef] [Medline]
    3. Vanichkachorn G, Newcomb R, Cowl CT, Murad MH, Breeher L, Miller S, et al. Post-COVID-19 syndrome (long haul syndrome): description of a multidisciplinary clinic at mayo clinic and characteristics of the initial patient cohort. Mayo Clin Proc 2021 Jul;96(7):1782-1791 [FREE Full text] [CrossRef] [Medline]
    4. Pfefferbaum B, North CS. Mental health and the Covid-19 pandemic. N Engl J Med 2020 Aug 06;383(6):510-512. [CrossRef] [Medline]
    5. O'Connor RC, Wetherall K, Cleare S, McClelland H, Melson A, Niedzwiedz C, et al. Br J Psychiatry 2021 Jun;218(6):326-333 [FREE Full text] [CrossRef] [Medline]
    6. Usher K, Durkin J, Bhullar N. The COVID-19 pandemic and mental health impacts. Int J Ment Health Nurs 2020 Jun;29(3):315-318 [FREE Full text] [CrossRef] [Medline]
    7. Heitzman J. Impact of COVID-19 pandemic on mental health. Psychiatr Pol 2020 Apr 30;54(2):187-198 [FREE Full text] [CrossRef] [Medline]
    8. Brooks SK, Webster RK, Smith LE, Woodland L, Wessely S, Greenberg N, et al. The psychological impact of quarantine and how to reduce it: rapid review of the evidence. Lancet 2020 Mar 14;395(10227):912-920 [FREE Full text] [CrossRef] [Medline]
    9. Wong AH, Roppolo LP, Chang BP, Yonkers KA, Wilson MP, Powsner S, et al. Management of agitation during the COVID-19 pandemic. West J Emerg Med 2020 May 22;21(4):795-800 [FREE Full text] [CrossRef] [Medline]
    10. Colizzi M, Sironi E, Antonini F, Ciceri ML, Bovo C, Zoccante L. Psychosocial and behavioral impact of COVID-19 in autism spectrum disorder: an online parent survey. Brain Sci 2020 Jun 03;10(6):341 [FREE Full text] [CrossRef] [Medline]
    11. Kawaoka N, Ohashi K, Fukuhara S, Miyachi T, Asai T, Imaeda M, et al. Impact of school closures due to COVID-19 on children with neurodevelopmental disorders in Japan. J Autism Dev Disord 2022 May;52(5):2149-2155 [FREE Full text] [CrossRef] [Medline]
    12. Beck C, Rossby L, Baldwin B. Correlates of disruptive behavior in cognitively impaired elderly nursing home residents. Arch Psychiatr Nurs 1991 Oct;5(5):281-291. [CrossRef] [Medline]
    13. Dobmeier R, Moran J. Dealing with disruptive behavior of adult learners. New Horizons Adult Educ Hum Resour Dev 2008;22(2):29-54. [CrossRef]
    14. McCormick RA, Smith M. Aggression and hostility in substance abusers: the relationship to abuse patterns, coping style, and relapse triggers. Addict Behav 1995;20(5):555-562. [CrossRef] [Medline]
    15. Fosse R, Olsen BM, Bjørkly S. Violence in acute psychiatric patients: a cumulative increased risk with polysubstance abuse? Int J Forensic Ment Health 2013 Jul;12(3):165-171. [CrossRef]
    16. Nielssen OB, Malhi GS, Large MM. Mania, homicide and severe violence. Aust N Z J Psychiatry 2012 Apr;46(4):357-363. [CrossRef] [Medline]
    17. Krakowski M, Volavka J, Brizer D. Psychopathology and violence: a review of literature. Compr Psychiatry 1986;27(2):131-148. [CrossRef] [Medline]
    18. Arciniegas DB. Psychosis. Continuum (Minneap Minn) 2015 Jun;21(3 Behavioral Neurology and Neuropsychiatry):715-736 [FREE Full text] [CrossRef] [Medline]
    19. Vindegaard N, Benros ME. COVID-19 pandemic and mental health consequences: systematic review of the current evidence. Brain Behav Immun 2020 Oct;89:531-542 [FREE Full text] [CrossRef] [Medline]
    20. Reising K, Farrington DP, Ttofi MM, Piquero AR, Coid JW. Childhood risk factors for personality disorder symptoms related to violence. Aggress Violent Behav 2019 Nov;49:101315. [CrossRef]
    21. Gilbert F, Daffern M. Illuminating the relationship between personality disorder and violence: contributions of the General Aggression Model. Psychol Violence 2011;1(3):230-244. [CrossRef]
    22. Moreno-Küstner B, Martín C, Pastor L. Prevalence of psychotic disorders and its association with methodological issues. A systematic review and meta-analyses. PLoS One 2018 Apr 12;13(4):e0195687 [FREE Full text] [CrossRef] [Medline]
    23. Maguire PA, Reay RE, Looi JC. Nothing to sneeze at - uptake of protective measures against an influenza pandemic by people with schizophrenia: willingness and perceived barriers. Australas Psychiatry 2019 Apr;27(2):171-178 [FREE Full text] [CrossRef] [Medline]
    24. Douglas KS, Guy LS, Hart SD. Psychosis as a risk factor for violence to others: a meta-analysis. Psychol Bull 2009 Sep;135(5):679-706. [CrossRef] [Medline]
    25. Witt K, van Dorn R, Fazel S. Risk factors for violence in psychosis: systematic review and meta-regression analysis of 110 studies. PLoS One 2013;8(2):e55942 [FREE Full text] [CrossRef] [Medline]
    26. Steinberg D, Hirsch SR, Marston SD, Reynolds K, Sutton RN. Influenza infection causing manic psychosis. Br J Psychiatry 1972 May;120(558):531-535. [CrossRef] [Medline]
    27. Kulaga SS, Miller CW. Viral respiratory infections and psychosis: a review of the literature and the implications of COVID-19. Neurosci Biobehav Rev 2021 Aug;127:520-530. [CrossRef] [Medline]
    28. Halstead S, Riccio M, Harlow P, Oretti R, Thompson C. Psychosis associated with HIV infection. Br J Psychiatry 1988 Nov;153:618-623. [CrossRef] [Medline]
    29. Becker Y. HSV-1 brain infection by the olfactory nerve route and virus latency and reactivation may cause learning and behavioral deficiencies and violence in children and adults: a point of view. Virus Genes 1995;10(3):217-226. [CrossRef] [Medline]
    30. Brown LK, Danovsky MB, Lourie KJ, DiClemente RJ, Ponton LE. Adolescents with psychiatric disorders and the risk of HIV. J Am Acad Child Adolesc Psychiatry 1997 Nov;36(11):1609-1617. [CrossRef] [Medline]
    31. Morag M, Morag A, Reichenberg A, Lerer B, Yirmiya R. Psychological variables as predictors of rubella antibody titers and fatigue--a prospective, double blind study. J Psychiatr Res 1999;33(5):389-395. [CrossRef] [Medline]
    32. Dantas-Duarte A, Morais-de-Jesus M, Nunes AP, Miranda-Pettersen K, Araújo-de-Freitas L, Netto LR, et al. Risk-taking behavior and impulsivity among HCV-infected patients. Psychiatry Res 2016 Sep 30;243:75-80. [CrossRef] [Medline]
    33. Mellins CA, Brackis-Cott E, Dolezal C, Abrams EJ. Psychiatric disorders in youth with perinatally acquired human immunodeficiency virus infection. Pediatr Infect Dis J 2006 May;25(5):432-437. [CrossRef] [Medline]
    34. Mohammed A, Sheikh TL, Gidado S, Abdus-Salam IA, Adeyemi J, Olayinka A, et al. Psychiatric treatment of a health care worker after infection with Ebola virus in Lagos, Nigeria. Am J Psychiatry 2015 Mar 01;172(3):222-224. [CrossRef] [Medline]
    35. Listings of WHO’s response to COVID-19. World Health Organization. 2020 Jun 29.   URL: https://www.who.int/news/item/29-06-2020-covidtimeline [accessed 2022-01-12]
    36. Viswanathan V, Sheppard JP, Kim BW, Plantz CL, Ying H, Lee MJ, et al. A quantitative relationship between signal detection in attention and approach/avoidance behavior. Front Psychol 2017 Feb 21;8:122 [FREE Full text] [CrossRef] [Medline]
    37. Perlis RH, Holt DJ, Smoller JW, Blood AJ, Lee S, Kim BW, Phenotype Genotype Project on Addiction and Mood Disorders. Association of a polymorphism near CREB1 with differential aversion processing in the insula of healthy participants. Arch Gen Psychiatry 2008 Aug;65(8):882-892 [FREE Full text] [CrossRef] [Medline]
    38. Gasic GP, Smoller JW, Perlis RH, Sun M, Lee S, Kim BW, et al. BDNF, relative preference, and reward circuitry responses to emotional communication. Am J Med Genet B Neuropsychiatr Genet 2009 Sep 05;150B(6):762-781 [FREE Full text] [CrossRef] [Medline]
    39. Makris N, Gasic GP, Kennedy DN, Hodge SM, Kaiser JR, Lee MJ, et al. Cortical thickness abnormalities in cocaine addiction--a reflection of both drug use and a pre-existing disposition to drug abuse? Neuron 2008 Oct 09;60(1):174-188 [FREE Full text] [CrossRef] [Medline]
    40. Perlis RH, Smoller JW, Mysore J, Sun M, Gillis T, Purcell S, et al. Prevalence of incompletely penetrant Huntington's disease alleles among individuals with major depressive disorder. Am J Psychiatry 2010 May;167(5):574-579 [FREE Full text] [CrossRef] [Medline]
    41. Kim BW, Kennedy DN, Lehár J, Lee MJ, Blood AJ, Lee S, Phenotype Genotype Project in Addiction and Mood Disorders (PGP). Recurrent, robust and scalable patterns underlie human approach and avoidance. PLoS One 2010 May 26;5(5):e10613 [FREE Full text] [CrossRef] [Medline]
    42. Blood AJ, Iosifescu DV, Makris N, Perlis RH, Kennedy DN, Dougherty DD, Phenotype Genotype Project on Addiction and Mood Disorders. Microstructural abnormalities in subcortical reward circuitry of subjects with major depressive disorder. PLoS One 2010 Nov 29;5(11):e13945 [FREE Full text] [CrossRef] [Medline]
    43. Viswanathan V, Lee S, Gilman JM, Kim BW, Lee N, Chamberlain L, et al. Age-related striatal BOLD changes without changes in behavioral loss aversion. Front Hum Neurosci 2015 Apr 30;9:176 [FREE Full text] [CrossRef] [Medline]
    44. Hyman SE. Manual of Psychiatric Emergencies. Boston, MA, USA: Little, Brown; 1988.
    45. Wilcoxon F. Individual comparisons by ranking methods. In: Kotz S, Johnson NL, editors. Breakthroughs in Statistics: Methodology and Distribution. New York, NY, USA: Springer; 1992:196-202.
    46. Smirnov NV. On the estimation of the discrepancy between empirical curves of distribution for two independent samples. Moscow Univ Math Bull 1939;2(16):2-4.
    47. Benjamini Y, Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Series B Stat Methodol 1995;57(1):289-300. [CrossRef]
    48. Cummings MJ, Baldwin MR, Abrams D, Jacobson SD, Meyer BJ, Balough EM, et al. Epidemiology, clinical course, and outcomes of critically ill adults with COVID-19 in New York City: a prospective cohort study. Lancet 2020 Jun 06;395(10239):1763-1770 [FREE Full text] [CrossRef] [Medline]
    49. Alsan M, Stantcheva S, Yang D, Cutler D. Disparities in Coronavirus 2019 reported incidence, knowledge, and behavior among US adults. JAMA Netw Open 2020 Jun 01;3(6):e2012403 [FREE Full text] [CrossRef] [Medline]
    50. Baena-Díez JM, Barroso M, Cordeiro-Coelho SI, Díaz JL, Grau M. Impact of COVID-19 outbreak by income: hitting hardest the most deprived. J Public Health (Oxf) 2020 Nov 23;42(4):698-703 [FREE Full text] [CrossRef] [Medline]
    51. Wada K, Yamada N, Sato T, Suzuki H, Miki M, Lee Y, et al. Corticosteroid-induced psychotic and mood disorders: diagnosis defined by DSM-IV and clinical pictures. Psychosomatics 2001;42(6):461-466. [CrossRef] [Medline]
    52. Rentero D, Juanes A, Losada CP, Álvarez S, Parra A, Santana V, et al. New-onset psychosis in COVID-19 pandemic: a case series in Madrid. Psychiatry Res 2020 Aug;290:113097 [FREE Full text] [CrossRef] [Medline]
    53. Correa-Palacio AF, Hernandez-Huerta D, Gómez-Arnau J, Loeck C, Caballero I. Affective psychosis after COVID-19 infection in a previously healthy patient: a case report. Psychiatry Res 2020 Aug;290:113115 [FREE Full text] [CrossRef] [Medline]
    54. Huarcaya-Victoria J, Herrera D, Castillo C. Psychosis in a patient with anxiety related to COVID-19: a case report. Psychiatry Res 2020 Jul;289:113052 [FREE Full text] [CrossRef] [Medline]
    55. Noone R, Cabassa JA, Gardner L, Schwartz B, Alpert JE, Gabbay V. Letter to the Editor: new onset psychosis and mania following COVID-19 infection. J Psychiatr Res 2020 Nov;130:177-179 [FREE Full text] [CrossRef] [Medline]
    56. Tănăsescu G. Aggressive behavior during Covid-19. In: Consciens 2020 Conference Proceedings. 2020 Presented at: ConScienS '20; November 22-23, 2020; Princeton, NJ, USA.
    57. Mazza M, Marano G, Lai C, Janiri L, Sani G. Danger in danger: interpersonal violence during COVID-19 quarantine. Psychiatry Res 2020 Jul;289:113046 [FREE Full text] [CrossRef] [Medline]
    58. Usher K, Bhullar N, Durkin J, Gyamfi N, Jackson D. Family violence and COVID-19: increased vulnerability and reduced options for support. Int J Ment Health Nurs 2020 Aug;29(4):549-552 [FREE Full text] [CrossRef] [Medline]
    59. Boserup B, McKenney M, Elkbuli A. Alarming trends in US domestic violence during the COVID-19 pandemic. Am J Emerg Med 2020 Dec;38(12):2753-2755 [FREE Full text] [CrossRef] [Medline]
    60. Ye B, Zeng Y, Im H, Liu M, Wang X, Yang Q. The relationship between fear of COVID-19 and online aggressive behavior: a moderated mediation model. Front Psychol 2021 Feb 15;12:589615 [FREE Full text] [CrossRef] [Medline]
    61. Douaud G, Lee S, Alfaro-Almagro F, Arthofer C, Wang C, McCarthy P, et al. SARS-CoV-2 is associated with changes in brain structure in UK Biobank. Nature 2022 Apr;604(7907):697-707 [FREE Full text] [CrossRef] [Medline]
    62. Mørch-Johnsen L, Nesvåg R, Faerden A, Haukvik UK, Jørgensen KN, Lange EH, et al. Brain structure abnormalities in first-episode psychosis patients with persistent apathy. Schizophr Res 2015 May;164(1-3):59-64 [FREE Full text] [CrossRef] [Medline]
    63. Padmanabhan JL, Tandon N, Haller CS, Mathew IT, Eack SM, Clementz BA, et al. Correlations between brain structure and symptom dimensions of psychosis in schizophrenia, schizoaffective, and psychotic bipolar I disorders. Schizophr Bull 2015 Jan;41(1):154-162 [FREE Full text] [CrossRef] [Medline]
    64. Ivleva EI, Clementz BA, Dutcher AM, Arnold SJ, Jeon-Slaughter H, Aslan S, et al. Brain structure biomarkers in the psychosis biotypes: findings from the bipolar-schizophrenia network for intermediate phenotypes. Biol Psychiatry 2017 Jul 01;82(1):26-39 [FREE Full text] [CrossRef] [Medline]
    65. Hummer TA, Wang Y, Kronenberger WG, Dunn DW, Mathews VP. The relationship of brain structure to age and executive functioning in adolescent disruptive behavior disorder. Psychiatry Res 2015 Mar 30;231(3):210-217. [CrossRef] [Medline]
    66. Baker RH, Clanton RL, Rogers JC, De Brito SA. Neuroimaging findings in disruptive behavior disorders. CNS Spectr 2015 Aug;20(4):369-381. [CrossRef] [Medline]
    67. Coughlin SS. Recall bias in epidemiologic studies. J Clin Epidemiol 1990;43(1):87-91. [CrossRef] [Medline]
    68. Siblerud R, Mutter J, Moore E, Naumann J, Walach H. A hypothesis and evidence that mercury may be an etiological factor in Alzheimer's disease. Int J Environ Res Public Health 2019 Dec 17;16(24):5152 [FREE Full text] [CrossRef] [Medline]
    69. Genuis SJ. Toxic causes of mental illness are overlooked. Neurotoxicology 2008 Nov;29(6):1147-1149. [CrossRef] [Medline]
    70. Barzman D, Gernert B, Delbello M. Quetiapine for chronic motor tic disorder. Am J Psychiatry 2004 Jul;161(7):1307. [CrossRef] [Medline]
    71. Wall CA, Rummans TA, Aksamit AJ, Krahn LE, Pankratz VS. Psychiatric manifestations of Creutzfeldt-Jakob disease: a 25-year analysis. J Neuropsychiatry Clin Neurosci 2005;17(4):489-495. [CrossRef] [Medline]

    ‎
    MAE: mean absolute error
    MGH SQ: Massachusetts General Hospital Subjective Question
    MVLR: multivariable logistic regression
    RMSE: root mean square error
    VIDB: violent ideation and disruptive behavior

    Edited by A Mavragani; submitted 14.01.22; peer-reviewed by HSJ Chew, J Wang, C Comeras-Chueca; comments to author 16.03.22; revised version received 06.04.22; accepted 14.05.22; published 16.08.22

    Copyright

    ©Sumra Bari, Nicole L Vike, Khrystyna Stetsiv, Sean Woodward, Shamal Lalvani, Leandros Stefanopoulos, Byoung Woo Kim, Nicos Maglaveras, Hans C Breiter, Aggelos K Katsaggelos. Originally published in JMIR Formative Research (https://formative.jmir.org), 16.08.2022.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.