<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Publishing DTD v2.0 20040830//EN" "http://dtd.nlm.nih.gov/publishing/2.0/journalpublishing.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink" article-type="research-article" dtd-version="2.0">
  <front>
    <journal-meta>
      <journal-id journal-id-type="publisher-id">JFR</journal-id>
      <journal-id journal-id-type="nlm-ta">JMIR Form Res</journal-id>
      <journal-title>JMIR Formative Research</journal-title>
      <issn pub-type="epub">2561-326X</issn>
      <publisher>
        <publisher-name>JMIR Publications</publisher-name>
        <publisher-loc>Toronto, Canada</publisher-loc>
      </publisher>
    </journal-meta>
    <article-meta>
      <article-id pub-id-type="publisher-id">v5i7e28738</article-id>
      <article-id pub-id-type="pmid">34283026</article-id>
      <article-id pub-id-type="doi">10.2196/28738</article-id>
      <article-categories>
        <subj-group subj-group-type="heading">
          <subject>Original Paper</subject>
        </subj-group>
        <subj-group subj-group-type="article-type">
          <subject>Original Paper</subject>
        </subj-group>
      </article-categories>
      <title-group>
        <article-title>Studying How Individuals Who Express the Feeling of Loneliness in an Online Loneliness Forum Communicate in a Nonloneliness Forum: Observational Study</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="editor">
          <name>
            <surname>Eysenbach</surname>
            <given-names>Gunther</given-names>
          </name>
        </contrib>
      </contrib-group>
      <contrib-group>
        <contrib contrib-type="reviewer">
          <name>
            <surname>Entwistle</surname>
            <given-names>Charlotte</given-names>
          </name>
        </contrib>
        <contrib contrib-type="reviewer">
          <name>
            <surname>Fiordelli</surname>
            <given-names>Maddalena</given-names>
          </name>
        </contrib>
      </contrib-group>
      <contrib-group>
        <contrib id="contrib1" contrib-type="author" corresp="yes">
          <name name-style="western">
            <surname>Andy</surname>
            <given-names>Anietie</given-names>
          </name>
          <degrees>PhD</degrees>
          <xref rid="aff1" ref-type="aff">1</xref>
          <address>
            <institution>Penn Medicine</institution>
            <addr-line>3400 Civic Blvd</addr-line>
            <addr-line>Philadelphia, PA, </addr-line>
            <country>United States</country>
            <phone>1 202 486 4095</phone>
            <email>andyanietie@gmail.com</email>
          </address>
          <ext-link ext-link-type="orcid">https://orcid.org/0000-0002-7043-3042</ext-link>
        </contrib>
      </contrib-group>
      <aff id="aff1">
        <label>1</label>
        <institution>Penn Medicine</institution>
        <addr-line>Philadelphia, PA</addr-line>
        <country>United States</country>
      </aff>
      <author-notes>
        <corresp>Corresponding Author: Anietie Andy <email>andyanietie@gmail.com</email></corresp>
      </author-notes>
      <pub-date pub-type="collection">
        <month>7</month>
        <year>2021</year>
      </pub-date>
      <pub-date pub-type="epub">
        <day>20</day>
        <month>7</month>
        <year>2021</year>
      </pub-date>
      <volume>5</volume>
      <issue>7</issue>
      <elocation-id>e28738</elocation-id>
      <history>
        <date date-type="received">
          <day>12</day>
          <month>3</month>
          <year>2021</year>
        </date>
        <date date-type="rev-request">
          <day>10</day>
          <month>5</month>
          <year>2021</year>
        </date>
        <date date-type="rev-recd">
          <day>4</day>
          <month>6</month>
          <year>2021</year>
        </date>
        <date date-type="accepted">
          <day>17</day>
          <month>6</month>
          <year>2021</year>
        </date>
      </history>
      <copyright-statement>©Anietie Andy. Originally published in JMIR Formative Research (https://formative.jmir.org), 20.07.2021.</copyright-statement>
      <copyright-year>2021</copyright-year>
      <license license-type="open-access" xlink:href="https://creativecommons.org/licenses/by/4.0/">
        <p>This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.</p>
      </license>
      <self-uri xlink:href="https://formative.jmir.org/2021/7/e28738" xlink:type="simple"/>
      <abstract>
        <sec sec-type="background">
          <title>Background</title>
          <p>Loneliness is a public health concern, and increasingly, individuals experiencing loneliness are seeking support on online forums, some of which focus on discussions around loneliness (loneliness forums). Some of these individuals may also seek support around loneliness on online forums not related to loneliness or well-being (nonloneliness forums). Hence, to design and implement appropriate and efficient online loneliness interventions, it is important to understand how individuals who express and seek support around loneliness on online loneliness forums communicate in nonloneliness forums; this could provide further insights into the support needs and concerns of these users.</p>
        </sec>
        <sec sec-type="objective">
          <title>Objective</title>
          <p>This study aims to explore how users who express the feeling of loneliness and seek support around loneliness on an online loneliness forum communicate in an online nonloneliness forum.</p>
        </sec>
        <sec sec-type="methods">
          <title>Methods</title>
          <p>A total of 2401 users who expressed loneliness in posts published on a loneliness forum on Reddit and had published posts in a nonloneliness forum were identified. Using latent Dirichlet allocation (a natural language processing algorithm); Linguistic Inquiry and Word Count (a psycholinguistic dictionary); and the word score–based language features <italic>valence</italic>, <italic>arousal</italic>, and <italic>dominance</italic>, the language use differences in posts published in the nonloneliness forum by these users compared to a control group of users who did not belong to any loneliness forum on Reddit were determined.</p>
        </sec>
        <sec sec-type="results">
          <title>Results</title>
          <p>It was found that in posts published in the nonloneliness forum, users who expressed loneliness tend to use more words associated with the Linguistic Inquiry and Word Count categories on sadness (Cohen <italic>d</italic>=0.10) and seeking to socialize (Cohen <italic>d</italic>=0.114), and use words associated with <italic>valence</italic> (Cohen <italic>d</italic>=0.364) and <italic>dominance</italic> (Cohen <italic>d</italic>=0.117). In addition, they tend to publish posts related to latent Dirichlet allocation topics such as relationships (Cohen <italic>d</italic>=0.105) and family and friends and mental health (Cohen <italic>d</italic>=0.10).</p>
        </sec>
        <sec sec-type="conclusions">
          <title>Conclusions</title>
          <p>There are clear distinctions in language use in nonloneliness forum posts by users who express loneliness compared to a control group of users. These findings can help with the design and implementation of online interventions around loneliness.</p>
        </sec>
      </abstract>
      <kwd-group>
        <kwd>loneliness</kwd>
        <kwd>Reddit</kwd>
        <kwd>nonloneliness</kwd>
        <kwd>mental health</kwd>
        <kwd>eHealth</kwd>
        <kwd>forum</kwd>
        <kwd>online forum</kwd>
        <kwd>communication</kwd>
        <kwd>natural language processing</kwd>
        <kwd>language</kwd>
        <kwd>linguistics</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec sec-type="introduction">
      <title>Introduction</title>
      <p>Loneliness is a public health challenge [<xref ref-type="bibr" rid="ref1">1</xref>]; it affects the well-being of individuals of all age groups [<xref ref-type="bibr" rid="ref2">2</xref>,<xref ref-type="bibr" rid="ref3">3</xref>] and has been linked to early death [<xref ref-type="bibr" rid="ref4">4</xref>,<xref ref-type="bibr" rid="ref5">5</xref>], depression [<xref ref-type="bibr" rid="ref6">6</xref>,<xref ref-type="bibr" rid="ref7">7</xref>], and heart disease [<xref ref-type="bibr" rid="ref8">8</xref>].</p>
      <p>Several prior works [<xref ref-type="bibr" rid="ref1">1</xref>,<xref ref-type="bibr" rid="ref9">9</xref>,<xref ref-type="bibr" rid="ref10">10</xref>] have analyzed data from social media platforms and online forums, some of which focus on discussions around loneliness (loneliness forums) to understand the support needs of individuals who express loneliness on these platforms. These individuals may also seek support and express concerns as it relates to loneliness, such as how to develop or maintain relationships, on online forums not focused on discussions around loneliness or well-being (nonloneliness forums). To better understand the support needs of users who express loneliness on online loneliness forums and to design and implement appropriate and efficient online interventions, it is important to study how these individuals communicate in nonloneliness forums.</p>
      <p>Using natural language processing methods, prior works determined that the language used in posts published on social media platforms and online forums can be used to gain insights into how users communicate on these forums and the types of support they seek and express, as it relates to their health and well-being. For example, language used in posts published on Facebook was used to train a machine learning model to predict patients risk for cardiovascular disease [<xref ref-type="bibr" rid="ref11">11</xref>]. In addition, language used on online forum posts were used to determine the support needs of users in substance use recovery forums [<xref ref-type="bibr" rid="ref12">12</xref>], a COVID-19 online forum [<xref ref-type="bibr" rid="ref13">13</xref>,<xref ref-type="bibr" rid="ref14">14</xref>], and an online cancer forum [<xref ref-type="bibr" rid="ref15">15</xref>-<xref ref-type="bibr" rid="ref17">17</xref>]. Regarding loneliness, prior works analyzed social media data belonging to individuals who expressed loneliness; for example, Guntuku et al [<xref ref-type="bibr" rid="ref1">1</xref>] analyzed Twitter posts from users who self-declared feeling lonely, and it was determined that the language used by these users in their Twitter posts was more associated with mental health concerns. Kivran-Swaine et al [<xref ref-type="bibr" rid="ref18">18</xref>] determined that posts in which users expressed loneliness tended to receive more responses compared to other posts by the same users. Ruiz et al [<xref ref-type="bibr" rid="ref9">9</xref>] showed that the more loneliness users express on social media, the less online relationships they had. Hunt et al [<xref ref-type="bibr" rid="ref10">10</xref>] showed that there is an association between less social media use and a decrease in the feeling of loneliness and depression.</p>
      <p>Similar to prior works, in this study, using natural language processing methods, language used on posts published on a nonloneliness online forum are analyzed to gain insights into the support needs and concerns of individuals who express loneliness in a loneliness online forum. Potentially, information gleaned from the analyses in this study will provide further insights into the support needs and concerns of individuals expressing loneliness on online loneliness forums, thereby informing loneliness interventions.</p>
    </sec>
    <sec sec-type="methods">
      <title>Methods</title>
      <sec>
        <title>Data and Design</title>
        <p>For the analysis in this study, data from Reddit was used. Reddit is made up of more than 1 million subforums (called subreddits) [<xref ref-type="bibr" rid="ref19">19</xref>] focused on discussions around specific topics such as depression, loneliness, and open-ended questions spanning various topics. In addition, Reddit allows members to join several of these forums; hence, it is possible to get posts published in a nonloneliness forum by users who are members of a loneliness forum. Using Google’s BigQuery [<xref ref-type="bibr" rid="ref20">20</xref>], which is a data warehouse with publicly accessible Reddit data sets published between December 2015 and August 2019, posts from Reddit forums focused on discussions around loneliness were identified by selecting the forums that contained the word “Lonely” in its name (eg, “/r/lonelyheartbeats,” “/r/iAMlonely,” and “/r/Lonely”). It was observed that the forum <italic>/r/Lonely</italic> had more posts and members compared to the other forums, which each had less than 200 published posts during the time period in which the data was collected. Hence, for the analysis in this study, usernames of members of <italic>/r/Lonely</italic> were used; specifically, the usernames of 9956 users who had published a total of 15,012 posts on the <italic>/r/Lonely</italic> forum were selected.</p>
        <p>To identify the other Reddit forums in which these users belong to and tend to publish posts, using the usernames from <italic>/r/Lonely</italic>, all the forums on Reddit were searched to determine the forums in which these <italic>/r/Lonely</italic> users tend to publish posts. It was observed that the forums with the most number of these users as members are <italic>/r/AskReddit</italic> (a forum in which users seek advice and ask open-ended questions on various topics) and <italic>/r/depression</italic> (a forum focused on discussions around depression) with posts by 24% (n=2401) and 20% (n=2031), respectively, of the <italic>/r/Lonely</italic> users (N=9956) in the data set. Since the focus of this study is to determine how individuals who express loneliness on an online loneliness forum communicate in nonloneliness forums (ie, forums not focused on loneliness or well-being), for all the analysis in this study, data from <italic>/r/AskReddit</italic> by 2401 users who expressed loneliness in <italic>/r/Lonely</italic> were used. The author reviewed the posts (N=4001) published on <italic>/r/Lonely</italic> by these users and observed that these users expressed feeling lonely in their <italic>/r/Lonely</italic> posts by stating that they were feeling lonely (eg, rephrased “I am a 25 years old female and I am always lonely”), implied that they were feeling lonely (eg, rephrased “I moved to a new city and I don’t know anyone”), or sought support as it relates to loneliness (eg, rephrased “Where can I find tools online to help with loneliness?”).</p>
        <p>Each of the 2401 users who posted on <italic>/r/Lonely</italic> and had published posts on <italic>/r/AskReddit</italic> were matched with a control group user who had no published posts on any loneliness forum on Reddit and had published posts on <italic>/r/AskReddit</italic> between December 2015 and August 2019.</p>
        <p><xref ref-type="table" rid="table1">Table 1</xref> shows information about the data set.</p>
        <table-wrap position="float" id="table1">
          <label>Table 1</label>
          <caption>
            <p>Summary of /r/AskReddit posts published between December 2015 and August 2019 by /r/Lonely users and a control group.</p>
          </caption>
          <table width="1000" cellpadding="5" cellspacing="0" border="1" rules="groups" frame="hsides">
            <col width="340"/>
            <col width="330"/>
            <col width="330"/>
            <thead>
              <tr valign="top">
                <td>Variables</td>
                <td>/r/Lonely users</td>
                <td>Control group</td>
              </tr>
            </thead>
            <tbody>
              <tr valign="top">
                <td>Users, n</td>
                <td>2401</td>
                <td>2401</td>
              </tr>
              <tr valign="top">
                <td>Posts, n</td>
                <td>25,834</td>
                <td>34,718</td>
              </tr>
            </tbody>
          </table>
        </table-wrap>
        <p>In this study, the following methods were used to determine language use differences in <italic>/r/AskReddit</italic> posts by users who express loneliness compared to the control group: a topic modeling approach, a dictionary-based approach, and a word score–based approach. Cohen <italic>d</italic>, which indicates the standardized difference between means, was used to report the effect sizes. In this study, only results with Cohen <italic>d</italic> greater than or equal to a threshold (ie, 0.10) and that are significant at Bonferroni-corrected <italic>P</italic> values &#60;.001 are reported.</p>
        <p>The topic modeling approach and the dictionary-based approaches were used because prior works used these approaches to gain insights from social media data about the language use differences between individuals in different genders [<xref ref-type="bibr" rid="ref21">21</xref>] and age groups [<xref ref-type="bibr" rid="ref22">22</xref>], and to determine the language use differences between users who express loneliness compared to a control group of users who did not express loneliness [<xref ref-type="bibr" rid="ref1">1</xref>]. The word score–based approach was used because prior work [<xref ref-type="bibr" rid="ref23">23</xref>] used these methods to better understand language features associated with persuasion in online forum posts and comments.</p>
      </sec>
      <sec>
        <title>Topic Modeling Approach</title>
        <p>In this section, the natural language processing topic modeling method latent Dirichlet allocation (LDA) [<xref ref-type="bibr" rid="ref24">24</xref>] was used. LDA works by, first, splitting words in Reddit posts into single words or tokens (tokenization). Second, words that co-occur together are clustered together; the cluster of words are referred to as topics, and based on the content words associated with each topic, a label can be assigned to the topics. For example, LDA could group the words “family,” “mom,” “dad,” “daughter,” and “son” as a reference to family. LDA assumes that the topics consist of a combination of words, and each Reddit post is made up of a combination of topics. Using the Dlatk package [<xref ref-type="bibr" rid="ref25">25</xref>], 20 LDA topics were generated from the <italic>/r/AskReddit</italic> posts associated with <italic>/r/Lonely</italic> users and the control group users; to determine the number of LDA topics, the number of topics varied between 5 and 50 topics by starting with 5 topics and incrementing by 2 topics up to 50 topics. A total of 20 topics had the most coherent topic themes when reviewed by the author. With the generated topics, using the Dlatk package [<xref ref-type="bibr" rid="ref25">25</xref>], the topic themes that frequently occurred in the <italic>/r/AskReddit</italic> posts by <italic>/r/Lonely</italic> users when compared with the control group users were identified.</p>
      </sec>
      <sec>
        <title>Dictionary-Based Approach</title>
        <p>In this approach, language from <italic>/r/AskReddit</italic> posts associated with the <italic>/r/Lonely</italic> users and the control group users were used to determine the prevalence of Linguistic Inquiry and Word Count (LIWC) [<xref ref-type="bibr" rid="ref26">26</xref>] dictionary word categories in posts associated with these groups of users. LIWC is a psycholinguistic dictionary made up of 73 predefined categories such as positive and negative emotions; each of these categories has a curated list of words associated with it. LIWC has been used in several prior works [<xref ref-type="bibr" rid="ref1">1</xref>,<xref ref-type="bibr" rid="ref22">22</xref>,<xref ref-type="bibr" rid="ref27">27</xref>]. Using the Dlatk package [<xref ref-type="bibr" rid="ref25">25</xref>], for each group of <italic>/r/AskReddit</italic> posts (ie, posts belonging to <italic>/r/Lonely</italic> users and the control group users), the proportion of token words associated with LIWC categories were determined.</p>
      </sec>
      <sec>
        <title>Word Score–Based Approach</title>
        <p>The word score–based features v<italic>alence</italic>, <italic>arousal</italic>, and <italic>dominance</italic> have been used by prior works to study communication strategies in an online forum [<xref ref-type="bibr" rid="ref23">23</xref>,<xref ref-type="bibr" rid="ref28">28</xref>]. <italic>Valence</italic> indicates the measure of the positive or negative denotation of a word; for example, “enjoyable” is a high <italic>valence</italic> word and “nightmare” is a low valence word. <italic>Arousal</italic> measures the emotional intensity expressed in a word; an example of a high <italic>arousal</italic> word is “exhilarated,” and an example of a low <italic>arousal</italic> word is “siesta.” Dominance indicates the measure of the locus of control expressed in a word; for example “powerful” is a high dominance word, and “weak” is a low dominance word. Mohammad [<xref ref-type="bibr" rid="ref28">28</xref>] provided a lexicon of human ratings for <italic>valence</italic>, <italic>arousal</italic>, and <italic>dominance</italic> for 20,000 words in English; using this lexicon, for each post in the data set, the average ratings of all content words in each of these word categories (ie, <italic>valence</italic>, <italic>arousal</italic>, and <italic>dominance</italic>) was computed.</p>
      </sec>
    </sec>
    <sec sec-type="results">
      <title>Results</title>
      <sec>
        <title>Topic Modeling Approach</title>
        <p><xref ref-type="table" rid="table2">Table 2</xref> shows the most significant LDA topics in <italic>/r/Askreddit</italic> posts by <italic>/r/Lonely</italic> users compared to the control group.</p>
        <table-wrap position="float" id="table2">
          <label>Table 2</label>
          <caption>
            <p>Results from latent Dirichlet allocation analysis on /r/AskReddit posts by /r/Lonely users compared to the control group users.</p>
          </caption>
          <table width="1000" cellpadding="5" cellspacing="0" border="1" rules="groups" frame="hsides">
            <col width="250"/>
            <col width="450"/>
            <col width="150"/>
            <col width="150"/>
            <thead>
              <tr valign="top">
                <td>Label</td>
                <td>Highly correlated words</td>
                <td>Cohen <italic>d</italic></td>
                <td>Mean (SD)</td>
              </tr>
            </thead>
            <tbody>
              <tr valign="top">
                <td>Relationships</td>
                <td>people, love, hate, relationship, find, can't, lose, meet, stop, married</td>
                <td>0.105</td>
                <td>0.038 (0.048)</td>
              </tr>
              <tr valign="top">
                <td>Family and friends/mental health</td>
                <td>family, friends, deal, talk, experience, depression, mental, care, service, dear</td>
                <td>0.10</td>
                <td>0.038 (0.053)</td>
              </tr>
            </tbody>
          </table>
        </table-wrap>
      </sec>
      <sec>
        <title>Dictionary-Based Approach</title>
        <p><xref ref-type="table" rid="table3">Table 3</xref> shows the different LIWC categories in <italic>/r/AskReddit</italic> posts most associated with <italic>/r/Lonely</italic> users compared to the control group users.</p>
        <table-wrap position="float" id="table3">
          <label>Table 3</label>
          <caption>
            <p>Results from LIWC analysis on /r/AskReddit posts by /r/Lonely users compared to the control group users.</p>
          </caption>
          <table width="1000" cellpadding="5" cellspacing="0" border="1" rules="groups" frame="hsides">
            <col width="370"/>
            <col width="320"/>
            <col width="310"/>
            <thead>
              <tr valign="top">
                <td>LIWC<sup>a</sup> category</td>
                <td>Cohen <italic>d</italic></td>
                <td>Mean (SD)</td>
              </tr>
            </thead>
            <tbody>
              <tr valign="top">
                <td>Social processes</td>
                <td>0.114</td>
                <td>0.157 (0.08)</td>
              </tr>
              <tr valign="top">
                <td>Sadness</td>
                <td>0.10</td>
                <td>0.083 (0.042)</td>
              </tr>
            </tbody>
          </table>
          <table-wrap-foot>
            <fn id="table3fn1">
              <p><sup>a</sup>LIWC: Linguistic Inquiry and Word Count.</p>
            </fn>
          </table-wrap-foot>
        </table-wrap>
      </sec>
      <sec>
        <title>Word Score–Based Approach</title>
        <p>Using Cohen <italic>d</italic>, the effect size between the features that represent the average <italic>valence</italic>, <italic>arousal</italic>, and <italic>dominance</italic> scores for posts in the data set and a feature that represents if a <italic>/r/AskReddit</italic> post was by a <italic>/r/Lonely</italic> user or a control group user was determined, as shown in <xref ref-type="table" rid="table4">Table 4</xref>.</p>
        <table-wrap position="float" id="table4">
          <label>Table 4</label>
          <caption>
            <p>Results from word score–based language feature analysis on /r/AskReddit posts by /r/Lonely users compared to the control group users.</p>
          </caption>
          <table width="1000" cellpadding="5" cellspacing="0" border="1" rules="groups" frame="hsides">
            <col width="500"/>
            <col width="500"/>
            <thead>
              <tr valign="top">
                <td>Attribute</td>
                <td>Cohen <italic>d</italic></td>
              </tr>
            </thead>
            <tbody>
              <tr valign="top">
                <td>Valence</td>
                <td>0.364</td>
              </tr>
              <tr valign="top">
                <td>Arousal</td>
                <td>–0.004</td>
              </tr>
              <tr valign="top">
                <td>Dominance</td>
                <td>0.117</td>
              </tr>
            </tbody>
          </table>
        </table-wrap>
      </sec>
    </sec>
    <sec sec-type="discussion">
      <title>Discussion</title>
      <sec>
        <title>Findings</title>
        <p>Using natural language processing methods, this study shows the distinction in language use in posts published on a nonloneliness forum by users who express and seek support around loneliness in an online loneliness forum compared to a control group of users. These language use differences reflect the support needs and concerns of these users. The findings from this study are summarized in this section.</p>
        <p>This study determined that users who express the feeling of loneliness in <italic>/r/Lonely</italic> tend to seek advice and ask questions about relationships on <italic>/r/AskReddit</italic> (<xref ref-type="table" rid="table2">Table 2</xref>) compared to the control group users. This finding is in line with prior work [<xref ref-type="bibr" rid="ref1">1</xref>] that determined that individuals who expressed loneliness on Twitter tend to publish Twitter messages related to themes about difficult interpersonal relationships.</p>
        <p>In addition, this study has findings that were not in prior work; specifically, it was observed that individuals who express loneliness in a loneliness forum tend to seek advice and ask questions about mental health concerns as it relates to their family members and friends; for example, the following are examples of <italic>/r/AskReddit</italic> posts (rephrased) by <italic>/r/Lonely</italic> users seeking advice as it relates to their relationships with family members and friends:</p>
        <disp-quote>
          <p>I need advice on how to deal with a family member / friend who keeps criticizing me.</p>
        </disp-quote>
        <disp-quote>
          <p>I need help, if one is struggling with mental health, what is the best way to explain it to family members and friends?</p>
        </disp-quote>
        <p>Using LIWC, it was observed that users who expressed loneliness in the loneliness forum tended to use more words associated with sadness and wanting to socialize in the nonloneliness forum.</p>
        <p>Using the word score–based language features v<italic>alence</italic>, <italic>arousal</italic>, and <italic>dominance</italic>, it was determined that the average v<italic>alence</italic> and <italic>dominance</italic> scores in <italic>/r/AskReddit</italic> posts are more associated with posts by users who express loneliness on <italic>/r/Lonely</italic>. A potential explanation for <italic>dominance</italic> being more associated with <italic>/r/AskReddit</italic> posts by <italic>/r/Lonely</italic> users is that some of these users seek support and express vulnerability in these posts; low dominance words suggest vulnerability, hence the association. Additionally, a potential explanation for v<italic>alence</italic> being more associated with <italic>/r/AskReddit</italic> posts by <italic>/r/Lonely</italic> users is that these users tend to use low <italic>valence</italic> words in these posts, hence the association.</p>
        <p>This study shows that users who express loneliness in a loneliness forum seek support and communicate differently from a control group of users in a nonloneliness forum. The findings from this study can aid in the design and implementation of online loneliness interventions; for example, given that users who express loneliness in the loneliness forum ask questions and seek advice as it relates to their relationships (with family and friends) and use more words associated with seeking to socialize and sadness (<xref ref-type="table" rid="table3">Table 3</xref>), online loneliness interventions can provide services in which advice and tips are given to users on how to develop, maintain, and navigate relationships.</p>
        <p>From this study’s findings, when designing and implementing online loneliness interventions, it is important to not only focus on user communication in loneliness forums but also look into how these users communicate in nonloneliness forums.</p>
      </sec>
      <sec>
        <title>Limitations</title>
        <p>In this study, data from Reddit users was used and may not be representative of all individuals (some of whom may not publish posts on online forums expressing their feeling of loneliness) who feel lonely.</p>
      </sec>
      <sec>
        <title>Ethics and Privacy</title>
        <p>The data set used for this study is publicly available. For all the analyses in this study, no user or moderator of any loneliness forum on Reddit (including <italic>/r/Lonely</italic>) was contacted. In addition, besides the usernames of <italic>/r/Lonely</italic> users, no other information from user profiles was used or accessed.</p>
      </sec>
      <sec>
        <title>Conclusion</title>
        <p>In this study, using natural language processing methods<italic>,</italic> it was determined that users who express loneliness in an online loneliness forum communicated differently in a nonloneliness forum when compared to a control group of users. The findings from this study can aid with the design and implementation of online loneliness interventions.</p>
      </sec>
    </sec>
  </body>
  <back>
    <app-group/>
    <glossary>
      <title>Abbreviations</title>
      <def-list>
        <def-item>
          <term id="abb1">LDA</term>
          <def>
            <p>latent Dirichlet allocation</p>
          </def>
        </def-item>
        <def-item>
          <term id="abb2">LIWC</term>
          <def>
            <p>Linguistic Inquiry and Word Count</p>
          </def>
        </def-item>
      </def-list>
    </glossary>
    <fn-group>
      <fn fn-type="conflict">
        <p>None declared.</p>
      </fn>
    </fn-group>
    <ref-list>
      <ref id="ref1">
        <label>1</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Guntuku</surname>
              <given-names>SC</given-names>
            </name>
            <name name-style="western">
              <surname>Schneider</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Pelullo</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Young</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Wong</surname>
              <given-names>V</given-names>
            </name>
            <name name-style="western">
              <surname>Ungar</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Polsky</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Volpp</surname>
              <given-names>KG</given-names>
            </name>
            <name name-style="western">
              <surname>Merchant</surname>
              <given-names>R</given-names>
            </name>
          </person-group>
          <article-title>Studying expressions of loneliness in individuals using Twitter: an observational study</article-title>
          <source>BMJ Open</source>
          <year>2019</year>
          <month>11</month>
          <day>04</day>
          <volume>9</volume>
          <issue>11</issue>
          <fpage>e030355</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://bmjopen.bmj.com/lookup/pmidlookup?view=long&#38;pmid=31685502"/>
          </comment>
          <pub-id pub-id-type="doi">10.1136/bmjopen-2019-030355</pub-id>
          <pub-id pub-id-type="medline">31685502</pub-id>
          <pub-id pub-id-type="pii">bmjopen-2019-030355</pub-id>
          <pub-id pub-id-type="pmcid">PMC6830671</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref2">
        <label>2</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Luhmann</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Hawkley</surname>
              <given-names>LC</given-names>
            </name>
          </person-group>
          <article-title>Age differences in loneliness from late adolescence to oldest old age</article-title>
          <source>Dev Psychol</source>
          <year>2016</year>
          <month>06</month>
          <volume>52</volume>
          <issue>6</issue>
          <fpage>943</fpage>
          <lpage>59</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="http://europepmc.org/abstract/MED/27148782"/>
          </comment>
          <pub-id pub-id-type="doi">10.1037/dev0000117</pub-id>
          <pub-id pub-id-type="medline">27148782</pub-id>
          <pub-id pub-id-type="pii">2016-22442-001</pub-id>
          <pub-id pub-id-type="pmcid">PMC8015413</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref3">
        <label>3</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Beutel</surname>
              <given-names>ME</given-names>
            </name>
            <name name-style="western">
              <surname>Klein</surname>
              <given-names>EM</given-names>
            </name>
            <name name-style="western">
              <surname>Brähler</surname>
              <given-names>E</given-names>
            </name>
            <name name-style="western">
              <surname>Reiner</surname>
              <given-names>I</given-names>
            </name>
            <name name-style="western">
              <surname>Jünger</surname>
              <given-names>C</given-names>
            </name>
            <name name-style="western">
              <surname>Michal</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Wiltink</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Wild</surname>
              <given-names>PS</given-names>
            </name>
            <name name-style="western">
              <surname>Münzel</surname>
              <given-names>T</given-names>
            </name>
            <name name-style="western">
              <surname>Lackner</surname>
              <given-names>KJ</given-names>
            </name>
            <name name-style="western">
              <surname>Tibubos</surname>
              <given-names>AN</given-names>
            </name>
          </person-group>
          <article-title>Loneliness in the general population: prevalence, determinants and relations to mental health</article-title>
          <source>BMC Psychiatry</source>
          <year>2017</year>
          <month>03</month>
          <day>20</day>
          <volume>17</volume>
          <issue>1</issue>
          <fpage>97</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://bmcpsychiatry.biomedcentral.com/articles/10.1186/s12888-017-1262-x"/>
          </comment>
          <pub-id pub-id-type="doi">10.1186/s12888-017-1262-x</pub-id>
          <pub-id pub-id-type="medline">28320380</pub-id>
          <pub-id pub-id-type="pii">10.1186/s12888-017-1262-x</pub-id>
          <pub-id pub-id-type="pmcid">PMC5359916</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref4">
        <label>4</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Newall</surname>
              <given-names>NEG</given-names>
            </name>
            <name name-style="western">
              <surname>Chipperfield</surname>
              <given-names>JG</given-names>
            </name>
            <name name-style="western">
              <surname>Bailis</surname>
              <given-names>DS</given-names>
            </name>
            <name name-style="western">
              <surname>Stewart</surname>
              <given-names>TL</given-names>
            </name>
          </person-group>
          <article-title>Consequences of loneliness on physical activity and mortality in older adults and the power of positive emotions</article-title>
          <source>Health Psychol</source>
          <year>2013</year>
          <month>08</month>
          <volume>32</volume>
          <issue>8</issue>
          <fpage>921</fpage>
          <lpage>4</lpage>
          <pub-id pub-id-type="doi">10.1037/a0029413</pub-id>
          <pub-id pub-id-type="medline">22888819</pub-id>
          <pub-id pub-id-type="pii">2012-21508-001</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref5">
        <label>5</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Penninx</surname>
              <given-names>BWJH</given-names>
            </name>
            <name name-style="western">
              <surname>van Tilburg</surname>
              <given-names>T</given-names>
            </name>
            <name name-style="western">
              <surname>Kriegsman</surname>
              <given-names>DMW</given-names>
            </name>
            <name name-style="western">
              <surname>Deeg</surname>
              <given-names>DJH</given-names>
            </name>
            <name name-style="western">
              <surname>Boeke</surname>
              <given-names>AJP</given-names>
            </name>
            <name name-style="western">
              <surname>van Eijk</surname>
              <given-names>JTM</given-names>
            </name>
          </person-group>
          <article-title>Effects of social support and personal coping resources on mortality in older age: the Longitudinal Aging Study Amsterdam</article-title>
          <source>Am J Epidemiol</source>
          <year>1997</year>
          <month>09</month>
          <day>15</day>
          <volume>146</volume>
          <issue>6</issue>
          <fpage>510</fpage>
          <lpage>9</lpage>
          <pub-id pub-id-type="doi">10.1093/oxfordjournals.aje.a009305</pub-id>
          <pub-id pub-id-type="medline">9290512</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref6">
        <label>6</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Joiner</surname>
              <given-names>JT</given-names>
            </name>
            <name name-style="western">
              <surname>Rudd</surname>
              <given-names>MD</given-names>
            </name>
          </person-group>
          <article-title>Disentangling the interrelations between hopelessness, loneliness, and suicidal ideation</article-title>
          <source>Suicide Life Threat Behav</source>
          <year>1996</year>
          <volume>26</volume>
          <issue>1</issue>
          <fpage>19</fpage>
          <lpage>26</lpage>
          <pub-id pub-id-type="medline">9173606</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref7">
        <label>7</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Wei</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Russell</surname>
              <given-names>DW</given-names>
            </name>
            <name name-style="western">
              <surname>Zakalik</surname>
              <given-names>RA</given-names>
            </name>
          </person-group>
          <article-title>Adult attachment, social self-efficacy, self-disclosure, loneliness, and subsequent depression for freshman college students: a longitudinal study</article-title>
          <source>J Counseling Psychol</source>
          <year>2005</year>
          <month>10</month>
          <volume>52</volume>
          <issue>4</issue>
          <fpage>602</fpage>
          <lpage>614</lpage>
          <pub-id pub-id-type="doi">10.1037/0022-0167.52.4.602</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref8">
        <label>8</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Holt-Lunstad</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Smith</surname>
              <given-names>TB</given-names>
            </name>
          </person-group>
          <article-title>Loneliness and social isolation as risk factors for CVD: implications for evidence-based patient care and scientific inquiry</article-title>
          <source>Heart</source>
          <year>2016</year>
          <month>07</month>
          <day>01</day>
          <volume>102</volume>
          <issue>13</issue>
          <fpage>987</fpage>
          <lpage>9</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="http://heart.bmj.com/lookup/pmidlookup?view=long&#38;pmid=27091845"/>
          </comment>
          <pub-id pub-id-type="doi">10.1136/heartjnl-2015-309242</pub-id>
          <pub-id pub-id-type="medline">27091845</pub-id>
          <pub-id pub-id-type="pii">heartjnl-2015-309242</pub-id>
          <pub-id pub-id-type="pmcid">PMC4941164</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref9">
        <label>9</label>
        <nlm-citation citation-type="confproc">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Ruiz</surname>
              <given-names>C</given-names>
            </name>
            <name name-style="western">
              <surname>Ito</surname>
              <given-names>K</given-names>
            </name>
            <name name-style="western">
              <surname>Wakamiya</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Aramaki</surname>
              <given-names>E</given-names>
            </name>
          </person-group>
          <article-title>Loneliness in a connected world: analyzing online activity and expressions on real life relationships of lonely users</article-title>
          <year>2017</year>
          <conf-name>The AAAI 2017 Spring Symposium on Wellbeing AI: From Machine Learning to Subjectivity Oriented Computing</conf-name>
          <conf-date>January 1, 2017</conf-date>
          <conf-loc>California</conf-loc>
          <fpage>726</fpage>
          <lpage>733</lpage>
          <pub-id pub-id-type="doi">10.7591/cornell/9781501705335.003.0006</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref10">
        <label>10</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Hunt</surname>
              <given-names>MG</given-names>
            </name>
            <name name-style="western">
              <surname>Marx</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Lipson</surname>
              <given-names>C</given-names>
            </name>
            <name name-style="western">
              <surname>Young</surname>
              <given-names>J</given-names>
            </name>
          </person-group>
          <article-title>No more FOMO: limiting social media decreases loneliness and depression</article-title>
          <source>J Soc Clin Psychol</source>
          <year>2018</year>
          <month>12</month>
          <volume>37</volume>
          <issue>10</issue>
          <fpage>751</fpage>
          <lpage>768</lpage>
          <pub-id pub-id-type="doi">10.1521/jscp.2018.37.10.751</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref11">
        <label>11</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Andy</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Guntuku</surname>
              <given-names>SC</given-names>
            </name>
            <name name-style="western">
              <surname>Adusumalli</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Asch</surname>
              <given-names>DA</given-names>
            </name>
            <name name-style="western">
              <surname>Groeneveld</surname>
              <given-names>PW</given-names>
            </name>
            <name name-style="western">
              <surname>Ungar</surname>
              <given-names>LH</given-names>
            </name>
            <name name-style="western">
              <surname>Merchant</surname>
              <given-names>RM</given-names>
            </name>
          </person-group>
          <article-title>Predicting cardiovascular risk using social media data: performance evaluation of machine-learning models</article-title>
          <source>JMIR Cardio</source>
          <year>2021</year>
          <month>02</month>
          <day>19</day>
          <volume>5</volume>
          <issue>1</issue>
          <fpage>e24473</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://cardio.jmir.org/2021/1/e24473/"/>
          </comment>
          <pub-id pub-id-type="doi">10.2196/24473</pub-id>
          <pub-id pub-id-type="medline">33605888</pub-id>
          <pub-id pub-id-type="pii">v5i1e24473</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref12">
        <label>12</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Andy</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Guntuku</surname>
              <given-names>S</given-names>
            </name>
          </person-group>
          <article-title>Does social support expressed in post titles elicit comments in online substance use recovery forums?</article-title>
          <source>arXiv.</source>
          <comment>Preprint posted online on November 10, 2020
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://arxiv.org/abs/2011.05103"/>
          </comment>
        </nlm-citation>
      </ref>
      <ref id="ref13">
        <label>13</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Stokes</surname>
              <given-names>DC</given-names>
            </name>
            <name name-style="western">
              <surname>Andy</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Guntuku</surname>
              <given-names>SC</given-names>
            </name>
            <name name-style="western">
              <surname>Ungar</surname>
              <given-names>LH</given-names>
            </name>
            <name name-style="western">
              <surname>Merchant</surname>
              <given-names>RM</given-names>
            </name>
          </person-group>
          <article-title>Public priorities and concerns regarding COVID-19 in an online discussion forum: longitudinal topic modeling</article-title>
          <source>J Gen Intern Med</source>
          <year>2020</year>
          <month>07</month>
          <volume>35</volume>
          <issue>7</issue>
          <fpage>2244</fpage>
          <lpage>2247</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="http://europepmc.org/abstract/MED/32399912"/>
          </comment>
          <pub-id pub-id-type="doi">10.1007/s11606-020-05889-w</pub-id>
          <pub-id pub-id-type="medline">32399912</pub-id>
          <pub-id pub-id-type="pii">10.1007/s11606-020-05889-w</pub-id>
          <pub-id pub-id-type="pmcid">PMC7217615</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref14">
        <label>14</label>
        <nlm-citation citation-type="confproc">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Andy</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Chu</surname>
              <given-names>B</given-names>
            </name>
            <name name-style="western">
              <surname>Fathy</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Bennett</surname>
              <given-names>B</given-names>
            </name>
            <name name-style="western">
              <surname>Stokes</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Guntuku</surname>
              <given-names>SC</given-names>
            </name>
          </person-group>
          <article-title>Understanding social support expressed in a COVID-19 online forum</article-title>
          <year>2021</year>
          <month>04</month>
          <day>01</day>
          <conf-name>12th International Workshop on Health Text Mining and Information Analysis</conf-name>
          <conf-date>April 2021</conf-date>
          <conf-loc>Online</conf-loc>
          <fpage>19</fpage>
          <lpage>27</lpage>
        </nlm-citation>
      </ref>
      <ref id="ref15">
        <label>15</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Yang</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Kraut</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Levine</surname>
              <given-names>JM</given-names>
            </name>
          </person-group>
          <article-title>Commitment of newcomers and old-timers to online health support communities</article-title>
          <source>Proc SIGCHI Conf Hum Factor Comput Syst</source>
          <year>2017</year>
          <month>05</month>
          <volume>2017</volume>
          <fpage>6363</fpage>
          <lpage>6375</lpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="http://europepmc.org/abstract/MED/31423492"/>
          </comment>
          <pub-id pub-id-type="doi">10.1145/3025453.3026008</pub-id>
          <pub-id pub-id-type="medline">31423492</pub-id>
          <pub-id pub-id-type="pmcid">PMC6696930</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref16">
        <label>16</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Yang</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Kraut</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Smith</surname>
              <given-names>T</given-names>
            </name>
            <name name-style="western">
              <surname>Mayfield</surname>
              <given-names>E</given-names>
            </name>
            <name name-style="western">
              <surname>Jurafsky</surname>
              <given-names>D</given-names>
            </name>
          </person-group>
          <article-title>Seekers, providers, welcomers, and storytellers: modeling social roles in online health communities</article-title>
          <source>Proc SIGCHI Conf Hum Factor Comput Syst</source>
          <year>2019</year>
          <month>05</month>
          <volume>2019</volume>
          <fpage>344</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="http://europepmc.org/abstract/MED/31423493"/>
          </comment>
          <pub-id pub-id-type="doi">10.1145/3290605.3300574</pub-id>
          <pub-id pub-id-type="medline">31423493</pub-id>
          <pub-id pub-id-type="pmcid">PMC6696924</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref17">
        <label>17</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Yang</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Yao</surname>
              <given-names>Z</given-names>
            </name>
            <name name-style="western">
              <surname>Seering</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Kraut</surname>
              <given-names>R</given-names>
            </name>
          </person-group>
          <article-title>The channel matters: self-disclosure, reciprocity and social support in online cancer support groups</article-title>
          <source>Proc SIGCHI Conf Hum Factor Comput Syst</source>
          <year>2019</year>
          <month>05</month>
          <volume>2019</volume>
          <fpage>31</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="http://europepmc.org/abstract/MED/31448374"/>
          </comment>
          <pub-id pub-id-type="doi">10.1145/3290605.3300261</pub-id>
          <pub-id pub-id-type="medline">31448374</pub-id>
          <pub-id pub-id-type="pmcid">PMC6708374</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref18">
        <label>18</label>
        <nlm-citation citation-type="confproc">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Kivran-Swaine</surname>
              <given-names>F</given-names>
            </name>
            <name name-style="western">
              <surname>Ting</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Brubaker</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Teodoro</surname>
              <given-names>R</given-names>
            </name>
            <name name-style="western">
              <surname>Naaman</surname>
              <given-names>M</given-names>
            </name>
          </person-group>
          <article-title>Understanding loneliness in social awareness streams: expressions and responses</article-title>
          <year>2014</year>
          <month>05</month>
          <day>05</day>
          <conf-name>Eighth International AAAI Conference on Weblogs and Social Media</conf-name>
          <conf-date>June 1-4, 2014</conf-date>
          <conf-loc>Ann Arbor, MI</conf-loc>
        </nlm-citation>
      </ref>
      <ref id="ref19">
        <label>19</label>
        <nlm-citation citation-type="confproc">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Fiesler</surname>
              <given-names>C</given-names>
            </name>
            <name name-style="western">
              <surname>Jiang</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>McCann</surname>
              <given-names>J</given-names>
            </name>
            <name name-style="western">
              <surname>Frye</surname>
              <given-names>K</given-names>
            </name>
            <name name-style="western">
              <surname>Brubaker</surname>
              <given-names>JR</given-names>
            </name>
          </person-group>
          <article-title>Reddit rules! Characterizing an ecosystem of governance</article-title>
          <year>2018</year>
          <month>06</month>
          <day>15</day>
          <conf-name>Twelfth International AAAI Conference on Web and Social Media</conf-name>
          <conf-date>June 25-28, 2018</conf-date>
          <conf-loc>Palo Alto, CA</conf-loc>
        </nlm-citation>
      </ref>
      <ref id="ref20">
        <label>20</label>
        <nlm-citation citation-type="confproc">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Fernandes</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Bernardino</surname>
              <given-names>J</given-names>
            </name>
          </person-group>
          <article-title>What is BigQuery?</article-title>
          <source>Proceedings of the 19th International Database Engineering &#38; Applications Symposium</source>
          <year>2015</year>
          <month>07</month>
          <conf-name>IDEAS '15</conf-name>
          <conf-date>July 13-15, 2015</conf-date>
          <conf-loc>Yokohama, Japan</conf-loc>
          <fpage>202</fpage>
          <lpage>203</lpage>
          <pub-id pub-id-type="doi">10.1145/2790755.2790797</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref21">
        <label>21</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Park</surname>
              <given-names>G</given-names>
            </name>
            <name name-style="western">
              <surname>Yaden</surname>
              <given-names>DB</given-names>
            </name>
            <name name-style="western">
              <surname>Schwartz</surname>
              <given-names>HA</given-names>
            </name>
            <name name-style="western">
              <surname>Kern</surname>
              <given-names>ML</given-names>
            </name>
            <name name-style="western">
              <surname>Eichstaedt</surname>
              <given-names>JC</given-names>
            </name>
            <name name-style="western">
              <surname>Kosinski</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Stillwell</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Ungar</surname>
              <given-names>LH</given-names>
            </name>
            <name name-style="western">
              <surname>Seligman</surname>
              <given-names>MEP</given-names>
            </name>
          </person-group>
          <article-title>Women are warmer but no less assertive than men: gender and language on Facebook</article-title>
          <source>PLoS One</source>
          <year>2016</year>
          <volume>11</volume>
          <issue>5</issue>
          <fpage>e0155885</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://dx.plos.org/10.1371/journal.pone.0155885"/>
          </comment>
          <pub-id pub-id-type="doi">10.1371/journal.pone.0155885</pub-id>
          <pub-id pub-id-type="medline">27223607</pub-id>
          <pub-id pub-id-type="pii">PONE-D-15-49892</pub-id>
          <pub-id pub-id-type="pmcid">PMC4881750</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref22">
        <label>22</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Schwartz</surname>
              <given-names>HA</given-names>
            </name>
            <name name-style="western">
              <surname>Eichstaedt</surname>
              <given-names>JC</given-names>
            </name>
            <name name-style="western">
              <surname>Kern</surname>
              <given-names>ML</given-names>
            </name>
            <name name-style="western">
              <surname>Dziurzynski</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Ramones</surname>
              <given-names>SM</given-names>
            </name>
            <name name-style="western">
              <surname>Agrawal</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Shah</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Kosinski</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Stillwell</surname>
              <given-names>D</given-names>
            </name>
            <name name-style="western">
              <surname>Seligman</surname>
              <given-names>MEP</given-names>
            </name>
            <name name-style="western">
              <surname>Ungar</surname>
              <given-names>LH</given-names>
            </name>
          </person-group>
          <article-title>Personality, gender, and age in the language of social media: the open-vocabulary approach</article-title>
          <source>PLoS One</source>
          <year>2013</year>
          <volume>8</volume>
          <issue>9</issue>
          <fpage>e73791</fpage>
          <comment>
            <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://dx.plos.org/10.1371/journal.pone.0073791"/>
          </comment>
          <pub-id pub-id-type="doi">10.1371/journal.pone.0073791</pub-id>
          <pub-id pub-id-type="medline">24086296</pub-id>
          <pub-id pub-id-type="pii">PONE-D-13-03858</pub-id>
          <pub-id pub-id-type="pmcid">PMC3783449</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref23">
        <label>23</label>
        <nlm-citation citation-type="confproc">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Tan</surname>
              <given-names>C</given-names>
            </name>
            <name name-style="western">
              <surname>Niculae</surname>
              <given-names>V</given-names>
            </name>
            <name name-style="western">
              <surname>Danescu-Niculescu-Mizil</surname>
              <given-names>C</given-names>
            </name>
            <name name-style="western">
              <surname>Lee</surname>
              <given-names>L</given-names>
            </name>
          </person-group>
          <article-title>Winning arguments: interaction dynamics and persuasion strategies in good-faith online discussions</article-title>
          <source>Proceedings of the 25th International Conference on World Wide Web</source>
          <year>2016</year>
          <month>04</month>
          <day>01</day>
          <conf-name>WWW '16</conf-name>
          <conf-date>April 11-15, 2016</conf-date>
          <conf-loc>Montréal, QC</conf-loc>
          <fpage>613</fpage>
          <lpage>624</lpage>
          <pub-id pub-id-type="doi">10.1145/2872427.2883081</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref24">
        <label>24</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Blei</surname>
              <given-names>DM</given-names>
            </name>
            <name name-style="western">
              <surname>Ng</surname>
              <given-names>AY</given-names>
            </name>
            <name name-style="western">
              <surname>Jordan</surname>
              <given-names>MI</given-names>
            </name>
          </person-group>
          <article-title>Latent Dirichlet allocation</article-title>
          <source>J Machine Learning Res</source>
          <year>2003</year>
          <month>01</month>
          <day>01</day>
          <fpage>993</fpage>
          <lpage>1022</lpage>
        </nlm-citation>
      </ref>
      <ref id="ref25">
        <label>25</label>
        <nlm-citation citation-type="confproc">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Schwartz</surname>
              <given-names>HA</given-names>
            </name>
            <name name-style="western">
              <surname>Giorgi</surname>
              <given-names>S</given-names>
            </name>
            <name name-style="western">
              <surname>Sap</surname>
              <given-names>M</given-names>
            </name>
            <name name-style="western">
              <surname>Crutchley</surname>
              <given-names>P</given-names>
            </name>
            <name name-style="western">
              <surname>Ungar</surname>
              <given-names>L</given-names>
            </name>
            <name name-style="western">
              <surname>Eichstaedt</surname>
              <given-names>J</given-names>
            </name>
          </person-group>
          <article-title>DLATK: Differential Language Analysis ToolKit</article-title>
          <year>2017</year>
          <month>06</month>
          <day>05</day>
          <conf-name>2017 Conference on Empirical Methods in Natural Language Processing: System Demonstrations</conf-name>
          <conf-date>September 2017</conf-date>
          <conf-loc>Copenhagen, Denmark</conf-loc>
          <fpage>55</fpage>
          <lpage>60</lpage>
          <pub-id pub-id-type="doi">10.18653/v1/d17-2010</pub-id>
        </nlm-citation>
      </ref>
      <ref id="ref26">
        <label>26</label>
        <nlm-citation citation-type="journal">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Pennebaker</surname>
              <given-names>JW</given-names>
            </name>
            <name name-style="western">
              <surname>Boyd</surname>
              <given-names>RL</given-names>
            </name>
            <name name-style="western">
              <surname>Jordan</surname>
              <given-names>K</given-names>
            </name>
            <name name-style="western">
              <surname>Blackburn</surname>
              <given-names>K</given-names>
            </name>
          </person-group>
          <article-title>The development and psychometric properties of LIWC2015</article-title>
          <source>University Texas Austin</source>
          <year>2015</year>
          <month>09</month>
          <day>15</day>
          <fpage>1</fpage>
          <lpage>10</lpage>
        </nlm-citation>
      </ref>
      <ref id="ref27">
        <label>27</label>
        <nlm-citation citation-type="confproc">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Guntuku</surname>
              <given-names>SC</given-names>
            </name>
            <name name-style="western">
              <surname>Buffone</surname>
              <given-names>A</given-names>
            </name>
            <name name-style="western">
              <surname>Nanyang</surname>
              <given-names>KJ</given-names>
            </name>
            <name name-style="western">
              <surname>Eichstaedt</surname>
              <given-names>JC</given-names>
            </name>
            <name name-style="western">
              <surname>Ungar</surname>
              <given-names>LH</given-names>
            </name>
          </person-group>
          <article-title>Understanding and measuring psychological stress using social media</article-title>
          <year>2019</year>
          <month>05</month>
          <day>15</day>
          <conf-name>Thirteenth International AAAI Conference on Web and Social Media</conf-name>
          <conf-date>June 11-14, 2019</conf-date>
          <conf-loc>Münich, Germany</conf-loc>
          <fpage>214</fpage>
          <lpage>225</lpage>
        </nlm-citation>
      </ref>
      <ref id="ref28">
        <label>28</label>
        <nlm-citation citation-type="confproc">
          <person-group person-group-type="author">
            <name name-style="western">
              <surname>Mohammad</surname>
              <given-names>S</given-names>
            </name>
          </person-group>
          <article-title>Obtaining reliable human ratings of valence, arousal, and dominance for 20,000 English words</article-title>
          <year>2018</year>
          <conf-name>56th Annual Meeting of the Association for Computational Linguistics</conf-name>
          <conf-date>July 2018</conf-date>
          <conf-loc>Melbourne, Australia</conf-loc>
          <fpage>174</fpage>
          <lpage>184</lpage>
          <pub-id pub-id-type="doi">10.18653/v1/p18-1017</pub-id>
        </nlm-citation>
      </ref>
    </ref-list>
  </back>
</article>
