This paper is in the following e-collection/theme issue:

Formative Evaluation of Digital Health Interventions (2491) Digital Mental Health Interventions, e-Mental Health and Cyberpsychology (1565) Mental Health Issues in Adolescence (457) Text-messaging (SMS, WeChat etc)-Based Interventions (663) Linguistics, Comprehension, and Learning (53)

Published on in Vol 8 (2024)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/51795, first published .
Linguistic Variables and Gender Differences Within a Messenger-Based Psychosocial Chat Counseling Service for Children and Adolescents: Cross-Sectional Study

Linguistic Variables and Gender Differences Within a Messenger-Based Psychosocial Chat Counseling Service for Children and Adolescents: Cross-Sectional Study

Linguistic Variables and Gender Differences Within a Messenger-Based Psychosocial Chat Counseling Service for Children and Adolescents: Cross-Sectional Study

Authors of this article:

Zeki Efe1 Author Orcid Image ;   Sabrina Baldofski1 Author Orcid Image ;   Elisabeth Kohls1, 2 Author Orcid Image ;   Melanie Eckert3 Author Orcid Image ;   Shadi Saee3 Author Orcid Image ;   Julia Thomas3 Author Orcid Image ;   Richard Wundrack3, 4 Author Orcid Image ;   Christine Rummel-Kluge1, 2 Author Orcid Image
Article Authors Cited by (1) Tweetations (5) Metrics

    Original Paper

    1Department of Psychiatry and Psychotherapy, Medical Faculty, Leipzig University, Leipzig, Germany

    2Department of Psychiatry and Psychotherapy, University Leipzig Medical Center, Leipzig University, Leipzig, Germany

    3Krisenchat gGmbH, Berlin, Germany

    4Department of Psychology, Chair of Personality Psychology, Humboldt Universität zu Berlin, Berlin, Germany

    Corresponding Author:

    Christine Rummel-Kluge, MD

    Department of Psychiatry and Psychotherapy

    Medical Faculty

    Leipzig University

    Semmelweisstr 10

    Haus 13

    Leipzig, 04103

    Germany

    Phone: 49 341 9724464

    Email: Christine.Rummel-Kluge@medizin.uni-leipzig.de


    Background: Text messaging is widely used by young people for communicating and seeking mental health support through chat-based helplines. However, written communication lacks nonverbal cues, and language usage is an important source of information about a person’s mental health state and is known to be a marker for psychopathology.

    Objective: The aim of the study was to investigate language usage, and its gender differences and associations with the presence of psychiatric symptoms within a chat counseling service for adolescents and young adults.

    Methods: For this study, the anonymized chat content of a German messenger–based psychosocial chat counseling service for children and adolescents (“krisenchat”) between May 2020 and July 2021 was analyzed. In total, 661,131 messages from 6962 users were evaluated using Linguistic Inquiry and Word Count, considering the following linguistic variables: first-person singular and plural pronouns, negations, positive and negative emotion words, insight words, and causation words. Descriptive analyses were performed, and gender differences of those variables were evaluated. Finally, a binary logistic regression analysis examined the predictive value of linguistic variables on the presence of psychiatric symptoms.

    Results: Across all analyzed chats, first-person singular pronouns were used most frequently (965,542/8,328,309, 11.6%), followed by positive emotion words (408,087/8,328,309, 4.9%), insight words (341,460/8,328,309, 4.1%), negations (316,475/8,328,309, 3.8%), negative emotion words (266,505/8,328,309, 3.2%), causation words (241,520/8,328,309, 2.9%), and first-person plural pronouns (499,698/8,328,309, 0.6%). Female users and users identifying as diverse used significantly more first-person singular pronouns and insight words than male users (both P<.001). Negations were significantly more used by female users than male users or users identifying as diverse (P=.007). Similar findings were noted for negative emotion words (P=.01). The regression model of predicting psychiatric symptoms by linguistic variables was significant and indicated that increased use of first-person singular pronouns (odds ratio [OR] 1.05), negations (OR 1.11), and negative emotion words (OR 1.15) was positively associated with the presence of psychiatric symptoms, whereas increased use of first-person plural pronouns (OR 0.39) and causation words (OR 0.90) was negatively associated with the presence of psychiatric symptoms. Suicidality, self-harm, and depression showed the most significant correlations with linguistic variables.

    Conclusions: This study highlights the importance of examining linguistic features in chat counseling contexts. By integrating psycholinguistic findings into counseling practice, counselors may better understand users’ psychological processes and provide more targeted support. For instance, certain linguistic features, such as high use of first-person singular pronouns, negations, or negative emotion words, may indicate the presence of psychiatric symptoms, particularly among female users and users identifying as diverse. Further research is needed to provide an in-depth look into language processes within chat counseling services.

    JMIR Form Res 2024;8:e51795

    doi:10.2196/51795

    Keywords

    e-mental health; chat counseling; crisis; helpline; linguistic; language; Linguistic Inquiry and Word Count; LIWC; psychiatric symptoms 


    Childhood and adolescence are known for their biological, social, and psychological changes as vulnerable periods, in which young people are at an increased risk for experiencing mental health problems. It is also known that an early age of onset of mental illness is a risk factor for poor mental health conditions in adulthood [1,2]. The use of mental health care services for adolescents and young adults can have a positive influence on their attitudes, beliefs, and behaviors, which are known to be important predictors of their later mental health [3]. A growing number of studies indicate that children, adolescents, and young adults use the internet to seek help for their mental health problems because the digital environment is familiar and easily accessible, offers anonymity, and accommodates their need for independence [4-8]. Nearly all young people aged 12 to 19 years in Germany (94%) own a smartphone [9].

    With the increased use of smartphones, text messaging has become the primary communication tool for today’s youth [10]. Studies on text messaging usage with mobile phones have shown that adolescents experience text messaging as a quick, easy, convenient, playful, and inexpensive way of communication [11-14]. In line with this trend, a number of crisis helplines and similar services have begun to offer online support services such as chat or email counseling [15-18]. Studies have shown that adolescents prefer texting to talking when seeking help for mental health problems and find it easier to write than to express serious concerns verbally [18,19]. Recent studies support the acceptance, feasibility, and usability of online support services, especially among young people [7,20].

    However, written language lacks nonverbal stimuli. Recent research has shown that facial expressions and prosody have an influence on the recognition of a speaker’s intention in face-to-face communication [21,22]. In fact, in comparison to face-to-face interactions, people report higher levels of miscommunication when texting. This might represent a barrier in messenger-based counseling and may make it difficult for young people to understand and interpret the intentions of online counselors [23,24]. This lack of nonverbal cues can also make it more difficult for crisis line counselors to establish and maintain a therapeutic relationship [25,26]. In some studies, counselors reported greater difficulty and a lower perceived ability to establish a therapeutic relationship in the digital environment compared to a face-to-face counseling or therapy setting [25,27]. In addition to nonverbal stimuli, language usage is an important source of information in the therapeutic context, and the way people use words conveys a great deal of information about themselves and their current situation [28]. Language reflects both conscious and unconscious thoughts and feelings [29]. Linguists distinguish 2 aspects in the study of language: the formal and the content features of language. The formal aspect concerns grammar, syntax, reaction speed, speech tempo, etc, whereas content features consider vocabulary and word choice [30]. The investigation of “lexical diversity” thus allows a better insight into the cognitive diversity of people [31,32].

    Thus, in the 1970s, it was evidenced for the first time that language use can be a specific marker for psychopathology, especially depression [30]. It was found that individuals with depression use more first-person singular pronouns (ie, “I,” “my,” “me,” and “mine”) in both spoken and written language [33,34], supporting cognitive theories of depression [33], which indicate that depression is associated with an increased self-focus. Recent research has shown that increased use of certain words, for example, sad (eg, “crying,” “grief,” and “sad”) or sleep (eg, “asleep” and “bed”), correlates positively with higher levels of depressive symptoms [35]. Further studies also found gender differences in language usage. For example, it was found that women tend to use more language related to thoughts, emotions, senses, negations, and verbs in the present or past tense than men [36]. Furthermore, women were shown to be more likely to use first-person singular pronouns than men, which is consistent with the higher prevalence of depression in women [36,37]. Besides first-person singular pronouns and negative emotion words, causation words (eg, “because”) were also found to be used more by people having depression [34,38]. In line with this, research confirmed the Seligman theory for learned helplessness, which postulates that individuals at risk for depression attribute the cause of a negative event as being internal, global, and stable, by showing that young adults with negative attributional styles were more likely to develop clinically significant depression than those without such attributional styles [39]. There are also studies indicating negative attributional styles as predictive factors for developing depressive symptoms when experiencing negative life events [40-42]. Due to the trend of digitalization in mental health care, linguistic investigations have been conducted in the digital environment as well. In doing so, a positive association was found between Twitter posts indicating loneliness and mental health problems of the users [43]. In another study, which examined the language usage of users of loneliness forums, it was found that these users tended to use words associated with sadness or a desire for social contact, that is, their overall language leaned toward words with negative valence [44]. Regarding associations with psychopathology, the newest findings indicate that individuals with depressive symptoms used fewer complex syntactic constructions, such as adverbial phrases, perhaps because these require greater cognitive effort [35]. The population of young smartphone users and texters remains a vulnerable and underserved group in crisis counseling, which is why further research on outcomes and the effectiveness of specific communication and counseling strategies is needed [45,46]. To date, there have been no attempts to examine the chat content of crisis counseling services with regard to their linguistic structure. For this purpose, anonymized chat messages from a messenger-based psychosocial chat counseling service, krisenchat (German for “crisis chat”), were used to examine (1) which linguistic indicators and gender differences can be identified within the messages of chat users and (2) how these linguistic indicators are associated with the presence of psychiatric symptoms. Based on the existing literature, it was hypothesized that female users would be more likely to use first-person singular pronouns, negations, and insight words than male users and users identifying as diverse. Additionally, it was hypothesized that higher use of first-person singular pronouns, negations, negative emotion words, and words indicating causation would be associated with a higher likelihood of the presence of psychiatric symptoms among users.


    Sampling and Data Collection

    For the purpose of this study, anonymized chat data from all users receiving counseling between May 2020 and July 2021 were extracted from the krisenchat database. Data extraction and preparation were performed by authors affiliated with krisenchat (ME, SS, JT, and RW) so that chat content remained within the krisenchat database. The anonymized chat data included metadata on the chat (total number of messages and words sent by users during the whole counseling process, and number of sessions) and information about the user that counselors identified and noted during the counseling process (sociodemographic information, such as gender and age, and topics of users’ concerns). krisenchat counselors were volunteers and had a background in psychosocial studies. In addition, they underwent a structured 2-month training in chat-based counseling. Regarding gender, counselors had 3 options (male, female, and diverse) to mark in their documentation. They were encouraged to record the identified gender and not the biological sex of the users. The gender “diverse” included individuals identifying as nonbinary or diverse, or indicating to be unsure about their gender identity. For more information about the study design and the nature of krisenchat, we referred to the initial evaluation study of krisenchat [20]. Linguistic variables were determined using Linguistic Inquiry and Word Count (LIWC; see below for details).

    The sample examined in this study was based on the previous evaluation of krisenchat, in which the sample consisted of those who completed a subsequent feedback survey after the counseling session [20]. Thus, out of a total of 11,031 users in the above-mentioned time period, 6962 (63.1%) completed a feedback survey. The chat messages of these 6962 users were analyzed. In total, 661,131 messages (mean 94.96, SD 259.46) from 26,614 chat sessions (mean 3.82, SD 6.24) with a total word count of 8,872,154 (mean 1274.37, SD 2954.57) were analyzed.

    Ethical Considerations

    The Medical Faculty of the University of Leipzig approved this study on August 3, 2021 (372/21-ek). Users were informed about the data protection and privacy policy of krisenchat when they first contacted the counseling service. The chat counseling only began after confirming the policy with “Yes.” Participants in the study confirmed informed consent via an opt-in function before taking part in the feedback survey.

    Measures

    Linguistic Variables

    LIWC is a software for dictionary-based quantitative text analysis [29]. LIWC performs an automated 1-word analysis based on a lexicon with more than 80 categories (ie, language variables, descriptors, linguistic dimensions, psychological dimensions, concerns, informal language, and punctuation) including a total of 18,711 words. In 2008, the German version of the lexicon was developed, and good equivalence was confirmed for the majority of LIWC categories [47]. The tool has been used in various studies on personality, social, and clinical psychological frameworks and for the analysis of therapeutic essays, everyday communication, or computer-based communication, and it can therefore be considered a reliable software program for quantitative text analysis [47-52].

    LIWC counts the number of words within the lexicon over a whole chat and assigns them to categories. The output file includes all categories of the lexicon. All variables, except summary variables, are expressed as percentages of the total word count of a respective chat. Based on previous findings [34-37,44,53,54], the following linguistic variables were considered in this study: first-person singular (eg, “I,” “me,” and “mine”) and first-person plural pronouns (eg, “we,” “us,” and “our”), negations (eg, “no,” “not,” and “never”), positive emotion words (eg, “love,” “nice,” and “sweet”), negative emotion words (eg, “hurt,” “worried,” and “sad”), cognitive process words such as words related to insight (eg, “think” and “know”), and words related to causation (eg, “because” and “effect”).

    Psychiatric Symptoms

    The presence of psychiatric symptoms was assessed during the counseling process and noted by krisenchat counselors. The identification of psychiatric symptoms was derived from the concerns reported by the users. Counselors distinguished between the presence of the following symptoms: depression, anxiety, suicidality, self-harm, addictive behavior, eating disorders, flashbacks, and obsessive-compulsive symptoms. Additionally, symptoms were summed up into a dichotomous variable “psychiatric symptoms” to indicate the presence or absence of psychiatric symptoms (0, “not present;” 1, “present”).

    Statistical Analysis

    Statistical analyses were performed using IBM SPSS Statistics version 27.0 (IBM Corp). A 2-tailed α value of .05 was applied to statistical testing. First, descriptive statistics were performed for sociodemographic variables and linguistic variables of the total sample. Additionally, Kruskal-Wallis H tests (because of nonnormality of the linguistic variables) were used to identify gender differences in use, that is, metadata for the number of sessions, messages, and words of each user. Then, a 1-way multivariate analysis of variance (1-way MANOVA) was conducted to test for gender differences in language usage controlling for word count. Gender was considered as an independent variable, and all 7 linguistic variables (ie, first-person singular pronouns, first-person plural pronouns, negations, positive emotion words, negative emotion words, insight words, and causation words) were considered as dependent variables. Post-hoc univariate ANOVAs were conducted separately for every linguistic variable. Bonferroni correction was applied to account for multiple testing. Then, binary logistic regression analysis was conducted to examine the predictive effect of linguistic variables (first-person singular and plural pronouns, negations, positive and negative emotion words, insight words, and causation words), age, and gender (recoded into a set of dummy variables with “male” as the reference variable) on the presence of psychiatric symptoms. The amount of explained variance as shown by Nagelkerke R2 was interpreted as follows: R2 >0.20, “acceptable” or small effect size; R2 >0.40, “good” or average effect size; and R2 >0.50, “very good” or large effect size [55]. Additionally, Spearman correlation coefficients (ρ) were reported between linguistic variables. Finally, with the aim to examine the deeper relationship between linguistic variables and psychiatric symptoms, explorative Spearman correlations (ρ) between all 7 linguistic variables and all categories of psychiatric symptoms (suicidality, self-harm, depression, anxiety, eating disorder symptoms, flashbacks, obsessive-compulsive symptoms, and addictive behavior) were computed and interpreted as follows: ρ=0.10, small effect size; ρ=0.30, moderate effect size; and ρ=0.50, large effect size [56].


    Sociodemographic Characteristics

    The average user was 17 years old (mean 16.55, SD 3.45 years; range 8-25 years), and most users were female (female: 4988/5978, 83.4%; male: 881/5978, 14.7%; diverse: 109/5978, 1.8%). A large number of all users (4841/6962, 69.5%) contacted the counseling service due to psychiatric symptoms. Further concerns identified were psychosocial distress (eg, school-related problems, family-related problems, bullying, etc; 2370/6962, 34.0%) or emotional distress (eg, grief, lovesickness, anger, and loneliness; 2101/6962, 30.2%) [20].

    The users participated in an average of 3.82 (SD 6.24) counseling sessions and sent an average of 94.96 (SD 259.46; range 2-11,512) messages with an average of 1274.37 (SD 2954.57) words throughout the counseling process. Additional testing indicated that there were gender differences in the numbers of sessions (χ22=22.849; P<.001), messages (χ22=14.863; P<.001), and words (χ22=33.036; P<.001). The results are presented in Table 1. Subsequent post-hoc tests indicated that female users attended a significantly higher number of sessions (z=−4.211; P<.001; r=0.05) and sent significantly more messages (z=−3.247; P<.001; r=0.04) and words (z=−5.349; P<.001; r=0.07) than male users, whereas there were no significant differences between female users and users identifying as diverse (number of sessions: P=.13; number of messages: P=.18; number of words: P=.22). Users identifying as diverse also attended a significantly higher number of sessions (z=−3.441; P<.001; r=0.04) and sent significantly more messages (z=−2.972; P<.001; r=0.04) and words (z=−3.639; P<.001; r=0.05) than male users.

    Table 1. Gender-specific differences in metadata (N=5978).
    Metadata variablesMalea, mean (SD)Femalea, mean (SD)Diversea, mean (SD)χ2 (df)bP value
    Session count3.23 (4.17)c4.17 (6.94)d5.43 (7.56)d22.849 (2)<.001
    Message count79.05 (145.20)c103.74 (287.43)d189.14 (512.32)d14.863 (2)<.001
    Word count1068.80 (1965.98)c1392.55 (3279.22)d2166.18 (4653.08)d33.036 (2)<.001

    aReduced sample size owing to missing data on gender.

    bTest statistic for the Kruskal-Wallis H test.

    c,dDifferent letters indicate significant differences between the groups, while same letters indicate no significant differences between the groups.

    Descriptive Statistics of Linguistic Variables

    In the total sample, the mean percentage of first-person singular pronouns among all words of a user during the whole counseling process was 11.59% (SD 2.46%), indicating that on average, more than one-tenth of all words written throughout all chat messages was a first-person singular pronoun (“I,” “me,” “my,” or “mine”). The next most used linguistic categories were positive emotion words (mean 4.85%, SD 1.70%) and insight words (mean 4.05%, SD 1.32%). The mean percentage of negations among all words of a user during the whole counseling process was 3.76% (SD 1.54%). Furthermore, negative emotion words were used with a mean percentage among all words of 3.23% (SD 1.27%). Causation words were used with a mean percentage among all words of 2.93% (SD 1.04%). Finally, first-person plural pronouns were least frequently used with a mean percentage among all words of 0.43% (SD 0.63%).

    Gender Differences in Linguistic Variables

    Gender-specific descriptive statistics are displayed in Table 2. A 1-way MANOVA showed statistically significant differences in linguistic variables between genders (F14, 11,932=8.945; P<.001; partial η²=0.01; Wilk Λ=0.979). Post-hoc univariate ANOVAs were conducted separately for every linguistic variable. Separate ANOVAs and respective Bonferroni-corrected post-hoc tests showed that when controlling for word count, there were statistically significant differences in the use of first-person singular pronouns between genders (F2, 5972=49.780; P<.001; partial η²=0.02), with female users (mean difference=0.90, 95% CI 0.68-1.12; P<.001) and users identifying as diverse (mean difference=0.98, 95% CI 0.37-1.56; P<.001) using more first-person singular pronouns than male users, whereas there was no significant difference between female users and users identifying as diverse (mean difference=0.08, 95% CI −0.66 to 0.50; P>.99). Furthermore, the use of negations differed significantly between genders (F2, 5972=4.915; P=.007; partial η²=0.002), with female users using significantly more negations than male users (mean difference=−0.16, 95% CI 0.03-0.30; P=.01), while there were no significant differences in the use of negations between male users and users identifying as diverse (mean difference=0.329, 95% CI −0.05 to 0.71; P=.12) and between female users and users identifying as diverse (mean difference=0.17, 95% CI −0.20 to 0.53; P=.84). Another significant difference was found in the use of negative emotion words between genders (F2, 5972=4.505; P=.01; partial η²=0.00), with female users using significantly more negative emotion words than male users (mean difference=0.12, 95% CI 0.01-0.23; P=.04; not significant after Bonferroni correction), while no significant differences in the use of negative emotion words were found between female users and users identifying as diverse (mean difference=0.22, 95% CI −0.07 to 0.52; P=.22) and between male users and users identifying as diverse (mean difference=0.11, 95% CI −0.42 to 0.21; P>.99). Finally, the results showed a significant difference in the use of insight words between genders (F2, 5972=15.215; P<.001; partial η²=0.01), with female users (mean difference=0.26, 95% CI 0.14-0.37; P<.001) and users identifying as diverse (mean difference=0.41, 95% CI −0.15 to 0.47; P=.007) using significantly more insight words than male users, while no significant differences were found between female users and users identifying as diverse (mean difference=0.16, 95% CI −0.42 to 0.21; P=.67). No overall significant differences were found between genders in the use of first-person plural pronouns (F2, 5972=3.006; P=.05; partial η²=0.00), positive emotion words (F2, 5972=0.489; P=.61; partial η²=0.00), and causation words (F2, 5972=2.434; P=.09; partial η²=0.00).

    Table 2. Gender-specific differences in language usage (N=5978).
    Linguistic variablesMalea (n=881), mean (SD)Femalea (n=4988), mean (SD)Diversea (n=109), mean (SD)P value
    First-person singular pronouns10.81 (2.60)b11.71 (2.43)c11.79 (2.55)c<.001
    First-person plural pronouns0.47 (0.84)0.42 (0.59)0.32 (0.43).05
    Negations3.62 (2.24)b3.76 (1.39)c,d3.94 (1.49)b,d.007
    Positive emotions4.87 (1.80)4.86 (1.67)4.68 (1.79).61
    Negative emotions3.13 (1.37)b3.24 (1.24)c3.02 (1.43)b,c.01
    Insight words3.81 (1.38)b4.07 (1.30)c4.20 (1.42)c<.001
    Causation words2.99 (1.12)2.92 (1.02)3.04 (0.92).09

    aReduced sample size owing to missing data on gender.

    b,c,dDifferent letters indicate significant differences between the groups, while same letters indicate no significant differences between the groups.

    Predicting Psychiatric Symptoms by Linguistic Variables

    The binomial logistic regression model was statistically significant (χ²8=25.0; P=.002), resulting in a small amount of explained variance, as shown by Nagelkerke R2=0.124 (Table 3). Of the 10 variables entered into the regression model, all but 3 contributed significantly to the presence of psychiatric symptoms: first-person singular and plural pronouns, negations, negative emotion words, causation words (all P<.001), and female gender (P=.005), while positive emotion words (P=.08), insight words (P=.90), and diverse gender (P=.57) showed no significant effects. Using first-person plural pronouns was associated with a lower likelihood of reporting psychiatric symptoms (odds ratio [OR] 0.39), as did using more causation words (OR 0.90). In contrast, a higher use of first-person singular pronouns was associated with an increased likelihood of reporting psychiatric symptoms (OR 1.05), as did using more negations (OR 1.11) or negative emotion words (OR 1.15). Finally, being female (OR 1.18) or having a higher age (OR 1.04) was also associated with an increased likelihood of the presence of psychiatric symptoms.

    Table 3. Prediction of psychiatric symptoms by language usage (N=6962).
    VariableBaSEWaldbP valueORc (95% CI)
    Age0.040.0122.04<.0011.04 (1.02-1.06)
    Gender





    		Female0.160.067.94.0051.18 (1.05-1.32)

    		Diverse0.120.210.33.571.13 (0.75-1.70)
    First-person singular pronouns0.050.0115.78<.0011.05 (1.03-1.08)
    First-person plural pronouns−0.940.05301.21<.0010.39 (0.35-0.44)
    Negations0.110.0232.14<.0011.11 (1.07-1.16)
    Positive emotions−0.030.023.09.080.97 (0.95-1.01)
    Negative emotions0.140.0243.45<.0011.15 (1.11-1.21)
    Insight words−0.010.020.02.901.00 (0.96-1.04)
    Causation words−0.110.0313.80<.0010.90 (0.85-0.94)
    Constant−0.870.2413.07<.0010.42 (N/Ad)

    aB: regression coefficient.

    bDegrees of freedom were 1 for all Wald statistics.

    cOR: odds ratio.

    dN/A: not applicable.

    The results of an additional correlation analysis between all linguistic variables are reported in Table 4. Among others, significant findings included a negative association between first-person singular pronouns and first-person plural pronouns (ρ=−0.24; P<.001). In line with this, first-person singular pronouns were positively correlated with negations (ρ=0.17; P<.001) and negative emotion words (ρ=0.15; P<.001).

    Table 4. Spearman correlation coefficients between linguistic variables (N=6962).
    VariableFirst-person singular pronounsFirst-person plural pronounsNegationsPositive emotionsNegative emotionsInsight wordsCausation words
    First-person singular pronouns







    		ρ1−0.24a0.17a0.04a0.15a0.29a−0.02

    		P value—b<.001<.001.002<.001<.001.22
    First-person plural pronouns







    		ρ−0.24a1−0.08a−0.01−0.10a−0.03a−0.07a

    		P value<.001—<.001.34<.001.01<.001
    Negations







    		ρ0.17a−0.08a1−0.02a0.020.13a0.04a

    		P value<.001<.001—.045.19<.001<.001
    Positive emotions







    		ρ0.04a−0.01−0.02a1−0.06a−0.03a0.04a

    		P value.002.34.045—<.001.023<.001
    Negative emotions







    		ρ0.15a−0.10a0.02−0.06a10.03a−0.07a

    		P value<.001<.001.19<.001—.023<.001
    Insight words







    		ρ0.29a−0.03a0.13a−0.03a0.03a10.04a

    		P value<.001<.01<.001.023.023—<.001
    Causation words







    		ρ−0.02−0.07a0.04a0.04a−0.07a0.04a1

    		P value.22<.001<.001<.001<.001<.001—

    aStatistical significance.

    bNot applicable.

    Associations Between Linguistic Variables and Psychiatric Symptoms

    Finally, an exploratory correlation analysis indicated evidence for the relationship between linguistic variables and various psychiatric symptoms, which are displayed in Table 5. Suicidality, self-harm, depression, and anxiety showed the most significant correlations to linguistic variables. In particular, the use of first-person singular pronouns was positively associated with suicidality (ρ=0.11; P<.001) and self-harm (ρ=0.10; P<.001). The use of first-person plural pronouns was negatively associated with suicidality (ρ=−0.10; P<.001) and depression (ρ=−0.14; P<.001). The use of negations was positively associated with suicidality (ρ=0.18; P<.001) and self-harm (ρ=0.12; P<.001). Finally, the use of negative emotion words was positively associated with depression (ρ=0.12; P<.001) and anxiety (ρ=0.15; P<.001).

    Table 5. Spearman correlation coefficients between linguistic variables and psychiatric symptoms (N=6962).
    VariableSuicidalitySelf-harmDepressionAnxietyEating disorder symptomsFlashbacksObsessive-compulsive symptomsAddictive behavior
    First-person singular pronouns








    		ρ0.11a0.10a0.08a−0.010.05a−0.020.00−0.01

    		P value<.001<.001<.001.46<.001.08.75.33
    First-person plural pronouns








    		ρ−0.10a−0.08a−0.14a−0.04a−0.04a−0.01−0.03a−0.04a

    		P value<.001<.001<.001<.001<.001.26.007<.001
    Negations








    		ρ0.18a0.12a−0.06a−0.06a0.01−0.02−0.010.00

    		P value<.001<.001<.001<.001.40.07.28.80
    Positive emotions








    		ρ−0.05a−0.020.010.00−0.02−0.02−0.02−0.03a

    		P value<.001.08.41.87.06.19.09.01
    Negative emotions








    		ρ0.04a0.03a0.12a0.15a−0.03a0.020.02a0.00

    		P value.002.03<.001<.001.02.18.045.82
    Insight words








    		ρ0.03a−0.000.08a−0.01−0.01−0.010.00a−0.04a

    		P value.005.73<.001.65.67.61.045<.001
    Causation words








    		ρ−0.06a−0.03a−0.03a−0.02−0.02−0.04a0.01−0.01

    		P value<.001.007.03.10.13<.001.63.32

    aStatistical significance.


    Principal Findings and Comparison With Prior Work

    The findings of this study provide first-time valuable insights into the psycholinguistic characteristics of children, adolescents, and young adults seeking psychosocial support through a messenger-based crisis counseling service (krisenchat). Previous findings that examined psycholinguistic characteristics in association with mental health, which however focused on texts (eg, from social media or online therapies), could be identified in the chat context as well. Specifically, linguistic variables were found to be associated with the presence of psychiatric symptoms [43,44,57]. The use of first-person singular pronouns, negations, and negative emotion words increased the likelihood of the presence of psychiatric symptoms, while the use of first-person plural pronouns and causation words was associated with a lower likelihood of the presence of psychiatric symptoms. Female gender was also associated with an increased likelihood of the presence of psychiatric symptoms, which is consistent with the higher prevalence of psychiatric symptoms in women [58]. Gender differences were found, with female users exhibiting more frequent use of certain linguistic features. Previous linguistic analyses, especially in the digital context, such as social media platforms, have been performed with a focus on the presence of psychiatric symptoms or associations with psychiatric symptoms, in particular, depressive symptoms [59]. Linguistic analyses of social media have proven useful in predicting depression, anxiety, loneliness, personality disorders, or other mental health issues [43,57,60-62]. As there are no other comparative studies in this field, the focus of the below comparison of the present results with previous findings relies on correlates of linguistic variables with the presence of psychiatric symptoms, especially depression or anxiety.

    First-Person Pronouns

    Starting with the most frequently used linguistic variable among those examined, an increased use of first-person singular pronouns was associated with a higher likelihood of the presence of psychiatric symptoms. Additionally, it was determined that they were used more often by female users and users identifying as diverse than by male users. In contrast, the linguistic variable of first-person plural pronouns was found to be the least used and did not predict the presence of psychiatric symptoms. Taking into account the most frequently mentioned concerns among the users of krisenchat (see [20]), involving psychiatric symptoms, such as depression or anxiety, and the higher prevalence of depression in female samples [63], these findings are consistent with previous findings on language usage. Thus, in line with cognitive theories of depression (eg, according to [33]) indicating that depression is associated with an increased self-focus, previous research showed that individuals with depression used more first-person singular pronouns (ie, “I,” “my,” “me,” and “mine”) in both spoken and written language [33,34]. Similarly, first-person plural pronouns (ie, “we,” “us,” and “our”) were used significantly less by depressed individuals, which may be attributed to social isolation or lack of social integration and social engagement in the context of depression [64-66]. However, the increased use of first-person singular pronouns may also be a marker for increased vulnerability to stress and negative emotionality and not directly for depression [67,68].

    Negations and Causation Words

    Negations were found to be significantly associated with the presence of psychiatric symptoms and were used more frequently by female users than male users or users identifying as diverse. Contradicting the hypothesis, the present results indicate that the use of causation words reduces the likelihood of the presence of psychiatric symptoms, while no significant differences were found between genders. In line with this finding, previous studies indicated that low use of causation words is associated with positive treatment outcomes in treatment for personality disorders [53]. It was found that the use of fewer cognitive words, such as causation words, was associated with a more coherent personal story [69]. A recent study underlined this finding by pointing out that patients having depression tend to use significantly more aligned sentences than bringing them into a logical chain compared with a healthy control group [34]. This cross-sectional study design does not allow to draw conclusions about the trend of the use of causation words throughout the chat counseling. The meaning of these divergent results deserves further specific longitudinal research on the development and change of language use across chat counseling.

    Negative and Positive Emotion Words

    Elevated use of negative emotion words was associated with an increase in the presence of psychiatric symptoms. Negative emotion words were found to differ between genders, that is, more negative emotion words were used by female users than by male users. This is in line with previous research indicating a significantly higher general use of emotion words by women than men, while men were found to use more anger words [70,71]. This is also in accordance with findings showing that more frequent use of negative emotion words, including anxiety, sadness, and anger words, was positively correlated with higher anxiety and depression levels [54].

    Depressive Symptoms

    Taken together, the results underline a higher likelihood of depression (or anxiety) in users using more self-focused language (first-person singular pronouns), and more negative and fewer positive (emotion) words. The response style theory of rumination in depression, which could also be proven for worry in generalized anxiety, explains that symptoms of repetitive self-focused negative thinking become habitual over time [72,73]. In terms of language usage, this theory suggests that people with high levels of depression or anxiety might communicate using more self-focused language (ie, more I-related pronouns), and more negative and fewer positive words, and that this tendency may become habitual and outside of conscious awareness. In line with this, linguistic analyses of text-based therapy found reductions in the use of first-person singular pronouns, even though language usage was not being focused on in the treatment [74]. Moreover, changes in the use of positive and negative emotion words and words indicating certainty (eg, “always” and “never”) could be identified during the treatment for depression. Researchers interpret these findings as changes in cognitive processes [75].

    Suicidality

    In relation to depression, it is also important to keep suicidality in mind because messenger-based chat counseling services are used in acute crises, such as suicidality [76]. In accordance with the present findings, previous studies found that suicidal behavior is associated with the use of more I-related pronouns [77]. Likewise, in accordance with the present findings, previous literary analyses indicated that suicidal poets also used fewer first-person plural pronouns than nonsuicidal poets [66]. In addition, these studies showed that the use of more absolutist language, that is, superlatives and intensifiers (eg, “absolutely,” “completely,” “all,” “none,” etc), was associated with suicidality [60,77,78].

    The results of this study indicate that by considering language usage, differences in the user population can be discovered and may also be linked to psychopathology. Thus, language usage should be integrated into the counseling strategy. In the context of computer-based analyses, it was found that in addition to the standardized diagnostic tools used to confirm a psychiatric diagnosis, linguistic research showed that systematic analyses of clients’ language may be used to reliably classify them into diagnostic groups [34]. Additionally, computer-based methods were shown to distinguish persons with depression from other clinical subgroups [79]. Thus, linguistic or, in general, qualitative analyses of text messages seem to be advisable to examine chat-based counseling services in more depth. This is even more true than for social media platforms because text messages are less influenced by social desirability, facilitating more granular visibility into changes in linguistic patterns [80].

    Implications for Counselors in Chat-Based Counseling

    Multiple implications for psychosocial chat counseling have emerged from the findings. First, counselors may use psycholinguistic analyses as an additional tool for assessing the mental health state of users. By monitoring the usage of specific linguistic features, counselors can identify individuals at risk for psychiatric symptoms and tailor interventions accordingly. For example, in cases of elevated use of first-person singular pronouns, negations, and negative emotion words, particularly for female users and users who identify as diverse, a more in-depth exploration of psychiatric symptoms, especially depression, anxiety, or suicidality, is recommended. Regardless of the actual cause, it seems advisable to monitor the frequency of first-person singular pronouns throughout the chat history. For example, establishing a word-counting function during chat counseling may provide counselors with additional valuable information. In turn, increased use of first-person plural pronouns as well as causation words can be identified as protective factors, and also expanded and used as such. For example, increased use of first-person plural pronouns on the counselor’s side might be helpful to create a sense of “unity” or belongingness. Taking social belongingness into account as a protective factor for depression, suicidality, or mental disorders in general, it seems advisable to create closeness or sociality through language to initially stabilize the user emotionally, for example, meet their need for attachment or create a base to make further recommendations [81].

    Furthermore, it can be helpful to mirror statements of users in the context of active listening (positively paraphrased) or to guide the users through targeted questioning techniques into positively formulated thoughts, goals, behavioral directions, etc (ie, in concrete terms) to avoid negations. This implication is also underscored by the fact that increased use of words referring to expectation, trust, and belongingness was associated with lower depression rates [80,81]. Furthermore, the establishment of logical chains, in the sense of causation, also appears to be of great importance. Thus, attempts should be made to bring concerns, feelings, etc into a logical coherent context in the sense of stimulus-reaction chains. Moreover, the use of insight words shows no influence on the presence of psychiatric symptoms, yet reflective functioning is known to be a protective factor [82]. An emphasis on self-reflection could potentially be integrated into psychoeducation by supporting an understanding of one’s own mind as an aspect of resilience and personal agency that can be fostered through social support and professional help [82]. In practice, repeated questions about what feeling is associated with particular problems or concerns, or identifying thoughts or cognition in terms of one’s own belief patterns, can help to practice such skills.

    Strengths and Limitations

    To the best of our knowledge, this study is the first to examine language usage in a messenger-based crisis counseling service among youth and young adults. In addition to the strength of the large sample size, this study acts as a reference and comparison for further studies in this area, in part because of the use of the internationally recognized method LIWC. Nevertheless, some limitations have to be taken into account. Owing to the retrospective study design, the data rely on convenience sampling, which limits the generalizability to a more mixed-gender population. It should also be noted that the counseling service is offered in German-speaking countries, which is why cultural and linguistic differences in language usage must be taken into account, and generalizability is limited. An international comparison between counseling services in different countries, cultures, and languages would provide insights into similarities and differences. Assuming that people in crisis reach out to the chat counseling service, high use of emotion words seems somewhat expected. Since the nature of the study was cross-sectional and the words were counted across all chat messages (ie, not in chronological order or within sessions), no data can be provided on the trend of the word count for either positive or negative emotion words, which is why no indications can be derived on whether the word count relates only to the beginning of the chat or also to the progression throughout the chat counseling. Longitudinal studies examining changes in language use across consecutive chat sessions may provide further insights into these associations. Consideration must also be given to the nested data structure, which cannot be clearly read owing to the format used to provide data by LIWC. Therefore, for future studies, in addition to the trend of the word count throughout the chat counseling, the consideration of levels (eg, within a message and during a session) is recommended. Therefore, a qualitative analysis of the chats may be beneficial for providing more in-depth insights into individual language usage as well as concerns. A qualitative approach would also ensure the quality of the classification of the presence of psychiatric symptoms. Likewise, no standardized measurement instruments were used, which in turn opens up further opportunities for future research, for example, implementing symptom-specific questionnaires to examine associations between (changes in) symptom severity and language usage.

    Conclusion

    This study underlines the options, possibilities, and chances of examining psycholinguistic characteristics in psychosocial online chat counseling services for children and adolescents. The identified associations between specific linguistic features and the presence of psychiatric symptoms provide valuable insights for the development of targeted interventions. By considering psycholinguistic findings in the counseling practice, counselors may enhance their understanding of the psychological processes of users and their interventions to offer a more targeted service for children and adolescents seeking help. Nevertheless, further research is needed to investigate the mechanisms underlying linguistic patterns and explore the effectiveness of linguistic-based interventions. At the same time, this would allow further research on the link between specific indicators and changes in specific psychopathology.

    Acknowledgments

    We acknowledge support from Leipzig University for open access publishing. The Federal Ministry of Health funded this project (ZMI1-2521FEP001).

    Authors' Contributions

    ZE, SB, and CRK designed the study. The data set was prepared by ME, SS, JT, and RW. ZE performed the statistical analysis. ZE and SB drafted the article. All authors approved the final manuscript.

    Conflicts of Interest

    ZE, SB, EK, ME, SS, JT, and RW confirm no conflicts of interest. ME, SS, JT, and RW are paid employees at krisenchat gGmbH. CRK received lecture honoraria from Recordati and Servier outside and independent of the submitted work.

    1. Patton GC, Ross DA, Santelli JS, Sawyer SM, Viner RM, Kleinert S. Next steps for adolescent health: a Lancet Commission. Lancet. Feb 01, 2014;383(9915):385-386. [CrossRef] [Medline]
    2. Solmi M, Radua J, Olivola M, Croce E, Soardo L, Salazar de Pablo G, et al. Age at onset of mental disorders worldwide: large-scale meta-analysis of 192 epidemiological studies. Mol Psychiatry. Jan 2022;27(1):281-295. [FREE Full text] [CrossRef] [Medline]
    3. Viner RM, Ozer EM, Denny S, Marmot M, Resnick M, Fatusi A, et al. Adolescence and the social determinants of health. Lancet. Apr 28, 2012;379(9826):1641-1652. [CrossRef] [Medline]
    4. Batchelor S, Stoyanov S, Pirkis J, Kõlves K. Use of Kids Helpline by Children and Young People in Australia During the COVID-19 Pandemic. J Adolesc Health. Jun 2021;68(6):1067-1074. [CrossRef] [Medline]
    5. Best P, Gil-Rodriguez E, Manktelow R, Taylor BJ. Seeking Help From Everyone and No-One: Conceptualizing the Online Help-Seeking Process Among Adolescent Males. Qual Health Res. Jul 2016;26(8):1067-1077. [CrossRef] [Medline]
    6. Gowen LK. Online Mental Health Information Seeking in Young Adults with Mental Health Challenges. Journal of Technology in Human Services. Apr 2013;31(2):97-111. [CrossRef]
    7. Pretorius C, Chambers D, Coyle D. Young People's Online Help-Seeking and Mental Health Difficulties: Systematic Narrative Review. J Med Internet Res. Nov 19, 2019;21(11):e13873. [FREE Full text] [CrossRef] [Medline]
    8. Subramaniam M, Taylor NG, St. Jean B, Follman R, Kodama C, Casciotti D. As simple as that?: tween credibility assessment in a complex online world. Journal of Documentation. May 11, 2015;71(3):550-571. [CrossRef]
    9. Feierabend S, Rathgeb T, Hediye K, Glöckler S. JIM-Studie 2020. Medienpädagogischer Forschungsverbund Südwest. URL: https://www.mpfs.de/fileadmin/files/Studien/JIM/2020/JIM-Studie-2020_Web_final.pdf [accessed 2022-12-11]
    10. Teens, Technology and Friendships. Pew Research Center. 2015. URL: https://www.pewresearch.org/internet/2015/08/06/teens-technology-and-friendships/ [accessed 2023-12-22]
    11. Agosto DE, Abbas J, Naughton R. Relationships and social rules: Teens’ social network and other selection practices. J Am Soc Inf Sci Tec. Apr 04, 2012;63(6):1108-1124. [CrossRef]
    12. Blair BL, Fletcher AC, Gaskin ER. Cell Phone Decision Making. Youth & Society. Sep 16, 2013;47(3):395-411. [CrossRef]
    13. Church K, de Oliveira R. What's up with whatsapp?: comparing mobile instant messaging behaviors with traditional SMS. In: MobileHCI '13: Proceedings of the 15th International Conference on Human-Computer Interaction with Mobile Devices and Services. Presented at: 15th International Conference on Human-Computer Interaction with Mobile Devices and Services; August 27-30, 2013, 2013;352-361; Munich, Germany. [CrossRef]
    14. Yoon C, Jeong C, Rolland E. Understanding individual adoption of mobile instant messaging: a multiple perspectives approach. Inf Technol Manag. Oct 31, 2014;16(2):139-151. [CrossRef]
    15. Gould MS, Chowdhury S, Lake AM, Galfalvy H, Kleinman M, Kuchuk M, et al. National Suicide Prevention Lifeline crisis chat interventions: Evaluation of chatters' perceptions of effectiveness. Suicide Life Threat Behav. Dec 2021;51(6):1126-1137. [FREE Full text] [CrossRef] [Medline]
    16. Mokkenstorm JK, Eikelenboom M, Huisman A, Wiebenga J, Gilissen R, Kerkhof AJFM, et al. Evaluation of the 113Online Suicide Prevention Crisis Chat Service: Outcomes, Helper Behaviors and Comparison to Telephone Hotlines. Suicide Life Threat Behav. Jun 2017;47(3):282-296. [CrossRef] [Medline]
    17. Perry Y, Werner-Seidler A, Calear A, Christensen H. Web-Based and Mobile Suicide Prevention Interventions for Young People: A Systematic Review. J Can Acad Child Adolesc Psychiatry. 2016;25(2):73-79. [FREE Full text] [Medline]
    18. Predmore Z, Ramchand R, Ayer L, Kotzias V, Engel C, Ebener P, et al. Expanding Suicide Crisis Services to Text and Chat. Crisis. Jul 2017;38(4):255-260. [CrossRef] [Medline]
    19. Zøllner L, Rask L, Konieczna A. Selvskadende adfærd blandt unge mellem 13-19 år. Del 2: Sociale medier, søvn og mistrivsel. Center for Selvmordsforskning. URL: https://viden.sl.dk/media/5545/selvskadende-adfaerd-blandt-unge-mellem-13-19aar-del-2.pdf [accessed 2023-12-22]
    20. Eckert M, Efe Z, Guenthner L, Baldofski S, Kuehne K, Wundrack R, et al. Acceptability and feasibility of a messenger-based psychological chat counselling service for children and young adults ("krisenchat"): A cross-sectional study. Internet Interv. Mar 2022;27:100508. [FREE Full text] [CrossRef] [Medline]
    21. Domaneschi F, Passarelli M, Chiorri C. Facial expressions and speech acts: experimental evidences on the role of the upper face as an illocutionary force indicating device in language comprehension. Cogn Process. Aug 2017;18(3):285-306. [CrossRef] [Medline]
    22. Hellbernd N, Sammler D. Prosody conveys speaker’s intentions: Acoustic cues for speech act perception. Journal of Memory and Language. Jun 2016;88:70-86. [CrossRef]
    23. Johnson A, Bostwick E, Anderson C. How Do Computer-Mediated Channels Negatively Impact Existing Interpersonal Relationships? In: Contexts of the Dark Side of Communication. New York, NY. Peter Lang; 2016.
    24. Kelly L, Keaten JA, Becker B, Cole J, Littleford L, Rothe B. “It's the American Lifestyle!”: An Investigation of Text Messaging by College Students. Qualitative Research Reports in Communication. Jan 2012;13(1):1-9. [CrossRef]
    25. Appleton R, Williams J, Vera San Juan N, Needle JJ, Schlief M, Jordan H, et al. Implementation, Adoption, and Perceptions of Telemental Health During the COVID-19 Pandemic: Systematic Review. J Med Internet Res. Dec 09, 2021;23(12):e31746. [FREE Full text] [CrossRef] [Medline]
    26. Wampold BE. The Great Psychotherapy Debate. England, UK. Routledge; 2013.
    27. Cook JE, Doyle C. Working alliance in online therapy as compared to face-to-face therapy: preliminary results. Cyberpsychol Behav. Apr 2002;5(2):95-105. [CrossRef] [Medline]
    28. Pennebaker JW, Mehl MR, Niederhoffer KG. Psychological aspects of natural language. use: our words, our selves. Annu Rev Psychol. 2003;54:547-577. [CrossRef] [Medline]
    29. Pennebaker J. The secret life of pronouns. New York, NY. Bloomsbury Press; 2011.
    30. Andreasen NG, Pfohl B. Linguistic analysis of speech in affective disorders. Arch Gen Psychiatry. Nov 1976;33(11):1361-1367. [CrossRef] [Medline]
    31. Johnson W. I. A program of research. Psychological Monographs. 1944;56(2):1-15. [CrossRef]
    32. Chotlos JW. IV. A statistical and comparative analysis of individual written language samples. Psychological Monographs. 1944;56(2):75-111. [CrossRef]
    33. Pyszczynski T, Greenberg J. Self-regulatory perseveration and the depressive self-focusing style: A self-awareness theory of reactive depression. Psychological Bulletin. 1987;102(1):122-138. [CrossRef]
    34. Smirnova D, Cumming P, Sloeva E, Kuvshinova N, Romanov D, Nosachev G. Language Patterns Discriminate Mild Depression From Normal Sadness and Euthymic State. Front Psychiatry. 2018;9:105. [FREE Full text] [CrossRef] [Medline]
    35. Morales MR. Multimodal Depression Detection: An Investigation of Features and Fusion Techniques for Automated Systems. City University of New York (CUNY). 2018. URL: https://academicworks.cuny.edu/cgi/viewcontent.cgi?article=3607&context=gc_etds [accessed 2023-12-22]
    36. Newman ML, Groom CJ, Handelman LD, Pennebaker JW. Gender Differences in Language Use: An Analysis of 14,000 Text Samples. Discourse Processes. May 15, 2008;45(3):211-236. [CrossRef]
    37. Shorey S, Ng ED, Wong CHJ. Global prevalence of depression and elevated depressive symptoms among adolescents: A systematic review and meta-analysis. Br J Clin Psychol. Jun 2022;61(2):287-305. [CrossRef] [Medline]
    38. Pennebaker JW, Mayne TJ, Francis ME. Linguistic predictors of adaptive bereavement. J Pers Soc Psychol. Apr 1997;72(4):863-871. [CrossRef] [Medline]
    39. Alloy LB, Abramson LY, Whitehouse WG, Hogan ME, Panzarella C, Rose DT. Prospective incidence of first onsets and recurrences of depression in individuals at high and low cognitive risk for depression. J Abnorm Psychol. Feb 2006;115(1):145-156. [CrossRef] [Medline]
    40. Seligman M. Helplessness: On depression, development, and death. New York, NY. W H Freeman/Times Books/Henry Holt & Co; 1975.
    41. Hankin BL, Abramson LY, Miller N, Haeffel GJ. Cognitive Vulnerability-Stress Theories of Depression: Examining Affective Specificity in the Prediction of Depression Versus Anxiety in Three Prospective Studies. Cognitive Therapy and Research. Jun 2004;28(3):309-345. [CrossRef]
    42. Joiner TE. A test of the hopelessness theory of depression in youth psychiatric inpatients. J Clin Child Psychol. Jun 2000;29(2):167-176. [CrossRef] [Medline]
    43. Guntuku SC, Schneider R, Pelullo A, Young J, Wong V, Ungar L, et al. Studying expressions of loneliness in individuals using twitter: an observational study. BMJ Open. Nov 04, 2019;9(11):e030355. [FREE Full text] [CrossRef] [Medline]
    44. Andy A. Studying How Individuals Who Express the Feeling of Loneliness in an Online Loneliness Forum Communicate in a Nonloneliness Forum: Observational Study. JMIR Form Res. Jul 20, 2021;5(7):e28738. [FREE Full text] [CrossRef] [Medline]
    45. Nesmith A. Reaching Young People Through Texting-Based Crisis Counseling. ASW. Jan 02, 2019;18(4):1147-1164. [CrossRef]
    46. Sawyer M, Sawyer A, La Greca A. Help-Seeking by Young People with Depressive Disorders. Prevention Researcher. 2012;19(4):11-14.
    47. Wolf M, Horn AB, Mehl MR, Haug S, Pennebaker JW, Kordy H. Computergestützte quantitative Textanalyse. Diagnostica. Apr 2008;54(2):85-98. [CrossRef]
    48. Back MD, Küfner ACP, Egloff B. “Automatic or the People?”. Psychol Sci. May 13, 2011;22(6):837-838. [CrossRef]
    49. Hai-Jew S. Extracting Linguistic Patterns from Texts with LIWC (“luke”) for Analysis. C2C Digital Magazine (Fall 2016 / Winter 2017). URL: https:/​/scalar.​usc.edu/​works/​c2c-digital-magazine-fall-2016--winter-2017/​extracting-linguistic-patterns-from-texts-liwc-analysis [accessed 2023-12-22]
    50. Pennebaker JW, Chung CK, Frazee J, Lavergne GM, Beaver DI. When small words foretell academic success: the case of college admissions essays. PLoS One. 2014;9(12):e115844. [FREE Full text] [CrossRef] [Medline]
    51. Proyer RT, Brauer K. Exploring adult Playfulness: Examining the accuracy of personality judgments at zero-acquaintance and an LIWC analysis of textual information. Journal of Research in Personality. Apr 2018;73:12-20. [CrossRef]
    52. Vergani M, Bliuc A. The evolution of the ISIS’ language: A quantitative analysis of the language of the first year of Dabiq magazine. Sicurezza, Terrorismo e societa. 2015:7-20. [FREE Full text]
    53. Arntz A, Hawke LD, Bamelis L, Spinhoven P, Molendijk ML. Changes in natural language use as an indicator of psychotherapeutic change in personality disorders. Behav Res Ther. Mar 2012;50(3):191-202. [CrossRef] [Medline]
    54. Dirkse D, Hadjistavropoulos HD, Hesser H, Barak A. Linguistic analysis of communication in therapist-assisted internet-delivered cognitive behavior therapy for generalized anxiety disorder. Cogn Behav Ther. 2015;44(1):21-32. [CrossRef] [Medline]
    55. Backhaus K, Erichson B, Plinke W, Weiber R. Multivariate Analysemethoden: Eine anwendungsorientierte Einführung. Berlin, Heidelberg. Springer; 2006.
    56. Ellis P. The Essential Guide to Effect Sizes: Statistical Power, Meta-Analysis, and the Interpretation of Research Results. Cambridge, United Kingdom. Cambridge University Press; 2010.
    57. Guntuku SC, Yaden DB, Kern ML, Ungar LH, Eichstaedt JC. Detecting depression and mental illness on social media: an integrative review. Current Opinion in Behavioral Sciences. Dec 2017;18:43-49. [CrossRef]
    58. Alonso J, Angermeyer MC, Bernert S, Bruffaerts R, Brugha TS, Bryson H, et al. ESEMeD/MHEDEA 2000 Investigators‚ European Study of the Epidemiology of Mental Disorders (ESEMeD) Project. Prevalence of mental disorders in Europe: results from the European Study of the Epidemiology of Mental Disorders (ESEMeD) project. Acta Psychiatr Scand Suppl. 2004(420):21-27. [CrossRef] [Medline]
    59. Chancellor S, De Choudhury M. Methods in predictive techniques for mental health status on social media: a critical review. NPJ Digit Med. 2020;3:43. [FREE Full text] [CrossRef] [Medline]
    60. Al-Mosaiwi M, Johnstone T. In an Absolute State: Elevated Use of Absolutist Words Is a Marker Specific to Anxiety, Depression, and Suicidal Ideation. Clin Psychol Sci. Jul 2018;6(4):529-542. [FREE Full text] [CrossRef] [Medline]
    61. Eichstaedt JC, Smith RJ, Merchant RM, Ungar LH, Crutchley P, Preoţiuc-Pietro D, et al. Facebook language predicts depression in medical records. Proc Natl Acad Sci U S A. Oct 30, 2018;115(44):11203-11208. [FREE Full text] [CrossRef] [Medline]
    62. Mairesse F, Walker MA, Mehl MR, Moore RK. Using Linguistic Cues for the Automatic Recognition of Personality in Conversation and Text. jair. Nov 28, 2007;30:457-500. [CrossRef]
    63. Hasin DS, Sarvet AL, Meyers JL, Saha TD, Ruan WJ, Stohl M, et al. Epidemiology of Adult DSM-5 Major Depressive Disorder and Its Specifiers in the United States. JAMA Psychiatry. Apr 01, 2018;75(4):336-346. [FREE Full text] [CrossRef] [Medline]
    64. Berkman LF, Glass T, Brissette I, Seeman TE. From social integration to health: Durkheim in the new millennium. Soc Sci Med. Sep 2000;51(6):843-857. [CrossRef] [Medline]
    65. Schwarzbach M, Luppa M, Sikorski C, Fuchs A, Maier W, van den Bussche H, et al. The relationship between social integration and depression in non-demented primary care patients aged 75 years and older. J Affect Disord. Feb 20, 2013;145(2):172-178. [CrossRef] [Medline]
    66. Stirman SW, Pennebaker JW. Word use in the poetry of suicidal and nonsuicidal poets. Psychosom Med. 2001;63(4):517-522. [CrossRef] [Medline]
    67. Lyons M, Aksayli ND, Brewer G. Mental distress and language use: Linguistic analysis of discussion forum posts. Computers in Human Behavior. Oct 2018;87:207-211. [CrossRef]
    68. Tackman AM, Sbarra DA, Carey AL, Donnellan MB, Horn AB, Holtzman NS, et al. Depression, negative emotionality, and self-referential language: A multi-lab, multi-measure, and multi-language-task research synthesis. J Pers Soc Psychol. May 2019;116(5):817-834. [CrossRef] [Medline]
    69. Klein K, Boals A. Coherence and Narrative Structure in Personal Accounts of Stressful Experiences. Journal of Social and Clinical Psychology. Mar 2010;29(3):256-280. [CrossRef]
    70. Mehl MR, Pennebaker JW. The sounds of social life: a psychometric analysis of students' daily social environments and natural conversations. J Pers Soc Psychol. Apr 2003;84(4):857-870. [CrossRef] [Medline]
    71. Thomson R, Murachver T. Predicting gender from electronic discourse. Br J Soc Psychol. Jun 2001;40(Pt 2):193-208. [CrossRef] [Medline]
    72. Watkins ER, Nolen-Hoeksema S. A habit-goal framework of depressive rumination. J Abnorm Psychol. Feb 2014;123(1):24-34. [CrossRef] [Medline]
    73. Watkins ER, Roberts H. Reflecting on rumination: Consequences, causes, mechanisms and treatment of rumination. Behav Res Ther. Apr 2020;127:103573. [CrossRef] [Medline]
    74. Nook EC, Hull TD, Nock MK, Somerville LH. Linguistic measures of psychological distance track symptom levels and treatment outcomes in a large set of psychotherapy transcripts. Proc Natl Acad Sci U S A. Mar 29, 2022;119(13):e2114737119. [FREE Full text] [CrossRef] [Medline]
    75. Stamatis CA, Meyerhoff J, Liu T, Sherman G, Wang H, Liu T, et al. Prospective associations of text-message-based sentiment with symptoms of depression, generalized anxiety, and social anxiety. Depress Anxiety. Dec 2022;39(12):794-804. [FREE Full text] [CrossRef] [Medline]
    76. Kohls E, Guenthner L, Baldofski S, Eckert M, Efe Z, Kuehne K, et al. Suicidal Ideation Among Children and Young Adults in a 24/7 Messenger-Based Psychological Chat Counseling Service. Front Psychiatry. 2022;13:862298. [FREE Full text] [CrossRef] [Medline]
    77. Homan S, Gabi M, Klee N, Bachmann S, Moser A, Duri' M, et al. Linguistic features of suicidal thoughts and behaviors: A systematic review. Clin Psychol Rev. Jul 2022;95:102161. [FREE Full text] [CrossRef] [Medline]
    78. Fekete S. The Internet - A New Source of Data on Suicide, Depression and Anxiety: A Preliminary Study. Archives of Suicide Research. Nov 2002;6(4):351-361. [CrossRef]
    79. Linguistic Inquiry and Word Count (LIWC). URL: https://www.liwc.app/ [accessed 2022-12-11]
    80. U.S. Smartphone Use in 2015. Pew Research Center. URL: https://www.pewresearch.org/internet/2015/04/01/us-smartphone-use-in-2015/ [accessed 2023-12-22]
    81. Santini ZI, Koyanagi A, Tyrovolas S, Mason C, Haro JM. The association between social relationships and depression: a systematic review. J Affect Disord. Apr 01, 2015;175:53-65. [CrossRef] [Medline]
    82. Kealy D, Rice SM, Seidler ZE, Oliffe JL, Ogrodniczuk JS. Reflective functioning and men's mental health: Associations with resilience and personal growth initiative. Stress Health. Oct 2021;37(4):706-714. [CrossRef] [Medline]

    ‎
    LIWC: Linguistic Inquiry and Word Count
    OR: odds ratio

    Edited by A Mavragani; submitted 15.08.23; peer-reviewed by JC Perez, Z Haime; comments to author 11.09.23; revised version received 29.09.23; accepted 29.11.23; published 12.01.24.

    Copyright

    ©Zeki Efe, Sabrina Baldofski, Elisabeth Kohls, Melanie Eckert, Shadi Saee, Julia Thomas, Richard Wundrack, Christine Rummel-Kluge. Originally published in JMIR Formative Research (https://formative.jmir.org), 12.01.2024.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.