Assessing suicide risk is challenging and consequential. Many patients who die by suicide deny suicidal thinking in their last conversation with a mental health care provider [1,2]. Similarly, many survivors of near-lethal suicide attempts who engaged in treatment at the time of attempt report concealing their intent from their providers beforehand [3]. Even those with more passive ideation tend to disclose and describe their suicidal experiences inconsistently across contexts and assessment methods [4,5]. Yet, to date, clinicians and researchers have had little choice but to rely almost exclusively on what patients say about their suicidal thoughts and behaviors [6].

While these reporting discrepancies may be due in part to fears of the consequences of disclosure (eg, hospitalization [7-9]), they also likely reflect the complexity of the suicidal experience and the limitations of our methods for capturing and quantifying it [6]. Indeed, most suicide assessments rely on responses to yes or no questions, even though many individuals possess ambivalent feelings toward suicide that do not lend themselves to clear-cut, binary responses [6,10]. Further, some experience suicidal thoughts more in the form of imagery than thoughts that can be easily articulated in words [11]. Prior research demonstrates the power of implicit processes pertinent to suicide, such as attentional biases toward suicide and implicit associations with death, both of which have been shown to predict subsequent suicidal thoughts and behaviors [12,13]. Taken together, these findings lend support to the call from Schechter et al [6] for the development of assessments that enable capture of suppressed, dissociated, and/or implicit material related to suicide, as well as calls from researchers for the investigation of markers that capture more than what is voluntarily reported by the patient, such as biochemical and behavioral markers [14].

Nonverbal behavior, the elements of interaction that extend beyond the words spoken, including facial expressions, head movements, gaze patterns, and vocal characteristics, convey important affective and interpersonal information in clinical contexts [15,16]. A growing body of research suggests that nonverbal behaviors, such as the facial action units (AUs) taxonomized in the facial action coding system (FACS) [17,18], contain salient cues in clinical settings. For example, in a study of emergency room patients who recently attempted suicide, patients’ facial expressions during a clinical interview predicted future suicide attempts over the next 2 years [19]. Importantly, these expressions, particularly chin-raising, predicted subsequent suicide attempts above and beyond the interviewing psychiatrist’s risk determination [19]. In other studies [20,21], particular AUs and combinations of AUs exhibited across various contexts have differentiated suicidal and nonsuicidal individuals, including inner brow-raisers, outer brow-raisers, brow-lowerers, lip corner-depressors, and Duchenne smiles, ie, smiles typically associated with genuine positive affect that involve both the cheek-raiser and the lip corner-puller. One previous study examined not just facial expressions but also head motion exhibited during suicide assessments and found reduced head motion to be associated with more severe suicidal ideation among hospitalized suicidal adults [22]; this same behavior has also emerged as a strong predictor of related psychopathology, such as depression [23]. While the extant research is limited to a small number of studies, and specific findings vary by sample and context, facial action and head motion exhibited during suicide assessments appear to contain rich information.

Decades of research in adjacent disciplines (eg, mother-infant communication) highlight the importance of attending to both members of the interacting dyad when examining nonverbal communication [24]. Yet only one previous study assessed the facial action of the clinical interviewer during an assessment with recent suicide attempters [19]. Results revealed that the interviewing psychiatrist’s facial actions predicted subsequent suicide attempts in the participant, while her written, explicit risk determination did not [19,25]. Similarly, researchers have observed that suicidal adolescents’ clinical interviewers adapt their own vocal prosody in response to suicidal adolescents’ presentations [26]. These findings suggest that important data pertaining to suicide risk may be captured and transmitted via the nonverbal behaviors of both members of the interviewer-participant dyad, warranting further research.

Nonverbal behaviors have likely received less attention in clinical research than verbal reports because of the time-intensive and subjective nature of quantifying them [27,28]. Recent advances in machine learning and affective computing afford the opportunity to quantify nonverbal information efficiently and systematically. Deep learning–based approaches, such as the Python-Based Automated Facial Affect Recognition (PyAFAR) software [27,28], enable objective, reliable, and efficient measurement of visual behavior, including facial AUs, head pose, and eye and mouth opening and closing for each frame of video [28]. The continuous, intensive time series data produced makes it possible to characterize dynamic, complex, and out-of-awareness features of the nonverbal channels of communication.

Participants were 70 young adults between the ages of 18 and 26 years (mean 21.32, SD 2.11 years) recruited from the community through social media advertisements (eg, Instagram [Meta], Facebook [Meta]) between October 2020 and December 2021. Participants were required to reside in New York State. Participants mostly identified as women. The sample was diverse in terms of race and ethnicity and sexual orientation, with about half the sample identifying as White, and slightly less than half of the sample identifying as heterosexual. See Table 1 for demographic characteristics.

Participants were screened for eligibility via an anonymous Qualtrics survey. To be eligible, young adults were required to endorse either: (1) presence of suicidal behavior in the past year (ie, suicide attempt, interrupted attempt, aborted attempt, or preparatory behaviors; n=35); or (2) no lifetime history of suicidal thoughts or behaviors (n=35), both assessed using an online version of the Columbia-Suicide Severity Rating Scale (C-SSRS; [29]). Exclusion criteria were the presence of any factor visibly impairing the young adult’s ability to effectively participate in the study.

Eligible young adults were contacted via email and scheduled for an hour-long online recorded video call. During the video call, master’s-level research assistants administered the C-SSRS interview as well as a brief self-report battery via Qualtrics (see Measures). Any participant reporting a history of suicidal or self-injurious thoughts or behaviors at any point during the call received a risk assessment and safety plan prior to the termination of the video call.

Suicidal participants were contacted via email 3 months following the baseline interview and were invited to complete an online self-report survey administered via Qualtrics. The follow-up survey assessed the presence or absence of suicidal thoughts and behaviors over the past 3 months as well as past-month suicidal ideation severity. Of the 33 suicidal participants in our analytic sample, 28 completed the follow-up survey and were included in our post hoc exploratory prospective analyses.

All participants provided informed consent at the start of the video call. Deception was used regarding the rationale for the video recording to minimize potential bias in nonverbal behaviors. Participants were initially informed that the video recordings were for response verification and training purposes. Full disclosure regarding the study’s true purpose was provided at the conclusion of the study, at which point participants had the option to withdraw and request video deletion. Video recordings were stored on a secure, Health Insurance Portability and Accountability Act (HIPAA)–compliant drive using a unique ID number, distinct from ID numbers used for all other participant data. All participants were compensated with a US $25 Amazon gift card. Participants who completed the 3-month follow-up online survey were compensated with an additional US $10 Amazon gift card. All study procedures were approved by the Teachers College, Columbia University Institutional Review Board (IRB #s 20‐274, 22‐135, 23‐260).

We pursued post hoc exploratory analyses testing whether any participant and interviewer nonverbal behaviors that differentiated groups at baseline were associated with ideation severity and/or suicidal behavior 3 months later among a subset of suicidal participants who completed a follow-up survey 3 months later. We use Mann-Whitney U tests, 2-tailed independent samples t tests, and bivariate correlation, as appropriate, and do not run predictive models due to the small sample size for these exploratory analyses (n=28). For comparison, we also test whether participants’ baseline ideation severity on the SIQ, or interviewers’ clinical ratings, predict the same. Clinical ratings were comprised of quantitative risk level assignments that interviewers made immediately following the interview based on information gathered and clinical intuition, ranging from 1 (“No risk at all”) to 5 (“Very high risk”). We do not run predictive models due to the small sample size for these exploratory analyses (n=28).

The final analytic sample for whom we retained video recordings comprised 66 participants (Table 1). The analytic sample included 33 suicidal participants and 33 nonsuicidal participants. Lifetime suicidal behaviors reported by suicidal participants included: suicide attempts (n=17, 26% of participants), interrupted attempts (n=23, 35%), aborted attempts (n=26, 39%), and preparatory behaviors (n=23, 35%). With regard to severity of current suicidal ideation and depressive symptoms, the mean ideation severity among suicidal participants was 69.97 (SD 34.71), compared to 9.61 (SD 7.55) among nonsuicidal participants and a clinical cut-off value of 41 [36]. Mean depression severity among suicidal participants fell in the “moderate” range (mean 12.65, SD 5.39), and in the “none/mild” range among nonsuicidal participants (mean 5.61, SD 3.90) [37]. Suicidal and nonsuicidal groups did not differ on age (t₆₂=1, Cohen d=0.25, 95% CI −0.24 to 0.74; P=.32), sex at birth (χ²₁=0.41, Φ=.08; P=.52), or racial composition (χ²₄=6.45, Cramér V=.32; P=.17). Gender identity differed significantly between groups (χ²₂=13.36, Cramér V=.46; P=.001), with the suicidal group comprising more men (n=8) and gender nonconforming and gender queer individuals (n=11) and fewer women (n=13), relative to the nonsuicidal group (men=5; gender nonconforming=1; women=26).

Mean suicidal ideation severity at follow-up (n=28) was 55.46 (SD 22.15), compared to 63.85 (SD 29.96) among the same group of participants at baseline (t₂₆=1.55; P=.13). Of these, 29% (8/28) reported engaging in suicidal behavior since baseline. Suicidal behaviors reported included preparatory behaviors (n=4), aborted attempt (n=3), and interrupted attempt (n=1). Due to small sample sizes, a composite variable was created and recoded as 3-month suicidal behavior (presence vs absence).

Participant nonverbal behaviors that differentiated suicidal and nonsuicidal participants at baseline (ie, EAR and MAR velocity) were not associated with subsequent suicidal ideation severity (r_EAR=−0.23, P=.19, r_MAR=−0.03; P=.89) or suicidal behavior (t_EAR(25)=−1.27, P=.22, U_MAR=41, z=−1.61; P=.12).

However, interviewer nonverbal behavior during the baseline C-SSRS was associated with subsequent suicidal ideation severity and suicidal behavior. We examined the 3 domains of interviewer behavior that differentiated suicidal and nonsuicidal participants at baseline (ie, head motion velocity, EAR velocity, and smiling) and found that interviewer lip corner pullers (ie, AU 12) during the C-SSRS were associated with participants’ suicidal ideation severity and behavior 3 months postbaseline, such that interviewers demonstrated more lip corner-pullers with participants who reported more severe suicidal ideation at follow-up (r=.38; P=.04) and who engaged in subsequent suicidal behavior (U=33, z=−2.39; P=.02). Of note, explicit clinical risk assessment ratings made by the interviewers immediately following the interview were not associated with participants’ subsequent suicidal behavior (t₂₆=−1.27; P=.22) or subsequent suicidal ideation severity (r=−0.04; P=.86). Participants’ baseline ideation severity was also not associated with subsequent suicidal behavior (t₂₅=−1.74; P=.09) or subsequent suicidal ideation severity (r=0.33; P=.09).

We used machine learning–based tools to characterize nonverbal behaviors of both participants and their clinical interviewers during a face-to-face C-SSRS video interview with suicidal and nonsuicidal young adults. Our primary aims were to test (1) whether nonverbal behaviors of participants and interviewers differ in interviews with suicidal versus nonsuicidal young adults and (2) the extent to which nonverbal behaviors can be used to predict group membership. Finally, in a post hoc exploratory analysis, we tested whether any participant or interviewer nonverbal behaviors that differentiated groups at baseline were associated with participants’ subsequent suicidal thoughts and behaviors 3 months later.

This study represents the first effort to our knowledge to characterize both participant and interviewer facial action and head motion using automated coding during a suicide assessment [19,20,22]. While nonverbal behaviors of both young adults and their interviewers differentiated suicidal (vs nonsuicidal) participants, we documented that interviewers’ nonverbal behaviors reflected differences across more dimensions of nonverbal behavior, were just as (if not more) discriminating, and corresponded to participants’ suicidal ideation and behavior 3 months later. In our study, the distress of the participant is thus more readily detected through the clinical interviewer.

While depression severity alone was a strong predictor of suicidal versus nonsuicidal group membership (correctly classifying 80% [52/65] of participants), the inclusion of important nonverbal behaviors of participants and interviewers improved classification accuracy slightly (86% [56/65]). Importantly, the integration of these nonverbal behaviors was especially critical for the detection of suicidal young adults; while depression severity alone correctly identified 75% (25/33) of suicidal participants, the model incorporating participant and interviewer nonverbal behaviors captured 88% (29/33). Our findings also suggest that interviewer nonverbal behaviors are particularly informative in predicting suicidal versus nonsuicidal group membership of participants; across all participant and interviewer nonverbal behaviors assessed, the model containing all interviewer nonverbal behaviors better predicted group membership than did the model containing all participant nonverbal behaviors: interviewer nonverbal behaviors alone identified 81% (27/33) of suicidal participants, while participant nonverbal behaviors alone captured 59% (19/33).

The importance of interviewer nonverbal behavior is further underscored by our exploratory prospective analyses, which revealed that interviewer nonverbal behavior, but not participant nonverbal behavior, was associated with participant suicidal ideation and behavior 3 months later. Strikingly, neither interviewer clinical ratings of participant suicide risk nor participant self-report of ideation severity were associated with suicidal thoughts or behavior 3 months later. Thus, neither interviewer nor participant seemed consciously aware of suicide risk. These interviewer findings replicate for the first time the seminal work of Heller et al [19], who found that an emergency room psychiatrist’s implicit, out-of-awareness nonverbal behavior predicted which of her patients went on to reattempt suicide over the next two years, while her explicit, written risk predictions did not. In both studies, the interviewer knew with her body what she did not know with her words.

In our study, interviewers’ more frequent use of the lip corner-puller (but not the Duchenne smile) was associated with participant suicidal ideation severity and behavior 3 months later. Non-Duchenne smiles, which contain only the lip corner-puller, have historically been referred to as the “masking smile,” “false smile,” “anticipatory smile,” and “miserable smile,” as they are widely thought to conceal negative affect [48]. Although the current state of research on the distinction between Duchenne and non-Duchenne smiling is mixed [49], non-Duchenne smiles may reveal underlying anxiety or ambivalence in certain contexts [50]. Interviewer lip corner-pullers in this context likely reflect a compensatory attempt to manage distress.

Our study contributes to growing empirical support for what has long been proposed by theoreticians: the clinician’s affective experience contains critical information about the patient [51-53]. Further, findings documented here call into question our tendency to focus exclusively on the verbal content in suicide assessments, and our tendency to focus exclusively on the patient. There is a great deal to learn from attending to the nonverbal experience of the clinical interviewer.

Importantly, this study sheds light on the use of machine learning–based tools for nonverbal behavior characterization. It is one of the first to examine the performance of an open-source, freely available software that uses deep learning to characterize facial and head behavior (ie, PyAFAR) on an entirely new dataset. Overall, performance of PyAFAR on our dataset can be considered acceptable for most AUs tested. Agreement between automated and manual coding about the absence of an AU was stronger than agreement about presence, and positive agreement rates were particularly low for the brow-lowerer, lip-depressor, and chin-raiser AUs. This is likely due to relatively low occurrence rates of these AUs in our reliability dataset. Additional intersystem reliability studies with longer video samples and therefore higher occurrence rates are needed to further assess agreement between automated coding and human coding. Still, there are many advantages of automated coding; human coding that took more than two years from training to completion could be produced in days by PyAFAR, with additional, rich information captured. Although PyAFAR does not produce results in real-time, its relative efficiency means that nonverbal behaviors can much more readily be incorporated into psychological research, addressing the current lack thereof that Hall et al [54] refer to as “a study of words without the music.” Our study demonstrates that automated coding has the power to efficiently detect clinically meaningful information exhibited via multiple nonverbal channels.

There are several limitations of this study. First, our sample, though sizable enough to detect effects, is small. Results of the prospective analyses on the suicidal subsample in particular should be interpreted with caution. Additionally, although our suicidal participants all reported past-year suicidal behavior, the degree of suicidality during the interview itself varied, and the number of participants with suicidal behavior at follow-up was small. Future research should seek to replicate this work with a larger sample and with more clinically acute participants, which would also enable a more fine-grained evaluation of possible sociodemographic differences in nonverbal behaviors (eg, by race and ethnicity, gender, education level). Relatedly, generalizability of study findings may be limited by the demographic makeup of our sample, which, while diverse with regard to race and ethnicity and sexual orientation, was predominantly female. There are also important limitations to our study design. All interviews were conducted via videoconference, and interviewers may differ in their ability to make emotional contact with a suicidal participant in this medium. Finally, this study is limited by its focus on each member of the dyad in isolation. Future research would benefit from conducting within-dyad analyses as well as between-group analyses to better understand the interactive processes that occur between interviewers and participants [24].

This study documents the importance of attending to much more than just what patients say in considering suicide risk. Attending to nonverbal behavior holds the potential of improving suicide risk detection. Moreover, attending to the interviewer, as well as the patient, will also improve the study of suicide risk. Finally, the time-intensive nature of quantifying nonverbal behaviors through human coding is side-stepped by the approach to automated affective computing used here, which affords the opportunity to quantify nonverbal information efficiently and systematically, highlighting the potential of transdisciplinary, multimodal approaches in suicide research.