This paper is in the following e-collection/theme issue:

Formative Evaluation of Non-Ehealth Innovations (134) Autism Spectrum Disorder (ASD) (109) Attitudes, Beliefs, and Health Behaviours in Human Factors Research (27) Psychological, -behavioural, -social, and/or -cultural Experiments and Interventions (52) Behavior Change (712)

Published on in Vol 8 (2024)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/55339, first published .
Behavioral Intervention for Adults With Autism on Distribution of Attention in Triadic Conversations: A/B-Tested Pre-Post Study

Behavioral Intervention for Adults With Autism on Distribution of Attention in Triadic Conversations: A/B-Tested Pre-Post Study

Behavioral Intervention for Adults With Autism on Distribution of Attention in Triadic Conversations: A/B-Tested Pre-Post Study

Authors of this article:

Onur Necip Tepencelik1 Author Orcid Image ;   Wenchuan Wei1 Author Orcid Image ;   Mirabel Luo2 Author Orcid Image ;   Pamela Cosman1 Author Orcid Image ;   Sujit Dey1 Author Orcid Image
Article Authors Cited by Tweetations Metrics

    Original Paper

    1Electrical and Computer Engineering, University of California San Diego, La Jolla, CA, United States

    2Carlsbad High School, Carlsbad, CA, United States

    Corresponding Author:

    Onur Necip Tepencelik, MSc

    Electrical and Computer Engineering, University of California San Diego

    UC San Diego

    9500 Gilman Dr

    La Jolla, CA, 92093

    United States

    Phone: 1 (858) 534 2230

    Email: otepence@ucsd.edu


    Background: Cross-neurotype differences in social communication patterns contribute to high unemployment rates among adults with autism. Adults with autism can be unsuccessful in job searches or terminated from employment due to mismatches between their social attention behaviors and society’s expectations on workplace communication.

    Objective: We propose a behavioral intervention concerning distribution of attention in triadic (three-way) conversations. Specifically, the objective is to determine whether providing personalized feedback to each individual with autism based on an analysis of their attention distribution behavior during an initial conversation session would cause them to modify their orientation behavior in a subsequent conversation session.

    Methods: Our system uses an unobtrusive head orientation estimation model to track the focus of attention of each individual. Head orientation sequences from a conversation session are analyzed based on five statistical domains (eg, maximum exclusion duration and average contact duration) representing different types of attention distribution behavior. An intervention is provided to a participant if they exceeded the nonautistic average for that behavior by at least 2 SDs. The intervention uses data analysis and video modeling along with a constructive discussion about the targeted behaviors. Twenty-four individuals with autism with no intellectual disabilities participated in the study. The participants were divided into test and control groups of 12 participants each.

    Results: Based on their attention distribution behavior in the initial conversation session, 11 of the 12 participants in the test group received an intervention in at least one domain. Of the 11 participants who received the intervention, 10 showed improvement in at least one domain on which they received feedback. Independent t tests for larger test groups (df>15) confirmed that the group improvements are statistically significant compared with the corresponding controls (P<.05). Crawford-Howell t tests confirmed that 78% of the interventions resulted in significant improvements when compared individually against corresponding controls (P<.05). Additional t tests comparing the first conversation sessions of the test and control groups and comparing the first and second conversation sessions of the control group resulted in nonsignificant differences, pointing to the intervention being the main effect behind the behavioral changes displayed by the test group, as opposed to confounding effects or group differences.

    Conclusions: Our proposed behavioral intervention offers a useful framework for practicing social attention behavior in multiparty conversations that are common in social and professional settings.

    JMIR Form Res 2024;8:e55339

    doi:10.2196/55339

    Keywords

    autism spectrum condition; social attention; social orienting; behavioral intervention; attention distribution; triadic conversation 


    Background

    Atypical patterns of attention in social interaction settings are a key characteristic of autism spectrum condition (ASC) [1]. Many studies of these patterns involve only two people, either as a face-to-face interaction or as a person-person-object interaction; very few analyze attention behavior in multiparty interactions. In this study, we used a head orientation estimation model to analyze the attention distribution of young adults with autism with no intellectual disabilities in triadic (three-way) conversation settings. Triadic conversations are foundational for understanding patterns in multiparty interactions as having three people in conversation is the smallest number that allows for the study of a speaker’s attention distribution across multiple listeners. In addition to the attention distribution analysis, we present a behavioral intervention framework to provide feedback on behavior relative to normative patterns. The remainder of this section includes a literature review on social attention patterns of individuals with autism and social orienting behavior in triadic interaction settings. Then, we present our experimental setting, the behavioral intervention program and results, and a discussion.

    Social Attention

    Social attention is the ability and motivation to attend to a social partner during interaction, as well as coordinating attention with them [2]. Social attention includes joint attention, social orienting, eye contact, nonverbal gestures, and following social cues, such as eyes, faces, hands, and voice. Differences in social attention are a core diagnostic feature of ASC [1]. A common practice in social attention research involves presenting participants with social and nonsocial cues and analyzing their responses, enabling comparisons between different types of cues as well as populations. Many studies showed that, during face-to-face interactions, individuals with autism spend less time looking at social stimuli such as an experimenter’s face and more time looking at nonsocial stimuli such as the background or objects present in the environment than participants without autism [1-3]. The gaze behavior of children with autism and neurotypical (NT) children was compared while they watched videos that they believed to be prerecorded or live and while engaging in real social interactions [4]. The authors found that, during real social interactions, children with low autism spectrum quotient (AQ) scores looked more at the experimenter than did children with high AQ scores, while for videos, children with low AQ scores looked less at the person in the live video than in the prerecorded video, compared with children with high AQ scores. In a study by Speer et al [5], participants were presented with four types of stimuli: social dynamic, social static, isolated dynamic, and isolated static. Participants with autism exhibited significant differences under the social dynamic stimuli with decreased fixation duration on eyes and increased fixation duration on the body. Many studies on social attention patterns involve laboratory settings where social stimuli are presented digitally, such as displaying social scenes or videos on a computer screen [3,5-8]. Such laboratory interactions can lead to very different results, both qualitatively and quantitatively, compared with live interactions [9-11] in part because human eyes can communicate information as well as receive it, a dual aspect that is missing when viewing others on a screen [12]. Hence, it is important to study social communication through scenarios where a social partner is physically present [13]. Over the last decade, there has been an increase in real interaction settings to study social communication. In this study, we present a live naturalistic social interaction setting where the attention patterns of individuals with autism are examined, enabled by the unobtrusive orientation estimation system presented in our previous study [14].

    Joint attention is the ability to achieve a common focus of attention with another person during a social interaction [7] and is one of the most well-studied diagnostic features of ASC [15-18]. These studies and others showed that starting from infancy, individuals with autism participate less in joint attention. More recently, Caruana et al [7] showed that adults with autism continue to have differences in joint attention, with fewer adults with autism responding to joint attention cues in their experiment, while also being slower to respond than IQ-matched controls.

    Triadic interactions are significantly harder to navigate than dyadic (two-way) interactions [19]. In a dyadic interaction, objects and the background are potential distractions. A triadic interaction has those as well as an additional person who is part of the interaction and may require attention. Prior research on triadic interactions involving individuals on the autism spectrum is limited but insightful. McParland et al [20] studied triadic conversations with low communicative intent (researchers speaking primarily with each other, with occasional input from a child) and dyadic conversations with high communicative intent (a researcher directly interacting with a child). The authors found that children with autism spent 12.3% less time looking at other people’s faces in these triadic conversations than the dyadic ones, and 9.7% less than typically developing (TD) children. Authors also found that children with autism made 57% more gaze fixations to people’s faces in these triadic conversations than the dyadic ones, while TD children displayed the reverse pattern, with 61% more gaze fixations in the dyadic conversations. In our study, we aimed to create a realistic, highly communicative setting where the individual with autism is engaging with other parties continuously.

    Social orienting is a rapid and involuntary attentional shift, typically by a change in gaze direction or head orientation in response to a social cue [21]. Studying the ability to follow another’s eye gaze is a common method of analyzing social orienting skills [22] and is a differentiating factor in autism [13,23-25]. Several studies analyzed head movements of people with autism [6,8,18,26,27]. Dawson et al [18] examined responses to various social cues such as hearing their own name or mother’s voice and concluded that children with autism are significantly less likely to respond to these cues with a reorientation of the head compared with TD children. Campbell et al [8] showed that fewer children with autism oriented to their names compared with TD children, and when they did respond by orienting, it took longer. Zhao et al [26] proposed a model to analyze head movement features such as rotation range and frequency in children with autism without intellectual disabilities during face-to-face interactions. The authors reported that children with autism had a significantly higher level of head movement stereotypy (repetitive, ritualistic head movements), as well as higher rotation range and frequency than TD children. Based on these results, the authors developed a machine learning model using the proposed head movement features to diagnose autism in children [27].

    Many studies analyzed attention in conversational settings, including the social modulation of gaze, which is the change in gaze orientation based on conversational role (eg, speaker or listener). In dyadic conversations, listeners generally gaze more at speakers compared with speakers looking at listeners [28-31]. However, Vertegaal et al [31] found that in group conversations, the gaze levels of speakers come close to those of listeners. The authors argued that one reason for this change was that speakers, when addressing a group, need to collect visual feedback from each individual and to maintain the signal that they are addressing each individual. In a similar conversational role comparison in a face-to-face setting, it was found that when the experimenter looked directly at the participant, adults with autism looked back at the experimenter’s face less than NT adults did, missing opportunities for reciprocal social gaze [32]. Bal et al [33] showed that individuals with autism might not regularly provide normative nonverbal communication cues, such as periodically making eye contact with a speaker and maintaining a body orientation generally toward them. In a recent study [34] of social attention over longer time spans, it was found that NT individuals display an initial increase in social attention, followed by a decay and a recovery, whereas high functioning individuals with autism exhibit a constant linear decay without any recovery of attention. In a web-based public-speaking experiment [35], it was found that children with autism with no intellectual disability made contact with the listeners less frequently than TD children. In our study, the individual with autism is the primary speaker of the triadic interaction, which leads to an analysis of attention distribution for the speaking phase of a conversation.

    The social attention differences discussed in this section could be attributed to reduced cognitive functioning or IQ, instead of differences in autistic traits. However, multiple studies indicated that reduced social attention in ASC is orthogonal to IQ [34]. Multiple meta-analytic studies [1,36] suggest that differences in social attention are not modulated by IQ matching between the NT and ASC groups and remain stable across IQ differences between groups [37]. Furthermore, various studies [38-40] involving control groups with developmentally delayed individuals showed that individuals with autism exhibit reduced social attention compared with both developmentally delayed and NT groups. These findings highlight the importance of addressing social attention differences in ASC, regardless of intellectual ability.

    Many studies outline the impact of these kinds of social attention differences. According to Chen et al [41], differences from society’s workplace communication norms are one reason that adults with autism have high unemployment rates despite often holding college degrees, average to high IQs, and various useful skills. Furthermore, it was reported [42] that many adults were terminated from jobs due to communication differences. For the large number of individuals with autism aging into adulthood each year, effective systems that allow situational practice and feedback of social attention could support successful transition to employment. One goal of this intervention study was to provide useful feedback to adults with autism who are seeking employment.

    Positive results were reported in various intervention studies involving individuals with autism. Video-based interventions were found effective in improving social communication skills [43]. Munandar et al [44] proposed a video-based intervention for college students with autism to improve their storytelling ability during job interviews, which led to positive results. Ferguson et al [45] initiated an intervention-based program for adults with autism with co-occurring intellectual disability, addressing verbal conversational skills and nonverbal behaviors such as eye contact and active listening. Promising behavioral results were reported for all pilot participants after the interventions, as well as overall positive feedback from the parents of participants. In a similar intervention program with eight sessions [46], children with autism with no intellectual disability showed significant improvements in eye contact and facial emotion recognition.

    Social Orienting in Triadic Conversations

    In this study, we examine triadic conversations, a complex environment that requires keeping track of social stimuli from the other two conversational parties simultaneously and distributing attention between them. Social orienting toward a speaker is important to show interest in the conversation, and distributing attention between the other two parties while speaking can help make everyone feel included in the conversation. Joint attention is often required as well if a conversational partner initiates it toward an external point of focus.

    Triadic interactions can be hard to navigate for both people with and those without autism. Auer [47] argued that one of the participants in a three-party constellation is in danger of being marginalized and may even become a bystander in extreme cases. Multiple studies [47,48] pointed out the importance of using practices such as “alternating gaze” to avoid schisms or marginalization of participants. Zima [49] argued that gaze patterns in triadic interactions are more diverse and complicated than in dyadic interactions. The author found that the gaze window pattern defined by Bavelas et al [50] occurs less frequently in triadic interactions, pointing to the difficulty of using interactive feedback in triadic storytelling activities. Arndt et al [51] showed that salespeople tend to share mutual gaze with customers who are similar to them and those who speak more; however, sharing mutual gaze equitably between customers rather than focusing on a particular customer helps with building trust and rapport. Vertegaal et al [52] argued that lack of gaze can decrease turn-taking efficiency in multiparty interactions by 25%. Many studies [53-56] showed that children were mainly excluded in pediatric visits involving triadic conversations with a parent and a pediatrician. Van Dulmen [53] argued that children could be capable of providing reliable information and feedback, and pediatricians might benefit from training to better communicate with children. These examples and others show that triadic interactions are an important setting to study both because they are common in social and professional settings and because they can act as a building block for behavior analysis of general multiparty interactions. Being able to navigate complex social interaction settings, such as triadic conversations, is important during job interviews to obtain employment or in workplaces to retain employment.

    The main objective of this study is to examine the head orientations of young adults with autism during triadic conversations, before and after the receipt of a behavioral intervention. We use the privacy-preserving LiDAR-based head orientation estimation system introduced in our previous study [14]. The model works from a surveillance viewpoint at an unobtrusive distance, with sensors at two ceiling corners about 10 ft from the participants, enabling orientation analysis for real social interaction settings without the need for wearable devices or digitally created social stimuli. In our previous study [57], we showed that the orientation estimation models can reveal differences between participants with and those without autism in various parameters related to attention distribution and social orienting. In this study, we build on that and present a system that analyzes a triadic conversation session and provides personalized feedback aimed at improving various behaviors related to social attention. In a subsequent session, we study the effectiveness of the intervention.

    Some of the studies presented earlier connect to this study with their focus on head movements [8,26,27], triadic interaction settings [20], or conversational settings [31-33]. However, none of these studies focus on attention distribution and head orientations in multiparty conversations. This work presents an understudied social attention setting, real triadic conversations, and presents a novel head orientation and attention distribution analysis and behavioral intervention framework regarding social communication differences in this setting.


    Participants

    Twenty-four individuals with autism, none of whom had intellectual disabilities, were involved in this intervention study. Twelve individuals with autism and eight individuals without autism with no intellectual disabilities were involved in our previous study [57], which helped design the intervention. Participants without autism were a mix of undergraduate and graduate students aged between 18 and 28 years at the time of participation. All 36 participants with autism for both the current and previous studies were recruited through a neurodiversity workforce training program organized at the University of California (UC) San Diego, which hosted two-month internship programs for employment-seeking individuals with autism every year since 2018. The interns were selected through interviews with more than 100 applicants. As per the inclusion criteria for our studies, participants with autism were at least 18 years of age and had previously received a community diagnosis of ASC. Except for a participant who was 43 years of age, the participant ages ranged from 18 to 28 years (mean 23.8 years, including the 43-year-old participant), with varying educational backgrounds. All participants were high school graduates, and most were in the process of additional education including at vocational schools, community colleges, and universities. Roughly half of the participants had some previous job experience, although it was generally short-term or part-time.

    The 24 participants with autism who took part in the intervention study are split into test and control groups, with 12 participants each. The study consists of three parts for the test group: an 8- to 12-minute initial triadic conversation, an intervention, and another 8- to 12-minute triadic conversation. The participants in the control group did not receive the intervention, taking part in two consecutive 8- to 12-minute triadic conversations.

    First Conversation Session and Initial Evaluation

    Participants converse while seated around an oval table; one interviewer is across from the participant and the other is seated to the side, so a head turn by the participant from one interviewer to the other corresponds to about 45 degrees as shown in Figure 1. The conversation is conducted in a semistructured interview style [58], which starts with the following question: “What do you do in your free time?” and continues based on the answers of the participant. The interviewers are instructed to make sure that the participant is the main speaker throughout the conversation, while they listen in an engaged way, make brief comments, ask follow-up questions, and shift topics when necessary.

    Throughout the conversation, the participant’s head orientation is estimated and recorded as a sequence using the model from our previous study [14]. Multiple studies [52,59,60] showed that head orientation is a good indicator of visual focus of attention, without the need to estimate gaze orientation. To analyze the participant’s attention distribution, we define two events based on their duration, contact and exclusion. Although there are studies of gaze patterns or orientation behaviors in triadic interactions, none define specific communication events pertaining to excluding participants and instead rely on qualitative analyses. Multiple studies [47,48] discussed the danger of exclusion in triadic interactions but did not specify how long it takes for someone to be considered excluded. We use the definitions of these events from our previous study [57]; the participant makes contact with an interviewer if their head orientation corresponds to that interviewer’s region (spanning 15 degrees to either side of the interviewer; see Figure 1) for three consecutive frames (about 2.5 seconds, given the frame rate of about 1.3 fps). Similarly, an exclusion happens in a 20-frame window where at least 15 frames fall in the region of one interviewer and none in the region of the other. To estimate the head orientations, we developed a neural network regression model that uses handcrafted geometric features extracted from human point clouds. The estimated head orientation sequence is analyzed continuously on a rolling basis; a contact or an exclusion is initiated as soon as one of the required conditions happens and is terminated as soon as the condition no longer holds. As presented in our previous study [57], these two events capture important aspects of social communication and can characterize distinct patterns of attention for individuals with and those without autism, covering two important parameters of gaze, duration, and frequency, as defined by Kleinke [30]. Table 1 shows the five domains for which the head orientation sequence is analyzed, the averages for these according to data from 8 participants without autism and 12 participants with autism, as well as the SD among the participants without autism and the values corresponding to 2 SDs from the mean [57]. We also showed in our previous study [57] that some of the behavioral differences between the groups with and those without autism shown in Table 1 are statistically significant.

    The duration, pace, topics, and conversational roles were kept very similar to prevent these external factors from confounding the results. Each participant was asked questions about the same set of topics in the same order. The interviewers asked only short questions and made brief comments when necessary, making the participant the main speaker of the conversation. In all 48 conversation sessions, the participant was the speaker 81%-93% of the time, disregarding the back channels and considering only those sequences where a speaker is talking for at least 2 seconds.

    ‎
    Figure 1. Triadic conversation setup. The participant’s head orientation is estimated continuously, and their focus of attention is analyzed using the interviewer regions depicted.
    Table 1. Statistical domains for attention-related behaviors for which the interventions are provided if a participant exceeds nonautistic average ±2 SDs. Numbers are in terms of frames for AvgCon, AvgNoCon and MaxExc: 1 frame≈0.75 seconds. Em dashes (—) represent directions for which no intervention was given.
    StatisticNAa average (SD)ASCb averageNA average + 2 SDsNA average – 2 SDs
    Average duration of contact (AvgCon)6.09 (0.94)7.737.974.21
    Average duration of NOT making contact with anyone (AvgNoCon)30.95 (15.11)58.661.17—
    Number of contacts per minute (NumCon)2.14 (0.47)1.44—1.21
    Maximum exclusion duration (MaxExc)9.38 (5.89)20.321.16—
    Total duration of exclusion percentage per session (ExcPct)3.13% (1.85%)7.4%6.83%—

    aNA: nonautistic.

    bASC: autism spectrum condition.

    Behavioral Intervention

    The intervention session was conducted by a researcher who was trained by a behavioral coach. The discussion starts with the researcher explaining the experimental setup and its purpose, including information on the importance of distributing attention in triadic interactions, using body and head orientations to facilitate engagement, and our orientation estimation models and tools to analyze attention distribution. The researcher also provides an example of an unwanted situation, where two parties of a triadic conversation orient directly toward each other and exclude the third person from the interaction. The researcher encourages the participant to reflect on their session, with questions such as “Do you think you made any social exclusions?” This part of the intervention is orchestrated identically for all participants in the test group.

    Based on the head orientation analysis for the first conversation session, for a given domain, additional intervention (data analysis and video examples) is provided to participants in the test group if they deviate from the NA mean by at least 2 SDs. We found in our previous study [57] that, on the five metrics shown in Table 1, the average value for the participants with autism differed from the average NA score by about 2 SDs. Note that for most metrics, deviations to only one side of the NA average are considered for intervention, since making more contacts per minute or making no exclusions at all is not a behavior that would suggest feedback. Each participant was provided feedback on up to three domains. If a participant exceeded the threshold of 2 SDs on more than three domains, the domains where the deviations are the largest were selected for the intervention.

    Next, the data analysis from the initial session is shared with the participant, focusing on up to three domains where the participant deviated from the NA average by more than 2 SDs. The participant is informed about the NA average and by how much they deviated from the NA average. The researcher helps the participant think about their attention distribution patterns during the first session with questions such as: “Why do you think your average contact duration was high?” or “Did you think you excluded one of the listeners?” Finally, participants are shown examples of attention distribution videos, taken from triadic conversation sessions involving participants without autism. Each domain has a different associated video that demonstrates the social attention aspect on which a participant is receiving intervention. While the participant watches the video, the researcher asks additional open-ended questions such as: “What do you notice about the participant’s head orientations in the video?” Each participant who received an intervention completed the same steps of discussion, data analysis, and video example detailed in this section. In total, the intervention session took 10-15 minutes. For participants in the control group, no intervention is given; there was merely a short break between the two conversation sessions.

    Second Conversation Session

    Participants took part in a second conversation session, similar to the first. Upon completion of the second conversation session, the researcher asked the test participant about how it felt to do a second session after the intervention, and whether they did anything differently. Finally, all participants were presented with the option to review and discuss their statistics and head orientation analysis.

    Ethical Considerations

    This study was approved by the UC San Diego Institutional Review Board (Protocol 210775; July 1, 2021). Participants gave their informed consent to the research team to analyze their individual data by signing a consent form describing the study and the data collection procedure. The privacy of the participants is protected through the assignment of deidentifying codes to each participant throughout this manuscript. Participants received US $15 for an hour of participation as compensation.


    Our chosen thresholds meant that 11 of 12 test participants received intervention on at least one domain, with most participants receiving interventions on two or three domains. Participant 8 did not receive an intervention as their behavior was consistent with NT norms for all five domains. The most common domains for intervention were related to exclusions, “maximum exclusion duration” (MaxExc) and “total duration of exclusion percentage per session” (ExcPct), with the thresholds suggesting feedback for 8 and 6 participants, respectively. For contact-related statistics, 6 participants received intervention on high “average duration of contact” (AvgCon) and 1 participant received intervention on low AvgCon, 4 participants received intervention on “number of contacts per minute” (NumCon), and 2 participants received intervention on “average duration of not making contact with anyone” (AvgNoCon). Figure 2 shows the data for exclusion-related domains, in comparison with the NA averages and the intervention region (2 SDs from the NA mean). Figures 2A and 2C show data for the test participants. Those participants receiving intervention for that domain (left side of plot) are marked with red and green dots, corresponding to their scores during the first and second conversation sessions. The test participants who did not receive intervention for that domain (right side of plot) are marked with magenta and cyan dots for their first and second sessions, as also is the case in Figures 2B and 2D, showing the control group. Figure 3 shows the data for Contact related statistics. We compared the performances in the first conversation session for the test and control groups to check for between-group differences; no significant difference was observed in any domain (P>.10 for all five domains).

    Figure 2A shows the results for MaxExc. Eight participants exhibited a maximum exclusion duration higher than the NA average by at least 2 SDs in their first session, meaning that one of the interviewers was excluded from the conversation for a long period of time, at least once. Hence, these participants received intervention about this behavior. According to an independent 1-tailed t test, the changes in MaxExc for the group of 8 participants who received the intervention are statistically significant compared with the control group, t18=−3.3; P=.004. To assess individual improvements, we use 1-tailed Crawford-Howell t tests [61-63], which allow for single case comparisons against a control group. This test accounts for the small size of the control group and the uncertainties over its mean and SD [62]. According to 1-tailed Crawford-Howell t tests, 6 participants out of the 8 who received the intervention made statistically significant improvements in MaxExc compared with the control group (exceptions are participants 6 and 10).

    Figure 2C shows the results for ExcPct. All participants apart from participant 6 made statistically significant improvements according to 1-tailed Crawford-Howell t tests. As a group, the 6 participants who received the intervention made statistically significant improvements in ExcPct compared with the control group, t16=−3.4; P=.004.

    Results are generally similar for the contact statistics. Figure 3A shows results for AvgCon for which 6 participants received intervention due to a high AvgCon and 1 received intervention due to a low AvgCon. As a group they made significant improvements and all but 2 (participants 6 and 10) improved compared with the control group. Similarly, Figure 3C presents results for NumCon, where 4 participants received intervention for having a low NumCon. We drop the independent t test for smaller intervention groups, but 1-tailed Crawford-Howell t tests confirm that 3 participants out of the 4 who received the intervention made significant improvements in NumCon, compared with the control group. Only 2 participants received intervention on AvgNoCon, both of whom improved according to 1-tailed Crawford-Howell t tests compared with the control group, as displayed in Figure 3E.

    Figure 4 displays cumulative results. In total, our model proposed an intervention on 27 domains, and 21 (78%) of these intervention cases resulted in statistically significant improvements in the second conversation session compared with the control group. For 15 intervention cases (56%), the participants not only showed significant improvements but also moved to the no-intervention region. Of the 11 participants who received intervention, 10 showed significant improvement in at least one domain on which they received feedback.

    We conducted statistical tests on the control group to check whether external factors such as familiarity with the interviewers or the conversation setting might have had a significant effect. For all five domains, the changes between the first and second conversation sessions are nonsignificant for the control group (P>.10 for all five domains).

    ‎
    Figure 2. Participant data for exclusion-related behaviors in first and second conversation sessions for test and control groups. (A) Maximum exclusion duration (MaxExc) statistics for the test group. (B) MaxExc statistics for the control group. (C) Exclusions percentage (ExcPct) statistics for the test group. (D) ExcPct statistics for the control group. NA: nonautistic.
    ‎
    Figure 3. Participant data for contact-related behaviors in first and second conversation sessions for test and control groups. (A) Average duration of contact (AvgCon) statistics for the test group. (B) AvgCon statistics for the control group. (C) Number of contacts per minute (NumCon) statistics for the test group. (D) NumCon statistics for the control group. (E) Average duration of not making contact with anyone (AvgNoCon) statistics for the test group. (F) AvgNoCon statistics for the control group. NA: nonautistic.
    ‎
    Figure 4. Cumulative statistics of interventions provided to participants. Interventions were provided on 27 statistical domains and 21 of those resulted in statistically significant improvements. NA: nonautistic.

    Principal Findings

    In this study, we propose a behavioral intervention based on a social attention analysis in triadic conversations using the estimated head orientations of the participants. In our previous study [57], differences in attention distribution between individuals with and those without autism were analyzed, and metrics related to exclusions and contact were quantified for the 2 groups. Using these metrics, we designed a study where participants receive personalized feedback on their attention patterns. Twelve participants on the autism spectrum formed the test group, of whom 11 received interventions on specific domains and 10 showed improvement in the postintervention session. The numerical results in the postintervention session suggest that the intervention session was effective compared with the control group of 12 individuals with autism who did not receive an intervention.

    In this study, the intervention session was conducted by the researcher who was trained by a behavioral coach. As the first iteration of this study, having the researcher present during the intervention session enabled us to answer the questions of the participants, observe their reactions, and potentially refine the approach. We concluded that the information presented during the intervention discussion is clearly understandable by the participants. Poststudy discussions with intervention recipients revealed that 4 participants found the discussion with the researcher to be the most informative and impactful way of delivering the intervention, while 3 preferred the data analysis part and 1 participant found the video example the most useful. These results suggest that providing feedback in multiple ways can be helpful and different people may prefer different forms of feedback.

    The results of this study suggest that an intervention on orientation behavior may enable individuals with autism to modify their attention behavior to become more similar to NT norms of social attention. Such adjustments could, in turn, help these individuals obtain and retain employment. As more companies move in the direction of making their interview practices and workplace environments friendly for applicants with autism and employees, the need for individuals with autism to adjust their own social communication patterns may decline. For the foreseeable future, however, some adjustment to fit in with the social communication norms of workplaces is likely to remain important for people on the autism spectrum to find and retain employment [41,42]. Many intervention studies [43-46] were proposed and achieved positive results in helping individuals with autism make those adjustments, focusing on different aspects of social attention and targeting different age groups. In this study, we focus on attention distribution and orientation behavior during triadic interactions, which is an understudied experimental setting. Our study is designed for adults with autism with no intellectual disabilities, which is a relatively undertargeted population compared with children or individuals with autism with intellectual disabilities. To the best of our knowledge, this intervention study is the first to use this experimental setting and target this population.

    Limitations and Future Work

    The main limitation of this study is the small number of participants involved. The thresholds for determining when to provide an intervention were created using the data from 8 participants without autism, while the main study involved 24 participants with autism. The Crawford-Howell t test accounts for the small sample size of the control group.

    The intervention sessions during this study were conducted with in-person feedback. In future iterations of this work, we aim to provide this feedback in a fully automated way, so that the method can be easily replicated and users can use the proposed models as self-practice tools. We will also address the low number of participants.

    Conclusions

    We proposed a triadic conversation study to provide a behavioral intervention on social attention patterns to individuals on the autism spectrum. This triadic conversation study was successful in providing feedback to participants both objectively and subjectively. Comparison between the first session and the postintervention session shows clear improvements, confirmed by significance tests in comparison with the control group. The proposed triadic conversation setup and feedback model can enable situational practice for triadic interactions, which are common in both social and professional settings.

    Acknowledgments

    This work was supported by the National Science Foundation under (grant DUE-1928604). We thank Shana Cohen and Trent Simmons for helpful discussions and Saygin Artiran for helping with data collection. We would also like to thank our participants for their involvement.

    Data Availability

    The data sets generated and analyzed during this study are not publicly available due to privacy restrictions.

    Conflicts of Interest

    None declared.

    1. Chita-Tegmark M. Social attention in ASD: A review and meta-analysis of eye-tracking studies. Res Dev Disabil. 2016;48:79-93. [CrossRef] [Medline]
    2. Barzy M, Ferguson HJ, Williams DM. Perspective influences eye movements during real-life conversation: Mentalising about self versus others in autism. Autism. 2020;24(8):2153-2165. [FREE Full text] [CrossRef] [Medline]
    3. Kaliukhovich DA, Manyakov NV, Bangerter A, Pandina G. Context modulates attention to faces in dynamic social scenes in children and adults with autism spectrum disorder. J Autism Dev Disord. 2022;52(10):4219-4232. [FREE Full text] [CrossRef] [Medline]
    4. von dem Hagen EA, Bright N. High autistic trait individuals do not modulate gaze behaviour in response to social presence but look away more when actively engaged in an interaction. Autism Res. 2017;10(2):359-368. [FREE Full text] [CrossRef] [Medline]
    5. Speer LL, Cook AE, McMahon WM, Clark E. Face processing in children with autism: Effects of stimulus contents and type. Autism. 2007;11(3):265-277. [CrossRef] [Medline]
    6. Martin KB, Hammal Z, Ren G, Cohn JF, Cassell J, Ogihara M, et al. Objective measurement of head movement differences in children with and without autism spectrum disorder. Mol Autism. 2018;9:14. [FREE Full text] [CrossRef] [Medline]
    7. Caruana N, Stieglitz Ham H, Brock J, Woolgar A, Kloth N, Palermo R, et al. Joint attention difficulties in autistic adults: an interactive eye-tracking study. Autism. 2018;22(4):502-512. [CrossRef] [Medline]
    8. Campbell K, Carpenter KL, Hashemi J, Espinosa S, Marsan S, Borg JS, et al. Computer vision analysis captures atypical attention in toddlers with autism. Autism. 2019;23(3):619-628. [FREE Full text] [CrossRef] [Medline]
    9. Laidlaw KEW, Foulsham T, Kuhn G, Kingstone A. Potential social interactions are important to social attention. Proc Natl Acad Sci USA. 2011;108(14):5548-5553. [FREE Full text] [CrossRef] [Medline]
    10. Risko EF, Laidlaw K, Freeth M, Foulsham T, Kingstone A. Social attention with real versus reel stimuli: Toward an empirical approach to concerns about ecological validity. Front Hum Neurosci. 2012;6:143. [FREE Full text] [CrossRef] [Medline]
    11. Freeth M, Foulsham T, Kingstone A. What affects social attention? Social presence, eye contact and autistic traits. PLoS One. 2013;8(1):e53286. [FREE Full text] [CrossRef] [Medline]
    12. Risko EF, Richardson DC, Kingstone A. Breaking the fourth wall of cognitive science. Curr Dir Psychol Sci. 2016;25(1):70-74. [FREE Full text] [CrossRef]
    13. Freeth M, Morgan E, Bugembe P, Brown A. How accurate are autistic adults and those high in autistic traits at making face-to-face line-of-sight judgements? Autism. 2020;24(6):1482-1493. [FREE Full text] [CrossRef] [Medline]
    14. Tepencelik ON, Wei W, Chukoskie L, Cosman PC, Dey S. Body and head orientation estimation with privacy preserving lidar sensors. IEEE; 2021. Presented at: 2021 29th European Signal Processing Conference (EUSIPCO); 2021 Aug 27:766-770; Dublin, Ireland. [CrossRef]
    15. Mundy P, Sigman M, Kasari C. A longitudinal study of joint attention and language development in autistic children. J Autism Dev Disord. 1990;20(1):115-128. [CrossRef] [Medline]
    16. Mundy P, Sigman M, Kasari C. Joint attention, developmental level, and symptom presentation in autism. Dev Psychopathol. 2008;6(3):389-401. [FREE Full text] [CrossRef]
    17. Charman T. Why is joint attention a pivotal skill in autism? Philos Trans R Soc Lond B Biol Sci. 2003;358(1430):315-324. [FREE Full text] [CrossRef] [Medline]
    18. Dawson G, Toth K, Abbott R, Osterling J, Munson J, Estes A, et al. Early social attention impairments in autism: Social orienting, joint attention, and attention to distress. Dev Psychol. 2004;40(2):271-283. [CrossRef] [Medline]
    19. Greene MG, Adelman RD. Beyond the dyad: Communication in triadic (and more) medical encounters. In: Martin LR, Robin DiMatteo M, editors. The Oxford Handbook of Health Communication, Behavior Change, and Treatment Adherence. England. Oxford University Press; 2013:136-154.
    20. McParland A, Gallagher S, Keenan M. Investigating gaze behaviour of children diagnosed with autism spectrum disorders in a classroom setting. J Autism Dev Disord. 2021;51(12):4663-4678. [FREE Full text] [CrossRef] [Medline]
    21. Morrisey MN, Reed CL, McIntosh DN, Rutherford MD. Brief report: Attentional cueing to images of social interactions is automatic for neurotypical individuals but not those with ASC. J Autism Dev Disord. 2018;48(9):3233-3243. [FREE Full text] [CrossRef] [Medline]
    22. Frischen A, Bayliss AP, Tipper SP. Gaze cueing of attention: Visual attention, social cognition, and individual differences. Psychol Bull. 2007;133(4):694-724. [FREE Full text] [CrossRef] [Medline]
    23. Leekam SR, Hunnisett E, Moore C. Targets and cues: Gaze-following in children with autism. J Child Psychol Psychiatry. 1998;39(7):951-962. [Medline]
    24. Gillespie-Lynch K, Elias R, Escudero P, Hutman T, Johnson SP. Atypical gaze following in autism: A comparison of three potential mechanisms. J Autism Dev Disord. 2013;43(12):2779-2792. [FREE Full text] [CrossRef] [Medline]
    25. Congiu S, Fadda R, Doneddu G, Striano T. Impaired representational gaze following in children with autism spectrum disorder. Res Dev Disabil. 2016;57:11-17. [CrossRef] [Medline]
    26. Zhao Z, Zhu Z, Zhang X, Tang H, Xing J, Hu X, et al. Atypical head movement during face-to-face interaction in children with autism spectrum disorder. Autism Res. 2021;14(6):1197-1208. [CrossRef] [Medline]
    27. Zhao Z, Zhu Z, Zhang X, Tang H, Xing J, Hu X, et al. Identifying autism with head movement features by implementing machine learning algorithms. J Autism Dev Disord. 2022;52(7):3038-3049. [CrossRef] [Medline]
    28. Kendon A. Some functions of gaze-direction in social interaction. Acta Psychol (Amst). 1967;26(1):22-63. [CrossRef] [Medline]
    29. Argyle M, Cook M. Gaze and Mutual Gaze. Cambridge. Cambridge University Press; 1976.
    30. Kleinke CL. Gaze and eye contact: A research review. Psychol Bull. 1986;100(1):78-100. [Medline]
    31. Vertegaal R, Slagter G, Van der Veer G, Nijholt A. Eye gaze patterns in conversations: There is more to conversational agents than meets the eyes. 2001. Presented at: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems; 2001:301-308; New York, NY. URL: https://doi.org/10.1145/365024.365119 [CrossRef]
    32. Freeth M, Bugembe P. Social partner gaze direction and conversational phase; factors affecting social attention during face-to-face conversations in autistic adults? Autism. 2019;23(2):503-513. [FREE Full text] [CrossRef] [Medline]
    33. Bal VH, Kim SH, Fok M, Lord C. Autism spectrum disorder symptoms from ages 2 to 19 years: Implications for diagnosing adolescents and young adults. Autism Res. 2019;12(1):89-99. [FREE Full text] [CrossRef] [Medline]
    34. Hedger N, Chakrabarti B. Autistic differences in the temporal dynamics of social attention. Autism. 2021;25(6):1615-1626. [FREE Full text] [CrossRef] [Medline]
    35. Jarrold W, Mundy P, Gwaltney M, Bailenson J, Hatt N, McIntyre N, et al. Social attention in a virtual public speaking task in higher functioning children with autism. Autism Res. 2013;6(5):393-410. [FREE Full text] [CrossRef] [Medline]
    36. Hedger N, Dubey I, Chakrabarti B. Social orienting and social seeking behaviors in ASD. A meta analytic investigation. Neurosci Biobehav Rev. 2020;119:376-395. [CrossRef] [Medline]
    37. Frazier TW, Strauss M, Klingemier EW, Zetzer EE, Hardan AY, Eng C, et al. A meta-analysis of gaze differences to social and nonsocial information between individuals with and without autism. J Am Acad Child Adolesc Psychiatry. 2017;56(7):546-555. [FREE Full text] [CrossRef] [Medline]
    38. Klin A, Lin DJ, Gorrindo P, Ramsay G, Jones W. Two-year-olds with autism orient to non-social contingencies rather than biological motion. Nature. 2009;459(7244):257-261. [FREE Full text] [CrossRef] [Medline]
    39. Shic F, Bradshaw J, Klin A, Scassellati B, Chawarska K. Limited activity monitoring in toddlers with autism spectrum disorder. Brain Res. 2011;1380:246-254. [FREE Full text] [CrossRef] [Medline]
    40. Chawarska K, Macari S, Shic F. Context modulates attention to social scenes in toddlers with autism. J Child Psychol Psychiatry. 2012;53(8):903-913. [FREE Full text] [CrossRef] [Medline]
    41. Chen JL, Leader G, Sung C, Leahy M. Trends in employment for individuals with autism spectrum disorder: A review of the research literature. Rev J Autism Dev Disord. 2014;2(2):115-127. [CrossRef]
    42. Muller E, Schuler A, Burton BA, Yates GB. Meeting the vocational support needs of individuals with Asperger syndrome and other autism spectrum disabilities. J Vocat Rehabil. 2003;18(3):163-175.
    43. Laver K, Wilkes-Gillan S. Video modelling interventions improve social communication skills for individuals with autism spectrum disorder. Aust Occup Ther J. 2018;65(4):340-341. [CrossRef] [Medline]
    44. Munandar VD, Bross LC, Zimmerman KN, Morningstar ME. Video-based intervention to improve storytelling ability in job interviews for college students with autism. Career Dev Transit Except Individ. 2020;44(4):203-215. [FREE Full text] [CrossRef]
    45. Ferguson EF, Drapalik KN, Liang J, Hua K, Feerst H, Mallory AB, et al. Social interaction skill intervention for autistic adults with intellectual disability and limited language: A pilot of the SKILL program. J Autism Dev Disord. 2021;51(5):1641-1657. [CrossRef] [Medline]
    46. Yun SS, Choi JS, Park SK, Bong GY, Yoo HJ. Social skills training for children with autism spectrum disorder using a robotic behavioral intervention system. Autism Res. 2017;10(7):1306-1323. [CrossRef] [Medline]
    47. Auer P. Gaze, addressee selection and turn-taking in three-party interaction. In: Oben B, Brône G, editors. Eye-Tracking in Interaction: Studies on the Role of Eye Gaze in Dialogue. Amsterdam, Netherlands. John Benjamins Publishing Company; 2018:197-231.
    48. Rühlemann C, Gee M, Ptak A. Alternating gaze in multi-party storytelling. Journal of Pragmatics. 2019;149:91-113. [FREE Full text] [CrossRef]
    49. Zima E. Gaze and recipient feedback in triadic storytelling activities. Discourse Process. 2020;57(9):725-748. [CrossRef]
    50. Bavelas JB, Coates L, Johnson T. Listener responses as a collaborative process: The role of gaze. J Commun. 2002;52(3):566-580. [FREE Full text] [CrossRef]
    51. Arndt AD, Khoshghadam L, Evans K. Who do I look at? Mutual gaze in triadic sales encounters. J Bus Res. 2020;111:91-101. [FREE Full text] [CrossRef]
    52. Vertegaal R, Van der Veer G, Vons H. Effects of gaze on multiparty mediated communication. 2000. Presented at: Graphics Interface; 2000 May 15:95-102; Montréal, Québec, Canada.
    53. van Dulmen AM. Children's contributions to pediatric outpatient encounters. Pediatrics. 1998;102(3 Pt 1):563-568. [CrossRef] [Medline]
    54. Tates K, Meeuwesen L. 'Let Mum have her say': Turntaking in doctor-parent-child communication. Patient Educ Couns. 2000;40(2):151-162. [FREE Full text] [CrossRef] [Medline]
    55. Tates K, Elbers E, Meeuwesen L, Bensing J. Doctor-parent-child relationships: A 'pas de trois'. Patient Educ Couns. 2002;48(1):5-14. [CrossRef] [Medline]
    56. Cahill P, Papageorgiou A. Triadic communication in the primary care paediatric consultation: A review of the literature. Br J Gen Pract. 2007;57(544):904-911. [FREE Full text] [CrossRef] [Medline]
    57. Tepencelik ON, Wei W, Cosman PC, Dey S. Analyzing head orientation of neurotypical and autistic individuals in triadic conversations. ArXiv. Preprint posted online November 1, 2023 . [CrossRef]
    58. Kallio H, Pietilä AM, Johnson M, Kangasniemi M. Systematic methodological review: Developing a framework for a qualitative semi-structured interview guide. J Adv Nurs. 2016;72(12):2954-2965. [CrossRef] [Medline]
    59. Stiefelhagen J, Yang J, Waibel A. Estimating focus of attention based on gaze and sound. New York, NY. Association for Computing Machinery; 2001. Presented at: Proceedings of the 2001 Workshop on Perceptive User Interfaces; 2001 Nov 15:1-9; Orlando, FL. URL: https://doi.org/10.1145/971478.971505 [CrossRef]
    60. Ba SO, Odobez JM. A study on visual focus of attention recognition from head pose in a meeting room. New York, NY. Springer Publishing; 2006. Presented at: International Workshop on Machine Learning for Multimodal Interaction; 2006 May 1:75-87; Bethesda, MD. [CrossRef]
    61. Crawford JR, Howell DC. Comparing an individual's test score against norms derived from small samples. Clin Neuropsychol. 1998;12(4):482-486. [CrossRef]
    62. Crawford JR, Garthwaite PH, Howell DC. On comparing a single case with a control sample: An alternative perspective. Neuropsychologia. 2009;47(13):2690-2695. [CrossRef] [Medline]
    63. Sokal RR, Rohlf FJ. Biometry. New York, NY. W H Freeman; 1995.

    ‎
    AQ: autism spectrum quotient
    ASC: autism spectrum condition
    AvgCon: average duration of contact
    AvgNoCon: average duration of not making contact with anyone
    ExcPct: total duration of exclusion percentage per session
    MaxExc: maximum exclusion duration
    NA: nonautistic
    NT: neurotypical
    NumCon: number of contacts per minute
    TD: typically developing
    UC: University of California

    Edited by A Mavragani; submitted 10.12.23; peer-reviewed by MDG Pimentel; comments to author 23.03.24; revised version received 17.04.24; accepted 27.05.24; published 12.08.24.

    Copyright

    ©Onur Necip Tepencelik, Wenchuan Wei, Mirabel Luo, Pamela Cosman, Sujit Dey. Originally published in JMIR Formative Research (https://formative.jmir.org), 12.08.2024.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.