Bystanders, those who witness but are not directly involved in cyberaggression, play a pivotal role in disrupting harmful dynamics [28]. Their responses vary in tone, target, and intention, ranging from aggressive actions (eg, threats and insults) to prosocial support [29,30]. Within cyberbullying, where victims often struggle to defend themselves, bystanders are particularly well studied [31]. Broadly, bystander interventions are categorized as aggressive, constructive, victim-focused, or constructive bullying-focused; aggressive strategies can escalate conflict [28,32], whereas constructive approaches, such as encouraging apologies, asking perpetrators to stop, or supporting victims with empathy or advice, are more effective in de-escalation [33-35]. Bystander interventions are uniquely embedded within ongoing interactions. Unlike post hoc moderation or long-term educational approaches, bystanders can intervene as harm emerges, offering immediate support to targets while signaling that aggressive behavior violates community norms [20,36]. This ability to reduce harm in real time and shape collective expectations makes bystander intervention a critical and complementary strategy, particularly in early-stage incidents where formal enforcement is yet to be enacted.

Large-scale discourse analyses reveal that bystanders use diverse strategies, including callouts, mocking, education, and moderation. For example, Ray et al [36] examined over 2 million Twitter and Reddit posts, finding that constructive approaches, such as educating perpetrators and mobilizing support, reduced hate speech and encouraged engagement, while mocking had little effect. Yet, intervention rates remained low, with bystanders active in only one of 6 Twitter threads and under 40% of Reddit discussions. These barriers highlight the need for scalable supports to help bystanders act effectively. Building on this, this study focuses on 5 constructive intervention subtypes, assessing both the linguistic quality (eg, tone and affect) and effectiveness (favorability, discussion shift, and behavior change) of LLM-generated responses.

LLMs have emerged as powerful tools for mitigating online harms, including cyberaggression [37,38]. Their capacity to understand and generate human-like language enables novel approaches to detecting harmful content, supporting victims, and fostering prosocial interactions [27]. For instance, Islam et al [37] proposed an LLM-based cyberbullying detection pipeline combining preprocessing, query generation, and classification for early intervention. Similarly, Vanpech et al [38] extended this approach using GPT-4 Vision to analyze images posted on Twitter and classify bullying content. These studies underscore LLMs’ potential in detection-focused interventions. Beyond detection, LLMs have been applied to bystander support and education. For example, Hedderich et al [18] developed an LLM-based chatbot platform for K-12 classrooms, enabling educators to simulate social media interactions and teach bystander strategies. Similarly, Wu et al [39] generated personalized persuasive messages aligned with strategies such as comforting, evoking, and scaffolding, showing that LLMs can adapt interventions to users’ contexts. Collectively, this work demonstrates growing interest in leveraging LLMs for intervention generation.

However, prior work has largely emphasized detection or educational use cases, leaving open questions about how LLMs can serve as in situ scaffolds for real-time bystander intervention during unfolding incidents of cyberaggression. In addition, prior studies emphasize technical alignment over psychological grounding [40]. Without integration of established bystander theories, LLM-generated interventions risk overlooking the cognitive and emotional processes driving real-world decisions. To address this gap, our study examines 5 intervention strategies rooted in bystander theory, using prompt engineering to generate responses that are linguistically appropriate, context aware, and psychologically informed. Therefore, we ask, “How can generative AI be designed to provide psychologically grounded, context-sensitive suggestions for bystanders intervening in cyberbullying scenarios?”

We collected 1000 cyberaggression examples from the 2 sources [41,42]. Of 1000 cyberaggression examples, 200 were collected from the dataset created by Huang et al [40], while 800 were collected from the dataset created by ElSherief et al [41].

We implemented 5 theoretically and empirically grounded strategies for prompting LLM-generated bystander intervention responses. Each strategy was derived from psychological literature and operationalized through specific prompts in the form of “dos and don’ts” to guide GPT responses. Table 1 presents a brief overview of the definition, psychological basis, and prompting guidelines for each bystander intervention (see Multimedia Appendix 1 for full details).

To generate bystander intervention responses, we prompted AllyGPT to first assess each aggressive message based on 2 dimensions: incivility level (low, moderate, or high) and direction (group targeted vs individual targeted). These dimensions are grounded in bystander intervention theory and empirical research on cyberaggression, which identifies severity appraisal and target identifiability as primary determinants of whether and how bystanders intervene. Bystander intervention theory conceptualizes intervention as a sequential process in which individuals must first interpret an event as sufficiently serious to warrant action and to assume responsibility [44], while empirical studies show that incident severity strongly predicts intervention likelihood and interacts with empathy and responsibility attribution [45]. Direction further shapes responses by influencing emotional salience, perceived responsibility, and social visibility: person-targeted aggression elicits clearer moral judgments and stronger empathic engagement, whereas group-targeted harm more often leads to diffusion of responsibility and reduced effectiveness of individualized responses [46]. Prior work also suggests that empathy-focused strategies are more effective in lower-severity contexts, while assertive or confrontational approaches are more appropriate in higher-severity or public settings [47]. Together, these insights informed AllyGPT’s selection and conditional combination of intervention strategies to generate context-sensitive bystander responses. Below, we present the prompt encoding this decision logic.

Through the above process, we generated 3 responses for each of the 1000 cyberaggression scenarios, resulting in a total of 3000 bystander responses (1000 per strategy) across the three intervention approaches: (1) standard policy reminder, (2) baseline GPT, and (3) AllyGPT.

To evaluate the linguistic differences across bystander intervention responses generated by 3 different strategies, we used the LIWC [57] to extract several linguistic features from the bystander responses. The features we extracted in this study included word count (total number of words), words per sentence (average sentence length), analytical thinking (degree of logical and formal thinking), authenticity (extent of honest and genuine expression), positive tone (prevalence of positive emotional language), negative tone (prevalence of negative emotional language), and netspeak (use of internet-specific shorthand and casual online expressions, eg, “lol” and “brb”).

The effectiveness of the bystander intervention messages was evaluated using three variables: (1) perceived favorability, (2) discussion shift, and (3) behavior change. Three trained coders independently rated each bystander intervention response on 3 effectiveness dimensions using a 5-point Likert-type scale (1=strongly disagree and 5=strongly agree). The first dimension, perceived favorability, assessed the extent to which a response would likely be viewed positively by other social media users (mean 3.88, SD 0.88). The second dimension, discussion shift, evaluated the potential for the response to redirect the conversation toward a more positive, objective, and/or less antagonistic tone (mean 3.95, SD 0.81). The third dimension, behavior change, measured the perceived likelihood of the response influencing the bully’s or agitator’s future online behavior (mean 3.45, SD 0.86).

Each coder was responsible for evaluating bystander intervention responses associated with all 1000 cyberaggression scenarios, ensuring exposure to all 3 intervention strategies. To achieve this, the intervention responses were divided into 3 batches so that each coder rated one-third of the responses from each strategy. For example, coder 1 evaluated strategy 1 responses from batch 1 (aggressive messages 1‐333), strategy 2 responses from batch 2 (messages 334‐666), and strategy 3 responses from batch 3 (messages 667‐1000; Table 2). This approach ensured that each coder rated the effectiveness of intervention responses across all 3 strategies while covering the full set of 1000 cyberaggression examples. Table 3 presents the example aggressive messages, bystander responses generated from the 3 strategies, and coder ratings of the 3 effectiveness variables.

Three graduate students in psychology (2 doctoral and one master’s) were trained as independent raters. Training involved a structured, multistep process combining group instruction, guided practice, and independent rating. Coders were first introduced to the study purpose, research questions, and their own experiences with social media and cyberaggression. They were then provided with a detailed coding manual containing sample scenarios, example ratings, and rationales for the 3 effectiveness dimensions (favorability, discussion shift, and behavior change).

As a group, coders practiced rating 3 scenarios, followed by discussion and feedback to build consensus. They next completed independent practice ratings on 3 additional scenarios, which were reviewed and discussed to refine consistency. Coders were also trained on the Qualtrics survey platform, including rating procedures and response scales. To establish intercoder reliability, each independently rated 25 additional scenarios; consensus agreement of 80% or greater was required before proceeding to the full study (1000 scenarios per coder and 3000 total).

A 1-hour booster session was conducted midway through coding to address procedural questions, clarify operational definitions, and review rating criteria. Coders jointly discussed 2 example responses, examining features linked to higher versus lower ratings across the dimensions. Intercoder agreement was computed using a group consensus method. Individual coder ratings for each of the 25 scenarios and the 3 effectiveness dimensions (perceived favorability, discussion shift, and behavior change) were compared to the group mean ratings. Agreement was considered achieved when coder ratings fell within one point of the group mean. For the 3 effectiveness dimensions, the Krippendorff α was 0.73 for favorability, 0.84 for discussion shift, and 0.89 for behavior change.

After completing the survey, each coder participated in a separate semistructured exit interview conducted by 2 researchers. Through the interviews, we explored coders’ views on the effectiveness of responses generated by the 3 strategies, their overall coding experience, and any questions about the study. To aid recall, coders were reminded of the rating scales and were shown their average scores across the 3 effectiveness dimensions. They also reviewed 2 example aggression scenarios with responses from all strategies: one where baseline GPT outperformed AllyGPT and another where AllyGPT outperformed baseline GPT across all dimensions. Examples were selected to ensure consistent trends across dimensions, randomly sampled from different coding batches, and paired with the corresponding policy reminder. Interviews lasted about 50 minutes, were audio recorded, transcribed, and deidentified. Coders received compensation for their participation.

A one-way ANOVA indicated that there was no statistically significant difference in mean values for perceived favorability, effectiveness in altering the discussion, and effectiveness in changing bullying behavior across intervention messages generated for high-level, medium-level, and low-level aggressive messages (Table 7).

Independent-samples t tests revealed significant differences in the perceived effectiveness of intervention messages depending on whether the aggression was group targeted or individual targeted. Responses addressing group-targeted aggression were rated as significantly more favorable than those addressing individual-targeted bullying (P<.001). Similarly, interventions for group-targeted bullying were seen as more effective in altering discussions than for individual-targeted bullying (P<.001). Finally, messages addressing group-targeted bullying were rated as more effective in changing bullying behavior than those addressing individual-targeted bullying (P=.001; Table 8).

We conducted a series of robust general linear models with heteroskedasticity-consistent SEs to examine the main and interaction effects of strategy (standard policy reminder, baseline GPT, and AllyGPT) and incivility level (low, moderate, and high) on 3 outcome measures. Pairwise contrasts were used to compare strategy conditions, and significant interactions were further examined using simple effects analyses. Overall, strategy type mattered more than incivility level. Across all outcomes, GPT-generated responses (both baseline and AllyGPT) were consistently rated higher than policy reminders. Incivility level alone did not significantly influence ratings, and very few strategy ×incivility interactions emerged. The only notable exception was that baseline GPT responses were perceived as more effective in changing behavior when paired with moderately incivil messages.

There was a significant main effect of strategy type, in which GPT-based responses were rated as more favorable than policy reminders. Incivility level and interaction effects were not significant (Table 9).

No significant main or interaction effects were observed. Although GPT-based responses were directionally more positive, the differences were not statistically significant.

Both baseline GPT and AllyGPT responses were rated as more effective than policy reminders. A small interaction was observed, with baseline GPT showing an added advantage under moderate incivility.

Figure 2 visualizes the interaction effects of strategy and incivility level on bystander intervention ratings across 3 outcome variables. Although the interaction effect was not statistically significant, exploratory pairwise comparisons indicated a pattern in which AllyGPT responses were rated more favorably and more effective for behavior change than baseline GPT responses at low levels of incivility. These differences were not observed at moderate or high levels of incivility.

GPT-based responses (both baseline and AllyGPT) were rated more favorably than policy reminders. Responses addressing person-targeted aggression were rated less favorably than those addressing group-targeted bullying. No significant interactions were observed (Table 10).

Baseline GPT responses were rated as most effective in shifting discussions, followed by AllyGPT, with policy reminders rated lowest. The direction of bullying did not significantly influence ratings. No interaction effect was found.

Both baseline GPT and AllyGPT responses were perceived as more effective than policy reminders. A significant interaction emerged: AllyGPT responses were rated as less effective in changing behavior when applied to person-targeted bullying compared to group-targeted bullying. Other interactions were statistically not significant.

Figure 3 visualizes the interaction effects of strategy type and specificity on 3 outcome variables.

The semistructured interviews with the coders provided nuanced perspectives on the perceived effectiveness of 3 different intervention strategies. First, the coders consistently described the standard policy reminder as robotic and impersonal. While these messages were generally viewed as inoffensive and safe, their lack of contextual adaptation made them feel generic. However, coders noted that this very standardization helped prevent the messages from seeming intrusive or out of place within online conversations: “It’s robotic but doesn’t feel like it’s coming out of nowhere. It kind of works because it doesn’t assume too much” (coder 1).

Baseline GPT responses were received more favorably than policy reminders due to their balance of specificity and neutrality. Coders noted that these responses avoided being too generic while also refraining from making direct or accusatory assumptions about the bully’s intent: “They don’t first attack the bully. They kind of try to educate the bully. It’s not a very aggressive message” (coder 2). The tone of these responses was described as gentle, constructive, and inclusive. One coder appreciated the frequent use of collective language (eg, “we” and “let’s”), which conveyed a shared commitment to respectful communication within online communities: “It’s using a lot of ‘we’ language... and I find that more favorable for viewing” (coder 1).

Coders described the responses generated by the AllyGPT prompting as the most specific, often incorporating detailed psychological framing and references to the nature of the aggression. While this specificity occasionally added persuasive depth and a human-like tone, it also introduced potential drawbacks. In cases where the response identified specific problematic content or implied motive (eg, racial bias), coders noted that such directness might provoke further defensiveness or hostility from bullies: “It’s kind of diving deeper and assuming the comment writer’s thought processes... It feels like there’s a little more of an argumentative tone” (coder 3). Although coders acknowledged that firmer messages could resonate in highly aggressive scenarios, they also emphasized that overt specificity or presumed interpretation could reduce the intervention’s acceptability, especially when assumptions do not align with the aggressor’s perceived intent.

Human coders widely agreed that the GPT-generated responses, particularly those from both baseline GPT and AllyGPT prompts, were generally more effective in shifting conversations toward a more positive and objective tone compared to the standardized policy reminders. While coders acknowledged that the standard policy reminder emphasized the importance of respectful communication, they felt this response “points out the need for more objectivity, but doesn’t really lead to a positive direction” (coder 1). As a result, they did little to steer the conversation in a constructive direction.

In contrast, coders perceived the baseline GPT responses as more successful in nudging conversations toward civility. These responses not only emphasized objectivity and respectful tone but also offered an alternative path for engagement, such as refocusing the discussion on shared understanding or mutual respect. This dual approach, highlighting a problem while proposing a constructive next step, was particularly appreciated by the coders. They also emphasized that the emotionally neutral and nonconfrontational tone used in these responses helped defuse tension. This approach, in their view, increased the likelihood that others would respond positively.

Coders noted that the customized responses generated by the AllyGPT model often provided in-depth reasoning for why a comment was harmful while stressing the importance of respectful dialogue and the responsibility of maintaining safe social spaces. However, the more assertive tone and direct confrontation of aggressive behavior raised concerns among coders. Some felt that these responses, while well-intentioned, could come across as accusatory or argumentative, especially if they appeared to infer the perpetrator’s motives.

A critical insight raised by coders concerned the focus of the intervention. While AllyGPT responses often centered on validating the victim’s experience, coders questioned the effectiveness of these responses when they failed to directly engage the perpetrator. One coder stated that, “It’s taking the focus away from the perpetrator and onto the victim... the conversation can’t go more positive if we’re not addressing that person” (coder 3). In contrast, baseline GPT responses were seen as doing a better job of engaging the perpetrator without provoking defensiveness. These messages reminded the aggressor to be mindful of their language and consider its impact, without making them feel targeted.

Coders’ perceptions of the effectiveness of each intervention strategy in changing aggressive behavior largely aligned with their earlier assessments of how well the responses altered the direction of the discussion. Across the board, coders emphasized that effective behavioral change requires directly, yet constructively, engaging the perpetrator in a way that encourages reflection and more respectful communication. Again, standard policy reminders were viewed as having a limited and neutral impact on changing aggressive behavior. While they did set a boundary and signal that harmful behavior was not tolerated, coders noted that these messages “didn’t suggest a more helpful or adaptive way for the perpetrator to act or speak” (coder 1).

Coders consistently perceived baseline GPT responses as striking an effective balance, being specific enough to address inappropriate behavior yet gentle enough to avoid provoking defensiveness. This balance, they felt, increased the likelihood that bullies would be receptive to the message and consider adjusting their behavior. One of the strengths of the baseline GPT responses, according to the coders, was the way they suggested alternative ways to engage in discussion, such as “Let’s focus on discussing specific policy points or actions that spark our concern or approval” (coder 2), redirecting attention to a shared goal or more constructive conversation topics.

Coders acknowledged that the responses generated by the AllyGPT model were tailored and direct in highlighting the specific aspects of a comment that were problematic. This directness sometimes added clarity and educational value. However, coders also expressed concern that these responses occasionally made assumptions about the bully’s intent or internal beliefs, such as labeling a comment as “discriminatory” or referring to an “insinuation,” which could be perceived as confrontational and reduce the effectiveness of the intervention. Instead of encouraging reflection, such responses risked escalating the situation or causing the perpetrator to disengage.

Overall, coders emphasized that for an intervention to be effective in promoting behavior change, it should maintain a respectful and nonjudgmental tone, avoid strong assumptions about motivation, and provide a clear, constructive alternative to harmful behavior.

Coders highlighted several strengths of GPT-generated responses compared to standard policy reminders. Most notably, they praised the model’s ability to show context awareness and personalization. Rather than offering one-size-fits-all statements, the interventions often reflected the specifics of the bullying scenario. As one coder noted, “I was impressed with its ability to focus in on the specific details of what was being said” (coder 3).

Coders also emphasized the conversational and human-like tone. They appreciated language that felt natural and approachable, contrasting sharply with the robotic style of policy reminders: “How personable does it sound for that response? That was really important for me” (coder 3). Such qualities were seen as enhancing rapport and trust.

Finally, GPT responses were perceived as more engaging and persuasive, with greater potential to redirect conversations toward civility. Their contextual sensitivity, approachable tone, and constructive framing made them feel better positioned to encourage reflection and behavior change. Overall, coders viewed GPT-generated responses as more relatable and impactful than standardized policy reminders.

Despite these strengths, coders also identified limitations. A key challenge was the subjectivity of evaluation, with judgments shaped by personal interpretation and uncertainty about real-world impact. To manage this ambiguity, some used policy reminders as a neutral reference point: “Policy reminder is also neutral it doesn’t add much but sets a benchmark” (coder 1).

Coders also doubted the effectiveness of any response, AI-generated or otherwise, against deeply entrenched or hostile beliefs. As one put it, “There’s not much that can be done to change someone making a hateful joke” (coder 2). Another added, “Some people are too inflexible... I don’t think any sort of response will change them” (coder 3).

Finally, while GPT often produced contextually appropriate responses, occasional lapses occurred when it misinterpreted situations: “There were a few responses where ChatGPT was totally off the mark or got off task” (coder 3). These errors underscored the need for refinement to improve accuracy in sensitive contexts.

Coders offered thoughtful recommendations for enhancing the design of GPT-generated intervention messages. One key suggestion was to acknowledge both the perpetrator’s and the victim’s perspectives, particularly in messages that challenge inappropriate behavior. Coders noted that responses were more persuasive and less likely to provoke defensiveness when they began by validating the perpetrator’s viewpoint before introducing critique.

Another recurring recommendation was to use a friendly, conversational tone rather than formal or emotionally charged language. Coders preferred responses that felt approachable and clear, avoiding complex vocabulary or accusatory phrasing: “I prefer more casual and natural language—nothing too serious or full of big words [...] Starting with friendly language, then pointing out the issue, and ending with a suggestion worked best” (coder 1). This approach was seen as helpful in reducing tension and guiding users toward reflection and behavioral change.

Finally, coders recommended balancing personalization with structural consistency. While they appreciated the human-like and tailored qualities of many GPT responses, they also valued a recognizable format that helped organize the message and ensure clarity. A common preference was for a response structure that included an opening statement, identification of the problematic issue, and a constructive suggestion for alternative behavior. As one coder put it, “The customized ones added a personal touch... but I appreciate the consistency of the standard responses too” (coder 3).

Together, our qualitative findings underscore the importance of bystander interventions that are empathetic, conversational, and contextually grounded. Coders consistently rated GPT-generated responses as more effective than policy reminders across all dimensions. Baseline GPT was praised for its balanced tone and gentle redirection, while AllyGPT stood out for clarity and victim-centered framing, although it was sometimes seen as overly direct or assumptive. In contrast, policy reminders were viewed as impersonal and unlikely to shape discussion or behavior. Overall, coders emphasized that effective interventions balance clarity with compassion, avoid excessive emotionality, and offer respectful alternatives (Table 11).