Differences in Safety Risks Across Languages in Health-Relevant Queries: Vulnerability Analysis of Large Language Model Responses

Introduction

Health systems worldwide exhibit persistent inequities in access to care and health outcomes across different population groups. Recent evidence indicates that health artificial intelligence (AI) algorithms often perpetuate these disparities, introducing biases that disadvantage certain communities [1-3]. While health information systems, including large language models (LLMs) such as ChatGPT, are widely regarded as transformative tools for improving access to health resources, concerns are growing that these technologies may exacerbate existing inequities rather than reduce them [4-7].

Health LLMs refer to general-purpose or domain-specific LLMs applied in health care contexts, where safety lapses can have serious consequences for patient well-being. A critical yet understudied dimension of these inequities involves access to harmful health information that can influence behaviors such as self-harm and violence. It is generally assumed that systems like ChatGPT will refuse to provide responses that could endanger health. Furthermore, it is implicitly assumed that the performance of LLMs across languages will remain similar. However, empirical evaluations of these assumptions remain limited in the health care context. This question is relevant for health informatics because the incidence of self-harm and related harms varies across linguistic and cultural groups, and if safeguards differ by language, they could amplify health disparities both within and across countries [8-10].

Differences in vulnerability to certain diseases are well documented and often attributed to social and behavioral factors across racial and linguistic groups [11]. In contrast, the vulnerabilities examined in this work, which focus on exposure to unsafe information that may enable self-harm or substance abuse, are preventable and stem from design gaps in health LLMs. Addressing these risks requires rigorous safety engineering and equitable language coverage to ensure that emerging health technologies do not amplify harm [12].

LLMs, such as ChatGPT-4, have significantly advanced natural language processing and are increasingly used in health-related applications [13]. However, despite their capabilities, these models are susceptible to jailbreaking—a process where adversarial prompts bypass built-in safety filters and elicit responses that violate content moderation policies. Jailbreaking is a critical component of security analysis because it exposes weaknesses in safeguards designed to prevent the dissemination of harmful or medically inappropriate content [14]. Understanding these vulnerabilities is essential for developing robust AI systems that can prevent the spread of dangerous information.

Existing research has begun to address health risks in LLM outputs and language disparities in jailbreak vulnerabilities [15], and recent studies have explored multilingual jailbreak attacks and their implications for safety alignment [16]. Work in medical contexts has also highlighted risks such as unsafe advice and multimodal jailbreak vulnerabilities [1,2]. However, these efforts remain fragmented and rarely provide systematic evaluations of jailbreak susceptibility across multiple languages for health-related tasks. Multilingual LLMs face unique safety challenges due to linguistic inequality in training data, as most safety alignment relies on high-resource languages like English. This imbalance increases vulnerability in low-resource languages, where harmful content detection is less effective [3]. While health-related risks such as hallucinations, misinformation, and misdiagnosis have been documented [5], the variance in jailbreak success rates across languages in health contexts remains underexplored. Users interacting in languages other than English may face disproportionate exposure to harmful outputs, exacerbating global health inequities. Specifically, the extent to which cipher-based attacks can successfully jailbreak LLMs in languages like Hindi and Spanish is not well understood but is critically important.

To guide this investigation, we pose the following research questions (RQs):

• RQ1: How do cipher-style adversarial attacks affect the safety performance of LLMs across different languages in health-related contexts?
• RQ2: How do different cipher techniques (emoji and permutation) influence the success of adversarial attacks across languages in health-related contexts?
• RQ3: Are the patterns of vulnerability across harm categories (self-harm, violence, and drugs) consistent across languages and cipher techniques in health-related adversarial attacks?

Methods

Study Design

We conducted an experimental study to assess the susceptibility of a consumer-facing LLM to cipher-based jailbreak attacks across 3 languages (English, Spanish, and Hindi). We used ChatGPT for this experiment as it was one of the most commonly used, freely accessible LLMs, and it was frequently being used to ask health-related questions. ChatGPT (GPT-4) was the flagship model from OpenAI at the time of data collection (April 2024). While all 3 languages had over a billion speakers, English was the most well-studied language in terms of natural language processing research, and Spanish was also relatively well resourced. Hindi, on the other hand, was widely spoken yet considered a low-resource language in natural language processing research [17]. For each language, we used 2 cipher techniques for jailbreak. Each combination of 3 languages and 2 cipher techniques consisted of a comprehensive evaluation using 1000 input queries. Thus, our study involved a total of 6000 query response pairs.

Cipher Techniques

Overview

This study investigates multilingual safety vulnerabilities in LLMs, focusing on adversarial prompt-rewriting strategies known as cipher attacks. Cipher attacks involve encoding or semantically altering harmful prompts to evade detection by safety filters [18]. Unlike direct translations of unsafe content, cipher attacks obscure intent through creative transformations, making it more difficult for LLMs to recognize and block harmful inputs.

A commonly used jailbreak strategy in the literature involves translating harmful prompts from high-resource languages (such as English) into low-resource languages (such as Hindi or Swahili). This technique exploits the fact that LLMs’ safety mechanisms are often less robust in low-resource languages, increasing the likelihood of harmful outputs [3,10,19]. Prior research has documented cross-language jailbreak vulnerabilities and emphasized the need for stronger multilingual safety alignment. Additionally, recent work has shown that inserting emojis into prompts can distort token embeddings and reduce the effectiveness of safety filters, particularly in judge LLMs [20]. Building on these insights, our study introduces a novel operationalization of cipher attacks that combines three underexplored dimensions: multilingual prompts, emoji-based encoding, and prompt permutation. To our knowledge, this is the first work to integrate these strategies in the context of health-related prompts, a domain where safety and interpretability are especially critical. We also depart from prior work by using detailed human-level coding to evaluate model responses, enabling context-sensitive and culturally grounded assessments that automated evaluators may miss.

Specifically, here we considered 2 different attack strategies.

Emoji Cipher Attack

In the creation of the cipher, we developed custom dictionaries for each language, mapping each alphabet letter to an emoji that starts with the same letter in that language. Each letter of the input prompt, as well as the expected responses, was replaced with its corresponding emoji from the dictionary to create the emoji-encoded text. A few examples of emoji representations of the alphabet are shown in Table 1.

Permutation Cipher Attack

The permutation cipher was created by randomly shuffling the positions of characters within each word of the input prompts and responses. This permutation of input prompts and responses added an extra layer of encoding and decoding for the LLM, which was intended to override the in-built safety guardrails. Some examples of permuted words are given in Table 2.

Data Sources

We used the BeaverTails dataset [14,21], an AI safety–focused collection containing human-labeled question-answer (QA) pairs across 14 harm categories. This is a well-cited public resource that aims to foster research on safety alignment in LLMs. It includes over 100,000 QA pairs in a wide variety of domains and includes expert feedback on harmlessness. From these categories, we selected the following three due to their relevance and potential for harm in the health context: (1) self-harm; (2) violence, aiding and abetting, and incitement (in this paper, we refer to this category as violence); and (3) drugs.

Once we identified relevant categories, we selected a subset of 1000 sentences from the BeaverTails dataset. Of the 1000 sentences, we had 167 cases for the self-harm category, 733 cases for the violence category, and 100 for the drugs category. This subset formed the English dataset. Second, we translated the English dataset into Spanish using Google Translate. The translations were reviewed for accuracy by a bilingual (Spanish-English) coauthor. This subset forms our Spanish dataset. Third, we translated the English sentences into Hindi, using Google Translate for transliteration in the Latin alphabet. These transliterations were also manually validated by a bilingual (Hindi-English) coauthor to ensure correctness. Thus, each of the 3 language datasets consisted of 1000 sentences, resulting in a total of 3000 sentences for analysis.

Experimental Procedure

To enable the model to interpret ciphered inputs, we used a 15-shot learning approach. Initial exploration showed that 15 examples were sufficient for ChatGPT-4 to reliably recognize and process both emoji and permutation ciphers. These examples were presented as input-output pairs to establish the mapping logic for each cipher type. We applied 2 types of cipher transformations (emoji cipher attack and permutation cipher attack) to all input prompts and expected responses. The encodings were applied consistently across prompts and responses to preserve semantic alignment while visually obfuscating the content.

We interacted with ChatGPT-4 via OpenAI’s application programming interface, submitting the ciphered prompts and recording the corresponding ciphered outputs. The outputs were then deciphered to recover the plain-text responses, referred to as “Output Text.” A team of human coders evaluated each sentence-response pair to determine whether the model’s output contained harmful content aligned with the original prompt. All responses were independently coded by 2 annotators. For Spanish and Hindi, coders were bilingual and fluent in the respective languages to ensure accurate interpretation and classification.

The experimental workflow is best described through the visualization shown below in Figure 1.

‎

The data collection workflow can be divided into two basic phases: (1) prompt generation for each of the 1000-set inputs and (2) vulnerability analysis of responses from GPT-4. In the prompt generation phase, 15 examples from the training set are used to create the ciphered versions for each entry in the 1000-item dataset. For the English language, each item in the training set and the 1000-item dataset goes directly to the cipher technique. However, each input goes through a translation process into the Spanish language and the transliterated Hindi language before being sent to the cipher process. This process results in 6000 input prompts (3 languages × 2 ciphers × 1000-item dataset). In the response collection phase, each item from the input prompts is fed to GPT-4 to output a ciphered response for each item. This ciphered response is then fed back to GPT-4 to convert the cipher back to the English language. For each input prompt, we collect a response and evaluate the results according to the codebook developed. The counts of these evaluations are tabulated and analyzed using the Pearson chi-square test of independence.

A structured codebook was developed to guide the classification of model responses into predefined outcome categories. This codebook, provided in Table 3, was used to ensure consistency and reliability across coders. The coding framework enabled a nuanced assessment of model behavior across languages and cipher types, supporting the broader goal of evaluating multilingual safety vulnerabilities in health-related contexts.

Outcome Measures

Each jailbreak attempt can have 1 of 5 different outcomes, as presented in Table 3. For analysis, we used 3 outcomes: success, failure, and others (a combination of all responses that were neither success nor failure).

The coders were trained in output labeling by 2 senior coauthors. A sample of 30 data points was coded by two of the authors, both fluent in the respective language. The intercoder agreement was found to be over 90% in all instances, and the student coauthors labeled the remaining data points on their own. Following a study by Abbasi et al [22], we use the attack success rate (ASR) and attack failure rate (AFR) to quantify the results. They represent the percentage of successful attacks and failed attacks, respectively, for each subset of 1000 attempts for the 2 ciphers and 3 languages.

A complete example of an item from the 1000-dataset is illustrated in Table S1 in Multimedia Appendix 1.

Ethical Considerations

All study procedures complied with OpenAI’s usage policies and ethical guidelines. No personal or sensitive information was used or generated during the research, and all data were securely stored with access restricted to authorized research personnel. The dataset was fully anonymized, and all interactions with the model were conducted to ensure participant privacy and data integrity. The overarching goal of this study is to contribute to the development of health LLMs that are safe and equitable across languages. By addressing multilingual vulnerabilities in health-related contexts, this work aims to support more inclusive and socially beneficial AI systems. The project did not require approval from the institutional review board because there were no external human participants, and the Hindi and Spanish coders are coauthors of this paper. We used the publicly available Beavertails dataset and did not include any human participant data.

Results

Overview

The detailed results for each of the permutation and emoji ciphers are presented in Table 4 (and the aggregation across the 2 ciphers is available in Table S2 in Multimedia Appendix 1). The total number of tests for each language and its splits across harm categories is shown in the second column. In both tables, the attack percentage success for each language and harm category within the language is given in the last column of the table. As can be seen, a sizable number (over 50% in every case) of prompts result in successful attacks.

We compare these high ASRs with a baseline to contextualize our interpretation. The baseline consists of a subset of the evaluation set used to produce responses to nonciphered, direct harmful prompts across the 3 languages, using the same proportional distribution of harm types. The results for this baseline setting are available in Tables S3 and S4 in Multimedia Appendix 1. The ASR is below 5% for each language in both types of cipher attacks. The baseline results indicate that the model (GPT-4) is generally robust to direct harmful prompts in all 3 languages. The substantial increase in vulnerability under cipher attacks underscores the importance of studying how this increased vulnerability varies across languages.

Next, we zoom in on subsets of results to answer the identified RQs.

Difference in Vulnerabilities Across Languages

Our first RQ focuses on differences in vulnerabilities across languages. The number of successful attacks across languages and cipher techniques is shown in Figure 2.

‎

The attacks were most successful in Hindi (83%) and least successful in Spanish (59.95%). A chi-squared test revealed a statistically significant association between language and cipher technique, χ²₂=17.21 and P<.001. More details on the test are available in Table S5 in Multimedia Appendix 1.

Not every attack results in a clearly identifiable success or failure. As shown in Table 3, our codebook also includes the categories of not sure, confused, and no output. Hence, to further understand the jailbreak outcomes, we study the differences in success (part of ASR above), failure, and others (the sum of not sure, confused, and no output). The results of jailbreak outcomes for both ciphers are summarized in Table 5.

A chi-square test of independence was performed to assess whether language is significantly associated with the observed outcomes for the emoji cipher (Table 5). The results show a significant difference in result rates across the languages at a 95% confidence level with χ²₄=468.17 and P<.001. Hence, the health safety risks of LLMs to both ciphers combined are different across languages. These risks are most severe for Hindi and least severe for the Spanish language. The major implication here is that the health safety risks of LLMs vary across languages.

Difference in Vulnerabilities Across Languages in Each Cipher Technique

Our second RQ focuses on the differences in vulnerabilities across the considered cipher types. Figure 2 shows that the emoji cipher is more successful than the permutation cipher for the Spanish and English languages, whereas the permutation cipher is more successful than the emoji cipher for the Hindi language.

Emoji Cipher

The results of jailbreak attempts across languages for emoji ciphers are summarized in Table 6. From Table 6, we can see that the rates of success, failure, and others differ for each language for the emoji cipher. For instance, the highest ASR is for Hindi, the highest AFR is for English, and others is for Spanish.

A chi-square test of independence was performed to assess whether language is significantly associated with the observed outcomes for the emoji cipher (Table 6). The results show a significant difference in result rates across the languages at the 95% confidence level with χ²₄=115.98 and P<.001. Hence, the health safety risks of LLMs to the emoji cipher are different across languages. They are most severe for Hindi and least severe for the Spanish language.

Permutation Cipher

The results of jailbreak attempts across languages for permutation ciphering are summarized in Table 7. We can see that the rates of success, failure, and others differ for each language for the permutation cipher. For instance, for the English and Hindi languages, the “others” results are below 80, but for Spanish, they are 191. Hindi has the highest ASR and the lowest AFR.

A chi-square test of independence was performed to assess whether language is significantly associated with the observed outcomes for the permutation cipher (Table 7). The results show a significant difference in result rates across the languages at the 95% confidence level with χ²₄=450.39 and P<0001. Hence, the health safety risks of LLMs to permutation ciphers are also different across languages. They are most severe for Hindi and least for the Spanish language.

Differences in Vulnerabilities Across Risk Categories by Cipher Techniques

Our third RQ studies how the patterns of vulnerability vary across harm categories (self-harm, violence, and drugs). We study these patterns for the permutation cipher, the emoji cipher, and then combined across these 2 ciphers.

Permutation Cipher

The results of jailbreak attempts across categories for permutation ciphering are summarized in Table 8. We can see that the rates of success, failure, and others are similar for all 3 categories. For each category, success gives the maximum number of cases, whereas others has the least number of cases. Also, from Table 4, the rates of success for each harm category are 65.07% (self-harm), 68.62% (violence), and 70.33% (drugs).

A chi-square test of independence was performed to assess whether categories are significantly associated with the observed outcomes for the permutation cipher (Table 9). The results do not show a significant difference in result rates across the categories at the 95% confidence level with χ²₄=6.25 and P=.18. Hence, health safety risks of LLMs to the permutation cipher are not statistically different across harm types. They are most severe for drugs and least for self-harm, but not significantly so.

Emoji Cipher

From Table 9, we can see that the rates of success, failure, and others are similar across the 3 categories. For each category, success accounts for the maximum number of cases, whereas others account for the least number of cases. The rates of success for each harm category are 67.07% (self-harm), 73.44% (violence), and 64.67% (drugs).

A χ² test of independence was performed to assess whether language is significantly associated with the observed outcomes for the emoji cipher (Table 9). The results show a significant difference in result rates across the languages at the 95% confidence level with χ²₄=43.11 and P<.001. Hence, the health safety risks of LLMs to the emoji cipher are different across harm types. They are most severe for violence and least severe for drugs.

All Ciphers

Table 10 summarizes the outcomes of jailbreak attempts across categories for both ciphers. The proportions of successful, failed, and other attempts are broadly similar across the 3 categories. In each category, successful attempts constitute the largest share, while the “other” category accounts for the fewest cases. The success rates by harm category are 66.07% for self-harm, 71.03% for violence, and 67.50% for drugs.

A chi-square test of independence was performed to assess whether language is significantly associated with the observed outcomes for the emoji cipher (Table 10). The results show a significant difference in result rates across the languages at the 95% confidence level with χ²₄=30.30 and P<.001. Hence, the health safety risks of LLMs to both ciphers are different across harm types. They are most severe for violence and least severe for self-harm.

Discussion

This study evaluated the multilingual safety vulnerabilities of LLMs in health-related contexts by applying cipher-style adversarial attacks. Across all analyses, more than half of the prompts bypassed safety filters, confirming that LLMs remain vulnerable to prompt obfuscation. We found statistically significant differences in attack success rates across languages, cipher types, and harm categories, with Hindi showing the highest vulnerability and Spanish the lowest.

RQ1: Language Differences in Safety Performance

ChatGPT-4’s safety varied notably by language, with Hindi prompts achieving an 83% ASR, compared to 66.6% in English and 59.95% in Spanish. These differences were consistent across cipher types and statistically significant. This finding aligns with recent work in medical informatics, which shows that multilingual safety research is heavily skewed toward English, creating dangerous blind spots in non-English use cases [8]. It also echoes concerns in clinical AI evaluations that call for ensuring equity in LLM performance across linguistic communities and health equity contexts. Our findings suggest that LLMs may be underprepared to detect harmful content in languages underrepresented in training data, potentially exacerbating inequities in multilingual health communication [23-25].

RQ2: Cipher Effectiveness Varies by Language

We found that the emoji cipher is more successful than the permutation cipher for the Spanish and English languages, but the permutation cipher is more successful than the emoji cipher for the Hindi language. This likely reflects differences in how LLMs process transliterations and semantic variations. Specifically, permutation ciphers on transliterated Hindi may disrupt pattern recognition more effectively than emojis. These results highlight the importance of developing cipher-aware safety mechanisms that are sensitive to language-specific encoding strategies. This aligns with broader research on security vulnerabilities in medical LLMs, such as prompt injection threats in clinical domains [26].

RQ3: Differential Vulnerability by Harm Category

ASRs were consistently high across self-harm, violence, and drug-related prompts. On aggregate, attacks were most successful within the violence category and least successful with drugs. Furthermore, while the health safety risks of LLMs to permutation cipher were not statistically different across harm types, they were found to be significant with the emoji cipher. This suggests a nuanced interplay between harm types, types of cipher attacks, and the language used. Future mitigation strategies should prioritize certain harm types for protection (eg, violence) but also recognize that the mitigation strategies should consider the attack type and the language used.

These results have several implications for health informatics. Ensuring equitable safety performance across languages is essential for LLMs embedded in clinical decision support systems, patient portals, and public health messaging. Health informatics frameworks should extend beyond English to include low-resource languages and diverse encoding styles. Tools like L2M3 [27], a multilingual medical LLM for underserved regions, point toward important directions in this area [26,28]. Moreover, the demonstrated success of cipher-based attacks highlights the need for multilayered defense mechanisms. Adversarial prompt injections and poisoned fine-tuning are known threats in medical LLMs; defenses should be adjusted to cover cipher techniques across languages [29,30]. Finally, transparent and comprehensive evaluation frameworks are vital. Benchmarks such as XSAFETY and MedHELM [31] advocate for rigorous, multilingual evaluations within health care AI. Incorporating adversarial testing, particularly with cipher-based scenarios, should be part of routine assessment for clinical LLMs [24].

Limitations

While our findings are based on the GPT-4 model available during the data collection period (April 2024), the central aim of this formative study is to motivate and ground new research into how vulnerabilities to cipher-style attacks may differ across languages in health-related tasks. Our approach involved detailed human-level coding to ensure depth and contextual accuracy in evaluating safety across languages. This method distinguishes our work from prior studies that relied on LLMs as evaluators and reflects our commitment to producing careful, interpretable results, even if it requires a longer analysis timeline.

The input prompts from BeaverTails were automatically translated into Hindi and English. While expert humans validated all such translations, we acknowledge that the use of automated tools, such as Google Translate, may have introduced some peculiarities that could influence specific outcomes. Each non-English language was reviewed by a bilingual human coder; hence, we do not provide intercoder reliability. Although this review process adds credibility to our process, we realize that not having multiple reviewers could be a limitation. Additionally, the study focused on a single dataset (BeaverTails) and 3 harm categories. While this approach allows for in-depth analysis, it may limit the generalizability of the findings to other datasets or types of harm.

Broader Implications

In summary, this work provides a reproducible framework for evaluating language-specific vulnerabilities in LLMs used in health care. The uneven distribution of safety across languages and attack styles highlights the urgent need for cipher-aware, multilingual safety alignment. As LLMs become increasingly integrated into health informatics infrastructure, ensuring robust and equitable safeguards for all languages becomes both a technical and public health imperative.

Future work can expand these foundations by exploring a broader range of languages, datasets, and harm categories, as well as by comparing model behaviors across newer versions of LLMs. We hope this study serves as a starting point for deeper investigations into language-specific vulnerabilities in health-related AI applications.

Conclusions

This study highlights critical safety vulnerabilities in LLMs used for health communication, particularly when exposed to cipher-style adversarial attacks. The findings show that ASRs vary significantly across languages, with Hindi prompts being the most susceptible. Differences in cipher effectiveness and harm categories further reveal that current safety mechanisms are not uniformly reliable, especially in multilingual contexts. These disparities raise important concerns for the equitable and secure deployment of LLMs in clinical and public health settings.

To address these risks, future work must prioritize multilingual safety alignment and develop defenses that account for language-specific encoding strategies. Evaluation frameworks should incorporate adversarial testing across diverse languages and harm types to ensure robust performance. As LLMs continue to shape health informatics tools and workflows, safeguarding their use across linguistic boundaries is essential for promoting inclusive and responsible AI in health care.