We conducted an experimental study to assess the susceptibility of a consumer-facing LLM to cipher-based jailbreak attacks across 3 languages (English, Spanish, and Hindi). We used ChatGPT for this experiment as it was one of the most commonly used, freely accessible LLMs, and it was frequently being used to ask health-related questions. ChatGPT (GPT-4) was the flagship model from OpenAI at the time of data collection (April 2024). While all 3 languages had over a billion speakers, English was the most well-studied language in terms of natural language processing research, and Spanish was also relatively well resourced. Hindi, on the other hand, was widely spoken yet considered a low-resource language in natural language processing research [17]. For each language, we used 2 cipher techniques for jailbreak. Each combination of 3 languages and 2 cipher techniques consisted of a comprehensive evaluation using 1000 input queries. Thus, our study involved a total of 6000 query response pairs.

We used the BeaverTails dataset [14,21], an AI safety–focused collection containing human-labeled question-answer (QA) pairs across 14 harm categories. This is a well-cited public resource that aims to foster research on safety alignment in LLMs. It includes over 100,000 QA pairs in a wide variety of domains and includes expert feedback on harmlessness. From these categories, we selected the following three due to their relevance and potential for harm in the health context: (1) self-harm; (2) violence, aiding and abetting, and incitement (in this paper, we refer to this category as violence); and (3) drugs.

Once we identified relevant categories, we selected a subset of 1000 sentences from the BeaverTails dataset. Of the 1000 sentences, we had 167 cases for the self-harm category, 733 cases for the violence category, and 100 for the drugs category. This subset formed the English dataset. Second, we translated the English dataset into Spanish using Google Translate. The translations were reviewed for accuracy by a bilingual (Spanish-English) coauthor. This subset forms our Spanish dataset. Third, we translated the English sentences into Hindi, using Google Translate for transliteration in the Latin alphabet. These transliterations were also manually validated by a bilingual (Hindi-English) coauthor to ensure correctness. Thus, each of the 3 language datasets consisted of 1000 sentences, resulting in a total of 3000 sentences for analysis.

To enable the model to interpret ciphered inputs, we used a 15-shot learning approach. Initial exploration showed that 15 examples were sufficient for ChatGPT-4 to reliably recognize and process both emoji and permutation ciphers. These examples were presented as input-output pairs to establish the mapping logic for each cipher type. We applied 2 types of cipher transformations (emoji cipher attack and permutation cipher attack) to all input prompts and expected responses. The encodings were applied consistently across prompts and responses to preserve semantic alignment while visually obfuscating the content.

We interacted with ChatGPT-4 via OpenAI’s application programming interface, submitting the ciphered prompts and recording the corresponding ciphered outputs. The outputs were then deciphered to recover the plain-text responses, referred to as “Output Text.” A team of human coders evaluated each sentence-response pair to determine whether the model’s output contained harmful content aligned with the original prompt. All responses were independently coded by 2 annotators. For Spanish and Hindi, coders were bilingual and fluent in the respective languages to ensure accurate interpretation and classification.

The experimental workflow is best described through the visualization shown below in Figure 1.

The data collection workflow can be divided into two basic phases: (1) prompt generation for each of the 1000-set inputs and (2) vulnerability analysis of responses from GPT-4. In the prompt generation phase, 15 examples from the training set are used to create the ciphered versions for each entry in the 1000-item dataset. For the English language, each item in the training set and the 1000-item dataset goes directly to the cipher technique. However, each input goes through a translation process into the Spanish language and the transliterated Hindi language before being sent to the cipher process. This process results in 6000 input prompts (3 languages × 2 ciphers × 1000-item dataset). In the response collection phase, each item from the input prompts is fed to GPT-4 to output a ciphered response for each item. This ciphered response is then fed back to GPT-4 to convert the cipher back to the English language. For each input prompt, we collect a response and evaluate the results according to the codebook developed. The counts of these evaluations are tabulated and analyzed using the Pearson chi-square test of independence.

A structured codebook was developed to guide the classification of model responses into predefined outcome categories. This codebook, provided in Table 3, was used to ensure consistency and reliability across coders. The coding framework enabled a nuanced assessment of model behavior across languages and cipher types, supporting the broader goal of evaluating multilingual safety vulnerabilities in health-related contexts.

Each jailbreak attempt can have 1 of 5 different outcomes, as presented in Table 3. For analysis, we used 3 outcomes: success, failure, and others (a combination of all responses that were neither success nor failure).

The coders were trained in output labeling by 2 senior coauthors. A sample of 30 data points was coded by two of the authors, both fluent in the respective language. The intercoder agreement was found to be over 90% in all instances, and the student coauthors labeled the remaining data points on their own. Following a study by Abbasi et al [22], we use the attack success rate (ASR) and attack failure rate (AFR) to quantify the results. They represent the percentage of successful attacks and failed attacks, respectively, for each subset of 1000 attempts for the 2 ciphers and 3 languages.

A complete example of an item from the 1000-dataset is illustrated in Table S1 in Multimedia Appendix 1.

All study procedures complied with OpenAI’s usage policies and ethical guidelines. No personal or sensitive information was used or generated during the research, and all data were securely stored with access restricted to authorized research personnel. The dataset was fully anonymized, and all interactions with the model were conducted to ensure participant privacy and data integrity. The overarching goal of this study is to contribute to the development of health LLMs that are safe and equitable across languages. By addressing multilingual vulnerabilities in health-related contexts, this work aims to support more inclusive and socially beneficial AI systems. The project did not require approval from the institutional review board because there were no external human participants, and the Hindi and Spanish coders are coauthors of this paper. We used the publicly available Beavertails dataset and did not include any human participant data.

The detailed results for each of the permutation and emoji ciphers are presented in Table 4 (and the aggregation across the 2 ciphers is available in Table S2 in Multimedia Appendix 1). The total number of tests for each language and its splits across harm categories is shown in the second column. In both tables, the attack percentage success for each language and harm category within the language is given in the last column of the table. As can be seen, a sizable number (over 50% in every case) of prompts result in successful attacks.

We compare these high ASRs with a baseline to contextualize our interpretation. The baseline consists of a subset of the evaluation set used to produce responses to nonciphered, direct harmful prompts across the 3 languages, using the same proportional distribution of harm types. The results for this baseline setting are available in Tables S3 and S4 in Multimedia Appendix 1. The ASR is below 5% for each language in both types of cipher attacks. The baseline results indicate that the model (GPT-4) is generally robust to direct harmful prompts in all 3 languages. The substantial increase in vulnerability under cipher attacks underscores the importance of studying how this increased vulnerability varies across languages.

Next, we zoom in on subsets of results to answer the identified RQs.

Our first RQ focuses on differences in vulnerabilities across languages. The number of successful attacks across languages and cipher techniques is shown in Figure 2.

The attacks were most successful in Hindi (83%) and least successful in Spanish (59.95%). A chi-squared test revealed a statistically significant association between language and cipher technique, χ²₂=17.21 and P<.001. More details on the test are available in Table S5 in Multimedia Appendix 1.

Not every attack results in a clearly identifiable success or failure. As shown in Table 3, our codebook also includes the categories of not sure, confused, and no output. Hence, to further understand the jailbreak outcomes, we study the differences in success (part of ASR above), failure, and others (the sum of not sure, confused, and no output). The results of jailbreak outcomes for both ciphers are summarized in Table 5.

A chi-square test of independence was performed to assess whether language is significantly associated with the observed outcomes for the emoji cipher (Table 5). The results show a significant difference in result rates across the languages at a 95% confidence level with χ²₄=468.17 and P<.001. Hence, the health safety risks of LLMs to both ciphers combined are different across languages. These risks are most severe for Hindi and least severe for the Spanish language. The major implication here is that the health safety risks of LLMs vary across languages.

Our second RQ focuses on the differences in vulnerabilities across the considered cipher types. Figure 2 shows that the emoji cipher is more successful than the permutation cipher for the Spanish and English languages, whereas the permutation cipher is more successful than the emoji cipher for the Hindi language.

Our third RQ studies how the patterns of vulnerability vary across harm categories (self-harm, violence, and drugs). We study these patterns for the permutation cipher, the emoji cipher, and then combined across these 2 ciphers.

ChatGPT-4’s safety varied notably by language, with Hindi prompts achieving an 83% ASR, compared to 66.6% in English and 59.95% in Spanish. These differences were consistent across cipher types and statistically significant. This finding aligns with recent work in medical informatics, which shows that multilingual safety research is heavily skewed toward English, creating dangerous blind spots in non-English use cases [8]. It also echoes concerns in clinical AI evaluations that call for ensuring equity in LLM performance across linguistic communities and health equity contexts. Our findings suggest that LLMs may be underprepared to detect harmful content in languages underrepresented in training data, potentially exacerbating inequities in multilingual health communication [23-25].

We found that the emoji cipher is more successful than the permutation cipher for the Spanish and English languages, but the permutation cipher is more successful than the emoji cipher for the Hindi language. This likely reflects differences in how LLMs process transliterations and semantic variations. Specifically, permutation ciphers on transliterated Hindi may disrupt pattern recognition more effectively than emojis. These results highlight the importance of developing cipher-aware safety mechanisms that are sensitive to language-specific encoding strategies. This aligns with broader research on security vulnerabilities in medical LLMs, such as prompt injection threats in clinical domains [26].

ASRs were consistently high across self-harm, violence, and drug-related prompts. On aggregate, attacks were most successful within the violence category and least successful with drugs. Furthermore, while the health safety risks of LLMs to permutation cipher were not statistically different across harm types, they were found to be significant with the emoji cipher. This suggests a nuanced interplay between harm types, types of cipher attacks, and the language used. Future mitigation strategies should prioritize certain harm types for protection (eg, violence) but also recognize that the mitigation strategies should consider the attack type and the language used.

These results have several implications for health informatics. Ensuring equitable safety performance across languages is essential for LLMs embedded in clinical decision support systems, patient portals, and public health messaging. Health informatics frameworks should extend beyond English to include low-resource languages and diverse encoding styles. Tools like L2M3 [27], a multilingual medical LLM for underserved regions, point toward important directions in this area [26,28]. Moreover, the demonstrated success of cipher-based attacks highlights the need for multilayered defense mechanisms. Adversarial prompt injections and poisoned fine-tuning are known threats in medical LLMs; defenses should be adjusted to cover cipher techniques across languages [29,30]. Finally, transparent and comprehensive evaluation frameworks are vital. Benchmarks such as XSAFETY and MedHELM [31] advocate for rigorous, multilingual evaluations within health care AI. Incorporating adversarial testing, particularly with cipher-based scenarios, should be part of routine assessment for clinical LLMs [24].

While our findings are based on the GPT-4 model available during the data collection period (April 2024), the central aim of this formative study is to motivate and ground new research into how vulnerabilities to cipher-style attacks may differ across languages in health-related tasks. Our approach involved detailed human-level coding to ensure depth and contextual accuracy in evaluating safety across languages. This method distinguishes our work from prior studies that relied on LLMs as evaluators and reflects our commitment to producing careful, interpretable results, even if it requires a longer analysis timeline.

The input prompts from BeaverTails were automatically translated into Hindi and English. While expert humans validated all such translations, we acknowledge that the use of automated tools, such as Google Translate, may have introduced some peculiarities that could influence specific outcomes. Each non-English language was reviewed by a bilingual human coder; hence, we do not provide intercoder reliability. Although this review process adds credibility to our process, we realize that not having multiple reviewers could be a limitation. Additionally, the study focused on a single dataset (BeaverTails) and 3 harm categories. While this approach allows for in-depth analysis, it may limit the generalizability of the findings to other datasets or types of harm.

In summary, this work provides a reproducible framework for evaluating language-specific vulnerabilities in LLMs used in health care. The uneven distribution of safety across languages and attack styles highlights the urgent need for cipher-aware, multilingual safety alignment. As LLMs become increasingly integrated into health informatics infrastructure, ensuring robust and equitable safeguards for all languages becomes both a technical and public health imperative.

Future work can expand these foundations by exploring a broader range of languages, datasets, and harm categories, as well as by comparing model behaviors across newer versions of LLMs. We hope this study serves as a starting point for deeper investigations into language-specific vulnerabilities in health-related AI applications.

This study highlights critical safety vulnerabilities in LLMs used for health communication, particularly when exposed to cipher-style adversarial attacks. The findings show that ASRs vary significantly across languages, with Hindi prompts being the most susceptible. Differences in cipher effectiveness and harm categories further reveal that current safety mechanisms are not uniformly reliable, especially in multilingual contexts. These disparities raise important concerns for the equitable and secure deployment of LLMs in clinical and public health settings.

To address these risks, future work must prioritize multilingual safety alignment and develop defenses that account for language-specific encoding strategies. Evaluation frameworks should incorporate adversarial testing across diverse languages and harm types to ensure robust performance. As LLMs continue to shape health informatics tools and workflows, safeguarding their use across linguistic boundaries is essential for promoting inclusive and responsible AI in health care.