Atopic dermatitis (AD) treatment has broadened since 2017, with several new, targeted, highly efficacious systemic therapies. Patients’ personal experiences with these novel therapies are largely unknown and unreported. Reddit is a rich source of patient perspectives on dermatology [1]. Previous studies have used traditional natural language processing (NLP) methods to extract meaningful information from unstructured social media data [2], but more relevant findings could potentially be extracted by applying large language models (LLMs) to such language data, including large-scale Reddit datasets, because tailored prompts can extract more specific and nuanced insights.

GPT-4o demonstrated superior performance across standard classification metrics (Table 1), with precision of 0.87, recall of 0.82, F₁-score of 0.82 (support-weighted across classes), and accuracy of 0.82, compared to VADER (precision: 0.42; recall: 0.38; F₁-score: 0.37; accuracy: 0.38) and DistilBERT (precision: 0.58; recall: 0.56; F₁-score: 0.56; accuracy: 0.56). The medications with the highest proportion of comments in the full dataset tagged as positive by GPT-4o (Figure 1) were upadacitinib (673/2107, 31.9%) and dupilumab (7724/25,926, 29.8%). Abrocitinib had a smaller percentage of negative sentiments than other medications (Figure 1). Dupilumab and upadacitinib had a high percentage of positive sentiment, which may suggest that many individuals are satisfied with the efficacy and safety of these medications.

Examples from the dataset illustrate the range of patient experiences. Positive comments expressed enthusiasm about treatment options, such as one patient’s response to abrocitinib approval: “Just saw it was just approved! Calling my dermo on Monday!” Negative sentiment reflected disappointment with treatment outcomes, as seen in one dupilumab user’s comment: “I thought Dupixent would provide relief too, but I’m still itchy and inflamed.” Neutral comments predominantly involved informational exchanges, such as questions about accessing treatment and questions about clinical trial participation.

GPT-4o showed substantially better agreement with clinician judgment than traditional NLP methods when classifying patient sentiment in Reddit comments about AD treatments. The superior performance across all metrics establishes the feasibility of applying LLMs to extract patient perspectives from unstructured social media data. However, it is important to note that we excluded comments when the annotators disagreed to ensure a clear reference standard. This approach potentially biased the validation set toward straightforward sentiment examples and likely inflated the reported performance metrics, as the models were not tested on ambiguous, mixed-emotion, or context-dependent language, which is common in real patient discussions. Simple task-specific prompts applied to LLMs may yield more detailed insights than traditional NLP methods. LLMs offer advantages through basic zero-shot prompts that can be adapted for specific tasks like drug-specific sentiment analysis in medical discussions, whereas general-purpose sentiment analysis tools may struggle with medical terminology and context without additional training or customization. However, LLM-based approaches also have limitations, including computational costs and the need for careful prompt design to ensure reliable results. The predominance of neutral sentiment across all medications shows that many comments serve informational or question-asking purposes rather than expressing clear sentiment. Clinically, the observed higher positive sentiment for upadacitinib and dupilumab in our dataset may reflect patient satisfaction with these treatments, while the lower negative sentiment observed with abrocitinib could indicate fewer patient-reported concerns. Similar approaches could illuminate real-world perspectives on treatments for psoriasis and other chronic skin conditions where multiple therapeutic options exist. While our findings provide initial insights for AD treatment discussions, they also suggest potential broader applications for analyzing patient perspectives across medical fields. Future studies should expand beyond simple sentiment categorization to capture more nuanced patient experiences, including mixed sentiments, specific concerns about side effects, cost considerations, and conditional satisfaction with treatment. Additionally, future research should explore temporal trends in sentiment as medications gain market share and correlate social media sentiment with real-world evidence databases and patient registries.