Evaluating Locally Run Large Language Models (Gemma 2, Mistral Nemo, and Llama 3) for Outpatient Otorhinolaryngology Care: Retrospective Study

Christoph Raphael Buhr   ^{1, 2} , MSc, MD ;   Christopher Seifen   ¹ , MD ;   Katharina Bahr-Hamm   ¹ , MD ;   Tilman Huppertz   ¹ , MD ;   Johannes Pordzik   ¹ , MD ;   Harry Smith   ³ , MSc ;   Tom Kelsey   ³ , MSc ;   Andrew Blaikie   ² , MD ;   Christoph Matthias   ¹ , MD ;   Sebastian Kuhn   ⁴ , MD ;   Jonas Eckrich   ¹ , MD

¹ Department of Otorhinolaryngology, University Medical Center of the Johannes Gutenberg-University Mainz, Mainz, Germany

² School of Medicine, University of St Andrews, St Andrews, United Kingdom

³ School of Computer Science, University of St Andrews, St Andrews, United Kingdom

⁴ Institute for Digital Medicine, Philipps University Marburg, University Hospital Giessen and Marburg, Marburg, Germany