This paper is in the following e-collection/theme issue:

Formative Evaluation of Digital Health Interventions (2491) Theme Issue: ChatGPT and Generative Language Models in Medical Education (86) Artificial Intelligence (1421) e-Learning and Digital Medical Education (705) Testing and Assessment in Medical Education (55)

Published on in Vol 7 (2023)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/48023, first published .
Accuracy of ChatGPT on Medical Questions in the National Medical Licensing Examination in Japan: Evaluation Study

Accuracy of ChatGPT on Medical Questions in the National Medical Licensing Examination in Japan: Evaluation Study

Accuracy of ChatGPT on Medical Questions in the National Medical Licensing Examination in Japan: Evaluation Study

Authors of this article:

Yasutaka Yanagita1 Author Orcid Image ;   Daiki Yokokawa1 Author Orcid Image ;   Shun Uchida1 Author Orcid Image ;   Junsuke Tawara2 Author Orcid Image ;   Masatomi Ikusaka1 Author Orcid Image
Article Authors Cited by (36) Tweetations (11) Metrics

Journals

  1. Kim J, Kim S, Choi J, Lee Y. Reliability of ChatGPT for performing triage task in the emergency department using the Korean Triage and Acuity Scale. DIGITAL HEALTH 2024;10 View
  2. Sallam M, Barakat M, Sallam M. A Preliminary Checklist (METRICS) to Standardize the Design and Reporting of Studies on Generative Artificial Intelligence–Based Models in Health Care Education and Practice: Development Study Involving a Literature Review. Interactive Journal of Medical Research 2024;13:e54704 View
  3. Chau R, Thu K, Yu O, Lo E, Hsung R, Lam W. Response to Generative AI in Dental Licensing Examinations: Comment. International Dental Journal 2024;74(4):897 View
  4. Yokokawa D, Yanagita Y, Li Y, Yamashita S, Shikino K, Noda K, Tsukamoto T, Uehara T, Ikusaka M. For any disease a human can imagine, ChatGPT can generate a fake report. Diagnosis 2024;11(3):329 View
  5. Pinto V, de Azevedo M, Wroclawski M, Gentile G, Jesus V, de Bessa Junior J, Nahas W, Sacomani C, Sandhu J, Gomes C. Conformity of ChatGPT recommendations with the AUA/SUFU guideline on postprostatectomy urinary incontinence. Neurourology and Urodynamics 2024;43(4):935 View
  6. Nakao T, Miki S, Nakamura Y, Kikuchi T, Nomura Y, Hanaoka S, Yoshikawa T, Abe O. Capability of GPT-4V(ision) in the Japanese National Medical Licensing Examination: Evaluation Study. JMIR Medical Education 2024;10:e54393 View
  7. Noda M, Ueno T, Koshu R, Takaso Y, Shimada M, Saito C, Sugimoto H, Fushiki H, Ito M, Nomura A, Yoshizaki T. Performance of GPT-4V in Answering the Japanese Otolaryngology Board Certification Examination Questions: Evaluation Study. JMIR Medical Education 2024;10:e57054 View
  8. Sato H, Ogasawara K. ChatGPT (GPT-4) passed the Japanese National License Examination for Pharmacists in 2022, answering all items including those with diagrams: a descriptive study. Journal of Educational Evaluation for Health Professions 2024;21:4 View
  9. Kawahara T, Sumi Y. GPT-4/4V's performance on the Japanese National Medical Licensing Examination. Medical Teacher 2024:1 View
  10. Zhu L, Mou W, Hong C, Yang T, Lai Y, Qi C, Lin A, Zhang J, Luo P. The Evaluation of Generative AI Should Include Repetition to Assess Stability. JMIR mHealth and uHealth 2024;12:e57978 View
  11. Jedrzejczak W, Skarzynski P, Raj-Koziak D, Sanfins M, Hatzopoulos S, Kochanek K. ChatGPT for Tinnitus Information and Support: Response Accuracy and Retest after Three and Six Months. Brain Sciences 2024;14(5):465 View
  12. Bharatha A, Ojeh N, Fazle Rabbi A, Campbell M, Krishnamurthy K, Layne-Yarde R, Kumar A, Springer D, Connell K, Majumder M. Comparing the Performance of ChatGPT-4 and Medical Students on MCQs at Varied Levels of Bloom’s Taxonomy. Advances in Medical Education and Practice 2024;Volume 15:393 View
  13. Hirano Y, Hanaoka S, Nakao T, Miki S, Kikuchi T, Nakamura Y, Nomura Y, Yoshikawa T, Abe O. GPT-4 Turbo with Vision fails to outperform text-only GPT-4 Turbo in the Japan Diagnostic Radiology Board Examination. Japanese Journal of Radiology 2024;42(8):918 View
  14. Yanagita Y, Yokokawa D, Fukuzawa F, Uchida S, Uehara T, Ikusaka M. Expert assessment of ChatGPT’s ability to generate illness scripts: an evaluative study. BMC Medical Education 2024;24(1) View
  15. Liu M, Okuhara T, Chang X, Shirabe R, Nishiie Y, Okada H, Kiuchi T. Performance of ChatGPT Across Different Versions in Medical Licensing Examinations Worldwide: Systematic Review and Meta-Analysis. Journal of Medical Internet Research 2024;26:e60807 View
  16. Rossettini G, Rodeghiero L, Corradi F, Cook C, Pillastrini P, Turolla A, Castellini G, Chiappinotto S, Gianola S, Palese A. Comparative accuracy of ChatGPT-4, Microsoft Copilot and Google Gemini in the Italian entrance test for healthcare sciences degrees: a cross-sectional study. BMC Medical Education 2024;24(1) View
  17. Hsieh C, Hsieh H, Lin H. Evaluating the performance of ChatGPT-3.5 and ChatGPT-4 on the Taiwan plastic surgery board examination. Heliyon 2024;10(14):e34851 View
  18. Ishida K, Hanada E. Potential of ChatGPT to Pass the Japanese Medical and Healthcare Professional National Licenses: A Literature Review. Cureus 2024 View
  19. Ishida K, Arisaka N, Fujii K. Analysis of Responses of GPT-4 V to the Japanese National Clinical Engineer Licensing Examination. Journal of Medical Systems 2024;48(1) View
  20. Jin H, Lee H, Kim E. Performance of ChatGPT-3.5 and GPT-4 in national licensing examinations for medicine, pharmacy, dentistry, and nursing: a systematic review and meta-analysis. BMC Medical Education 2024;24(1) View
  21. Sallam M, Al-Salahat K, Eid H, Egger J, Puladi B. Human versus Artificial Intelligence: ChatGPT-4 Outperforming Bing, Bard, ChatGPT-3.5 and Humans in Clinical Chemistry Multiple-Choice Questions. Advances in Medical Education and Practice 2024;Volume 15:857 View
  22. Fujimoto M, Kuroda H, Katayama T, Yamaguchi A, Katagiri N, Kagawa K, Tsukimoto S, Nakano A, Imaizumi U, Sato-Boku A, Kishimoto N, Itamiya T, Kido K, Sanuki T. Evaluating Large Language Models in Dental Anesthesiology: A Comparative Analysis of ChatGPT-4, Claude 3 Opus, and Gemini 1.0 on the Japanese Dental Society of Anesthesiology Board Certification Exam. Cureus 2024 View
  23. Yanagita Y, Yokokawa D, Uchida S, Li Y, Uehara T, Ikusaka M. Can AI-Generated Clinical Vignettes in Japanese Be Used Medically and Linguistically?. Journal of General Internal Medicine 2024;39(16):3282 View
  24. Ramgopal S, Varma S, Gorski J, Kester K, Shieh A, Suresh S. Evaluation of a Large Language Model on the American Academy of Pediatrics' PREP Emergency Medicine Question Bank. Pediatric Emergency Care 2024;40(12):871 View
  25. Song E, Lee S. Comparative Analysis of the Response Accuracies of Large Language Models in the Korean National Dental Hygienist Examination Across Korean and English Questions. International Journal of Dental Hygiene 2024 View
  26. Liu M, Okuhara T, Chang X, Okada H, Kiuchi T, Khlaif Z. Performance of ChatGPT in medical licensing examinations in countries worldwide: A systematic review and meta-analysis protocol. PLOS ONE 2024;19(10):e0312771 View
  27. Aster A, Laupichler M, Rockwell-Kollmann T, Masala G, Bala E, Raupach T. ChatGPT and Other Large Language Models in Medical Education — Scoping Literature Review. Medical Science Educator 2024 View
  28. Chen C, Li X, Luo H. Evaluation of accuracies of large language models in answering clinical questions related to Mediterranean diet on cardiodiabesity. Interdisciplinary Nursing Research 2024;3(3):157 View
  29. Sugawara Y, Hirakawa Y, Nangaku M. Telemedicine in nephrology: future perspective and solutions. Clinical Kidney Journal 2024;17(Supplement_2):ii1 View
  30. Ho C, Tian T, Ayers A, Aaron R, Phillips V, Wolf R, Mathioudakis N, Dai T, Klonoff D. Qualitative metrics from the biomedical literature for evaluating large language models in clinical decision-making: a narrative review. BMC Medical Informatics and Decision Making 2024;24(1) View
  31. Miyazaki Y, Hata M, Omori H, Hirashima A, Nakagawa Y, Eto M, Takahashi S, Ikeda M. Performance and Errors of ChatGPT-4o on the Japanese Medical Licensing Examination: Solving All Questions Including Images with Over 90% Accuracy (Preprint). JMIR Medical Education 2024 View
  32. Morishita M, Fukuda H, Yamaguchi S, Muraoka K, Nakamura T, Hayashi M, Yoshioka I, Ono K, Awano S. An exploratory assessment of GPT-4o and GPT-4 performance on the Japanese National Dental Examination. The Saudi Dental Journal 2024;36(12):1577 View
  33. Shen Y, Xu Y, Ma J, Rui W, Zhao C, Heacock L, Huang C. Multi-modal large language models in radiology: principles, applications, and potential. Abdominal Radiology 2024 View
  34. Maraqa N, Samargandi R, Poichotte A, Berhouet J, Benhenneda R. Comparing performances of french orthopaedic surgery residents with the artificial intelligence ChatGPT-4/4o in the French diploma exams of orthopaedic and trauma surgery. Orthopaedics & Traumatology: Surgery & Research 2024:104080 View
  35. Kaewboonlert N, Poontananggul J, Pongsuwan N, Bhakdisongkhram G. Factors Associated with Accuracy of Large Language Models Artificial Intelligence in Basic Medical Science Examinations: A Cross-Sectional Study (Preprint). JMIR Medical Education 2024 View
  36. Jin H, Kim E. Performance of GPT-3.5 and GPT-4 on the Korean Pharmacist Licensing Examination: Comparison Study. JMIR Medical Education 2024;10:e57451 View