TY  - JOUR
AU  - Abuyaman, Omar
PY  - 2023
DA  - 2023/11/10
TI  - Strengths and Weaknesses of ChatGPT Models for Scientific Writing About Medical Vitamin B12: Mixed Methods Study
JO  - JMIR Form Res
SP  - e49459
VL  - 7
KW  - AI
KW  - ChatGPT
KW  - GPT-4
KW  - GPT-3.5
KW  - vitamin B12
KW  - artificial intelligence
KW  - language editing
KW  - wide range information
KW  - AI solutions
KW  - scientific content
AB  - Background: ChatGPT is a large language model developed by OpenAI designed to generate human-like responses to prompts. Objective: This study aims to evaluate the ability of GPT-4 to generate scientific content and assist in scientific writing using medical vitamin B12 as the topic. Furthermore, the study will compare the performance of GPT-4 to its predecessor, GPT-3.5. Methods: The study examined responses from GPT-4 and GPT-3.5 to vitamin B12–related prompts, focusing on their quality and characteristics and comparing them to established scientific literature. Results: The results indicated that GPT-4 can potentially streamline scientific writing through its ability to edit language and write abstracts, keywords, and abbreviation lists. However, significant limitations of ChatGPT were revealed, including its inability to identify and address bias, inability to include recent information, lack of transparency, and inclusion of inaccurate information. Additionally, it cannot check for plagiarism or provide proper references. The accuracy of GPT-4’s answers was found to be superior to GPT-3.5. Conclusions: ChatGPT can be considered a helpful assistant in the writing process but not a replacement for a scientist’s expertise. Researchers must remain aware of its limitations and use it appropriately. The improvements in consecutive ChatGPT versions suggest the possibility of overcoming some present limitations in the near future. 
SN  - 2561-326X
UR  - https://formative.jmir.org/2023/1/e49459
UR  - https://doi.org/10.2196/49459
UR  - http://www.ncbi.nlm.nih.gov/pubmed/37948100
DO  - 10.2196/49459
ID  - info:doi/10.2196/49459
ER  -