background

Diabetes affects millions worldwide. Primary care physicians provide a significant portion of care, and they often struggle with selecting appropriate medications.

objective

We sought to develop a model that accurately predicts what drug an endocrinologist would prescribe based on the current measurements. The goal was to create a system that would assist non-specialists in choosing medications, thereby potentially improving diabetes treatment outcomes. Based on the performance of prior studies, we set a performance target of achieving a receiver operating characteristic area under the curve (ROC-AUC) above 0.95.

methods

A Transformer-based encoder-decoder model predicts whether 44 types of diabetes drugs will be prescribed. The model uses sequences of age, sex, history for 12 lab tests, and prescribed drug history as inputs. We assessed the model using the Electronic Health Records (EHRs) from 7034 diabetes patients seeing endocrinologists between 2012 and 2022 at the University of Tokyo Hospital. We assessed model performance trained on data subsets spanning different time periods (2, 5, and 10 years) using micro- and macro-averaged ROC-AUC on a hold-out test set comprising data solely from 2022. The model's performance was compared against LightGBM.

results

The model trained on data from the past 5 years (2017-2021) yielded the best predictive performance, achieving a micro average (95% CI) ROC-AUC of 0.993 (0.992, 0.994) and a macro average (95% CI) ROC-AUC of 0.988 (0.980, 0.993). The model achieved an ROC-AUC above 0.95 for 43 out of 44 drugs. These results surpassed the predefined performance target and outperformed both previous studies and the Light GBM model’s micro-average ROC AUC of 0.988 (0.985, 0.990) in terms of prediction accuracy. Furthermore, training the model with short-term data from the past 5 years yielded high accuracy compared to using data from the past 10 years, suggesting that learning from more recent prescribing patterns might be advantageous.

conclusions

The proposed model demonstrates the feasibility of accurately predicting the next prescribed drugs. This model, trained from past prescriptions of endocrinologists, has the potential to provide information that can assist non-specialists in making diabetes treatment decisions. Future studies will focus on incorporating important factors such as prescription contraindications and constraints to enhance safety, as well as leveraging large-scale clinical data across multiple hospitals to improve the generalizability of the model.

clinicaltrial