TY  - JOUR
AU  - Gendrin, Aline
AU  - Souliotis, Leonidas
AU  - Loudon-Griffiths, James
AU  - Aggarwal, Ravisha
AU  - Amoako, Daniel
AU  - Desouza, Gregory
AU  - Dimitrievska, Sashka
AU  - Metcalfe, Paul
AU  - Louvet, Emilie
AU  - Sahni, Harpreet
PY  - 2023
DA  - 2023/6/22
TI  - Identifying Patient Populations in Texts Describing Drug Approvals Through Deep Learning–Based Information Extraction: Development of a Natural Language Processing Algorithm
JO  - JMIR Form Res
SP  - e44876
VL  - 7
KW  - algorithm
KW  - artificial intelligence
KW  - BERT
KW  - cancer
KW  - classification
KW  - data extraction
KW  - data mining
KW  - deep-learning
KW  - development
KW  - drug approval
KW  - free text
KW  - information retrieval
KW  - line of therapy
KW  - machine learning
KW  - natural language processing
KW  - NLP
KW  - oncology
KW  - pharmaceutic
KW  - pharmacology
KW  - pharmacy
KW  - stage of cancer
KW  - text extraction
KW  - text mining
KW  - unstructured data
AB  - Background: New drug treatments are regularly approved, and it is challenging to remain up-to-date in this rapidly changing environment. Fast and accurate visualization is important to allow a global understanding of the drug market. Automation of this information extraction provides a helpful starting point for the subject matter expert, helps to mitigate human errors, and saves time. Objective: We aimed to semiautomate disease population extraction from the free text of oncology drug approval descriptions from the BioMedTracker database for 6 selected drug targets. More specifically, we intended to extract (1) line of therapy, (2) stage of cancer of the patient population described in the approval, and (3) the clinical trials that provide evidence for the approval. We aimed to use these results in downstream applications, aiding the searchability of relevant content against related drug project sources. Methods: We fine-tuned a state-of-the-art deep learning model, Bidirectional Encoder Representations from Transformers, for each of the 3 desired outputs. We independently applied rule-based text mining approaches. We compared the performances of deep learning and rule-based approaches and selected the best method, which was then applied to new entries. The results were manually curated by a subject matter expert and then used to train new models. Results: The training data set is currently small (433 entries) and will enlarge over time when new approval descriptions become available or if a choice is made to take another drug target into account. The deep learning models achieved 61% and 56% 5-fold cross-validated accuracies for line of therapy and stage of cancer, respectively, which were treated as classification tasks. Trial identification is treated as a named entity recognition task, and the 5-fold cross-validated F1-score is currently 87%. Although the scores of the classification tasks could seem low, the models comprise 5 classes each, and such scores are a marked improvement when compared to random classification. Moreover, we expect improved performance as the input data set grows, since deep learning models need to be trained on a large enough amount of data to be able to learn the task they are taught. The rule-based approach achieved 60% and 74% 5-fold cross-validated accuracies for line of therapy and stage of cancer, respectively. No attempt was made to define a rule-based approach for trial identification. Conclusions: We developed a natural language processing algorithm that is currently assisting subject matter experts in disease population extraction, which supports health authority approvals. This algorithm achieves semiautomation, enabling subject matter experts to leverage the results for deeper analysis and to accelerate information retrieval in a crowded clinical environment such as oncology. 
SN  - 2561-326X
UR  - https://formative.jmir.org/2023/1/e44876
UR  - https://doi.org/10.2196/44876
UR  - http://www.ncbi.nlm.nih.gov/pubmed/37347514
DO  - 10.2196/44876
ID  - info:doi/10.2196/44876
ER  -