Comparing traditional NLP methods and LLM-based extraction for identifying biomarkers in lung cancer

We analyzed 27 de-identified clinical notes from 29 lung cancer cases with biomarker data. The notes were sourced from different hospitals, after ethical committee approval, ensuring variability in documentation. Notes were processed by "SLM & LLM-based NER model" and "pre-transformed traditional NER." F1 scores of both the models were compared for clinically relevant attributes of genomic markers, such as categorical results, exonic location, variant type, and genomic alterations.

The SLM and LLM based NER model outperformed the traditional NER model in identifying the biomarker entity, variant type and categorical results (Table). In addition, a qualitative assessment of other attributes like exon location and genomic alterations which were not available through traditional NER models and could be extracted satisfactorily through the SLM and LLM based NER model for e.g. MET exon 14 and EGFR genomic alteration had F1 score of 0.8 and 0.75, respectively.

In precision oncology, identifying biomarker variants is crucial for targeted interventions. Clinical notes are a rich source of patient information including genomic data, making them key evidence to enrich the database. Our study demonstrates that SLM and LLM based NER models are better at distinguishing contextual information, improving their ability to perform precise information extraction, such as differentiating between 'EGFR' as a biomarker and 'eGFR’ as lab test and hence can significantly aid in extracting precision oncology data from unstructured clinical notes. This approach enhances the ability to support personalized treatments and clinical trials.