Methods
oncertAI RWD was used to train ML and NLP models for mets date in Non-Small Cell Lung Cancer (NSCLC). Labels were generated for curated mets date from unstructured EHR notes. To train the
model, possible mets dates, along with contextual information from patient documents, were extracted using NLP. Model features were developed from density of lab testing and cancer medication dates before and after rank ordered NLP extracted mets dates. An XGBoost classifier was trained on these features to select the optimal date called the integrated ML (IML) mets date. Patient360TM dataset comprising of expert curated clinical variables served as the ground truth (GT) for validation. Real-world
overall survival (rwOS) estimated from mets date until date of death, was compared between GT versus, NLP-extracted mets date and the IML estimated mets date. Model performance was also assessed by subgroups of age, gender, ECOG, and histology to assess bias induced by covariates.
Results
11,192 metastatic NSCLC patients from Patient360TM were included. Median rwOS from the GT mets date was 16.9 mos (95% CI: 16.4-17.5) compared to 13.0 mos (95% CI: 12.5-13.6) with NLP extracted mets date (diff=4.4 mos, logrank p<.005) and 17.5 mos (95% CI: 16.9-18.1) with IML estimated mets date (diff=0.6 mos, logrank p=0.76). The adjusted standardized mean difference (aSMD) were
0.1062 for GT vs. NLP and 0.0014 for GT vs. IML. Stratified analysis did not identify any differential results by subgroups. aSMDs for subgroups comparing GT vs. IML were between 0.001 to 0.09 and median rwOS differences with p>0.05.
Conclusions
ML-based models trained on expert curated RWD can significantly enhance the accuracy of
NLP extracted date by leveraging features from structured EHR when generating RWD from
unstructured EHR documents. This approach can be scaled across multiple date events to capture the temporal sequence of health events and generate meaningful insights about cancer patients’ journey. Future applications of this work across tumor types can provide a reliable approach to scale rapid insights from EHR documents.
.svg)
