A multi-agent AI system integrating real-world data and competitive intelligence to generate automated trial eligibility recommendations

The framework integrates Precision360 RWD and a vectorized ClinicalTrials.gov database accessed through internal APIs and uses proprietary LLMs as the reasoning engine. Draft eligibility criteria are evaluated by three agents: (1) a CI agent applying embedding-based similarity with a 30% biomarker/indication weighting to prioritize trials within the same disease context; (2) a SoC agent benchmarking criteria against real-world treatment pathways; and (3) an ET agent detecting recently approved therapies. Each agent issues criterion-level recommendations with rationale, which are consolidated into a unified output. Validation included 30 oncology trials (NSCLC, breast, colorectal), with two clinical scientists ( > 5yr experience) independently rating outputs on a 1–5 scale. Weighted F1-scores measured concordance (predefined threshold ≥0.80).

Overall agent–scientist concordance was 95% (range 93–98%). The CI agent performed best with 5–10 competitor trials and correctly identified biomarker and indication alignment for 87% of evaluations (26/30). Across trials, the agents identified opportunities to broaden eligibility that were consistent with competitive trial standards and clinical scientist assessment. Example recommendations included alignment of eligibility thresholds with ranges observed in competitor trials within the same disease and treatment setting, such as hematologic criteria (e.g., ANC ≥1500 to ≥1200/µL), renal function cutoffs (creatinine clearance ≥45 to ≥40 mL/min), and functional status requirements (ECOG ≤1 to ≤2). These suggestions were derived from competitive trial patterns and were intended to flag potential opportunities for protocol adjustment. The SoC and ET agents each achieved F1 ≥0.92. In an EGFR exon19del example, the SoC agent found osimertinib representing ~60% of first-line RWD use, while the ET agent identified rising adoption of amivantamab with marked growth in 2024 and ~40% year-over-year increase, supporting inclusion of this regimen as an acceptable prior therapy.

This multi-agent AI framework enables automated, transparent evaluation of eligibility criteria with expert-level concordance and identifies optimization opportunities informed by competitive and RWD evidence. It helps identify overly restrictive criteria that limit enrollment feasibility and contribute to protocol amendments and is extensible across therapeutic areas.