Methods
This study analyzed over 500,000 registered trials from ClinicalTrials.gov, focused on assessing the intelligence of 11 major CRC trials from leading sponsors. Our AI system evaluates trial competition across four key dimensions: how similar the trials are overall, overlap in patient eligibility criteria, similarity of clinical endpoints, and whether trials are recruiting during the same timeframe. The system uses three specialized AI agents working in sequence: a Screener that identifies potentially relevant trials, a Matcher that performs detailed comparisons, and a Ranker that generates final competitiveness scores. The technology converts trial text into numerical representations (called embedding) that allow precise similarity measurements between different trials, using OpenAI’s text-embedding-large model and Claude-4-Sonnet LLM for semantic understanding.
Results
The system successfully distinguished between truly competing and non-competing trials. For example, trials targeting opposite patient populations (MSI-H/dMMR versus MSS/pMMR) received low cosine similarity scores of 0.1, correctly identifying them as distinct. Conversely, trials with similar HER2+ eligibility criteria, despite different wording, received high similarity scores of 0.71, flagging them as potentially competitive. When analyzing the most similar trials, the system achieved high accuracy with average similarity scores of 0.78 for trial summaries and 0.71 for eligibility criteria. As expected, expanding the analysis to include more trials (from 5 to 50) included less relevant studies, with scores appropriately decreasing to reflect lower competition levels.
Conclusions
Our AI-powered system provides clinicians and trial teams with a valuable tool to quickly and efficiently analyze competitive trials. Its scalability and indication-agnostic design enable broad application across therapeutic areas for competitive intelligence and strategic trial planning.
.svg)
