NIH/NCI Request for Information on Benchmarks for AI in Cancer Research
The National Institutes of Health (NIH) National Cancer Institute (NCI) has released a request for information (RFI) seeking input on benchmark development and benchmark datasets needed for the utilization, validation, and application of artificial intelligence (AI) in cancer research and care. The RFI aims to elucidate and evaluate opportunity spaces for AI in cancer research with regards to AI benchmark development, dataset characteristics, and structural barriers in order to advance cancer prevention, detection, and treatment.
Specifically, NCI is soliciting community feedback to answer any or all of the following questions:
“What are AI-relevant use cases or tasks in cancer research and care that could be advanced through the availability of high-quality benchmarks? Please be as specific as possible; e.g., brain tumor image segmentation; cancer treatment extraction from EHR data; somatic variant calling from long read sequencing data. Of particular interest are 1) use cases and tasks with broad relevance; and 2) use cases and tasks where benchmarks are currently scarce.
What are the desired characteristics of benchmarks for these use cases, including but not limited to considerations of quality, utility, and availability?
What datasets currently exist that could contribute to are be adopted for benchmarking? Please include information about their size, annotation, and availability, as well as AI use cases they could support.
What are the biggest barriers to creating and/or using AI benchmarks in cancer research and care?”
NCI welcomes input from various stakeholders including researchers, scientists, administrators, and healthcare professionals. Responses to the RFI are due July 29, 2025, and can be submitted here.