Scientists involved in the project emphasize that capability should be judged
Researchers at Stanford University have launched Terminal-Bench-Science, a benchmark designed to evaluate AI agents on authentic scientific research tasks. Developed by the team behind Terminal-Bench in collaboration with domain experts across multiple disciplines, the initiative shifts evaluation power from model developers to practicing scientists who define what constitutes meaningful AI assistance in research. The benchmark focuses on end-to-end workflows drawn directly from researchers' own projects, including literature review, hypothesis formation, experimental design, and data interpretation. Unlike traditional AI assessments that rely on standardized datasets or synthetic tasks, Terminal-Bench-Science uses real-world scientific challenges to measure how well AI agents support actual discovery processes.
Breaking news
Hugging Face Launches $399 Robot Duck for AI Experimentation
5 things I would tell someone setting up their first home server this weekend
AI Engineer Notebooks Offer Free Hands-On LLM Training on Colab
Anthropic and OpenAI headline AI Stage at TechCrunch Disrupt 2026Scientists involved in the project emphasize that capability should be judged by utility in genuine research contexts, not performance on isolated metrics. How Scientists Shape the Evaluation Criteria By centering the benchmark on researcher-defined tasks, Terminal-Bench-Science ensures that progress in AI aligns with the practical needs of scientific work. Domain experts from fields such as biology, chemistry, and physics contributed workflows reflecting common bottlenecks in their research, such as troubleshooting experimental protocols or synthesizing findings from fragmented literature. This approach aims to reveal whether AI agents can truly augment scientific ## Can AI Agents Keep Up with the Pace of Scientific Inquiry? Early testing shows that while current AI agents excel at retrieving information and generating text, they struggle with iterative Scientists note that agents often fail to recognize when a hypothesis needs revision based on contradictory data, highlighting a gap between pattern recognition and true investigative thinking.
The benchmark will be updated regularly as new workflows are submitted by researchers worldwide. Frequently Asked Questions What makes Terminal-Bench-Science different from other AI benchmarks? It uses actual scientific workflows provided by researchers rather than artificial tasks, ensuring evaluation reflects real research demands and priorities set by scientists themselves. Which scientific fields are currently represented in the benchmark? The initial workflows span biology, chemistry, and physics, with plans to expand to earth sciences, engineering, and social sciences as more researchers contribute. How can scientists participate in Terminal-Bench-Science? Researchers can submit their own research workflows through the project’s website to be included in future benchmark iterations, helping shape how AI capabilities are measured in science.
