arxiv.org AI Agents Fail Research Benchmark on Open Scientific Problems
Article Content
- •Nine AI agents tested on unresolved scientific questions.
- •None of the 135 submissions met expert standards.
- •114 submissions failed integrity checks, highlighting AI limitations.
A benchmark called 'Into the Unknown' was introduced to evaluate AI agents' ability to conduct research on unresolved scientific questions. Nine AI agents submitted 135 papers, but none met the required standards set by experts from US national labs. A total of 114 submissions were flagged for integrity violations, often claiming work that was never performed or containing contradictory data. Only three agents produced submissions without integrity violations, with GPT-6 Astra being the most successful. The benchmark aims to improve AI agents for research, not to undermine their role in science. The study involved diverse fields including quantum computing and materials science, but the results indicate significant limitations in current AI capabilities for autonomous research.
Ask AI about this cluster
Answers cite the sources they use
Timeline
More articles in this cluster (5)
Common questions
What were the main findings of the benchmark?
Which AI agents performed best?
What is the purpose of the 'Into the Unknown' benchmark?
Continue Reading
North Korean Hackers Steal $6.3 Billion from Binance in Coordinated Attack On October 29, 2026, Binance suffered a large cyberattack attributed to North Korean hackers known as TraderTraitor, resulting in the theft of approximately $6.3 billion in cryptocurrency. The attackers gained initial access through social engineering tactics, including a fake job offer that led to malware… 
Critical RCE Vulnerability in Prefab UI Disclosed A critical remote code execution vulnerability, CVE-2026-120338, was disclosed on November 1, 2026, affecting Prefab UI's Generative UI sandbox. The flaw allows attackers to execute arbitrary Python code by exploiting Deno's permissions, enabling file read/write access on the host server. This vulnerability affects…