Google DeepMind says two AI agents reproduced a known 1024-bit RSA cryptography computation in three days and 22 hours

Overview Debates Related claims Topics Sources

441e85161ClaimNews event

Epoch AI unveiled Inference Performance Index, a new benchmark that measures AI chips by how many users they can serve

441e85161ClaimNews event

Google DeepMind and Artificial Analysis unveiled the MKA benchmark, which grades autonomous agents on complex, visually heavy professional work

441e85161ClaimNews event

Google DeepMind revealed Gemini Robotics 2.5, a vision-language-action model that generates robot actions, and ASIMOV-Jailbreak, an open-source benchmark of prompts designed to trick robotic AI into unsafe actions

441e85161ClaimNews event

Epoch AI introduced the Automation Reports, a benchmark testing whether frontier AI models can automate work, using tasks drawn from the organization's own internal research

441e85161News eventClaim

Epoch AI rolled out InnovationEval, which tests how well AI can automate research itself

441e85161ClaimNews event

Nous Research launched the Hermes Index, a leaderboard that ranks frontier models on their performance as agents inside the company's Hermes Agent framework and tracks the cost of each task

Vendor-reported benchmarks are insufficient to justify enterprise adoption of new AI models

Vendor-run launch benchmarks overstate real-world speed and cost gains

AI benchmark results must be verified by independent third parties

Benchmark scores are a poor proxy for real scientific capability

AI benchmarks no longer reflect real-world capability

Frontier AI models should undergo mandatory third-party safety evaluations before release

AI companies optimize for benchmark scores rather than usefulness

AI agent skills look great in benchmarks but fall apart under realistic conditions according to researchers