Arca guides artificial intelligence model selection using proprietary internal benchmarks rather than relying on external public benchmarks

Overview Debates Related claims Topics

Luna was rejected by Arca after internal benchmarking showed it was too slow and expensive for Arca's operational requirements

GPT-6 showed little performance improvement on existing financial benchmarks due to saturated evaluation tasks, ambiguous prompts, and erroneous answer keys

OpenAI discontinued tracking the GDP eval benchmark because the benchmark became saturated and flawed

Artificial intelligence models struggle more with subjective human taste and formatting nuance than with objective factual accuracy

Artificial intelligence models exhibit lower rates of factual hallucination than observed during 2022 and 2023

Artificial Analysis removed long-context reasoning benchmark AA-LCR from the Intelligence Index Version 5

Google's Gemini 4 Carbon gained 8 points on Terminal Bench 4.0 and 6 points on Frontier Suite compared to Gemini 4 Argon

Claude Haiku 5.5 scored 36 points on the Artificial Analysis Intelligence Index Version 5

Google's Gemini 4 Carbon scored 59 points on the Artificial Analysis Intelligence Index Version 5, ranking in first place ahead of Opus 5.5

Google's Gemini 4 Carbon recorded a 12-point increase on Exploit Gym and a 3-point increase on the CWE Benchmark compared to Gemini 4 Argon

Artificial Analysis added Frontier Code 2.0, a private benchmark evaluating real-world coding pull requests, to the Intelligence Index Version 5

Google's Gemini 4 Carbon reached the top of the leaderboard on six of eight key benchmarks, scoring ahead of GPT-6 Astra and Opus 5.5

Kimi 3.1 Pro scored 34% on Terminal Bench Science, while GPT-6 Astra scored 63%

Artificial Analysis upgraded its Humanity's Last Exam benchmark to the HLE Diamond evaluation in the Intelligence Index Version 5

Anthropic's Sonnet 5.5 dropped from second to fourth place on the Artificial Analysis Intelligence Index Version 5

A majority of the benchmark evaluations used in Artificial Analysis Intelligence Index Version 5 are constructed using private datasets

Google's Gemini 4 Carbon tied for second place with Sonnet 5.5 on the Artificial Analysis Coding Index, scoring 3 points behind Opus 5.5

DeepSeek-V4.1 Flash and GLM-5.3 scored 32 points on Artificial Analysis Intelligence Index Version 5

GPT-6.1 Tera placed fifth on the Artificial Analysis Intelligence Index Version 5

Google's Gemini 4 Carbon improved by 7 points on OSWorld and 8 points on Humanity's Last Exam compared to Gemini 4 Argon