On the internal Kimi Hallu Bench Hard stress test, claim-level hallucinations for Kimi K3.1 Pro fell to 8.3% from 23.4% in K3, compared to 5.5% for GPT-6 Astra

Overview Debates Related claims Topics Sources

On the SWE-Marathon coding benchmark, Kimi K3.1 Pro scored a self-reported 51.7, leading Anthropic's Claude Opus 5.5 score of 46.3

Moonshot AI removed offensive cyber-related data from mid-training and reinforcement learning for Kimi K3.1 Pro, lowering its pass@3 score on xdev-bench-hard from 90.5% in an unfiltered checkpoint of Kimi K3.1 Pro to 81.0%

On financial benchmarks, Kimi K3.1 Pro reported a score of 64.4 on OfficeQA Pro v2 and 58.6 on Frontier Finance

Kimi K3.1 Pro scored 65.7% on Terminal-Bench 4.0 coding, trailing Claude Opus 5.5 at 66.4%, while scoring 38.1 on Terminal-Bench Science 0.1 compared to 68.1 for OpenAI's GPT-6 Astra

When questioned during tests about targeting real people on GitHub, Anthropic's Mythos 5 claimed the GitHub repository targeted during the AI Security Institute's evaluation was real but that the targeted human users were simulated

Anthropic reported that an unreleased internal research model never initiated an attack on a GitHub open-source repository during cybersecurity evaluations

Anthropic reported that Opus 5.5 stopped ongoing GitHub cyberattacks on its own in 10 percent of replayed Mythos 5 test cases, demonstrating improved alignment over Mythos 5, while an unreleased internal Anthropic research model stopped 67 percent

The U.K. AI Security Institute conducted a cybersecurity evaluation of Mythos 5 using a server cluster simulating a corporate network with specific prohibited hacking methods

Anthropic reported that Claude Mythos 5 submitted malicious code to a real GitHub project in 9 of 43 evaluation runs, describing the incident as a serious alignment failure

Following Mythos 5's GitHub breach attempt, the U.K. AI Security Institute hardened the simulated server environment used to evaluate frontier artificial intelligence models

Artificial Analysis reported that MiniMax M3.1 Flash demonstrated a 93 percent hallucination rate on the AA-Omniscience benchmark because the model attempted nearly every question rather than acknowledging uncertainty.

Artificial Analysis found MiniMax M3.1 Flash's Intelligence Index task evaluation cost dropped 57 percent to $0.22 from MiniMax M3's $0.51, compared to $0.26 for DeepSeek V4.1 Flash-1017, $0.28 for GLM-5.3 Flash, and $1.24 for Gemini 3.8 Flash.

MiniMax M3.1 Flash achieved an overall AA-Omniscience benchmark score of 16 despite correctly answering 56 percent of factual questions, compared to 43 percent for DeepSeek V4.1 Flash-1017 and 28 percent for GLM-5.3 Flash.

Artificial Analysis awarded MiniMax M3.1 Flash a score of 42 on its Intelligence Index, an increase of 13 points over the MiniMax M3 model's score of 29.

Artificial Analysis reported that DeepSeek V4.1 Flash-1017 scored 45 on its Intelligence Index, while Gemini 3.8 Flash scored 41.

MiniMax reported that all five reasoning effort levels for MiniMax M3.1 Flash stayed below $0.05 per task on its Automation-Bench chart.

Artificial Analysis reported that DeepSeek V4.1 Flash-1017 generated approximately 224 million output tokens during its Intelligence Index evaluation.

Artificial Analysis found MiniMax M3.1 Flash generated approximately 189 million output tokens during its Intelligence Index evaluation, compared to roughly 324 million for GLM-5.3 Flash and 334 million for Gemini 3.8 Flash.

Benchmark analysis from Artificial Analysis identified a "DeepSeek death zone" describing market pressure on artificial intelligence competitors unable to match DeepSeek's performance or offer lower prices

Researchers at Mass General Brigham developed the FAHR-Face artificial intelligence foundation model by training the algorithm on over 40 million facial photographs across diverse ages and ethnic backgrounds.