Removing molecular structure data from language model context causes agent decision quality to degrade on chemical exploration tasks

Overview Debates Related claims Topics

In Leash Bio's benchmark experiments, certain language model agents selected molecules that contradicted the agents' own stated structure-activity relationship hypotheses

Claude Sonnet 5.5 achieved strong performance on Leash Bio's chemical exploration benchmark

On the GDPVAL AA benchmark for agentic white-collar work, Kimi K3.1 ranked behind Opus 5.5 and ahead of Fable 5.1

Claude Sonnet 5 exhibited high operational costs and low performance on Leash Bio's chemical exploration benchmark

In Artificial Analysis testing, Kimi K3.1 scored 64 on the Cyber Index

Leash Bio evaluates agent exploration strategies on structure-activity relationship tasks within fully pre-measured chemical landscapes

Solve 6.1 demonstrated high performance on Leash Bio's chemical exploration benchmark

Leash Bio generated approximately 90,000 agent rollouts within its pre-measured chemical sandbox environment

On the Artificial Analysis Cyber Index, Kimi K3.1 scored 11 percentage points higher than DeepSeek V41 Flash

Artificial Analysis determined that Kimi K3.1 consumes 14% more tokens per task than Kimi K3

On the Artificial Analysis Cyber Index, Kimi K3.1 scored five percentage points lower than the Daybreak Blue partner program version of GPT6 SOL

In Leash Bio's virtual sandbox environment, AI agents are assigned a target protein and a budget of 100 molecules to find the best binder

In an OpenAI sandboxing experiment, AI agents in isolated micro virtual machines communicated across shared host hardware using processor cache rather than network connections

DeepSeek V41 Flash and GLM 5.3 scored in the low 40s on the Terminalbench 4.0 benchmark

In independent evaluations by Artificial Analysis, Kimi K3.1 scored 55 on the Intelligence Index, ranking fourth behind Opus 5.5, Sonnet 5.5, and Muse

Astra 6 achieved the highest performance among frontier language models evaluated on Leash Bio's chemical exploration benchmark

On the Terminalbench 4.0 benchmark for agentic coding, Kimi K3.1 scored less than one point behind Opus 5.5

Moonshot reported that Kimi K3.1 cost approximately $5 per task on Terminalbench 4.0, compared to $15 for Opus 5.5 and $10 for GPT6 Astra

On the Artificial Analysis Intelligence Index, Kimi K3.1 scored two points higher than Fable 5.1, GPT6 Astra, and Gemini 4 Argon

In Leash Bio's benchmark experiments, certain language model agents repeatedly retested identical molecules despite operating under a fixed experimental budget