Ablating approximately 90% of individual neurons by retaining only extreme activations causes minimal performance degradation on standard language model benchmarks

Overview Debates Related claims Topics

The Neuron Scientist agent generates paragraph-length hypotheses describing the specific conditions under which an individual neuron activates

Ablating 80% to 90% of neurons in the Qwen 3.6-35B A3B language model improved performance on ARC-AGI benchmark puzzles

Nick Cammarata authored the essay Neural Networks are Already Clean

Noam Shazeer proposed the SwiGLU gated multi-layer perceptron architecture in a 2020 paper

Un-fine-tuned sparsified language models match baseline performance on reasoning tasks when evaluated using pass-at-4 or pass-at-8 sampling regimes

The Neuron Scientist agent utilizes experimental tools including generating counterfactual sentence pairs and steering activations to refine neuron hypotheses

Specific reasoning neurons in language models fire to modulate confidence, induce delays, trigger self-checking, or pivot problem-solving directions

Kurt Tigges and Nick Cammarata co-authored the research paper Neurons as an Interpretable Basis

Performance on factual question-answering benchmarks such as MedQA and GPQA Diamond can be restored to within one point of baseline after model sparsification and fine-tuning

Hypotheses generated by the Neuron Scientist agent are evaluated by prompting an independent language model to predict token-by-token activation levels on unseen text

Individual neurons in language model multi-layer perceptrons can express a compact set of distinct meanings rather than hundreds of unrelated meanings

Sparsified Qwen 3.6-35B A3B models maintain performance parity with the baseline model on coding tasks across context lengths up to 32,000 tokens

Selecting the top 64 activating neurons per expert in Qwen 3.6-35B A3B ablates approximately 88% of all neuron activations

Individual neurons in language models frequently function as verbs or control flow operators rather than static entity detectors

Swapping double-negative activations from an unrelated text sequence can restore reasoning capabilities in an ablated Qwen 3.6-35B A3B model

Neural networks employing ReLU or GELU activation functions are more difficult to interpret than networks using SwiGLU activation functions

ReGLU gated multi-layer perceptrons use the ReLU activation function instead of SiLU, preventing the occurrence of double-negative activation values

Double-negative activations occur in SwiGLU when the gate projection and up projection both produce negative values that multiply into a positive product

Patching predicted neuron activations based on Neuron Scientist descriptions into an ablated model recovers 74% of the damage measured by Kullback-Leibler divergence

Ablating individual neurons within attribution circuits can identify whether specific input tokens cause a persona steering vector to fail