Specific reasoning neurons in language models fire to modulate confidence, induce delays, trigger self-checking, or pivot problem-solving directions

Overview Debates Related claims Topics

Fine-tuning a 90% sparsified language model on 100 million tokens recovers the majority of its reasoning performance degradation

The Neuron Scientist agent generates paragraph-length hypotheses describing the specific conditions under which an individual neuron activates

Ablating 80% to 90% of neurons in the Qwen 3.6-35B A3B language model improved performance on ARC-AGI benchmark puzzles

Nick Cammarata authored the essay Neural Networks are Already Clean

Noam Shazeer proposed the SwiGLU gated multi-layer perceptron architecture in a 2020 paper

Un-fine-tuned sparsified language models match baseline performance on reasoning tasks when evaluated using pass-at-4 or pass-at-8 sampling regimes

The Neuron Scientist agent utilizes experimental tools including generating counterfactual sentence pairs and steering activations to refine neuron hypotheses

Kurt Tigges and Nick Cammarata co-authored the research paper Neurons as an Interpretable Basis

Ablating approximately 90% of individual neurons by retaining only extreme activations causes minimal performance degradation on standard language model benchmarks

Individual neurons in language model multi-layer perceptrons can express a compact set of distinct meanings rather than hundreds of unrelated meanings

Selecting the top 64 activating neurons per expert in Qwen 3.6-35B A3B ablates approximately 88% of all neuron activations

Individual neurons in language models frequently function as verbs or control flow operators rather than static entity detectors

Swapping double-negative activations from an unrelated text sequence can restore reasoning capabilities in an ablated Qwen 3.6-35B A3B model

Neural networks employing ReLU or GELU activation functions are more difficult to interpret than networks using SwiGLU activation functions

ReGLU gated multi-layer perceptrons use the ReLU activation function instead of SiLU, preventing the occurrence of double-negative activation values

Double-negative activations occur in SwiGLU when the gate projection and up projection both produce negative values that multiply into a positive product

Patching predicted neuron activations based on Neuron Scientist descriptions into an ablated model recovers 74% of the damage measured by Kullback-Leibler divergence

Ablating individual neurons within attribution circuits can identify whether specific input tokens cause a persona steering vector to fail

In SwiGLU multi-layer perceptrons, distinct sets of downstream reader neurons activate depending on context to parse polysemantic neuron outputs

Transcoders applied to multi-layer perceptron layers predominantly recapitulate individual neurons or sparse combinations of co-activating neurons