ReGLU gated multi-layer perceptrons use the ReLU activation function instead of SiLU, preventing the occurrence of double-negative activation values