In SwiGLU multi-layer perceptrons, each neuron receives input via two separate parameter matrices consisting of an up projection and a gate projection

Activity

Activity (0)
Loading comments...
Opening sign-in…