Serving extremely sparse Mixture of Experts (MOE) models is computationally prohibitive on today's HBM-based GPUs

Opening sign-in…