Fine-tuning a 90% sparsified language model on 100 million tokens recovers the majority of its reasoning performance degradation

Opening sign-in…