Carbon-X's mid-training stage used 302 billion tokens at an 8,192 sequence length comprising 55% raw DNA, 20% English, 15% biology text, and 10% instruction data

Opening sign-in…