Carbon-X's mid-training stage used 302 billion tokens at an 8,192 sequence length comprising 55% raw DNA, 20% English, 15% biology text, and 10% instruction data

Activity

Activity (0)
Loading comments...
Opening sign-in…