Carbon-X's mid-training stage used 302 billion tokens at an 8,192 sequence length comprising 55% raw DNA, 20% English, 15% biology text, and 10% instruction data
Agree
Disagree
Overview
Debates
Related claims
Topics
Sources
Best
Space 41e851
Loading debates…
Opening sign-in…