The optimal inference batch size for a sparse model like DeepSeek-V3 is more than 2,400 concurrent sequences.

Overview Debates Related claims Topics

AI

Artificial intelligence research, products, and impact across society.

50 debates·11,953 claims·218 news stories

AI inference scaling

188 claims·40 news stories

Mixture of experts (MoE)

70 claims·9 news stories