Mistral Large 4 utilizes a sparsely activated architecture containing 1.05 trillion total parameters and 52 billion active parameters, offering a 1 million token context capacity

Overview Debates Related claims Topics Sources

Moonshot AI released its Kimi K3 model in July 2026, a 2.8-trillion-parameter mixture-of-experts system built to compete against artificial intelligence offerings from OpenAI and Anthropic

Nous Research trained Arete-27B using supervised fine-tuning on approximately 53 million tokens over two epochs without reinforcement learning or preference optimization

The training dataset for Arete-27B contains 34,878 prompts across seven value categories, with 29,236 accepted teacher answers forming the fine-tuning split

Inkling 2 Small features 594 billion total parameters with a sparse architecture that activates 17 billion parameters per token

Pipecat is an open-source framework that orchestrates voice and video streams while linking speech-to-text, large language models, and text-to-speech components to minimize latency in AI applications.

Brave Software cited in a blog post that Anthropic used its Search API to train the Claude 2 artificial intelligence model.

Deepgram expanded its Nova-3 speech recognition model to support 55 languages as part of its product strategy.

Baidu pre-trained ERNIE 5.5 Flash Base on 33.5 trillion tokens using the Muon optimizer with a 128,000-token training context window

Baidu released ERNIE 5.5 Flash Base, a 280-billion-parameter text-only sparse Mixture-of-Experts foundation model that activates 14 billion parameters per token

ERNIE 5.5 Flash Base utilizes a Multi-Router Latent MoE architecture containing 512 routed experts running in a 2048-dimensional latent space alongside one shared expert

ERNIE 5.5 Flash Base implements ECHO speculative decoding, which reuses the final backbone layer as a draft model adding 0.36 billion parameters excluded from headline counts

ERNIE 5.5 Flash Base features a hybrid attention mechanism across 43 layers, using Heavily Compressed Attention in 37 layers and Multi-head Latent Attention with sparse attention in 6 layers

ERNIE 5.5 Flash Base incorporates a tokenizer with 201,216 entries that groups numbers right-to-left in chunks of up to three digits

Ricursive Intelligence pretrained Ripple-65B Preview from scratch on 12 trillion text and code tokens at an 8,000 sequence length

Ricursive Intelligence reported requiring roughly 70 times fewer reinforcement learning rollouts for Ripple-65B Preview than the 100 million rollouts reported for Reflection Beam

Ricursive Intelligence reached target reinforcement learning performance for Ripple-65B Preview using 1.4 million rollouts

Post-training for Ricursive Intelligence's Ripple-65B Preview incorporated synthetic data pipelines, long-context stages at 2 million sequence length, and reinforcement learning using truncated importance sampling

Ricursive Intelligence's Ripple-65B Preview features a dense Transformer architecture with 65 billion parameters across 80 layers and a model dimension of 5120

Pretraining of Ripple-65B Preview achieved roughly 40% model FLOPs utilization on NVIDIA hardware without custom kernels or quantization during training

The experimental NVFP4 checkpoint of Ricursive Intelligence's Ripple-65B Preview quantizes MLP layers while keeping attention in BF16 to target local hardware such as NVIDIA DGX Spark