Gemma 3
Google DeepMind — Gemma
partial detail 18 method fields published
Evaluation
Enabling long context ablation
scaling 4B/12B/27B models from 32K to 128K sequences at the end of pre-training via RoPE rescaling (scaling factor 8; RoPE base frequency of global layers increased from 10k to 1M, local kept at 10k); perplexity across context lengths
How it was run
Open a release to see the setup, scoring, and other details that the lab published. Only fields the lab actually disclosed are shown.