Skip to content
Inside the Frontier

Evaluation

Verbatim Memorization

discoverable memorization (Nasr et al. 2023): 10,000 documents sampled per corpus, first 50 tokens used as prompt, text classified memorized if the subsequent 50 generated tokens exactly match the ground-truth continuation; methodology follows Anil et al. (2023)

Memorization & privacy Google DeepMind — Gemma
Ask about this evaluation

How it was run

Open a release to see the setup, scoring, and other details that the lab published. Only fields the lab actually disclosed are shown.

Gemma 1
Google DeepMind — Gemma

rich detail 29 method fields published

See how this test was run
Gemma 2
Google DeepMind — Gemma

rich detail 29 method fields published

See how this test was run