Evaluation

Natural language capability benchmarks (CodeGemma)

performance on question answering (BoolQ, PIQA, TriviaQA), natural language (ARC-Challenge, HellaSwag, MMLU, WinoGrande) and mathematical reasoning (GSM8K, MATH) for the two 7B CodeGemma models vs the instruction-tuned Gemma 7B model

Core capabilities Google DeepMind — Gemma

Ask about this evaluation

How it was run

Open a release to see the setup, scoring, and other details that the lab published. Only fields the lab actually disclosed are shown.

CodeGemma Apr 2024

Google DeepMind — Gemma

partial detail 18 method fields published

See how this test was run