Evaluation

Codex HumanEval

python function synthesis

Coding ability Anthropic

How it was run

Open a release to see the setup, scoring, and other details that the lab published. Only fields the lab actually disclosed are shown.

Claude 2 Jul 2023

Anthropic

rich detail 19 method fields published

See how this test was run

Claude Instant 1.2 Aug 2023

Anthropic

rich detail 17 method fields published

See how this test was run