Evaluation

METR data deduplication

2-8 hour software engineering capability via a METR public task

Autonomy & self-improvement Anthropic

How it was run

Open a release to see the setup, scoring, and other details that the lab published. Only fields the lab actually disclosed are shown.

Claude 3.7 Sonnet Feb 2025

Anthropic

rich detail 23 method fields published

See how this test was run

Claude Opus 4, Claude Sonnet 4 May 2025

Anthropic

rich detail 21 method fields published

See how this test was run