Evaluation
Novel compiler task
ability to create a compiler for a novel and unusual programming language given only a specification and test cases
Autonomy & self-improvement
Anthropic
How it was run
Open a release to see the setup, scoring, and other details that the lab published. Only fields the lab actually disclosed are shown.
Claude Opus 4, Claude Sonnet 4
Anthropic
rich detail
23 method fields published
See how this test was run