Evaluation

SWE-Lancer

Performance on real-world, economically valuable full-stack software engineering tasks

Coding ability OpenAI

How it was run

Open a release to see the setup, scoring, and other details that the lab published. Only fields the lab actually disclosed are shown.

Deep Research Feb 2025

OpenAI

rich detail 14 method fields published

See how this test was run

GPT-4.5 Feb 2025

OpenAI

rich detail 14 method fields published

See how this test was run

o3/o4-mini Apr 2025

OpenAI

rich detail 14 method fields published

See how this test was run