Living definition
What is OpenAI SWE-bench Verified?
AI Coding & Software AgentsAI & Agents
- As of
- 2026-10-24
- Revision
- 2026-10-06.1
- Method
- v1.4.0
Definition
The term, in context
AI-assisted draft · approved dataset
SWE-bench is a benchmark that evaluates how well AI coding agents can resolve real-world GitHub issues, and the human-audited subset of the dataset released by OpenAI filters out mislabeled or unsolvable problems, scoring agents by generating code patches that are checked against each repository's actual test suite.
Live readiness status
Status as of the current dateline
AI-assisted assembly · derived results
As of 2026-10-24, verified readiness is 25 (claimed 55, reported 35, gap 30) — Strong evidence strength; current signals suggest Use SWE-bench Verified as a supplementary historical benchmark, with independent patch review.
The readiness fact belongs to the canonical Readiness Verdict for OpenAI SWE-bench Verified.
Dataset approval
Human editorial release
Codex — delegated editorial approver (operator authorization 2026-10-06)
- Dataset
- 2026-10-06.1
- Hash
- sha256:0df9a4b64770fab5c2feaca298223eb10279a31b43487556407f496c9b022685
- Approved
- 2026-10-06