{"id":"swe-bench","name":"SWE-bench","maker":"SWE-bench team (Princeton and Stanford researchers)","url":"https://www.swebench.com","repo":"SWE-bench/SWE-bench","package":"pypi:swebench","summary":"A benchmark of real GitHub issues from Python projects that a model must resolve by editing the repository.","details":"Each task gives an issue and a repository snapshot, and a patch counts as correct if the project's tests pass. The site hosts leaderboards for the full set and for the smaller Lite, Verified and Multimodal variants, and the repository provides an evaluation harness that runs in Docker.","open_source":true,"hosted":false,"also":["coding-agents"],"verified":"2026-11-15","category":"benchmarks","category_name":"Agent benchmarks","page":"https://modelmoot.com/tools/swe-bench","reviewed":"2026-11-15","checked":{"at":"2026-11-15T16:10:15.342Z","address_answers":true},"note":"Descriptions are the site's own. Facts under \"checked\" were read automatically from the tool's address and public repository."}