{"id":"openai-evals","name":"OpenAI Evals","maker":"OpenAI","url":"https://github.com/openai/evals","repo":"openai/evals","summary":"A framework and registry from OpenAI for evaluating language models and systems built on them.","details":"A Python repository with a command-line runner, a registry of evaluation definitions in YAML and code, and templates for model-graded checks. It targets OpenAI models first but can wrap other completion functions. The repository is public under the MIT licence.","open_source":true,"hosted":false,"speaks":["openai-api"],"verified":"2026-11-15","category":"observability","category_name":"Observability and evaluation","page":"https://modelmoot.com/tools/openai-evals","reviewed":"2026-11-15","checked":{"at":"2026-11-15T15:07:43.409Z","address_answers":true},"note":"Descriptions are the site's own. Facts under \"checked\" were read automatically from the tool's address and public repository."}