{"id":"helm","name":"HELM","maker":"Stanford Center for Research on Foundation Models","url":"https://crfm.stanford.edu/helm","repo":"stanford-crfm/helm","package":"pypi:crfm-helm","summary":"A framework and set of leaderboards for holistic evaluation of language models across many scenarios and metrics.","details":"Runs models over standard scenarios and reports several metrics such as accuracy, resilience to perturbation, fairness and efficiency, and publishes leaderboards for general and specialised domains. The code is open source under Apache 2.0.","open_source":true,"hosted":false,"verified":"2026-11-15","category":"benchmarks","category_name":"Agent benchmarks","page":"https://modelmoot.com/tools/helm","reviewed":"2026-11-15","checked":{"at":"2026-11-15T14:03:44.997Z","address_answers":true,"repository":{"at":"2026-11-15T23:25:51.807Z","archived":false,"licence":"Apache-2.0","language":"Python","last_change":"2026-09-01T01:33:19.000Z","latest_release":{"tag":"v0.5.16","published":"2026-04-30T05:05:48.000Z"}}},"note":"Descriptions are the site's own. Facts under \"checked\" were read automatically from the tool's address and public repository."}