source: hugging face blog: featuring every eval ever results on hugging face model pages

level: technical

every eval ever launched in february 2026 as a project of the evaleval coalition to improve how ai evaluation results get reported. hugging face community evals also launched in february 2026 to decentralize benchmark score reporting on the hub. together they address gaps in how users, researchers, and policymakers trust and compare evaluations. evaluation results are often scattered across papers, leaderboards, and blog posts in different formats, making comparisons difficult. the same model on the same benchmark can return different scores depending on who ran it and how, due to unreported evaluation settings.

eee provides a single json schema for evaluation results that records who ran the evaluation, which model, how it was accessed, generation settings, what the metric means, and optionally per-sample outputs. the schema was built with feedback from researchers and accepts results from any source. the datastore on hugging face has grown to around 229,000 results across more than 22,000 models and 2,200 benchmarks from 31 reporting formats. a new converter lets contributors send eee records to hugging face community evals by generating the required yaml files, so results appear on model pages and leaderboards with a source badge linking back to the full eee record.

the converter maps eee fields to community evals fields, handles four official benchmarks (mmlu-pro, gpqa, hle, gsm8k), and audits existing scores before pushing. it checks for duplicates and conflicts, writes local previews, and only opens pull requests after user confirmation. this integration makes evaluation results both visible on model pages and interpretable through full structured records, improving transparency and trust in model comparisons.

why it matters: it standardizes and links evaluation results, making model comparisons more reliable and transparent for ai practitioners.

source: hugging face blog: featuring every eval ever results on hugging face model pages