Provenance of the process

benchmark version 1.1.0 · held_out results v1.0.0 · splits v1.0/held_out, v1.1/multi_grid
updated 2026-09-29 13:18 UTC · git 7243f310ddbc

Test suite

364 test functions across 64 files (counted from source at build time) — unit, integration (incl. leakage, splits audit, freshness, contract conformance, per-baseline runners), e2e smoke, and data-integrity checks that run only where the real data lives.

Continuous integration

.github/workflows/ci.ymlruff + mypy + pytest (torch-free) on 3.10/3.12; the leakage audit re-runs as a named step so a leaky split fails legibly
.github/workflows/nightly.ymlthe same gate on fresh runners nightly — catches dependency/runner drift

Hooks

.githooks/pre-commitruns the freshness gate when rendered views are staged — a stale RESULTS.md / leaderboard.html / audit page cannot be committed
.pre-commit-config.yamlruff lint+format, mypy, hygiene hooks, and the same results-fresh gate for pre-commit users

Change log

CHANGELOG.mdversioned change log

Progress itself is an append-only ledger: the sweep loop commits a snapshot (chore(results): sweep progress snapshot — … done … pending …) with the coverage summary embedded in every message, so the history of the matrix is in git log.