EvEMTBench — results audit

EvEMTBench is a protocol-driven ML benchmark for power-system protection on EMT waveform windows. Labels are exact by construction — every window's label derives from the simulation script that injected the event — so the honest risks are leakage, held-out contamination, stale published tables and trivial-task artefacts. Each has a mechanical guard:

Frozen, audited splitsEpisode-disjoint, family-pure train/val/test/benchmark splits are committed to the repo and re-audited by named tests on every CI run — no training window shares an episode with an evaluation window (held_out v1.0; multi_grid pretrain v1.1, topology-grouped).
code tour → · src/evemtbench/splits

Analytic floor self-checkThe majority baseline must land on the closed-form 1/k chance floor and no learned baseline may fall below it — a known-answer check over the whole results matrix, re-runnable in seconds.
code tour → · src/evemtbench/coverage.py

Never-trained benchmark family + attestationThe benchmark family is scored but never fitted or model-selected on; every run commits a machine-readable no-tuning attestation alongside git SHA, lockfile hash and seeds in its run record.
docs/determinism.md

Byte-fresh rendered viewsRESULTS.md, leaderboard.html and this audit site are pure functions of outputs/ — a pre-commit gate fails any commit where the committed pages differ from a fresh rebuild, so a stale number cannot be published.
code tour → · hpc/check_results_fresh.py

How to read a claim

Field	What it gives the reviewer
Value	the measured number, read from the committed result file at build time
Script	link to the exact source that produces it
Command	the verbatim invocation to reproduce it
Artifact	the output (result.json / rendered file) it writes
Runtime & Hardware	wall-clock and the machine it ran on
Verified by	the test / gate that guards it
Status	measured vs in progress vs deferred — a plan is never dressed up as a result