tier 1 · switch_detection · global view · 2 classes · all cells measured
Headline metric balanced_accuracy (↑ higher is better); full metric set: balanced_accuracy · macro_f1 · missed_event_rate · false_event_trip_rate · accuracy. Metrics are a frozen contract (src/evemtbench/evaluation/metrics.py); label derivation: src/evemtbench/tasks/labels.
event switch_cap_on · grid double_line (adapt_grid/test split) · sample 181, window #4163, cubicle 3 (max-RMS) · 50 ms / 480 samples @ 9.6 kHz · top: 3-phase current, bottom: 3-phase voltage · figure provenance in assets/manifest.json
| task | grid | baseline | headline | test | benchmark | params | fit/seed | trace |
|---|---|---|---|---|---|---|---|---|
| switch_detection_global | cigre_mv | majority | balanced_accuracy ↑ | 0.500 ± 0.000 (n=5) | 0.500 ± 0.000 (n=5) | — | 0s | trace: output · eval-code · log · train-code · config · W&B |
| switch_detection_global | cigre_mv | random_forest | balanced_accuracy ↑ | 0.754 ± 0.002 (n=5) | 0.816 ± 0.005 (n=5) | — | 6.7m | trace: output · eval-code · log · train-code · config · W&B |
| switch_detection_global | cigre_mv | mlp | balanced_accuracy ↑ | 0.688 ± 0.002 (n=5) | 0.710 ± 0.001 (n=5) | 42.9M | 36.6m | trace: output · eval-code · log · train-code · config · W&B |
| switch_detection_global | cigre_mv | gru | balanced_accuracy ↑ | 0.718 ± 0.127 (n=5) | 0.758 ± 0.102 (n=5) | 117k | 28.3m | trace: output · eval-code · log · train-code · config · W&B |
| switch_detection_global | cigre_mv | cnn | balanced_accuracy ↑ | 0.781 ± 0.012 (n=5) | 0.822 ± 0.014 (n=5) | 124k | 37.9m | trace: output · eval-code · log · train-code · config · W&B |
| switch_detection_global | cigre_mv | resnet | balanced_accuracy ↑ | 0.865 ± 0.011 (n=5) | 0.914 ± 0.014 (n=5) | 602k | 37.2m | trace: output · eval-code · log · train-code · config · W&B |
| switch_detection_global | double_line | majority | balanced_accuracy ↑ | 0.500 ± 0.000 (n=5) | 0.500 ± 0.000 (n=5) | — | 0s | trace: output · eval-code · log · train-code · config · W&B |
| switch_detection_global | double_line | random_forest | balanced_accuracy ↑ | 0.721 ± 0.001 (n=5) | 0.630 ± 0.016 (n=5) | — | 1.5m | trace: output · eval-code · log · train-code · config · W&B |
| switch_detection_global | double_line | mlp | balanced_accuracy ↑ | 0.701 ± 0.000 (n=5) | 0.730 ± 0.002 (n=5) | 12.0M | 10.3m | trace: output · eval-code · log · train-code · config · W&B |
| switch_detection_global | double_line | gru | balanced_accuracy ↑ | 0.769 ± 0.029 (n=5) | 0.826 ± 0.032 (n=5) | 69k | 9.3m | trace: output · eval-code · log · train-code · config · W&B |
| switch_detection_global | double_line | cnn | balanced_accuracy ↑ | 0.729 ± 0.025 (n=5) | 0.816 ± 0.017 (n=5) | 68k | 10.6m | trace: output · eval-code · log · train-code · config · W&B |
| switch_detection_global | double_line | resnet | balanced_accuracy ↑ | 0.780 ± 0.004 (n=5) | 0.808 ± 0.017 (n=5) | 530k | 26.4m | trace: output · eval-code · log · train-code · config · W&B |
| switch_detection_global | testgrid_110kv | majority | balanced_accuracy ↑ | 0.500 ± 0.000 (n=5) | 0.500 ± 0.000 (n=5) | — | 0s | trace: output · eval-code · log · train-code · config · W&B |
| switch_detection_global | testgrid_110kv | random_forest | balanced_accuracy ↑ | 0.682 ± 0.002 (n=5) | 0.592 ± 0.006 (n=5) | — | 2.5m | trace: output · eval-code · log · train-code · config · W&B |
| switch_detection_global | testgrid_110kv | mlp | balanced_accuracy ↑ | 0.711 ± 0.001 (n=5) | 0.734 ± 0.003 (n=5) | 26.7M | 19.6m | trace: output · eval-code · log · train-code · config · W&B |
| switch_detection_global | testgrid_110kv | gru | balanced_accuracy ↑ | 0.725 ± 0.057 (n=5) | 0.805 ± 0.042 (n=5) | 92k | 17.5m | trace: output · eval-code · log · train-code · config · W&B |
| switch_detection_global | testgrid_110kv | cnn | balanced_accuracy ↑ | 0.694 ± 0.021 (n=5) | 0.823 ± 0.002 (n=5) | 94k | 21.8m | trace: output · eval-code · log · train-code · config · W&B |
| switch_detection_global | testgrid_110kv | resnet | balanced_accuracy ↑ | 0.786 ± 0.006 (n=5) | 0.785 ± 0.031 (n=5) | 564k | 31.5m | trace: output · eval-code · log · train-code · config · W&B |
| switch_detection_global | ieee39 | majority | balanced_accuracy ↑ | 0.500 ± 0.000 (n=5) | 0.500 ± 0.000 (n=5) | — | 0s | trace: output · eval-code · log · train-code · config · W&B |
| switch_detection_global | ieee39 | random_forest | balanced_accuracy ↑ | 0.922 ± 0.000 (n=5) | 0.778 ± 0.055 (n=5) | — | 11.5m | trace: output · eval-code · log · train-code · config · W&B |
| switch_detection_global | ieee39 | mlp | balanced_accuracy ↑ | 0.754 ± 0.015 (n=5) | 0.760 ± 0.016 (n=5) | 103.4M | 1.6h | trace: output · eval-code · log · train-code · config · W&B |
| switch_detection_global | ieee39 | gru | balanced_accuracy ↑ | 0.720 ± 0.041 (n=5) | 0.789 ± 0.021 (n=5) | 211k | 1.7h | trace: output · eval-code · log · train-code · config · W&B |
| switch_detection_global | ieee39 | cnn | balanced_accuracy ↑ | 0.783 ± 0.011 (n=5) | 0.769 ± 0.044 (n=5) | 234k | 1.7h | trace: output · eval-code · log · train-code · config · W&B |
| switch_detection_global | ieee39 | resnet | balanced_accuracy ↑ | 0.888 ± 0.013 (n=5) | 0.654 ± 0.092 (n=5) | 744k | 1.6h | trace: output · eval-code · log · train-code · config · W&B |
headline & reported metrics for this view — the metric set is a frozen contract; each links to the exact function that computes it
| metric | definition | entry |
|---|---|---|
balanced_accuracy | mean of per-class recall — the majority class cannot buy a good score | evaluate() |
macro_f1 | unweighted mean of per-class F1 (zero_division=0) | evaluate() |
missed_event_rate | FN-rate among positives (missed events) | evaluate() |
false_event_trip_rate | FP-rate among negatives (tripping on a benign event) | evaluate() |
accuracy | fraction of exact-match predictions | evaluate() |
PYTHONPATH=src python -m evemtbench.baselines.runner --task switch_detection_global --protocol held_out \
--grid <grid> --baseline <baseline> --seeds 0 1 2 3 4