Task — fault_classification_line

benchmark version 1.1.0 · held_out results v1.0.0 · splits v1.0/held_out, v1.1/multi_grid
updated 2026-09-29 13:18 UTC · git 7243f310ddbc

tier 2 · fault_class · line view · 9 classes · all cells measured

What this task is. Given a faulted window, name the fault type from the nine-class vocabulary (which phases are involved and whether ground participates). Fault typing drives which poles a breaker trips and how the event is analysed afterwards. The class vocabulary is frozen for the benchmark's major version. Line view: the cubicles at both ends of one line (12 channels), the information basis of a line-differential scheme.

How it's scored

Headline metric balanced_accuracy (↑ higher is better); full metric set: balanced_accuracy · macro_f1 · accuracy. Metrics are a frozen contract (src/evemtbench/evaluation/metrics.py); label derivation: src/evemtbench/tasks/labels.

Example window

example waveform window

event flt_1phg_incipient_w_arc · grid double_line (adapt_grid/test split) · sample 197, window #4531, cubicle 3 (max-RMS) · 50 ms / 480 samples @ 9.6 kHz · top: 3-phase current, bottom: 3-phase voltage · figure provenance in assets/manifest.json

Results (held_out)

fault_classification_line — every grid × baseline; click headers to sort
taskgridbaselineheadlinetestbenchmarkparamsfit/seedtrace
fault_classification_linecigre_mvmajoritybalanced_accuracy ↑0.111 ± 0.000 (n=5)0.111 ± 0.000 (n=5)—0strace: output · eval-code · log · train-code · config · W&B
fault_classification_linecigre_mvrandom_forestbalanced_accuracy ↑0.500 ± 0.005 (n=5)0.531 ± 0.003 (n=5)—12strace: output · eval-code · log · train-code · config · W&B
fault_classification_linecigre_mvmlpbalanced_accuracy ↑0.523 ± 0.002 (n=5)0.549 ± 0.002 (n=5)3.1M1.0mtrace: output · eval-code · log · train-code · config · W&B
fault_classification_linecigre_mvgrubalanced_accuracy ↑0.533 ± 0.009 (n=5)0.552 ± 0.005 (n=5)56k2.2mtrace: output · eval-code · log · train-code · config · W&B
fault_classification_linecigre_mvcnnbalanced_accuracy ↑0.576 ± 0.022 (n=5)0.589 ± 0.028 (n=5)52k2.0mtrace: output · eval-code · log · train-code · config · W&B
fault_classification_linecigre_mvresnetbalanced_accuracy ↑0.665 ± 0.010 (n=5)0.700 ± 0.010 (n=5)510k7.0mtrace: output · eval-code · log · train-code · config · W&B
fault_classification_linedouble_linemajoritybalanced_accuracy ↑0.143 ± 0.000 (n=5)0.143 ± 0.000 (n=5)—0strace: output · eval-code · log · train-code · config · W&B
fault_classification_linedouble_linerandom_forestbalanced_accuracy ↑0.661 ± 0.004 (n=5)0.555 ± 0.003 (n=5)—13strace: output · eval-code · log · train-code · config · W&B
fault_classification_linedouble_linemlpbalanced_accuracy ↑0.704 ± 0.005 (n=5)0.716 ± 0.001 (n=5)3.1M1.1mtrace: output · eval-code · log · train-code · config · W&B
fault_classification_linedouble_linegrubalanced_accuracy ↑0.697 ± 0.006 (n=5)0.719 ± 0.005 (n=5)56k2.0mtrace: output · eval-code · log · train-code · config · W&B
fault_classification_linedouble_linecnnbalanced_accuracy ↑0.792 ± 0.019 (n=5)0.811 ± 0.016 (n=5)52k2.1mtrace: output · eval-code · log · train-code · config · W&B
fault_classification_linedouble_lineresnetbalanced_accuracy ↑0.863 ± 0.007 (n=5)0.873 ± 0.023 (n=5)510k6.3mtrace: output · eval-code · log · train-code · config · W&B
fault_classification_linetestgrid_110kvmajoritybalanced_accuracy ↑0.143 ± 0.000 (n=5)0.143 ± 0.000 (n=5)—0strace: output · eval-code · log · train-code · config · W&B
fault_classification_linetestgrid_110kvrandom_forestbalanced_accuracy ↑0.631 ± 0.003 (n=5)0.631 ± 0.009 (n=5)—11strace: output · eval-code · log · train-code · config · W&B
fault_classification_linetestgrid_110kvmlpbalanced_accuracy ↑0.679 ± 0.005 (n=5)0.715 ± 0.001 (n=5)3.1M1.0mtrace: output · eval-code · log · train-code · config · W&B
fault_classification_linetestgrid_110kvgrubalanced_accuracy ↑0.681 ± 0.005 (n=5)0.718 ± 0.003 (n=5)56k1.8mtrace: output · eval-code · log · train-code · config · W&B
fault_classification_linetestgrid_110kvcnnbalanced_accuracy ↑0.706 ± 0.012 (n=5)0.743 ± 0.021 (n=5)52k1.7mtrace: output · eval-code · log · train-code · config · W&B
fault_classification_linetestgrid_110kvresnetbalanced_accuracy ↑0.805 ± 0.036 (n=5)0.834 ± 0.015 (n=5)510k6.2mtrace: output · eval-code · log · train-code · config · W&B
fault_classification_lineieee39majoritybalanced_accuracy ↑0.143 ± 0.000 (n=5)0.143 ± 0.000 (n=5)—0strace: output · eval-code · log · train-code · config · W&B
fault_classification_lineieee39random_forestbalanced_accuracy ↑0.661 ± 0.002 (n=5)0.679 ± 0.001 (n=5)—13strace: output · eval-code · log · train-code · config · W&B
fault_classification_lineieee39mlpbalanced_accuracy ↑0.674 ± 0.005 (n=5)0.715 ± 0.000 (n=5)3.1M2.1mtrace: output · eval-code · log · train-code · config · W&B
fault_classification_lineieee39grubalanced_accuracy ↑0.671 ± 0.007 (n=5)0.715 ± 0.002 (n=5)56k2.1mtrace: output · eval-code · log · train-code · config · W&B
fault_classification_lineieee39cnnbalanced_accuracy ↑0.666 ± 0.013 (n=5)0.713 ± 0.002 (n=5)52k1.2mtrace: output · eval-code · log · train-code · config · W&B
fault_classification_lineieee39resnetbalanced_accuracy ↑0.682 ± 0.006 (n=5)0.716 ± 0.003 (n=5)510k3.2mtrace: output · eval-code · log · train-code · config · W&B

Metric definitions

headline & reported metrics for this view — the metric set is a frozen contract; each links to the exact function that computes it

metricdefinitionentry
balanced_accuracymean of per-class recall — the majority class cannot buy a good scoreevaluate()
macro_f1unweighted mean of per-class F1 (zero_division=0)evaluate()
accuracyfraction of exact-match predictionsevaluate()

Reproduce one cell

PYTHONPATH=src python -m evemtbench.baselines.runner --task fault_classification_line --protocol held_out \
    --grid <grid> --baseline <baseline> --seeds 0 1 2 3 4