Bomatter and Gouk benchmark: EEG discriminative OOD detection is near random, generative is effective
[Daily Paper] OOD Detection for EEG-based Machine Learning in High-Risk Environments
Bomatter and Gouk's benchmark study evaluated OOD detection methods for clinical EEG models and found that discriminative detectors are close to random under realistic covariate shift (AUROC≈0.50), while generative latent-space methods such as SITN lead by a wide margin. The study applied five types of controlled perturbations on TUAB (over 2,300 people) and CAUEEG (1,122 people): sampling-rate shift, channel permutation, re-referencing, and low-pass and high-pass filtering. On TUAB, SITN reached an AUROC of 0.998 under the Cz re-referencing shift, while MSP reached only 0.517. MSP also exhibited overconfidence anomalies: on mixed data under the TUAB 200 Hz resampling shift, the accuracy of its highest-confidence 0.1% interval fell to 47.0%. Based on this, the paper proposes a two-stage abstention pipeline: first use SITN to filter unfamiliar data, then use calibrated MSP to assess uncertainty, and refer cases that fail the threshold to a human neurologist; on a 50/50 mixed test pool under the 200 Hz shift, it raised accuracy from 67.5% to 84.1% at 43.7% coverage.