SCARCE: Embedding-Based Scalable Cascade Analysis for Rare Event Characterization

The researchers propose SCARCE, which replaces handcrafted performance functions with learned latent representations and a geometric ruler to estimate rare-event probabilities. The method constructs nested intermediate events directly from data via adaptive thresholds, and uses a nonnegative supermartingale to give a high-probability upper bound that still holds under early stopping. On the MNIST misclassification task, SCARCE's mean absolute error is about 400–500 times lower than grid-search traditional subset simulation, and it eliminates systematic overcounting. The study also examines PAIR-style LLM jailbreaks under a fleet-level threat model: on Llama-Guard-3-8B hidden states, the PCA-based ruler has a mean relative error of 2.6% at η≥10^-3, versus a mean bootstrap relative half-width of 27.9% for the finite-sample reference; after recalibration and transfer to a GCG-style corpus, the relative error is 2.93%. The directional criterion KL(p_good||p_bad) is consistent with the ordering of estimation error (Spearman ρ=0.83).