Unreasonable confidence from simulator misconfiguration: directional robustness on the SPEED+ sim-to-real benchmark
Unreasonable Confidence from Simulator Misspecification: Directional Robustness on the SPEED+ Sim-to-Real Benchmark
AI summary
A monocular pose estimator trained on synthetic spacecraft images performs poorly when transferred to the real sunlamp/lightbox domain on the SPEED+ benchmark, and incorrect poses still carry high confidence: on synthetic data, 90% conformal calibration intervals cover only 9–15% and 21–38% of rotation errors. Expanding data and features reduces synthetic rotation error from 1.65 to 1.03 rad, yet improves sunlamp by only 0.05 rad.