FoMo-FD: Monitoring execution failures of surgical robot imitation policies with flow matching world models

Zhefeng Huang et al. propose FoMo-FD, an execution-level runtime failure monitor for surgical robot imitation policies, trained only on successful execution data without needing failure demonstrations. It compresses multi-view camera images into visual latent variables via frozen DINOv2 and a feature-space β-VAE, uses a flow matching world model to predict nominal visual-action dynamics over a short time window, and then back-transports the actual endpoint latent variables to the base Gaussian space, comparing a nonconformity score against a threshold calibrated with conformal prediction to trigger alerts. In the Needle Pickup and Ring over Post simulation tasks and the Tissue Retraction and Shunt Insertion tasks on the dVRK, covering 20 failure modes, 320 failures, and 80 successful rollouts, the wrist camera monitoring achieved a 96.6% failure detection rate with a 1.3% false positive rate, while the fixed camera achieved 45.9% and 10.0%.