Vision Wormhole: Latent-space communication in heterogeneous multi-agent systems

Researchers propose Vision Wormhole, which repurposes the visual input interface of vision-language models as a continuous latent-space communication channel between frozen heterogeneous agents. Its general-purpose visual codec encodes the sender's latent-space rollout into a fixed-size message, which after mapping through a shared reference space is decoded into the receiver's image token range; N models use a hub-and-spoke architecture requiring only O(N) components. Each model independently learns the codec through self-distillation on anchor text, and shared anchor alignment allows the codec to be reused across different communication partners. On four VLM families, six team configurations and nine reasoning benchmarks, Vision Wormhole improves accuracy by an average of 6.0 percentage points over text-mediated multi-agent systems, and batch-normalized end-to-end runtime achieves a 1.69× geometric mean speedup.