Jie Zhang discusses VLM prompt injection defense ideas
AI summary
When @JieZhang_ETH proposed this project, I joked that I knew the idea came from him rather than an LLM, because it was too quirky. That's a compliment! When we are flooded with AI junk research, going off the beaten path and trying quirky things is more important than ever.
Full text · Original
When @JieZhang_ETH pitched this project, I joked that I knew the idea was from him and not an LLM as it was too weird.
I meant this as a compliment!
As we get inundated with AI slop research, it's more important than ever to wander off the beaten path and try weird things
1/ We know VLMs mostly learn instruction-following from text, so for them an image is something to describe, not obey. Can we turn that into a prompt injection defense? 🧵View the quoted post on X