SusVibes evaluation: coding agents without safety instructions produced secure solutions only 11.8% of the time
SusVibes : 11,8 % des solutions générées par IA sont sécurisées | Brief IA
AI summary
In the SusVibes evaluation, coding agents that were not given safety instructions produced secure solutions only 11.8% of the time, and the highest functional correctness rate of responses was 57%. Security expert Alex Ștefănescu warns that completion features in tools such as GitHub Copilot may suggest nonexistent packages, and attackers register those names on npm or PyPI in advance, so importing them can cause harm. Mimecast's AI-assisted changes are approved only by humans.