CyberSecEval 2: A broad cybersecurity evaluation suite for large language models

Researchers released CyberSecEval 2, a cybersecurity evaluation suite that quantifies the security risks and capabilities of large language models, adding two new categories of tests: prompt injection and code interpreter abuse. Evaluations of models including GPT-4, Mistral, Meta Llama 3 70B-Instruct, and Code Llama show that the prompt injection test success rates of all tested models range from 26% to 41%, and eliminating attack risk through conditioning remains an unsolved problem. The paper proposes a trade-off between safety and utility, quantified by the false refusal rate (FRR), and finds that many models can complete