Poster Paper: Rethinking LLM Code Generation Security Through Real-World Risk Scenarios

The researchers constructed a security benchmark for large language model code generation based on real-world development risk scenarios, containing 2,700 test cases; evaluation of eight advanced large language models showed that the average vulnerability rate of each model under risk scenarios exceeded 56%. From the developer's perspective, the benchmark covers three types of risk scenarios, namely ambiguous requirements, underspecified operational context, and conflicts between security and functionality, to address the fact that existing benchmarks rely on explicit security requirements and struggle to reflect real development situations where prompts are often vague or incomplete. The authors demonstrate that security-aware prompting can substantially mitigate these risks, with improvements of up to 45%.