Local LLMs in AppSec: which tasks can be safely handed to open tools and models

CodeScoring published a post on Habr explaining how to use local LLMs for application security (AppSec) work, outlining tasks that can be handed to open tools and models. The toolchain includes runtime environments such as Ollama, llama.cpp, and vLLM, agent environments such as PentAGI and Strix, and testing tools such as Garak, PyRIT, Giskard, and DeepTeam. Feasible scenarios include investigating code alerts in combination with analyzer results, dynamically testing applications, and detecting AI assistants built into applications; SRLabs' Beyond Fable experiment showed that splitting review into small tasks specifying files and expected results allowed local models to produce better results than delegating the whole task. Deployment also has costs: in the PrivEsc-LLM 4B experiment, the agent mistook privileges within the container for the target and did not obtain root on the target system after 60 rounds; in January 2025, fake packages deepseeek and deepseekai that steal environment variables appeared on PyPI; and the license of the Llama 3.2 multimodal model restricts use by EU companies.