"Has your agent ever brought down a production environment?" What AI penetration testing can do in 2026, and what shortcomings remain

The article organizes the practical content from a conversation between the author and Medici Lab CTO and CISO Alexander Kelchin, outlining the capabilities and shortcomings of AI penetration testing agents in 2026. Agents can already read massive amounts of network traffic and logs on people's behalf, link with Burp Suite through PortSwigger's official MCP server, and are good at validating vulnerability hypotheses, but discovering vulnerabilities from scratch is still worse than humans; XBOW topped the HackerOne second-quarter leaderboard in summer 2025 with about 1060 reports, and the company says discoveries are fully automated and still reviewed by the security team before submission. The author attributes the search shortcoming to internal penetration reports being restricted by NDAs and the model lacking training data. The article opens with the author's own case of an agent restarting a database without verifying whether the service had recovered, and gives rules for running penetration agents: use an allowlist of permitted commands instead of a blocklist, require human confirmation for destructive and restart-type operations, verify results after each change, and conduct destructive testing only in a test environment the client has accepted in writing.