Three AI models agreed on only one of 121 potential code defects
Three AI Models Agree on Just One of 121 Potential Code Flaws
Software security company Echo had Claude Mythos 5.1, GPT-5.6 Sol, and GLM-5.3 audit the same batch of open-source code; of 121 potential issues, only 1 was flagged by all three models simultaneously. These issues were found by Echo using its own analysis framework and a unified verification process in 2 Maven libraries and 1 npm package, of which 100 were raised by only a single model. Nine runs produced a total of 446 initial findings, which after cleanup left 152, 74 rated High or Critical, 55 subjected to automated proof-of-concept testing, and 42 passed. Echo says this is a controlled comparison rather than a comprehensive benchmark, automated verification does not equate to confirmation by project maintainers, and project names and technical details will not be disclosed until each maintainer has been notified.