Skip to content
Infosecurity Magazine·Indexed by this site · Originally published· Evaluation →· Its place in the scan record →

Three AI models agreed on only one of 121 potential code defects

Three AI Models Agree on Just One of 121 Potential Code Flaws

AI summary

Software security company Echo had Claude Mythos 5.1, GPT-5.6 Sol, and GLM-5.3 audit the same batch of open-source code; of 121 potential issues, only 1 was flagged by all three models simultaneously. These issues were found by Echo using its own analysis framework and a unified verification process in 2 Maven libraries and 1 npm package, of which 100 were raised by only a single model. Nine runs produced a total of 446 initial findings, which after cleanup left 152, 74 rated High or Critical, 55 subjected to automated proof-of-concept testing, and 42 passed. Echo says this is a controlled comparison rather than a comprehensive benchmark, automated verification does not equate to confirmation by project maintainers, and project names and technical details will not be disclosed until each maintainer has been notified.

Read originalinfosecurity-magazine.com