On the Trade-off between Transparency and Security in Adversarial Machine Learning

The researchers examine the conflict between transparency and security through transferable adversarial example attacks, in which the attacker uses a surrogate model to maliciously perturb inputs in order to deceive the defender's target model. A large-scale empirical evaluation of nine attacks on 181 models finds that attacks are more likely to succeed when the attacker and defender make the same choice about whether to defend, so concealment may benefit the defender. The paper models the problem as a Nash game and a Stackelberg game respectively and compares the expected outcomes; the analysis shows that merely knowing whether the defender's model is defended is sometimes enough to harm its security, indicating that the transparency of AI systems may conflict with security, and that game-theoretic reasoning can be used to reveal such conflicts between transparency and security.