Skip to content
Xianzhi Community (Aliyun)·Indexed by this site · Originally published· Event →· Its place in the scan record →

AI can attack and defend for you, but it cannot bear responsibility for you: a retrospective on one irreversible write

AI summary

A red team practitioner reviews an agent incident: during an authorized test, the AI actually wrote a piece of data in order to prove that an interface had a missing authorization flaw (the author rates it CVSS 7.5 per CWE-862); the interface is append-only, and the record was subsequently read out by the front end and appeared in a display slot on the official website; after attempts to delete it failed, the asset owner was contacted for handling. The author believes the model has no concept of irreversibility, and that the reproducible evidence required by the prompt was concretized into a write operation through successive handoffs. Local reproduction shows that on an append-only service, only a pre-write snapshot can achieve rollback, and dry-run validation is slower and offers no protection; in the guardrail experiment, a whitelist plus irreversible sign-off stopped 8 of the 9 actions that should have been stopped with no false blocks, but full data export within authorization remains a blind spot for method-level policy. The article concludes with practices such as tool contract wording, four-layer gates, authorization letter granularity and reversibility grading, emphasizing that AI is responsible for discovery and humans are responsible for deciding and bearing responsibility.

Read originalxz.aliyun.com