Auditable Authorization for LLM Agents: Empirical Evidence Across Model, Harness, and Workflow Layers
Auditable Authorization for LLM Agents: Empirical Evidence Across Model, Harness, and Workflow Layers
Translation in progress — untranslated text is omitted.
The researchers propose an auditable authorization method for LLM agents, formalizing human approval as refusal safety and bounded authorization integrity, and auditing at three execution boundaries: model selection invocation, Harness scheduling recovery, and workflow routing to protected effects. The experiments cover three models, five prompt sets, four runtime approval designs, two framework replays with CrewAI and n8n, and 100 public workflow sources. Results show that in the paired model panel, 7 of 300 approvals were not completed and the refused effects were zero; parallel confirmation lost approved branches in three tasks for each model; the recovery process missed 336/672 CrewAI approvals; and 4 of 59 comparable n8n workflows still executed after Reject. The authors state that comparing intent, generated invocation, dispatch, and native effects can locate the first observable divergence and its repair target, and that the resulting audit contract makes refusal safety and authorization integrity independently verifiable, turning human approval from a procedural formality into an executable control boundary between intent and real effects.
- In critical agent systems, human approval is often the last means of control, but the interface recording a decision does not mean the decision controlled the work: approved work may not be completed, rejected effects may still occur, and real execution traces are difficult to make public and reproduce.
- The authors formalize two obligations, refusal safety and bounded authorization completeness, and use paired Approve/Reject decisions, exact traces and native business-state witnesses (rather than LLM judges) to audit three models, four runtime approval designs and 100 public n8n workflows separately at the model, harness and workflow layers.
- In the paired panel, 7 of 300 approvals were incomplete and rejections leaked zero; Google ADK merged parallel approval lost one approved branch on 3/3 tasks; CrewAI recovery missed 336/672 approved effects; 4/59 n8n workflows still executed the protected effect after Reject.
- The authors note: risk-oriented checkpoints and repeated transition episodes do not estimate incidence rates in deployment; the descriptive Wilson intervals are not a claim of independence; AI annotation agreement is low (κ 0.190–0.477), and native effects remain the primary endpoint; synthetic triage accuracy describes only its pre-declared fixed examples.