The paper proposes the AuthBench benchmark to evaluate whether coding agents can infer minimal permission boundaries.

Before executing terminal tasks, can code agents autonomously infer the minimal file permission boundary that both satisfies task execution and excludes unnecessary sensitive resources.

• Scenario and Importance: Code agents are shifting from simple code completion to multi-turn autonomous workflows in the Shell. However, manually granting task-level fine-grained permissions cannot scale with autonomy, and existing systems generally rely on broad default permissions, leading to a trust-authorization mismatch.
• Disconnect in Existing Evaluations: Existing capability benchmarks default to providing sufficient permissions and only measure completion rates; safety benchmarks measure behavior in established environments; runtime execution frameworks presuppose that policies already exist; none address the source of permission policy generation itself.
• Core Observation and Challenge: Least privilege requires policies to be above the task-sufficient boundary $ \pi^* $ and below the attack-enabling boundary. Frontier models are not simply unidirectionally miscalibrated between conservative and aggressive, but commonly exhibit both under-authorization (missing permissions necessary for the execution chain) and over-authorization (opening unnecessary or sensitive resources).
• Proposed Solution in the Paper: Formalizes the permission-boundary inference task, constructs the AuthBench benchmark containing 120 real terminal tasks, and proposes the Sufficiency-Tightness Decomposition method.
• Threat Model:
  • Attacker Goal: Induce the agent to access sensitive files, exploit insecure auxiliary scripts, or trigger sensitive credential exfiltration/compromised endpoints.
  • Knowledge and Capabilities: The tested agent can only inspect the terminal environment and instructions through read-only operations before execution; sensitive tasks have realistic attack surfaces pre-positioned (e.g., credential files, exfiltration tools curl/tar, insecure scripts).
  • Victim System: Docker-based terminal environment and the execution agent OpenClaw.

The paper categorizes related work into three directions: agent benchmarks, runtime permission control, and agent reasoning planning.

Agent Capability and Safety Benchmarks

• SWE-bench (Jimenez et al., 2023), Terminal-Bench (Merrill et al., 2026), and SWE-PolyBench (Rashid et al., 2025): Evaluate code agents' task completion ability in repository-level and command-line environments.
• ToolEmu (Ruan et al., 2023), AgentHarm (Andriushchenko et al., 2024), Agent-SafetyBench (Zhang et al., 2024), and AgentDojo (Debenedetti et al., 2024): Evaluate agents' harmful behavior, malicious instruction compliance, or prompt injection defense in environments with given tools and permissions.
• GrantBox (Zhang et al., 2026): Evaluates agents' privilege usage on real tools. The paper notes that such work evaluates how models use already-granted permissions (execution safety), rather than synthesizing least-privilege policies before execution.

Least Privilege and Runtime Access Control

• Progent (Shi et al., 2025), AgentBound (Bühler et al., 2025), AgentSpec (Wang et al., 2025), and Pro2Guard (Wang et al., 2025): Regulate and intercept agents' runtime behavior given existing policies or rules.
• Semantic task and scope matching research (El Helou et al., 2025; South et al., 2025): Studies delegation mechanisms where authorization should be tied to user intent. The paper notes these works focus on representation and execution mechanisms, not solving the generation problem of inferring file-level boundaries from concrete tasks.

Long Chain-of-Thought and Agent Reasoning Boundaries

• ReAct (Yao et al., 2022) and long chain-of-thought research (Chen et al., 2024; Chen et al., 2025): Explore agent planning and multi-step tool use, analyzing benefits and saturation when reasoning computation increases. The paper extends this perspective to permission inference.

Comparison Baselines and Benchmarks

• Comparison configurations: Use Full-Access (unrestricted read/write/execute, representing the TSR upper bound and safety lower bound) and Golden-Permission (a static golden policy derived from a human-reviewed safe oracle trace) as reference controls; the baseline method is the Direct baseline, which generates policies in a single direct pass.
• Benchmark environment: AuthBench, built from real command-line tasks (including 80 standard and 40 sensitive tasks).

Essential Difference from Prior Work The authors state that existing safety research focuses on "execution safety" after the model has been granted permissions, whereas this paper shifts focus to "authorization safety" before permissions are granted, studying the independent semantic capability of agents to infer task-scope permission boundaries before acting.

The overall approach is to model permission boundary inference as a two-stage discover-then-audit process, proposing the Sufficiency-Tightness Decomposition.

Problem Formalization

• Input and Output: The input is a natural language task instruction I and a terminal environment E. The authorization model $ f_\theta $ inspects the environment through read-only operations before execution, generating a file-level whitelist permission policy $ \pi = (\pi_\text{read}, \pi_\text{write}, \pi_\text{execute}) $, and any path not covered is denied.
• Optimization Objective: Minimize the redundant scope of the policy beyond the task sufficiency boundary, i.e., min_π d(π, π^∗) subject to \pi^* \sqsubseteq \pi. The task sufficiency boundary $ \pi^* $ strictly depends on the specific execution agent A (the combination of model M and execution scaffold H), not solely determined by the task instruction.

Phase 1: Sufficiency Reasoning

• Forward Simulation of Execution Chain: The model forward-simulates the execution plan that the downstream execution agent might take based on the task goal, parsing the complete transitive toolchain, including interpreters, compilers, package managers, and wrapper scripts.
• Coverage-First Principle: The prompt explicitly requires prioritizing coverage over minimality, generating a candidate policy $ \pi_\text{suf} $, preferring to temporarily over-authorize rather than omit paths necessary for execution.

Phase 2: Tightness Audit

• Item-by-Item Backward Review: Receive $ \pi_\text{suf} $ and generate a subset $ \pi_\text{final} \subseteq \pi_\text{suf} $. Each entry is independently audited against three criteria:
  • Task grounding: Check whether there is explicit evidence in the instruction or environment (e.g., Makefile, import statements, shebang lines). If no evidence, delete.
  • Scope minimality: Evaluate whether broad wildcards (e.g., directory globs) can be narrowed to specific files.
  • Sensitivity check: Check whether it overlaps with system-sensitive files (e.g., credentials, keys, system directories); delete if there is no sufficient necessary reason.
• One-Way Pruning Constraint: Phase 2 only allows retaining or removing entries from Phase 1, and prohibits introducing any new entries that did not appear previously.

Success Criteria and Evaluation Method

• Two-Axis Evaluation Protocol:
  • Permission Tightness: The static dimension computes per-axis Precision, Recall, and F1 against the gold label $ S_\text{gold} $ within the task-facing scope; the execution dimension uses the utility verifier $ V_u $ to test task success rate (TSR).
  • Security Exposure: The static dimension computes the sensitive file exposure rate (SER); the execution dimension uses the attack verifier $ V_a $ to test attack success rate (ASR).
• Appendix G.1 and G.3 provide specific prompt templates.

The paper evaluates 9 frontier models on AuthBench and provides an empirical analysis of reasoning intensity and two-stage decomposition.

Experimental Setup

• Models under test (all 9): GPT-5, GPT-5.3-Codex, GPT-5.4, Claude Opus 4.6, Gemini 3.1 Pro Preview, Kimi K2.5, MiniMax M2.7, Qwen3-Coder-480B, Qwen3.5-397B. Models with reasoning level configurations run at the highest available reasoning intensity.
• Dataset: AuthBench, with 120 terminal tasks (80 standard, 40 sensitive), covering 10 professional engineering domains.
• Baselines and controls: Full-Access, Golden-Permission; each model's own Direct baseline.
• Metric definitions: Precision (P), Recall (R), F1 for read/write/execute axes; Task Success Rate (TSR); Sensitive File Exposure Rate (SER); Attack Success Rate (ASR).
• Review and execution mechanism: Policies are run by a fixed execution agent, using GPT-5 as the execution base and OpenClaw as the harness. Execution results are judged by deterministic code verifiers $ V_u $ and $ V_a $.

Main Results

Main experimental results are shown in the table below (per Table 1):

Swipe horizontally to read the table

Model	Std TSR(%)	Std R-F1	Std W-F1	Std E-F1	Sens TSR(%)	Sens SER(%)	Sens ASR(%)
Full-Access	83.3	Not reported	Not reported	Not reported	94.0	Not reported	65.8
Golden-Perm.	77.1	Not reported	Not reported	Not reported	81.7	Not reported	0.0
Gemini 3.1 Pro	75.4	78.0	85.3	49.0	85.8	34.8	28.3
GPT-5	63.3	83.3	86.8	53.7	76.7	33.6	23.3
GPT-5.4	52.6	73.7	83.2	53.3	61.1	21.1	19.4
Claude Opus 4.6	61.3	84.7	87.4	53.0	61.5	47.0	25.6
Kimi K2.5	60.0	74.1	84.5	54.4	70.0	74.5	28.3
GPT-5.3-Codex	58.8	75.8	78.7	46.7	65.8	42.6	15.8
Qwen3-Coder	52.1	64.1	73.1	49.1	63.3	65.0	20.8
Qwen3.5-397B	45.6	71.8	83.6	59.6	42.9	71.9	17.6
MiniMax M2.7	42.1	66.7	77.6	54.6	48.3	64.2	15.8

• Authors' interpretation: Gemini 3.1 Pro achieves the highest TSR (standard 75.4%, sensitive 85.8%), but sensitive exposure SER is as high as 34.8%; high-precision models (e.g., Claude Opus 4.6 and GPT-5) perform better on static metrics but have lower TSR due to missing key permissions in the execution chain.

Authorization Attractor Analysis

• What was tested: Vector field displacement from low to high reasoning intensity was analyzed on Claude Opus 4.6, Gemini 3.1 Pro Preview, and GPT-5.4 (Figure 5).
• Results: Increasing reasoning did not make models converge to the zero-defect point (0,0), but rather to their respective biased attractors: Gemini tends to guarantee task coverage but with broader permissions; GPT-5.4 and Claude Opus 4.6 tend to minimize permissions but often miss paths necessary for execution.
• Authors' interpretation: The authors state that reasoning scaling only makes models more consistent with their existing trade-off preferences and cannot eliminate the inherent conflict between sufficiency and tightness through single direct generation.

Sufficiency-Tightness Decomposition Evaluation

• What was tested: Comparison of S-T Decomposition vs. Direct baseline on three representative models (Table 2).
• Results: Claude Opus 4.6's TSR on sensitive tasks rose from 61.5% to 75.0%, ASR dropped from 25.6% to 15.0%; GPT-5.4's sensitive TSR rose from 61.1% to 76.9%, ASR dropped from 19.4% to 15.4%; Gemini 3.1 Pro's sensitive SER dropped from 34.8% to 15.7%, ASR dropped from 28.3% to 12.5%, sensitive TSR dropped from 85.8% to 75.0%.
• Authors' interpretation: Two-stage decomposition can correct model-specific unilateral defects: tightness-biased models compensate for execution coverage gaps, while broad-biased models eliminate over-exposure.

Execution Base Robustness Check (Table 4)

• What was tested: Fixed the generated authorization policy and switched OpenClaw's execution model from GPT-5 to Claude Sonnet 4.6 or Gemini 3 Flash Preview.
• Results: Under Claude Sonnet 4.6 execution, Full-Access's sensitive ASR dropped to 7.5%; under Gemini 3 Flash it rose to 82.5%. The sensitive ASR of policies generated by each model ranged from 2.5%–12.5% under Claude Sonnet 4.6 and from 17.5%–35.0% under Gemini 3 Flash.
• Authors' interpretation: Changing the execution base does not affect the core conclusion that authorization inference itself is an independent bottleneck.

Other Ablations and Analyses

• Execution-stage recall analysis (Table 6): When Execute recall is in [0, 0.25), TSR is 55.2% (n=116), while at 1.0, TSR is 73.3% (n=288).
• Full closure coverage analysis (Table 8): When read/write/execute recall all reach 1.0, TSR is 82.2% (n=202); when not fully covered, TSR is 47.7% (n=375).
• Error timing (Figure 8): The first issue causing execution failure mainly occurs in the first two steps of reasoning progress (0–40%), concentrated in tool and path resolution.
• Sensitive task endpoint distribution (main text): Among the 40 sensitive tasks, 20 terminate at remote receivers or upload packages, and 20 terminate at local dangerous outputs or unsafe states.
• Negative results and costs: After S-T decomposition, Gemini 3.1 Pro's standard TSR dropped from 75.4% to 65.0%, and sensitive TSR dropped from 85.8% to 75.0% (Table 2).

Limitations and future directions noted by the authors

• Limited policy language scope: AuthBench currently only evaluates file-level read, write, and execute permissions in terminal tasks, and does not cover network access, inter-process communication, database credentials, cloud resources, browser state, or API-level permissions (Appendix B).
• Extrapolation of the multi-objective attractor phenomenon: The current reasoning intensity analysis only targets model families with configurable reasoning levels. The authors note that future work needs to verify whether similar attractor behavior exists in other multi-objective decisions such as tool selection, data sharing, network access, and sub-agent delegation (Appendix B).
• Multi-agent attribution challenge: AuthBench currently attributes execution failures to permission policy deficiencies rather than to individual agents in the pipeline (Appendix B).
• Epistemic boundaries and calibration: Explore training techniques that enable models to recognize their own knowledge boundaries, to better calibrate authorization decisions (Appendix B).

Experimental coverage

• Task and environment coverage: The benchmark tested is AuthBench, which includes 120 Docker-based Linux terminal tasks (80 standard tasks, 40 sensitive tasks), covering 10 specialized engineering domains.
• Model coverage: 9 frontier large language models were tested (GPT-5, GPT-5.3-Codex, GPT-5.4, Claude Opus 4.6, Gemini 3.1 Pro Preview, Kimi K2.5, MiniMax M2.7, Qwen3-Coder-480B, Qwen3.5-397B).
• Execution environment configuration: The main evaluation uses a fixed GPT-5 + OpenClaw as the execution end, and robustness experiments supplement with Claude Sonnet 4.6 and Gemini 3 Flash Preview.
• Two-stage decomposition test scope: The S-T decomposition experiment was only quantitatively compared on three models: Claude Opus 4.6, GPT-5.4, and Gemini 3.1 Pro.
• Reporting detail boundaries: The paper does not report actual policy generation and interception evaluations for non-file-system permissions (network/API).

This paper studies the ability of code agents to autonomously infer minimal file permission boundaries before executing terminal tasks.

• Core problem: As code agents gain broader permissions, coarse-grained default permissions can introduce security risks, but models autonomously inferring task-level permissions often face a dilemma between the permissions needed for execution and security tightening.
• Benchmark and phenomena: The paper constructs AuthBench, containing 120 terminal tasks, and finds that frontier models generally exhibit both under- and over-authorization; increasing reasoning compute causes models to converge to their own characteristic authorization attractors, entrenching inherent trade-off patterns rather than approaching a safe window.
• Key results: On sensitive tasks, Full-Access has an ASR of 65.8%, while the golden permission policy has an ASR of 0.0% (Table 1); Gemini 3.1 Pro achieves the highest TSR (85.8%), but its sensitive file exposure rate SER reaches 34.8% (Table 1).
• Method effectiveness: The proposed two-stage sufficiency-tightness decomposition first ensures coverage and then performs item-by-item audit, improving the TSR of tightness-biased models on sensitive tasks by up to 15.8% (GPT-5.4 rising from 61.1% to 76.9%), and reducing ASR on all tested models (Table 2).
• Authors' insight: The authors believe that single-pass direct policy generation has reasoning limits, and decomposing the authorization decision into forward coverage and backward audit can effectively break the defect of a single-model attractor.