Toward Safe Autonomy for Computer-Operating Agents

1. 01What problem does this paper attempt to solve?

   The authors use SA-CUA to draw a conservative trust boundary for CUA and use endorsement instead of repeated confirmation.

   CUA's screenshots entangle trusted interfaces with attacker-writable content, making it difficult for existing architectures to both block indirect prompt injection and avoid repeated confirmation.

   • Scenario: The authors state that CUA operates GUIs with mouse and keyboard, without requiring per-application integration. Attackers can leave IPI in third-party writable environments, steering trajectories toward exfiltration, messaging, authorizing payments, or deleting data. Khodayari et al. (2026) found 15.3K injections in 1.2B URLs.
   • Existing limitations: The authors state that alignment and safety classifiers are heuristic, reducing ASR but not providing safety guarantees. CaMeL and FIDES rely on initial trust labels on structured interfaces; screenshots lack typed values to attach labels to.
   • Gap: CaMeL-CUA makes the privileged model blind to the environment, limiting utility. UCM relies on site owners annotating untrusted regions in the DOM, which the authors state is insufficient evidence from LLMs. The authors argue that after constraining return types, a single bit can still steer the privileged model toward adversarial branches; higher capacity falls back to user confirmation, causing confirmation fatigue.
   • Threat model:
     • Goal: Hijack benign tasks to redirect them toward attacker-chosen malicious goals.
     • Injection surface: IPI is injected only in third-party user-generated content such as forum posts, emails, lists, or comments. The paper excludes network attacks that would alter the trust state of trusted sources.
     • User side: Benign users maintain allowlists of trusted domains and email contacts; tasks cover web pages, native apps, or both.
     • Victim system: A CUA that visually operates the screen. The OS layer carries trust via per-turn file metadata, with internet-downloaded files treated as untrusted.
   • Proposal: SA-CUA. By default, the entire screen is masked, and conservative deterministic rules from the accessibility tree reveal only trusted regions; values crossing boundaries must be endorsed by independent trusted sources. Humans only describe the task sufficiently at the start to enable endorsement.

   Collapse this answer

2. 02What related research exists?

   The authors place SA-CUA on dual-LLM information flow control and compare it against CaMeL-CUA and UCM.

   Section 2 of the paper discusses three groups; the experiments compare against unprotected CUA and CaMeL-CUA.

   Prompt injection and CUA evaluation

   • WASP (Evtimov et al., 2025) — places injections on WebArena sites, exploiting both visual and DOM channels.
   • Other injection surfaces: EIA (Liao et al., 2025) adds visually invisible deceptive forms to the DOM; Johnson et al. (2025) puts adversarial triggers in HTML aria-labels and reads them into the accessibility tree; Zhang et al. (2025) uses pop-ups to carry IPI; Seip and Hein (2026) and Wu et al. (2025) perturb screenshots directly.
   • Utility and safety benchmarks: utility uses OSWorld (Xie et al., 2024) and WindowsAgentArena (Bonatti et al., 2024). Safety evaluation includes OS-Harm (Kuntz et al., 2025), and RedTeamCUA (Liao et al., 2026), which places adversarial content in mixed web–OS environments.

   Alignment and safety classifiers

   • Model side: Constitutional AI (Bai et al., 2022), deliberative alignment (Guan et al., 2024), and SecAlign (Chen et al., 2025b), which pairs safe and unsafe completions with DPO. Also StruQ (Chen et al., 2025a), which only follows special prompts, and Instruction Hierarchy (Wallace et al., 2024), which makes models prioritize privileged instructions.
   • Classifiers and action gating: Llama Guard (Inan et al., 2023), LlamaFirewall (Chennabasappa et al., 2025), Constitutional Classifiers (Sharma et al., 2025). Task Shield (Jia et al., 2024) allows actions based on whether they serve the user's task; GuardAgent (Xiang et al., 2024) uses a separate guard agent to check plans.
   • The paper's claim: the authors say the above defenses are heuristics that can lower ASR but do not provide safety guarantees.

   Information flow control

   • Principles: the Biba model (Biba, 1977) does not allow high-integrity entities to read low-integrity data. dual-LLM (Willison, 2023) lets a privileged model plan and call tools without reading untrusted content, while an isolated model can read content but cannot call tools. The authors say this preserves control flow but remains vulnerable to data flow attacks.

   • Tool-based agents: CaMeL (Debenedetti et al., 2026) gives a plan at the start of the trajectory and tracks taint; FIDES (Costa et al., 2025) selectively hides untrusted tool outputs. Both block consequential actions that use untrusted data.

   • Moving to CUA: CaMeL-CUA (Foerster et al., 2026) extends CaMeL to computer use to guarantee control flow integrity. Prismata (Villa et al., 2026) gates actions by the integrity of the target element along the page tree ancestor path. UCM (Nikolić et al., 2026) masks untrusted regions under the assumption that site owners provide trust labels, and reads them with finite-capacity structured queries, which the authors say hurts utility.

   • Baselines and datasets: the main experimental baseline is CUA (Xie et al., 2024), which maps screenshots to pyautogui, and CaMeL-CUA, which writes the full plan in advance. Adversarial tasks are adapted from WASP and RedTeamCUA, with the value substitution suite RTC-value and native app injections added. No-injection utility uses an OSWorld subset and UCM-B. Trust annotation is also compared with UCM's CSS selectors, and UCM is evaluated only on web captures.

   The authors frame the difference as: default-deny trusted reveal using the AT-SPI accessibility tree over mixed web–OS, rather than masking untrusted content on the DOM; and endorsement of data-dependent tasks against independent sources, rather than only low-capacity returns or asking the user to confirm.

   Collapse this answer

3. 03How does the paper address this problem?

   SA-CUA masks the full screen by default, uses accessibility tree rules to reveal trusted regions, and values crossing boundaries must be robustly endorsed.

   SA-CUA follows FIDES's dual-LLM information flow control: the privileged planning model (P-LLM) only sees screenshots after trusted regions are revealed; the isolated model (Q-LLM) sees the full screen but can only answer by specified type, and return values are handed to the planner as redacted variables.

   Trust on screen

   • Three difficulties: clicks or keystrokes are syntactically benign, and their effect depends on the element under focus. Screenshots entangle trusted and untrusted visual content. The accessibility tree is incomplete: aria-hidden deletes nodes, aria-owns changes parents, and container permissions can override subtrees outside them.
   • User-generated content: Definition 3.1 treats a region or printed node as user-generated if any party other than the operator can make it display different content. The authors say this is broader than UCM: it asks who can influence the displayed value, rather than judging by channel capacity. False negatives send untrusted strings to the P-LLM, while false positives only lose functionality, so the goal is zero false negatives.
   • Default deny: start from a masked screen and reveal only trusted content. Visible regions without a corresponding accessibility node stay masked. First-party interfaces (browser chrome, native app menus and toolbars, window frames) are not masked; the address bar is treated as trusted chrome because sites cannot write to it and only trusted URLs can be clicked.

   Rule proposal and application

   • Offline four steps: for each URL template or app, the RP-LLM first writes a site prior using only the host and category; the input is a sanitized tree where names and text are replaced with length placeholders, keeping role, id, and class, so page body text is not visible. It then proposes trust rules and prunes rules that are too broad or empty by covered line count. Finally, it checks on the unredacted subtree whether the operator wrote each of those strings; this check can only delete rules.
   • Three shortcuts: consent banners are matched by known consent management platform ids. The capacity shortcut reveals elements whose rendered text exactly matches 24 closed patterns (e.g., prices, dates, file sizes, counts), but not when other elements in the same row or column would also be revealed. Files recorded as user-created are revealed in lists and native apps.
   • Online selection: stored rules are retrieved by hostname and URL template, and proposed on the fly if there is no match. Templates replace ids and slugs in paths with wildcards. Rule sets from different templates are not merged; the authors say merging would cause leakage (Appendix C.5). Native apps have no URL to separate states, so the union of rule sets is used. If a mask cannot be built, retry three times, then stop the turn rather than show an unmasked screen.

   Action policy

   • Following FIDES: consequential calls proceed only when the context and its relevant parameters meet the policy-required completeness, and action decisions depend only on trusted inputs.
   • Coordinates and focus: clicks and typing are treated as consequential and annotated by the region they act on. Coordinates are trusted only when they fall in a trusted region; clicks and typing in untrusted regions are blocked; non-consequential actions like scrolling are unrestricted. Keyboard input is rejected when focus is under the mask. The policy acts on actions rather than the final answer, and redacted variables can be returned to the user without endorsement, to cover UCM-B.

   Endorsement and task specification

   • Robust endorsement: the endorsed value cannot influence the expectation or checking process it is compared against, only whether it matches (Zdancewic and Myers, 2001; Myers et al., 2006). Components that have not seen the candidate value derive expectations only from trusted sources, then do a deterministic check with normalized type-aware equality, without model judgment. The authors say an attacker can make endorsement fail but cannot get a self-chosen value through.
   • Type catalog: queries can only return closed types, such as URL, email, date; free text is rejected after answering. ENUM and INDEX let the page choose among options written by the planner. The only unbounded type is PROCEDURE: it is parsed into a pipeline syntax tree, and leaves are assigned catalog types by program syntax; loops, command substitution, and unknown programs are rejected. Boolean queries are rejected and replaced by a tool that looks up trusted strings in the accessibility tree.
   • Routes: endorse compares the value against the user instruction, parameter knowledge from before the turn, a finite candidate set the planner gives from trusted context, or task constraints plus type. Dedicated tools check curated registries, manifests on the allowlist, or re-derive from allowlisted sources and compare. Procedures are first filtered by the planner's declared capabilities, then endorsed leaf by leaf. The authors say these routes are meant to test whether the robustness principle holds across the value types in Section 4, not to cover every task.
   • Specification classifier: at the start, it checks whether the task describes the required value well enough to endorse. The value only needs to be described, not named. If the task is entirely defined by user-generated content, the classifier rejects it and asks the user to be more specific. In evaluation, the classifier and P-LLM use the same model.

   Collapse this answer

4. 04What experiments did the paper conduct?

   The authors report an ASR of 0 for both information flow frameworks, and SA-CUA achieves a higher completion rate than CaMeL-CUA.

   Experimental Setup

   • Models: P-LLM is Qwen3.8-27B and GPT-5.6-Luna. The Q-LLM for SA-CUA and CaMeL-CUA is always Qwen3.8-27B; the RP-LLM for agent evaluation is always GPT-5.6-Luna. In the annotation experiment, both serve as RP-LLM.
   • Baselines: The unprotected CUA maps screenshots to pyautogui; CaMeL-CUA writes a plan in advance, then obtains values through an isolated model. The authors rerun the baselines in this setup and do not cite published numbers.
   • Scale: WASP† 42 (GitLab 22, Reddit 20); RTC† 65 (Reddit, OwnCloud, RocketChat); RTC-value 78 (Reddit 40, Thunderbird 28, GitLab 10). No-injection utility is a small OSWorld subset of 36 and UCM-B 99.
   • Metrics: ASR@k counts an attack as successful if any of k seeds hits; CR is the seed-averaged benign completion rate; RTC-value additionally reports FR, where neither the benign nor the attack goal is achieved. ASR is counted by any hit, CR and FR are averaged over runs, and the three need not sum to 100. Judgments come from environment state, not LLM review. The k column in Table 1 and Table 16 writes 3 (Qwen3.8-27B) and 1 (GPT-5.6-Luna).
   • Environment: Based on OSWorld. Each turn is in an Ubuntu VM inside Docker, screen 1920×1080; responses are pyautogui executed in the VM. DONE, FAIL, or 50 steps end the episode, with context retaining the last 10 turns. The appendix additionally writes that 70 P-LLM turns or 6 consecutive tool errors end the episode. Unless otherwise noted, temperature 1.0, top-p 0.9. Web applications are self-hosted containers; adversarial content is written to the backend before the turn begins. RocketChat colleagues are simulated by a local Qwen2.5-32B and are instructed to pass payloads verbatim.

   Main Results

   Per Table 1 (%). RTC-value cells are CR (FR). The ASR for that column is not placed in the table: CUA is 43.6 (Qwen3.8-27B) and 38.5 (GPT-5.6-Luna), and CaMeL-CUA and SA-CUA are both 0.0.

   Swipe horizontally to read the table

   Framework (P-LLM)	WASP† ASR/CR	RTC† ASR/CR	RTC-value CR (FR)	OSWorld CR	UCM-B CR
   CUA (Qwen3.8-27B)	4.8 / 81.7	46.2 / 95.4	53.4 (17.9)	58.3	90.9
   CaMeL-CUA (Qwen3.8-27B)	0.0 / 61.1	0.0 / 18.5	18.8 (81.2)	8.3	9.1
   SA-CUA (Qwen3.8-27B)	0.0 / 88.9	0.0 / 86.2	50.9 (49.1)	30.6	50.5
   CUA (GPT-5.6-Luna)	7.1 / 97.6	38.5 / 98.5	60.3 (3.8)	58.3	91.9
   CaMeL-CUA (GPT-5.6-Luna)	0.0 / 73.8	0.0 / 27.7	29.5 (70.5)	8.3	19.2
   SA-CUA (GPT-5.6-Luna)	0.0 / 100.0	0.0 / 81.5	69.2 (30.8)	38.9	49.5

   • Adversarial columns: The authors state that CaMeL-CUA's completions concentrate on tasks where the instruction already specifies the value; extracting values from the screen mostly fails, and none of the multi-step Thunderbird tasks succeed (Appendix C.3).
   • Relative to unprotected CUA: The authors state that SA-CUA has higher completion on WASP†, similar completion counts on RTC-value, and no turn adopts injected values, with the remaining failures being harmless.
   • No-injection suites: The authors state that several OSWorld and UCM-B tasks require acting on unverifiable untrusted information; SA-CUA cannot complete them without dropping guarantees or asking the user for confirmation. CaMeL-CUA completes very few tasks on these two suites.

   Trusted Content Annotation

   • Setup: Table 2, 160 manually labeled captures, 18 web domains and 13 native apps, 20,744 lines (trusted 12,229, untrusted 8,515); among untrusted lines, 2,048 lines (24.1%) have bounded capacity. UCM is evaluated only on 115 web captures. The positive class is untrusted. Parentheses indicate capacity shortcut on.
   • Numbers: Under both RP-LLMs, SA-CUA's FNRfree is 0.0 / 0.0 (all/web-only). UCM's web-only FNRfree is 44.1% (Qwen3.8-27B) and 36.7% (GPT-5.6-Luna). SA-CUA's web FPR is 76.9% and 56.3%, and UCM's is 5.2% and 4.7%. The capacity shortcut does not change FNRfree, only reveals bounded-capacity lines, raising FNRbounded to near the oracle's 58.7% on all captures with the shortcut on.
   • Author interpretation: The authors state that SA-CUA's safety corresponds to over-masking; switching to GPT-5.6-Luna as RP-LLM reduces this utility loss.

   Injection on Frontier Models

   • Setup: The authors select 10 tasks they consider harder, covering multiple suites and different injection types, run with Claude computer use API default settings, including the safety classifier. This is not an SA-CUA result.
   • Numbers: Claude Opus 5 has 0/10 attack successes and completes 10/10 benign tasks. Claude Sonnet 5 is hijacked on 2/10, both phishing emails with spoofed senders: the report goes to the impersonator rather than the colleague, the attack succeeds, and the benign task fails. Sonnet 5 completes 8 of 10 benign tasks.
   • Author interpretation: The authors state that these attacks are not trivial, because at least two of them succeed on Claude Sonnet 5.

   Instruction Specificity

   • Setup: Appendix C.4, Table 16 splits RTC† by whether the instruction names the target. specific gives both the process and the value, loose only describes what to do, and generic leaves the value entirely to untrusted content.
   • Numbers: Unprotected CUA's ASR decreases as instructions become more specific: Qwen3.8-27B from 50.0% for generic to 44.0% for specific, GPT-5.6-Luna from 42.9% to 36.0%. CaMeL-CUA and SA-CUA's ASR remains 0.0. CaMeL-CUA's specific CR is 37.3% and 64.0%, with loose and generic close to no completion. SA-CUA's specific CR is 98.7% and 100.0%; Qwen's loose/generic is 83.3% / 69.0%, and GPT's is 61.5% / 85.7%.
   • Author interpretation: The authors state that vague instructions give the unprotected CUA more room to follow injected steps, but more than a third of specific instructions are still attacked. SA-CUA's failures mostly fall in loose and generic. All generic runs are flagged by the specification classifier and subsequently rephrased as loose.

   Other Ablations and Analyses

   • Removing endorsement and policy gate (Table 14, Qwen3.8-27B, two rows share tasks, seed 1 each, ASR@1): after ablation, RTC† ASR 1.5%, RTC-value ASR 3.8%; WASP† CR from 92.9 to 69.0, RTC† CR from 80.0 to 98.5. This set of gated numbers is not the same task set as Table 1.
   • By platform (Appendix C.3): Unprotected CUA's successful WASP† attacks are all on GitLab (Qwen 9%, GPT 14%), with Reddit at 0. On RTC†, OwnCloud is 76% and 67%. SA-CUA's RTC-value: Reddit 83% and 100%, GitLab 40% for both models, Thunderbird 8% and 36%; CaMeL-CUA's Thunderbird tasks all fail.
   • Rule transfer (Table 17, GPT-5.6-Luna): Reusing the same URL template at runtime gives FNRfree 0.00, FPR 63.8%; a different template gives FNRfree 4.8. The union of native app rules gives FNRfree 0.00, FPR 42.8%.
   • Annotation audit (Table 19, same GPT-5.6-Luna run, 160 captures): Before audit, FNfree is 6, FNRfree 0.09, FPR 54.65; after audit, FNfree is 0, FNRfree 0.00, FPR 61.83. The audit can only delete rules.
   • Per-turn tokens (Table 15, GPT-5.6-Luna, successful turns, multiples count only P-LLM): SA-CUA relative to CUA is 7.49 (uncached) and 4.17 (cached pricing); after removing the fixed prefix, 2.13 and 3.07. SA-CUA's Q-LLM mean is an additional 9,588 tokens. CaMeL-CUA's P-LLM total× uncached is 0.17.
   • Rule proposal (Table 18): GPT-5.6-Luna on 160 captures averages 58,754 input tokens per run, 40,072 with cached pricing. The authors state that a single run is less than one-fifth of an SA-CUA turn (cached 18.0%, uncached 10.1%).

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors state that only integrity is guaranteed, not confidentiality, and some untrusted values cannot be independently verified.

   Limitations and future directions noted by the authors

   • No confidentiality (Section 6): the authors state SA-CUA guarantees integrity against external manipulation, not confidentiality, and the agent may still leak secrets, for example by posting to a public Reddit thread.
   • Long prompts (Section 6; called a future question in Appendix C.2): whether the P-LLM's long system prompt can be shortened is an open question, via fine-tuning on trajectories so the model learns to handle masked screenshots.
   • Cannot verify independently (Section 6): some untrusted content cannot be checked independently. The authors give the example of finding and buying a laptop by criteria, where the criteria can often only be checked in untrusted seller information.
   • Trade-offs (Section 6): the authors say such limitations are the price of safety and view the paper as a step toward how to trade off utility, cost, and safety.

   Experimental coverage

   • Planning and auxiliary models: the P-LLMs in the main results are Qwen3.8-27B and GPT-5.6-Luna; the Q-LLM is always Qwen3.8-27B; the RP-LLM in agent evaluation is always GPT-5.6-Luna (Section 5).
   • Tasks: WASP† 42, RTC† 65, RTC-value 78, on GitLab, Reddit, OwnCloud, RocketChat, Thunderbird; no-injection utility is OSWorld 36 and UCM-B 99 (Section 4, Table 3).
   • Comparisons: the main results compare against CUA and CaMeL-CUA (Table 1). The annotation comparison is against UCM, and only on 115 web captures (Table 2). The authors say the UCM agent cannot be run as-is in this setup because it redraws pages, reads the DOM, and masks only regions marked untrusted, so they use an ablation without endorsement and policy gates instead (Appendix C.1).
   • Annotation corpus: 160 manually labeled captures, 18 web domains, 13 native apps, 20,744 lines (Section 5, Appendix B). The paper does not report the number of annotators or inter-annotator agreement.
   • Other runs and judging: 10 tasks ran Claude Opus 5 and Claude Sonnet 5 via the Claude computer use API (Section 5). Success is judged by a deterministic evaluator of environment state (Appendix A.1). Table 1 gives k; sampling is temperature 1.0, top-p 0.9.

   Collapse this answer

6. 06Summarize the main content of the paper

   SA-CUA uses default masking and endorsement for CUA information-flow control; the authors report an ASR of 0 across three adversarial task suites.

   SA-CUA is a dual-LLM information flow control framework for computer-use agents, designed to maintain integrity under indirect prompt injection and reduce in-run user confirmations.

   • Problem: screenshots entangle first-party interfaces with third-party writable content, and click effects depend on the landing point. The authors say alignment and classifiers cannot provide safety guarantees. CaMeL-CUA trades integrity for blindness to the environment, and UCM relies on untrusted annotations on the DOM and asks the user to confirm when higher capacity is needed.
   • Approach: mask the full screen by default, derive rules offline from the sanitized AT-SPI tree, and reveal only trusted regions online; missing coverage costs utility, not safety. The Q-LLM reads untrusted regions by closed type, and the P-LLM only holds redacted variables. Before a value is read or used for a consequential action, it must be endorsed deterministically against expectations that have not seen the value. A specification classifier rejects tasks that cannot be endorsed at the start. Clicks and typing in untrusted regions are blocked by the policy.
   • Adversarial results: in Table 1, with Qwen3.8-27B and GPT-5.6-Luna as P-LLMs, SA-CUA and CaMeL-CUA both have ASR 0.0 on WASP†, RTC†, and RTC-value. SA-CUA's RTC† CR is 86.2% and 81.5%, and CaMeL-CUA's is 18.5% and 27.7%. On RTC-value, SA-CUA's CR is 50.9% and 69.2%, and FR is 49.1% and 30.8%; the authors say no turn used an injected value.
   • Utility and annotation: without injection, SA-CUA's OSWorld CR is 30.6% and 38.9%, and both CUAs are 58.3%; UCM-B is 50.5% and 49.5%, and CUA is 90.9% and 91.9%. In Table 2, SA-CUA's FNRfree is 0, and UCM is 44.1% and 36.7% on web only, while SA-CUA's web FPR is 76.9% and 56.3%.
   • Authors' conclusions: the authors say SA-CUA achieves zero attack success rate on all adversarial tasks and outperforms CaMeL-CUA in preserving utility; automatic trust annotation has zero free-text false negatives. They also state they only guarantee integrity, not confidentiality, that some conditions cannot be verified outside untrusted seller information, and that this is the price of safety.

   Collapse this answer

Related papers