Beyond Prompts: Runtime Contracts for Coding Agents
Beyond Prompts: Runtime Contracts for Coding Agents
Runtime contracts enforced during coding agent execution produced no oracle-judged rule violations across 104 paired runs, whereas without rules 21% to 55% of runs violated rules. The study conducted three experiments across two model families (240, 112 and 108 runs), comparing runtime contracts with prompt rules, retrieval-based guidance and post-hoc checks; violations were judged by an oracle different from the in-loop checking method and validated by two human reviewers and one model reviewer. Two confirmatory studies found no loss of functional correctness; in Study 3, targeting known failure cases (the protocol was hash-fixed before the first run), enforcement reduced the proportion of violating runs by 31 percentage points relative to no rules (Holm-corrected p = 0.014), and the 17 percentage point difference from prompt rules was not significant (p = 0.12). Prompt rules eliminated some violation types but could not stop recursive deletion of the scratch directory, and post-hoc checks could repair file edits but could not retract commands already executed. The experiments used low-cost models and a small test fixture.