AgenTest: Mining business rules to test business logic flaws in LLM agents
AgenTest: Mining Business Rules to Test LLM Agents for Business Logic Bugs
The authors propose AgenTest, which mines business rules from system prompts, tool definitions, and domain knowledge and generates Given–When–Then tests, detecting 89/121 rule violations on the three τ²-bench agents.
- Tool-based LLM agents must follow business rules when executing tasks, but these rules are scattered across system prompts and tool definitions or implicit in domain knowledge; existing tests only evaluate task success rates or predefined policies and struggle to systematically find unwritten business logic bugs.
- AgenTest mines business rules from system prompts, tool definitions, and LLM domain knowledge, deduplicates and filters them, and refines them into Given–When–Then specifications; it then constructs, for each specification, an environment state satisfying the preconditions and a simulated user, drives multi-turn dialogue with an interaction strategy library, and finally judges violations with a hybrid oracle combining rule checks and LLM judgment.
- On the airline, retail, and telecom agents of τ²-bench, AgenTest produces 551 refined specifications and detects 89 and 121 unique bugs on DeepSeek-V4-Flash and Qwen-3.8-Flash respectively, which the authors call 3.4 times and 2.5 times the total of all baselines combined; about 60% of bug-containing runs cannot be detected from the final state alone.
- The authors note that the evaluation covers only three domains of one benchmark and two underlying models, and requires system prompts, tool definitions, and a resettable environment with a known data model; the effectiveness of the interaction strategies in unseen domains was not tested (Threats to Validity).
- Evaluated models
- DeepSeek-V4-FlashQwen-3.8-Flash
- Benchmarks
- τ²-bench (airline, retail, telecom)
- Metrics
- unique bugs(违反的精炼规格数)runs/bugprecisionagreement ratebug rate
The researchers propose AgenTest, which mines business rules from system prompts, tool definitions, and domain knowledge, refines them into Given-When-Then specifications, and tests them one by one to discover business logic flaws in LLM agents. For each specification, the method constructs an environment state satisfying the Given conditions and a simulated user, drives the agent through multi-turn interaction, and uses a hybrid oracle to check whether messages, tool calls, and state changes satisfy the Then conditions. On the airline, retail, and telecom agents in tau2-bench and two underlying models, AgenTest refines 551 specifications and detects violations of 89 and 121 distinct specifications on the two models, respectively, which is 3.4 times and 2.5 times the total of all baselines; only AgenTest detects cases where the agent improperly accesses or modifies other users' data. About 60% of runs that trigger flaws cannot be discovered from the final state alone and require checking the full interaction against explicit business rules.
Paper analysis
What problem does this paper attempt to solve?
How to automatically test business logic bugs in tool-based LLM agents when there are no explicit business specifications.
Core problem: tool-using LLM agents must comply with business rules, but these rules rarely exist as explicit, testable specifications, making business-logic bugs hard to find systematically.
- Problem setting: customer-service agents modify orders, issue refunds, and cancel subscriptions through backend APIs; the authors argue they must not only complete user requests but also comply with business rules that constrain when and how operations may be performed; whether rules are followed may vary with dialogue wording and context, bringing risks such as unauthorized operations and erroneous transactions (§1).
- Shortcomings of existing methods: the authors say one line of existing work targets general security threats such as jailbreaks and prompt injection, not business rules for legitimate operations; another line, such as τ²-bench, evaluates with predefined tasks and manually specified expected outcomes, requiring a person to specify in advance what to test and how to judge it (§1).
- Core observation: rules are scattered across system prompts and tool definitions, or implicit in domain semantics. In the example in Figure 1, an airline agent modifies someone else's booking and charges a fee without verifying the requester's identity; the task appears successful and the final state is legitimate, but it violates the rule that "only the booking owner may request a modification", which is not stated in any single artifact.
- Method proposed in the paper: AgenTest turns each business rule into an explicit test objective: mine rules from system prompts, tool definitions, and domain knowledge, refine them into Given–When–Then specifications, then use the specifications to drive environment construction, multi-turn interaction, and violation detection (§1, §3).
- Problem formalization: given an agent A and an environment E but no explicit specification set, the goal is to automatically derive a specification set S, operationalize each specification into an executable test, and discover as many business-logic bugs (s, τ) as possible within the test budget (§2.2).
What related research exists?
Related work falls into two categories: tool-based agent testing and specification-driven testing. The authors state that no work treats each business specification as an explicit test target.
Benchmark evaluation of tool-using agents
- τ²-bench [6], τ-bench [30], AgentBench [21], WorkArena [13], WebArena [33]—evaluate conversational or tool-using agents with predefined tasks; MANTRA [2] synthesizes tasks from process documents. This paper evaluates directly on the three agents of τ²-bench.
- AgentHarm [3], AgentDojo [11], InjecAgent [32], etc.—safety benchmarks focused on harmful actions and prompt injection. The paper notes that such work does not target business rules for legitimate operations.
Automated testing of tool-using agents
- IntellAgent [19]—a multi-agent framework that samples policy combinations from policy text to generate scenarios and initial databases, with an LLM judging the dialogue; it is what the paper calls the closest open-source method and serves as the main baseline.
- Other automated testing work [15, 18, 20, 23]—generate policy-based scenarios, mine workflow boundaries, coverage-guided fuzzing, or adversarial dialogue; other work evaluates recorded trajectories against prompt rules [28] or enforces temporal constraints at runtime [17].
Specification-driven testing
- Test generation and conformance checking based on formal models [14], guiding tests with natural-language requirements and behavioral scenarios [25, 29], and message and sequence generation for fuzzing guided by protocol knowledge [22]. The paper notes that these methods rely on explicit requirements, behavioral models, or protocol specifications, whereas tool-using agents often lack independent business specification documents.
Baselines and benchmarks: the comparison baselines are τ²-bench's 278 human tasks (50, 114, and 114 for the three agents respectively), IntellAgent, and MANTRA (whose released suite contains 44, 57, and 31 tasks), all run on the airline, retail, and telecom agents of τ²-bench (§4.1).
Authors' positioning: no existing work treats each business specification as an explicit test objective, so implicit specifications and specifications requiring specific states are rarely tested; AgenTest mines each specification, constructs its applicable scenarios, and checks behavior accordingly (§7).
How does the paper address this problem?
AgenTest has two stages: mine and refine business rules into Given–When–Then specifications, then use the specifications to drive environment construction, multi-turn interaction, and hybrid oracle detection.
Overall approach: turn each business rule into an explicit test objective and use it to drive the entire testing process. AgenTest consists of two stages: specification mining and refinement, and specification-driven testing (Figure 2).
Business rule mining (§3.2.1)
- Three sources: tool definitions (parameter constraints and cross-tool call order constraints), the full system prompt (extract rules while preserving context and bind them to relevant tools), and LLM domain knowledge (infer unstated rules for each tool along four dimensions: parameters, environment state, order, and behavioral norms, then infer cross-call constraints in three patterns).
- Coverage and binding checks: track whether prompt paragraphs are cited; for uncited paragraphs, an LLM judges whether they contain missed rules; screen whether rules apply to more tools, and new bindings must include citations and be verified.
- Deduplication and filtering: two layers of LLM filtering—first merge equivalent rules (keeping the more reliable source: schema over prompt over domain knowledge), then check whether every concept in domain-knowledge rules is defined in the data model, prompt, or tool parameters; rules with no definition or that only constrain tool return values are removed.
Rule refinement (§3.2.2)
- Scenario derivation: the LLM reads rule conditions as a disjunction C = C1 ∨ ⋯ ∨ Cn, and each conjunctive branch Ci yields a permitted scenario; negating the whole yields prohibited scenarios, and the authors emphasize that the negation must be of the whole, otherwise negating only one branch may still leave the operation permitted.
- GWT encoding: each scenario is encoded as a specification s = (G, W, T), where Given states the applicability conditions, When states the triggering request, and Then states the required, prohibited, or permitted behavior; Given guides environment construction, When guides the user task, and Then provides assertions for the oracle to extract.
Test-driven construction (§3.3.1)
- Environment setup: split Given into atomic clauses and classify them (environment conditions, user-side conditions, ownership relations), map them to conditions of the form ⟨table, field, op, value⟩ and verify them against the data model in code; search for objects satisfying the conditions via foreign-key relations, and if none are found, modify attributes of existing objects; specifications whose preconditions cannot be established are not tested.
- User task and simulator setup: convert user-side conditions into what the user conveys in the dialogue, derive the target task from When and bind it to the selected object; when the object belongs to someone else, the simulator does not initialize the owner's private information.
Adaptive test generation and execution (§3.3.2)
- Strategy library: classify specifications into 13 structural categories by the constraint type of Then (Table 1), enumerate user variations in timing, form, completeness, wording, and boundary proximity, keep variants that still distinguish compliant from non-compliant responses, yielding 67 domain-independent strategies plus one default strategy (Table 2).
- Selection and adaptation: an LLM matches applicable strategies to each specification; later rounds use bug experience memory to boost strategies that previously exposed bugs and downweight strategies that repeatedly fail. Each selected pair is bound by an LLM to a concrete object and task, with expected behavior fixed before execution and with tool-reference checks and compliant tool-sequence replay.
- Multi-turn interaction: each round resets the environment, user state, and dialogue history, and the simulated user generates responses following strategy instructions; messages, tool calls, user actions, and initial and final states are recorded; the process stops when the budget is exhausted or no usable specification–strategy pairs remain.
Hybrid bug detection (§3.3.3)
- Assertion extraction: before execution, collect the target specification and other specifications that constrain the same tools and whose Given holds; an LLM extracts constraints from Then and binds objects; parameter, state, and event-order constraints become executable predicates, dialogue constraints remain as natural-language assertions, and all retain the force of "required/prohibited/permitted".
- Oracle judgment: executable assertions use deterministic checks; dialogue assertions are judged by an LLM with supporting citations, and citations must be verified against the trajectory; judgments without support are recorded as undetermined. A run that violates any assertion of the refined specification under test is recorded as a violation.
What experiments did the paper conduct?
On τ²-bench's three agents and two underlying models, answer four RQs: specification quality, bug-finding effectiveness, component contribution, and bug analysis.
Experimental Setup
- Agents under test: the airline, retail, and telecom agents of τ²-bench, with 15, 17, and 13 tools respectively; telecom additionally has 30 user-side tools; the underlying models are DeepSeek-V4-Flash and Qwen-3.8-Flash, and each model constitutes a different agent under test (§4.1).
- Baselines: the τ²-bench task set (278 human tasks), IntellAgent (does not support telecom user-side tools and runs only airline and retail), MANTRA (released suites of 44, 57, and 31 tasks, single-turn with no simulated user).
- AgenTest configuration: GPT-4.1 for specification extraction, Claude Opus 5.5 for test case synthesis and LLM judging, DeepSeek-V4-Flash as the simulated user; specifications are mined only once per domain and shared by the two underlying models (§4.1).
- Bug confirmation: two authors independently review all AgenTest runs and all violations reported by the baselines, disagreements are resolved by discussion, and only manually confirmed violations count as bugs.
- Counting method: deduplicated by violated refined specification, one specification counts as one unique bug; turn rules written in all three prompts are excluded (under the default policy the two models violate them in 69.3% and 32.2% of cases respectively).
- Run scale: in RQ2, AgenTest runs three rounds, with each of the 504 testable specifications run once under the default policy plus two rounds of policy iteration (Table 5 totals 1,512 runs).
Main Results
According to Table 4 (unique bugs; outside parentheses is the total across the three agents; Runs/bug is the number of runs required per bug):
Swipe horizontally to read the table
Method DS total DS Runs/bug Qwen total Qwen Runs/bug τ²-bench 5 55.6 8 34.8 IntellAgent 20 51.0 41 24.9 MANTRA 6 110.0 12 55.0 AgenTest 89 17.0 121 12.5 Note: DS is DeepSeek-V4-Flash, Qwen is Qwen-3.8-Flash; IntellAgent was not run on telecom (shown as "–" in the table). By domain, AgenTest finds 23/41/25 (DS) and 37/52/32 (Qwen) on airline/retail/telecom respectively.
- The authors state that the number of bugs detected by AgenTest is 3.4 times (89 vs. 26) and 2.5 times (121 vs. 49) the sum of the three baselines, and that it requires the fewest runs per bug.
- Figure 4 shows that of AgenTest's 89 and 121 bugs, 83 and 97 respectively were not found by any baseline; of the baselines' combined 26 and 49, AgenTest found 6 and 24. The authors state that the bugs unique to the baselines all occur in long tasks chaining multiple objectives, whereas AgenTest tests only one specification at a time.
- In the equal-budget comparison in Figure 3, the authors state that AgenTest leads at every budget above 44 runs (DS) and 178 runs (Qwen); below that budget IntellAgent is slightly higher, which the authors explain as its few bugs recurring across many dialogues.
RQ1: Specification Extraction Quality
- Process numbers: mining yields 483 specifications, deduplication removes 120, binding fixes add 25, filtering removes 19 domain-knowledge specifications, leaving 369; refinement yields 551 (an average of 1.5 refined specifications per specification) (Table 3).
- Correctness: two authors independently annotate and a third arbitrates; 88.0% of refined specifications are correct (κ=0.88); by source, tool definitions are all correct, system prompts 91.2%, domain knowledge 80.6%.
- Error causes: of the 66 incorrect specifications, 30 are unsupported by any source, 19 contradict the prompt or tool documentation, and 17 lose conditions.
RQ3: Component Effectiveness (DeepSeek-V4-Flash)
- Test-driven construction: of the 551 specifications, 504 (91.5%) can be construction-driven; 47 fail, of which 26 specifications are incorrect and 21 require states the environment cannot hold. In 1,503 of 1,512 runs (99.4%), the simulated user states the user-side conditions and triggers the When (Table 5).
- Interaction policy: the default policy alone violates only 6 specifications, randomly selecting policies violates 84, and the full AgenTest violates 89 (Table 6); the authors state that the interaction policy contributes most of the bugs, and bug experience memory adds 5 more on top of that.
- Hybrid oracle: agreement with manual review is 97.6% (1,476/1,512), precision 82.6% (161/195); the precision of the baseline judges is τ²-bench 25.0% (7/28), IntellAgent 46.5% (93/200), and MANTRA 5.7% (16/281) (Table 7). In default-policy cases, 85% (492/579) of checks are deterministic rules.
RQ4: Bug Symptoms and Root Causes
- Symptom distribution: invalid parameters are the most common (DS 43/89, Qwen 46/121), followed by prohibited operations (13, 22) and missing prerequisite steps (12, 25); the four symptom types involving business data tool calls together account for 86.5% and 85.1% (Table 8).
- Observability: 60% (DS) and 59% (Qwen) of bug-containing runs cannot be detected from the final state alone, and of these 42% and 37% do not change the database.
- Root causes: implicit rules, overlooked explicit rules, and misused explicit rules account for 50.6%/30.3%/19.1% for DS and 36.4%/26.4%/37.2% for Qwen respectively; unauthorized operations almost all stem from implicit rules (8/9 and 9/10). Annotation κ=0.93.
Other Ablations and Analyses
- Equal-budget curves: within the 278-run budget of τ²-bench, AgenTest finds on average 20.3 (DS) and 27.7 (Qwen) bugs, versus 5 and 8 for the baselines; within the 660-run budget of MANTRA it finds 45.2 and 61.7, versus 6 and 12 for the baselines (§4.3).
- Bug repetition: each IntellAgent bug is exposed by an average of 5.8 and 8.4 dialogues, MANTRA by 3.0 and 6.9 runs, and AgenTest by 1.3 and 1.3 (§4.3).
- Of IntellAgent's 93 confirmed-violation dialogues on DS, 70 violate the same rule "must not state information not provided by the user or tools"; on Qwen, 169 of 280 do (§4.3).
What are some points that can be further explored?
The authors list three types of threats: internal, construct, and external validity, and plan to improve the validation of domain-inferred specifications and multi-specification interaction testing.
Limitations and future directions noted by the authors
- Internal validity: Errors in extraction, user simulation, and LLM judgments may affect results; results may vary across runs, although they are consistent across the two underlying models (§6).
- Construct validity: Counting bugs by refined specifications depends on AgenTest's mining and refinement approach, and incomplete specifications may limit AgenTest's exploration scope (§6).
- External validity: The evaluation covers only three domains of one benchmark and two underlying models, and may not generalize to other agents or deployments; AgenTest requires system prompts, tool definitions, and a resettable environment with a known data model; the effectiveness of the interaction strategies in unseen domains was not tested (§6).
- Future directions: Improve validation of domain-inferred specifications and test interactions among multiple specifications (§8).
- Observation in the discussion: Bugs found only by the baselines all occur in long tasks with multiple chained objectives, and the authors say combining refined specifications of shared tools into multi-objective tests could serve as a supplement (§5).
Experimental coverage
- The subjects under test are the three agents of τ²-bench: airline, retail, and telecom, with two underlying models: DeepSeek-V4-Flash and Qwen-3.8-Flash (§4.1).
- The comparison baselines are the τ²-bench task set, IntellAgent, and MANTRA, three in total; IntellAgent runs only on airline and retail (§4.1).
- The main AgenTest experiment has three rounds: 504 testable specifications each run once with the default strategy plus two rounds of strategy iteration, for 1,512 runs in total (Table 5); the component ablation is conducted only on DeepSeek-V4-Flash (§4.4).
- All reported violations were confirmed by manual review by two authors; specification correctness annotation κ=0.88, bug symptom and root cause annotation κ=0.93 (§4.1, §4.2, §4.5).
- The paper does not report AgenTest's end-to-end runtime or LLM call cost.
Summarize the main content of the paper
AgenTest mines scattered and implicit business rules into Given–When–Then specifications and drives testing with them, detecting 89/121 rule violations on τ²-bench.
AgenTest is a specification-driven testing method for LLM agents that directly manipulate business data through tools, addressing the problem that business rules are scattered across system prompts and tool definitions or implicit in domain knowledge and thus hard to test systematically.
- Method: mine rules from three sources—system prompts, tool definitions, and LLM domain knowledge—deduplicate and filter for applicability, split scenarios by condition structure and encode them as Given–When–Then specifications; for each specification, construct an environment state satisfying the preconditions and a simulated user, drive multi-turn dialogue with 67 domain-independent interaction strategies plus bug experience memory, and judge violations with a hybrid oracle consisting of deterministic checks and LLM judgments with citation verification.
- Specification quality: produces 551 refined specifications on the three agents of τ²-bench, with 88.0% manually annotated as correct (κ=0.88), and errors concentrated mainly in rules inferred from domain knowledge (Table 3).
- Main results: detects 89 and 121 unique bugs on DeepSeek-V4-Flash and Qwen-3.8-Flash respectively, which the authors call 3.4 times and 2.5 times the total of the three baselines combined (26 and 49), with 83 and 97 of them found by no baseline (Table 4, Figure 4).
- Component contribution: using only the default strategy finds only 6 bugs, with interaction strategies contributing the vast majority; the hybrid oracle has 82.6% precision, versus at most 46.5% for baseline judges (Table 6, Table 7).
- Bug characteristics: invalid parameters are the most common symptom; 60% (DS) and 59% (Qwen) of bug-containing runs cannot be detected from the final state alone; unauthorized operations almost all stem from implicit rules not stated in any artifact (Table 8).
- Authors' conclusion: business-logic bugs arise both from unstated rules and from stated rules that are overlooked or misused, and different root causes require different fixes; structured rules can be enforced in tools, and refined specifications depend only on prompts and tool definitions, so they can be reused as a regression suite when the underlying model changes (§5).