Comparative Analysis of Agent Evaluation Frameworks: Stability, Risk Detection and Risk Discovery for Enterprise Analytical Agents
Comparative Analysis of Agent Evaluation Frameworks: Stability, Detection, and Discovery in Enterprise Analytics Agents
The authors compare three agent evaluation frameworks on stability, known-risk detection, and unknown-risk discovery, report trace-level evaluation F1=0.90, and recommend a dual-framework strategy.
- Enterprise analysis agents produce syntactically correct but semantically wrong outputs in multi-tool workflows, and manual spot checks easily miss them. The authors aim to compare three types of evaluation frameworks on stability, known-failure detection, and unknown-failure discovery.
- Using frozen execution trajectories, Strands Evals, PromptFoo, and Agenta are each tested on two supply-chain analysis agents. The three stages respectively measure repeated-evaluation stability, detection performance on 72 annotated failure cases, and the ability of automatically generated tests to discover new failures.
- The authors report that 35/36 configurations were stable; Strands achieved the best F1=0.90 in Phase 2, and PromptFoo had similar recall but more false positives; Phase 3 discovered 24 new failures in total, with zero overlap between functional and security testing.
- The authors state that they have not yet evaluated whether the framework can identify real improvements or regressions after agent changes, nor have they packaged the process into a reusable toolkit. The experiments cover only two supply chain analysis agents, three frameworks, and 11,700 evaluations.
- Evaluated models
- Claude Sonnet 4.5
- Benchmarks
- Agent A benchmarkAgent B benchmarkPhase 2 benchmark
- Metrics
- average within-case standard deviationdecision consistencyprecisionrecallF1FPRfalse discovery ratenovel failure count
In a controlled environment, researchers conducted a three-stage comparison of three evaluation frameworks—trajectory-based LLM review, text-only LLM review plus red teaming, and deterministic heuristics—on frozen execution trajectories of two enterprise analytical agents. In the first stage, repeated measurements showed that 97% of evaluation configurations reached the stability threshold, and the stability patterns were independent of the specific agent. In the second stage, known-risk detection was measured on 72 genuinely labeled test cases; the trajectory-based framework achieved the best precision-recall balance (F1=0.90), while text-only evaluation had comparable recall but 5 times as many false positives. In the third stage, 24 new failures were found, with zero overlap between functional tests and safety red teaming. The authors recommend a dual-framework strategy: trajectory-based evaluation for functional assessment and adversarial red teaming for safety validation.
Paper analysis
What problem does this paper attempt to solve?
The authors aim to compare three types of agent evaluation frameworks on stability, detection, and discovery.
The core question is how to reliably evaluate failures of enterprise analysis agents in complex multi-tool workflows. The authors focus on how different evaluation frameworks perform on repeated-measurement stability, known-failure detection, and unknown-failure discovery.
- Application scenario: enterprise teams are deploying analysis agents that generate SQL, call tools, and synthesize answers; erroneous outputs carry direct business consequences and often escape manual review.
- Shortcomings of existing methods: the authors state that many teams rely on ad-hoc spot-checking; in this study, initial manual labeling missed 9 of 72 ground-truth failures (12.5%), and these failures are syntactically valid but semantically wrong.
- Core observation: the authors argue that the level of information the evaluator can see determines evaluation quality; functional evaluation and security evaluation cover different failure surfaces.
- What the paper proposes: a three-phase comparative study examining evaluation stability (RQ1), known-risk detection (RQ2), and unknown-risk discovery (RQ3), comparing three frameworks: Strands Evals, PromptFoo, and Agenta.
- Subjects of study: two retail supply chain forecasting analysis agents; Agent A performs category-level demand forecasting and Agent B performs product-level inbound lead time forecasting; both are evaluated on frozen execution traces in a controlled research setting.
What related research exists?
Related work mainly covers three lines: LLM judging, consistency measurement, and adversarial testing.
LLM-as-a-judge and Agent Evaluation
- Yu [10], Zheng et al. [11]: discuss the adoption of LLM-as-a-judge in evaluating agent systems, and its agreement with human annotation.
- Li et al. [5]: document systematic inter-judge disagreement; the authors use this to argue that evaluation reliability is a key problem.
- Authors' positioning: prior comparisons have focused more on differences in judging method, whereas this paper compares the levels of information available to the judge: full execution trace, text-only output, and string representation.
Randomness and Consistency Measurement
- Atil et al. [2]: analyze LLM stability.
- Mustahsan et al. [8]: propose using ICC to measure inter-run consistency in agentic evaluation; the authors argue that ICC is subject to ceiling effects in better-performing agents.
- Bland and Altman [3], Livingston and Lewis [6]: correspond respectively to within-case standard deviation and pass/fail classification consistency, which the authors adopt as stability metrics.
Adversarial Testing and Information Leakage
- Agrawal et al. [1]: study prompt leakage and defense strategies.
- Mohammadi et al. [7]: survey evaluation and benchmarking of LLM agents, covering prompt injection and information leakage risks.
- Authors' positioning: the authors state that this is the first work to quantitatively compare functional evaluation and security evaluation on the same set of agents, and to argue that the two are complementary rather than substitutable.
Baselines and Benchmarks
- Compared frameworks: Strands Evals, PromptFoo, Agenta.
- Benchmarks used: two self-built supply chain analysis agent test sets, namely the Agent A benchmark and the Agent B benchmark; Phase 2 used 72 annotated test cases.
How does the paper address this problem?
Using frozen trajectories and a three-stage process, the authors separately measure stability, known-failure detection, and unknown-failure discovery.
The overall approach is to place three evaluation frameworks representing different levels of information access on the same set of frozen execution traces for a three-phase comparison. The core frameworks are Strands Evals (trace-based LLM judge), PromptFoo (text-only LLM judge with red-teaming), and Agenta (deterministic heuristics).
Evaluation Subjects and Framework Setup
- Candidate agents: two retail supply chain forecasting analysis agents. Agent A analyzes category-level demand forecasts, and Agent B analyzes product-level inbound lead time forecasts; both use 9 tools.
- Framework differences: Strands reads the full OpenTelemetry execution trace; PromptFoo sees only text input and injected context strings; Agenta sees only string representations.
- Requirement mapping: the authors map five evaluation requirements to each framework's implementation, including groundedness, intent, accuracy, tool correctness, and SQL correctness (Table 1).
Phase 1: Stability Measurement
- Sample and repetition: 20 frozen test cases per agent; LLM frameworks used 3 judge models × 5 repetitions, and Agenta was repeated 5 times.
- Metrics: decision consistency (whether repeated evaluations give the same pass/fail) and average within-case standard deviation; the stability threshold is set at ≤ 0.15.
- Threshold meaning: under 0–1 scores and a 0.5 pass/fail boundary, a within-case std of 0.15 means scores near the boundary have a non-trivial probability of flipping across it.
- Scale: 5,605 evaluations in total.
Phase 2: Known-Risk Detection
- Test set: 36 test cases per agent, 72 in total, covering 6 failure types; each case is independently annotated on 5 dimensions to avoid cross-contamination.
- Annotation method: each agent was annotated by one lead domain expert, and ambiguous cases were cross-reviewed by two other team members.
- Evaluation protocol: all three frameworks ran on frozen traces, repeated 3 times with the median taken, and a pass/fail threshold of > 0.5; the judge was Claude Sonnet 4.6.
- Scale: 2,415 evaluations in total.
Phase 3: Unknown-Risk Discovery
- Generation method: framework-specific generators were added on top of the Phase 2 baseline; Strands used ExperimentGenerator to generate 20 functional test cases per agent, and PromptFoo used red-team to generate 42–48 adversarial cases per agent.
- Classification method: the same domain expert classified each flagged failure as novel, known, or false positive.
- Evaluation setup: LLM frameworks were repeated 3 times, and the judge remained Claude Sonnet 4.6.
- Scale: 202 test cases, 3,680 evaluations in total.
What experiments did the paper conduct?
Experiments show that Strands has the best detection balance, PromptFoo has more false positives, and Agenta has the highest stability but weak semantic detection.
Experimental Setup
- Subjects under test: two analysis agents, Agent A and Agent B, with Claude Sonnet 4.5 as the underlying generative model (Appendix F).
- Evaluation frameworks: Strands Evals, PromptFoo, Agenta.
- Judge models: Phase 1 used Claude Sonnet 4.6, Claude Haiku, Qwen3 32B; the main experiments in Phase 2/3 used Claude Sonnet 4.6.
- Datasets: Phase 1 had 20 frozen cases per agent; Phase 2 had 36 cases per agent, 72 in total; Phase 3 had 202 test cases in total.
- Metrics: stability used average within-case standard deviation and decision consistency; detection used precision, recall, F1, FPR; discovery used novel failures and false discovery rate.
- Repetition and decision rule: LLM frameworks were repeated 3 times in Phase 2/3 and the median was taken, with a pass/fail threshold of > 0.5; the paper does not report a manual annotation agreement coefficient.
Main Results
Swipe horizontally to read the table
Framework Precision Recall F1 FPR Source Strands 88.2% 90.9% 0.896 10.3% Table 5 PromptFoo 60.4% 87.9% 0.716 48.7% Table 5 Agenta 45.8% 100% 0.629 100% Table 5 - Authors' interpretation: Strands achieves the best precision-recall balance on the combined results of the two agents, and this ranking is consistent in each agent's individual results (Appendix B).
- Authors' interpretation: the gap mainly comes from whether execution context is needed; on SQL correctness, both LLM frameworks reach 89% recall, but on tool correctness Strands is 89% and PromptFoo is 11%.
- Authors' interpretation: Agenta is reliable on structural SQL errors, but has 0% recall on Agent B's semantic errors, because regex cannot see such errors.
Stability Experiment
- Overall results: 35 of 36 configurations (97%) reached the stability threshold (Table 2). Agenta had an average std of 0.000 and decision consistency of 100%; PromptFoo was 0.025 and 92%; Strands was 0.053 and 82%.
- Judge differences: Claude Haiku was the most stable (0.029), followed by Sonnet 4.6 (0.035) and Qwen3 32B (0.072). The only unstable configuration was Qwen3/style in Strands (0.207, 20% decision consistency).
- Authors' interpretation: stability is a property of the (framework, model) combination, not of the model alone; variance is concentrated in complex, data-intensive cases (Appendix K).
Unknown Risk Discovery Experiment
- Number discovered: Phase 3 discovered 24 novel failures in total, with zero overlap between functional testing and security red-teaming discoveries (Table 6). Strands discovered 9 functional failures, PromptFoo discovered 15 security failures, and Agenta discovered 0.
- Failure types: Strands discovered SQL bugs, tool workflow gaps, and schema fabrication; PromptFoo discovered schema metadata leakage, storage path exposure, system prompt disclosure, and excessive agency under adversarial prompt injection (Table 7).
- Authors' interpretation: functional testing and security testing target different failure surfaces and are therefore complementary; the number of security findings increases with schema complexity, and Agent B's security findings are 4 times those of Agent A.
Manual Misses and Review
- Misses: across the two agents, 9 failures in total were initially labeled pass by humans and later changed to fail after evaluator prompting; of these, Agent A had 5 SQL partition errors and Agent B had 4 calculation interpretation mistakes.
- Specific pattern: these errors are syntactically valid but semantically wrong, making them hard for manual review to detect.
- Authors' interpretation: an LLM evaluator with a domain rubric and trace access can serve as a second reviewer.
Other Ablations and Analyses
- Label revision sensitivity: after excluding the 9 reclassified cases, combined precision changes from 0.86 to 0.82, recall from 0.91 to 0.88, and FPR remains 0.13 (footnote 1).
- Judge scale analysis: on 10 Agent A test cases, the score correlation between Claude Opus 4 and Sonnet 4.6 is r = 0.97, but latency is about 3.5 times higher (Table 20).
- Variance drivers: the within-case std dev of trace-based judging rises as the number of tools increases (m = 0.017), while text-only judging shows no such relationship (m = -0.006) (Appendix K).
What are some points that can be further explored?
The authors explicitly propose two follow-up directions: toolkitization and discriminative-power evaluation.
Limitations and Future Directions Noted by the Authors
- Toolkit packaging: the authors state that the current dual-framework strategy still requires manually configuring evaluators, collecting and freezing traces, running red-team generation, and interpreting results; future work could package it into a reusable toolkit (Future Work).
- Discriminative power evaluation: the authors state that this paper does not test whether the frameworks can detect real improvements or regressions when the agent changes, which is exactly the capability needed for CI/CD gating (Future Work).
- Controlled perturbation design: the authors propose starting from a baseline agent, constructing variants with known quality differences, and evaluating directional accuracy, effect size, sensitivity, and specificity on the Phase 2 benchmark (Future Work).
Scope of Experimental Coverage
- Agents under test: only two retail supply chain analysis agents, Agent A and Agent B (3.1).
- Evaluation frameworks: only three frameworks are compared: Strands Evals, PromptFoo, Agenta (3.2).
- Evaluation scale: 11,700 evaluations across the three phases; Phase 1 is 5,605, Phase 2 is 2,415, and Phase 3 is 3,680 (Appendix E).
- Judge coverage: the main-experiment judge is Claude Sonnet 4.6; Phase 1 also compared Claude Haiku and Qwen3 32B; Appendix J compares Claude Opus 4 only on 10 Agent A cases.
- Not reported in the paper: the manual annotation agreement coefficient, and the complete query budget or time cost for the cases produced by each generator in Phase 3.
Summarize the main content of the paper
The authors use a three-stage experiment to show that the information visible to the evaluator determines the quality of agent evaluation.
This paper compares three agent evaluation frameworks on stability, detection capability, and unknown-risk discovery in enterprise analysis scenarios. The problem the authors focus on is that syntactically correct but semantically wrong agent outputs often escape manual spot checks.
- Method: Compare Strands Evals, PromptFoo, and Agenta on the same set of frozen execution trajectories, representing trace-based LLM judging, text-only LLM judging with red-teaming, and deterministic heuristics, respectively.
- Stability results: 35 of 36 configurations (97%) reach the stability threshold; Agenta is the most stable, and Strands has higher variance but is still mostly stable (Table 2).
- Detection results: On 72 known failure cases, Strands achieves the best balance, F1=0.896; PromptFoo's recall is close but its FPR is higher; Agenta fails on semantic errors (Table 5).
- Discovery results: Phase 3 discovers 24 new failures in total, with zero overlap between functional testing and security red-teaming; Strands finds functional problems, and PromptFoo finds security problems (Table 6, Table 7).
- Manual comparison: 9 ground-truth failures were initially labeled pass by humans and were later re-judged as fail with the evaluator's prompt.
- Authors' conclusion: The authors believe that the level of information visible to the evaluator determines evaluation quality, that functional evaluation and security evaluation are complementary, and recommend using Strands for functional evaluation and PromptFoo for security validation.