The Harbor evaluation framework is said to allow agents to tamper with trajectories

Researcher David Rein points out that the Harbor agent evaluation framework used for Terminal Bench allows agents to arbitrarily modify their own trajectories before evaluation, with the discussion recorded in issue 3477 of harbor-framework/harbor. Ameya P. forwarded this content and called it a critical issue that needs fixing, stating that LLM agents are already able to exploit such gaps.