UK AISI 开源 Transect 智能体评测分析工具

1. 22:00
   X @AISecurityInstSelected

   UK AISI 开源 Transect，将 Agent 轨迹转为可交互时间线

   UK AISI 发布开源工具 Transect，用于把 AI 智能体的运行轨迹转换成可交互时间线。该机构指出，随着智能体编写代码、运行实验并相互协作，复杂评测越来越难解读，单一评测分数会掩盖智能体达成该分数的具体过程。Transect 正是针对这一问题，让评测者能够查看轨迹中的过程细节。

2. 08:00
   UK AI Security InstituteSelected

   UK AISI open-sources Transect, turning large-scale agent evaluation records into verifiable reports.

   UK AISI releases the open-source Python package Transect, built on Inspect Scout, which places activity labels, token usage, and recorded events from agent evaluation logs onto a single turn-indexed timeline, generating an interactive report that reviewers can use to jump to corresponding log segments to verify the basis of automated analysis. Users need to provide evaluation logs, task context, and the activity categories they wish to distinguish. Transect uses user-selected LLM-as-judges to label activity segments and can classify sub-agents based on delegation instructions. In an open-ended AI research evaluation, AISI used Transect to track the read/write flow of four documents—research plan, baseline code, experiment draft, and blind review—across multiple agents, showing that sub-agents first collaborate on the research plan and codebase, and after the experiment begins, focus collaboration on experiment design and data. Transect saves activity labels and individual model judgments, allowing analysis to be reopened without calling the model again; when judgments are repeated or multiple judge models are used, reviewers can see where judgments diverge.

1. 21:52
   arXiv

   Transect：为长程 LLM Agent 评测保留可观测性的开源工具

   研究者发布开源包 Transect，用于在长程 LLM Agent 评测中保留可观测性。该工具基于 Inspect Scout 构建，用户在可复用的评测族配置中指定任务上下文与行为词汇，判分模型与分析设置另行提供。其可导航报告把记录事件、token 用量、子 Agent 活动与模型生成的行为标签对齐到同一按回合的时间线上，评审者可快速把握一次运行的脉络、把任一标签或事件追溯到来源回合，并导出底层数据表做跨运行分析。作者在一项生成近 1300 万 token 的 AI R&D 评测上演示了该流程，将 Agent 工作划分为与研究技能分类对应的行为阶段、子 Agent 委派与交互以及 token 使用；合并视图显示工作集中在操作性任务与稿件产出，几乎没有持续假设生成阶段的证据。

Current attention54·Peak in the comparable period57（Oct 8 14:00）·Change over the comparable past 24 hours 0%