Skip to content

Translation in progress Some entries still include text in the original language.

Oct 12 · Monday

All papers

Found 9 papers

Another 265papers do not have category tags yet and are not included in filtered results.

  1. Paper audits LLM judge reliability: repeated agreement does not equal correctness

    Auditing LLM judge reliability and defining trustworthy verdict rate and accuracy upper bound

    Published
    Evaluated models
    GPT-5.2, GPT-4o, Gemini-2.5-Pro and others6
    AbstractThe authors claim that comparing judges should look at T, not just accuracy.

    The authors audit whether LLM-as-a-Judge decisions can still be treated as actionable ground truth under repeated queries, swapped candidate order, and changed prompt formats.

    Paper analysisRead analysis
  2. Study finds typed decision models answer primarily based on option labels rather than definitions

    揭示类型化决策模型按选项标签而非定义规则作答

    Published
    Evaluated models
    laya-typed-decisions, laya, laya-multilingual and others9

    Abstract揭示类型化决策模型被标签误导的机理,证实源于提示词渲染并给出实操建议。

    Paper analysisRead analysis
  3. 论文:安全路由评测在分布偏移下失效,基线选取偏差被低估

    揭示安全路由评测在分布偏移下的虚假下限与选择偏差

    Published
    Evaluated models
    claude-3-5-sonnet, claude-3-opus, claude-sonnet-4-6 and others9

    Abstract论文揭示安全路由评估在分布偏移下的虚假下限,指出防御重心应转向外部动作级强制。

    Paper analysisRead analysis
  4. 研究发现护栏模型在基座模型不确定处过度自信

    诊断护栏相对基座在对抗提示上的置信失校

    Published
    Evaluated models
    Llama-Guard 7B, Llama-Guard-2 8B, Llama-Guard-3 8B and others10
    Abstract对抗包装使 guard 高置信漏检,而对应 base 在同一输入上常更不确定。

    诊断性分析:五个提示词分类 guard 的置信度,在清洁输入上可以较校准,在越狱包装后则变成高置信度的错误判定。

    Paper analysisRead analysis
  5. RewardExplainer:用反事实偏好反馈学习奖励模型解释

    提出 RewardExplainer,用反事实偏好反馈学习奖励模型解释

    Published
    Evaluated models
    FsfairX-LLaMA3-RM-v0.1, Skywork-Reward-V2-Qwen3-1.7B, RM-Mistral-7B and others6
    AbstractRewardExplainer 用双向反事实反馈训练 RM 解释器,并用以发现偏差、定向微调原 RM。

    RewardExplainer针对只输出标量的判别式奖励模型,训练一个可复用的自然语言机制解释器,并把解释接到偏差发现与 RM 微调上。

    Paper analysisRead analysis
  6. LLM Judge 验证在稀疏重叠下的问题:从推理到设计

    分析稀疏重叠下 LLM judge 与人类一致性的验证误差并规划重叠分配

    Published
    Evaluated models
    Gemini Flash, Gemini Pro, GPT-4o-mini and others10
    Abstract稀疏重叠主导 judge 验证的错误决策。

    这篇工作研究标注重叠很稀时,怎样验证 LLM judge 是否足以替代人类,并规划重叠该留多少、怎么分配。。部署前的标准做法是在校准集上比较 judge 与人类的一致性。

    Paper analysisRead analysis
  7. 研究提出医疗答案事实核查失败分类法,指出检索后验证范式的固有局限

    归纳开放式医学长答案检索核验的失败分类

    Published
    Evaluated models
    GPT-5.4, Mistral Small 3 (24B-Instruct-2501), Mistral Small 4 (119B-2603) and others8
    • ComponentRAG
    Abstract作者把开放式低召回归为证据是否在库、能否被定位,以及与声明是否对齐。

    这篇工作诊断开放式医学事实核验的 retrieve-then-verify 管线断在哪里,而不是提出新的核验器。

    Paper analysisRead analysis
  8. 研究称 GPT 系列安全训练把性别歧视转化而非消除

    提出 harm laundering,证明安全训练改写而非消除性别歧视

    Published
    Evaluated models
    gpt2, gpt2-medium, gpt2-large and others15
    AbstractGPT 谱系上毒性下降与表征伤害上升并存。

    Wyer、Black 与 Al Moubayed 用 OpenAI GPT 谱系检验:毒性分数下降是否等于歧视减少。 他们的答案是否定的,并把“显性伤害被改写成评测指标看不见的形式”称为 harm laundering。

    Paper analysisRead analysis
You have reached the end