PORT: improving preference optimization robustness with token-level reweighting
Start here
An arXiv paper released on November 10, 2026 proposes the PORT method, which improves the alignment robustness of large language models under noisy preference data through token-level reweighting. The method uses the empirical cumulative distribution function of token log-probabilities as a forward-pass-only proxy to selectively suppress corrupted tokens without additional backpropagation, and theoretically lowers the upper bound of gradient bias caused by noisy preference labels. Experiments under various noise settings show that PORT consistently improves robustness and outperforms existing sequence-level baselines.
Generated by AI from reports · Updated 2 days ago
Attention and developments
Attention, hourly
Now 23, -2% in 24 hours·Peak 26 (Nov 10 20:00)
Related research
This event's paper
- PORT: Preference Optimization via Robust Token-Level Reweighting
arXiv · 2026-11-09 · Deep reading
Progress timeline
- arXiv: Alignment, Deception & OversightPORT: Preference optimization via robust token-level reweighting
PORT improves the robustness of large language model alignment under noisy preference data through token-level reweighting. The method uses the empirical cumulative distribution function (CDF) of token log-probabilities for reweighting, serving as a forward-pass-only proxy to selectively suppress corrupted tokens without additional backpropagation, and theoretically lowers the upper bound on gradient bias caused by noisy preference labels. Experiments under multiple noise settings show that PORT consistently improves robustness and outperforms existing sequence-level baselines.
Related discussion
Attention trend
New sources per day
- Nov 10: 1 new sources (1 outlets, 0 accounts)
- Nov 11: 0 new sources (0 outlets, 0 accounts)
- Nov 12: 0 new sources (0 outlets, 0 accounts)
- Nov 13: 0 new sources (0 outlets, 0 accounts)