Skip to content

Agentic on-policy distillation implicitly reshapes LLMs' ReAct patterns

Agentic On-Policy Distillation Implicitly Reshapes LLMs' ReAct Patterns

Hugging Face Daily Papers· arXiv:2611.19244· Deyang Kong, Zhaoyi Li, Sijian Ren, Yuan Wei, Ying Wei, Defu Lian, Shikun Zhang, Wei Ye·Indexed by this site · 2 days ago· Originally publishedAI updates· Its place in the scan record →
AI summary

The paper studied how on-policy distillation (OPD) reshapes the ReAct interaction patterns of language model agents across three domains—math, ALFWorld and τ²-bench—covering reasoning, actions and observation acquisition. Results showed that general reasoning supervision reduced the tool-call ratio on math tasks from 70.4% to 14.4% while raising accuracy from 16.7% to 38.0%; code interpreter supervision improved household action and retail query performance. ALFWorld supervision, however, dropped τ²-bench accuracy from 41.5% to 6.6%, accompanied by frequent tool calls that violated user information dependencies. The authors proposed a selective OPD that uses only source-domain training trajectories, retaining the top 95% of positions based on changes in teacher preference before and after domain-specific RL, raising τ² accuracy from 11.5% to 29.9% at the cost of impaired source-domain learning.

Read originalhuggingface.co