MLExpert: A General-Purpose Agent Model for Autonomous Machine Learning Engineering

How to train models to autonomously organize experiments in long-horizon machine learning engineering and learn multi-step decisions from delayed execution feedback.

• Scenario: the authors view MLE as a concrete setting toward AI for AI (AI4AI) and recursive self-improvement (RSI). It requires repeated execution, feedback analysis, and revision within a time budget, not one-shot generation.
• Existing shortcomings: the authors state that AIDE, MLE-STAR, R&D-Agent, etc., rely on hand-designed control logic to arrange long-horizon experiments. ML-Agent and Frontis-MA1 mainly train single-step decisions or local operators in external processes.
• Difficulties listed by the authors: an environment that leaves action selection to the model while preserving experiment state; long trajectories mix useful exploration, failure recovery, and degrading decisions. Feedback from one experiment corresponds to many previous steps, and there is policy mismatch during asynchronous collection.
• This paper proposes: post-training for autonomous long-horizon MLE, and training a 35B MoE agent model MLExpert. Supporting components are Kaggle-MLE Factory, Interaction Harness, hierarchical agentic SFT, and Windowed Interaction RL (WIRL).

The authors divide existing work into externally orchestrated research systems and post-training that mainly trains single-step or local operations; their own object is model-driven long-horizon interaction strategies.

Externally orchestrated automated ML research

• AIDE, MLE-STAR (2025) and R&D-Agent (2025): use iterative search, refinement, or multi-role collaboration to improve candidate solutions. The introduction states these systems rely on hand-designed control logic to organize long-horizon experiments.
• MLEvolve, MARS (2026): use tree or graph search, memory, and planning to organize long-horizon experiments. InternAgent (2026) and FM-Agent (2025) extend autonomous research to machine learning, algorithm optimization, and engineering tasks.
• Arbor, Argus (2026): maintain persistent research state, allowing models to design, continue, and revise experiments. The authors state this raises the requirements for planning experiments and maintaining long-horizon optimization.

Training MLE with interactive experience

• ML-Agent (2025): multi-turn demonstrations plus single-step RL. The authors, in the introduction, place it alongside Frontis-MA1, stating it mainly trains single-step or local operations in external processes.
• RL for MLE Agents, MLE-RL, AceGRPO (2026): study online MLE training with execution feedback, asynchronous sampling, or intermediate trajectory states. Learning to Ideate (2026) trains models to generate algorithm improvement suggestions.
• Frontis-MA1 (2026): performs SFT and RL on program evolution operators, then integrates into an external evolution framework. The authors summarize the above lines as mainly training single-step or local components in external processes. SandMLE (2026) does trajectory-level on-policy RL in a micro synthetic environment.

Long-horizon agent post-training

• SWE-Gym (2024), Search-R1 (2025), WebRL (2025): learn multi-turn tool use with executable environment trajectories or environment feedback.
• AgentFlow (2026): directly optimizes multi-turn planning policies and trains intermediate decisions with trajectory-level outcomes.
• Agents-A1, Ornith-1.5 (2026): the former uses long-horizon knowledge-action trajectories, multi-stage training, and multi-teacher on-policy distillation; the latter uses RL to jointly optimize task generation, scaffold construction, and rollout.

Baselines and benchmarks

• Comparison models: same-scale are Qwen3.6-35B-A3B, Ornith-1.5-35B-A3B, Nex-N2.5-mini; larger models are DeepSeek-V4-Pro Preview, MiniMax M3, GLM-5.3, Kimi-K3, GPT-5.5, GPT-5.6 Sol. RL comparisons are trajectory-level GRPO and PPO-style. Benchmarks are MLE-Bench Lite and NatureBench Lite.

The authors state they train model-driven long-horizon policies on real MLE tasks; WIRL uses execution windows, learns multi-turn decisions from delayed feedback, and supports asynchronous updates.

Overall, it is model-driven long-horizon interaction, not an external workflow. Kaggle-MLE Factory provides executable tasks, Interaction Harness manages experiment state; MLExpert is initialized from Qwen3.6-35B-A3B and post-trained with hierarchical agentic SFT and WIRL.

Problem setup

• Task: a task is written as e=(d_e, D_e, m_e), i.e., description, data, and metric. A candidate solution y is a complete pipeline including data processing, features, training, and prediction; S_e(y) is normalized by metric direction, larger is better. Time budget is B_e.
• Visibility: following MLE-bench, the agent sees the task description and public data, and improves solutions by executing code and self-built validation. Final submissions are independently scored on hidden test labels.
• Interaction: the trajectory alternates observations and actions. Each step's action may include reasoning and one or more tool calls; experiment state is placed in an external workspace and exposed via tool observations. The model does not follow a predetermined workflow for when to analyze, revise, execute, or change direction.
• Objective: maximize the expected score of the final selected solution, J(π_θ;e)=𝔼_{τ∼ pπθ(·|e,Be)}[S_e(ŷ_τ)]. The trajectory distribution is induced jointly by the policy and the interaction environment.

Task factory and interaction environment

• Factory: uses a multi-agent pipeline to convert historical Kaggle competitions into executable tasks, including public workspace, hidden test split, and unified evaluation interface. discovery filters by status, size, and submission format; analyzer extracts structure and modality; generator prepares data, hidden label split, and scorer; validator checks and iteratively fixes.
• Scale: Appendix A starts from a Meta-Kaggle snapshot with 11,649 entries. Default filters select completed competitions, at least 50 teams, data no more than 200 GB. Manual review of 160 registered tasks excludes 6 with unresolved label or scoring issues, keeping 154 for trajectory sampling.
• Harness: runs in a ReAct-style loop, with branching and optimization paths decided by the model, not an external search strategy. Solution nodes contain code, execution logs, and metadata; parent-child revisions form a traceable experiment tree.
• Tools and context: the model uses execute_code to launch experiments and update_answer to maintain submission candidates; it may call invalidate_node based on experimental evidence to exclude untrusted candidates. When context is nearly full, early messages are summarized. analyze delegates analysis to a sub-agent in a separate context, returning only the summary to the main model. Appendix B lists all tools.

Hierarchical agentic SFT

• Two stages: the first stage uses StepFun's programming and tool-use trajectories to learn basic tool calls and code revision. The second stage uses MLE teacher trajectories and literature-derived scientific experiment trajectories to learn algorithm design, experimental analysis, and iterative optimization.
• Demonstration types: if the first complete solution executes successfully and passes a quality threshold, extract the prefix from task start to that execution feedback. Also use long-horizon optimization trajectories from full interactions, and analysis sub-agent trajectories.
• Outcome-guided truncation: for trajectories with leaderboard references, filter by private leaderboard relative rank b_j, larger is better, threshold η=0.4. When the average relative degradation after the peak exceeds δ=0.05, anchor at the first submission with the highest score in the trajectory; otherwise use the last submission. The prefix ends at a complete tool interaction boundary near the anchor, and may extend up to ten additional complete tool interactions.
• Loss: supervised samples retain existing context, then append a continuous segment of model actions and environment feedback. The loss is applied only to newly generated reasoning, text, and tool call tokens; tool feedback and inherited history are only conditioning and are masked. Teachers and corpus are in the next section.

WIRL

• Window: the main model decisions from after the last experiment result to the next experiment started by execute_code form an execution window u_m. The first window starts from the initial task state. Feedback from completed experiments can be used for updates before the entire trajectory ends.
• Reward: execution errors caused by model actions are -1.0; no such errors but no scoreable submission is -0.6; a valid submission below the task's medal threshold is -0.4+T_m; bronze or silver is 1.0+T_m; gold is 2.0+T_m. T_m is 0 for the first valid submission, then -0.2, 0, or 0.2 depending on whether the relative change from the previous valid submission is worse, same, or better, considering metric direction.
• Shared weights: unlike GRPO, WIRL does not do within-group relative normalization; it directly uses verifiable execution rewards as shared weights for the main model output tokens participating in training within the window. Windows with infrastructure errors or no scoring signal are not included in training and are not recorded as model error rewards.
• Asynchronous correction: windows are batched asynchronously. The ratio q of rollout log-probability recorded at collection time to the updated policy is weighted only if within [0.2, 4.0]; outside this range, it is not included in the policy gradient. The loss is normalized by the number of valid output tokens in the window, including a KL term relative to a fixed SFT reference model and an entropy term. Appendix C.2 gives the full policy loss.

Data and training configuration

• Teachers: MLE trajectories are generated by GLM-5.2. Scientific experiment trajectories are generated by GLM-5.2 and Qwen3.8-Max. Science tasks follow NatureBench's construction, from 164 Nature sub-journal papers; the evaluation set does not overlap with NatureBench.
• Corpus: in Table 7, MLE-domain SFT retains 636 source trajectories: 412 MLE covering 236 tasks, 224 scientific experiments covering 77 tasks. After segmentation and repeated sampling, there are 5,539 input records. Training tasks overlapping with MLE-Bench or NatureBench are excluded.
• SFT hyperparameters: in Table 8, general stage 1 epoch, sequence 131,072, global batch 32, learning rate 2×10−5 to 10−6. MLE stage 8 epochs, sequence 262,144, batch 8, learning rate 10−5 to 10−6. Both stages use cosine decay, warmup ratio 0.05.
• WIRL hyperparameters: 32 windows per batch, sequence 131,072, learning rate 10−6 to 10−7, no warmup. Sampling temperature 0.6, top-p 0.95, top-k 20. Clipping parameters (0.3, 3.0), KL coefficient 10−3, entropy coefficient 10−4.

主评测是MLE-Bench Lite上的奖牌率；作者另外报告科学任务、scaffold优化和算法组件优化，并比较训练阶段与RL变体。

实验设置

• 基准：MLE-Bench Lite是MLE-Bench的22个历史Kaggle任务。NatureBench沿用Frontis-MA1列出的10任务Lite子集。迁移实验为AIME scaffold优化，以及MLS-Bench-Lite中12个领域各1个任务。
• 模型：MLExpert从Qwen3.6-35B-A3B初始化。同规模对照为该基座、Ornith-1.5-35B-A3B、Nex-N2.5-mini；Table 1将这组标为35B total、约3B active。更大对照为DeepSeek-V4-Pro Preview、MiniMax M3、GLM-5.3、Kimi-K3、GPT-5.5 xhigh、GPT-5.6 Sol。
• 指标：MLE-Bench报告Gold、Silver、Bronze、Any Medal和超过人类中位分的Med+，都按22个任务计百分比，主指标为Any Medal。没有有效分数的提交计为无奖牌。
• 判定：分数来自隐藏测试上的任务评分或官方verifier，论文未使用LLM裁判。NatureBench中Match-SOTA为g≥0，Surpass-SOTA为g>0.1；无有效提交的实例差距记为-1，任务分g为实例均值。
• 预算：每个MLE任务最多3,000步、12小时，20个vCPU、220 GB内存、单张NVIDIA H200。NatureBench每任务4小时，不含打分时间，本文评测用H200。
• 重复：MLExpert与同规模基线在MLE-Bench上评3次并报告SEM；更大前沿模型在MLExpert Harness上各1次。通用harness按表分列Claude Code或Codex。与评测集重叠的训练任务被排除。

主结果

据Table 1，单位为%。下表只列8行；省略Ornith-1.5-35B-A3B（Any Medal 39.4，Claude Code 47.0）和DeepSeek-V4-Pro Preview（Any Medal 63.6，通用harness未报告）。

Swipe horizontally to read the table

模型	Med+	Gold	Any Medal	通用harness
Qwen3.6-35B-A3B	62.1	22.7	37.9	42.4（Claude Code）
Nex-N2.5-mini	66.7	24.2	47.0	36.4（Claude Code）
MiniMax M3	81.8	54.5	68.2	54.5（Codex）
GLM-5.3	90.9	59.1	72.7	77.3（Claude Code）
Kimi-K3	95.5	63.6	72.7	72.7（Claude Code）
GPT-5.5 xhigh	81.8	45.5	72.7	68.2（Codex）
GPT-5.6 Sol	90.9	59.1	77.3	72.7（Codex）
MLExpert	84.8	50.0	72.7	69.7（Claude Code）

• 作者的比较：作者称MLExpert在所评同规模模型中Any Medal最高，并与GLM-5.3、Kimi-K3处于同一水平。同表中GPT-5.6 Sol的Any Medal更高。通用harness列里GLM-5.3也高于MLExpert。
• 换harness：作者称Claude Code下相对基座的优势仍在，并超出所评最强同规模基线22.7个百分点，因此认为增益不限于MLExpert Harness。Kimi-K3的Claude Code与GPT-5.6 Sol的Codex数字来自Frontis-MA1，其余为本文评测。
• 重复：附录Table 9对MLExpert与同规模基线给出三次评测的SEM，更大前沿模型为单次结果。附录D.1另给出Claude Code上三次评测的SEM。

NatureBench Lite

• 相对基座：Table 2在Claude Code下，MLExpert的Surpass-SOTA为50.0%（5/10），基座为0.0%（0/10）；Match-SOTA为70.0%（7/10），基座为40.0%（4/10）。作者称后训练提高了科学任务优化。
• 并不处处更高：作者称Surpass-SOTA高于Nex-N2.5-mini的20.0%，并与GLM-5.3的50.0%相同。附录D.2写明Nex的Match-SOTA为80.0%，高于MLExpert的70.0%。Kimi-K3的Surpass-SOTA为70.0%（7/10）。
• 条件：MLExpert为单次评测。DeepSeek-V4-Pro Preview与MiniMax M3的数字取自Frontis-MA1的Table 2，外部结果保留原评测条件。Table 2中GPT-5.5使用Codex，Match-SOTA为100.0%（10/10），Surpass-SOTA为40.0%（4/10）；论文未写这是否与Table 1的GPT-5.5 xhigh为同一次设置。

跨任务迁移

• AIME scaffold：固定求解器为gpt-4.1-nano-2025-04-14。Table 5中，MLExpert优化的scaffold留出平均准确率43.7%、每次8.40次调用；基座为23.3%与5.74次；MLExpert-SFT为38.1%与3.51次。Initial为20.0%与1.00次，Majority@5为24.8%与5.00次。
• 作者解读：作者称后训练提高了设计工作流的能力，增益延伸到未参与优化的年份。附录称MLExpert-SFT以大约少59%的调用接近Kimi-K3的40.0%；WIRL再把留出准确率从38.1%提到43.7%，调用增至8.40。GLM-5.3为48.1%、9.39次。
• 算法组件：MLS-Bench-Lite选取12个领域各1个任务，同一5小时预算。正文写平均verifier reward从0.322到0.425；Table 12的算术平均为0.321543与0.425198。作者称10个任务更高、1个持平、1个更低，并认为能力可迁到组件优化。

训练阶段与RL对照

• 阶段：Table 3在MLExpert Harness上，Any Medal由Base 37.9、General Agent SFT 47.0、MLE SFT 62.1到WIRL 72.7。作者称每一阶段都有额外增益，WIRL用在线执行反馈超过仅教师监督。
• 对照低于SFT：同一MLE-SFT初始化下，GRPO为56.1，PPO-style为60.6，都低于MLE-SFT的62.1。去掉趋势奖励为66.7，去掉策略校正为65.2，完整WIRL为72.7（Table 4）。作者称这支持按执行窗口学习，且两项设计都贡献了最终增益。
• 等待比例：Figure 5对应的正文写，记录到的更新上GRPO平均等待比例87.6%，WIRL为39.3%，相差48.3个百分点。作者称异步窗口采集降低了相对轨迹级GRPO的learner等待。

其他消融与分析

• Figure 3 / MLSP 2013 Birds：正文写MLExpert的ROC-AUC从0.58到0.964，MLExpert-SFT从0.60到0.88；图中还标有0.48、0.74、0.6。作者称该例是持续实验和执行反馈下的继续改进。
• Table 12例外：3D Scene Densification两者均为0.000000；Exogenous-Variable Target Forecasting上Base为0.445408，MLExpert为0.427787。
• Table 11留出年准确率：MLExpert在2023/2024/2026为43.3、43.3、44.4；MLExpert-SFT为35.6、42.2、36.7。每年为30题评3次。
• 附录D.1，Claude Code的Any Medal：Qwen 42.42±4.01，Ornith 46.97±3.03，Nex 36.36±6.94，MLExpert 69.70±1.52，均为三次。
• 附录D.1阶段SEM：Base 37.88±5.46，General Agent SFT 46.97±1.52，MLE SFT 62.12±1.52，WIRL 72.73±2.62。
• Table 10：GRPO 56.06±5.46，PPO-style 60.61±4.01，无趋势奖励66.67±3.03，无策略校正65.15±5.46，WIRL 72.73±2.62。

作者没有单列局限或后续工作。下面先记录这一点，再按论文实际做过的实验列出覆盖范围。

作者指出的局限与后续方向

论文未专门讨论局限。正文、结论和附录都没有单独写出局限、不足或未来工作。

实验覆盖范围

• MLE-Bench Lite：22个历史Kaggle任务；指标为Med+、Gold、Silver、Bronze、Any Medal，按22个任务计百分比。每个任务最多3,000步、12小时，20个vCPU、220 GB内存、单张H200（第5.1节）。
• 重复与来源：MLExpert和同规模基线在MLExpert Harness与Claude Code上各评3次；Table 1里更大前沿模型在统一harness上各1次。Kimi-K3的Claude Code与GPT-5.6 Sol的Codex结果来自Frontis-MA1，其余为本文评测（附录D.1）。
• NatureBench Lite：10个任务；MLExpert为单次评测，每任务4小时且不含打分，使用H200。DeepSeek-V4-Pro Preview与MiniMax M3的数字取自Frontis-MA1的Table 2。论文未报告该基准上其余模型的重复次数（附录D.2）。
• AIME：求解器固定为gpt-4.1-nano-2025-04-14。开发用AIME 2025全部30题、每候选评3次；留出为AIME 2023、2024、2026，各30题、每年3次。搜索上限4小时、1,000步，每题最多10次内部调用（附录D.3）。
• MLS-Bench-Lite与训练侧：12个领域各1个任务，5小时、最多2,000步，用官方Harbor verifier。训练排除与MLE-Bench、NatureBench重叠的任务；MLE教师轨迹由GLM-5.2生成，科学轨迹由GLM-5.2和Qwen3.8-Max生成（第5.1节、附录C.1）。

MLExpert is a 35B MoE agent model initialized from Qwen3.6-35B-A3B, using post-training to learn to autonomously organize long-horizon machine learning experiments.

• Problem: The feedback from one experiment corresponds to many previous steps, and a complete run contains multiple high-cost experiments. The authors want the model to arrange experiments itself and use intermediate execution results before the trajectory ends.
• Method: Kaggle-MLE Factory turns historical competitions into executable tasks; Interaction Harness leaves branch selection to the model in a ReAct-style loop. Hierarchical SFT uses execution results to select teacher trajectories; WIRL groups decisions between adjacent experimental results into windows, shares medal tiers and local score trend rewards, and performs asynchronous collection and token-level correction.
• MLE-Bench Lite: On MLExpert Harness, Any Medal is 72.7%, and the base model is 37.9% (Table 1). The authors state it is comparable to Kimi-K3 and GLM-5.3; in the same table, GPT-5.6 Sol is 77.3%. On Claude Code, MLExpert is 69.7%, and the base model is 42.4%.
• Other evaluations: On NatureBench Lite and Claude Code, Surpass-SOTA is 50.0% (5/10), and the base model is 0.0% (Table 2); Nex's Match-SOTA is 80.0%, higher than MLExpert's 70.0%. With GPT-4.1-nano fixed, the MLExpert-optimized scaffold has a held-out accuracy of 43.7% (Table 5). On MLS-Bench-Lite, the main text states average reward goes from 0.322 to 0.425, with 1 of 12 tasks lower.
• Stages: On the same harness, Any Medal goes from base 37.9, general SFT 47.0, MLE SFT 62.1, to WIRL 72.7 (Table 3). Trajectory-level GRPO and PPO-style are 56.1 and 60.6, lower than MLE SFT's 62.1 (Table 4).
• Authors' conclusion: The authors state these results support treating long-horizon optimization decisions as a trainable model capability, and believe this capability can transfer to scientific experiments, agent scaffolds, and algorithm component optimization.