arXiv · AI· Tan Yu, Alexander Bukharin, Khushi Bhardwaj, Jennifer Williams, Zirui Liu, Jonathan Lingjie Li, Soumye Singhal, Joseph Jennings, Sanjeev Satheesh, Yash Jain, Ashish Vaswani, Venkat Krishna Srinivasan, Matthew Papakipos, Hyunwoo Kim, Jian Zhang, Oleksii Kuchaiev, Markus Kliegl, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Jonathan Cohen, Jiantao Jiao·· 2026-10-08AI 评分33
基于基座模型轨迹预测后训练编码智能体性能
Before They Can Solve: Predicting Post-Training Coding-Agent Performance from Base Models
AI 导读
研究提出通过重放后训练轨迹中的“决定性步骤”来预测基座模型的编码潜力,无需从冷启动驱动测试框架。 构建三项评估指标:关键动作BPB、补丁MCQ和前缀条件pass@K。在10对公开模型上,指标排名与SWE-bench Verified pass@1高度一致,可应用于未来编码基准。
来源:arXiv · AI · arxiv.org