arXiv · AI· Vikash Singh, Weicong Chen, Debargha Ganguly, Yanyan Zhang, Mohsen Hariri, Sreehari Sankar, Wang Yang, Chaoda Song, Alexander Nemecek, Osama Zafar, Yuting Shao, Kai Ye, Biyao Zhang, Xinpeng Li, Shouren Wang, Erman Ayday, Jing Ma, Sanmukh Rao Kuppannagari, Vipin Chaudhary·· 9 天前AI 评分30
GRPO 中密集奖励是否物尽其用?匹配算力验证下的性能评估
Are Dense Rewards Worth Their Tokens in GRPO?
AI 导读
summary_zh: 密集奖励在匹配算力下未优于 GRPO,Qwen2.5-1.5B 上低 14.3 个百分点。 Monte-Carlo 奖励消耗 91% 生成 token;匹配算力后排序反转,且梯度测试高估收益 1-2 分。
来源:arXiv · AI · arxiv.org