跳到正文
arXiv · AI· Terry Yue Zhuo, Xiaolong Jin, Hange Liu, Juyong Jiang, Tianyang Liu, Chen Gong, Bhupesh Bishnoi, Vaisakhi Mishra, Marek Suppa, Noah Ziems, Saiteja Utpala, Ming Xu, Guangyu Song, Kaixin Li, Yuhan Cao, Bo Liu, Zheng Liu, Sabina Abdurakhmanova, Wenhao Yu, Mengzhao Jia, Jihan Yao, Kenneth Hamilton, Kumar Shridhar, Minh Chien Vu, Dingmin Wang, Jiawei Liu, Zijian Wang, Qian Liu, Binyuan Hui, Meg Risdal, Ahsen Khaliq, Atin Sood, Zhenchang Xing, Wasi Uddin Ahmad, John Grundy, David Lo, Banghua Zhu, Xiaoning Du, Torsten Scholak, Leandro von Werra·· 4 天前AI 评分48

BigCodeArena 如何通过执行揭示代码生成中更可靠的人类偏好

BigCodeArena: Unveiling More Reliable Human Preferences in Code Generation via Execution

AI 导读

开源平台 BigCodeArena 通过实时执行环境评估 LLM 代码生成,发现 4,700 个多轮样本中人类偏好在细粒度领域存在未被探索的模式。 平台收集了 10 种语言、8 类执行环境下 14,000 个原始会话,并据此构建了 BigCodeReward 和 AutoCodeArena 两个基准。

来源:arXiv · AI · arxiv.org