arXiv · AI· Terry Yue Zhuo, Minh Chien Vu, Jenny Chim, Han Hu, Wenhao Yu, Ratnadira Widyasari, Imam Nur Bani Yusuf, Haolan Zhan, Junda He, Indraneil Paul, Simon Brunner, Chen Gong, Thong Hoang, Armel Randy Zebaze, Xiaoheng Hong, Wen-Ding Li, Jean Kaddour, Ming Xu, Zhihan Zhang, Prateek Yadav, Naman Jain, Alex Gu, Zhoujun Cheng, Jiawei Liu, Qian Liu, Zijian Wang, Binyuan Hui, Niklas Muennighoff, David Lo, Daniel Fried, Xiaoning Du, Harm de Vries, Leandro Von Werra·· 4 天前AI 评分23
BigCodeBench 发布:大模型函数调用得分最高仅 60%,远落后于人类 97% 表现
BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions
AI 导读
包含 1,140 个任务,覆盖 139 个库与 7 个领域,平均分支覆盖率 99%。 已获 ICLR 2025 Oral 录用,由 BigCode 社区构建。
来源:arXiv · AI · arxiv.org