arXiv · ML· Qiuyang Mang, Wenhao Chai, Runyuan He, Dongruixuan Li, Zihan Zheng, Yujun He, Xuanyi Zhou, Hao Wang, Siddhant Ray, Bo Peng, Ruyi Ji, Shuo Yang, Haocheng Xi, Boru Chen, Xinyang David Han, Melissa Pan, Mayank Mishra, Boxi Yu, Sijia Liu, Kerem Akillioglu, Yi Pan, Zirui Wang, Mert Cemri, Diogo Antunes, Yichuan Wang, Aiwei Ivy Zhang, Zhenyang Chen, Qizheng Zhang, Kaiyuan Liu, Wenyuan Huang, Sherry Ruan, Shu Liu, Dacheng Li, Zerui Cheng, Zeyu Shen, Shang Zhou, Hanfei Chen, Yuzhong Zhang, Bohan Lyu, Qu Ao, Han Zheng, Chonghe Jiang, Jingzhuo Hu, Xiaokun Chen, Kunjun Li, Zhiqing Zhong, Haoye Tian, Tong Zhao, Yunzhong He, Hanchen Li, Ofir Press, Jiantao Jiao, Kunle Olukotun, Karthik R. Narasimhan, Jingbo Shang, Koushik Sen, Matei Zaharia, Alex Dimakis, Ion Stoica, Joseph E. Gonzalez, Alvin Cheung·· 29 天前AI 评分54
FrontierCS 2:面向计算机科学与研发的大模型Agent评测基准
FrontierCS 2: A Benchmark for Computer Science R&D
AI 导读
研究团队发布了FrontierCS 2,这是一个包含48个开放性任务的评测基准,用于评估LLM Agent在计算机科学与研发中的贡献能力。该基准要求Agent在现有研究代码仓库中提交补丁,最终通过保留的测试集进行评估,任务涵盖机器学习系统、高性能计算、数据库、安全与机器人等13个领域。
来源:arXiv · ML · arxiv.org