arXiv · NLP· Zheyuan Deng, Taoran Li, Jingwen Zhang, Yuante Li, Dachuan Zhao, Fulei Zhang, Yunkai Zhang, Zhikun Xu, Yajie Liao, Haichao Zhang, Haodi Hu, Cong Wang, Chen Yang, Siyang Wu, Defu Cao, Xu Hu, Haoming Li, Haixin Wang, Robert George, Weizhi Zhang, Yuchu Wang, Zhejian Peng, Yueying Li, Wanting Jiang, Shushu Liang, Zhiyuan Yu, Yifan Wang, Jintao Huang, Yijun Wang, Yuexing Hao, Xiaomin Li·· 2 小时前AI 评分38
PersonaChatBench：基于多样化模拟用户评估聊天助手
PersonaChatBench: Evaluating Chat Assistants with Diverse Simulated Users
AI 导读
summary_zh: 研究团队推出 PersonaChatBench 框架，通过固定任务搭配不同“私人物设卡”模拟用户，以评估聊天助手对多样化需求的响应。 该框架在六个领域测试了 20 个语言模型，结果显示模型任务表现各异，且强任务性能未必能良好满足个体特定需求。
来源：arXiv · NLP · arxiv.org