跳到正文
arXiv · ML· Christian Moya, Purav Matlia, Elliott Thornley, Guang Lin·· 24 天前AI 评分25

Selective Tie Learning: Mitigating Spurious Correlations in Preference Optimization

Selective Tie Learning: Mitigating Spurious Correlations in Preference Optimization

AI 导读

偏好优化常会学到虚假相关,引入“同分”(ties)数据可减少此类问题。该文提出基于隐式奖励幅度筛选同分样本的方法,在逻辑线性策略、神经模型及大语言模型中均有效降低对虚假特征的依赖。

来源:arXiv · ML · arxiv.org