arXiv · AI· Shuyang Xiang·· 10 小时前AI 评分35
A Conditional No-Hair Theorem for Transformers: How Language Models Forget Their Prompt
A Conditional No-Hair Theorem for Transformers: How Language Models Forget Their Prompt
AI 导读
研究指出，自回归 transformer 中 prompt 对 hidden states 的影响随生成 token 数 $t$ 呈幂律衰减 $t^{-\alpha}$。
来源：arXiv · AI · arxiv.org