arXiv · AI· Yue He, Jiahui Hu, Shaman Tang, Hua Zhou, Qiman Wu, Menglin Li, Yizhao Wang, Yuheng Zhang, Da Zhu, Zhou Wang, Anhua Liu, Hanlin Chen, Mingyuan Wang, Rui Xin, Yuhua Wei, Shuang Su, Siqi Liu, Kai Yang, Weichao Huang, Hang Zhang, Honggou Yang, Xianming Liu·· 21 天前AI 评分32
Turing LLM 后训练:校准蒸馏与高效动态 MoE 执行
Turing LLM Post-Training: Calibrated Distillation and Efficient Dynamic-MoE Execution
AI 导读
论文提出针对动态 MoE 模型的后训练框架,结合校准多教师在线策略蒸馏(C-MOPD)与高效执行路径,在 Turing-20B-A2B 上实现高分专家整合。
来源:arXiv · AI · arxiv.org