R/ArtificialInteligence· /u/No-Peanut-6988·· 2026-10-14AI 评分46
vLLM、SGLang 等 LLM 服务在用户停止读取后仍继续生成并计费的研究
Research: Does your LLM app keep paying for tokens after the user hits stop? Measured cancelled streams on vLLM, SGLang, Bedrock and OpenRouter
AI 导读
研究实测显示,vLLM 0.11.0 与 SGLang 0.5.21 仅在关闭 HTTP 连接后才停止生成;若客户端仅停止读取而不关闭连接,两个引擎会继续生成剩余答案,误差在 4% 以内。
来源:R/ArtificialInteligence · reddit.com