fix(stream): 流中断时干净收尾,避免拼接错乱回复 - #92
Open
Sunsh1neY wants to merge 1 commit into
Open
Conversation
上游连接经代理中断(如 "[SSL] record layer failure")后,原逻辑会整段 重新生成并重试。由于已流式输出的内容无法撤回,重试要么因前缀校验失败 ("Gemini stream content changed during retry")导致流被静默丢弃——客户端 看到回复说一半就停;要么在重试内容恰好以已输出文本为前缀时,把两次不同 的生成拼接成一条前后矛盾的回复。 - generate_stream: 一旦有部分内容已输出,跳过重试并干净结束流,客户端收到 格式完整(finish_reason + [DONE])的响应,可自行请求续写 - _get_httpx_client: 禁用 keep-alive 连接复用(Connection: close),避免 复用已被代理/上游关闭的连接导致流中途 SSL 断裂 现有 18 个测试全部通过。
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
问题
通过 HTTP 代理访问上游时,流式响应偶发中途断开(日志表现为
[SSL] record layer failure,常见于代理复用/回收 keep-alive 连接后)。此时generate_stream的重试逻辑会整段重新生成,而已流式输出的内容无法撤回,导致两种用户可见的故障:Gemini stream content changed during retry,重试耗尽后流被静默丢弃(无 finish_reason、无结束标记);修复
generate_stream:一旦有部分内容已输出,跳过重试并干净结束流。客户端收到格式完整的响应(finish_reason: "stop"+data: [DONE]),可自行发起续写;_get_httpx_client:禁用 keep-alive 连接复用(Connection: close),避免复用已被代理/上游关闭的连接导致流中途 SSL 断裂。代价为每请求一次额外的 TLS 握手,对话场景无感。测试
pytest tests/ -q);Stream interrupted after partial output (N chars), ending cleanly,客户端拿到正常收尾的流;连续 3 次长回复流式请求均干净结束,不再出现重试拼接。