Skip to content

fix(cookie): support Netscape cookies.txt and auto-refresh XSRF token - #88

Open
usmanovbf wants to merge 17 commits into
Sophomoresty:mainfrom
usmanovbf:fix/cookie-file-xsrf-refresh
Open

fix(cookie): support Netscape cookies.txt and auto-refresh XSRF token#88
usmanovbf wants to merge 17 commits into
Sophomoresty:mainfrom
usmanovbf:fix/cookie-file-xsrf-refresh

Conversation

@usmanovbf

@usmanovbf usmanovbf commented Aug 24, 2026 •
edited
Loading

Copy link
Copy Markdown

Summary

  • Parse Netscape-format cookies.txt directly in load_cookie() (single-file and modular) - no manual conversion
  • Fetch the XSRF token (SNlM0e) from the app page at startup and auto-refresh BL + XSRF on HTTP 400/405 retries, so cookie sessions survive token rotation without a restart
  • Detect the new JSPB BardErrorInfo",[code] error envelope (old format still matched) and surface known codes as clear messages instead of silent content: null
  • Drop stale 200/50-char length heuristics that discarded valid short wrb.fr payloads
  • SSE streams end with a finish chunk + [DONE] on upstream failure, so clients never hang
  • Fail fast on hard rejections (1060 IP block, 1037 quota) but keep retries for transient 1013
  • Return 400 (not 500) for invalid JSON request bodies; harden the cookie cache key with file size
  • Port the extra-fields model mechanism (gemini-3.1-pro-enhanced) to the single-file build - both builds now list the same 9 models
  • Add probe_upstream.py (one-shot reachability check) and README/README_CN updates

Problem

Running with a Netscape cookie file failed twice over:

  1. load_cookie() only understood JSON {"cookie": ...} or a raw k=v; k2=v2 string, so the whole tab-separated file was sent upstream as the Cookie header -> Invalid header value.
  2. Authenticated StreamGenerate now requires the XSRF token (at = SNlM0e from the app page). The code could send it but never fetched it -> HTTP 400 with an xsrf error body.
  3. Long-running servers died on token rotation until restart.
  4. Google changed the error envelope to JSPB; the old regex missed it, so upstream rejections (e.g. 1060 IP block) returned HTTP 200 with content: null instead of an error.

Testing

  • 23/23 unit + live-server integration tests (python -m unittest discover -s tests), including new coverage for both error formats, short payloads, SSE error finish chunks and the mocked success path
  • End-to-end against live Gemini with a real cookies.txt: non-streaming and streaming, single-file and modular builds (successful generations verified; later runs hit upstream 1060 from the test machine and were verified to report it correctly - HTTP 502 with a clear message, [error] finish chunk for streams)
  • Simulated token expiry by corrupting xsrf_token mid-session: both builds auto-recovered in ~4s

Bu added 17 commits August 25, 2026 00:21
Bu Bu
fix(cookie): support Netscape cookies.txt and auto-refresh XSRF token
c343aaf 
- Parse Netscape-format cookie files (tab-separated cookies.txt) instead
  of sending raw file contents as the Cookie header
- Extract SNlM0e XSRF token from the app page at startup when cookies
  are configured; required by StreamGenerate for authenticated requests
- On HTTP 400/405, refresh BL and XSRF token mid-retry and rebuild the
  request so long-running servers survive token rotation
- Sync modular package (refresh_bl_and_xsrf) with single-file script
Bu Bu
fix(upstream): detect new BardErrorInfo format and surface clear errors
ad15403 
- Google changed the upstream error envelope from 'BardErrorInfo [code]'
  to JSPB 'application.BardErrorInfo",[code]'; the old regex missed it,
  so rejected requests returned HTTP 200 with content:null instead of an
  error
- Map known codes to human-readable hints (1060 = IP temporarily blocked
  / unsupported region, 1037 = usage limit, 1013 = transient)
- Drop the 200-char line-length and 50-char payload heuristics that
  silently skipped valid short wrb.fr payloads
- Streaming SSE responses now end with a finish chunk on error so
  clients do not hang on dropped streams
- Add tests/test_upstream_errors.py covering both error formats and the
  short-line regression
Bu Bu
chore: gitignore Netscape cookies.txt and config backup
98a36ca
Bu Bu
fix(upstream): fail fast on hard BardErrorInfo rejections in streams
f5ea291 
StreamGenerate streaming paths retried BardErrorInfo rejections three
times pointlessly - an IP block (1060) or quota rejection (1037) does
not clear within a retry loop. Raise immediately instead; connection
and transient errors keep their retry behaviour. 1060 responses now
return in ~1s instead of after the full retry cycle.
Bu Bu
docs(upstream): add recovery guidance to IP block error message
0af20f7
Bu Bu
fix(server): return 400 for invalid JSON request bodies
962bcf9 
Malformed JSON fell through the generic handler and produced HTTP 500;
clients cannot distinguish their own bad request from a server fault.
Matches the modular package, which already answers 400 invalid JSON.
Bu Bu
test: cover SSE error finish chunk on upstream failure
c349471 
Live-server test that a failing generate_stream (BardErrorInfo 1060)
yields the partial content, an [error] finish chunk and [DONE], so
clients terminate instead of hanging on a dropped stream.
Bu Bu
test: add mocked success-path integration test
63db2ed 
Feed server.generate() a Russian answer and assert the full OpenAI
completion shape survives JSON encoding end to end.
Bu Bu
docs: Netscape cookies.txt works directly, no conversion needed
f722b88
Bu Bu
fix(cookie): include file size in cookie cache key
ad820bc 
mtime-only caching can miss a same-second rewrite on filesystems with
1s timestamp resolution; (mtime, size) catches content changes in one
write.
Bu Bu
feat: add probe_upstream.py - one-shot upstream reachability check
7bd4036 
Anonymous StreamGenerate probe; prints blocked/UNBLOCKED and exits
nonzero while the IP is blocked. Useful to tell 'Google blocked this
IP' apart from a proxy regression before debugging code.
Bu Bu
docs: mention probe_upstream.py in rate-limit troubleshooting
ea35e0b
Bu Bu
feat(models): port gemini-3.1-pro-enhanced to single-file build
ebec871 
The modular package gained an extra-fields mechanism (inner[31]/inner[80]
payload overrides for the enhanced Pro variant) that the single-file
script never received, leaving the two builds listing different models
(8 vs 9). Thread extra_fields through resolve/generate/stream paths and
widen the payload array to 102 slots, matching the package.
Bu Bu
docs: sync model tables with current model list (9 models)
78602b9
Bu Bu
fix(upstream): keep retries for transient BardErrorInfo 1013
ad4da2e 
The fail-fast guard stopped retrying every BardErrorInfo, but 1013 is
documented as transient and clears on retry; only hard rejections
(1060 IP block, 1037 quota) skip the retry loop.
Bu Bu
fix(upstream): fail fast on HTTP 429 instead of retrying
397b5a7 
Retrying an IP rate limit amplifies it. Surface 429 immediately with
guidance so clients back off, matching the fail-fast behavior used for
hard BardErrorInfo rejections.
Bu Bu
fix(upstream): recognize error code 1185 in live traffic
28b44dc
@kirigayakazima kirigayakazima mentioned this pull request Sep 5, 2026
Open
luxizai pushed a commit to luxizai/gemini-web2api that referenced this pull request Sep 11, 2026
@luler
Merge PR Sophomoresty#88: fix(cookie): support Netscape cookies.txt a…
cc9660e 
…nd auto-refresh XSRF token
mucsbr pushed a commit to mucsbr/gemini-web2api that referenced this pull request Sep 29, 2026
@beingSbr
fix(cookie): 支持 Netscape cookies.txt 并自动刷新 XSRF/BL (Sophomoresty#88)
24eee95 
- load_cookie 直接解析 Netscape 格式 cookies.txt(含 #HttpOnly_ 前缀),
  无需手动转换;包版缓存键升级为 (mtime, size)
- 新增 refresh_bl_and_xsrf():带 cookie 抓取应用页面,刷新 SNlM0e
  XSRF token 与 gemini_bl;启动时及 HTTP 400/405 重试时自动执行
  (body/headers 相应移入重试循环内构建)
- HTTP 429 立即失败不再重试(立即重试会延长封锁)
- BardErrorInfo 兼容新版 JSPB 信封格式,已知错误码给出明确提示
  (1060 IP 封锁/1037 限额/1013 瞬时/1185 拒绝);硬拒绝不再重试
  (1013 瞬时错误除外)
- SSE 流异常时补发 finish chunk + [DONE],客户端不再悬挂
- 新增 probe_upstream.py 诊断脚本与 tests/test_upstream_errors.py

与 Sophomoresty#87/Sophomoresty#92/Sophomoresty#99 的调和:保留 Sophomoresty#87 严格解析与 _next_reqid、Sophomoresty#92 干净收尾、
Sophomoresty#99 cookie 文件字段同步;Sophomoresty#88 测试中对旧解析函数的引用改写为等价的
extract_response_text 断言。

Original PR by @usmanovbf
mucsbr pushed a commit to mucsbr/gemini-web2api that referenced this pull request Sep 29, 2026
@beingSbr
feat: OpenAI 兼容流式 tool_calls、宽容工具解析与 ticket 模型路由 (Sophomoresty#100)
caad458 
Sophomoresty#96 的超集,作者实测验证:

- 流式 tool_calls 按 OpenAI 规范发 chunk(必需的 index 字段 + 参数分片
  + tool_calls finish),AI SDK 类客户端可正确组装调用
- parse_tool_calls 宽容解析 5 种实际输出格式(tool_call/function_call/
  json 围栏、[tool_call: ...] 括号速记、裸 JSON 对象),非法围栏不动,
  未声明的工具名被过滤
- 模型路由修复(Issue Sophomoresty#82):slot79/80 仅在携带 X-Goog-Ext-525001261-Jspb
  ticket header 时生效,否则上游回落到账号默认模型。新增 model_tickets
  配置 + ticket_for() + upstream_echo/check_routing 路由回显校验,
  ticket 失效时打印告警
- 模型列表对齐 Web UI 实测路由:6 个模型按 (family, variant) 二元组定义,
  inner[80] 变体字段随请求发送;3.x 点版本由服务端家族默认决定,
  移除无法区分的 3.7/3.8/auto/flash-lite 别名(未知模型名仍回落默认)
- refresh_auth():SNlM0e 数分钟轮转,cookie 会话下自动重取并回写
  cookie JSON 文件;400-xsrf 错误精确识别
- 新增 AGENTS.md 镜像规则文档、tests/test_openai_compat.py,
  test_modular_sync.py 重写(家族/变体映射、ticket、回显解析)

与 Sophomoresty#87/Sophomoresty#88/Sophomoresty#92/Sophomoresty#99 的调和:保留严格解析、_reqid 递增、干净收尾、
Netscape/429/错误码处理与 per-attempt 重建;refresh_bl_and_xsrf 重构
为复用 _extract_auth_from_html;修复原 PR 中 dict|None 联合类型语法
以保持 Python 3.8 兼容。

Original PR by @vchieu (含 @imtiyazakiwat Sophomoresty#96 的工作)
mucsbr pushed a commit to mucsbr/gemini-web2api that referenced this pull request Sep 29, 2026
@beingSbr
feat(chat): 流式转发带工具请求的正文，tool_call 围栏缓冲解析 (Sophomoresty#93)
7c7e188 
带工具的流式请求此前被降级为非流式,agent 类客户端(每轮都带工具
定义)永远看不到流式效果;且生成中断导致围栏未闭合时,旧解析会静默
丢弃整个工具调用,agent 下一轮重试再失败形成死循环。

新增第三分支处理 tools+stream:正文 delta 实时转发;检测到
```tool_call 围栏开始标记后停止转发转入缓冲(标记可能跨 delta
边界,常规转发保留末尾 len(marker)-1 字符);流结束后统一解析,
工具调用按 OpenAI 规范以带 index 的 tool_calls delta 发出并以
tool_calls finish 收尾;围栏残缺时原文转发不静默丢弃。

与 Sophomoresty#100 的调和:本分支取代其「生成完毕后一次性发 tool_calls」的
路径,_stream_tool_calls/stream_tool_calls 随之移除;tool_calls
过滤采用 tool_names;错误收尾沿用 Sophomoresty#88 的 [error] chunk。
test_openai_compat 两个流式测试相应改为 mock generate_stream。

Original PR by @Sunsh1neY
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@usmanovbf