Skip to content

fix: classify non-standard embedding 5xx as unavailable - #761

Draft
justrach wants to merge 1 commit into
release/0.2.5856from
fix/semantic-provider-5xx-classification
Draft

justrach wants to merge 1 commit into
release/0.2.5856from
fix/semantic-provider-5xx-classification

Conversation

@justrach

@justrach justrach commented Sep 26, 2026 •

Copy link
Copy Markdown
Owner

Draft — do not merge yet. A later clean exact-head full-suite run hit the unchanged semantic retry test's EmbeddingTimeout flake, which also reproduces on the release base. PR-ready preflight has not accepted the available local test evidence. This classification fix does not restore the hosted provider.

What changed

Classify every numeric HTTP 5xx response from the embedding provider as EmbeddingProviderUnavailable, including non-standard codes such as 530 that Zig's status enum does not name. Keep HTTP 429 as EmbeddingRateLimited and 4xx responses as EmbeddingProviderRejected. Add mock HTTP coverage for 403 and 530 alongside the existing 429 and 500 cases.

Why

A disposable, device-enrolled synthetic semantic-index run received HTTP 530 on signed embedding requests. The previous switch listed only standard 5xx enum members and put 530 in its default EmbeddingProviderRejected branch, misdiagnosing a server-side status as a client rejection. Numeric 500–599 classification follows HTTP status classes without special-casing one provider code. The trade-off is treating any provider-specific 5xx as unavailable even if that provider meant it to be permanent; 4xx and 429 retain their existing classes. Treating 530 as a compact-transport negotiation error was rejected because a legacy retry also received 530 before this change.

This fixes classification, not the hosted service. No model, ranking weight, ANN candidate breadth, local-CPU embedding, credential, or telemetry policy changes.

Verification — exact head 7063ca9f681fe00673528ffa6f13ff71cec3abe2

  • Local: zig build test-semantic -Dtest-filter='provider 4xx and 5xx responses fail closed with retryable classes' --summary all --cache-dir .graff/verify/pr761-direct-cache — 1/1 passed in a fresh, clean exact-head run after PR creation.
  • Local (first clean exact-head run): zig build test --summary all — 33/33 steps succeeded; 1,621 tests passed and 13 skipped.
  • Local (later direct fresh-cache run): zig build test --summary all --cache-dir .graff/verify/pr761-direct-cache — 31/33 steps succeeded; 1,620 passed, 13 skipped, one EmbeddingTimeout in the unchanged explicit-index retry test. The same test failed on clean release/0.2.5856 at 565238036b7a2e29dc13972a45f5d59910d3fbfd; do not treat the full suite as consistently green.
  • Diagnostic, not acceptance: zig build -Doptimize=ReleaseFast succeeded. A copied candidate (SHA-256 80b97bffb9aaa42ef41f7c664f9833b44277af7c5406a109873557b99907e389) ran isolated signed semantic-index with telemetry disabled: it exited with EmbeddingProviderUnavailable, not EmbeddingProviderRejected or a panic. The provider is still unhealthy; this is not a live-retrieval pass.
  • Local: python3 scripts/e2e_mcp_test.py --binary .graff/verify/hosted-probe/codedb-5xx-candidate --project "$PWD" — 76/76 passed on this commit, including sensitive-path and semantic-boundary cases.
  • Remote: exact-head bench passed (8m49s) and macos-resource-regression passed (3m50s); guard skipped while draft. The PR-ready preflight has not accepted the local check receipt/test reachability, so no ready-state write occurred.

Do not tag, sign, or publish a release from this PR alone: the hosted provider still needs a successful live retrieval check, and separate release gates remain open.

A signed synthetic hosted request returned HTTP 530. The previous enum-only status switch treated that server-side failure as a client rejection, so explicit index builds skipped their bounded retry budget and diagnostics blamed the client. Classify the numeric 500–599 range as provider unavailable while preserving 429 and 4xx handling; add HTTP-mock coverage for 403 and 530. This does not restore the hosted service or alter ranking or retrieval breadth.

Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant