Repository navigation
release: 0.2.5860 - #764
Conversation
semantic-index sends 100-item batches with eight requests in flight by default, cutting round trips for hosted index builds. Index builds now wait out hosted rate limits with a doubling backoff capped at 30 seconds instead of failing after a few quick retries. Interactive context calls still fail fast to local results.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c73cb97081
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // The hosted lane accepts at most 100 inputs per request, the provider's own | ||
| // batch ceiling. Larger batches cut per-item latency and round trips; query | ||
| // reranking stays bounded separately by `max_documents`. | ||
| pub const max_index_documents: usize = 100; |
There was a problem hiding this comment.
When a custom OpenAI-compatible endpoint uses 3072 or 4096 dimensions and the repository supplies a full batch, this raises the response to 100 JSON-float vectors while max_response_bytes remains a fixed 2 MiB; ordinary decimal embeddings can therefore overflow the response writer and make semantic-index fail, whereas the previous 25-item batches fit. Keep smaller batches for custom endpoints or calculate a safe batch size from dimensions and the response cap.
Useful? React with 👍 / 👎.
| err == error.EmbeddingTimeout; | ||
| if (!retryable or attempt + 1 == max_index_embedding_attempts) return err; | ||
| const delay_ms = index_embedding_retry_base_ms << @intCast(attempt); | ||
| const delay_ms = indexRetryDelayMs(attempt); |
There was a problem hiding this comment.
When the hosted lane is already returning 429, 5xx, or timeout responses as a build starts, the new retry loop is never reached because src/semantic_index.zig:714 still performs the required calibration through embedRemoteTexts directly. The entire index build consequently fails before its first chunk batch despite the advertised bounded backoff; route the calibration request through the same index-build retry path.
Useful? React with 👍 / 👎.
1 participant
semantic-indexsends 100-item batches, eight in flight by default (CODEDB_SEMANTIC_INDEX_CONCURRENCYstill overrides, 1-8).contextcalls are unchanged.Verification:
zig build test --summary all.