docs: remove stale files and apply inline feedback#86
Conversation
| Proxy ──────────────────────────────────────► Charon | ||
| │ POST /staging (resolve + open staging) (context resolution + storage) | ||
| │ PUT /staging/{id}/complete (store) | ||
| │ GET /responses/{id} (resolve chain) (context resolution + storage) |
There was a problem hiding this comment.
Two of the three Charon calls listed here do not exist as named. (1) GET /responses/{id} returns a single turn's blob (HandleRetrieve in internal/server/handlers.go:412); it does NOT resolve a chain — the chain walk lives in POST /staging?prev=.... Labelling it (resolve chain) here is misleading. (2) POST /responses/{id} is not a route — the buffered store is POST /responses (no path id), registered at internal/server/server.go:38 and bound to HandleBufferedStore. There is no Charon endpoint with a {response_id} path suffix anywhere. Suggest: drop the first line, change POST /responses/{id} to POST /responses, and reword the GET line to (point retrieve).
| - The inference server assigns the canonical response ID, returned in the first streaming chunk or response body. | ||
| - Proxy emits `response.created` to the client using that inference-server-assigned ID. | ||
| - If `store: true`: proxy uses the buffered `POST /responses` path (or the streaming staging protocol) to commit the response. | ||
| - If `store: true`: proxy calls `POST /responses/{canonical_id}` with the request and response blobs. |
There was a problem hiding this comment.
Two accuracy issues here. (1) "No Charon resolve call" is wrong — cmd/proxy/handler.go:92 unconditionally calls h.charon.Resolve(ctx, prevID, tenantKey, requestBlob) for every non-streaming turn, including new chains where prevID == "". What new chains skip is the prior-chain walk inside Resolve (the resolver still opens a staging record for the new turn). (2) "proxy calls POST /responses/{canonical_id}" — the endpoint is POST /responses (no path id); see comment at L139.
|
|
||
| **Continuation** (has `previous_response_id`) — streaming path: | ||
| **Continuation — buffered path** (`store: true`, no streaming): | ||
| 1. Proxy calls `GET /responses/{previous_response_id}` — Charon returns the resolved flat context. |
There was a problem hiding this comment.
This three-step GET-then-POST flow has no backing API. GET /responses/{id} returns a single turn's blob (no chain resolve, no flat context); there is no GET-returns-flat-context endpoint. The actual non-streaming continuation flow in the proxy is: POST /staging?prev=... (resolve+open staging), inference, then PUT /staging/{id}/chunks/0 + PUT /staging/{id}/complete reusing the staging handle. The "buffered POST" endpoint POST /responses accepts prev_id in its body and resolves+stores in one call, but the proxy in production never uses it (pkg/charon/client.go:118 notes it is for "direct root write in tests"). Recommend deleting this section or re-titling to "Continuation — non-streaming (via staging protocol)" with the actual call sequence.
| Charon exposes an internal HTTP API consumed only by the proxy. It is **not** required to conform to the OpenAI Responses API specification — it is designed for operational efficiency as an internal service. | ||
|
|
||
| ### Buffered store: `POST /responses` | ||
| ### Buffered store: `POST /responses/{id}` |
There was a problem hiding this comment.
Section heading says POST /responses/{id}; the actual endpoint is POST /responses. (internal/server/server.go:38: mux.HandleFunc("POST /responses", h.HandleBufferedStore).) Per pkg/charon/client.go:149, the SDK calls c.baseURL + "/responses". There is no {id} segment in this path anywhere in the codebase.
| POST /responses/{response_id} | ||
| body: { | ||
| "response_id": "resp_...", | ||
| "previous_response_id": "resp_..." | null, |
There was a problem hiding this comment.
Sample body shape is wrong in three places vs the actual request struct. From pkg/charon/client.go:135-145 and internal/server/handlers.go:103-108: the JSON keys are prev_id (NOT previous_response_id), response_id, request_blob, response_blob; there is no tenant_key field at all — tenant isolation is carried in the X-Tenant-Key request header (internal/server/handlers.go:123, set by the SDK at pkg/charon/client.go:154). Suggested body:
POST /responses
X-Tenant-Key: <tenant>
body: {
"prev_id": "resp_..." | <absent>,
"response_id": "resp_..." | <absent>,
"request_blob": "<base64>",
"response_blob": "<base64>"
}
The response_id belongs in the body (it may be omitted to let the server mint a UUID) — it is NOT in the URL.
| ``` | ||
| POST /responses body: { response_id, previous_response_id=null, request_blob, response_blob } | ||
| → 201 Created; node committed atomically | ||
| GET /responses/{prev_response_id} → Charon returns flat_context |
There was a problem hiding this comment.
This ID-flow diagram reuses the same API mistake as L74-77: it claims GET /responses/{prev_response_id} -> flat_context. No such endpoint exists. Also POST /responses/{canonical_id} is wrong (path should be /responses) and previous_response_id should be prev_id. If you want this diagram to reflect what the proxy actually does today, replace both lines with the staging-protocol sequence (same as the streaming flow above): POST /staging?prev=... to PUT /staging/{id}/chunks/0?response_id=... to PUT /staging/{id}/complete?response_id=...&total=1 — that is what cmd/proxy/handler.go:92,111 actually executes.
| POST /staging?prev={prevID} → staging_id, flat_context (resolve + open staging) | ||
| PUT /staging/{id}/chunks/{k} → next_expected (repeated per batch; out-of-order OK) | ||
| PUT /staging/{id}/complete?total={N} → committed (seal and write node atomically) | ||
| POST /responses/{canonical_id} body: { previous_response_id=null, request_blob, response_blob } |
There was a problem hiding this comment.
Path is POST /responses (no /{canonical_id} segment), and the body field is prev_id with omitempty — for a new chain you typically omit it entirely rather than setting it to null. Compare the actual struct: pkg/charon/client.go:135 json:"prev_id,omitempty".
| ## `store: false` Semantics | ||
|
|
||
| When the client sets `store: false`, the proxy skips the store call to Charon after inference. The response is never written to durable storage. | ||
| When the client sets `store: false`, the proxy skips the store call to Charon after inference. The response is never written to durable storage. When `store: false`, the proxy also skips the staging open call — there is no `/staging` interaction, only the prior-chain resolve (if a `previous_response_id` is present). |
There was a problem hiding this comment.
Backwards. cmd/proxy/handler.go:92 unconditionally calls h.charon.Resolve(ctx, prevID, ...) (= POST /staging?prev=...), which mints a staging_id and opens a staging record before the proxy knows whether the turn will be stored. What store: false actually skips is the Store call (line 109). The orphaned staging record left behind is reaped by the TTL reaper — fine, but it means the proxy always opens staging even when the turn is store: false, which is the opposite of what this paragraph says. Reword: "When store: false, the proxy skips the store call to Charon. The POST /staging call at resolve time still runs (so the resolved flat_context is returned), and any staging record left dangling is reaped by the TTL reaper."
| These are order-of-magnitude estimates for a single Charon instance on commodity server hardware (8–32 cores, NVMe SSD). They are not benchmarks; actual numbers depend on blob size, chain depth, and hardware. | ||
|
|
||
| **Write throughput — new response (`POST /responses` buffered path or `PUT /staging/{id}/complete`):** | ||
| **Write throughput — new response (`POST /responses/{id}` buffered path or `PUT /staging/{id}/complete`):** |
There was a problem hiding this comment.
Same endpoint-naming bug as above: this line and the resolve-latency line just below (L279) both reference POST /responses/{id} and GET /responses/{id} (full chain) — neither exists. Correct wording: "POST /responses buffered path" and remove the GET /responses/{id} note from the resolve-latency heading (chain resolution latency applies only to POST /staging?prev=...; GET /responses/{id} is a single-key point read with no chain walk).
Summary
docs/terminology.md,docs/implementation-plan.md, and all fourdocs/adr/files per inline feedback markersdocs/architecture.md: fix API interaction flows (new chain / buffered / streaming paths), add proxy multi-tenancy and tenant namespacing, drop redundant sections (KV cache principle, streaming ingest duplication), clarify deployment modes, add "Why two components?" rationale, fix response ID lifecycle to reflect staging ID vs. canonical ID correctlydocs/storage.md: add "Why a Key-Value Store?" section, add chain reconstruction strategy table (moved from architecture.md), remove low-level key schema table and Go struct/pseudocode, trimstore: falseduplicationTest Plan
go test ./...)<comment>markers remain in anydocs/fileadr/or deleted-file cross-references remainarchitecture.mdandstorage.mdtop-to-bottom to confirm consistencyNote on history rewrite
After this lands, the plan calls for
git filter-repoto remove the deleted files from all prior commits and a force-push to erase them from history entirely (Phase 6 ofthoughts/shared/plans/006_docs_cleanup.md).