Skip to content

docs: remove stale files and apply inline feedback#86

Open
elevran wants to merge 1 commit into
mainfrom
docs-cleanup
Open

docs: remove stale files and apply inline feedback#86
elevran wants to merge 1 commit into
mainfrom
docs-cleanup

Conversation

@elevran

@elevran elevran commented Jul 8, 2026

Copy link
Copy Markdown
Owner

Summary

  • Delete docs/terminology.md, docs/implementation-plan.md, and all four docs/adr/ files per inline feedback markers
  • Rewrite docs/architecture.md: fix API interaction flows (new chain / buffered / streaming paths), add proxy multi-tenancy and tenant namespacing, drop redundant sections (KV cache principle, streaming ingest duplication), clarify deployment modes, add "Why two components?" rationale, fix response ID lifecycle to reflect staging ID vs. canonical ID correctly
  • Rewrite docs/storage.md: add "Why a Key-Value Store?" section, add chain reconstruction strategy table (moved from architecture.md), remove low-level key schema table and Go struct/pseudocode, trim store: false duplication

Test Plan

  • All Go tests pass (go test ./...)
  • No <comment> markers remain in any docs/ file
  • No adr/ or deleted-file cross-references remain
  • Read architecture.md and storage.md top-to-bottom to confirm consistency

Note on history rewrite

After this lands, the plan calls for git filter-repo to remove the deleted files from all prior commits and a force-push to erase them from history entirely (Phase 6 of thoughts/shared/plans/006_docs_cleanup.md).

@elevran elevran left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Submitted.

Comment thread docs/architecture.md
Proxy ──────────────────────────────────────► Charon
│ POST /staging (resolve + open staging) (context resolution + storage)
│ PUT /staging/{id}/complete (store)
│ GET /responses/{id} (resolve chain) (context resolution + storage)

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two of the three Charon calls listed here do not exist as named. (1) GET /responses/{id} returns a single turn's blob (HandleRetrieve in internal/server/handlers.go:412); it does NOT resolve a chain — the chain walk lives in POST /staging?prev=.... Labelling it (resolve chain) here is misleading. (2) POST /responses/{id} is not a route — the buffered store is POST /responses (no path id), registered at internal/server/server.go:38 and bound to HandleBufferedStore. There is no Charon endpoint with a {response_id} path suffix anywhere. Suggest: drop the first line, change POST /responses/{id} to POST /responses, and reword the GET line to (point retrieve).

Comment thread docs/architecture.md
- The inference server assigns the canonical response ID, returned in the first streaming chunk or response body.
- Proxy emits `response.created` to the client using that inference-server-assigned ID.
- If `store: true`: proxy uses the buffered `POST /responses` path (or the streaming staging protocol) to commit the response.
- If `store: true`: proxy calls `POST /responses/{canonical_id}` with the request and response blobs.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two accuracy issues here. (1) "No Charon resolve call" is wrong — cmd/proxy/handler.go:92 unconditionally calls h.charon.Resolve(ctx, prevID, tenantKey, requestBlob) for every non-streaming turn, including new chains where prevID == "". What new chains skip is the prior-chain walk inside Resolve (the resolver still opens a staging record for the new turn). (2) "proxy calls POST /responses/{canonical_id}" — the endpoint is POST /responses (no path id); see comment at L139.

Comment thread docs/architecture.md

**Continuation** (has `previous_response_id`) — streaming path:
**Continuation — buffered path** (`store: true`, no streaming):
1. Proxy calls `GET /responses/{previous_response_id}` — Charon returns the resolved flat context.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This three-step GET-then-POST flow has no backing API. GET /responses/{id} returns a single turn's blob (no chain resolve, no flat context); there is no GET-returns-flat-context endpoint. The actual non-streaming continuation flow in the proxy is: POST /staging?prev=... (resolve+open staging), inference, then PUT /staging/{id}/chunks/0 + PUT /staging/{id}/complete reusing the staging handle. The "buffered POST" endpoint POST /responses accepts prev_id in its body and resolves+stores in one call, but the proxy in production never uses it (pkg/charon/client.go:118 notes it is for "direct root write in tests"). Recommend deleting this section or re-titling to "Continuation — non-streaming (via staging protocol)" with the actual call sequence.

Comment thread docs/architecture.md
Charon exposes an internal HTTP API consumed only by the proxy. It is **not** required to conform to the OpenAI Responses API specification — it is designed for operational efficiency as an internal service.

### Buffered store: `POST /responses`
### Buffered store: `POST /responses/{id}`

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Section heading says POST /responses/{id}; the actual endpoint is POST /responses. (internal/server/server.go:38: mux.HandleFunc("POST /responses", h.HandleBufferedStore).) Per pkg/charon/client.go:149, the SDK calls c.baseURL + "/responses". There is no {id} segment in this path anywhere in the codebase.

Comment thread docs/architecture.md
POST /responses/{response_id}
body: {
"response_id": "resp_...",
"previous_response_id": "resp_..." | null,

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sample body shape is wrong in three places vs the actual request struct. From pkg/charon/client.go:135-145 and internal/server/handlers.go:103-108: the JSON keys are prev_id (NOT previous_response_id), response_id, request_blob, response_blob; there is no tenant_key field at all — tenant isolation is carried in the X-Tenant-Key request header (internal/server/handlers.go:123, set by the SDK at pkg/charon/client.go:154). Suggested body:

POST /responses
X-Tenant-Key: <tenant>
body: {
  "prev_id":       "resp_..." | <absent>,
  "response_id":   "resp_..." | <absent>,
  "request_blob":  "<base64>",
  "response_blob": "<base64>"
}

The response_id belongs in the body (it may be omitted to let the server mint a UUID) — it is NOT in the URL.

Comment thread docs/architecture.md
```
POST /responses body: { response_id, previous_response_id=null, request_blob, response_blob }
→ 201 Created; node committed atomically
GET /responses/{prev_response_id} → Charon returns flat_context

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This ID-flow diagram reuses the same API mistake as L74-77: it claims GET /responses/{prev_response_id} -> flat_context. No such endpoint exists. Also POST /responses/{canonical_id} is wrong (path should be /responses) and previous_response_id should be prev_id. If you want this diagram to reflect what the proxy actually does today, replace both lines with the staging-protocol sequence (same as the streaming flow above): POST /staging?prev=... to PUT /staging/{id}/chunks/0?response_id=... to PUT /staging/{id}/complete?response_id=...&total=1 — that is what cmd/proxy/handler.go:92,111 actually executes.

Comment thread docs/architecture.md
POST /staging?prev={prevID} → staging_id, flat_context (resolve + open staging)
PUT /staging/{id}/chunks/{k} → next_expected (repeated per batch; out-of-order OK)
PUT /staging/{id}/complete?total={N} → committed (seal and write node atomically)
POST /responses/{canonical_id} body: { previous_response_id=null, request_blob, response_blob }

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Path is POST /responses (no /{canonical_id} segment), and the body field is prev_id with omitempty — for a new chain you typically omit it entirely rather than setting it to null. Compare the actual struct: pkg/charon/client.go:135 json:"prev_id,omitempty".

Comment thread docs/architecture.md
## `store: false` Semantics

When the client sets `store: false`, the proxy skips the store call to Charon after inference. The response is never written to durable storage.
When the client sets `store: false`, the proxy skips the store call to Charon after inference. The response is never written to durable storage. When `store: false`, the proxy also skips the staging open call — there is no `/staging` interaction, only the prior-chain resolve (if a `previous_response_id` is present).

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Backwards. cmd/proxy/handler.go:92 unconditionally calls h.charon.Resolve(ctx, prevID, ...) (= POST /staging?prev=...), which mints a staging_id and opens a staging record before the proxy knows whether the turn will be stored. What store: false actually skips is the Store call (line 109). The orphaned staging record left behind is reaped by the TTL reaper — fine, but it means the proxy always opens staging even when the turn is store: false, which is the opposite of what this paragraph says. Reword: "When store: false, the proxy skips the store call to Charon. The POST /staging call at resolve time still runs (so the resolved flat_context is returned), and any staging record left dangling is reaped by the TTL reaper."

Comment thread docs/architecture.md
These are order-of-magnitude estimates for a single Charon instance on commodity server hardware (8–32 cores, NVMe SSD). They are not benchmarks; actual numbers depend on blob size, chain depth, and hardware.

**Write throughput — new response (`POST /responses` buffered path or `PUT /staging/{id}/complete`):**
**Write throughput — new response (`POST /responses/{id}` buffered path or `PUT /staging/{id}/complete`):**

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same endpoint-naming bug as above: this line and the resolve-latency line just below (L279) both reference POST /responses/{id} and GET /responses/{id} (full chain) — neither exists. Correct wording: "POST /responses buffered path" and remove the GET /responses/{id} note from the resolve-latency heading (chain resolution latency applies only to POST /staging?prev=...; GET /responses/{id} is a single-key point read with no chain walk).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant