[Feature]: Codiv (api.codiv.ai) as a JEV-compatible decision provider with pacing, retry and fallback #7083

Activity

1. github-actions

   closed this as not plannedon Oct 13, 2026
2. github-actions commented on Oct 13, 2026

   

   github-actionsbot

   on Oct 13, 2026 – with GitHub Actions

   Last edited by github-actions

   Contributor

   More actions

   Issue reopened

   The report now contains the information required by the automated check. Thanks for updating it.
3. nordz0r

   changed the title [-]Add Codiv (api.codiv.ai) as a Jev-compatible decision provider with pacing, retry and fallback[/-] [+][Feature]: Codiv (api.codiv.ai) as a JEV-compatible decision provider with pacing, retry and fallback[/+] on Oct 13, 2026
4. github-actions

   added

   enhancementNew feature or requestNew feature or request

   proxyHTTP proxy, routing, reverse-proxy / management authHTTP proxy, routing, reverse-proxy / management auth

   on Oct 13, 2026
5. github-actions

   reopened this on Oct 13, 2026
6. nordz0r commented on Oct 13, 2026

   

   nordz0r

   on Oct 13, 2026 – with Cursor

   ContributorAuthor

   More actions

   Clarifying the fallback requirement (item 3):

   • Codiv must be usable either as the primary decision backend or as a fallback, in any order: jev (TypeSafe) followed by codiv, or codiv followed by jev.
   • Move to the next backend on a network error, timeout, 429, 529, 5xx, or an exhausted quota (insufficient_quota). Do this only after the per-provider retryOn429 budget is spent, and don't retry insufficient_quota against the same backend.
   • Each backend keeps its own model id (jev-latest / openjev-0.1), key, headers and pacing.
   • The request history should record which backend actually made the decision (e.g. jevDecision.backend, plus fallbackFrom when one was skipped).
   • After the last backend fails, keep today's fail-open behaviour (route to the first eligible target).
7. Ingwannu commented on Oct 16, 2026

   

   Ingwannu

   on Oct 16, 2026

   Owner

   More actions

   Current dev@4ae49360b7 already supports selecting one System One decision provider through decisionProvider, with destination-owned credentials, model, timeout and fail-open routing; see structure/providers/jev-decision.md. That is not the ordered multi-backend chain, per-backend pacing/retry or telemetry proposed here.

   A focused design should extend that decision-service boundary rather than retarget the canonical jev credential. Please specify the Codiv protocol/endpoint and authenticated capability evidence, credential ownership, a single total deadline and send budget across backends, cancellation semantics, and the exact fail-open result after all backends fail. Caller history and credentials must not become fallback payloads.

   Keeping this as needs-design; no canonical provider promotion or live request is approved by this triage.
8. Ingwannu

   added

   needs-design설계 필요. Implementation waits on a written design.설계 필요. Implementation waits on a written design.

   on Oct 16, 2026
9. nordz0r commented on Oct 16, 2026

   

   nordz0r

   on Oct 16, 2026

   Last edited by nordz0r

   ContributorAuthor

   More actions

   Design: ordered decision-backend chain (implements this proposal)

   

   

   (animated PR Lens diagrams: architecture — chain runner dispatching TypeSafe → Codiv with
   per-row pacing/retry/headers; data flow — fallback after a 429/529 budget exhaustion and the
   fail-open result after all backends fail)

   Design drafted for the four items in the issue. Summary below; it addresses the triage
   requirements: protocol/capability evidence, credential ownership, a single total deadline and
   send budget, cancellation semantics, and the exact fail-open result.

   Protocol verification (Codiv)

   • POST https://api.codiv.ai/v1/systemone is wire-compatible with TypeSafe JEV: same
     request/response fields, noul/choice/score answer shapes with legend/confidence, the same
     error shapes (422 validation lists; {"detail": {"error_type", "message"}} otherwise), and the
     x-typesafe-request-id header — per codiv.ai/docs/guides/jev-compatibility.md (checked
     2026-10-17). The existing question builder and answer parser therefore work unchanged: Codiv is
     just a jev-decision row with baseUrl: https://api.codiv.ai/v1/systemone. No new adapter, no
     registry entry.
   • Models: openjev-latest (canonical) or openjev-0.1 (pinned); jev-latest is accepted as an
     alias, so a row may even keep the default model.
   • criteria accepts {option: description-or-null}, so the self-hosted (descriptive-criteria)
     question path works as-is. Codiv extensions (images, steps, samples, think,
     sequential) are purely additive; opencodex never sends them, requests stay Jev-identical.

   1. Config shape

   {
     "providers": {
       "codiv": {
         "adapter": "jev-decision",
         "baseUrl": "https://api.codiv.ai/v1/systemone",
         "apiKey": "${CODIV_API_KEY}",
         "headers": { "User-Agent": "my-client/1.0" },
         "requestPacing": { "enabled": true, "requestsPerMinute": 250, "maxConcurrentRequests": 8 },
         "retryOn429": { "enabled": true, "attempts": 3, "intervalMs": 5000, "maxIntervalMs": 30000 }
       }
     },
     "combos": {
       "jev-auto": {
         "strategy": "jev",
         "decisionProviders": [
           { "provider": "jev" },
           { "provider": "codiv", "model": "openjev-0.1" }
         ],
         "decisionTimeoutMs": 4000,
         "targets": [ "..." ]
       }
     }
   }

   New optional combo field decisionProviders: [{ provider, model? }, …] (1–4 entries, no
   duplicate provider ids), mutually exclusive with decisionProvider / decisionModel. Every
   non-jev entry must pass exactly today's decisionProvider validation (adapter: "jev-decision", HTTPS System One endpoint or local HTTP /systemone, own-key ownership,
   disabled/model-less refusal under requireUsableDecisionService) — shared helper so the two
   boundaries cannot drift. Without the field, behavior is byte-identical to today (existing
   tests, including the TypeSafe request golden, stay authoritative).

   2. Chain semantics

   flowchart TD
       A[decision request] --> B{decisionModel?}
       B -- yes --> M[model backend, unchanged]
       B -- no --> C[chain: decisionProviders or implicit single]
       C --> E[exchange + pacing slot]
       E -->|apply| D[decision: backend + provider + fallbackFrom]
       E -->|no_choices / no_state / invalid| F[fail open, gate reported]
       E -->|missing_key| G{more backends?}
       G -- yes --> C
       G -- no --> F
       E -->|http 429/529| H[retryOn429 budget: Retry-After or backoff]
       H --> E
       H -->|budget spent| G
       E -->|network / timeout / redirect / malformed| G
   

   Loading

   Gate classification:

   • Question-shaped, terminal for the whole chain: no_choices, no_state, invalid —
     properties of the question, not the service; another backend cannot answer them.
   • Row-shaped, advance: missing_key — a disabled or keyless row is skipped, which is the
     point of the chain. On a degenerate single-entry chain this keeps today's missing_key
     fail-open result.
   • Service-shaped, retry then advance: http, network, timeout, redirect, malformed.
   • After the last backend: exactly today's fail-open — route to the first eligible target,
     with the last gate recorded.

   3. Retry, pacing, 529, headers

   • Per-backend retryOn429 (the existing policy block; presence = opt-in, defaults attempts=3 /
     intervalMs=5000 / maxIntervalMs=60000 / Retry-After respected) retries 429 and 529. The
     exchange gains an optional status on its http gate; the retry loop lives in a chain runner,
     not inside the exchange, so the exchange's load-bearing cancellation contract stays untouched.
     Waits honor Retry-After up to the 60 s ceiling, else capped exponential backoff; a wait is
     bounded by the remaining chain deadline and aborts with the caller's reason.
   • requestPacing (requestsPerMinute, maxConcurrentRequests, per-model overrides) applies to
     decision sends through the same slot primitives the inference path uses; an absent/disabled
     block is a zero-overhead no-op. A pacing-queue overload advances the chain (same as a 429),
     it does not fail the decision.
   • The row's headers are forwarded on decision requests (Codiv's edge requires a User-Agent;
     without it Cloudflare answers 403/1010). Credential-shaped configured headers
     (authorization, x-api-key, x-goog-api-key) are filtered before merge; the row's own
     apiKey remains the only credential, and Authorization/Content-Type are applied last.

   4. Deadline and send budget

   • One total deadline: decisionTimeoutMs (default 4 s, bounds 1000..120000 unchanged) budgets
     the whole chain; each backend's attempts draw from the remainder. A timeout gate is
     deadline expiry during that backend's turn.
   • Send clamp: at most 12 sends per decision regardless of configured attempt budgets — insurance
     against a 4×20-attempt config with a 120 s deadline.
   • Cancellation: the caller's signal wins at every existing checkpoint (pre-endpoint, pre-POST,
     post-response, parse); retry waits re-check it. Caller history and caller credentials never
     become fallback payloads — each backend sends only its own bounded question state.

   5. Telemetry

   • Request history records jevDecision.provider (the id of the backend that decided) and
     jevDecision.fallbackFrom (ids tried and skipped before it), alongside the existing
     jevDecision.backend. Ids only — no extracted state, no credentials; privacy:scan stays
     green. The per-backend usage aggregate is unchanged; per-provider detail lives in history.

   Out of scope here

   The data-plane passthrough route (item 4 of the issue) is a separate unit: a new authenticated
   ingress with its own admission scope and spend accounting. The chain runner is built
   question-agnostic so the passthrough can reuse it later.

   Related: #6268, #6384, #6665.