Skip to content

Preserve uncertainty in context limits #23

Open

Description

@ehrvs

Problem

When PAIR omits context_length (uncertain capacity), the client substitutes /api/show metadata or a 32,768 fallback, which may exceed the actual runtime capacity.

Evidence

  • codex-rs/local/models/src/lib.rs:46-47 FALLBACK_CONTEXT_WINDOW = 32_768; lib.rs:104 calls /api/show for Ollama-shaped entries lacking context_length; v1.rs:40 parses context_length as optional.

Proposal

Preserve the uncertainty: use an explicit safe limit or validated runtime capacity instead of the metadata/32k substitute when PAIR omits the field.

Acceptance criteria

  • Test: PAIR omits context_length while the loaded window is smaller than 32,768; the client does not advertise a window larger than the safe/validated limit.
  • Behaviour with context_length present is unchanged.

Refs: codex #18-#21; PAIR #6, #8.

🤖 Generated with Claude Code

https://claude.ai/code/session_01RPCzgwU247gMtjybR5v4iR

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions