Skip to content

🪜 feat: Project Accepted Model Tool Calls - #560

Merged
danny-avila merged 11 commits into
mainfrom
lia/agents-api-chat-parity
Sep 24, 2026
Merged

danny-avila merged 11 commits into
mainfrom
lia/agents-api-chat-parity

Conversation

@lia-by-librechat

@lia-by-librechat lia-by-librechat Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Provide an opt-in OpenAI tool-call projector driven by accepted graph results rather than provider fragments. SDK and provider-executed calls never become client instructions, even when the graph bypasses ToolNode. A single-agent host must explicitly configure clientDelegatedToolNames to return a valid call to the OpenAI client instead of executing it locally. Mixed client/graph batches fail closed.

Design

  • Stream and invoke normalize model output into SDK-owned snapshots before inspecting tool descriptors. Frozen invoke results and reusable stream chunks keep their original values and accounting identity.
  • Only complete object-valued parsed calls may enter eager host execution. Supported sealed raw fragments retain their separate reconstruction path. Malformed-call diagnostics still reach ToolNode repair.
  • Each accepted call carries trusted graph disposition (SDK, provider, or explicitly delegated client). The projector formats only the latter. ToolNode claims remain a secondary guard; an unclaimed call is not proof of client ownership.
  • Host-confirmed natural completion controls final publication. Cancellation, mixed ownership and malformed results fail closed. The host retains HTTP backpressure, persistence and side-effect responsibility.

Usage and limits: README.md. Proposed decision: docs/adr/0010-project-accepted-model-results.md.

Verification

Pushed head cb66b3f486c336cfc5717e8fba1789b5312734b3: 583 passing tests across 25 focused suites; one existing benchmark skipped. TypeScript typecheck, zero-warning scoped lint, import order, CJS/ESM/declaration build and circular-dependency check pass locally. Regressions cover real graph and SSE output, externally executed server tools, frozen invoke results, explicit client handoff, mixed batches, eager host dispatch, fallback, replay/approval, preemption and tracing. CI for this head is running.

Scope

No merge, npm release or LibreChat cutover. The client handoff is deliberately single-agent and rejects mixed client/graph batches. No eager side-effect rollback or durable wire delivery is promised. Live-provider and live-Langfuse-project checks were not run.

Related: LibreChat-AI/LibreChat#13987, LibreChat-AI/LibreChat#16206 and #260.

@lia-by-librechat

Copy link
Copy Markdown
Contributor Author

Review handoff for exact pushed head fd208ff6f80e4052f15b6b935b1550d1b11e71b4: the opt-in OpenAI tool-call projector preserves complete-before-publish identity/index behavior across SDK graph run steps without changing existing handler exports or wire framing. Please review native event matching, incomplete fragments, abort/reentrant emission and API compatibility; LibreChat integration is a separate follow-up.

@danny-avila

Copy link
Copy Markdown
Collaborator

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-24T19:52:47.786153Z cb66b3f Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fd208ff6f8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/openai/toolProjection.ts Outdated
Comment thread src/openai/toolProjection.ts Outdated
Comment thread src/openai/toolProjection.ts Outdated
Comment thread src/openai/toolProjection.ts
Comment thread src/openai/toolProjection.ts Outdated
Comment thread src/openai/toolProjection.ts Outdated
@lia-by-librechat

Copy link
Copy Markdown
Contributor Author

Review handoff for exact remote head c26f6dc4b16281ad6f5617189ca9e5ad775f9069: addresses all six Codex findings with real ChatModelStreamHandler/StandardGraph regressions. Snapshot/identity reconciliation, late declarations, fallback attempt isolation, reentrant abort, early indexed arguments and required step IDs are covered. Also checks text-only fallback, late failed-primary chunks, parallel nodes and graph-segment boundaries. Focused burn-in: 149 tests across 6 suites passed; tsc --noEmit, touched-file lint (zero warnings), Prettier, build and CJS/ESM export checks passed. Default SDK handlers are unchanged; new projector hosts must call observeModelAttempt before model chunks/end events. Please review this head; the previous review covered fd208ff.

@danny-avila

Copy link
Copy Markdown
Collaborator

@codex review the latest head

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c26f6dc4b1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/openai/toolProjection.ts Outdated
Comment thread src/openai/toolProjection.ts Outdated
@lia-by-librechat lia-by-librechat Bot changed the title 🪜 feat: Stabilize API Tool Calls Across Run Steps 🪜 feat: Project Accepted Model Tool Calls Sep 24, 2026
@lia-by-librechat

Copy link
Copy Markdown
Contributor Author

Review handoff for exact remote head 79288adf8df17a8ebb0b8dc429c96a57632fee7b: this replaces fragment/attempt reconstruction, rather than adding another inference rule. Graph emits an awaited accepted-result event after primary/fallback selection, overflow recovery and usage accounting. Both streaming and invoke-only paths use it; callback echoes cannot forge it. The projector has no raw-fragment caches, no observeModelAttempt obligation, bounded retained calls/bytes, sanitized failures and terminal cancellation/writer semantics. 630 tests across 22 focused suites passed, plus typecheck, zero-warning touched-file lint, CJS/ESM builds/exports and circular checks. Measured projector coverage is 100% (not a universal correctness claim). Please review the new ownership boundary, payload isolation and publication lifecycle. No default handler, LibreChat consumer, release or merge is included.

@danny-avila

Copy link
Copy Markdown
Collaborator

@codex review the latest head

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 79288adf8d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/openai/toolProjection.ts
Comment thread src/openai/toolProjection.ts Outdated
@lia-by-librechat

Copy link
Copy Markdown
Contributor Author

Review handoff for exact remote head 557068f6b1b239d23f6ed29751f327101cdad805: addresses both new P2 findings. Streaming projection and the public finalizer share one tracker; strict JSON-tree encoding rejects non-JSON objects without calling conversion hooks and checks retained-output limits during traversal. Both original failures were reproduced first (8 failing tests). Final local verification: 681 tests across 24 focused suites, full SDK tsc --noEmit, touched-file lint/formatting, package build, CJS/ESM exports and circular checks passed. No fresh review or green CI is claimed for this new head yet. Please review the tracker/finalizer composition, JSON semantic validation and resource-bound failure paths. No release, merge or LibreChat cutover is included.

@danny-avila

Copy link
Copy Markdown
Collaborator

@codex review the latest head

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 557068f6b1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/graphs/Graph.ts Outdated
Comment thread src/openai/toolProjection.ts Outdated
Comment thread src/graphs/Graph.ts
@lia-by-librechat

Copy link
Copy Markdown
Contributor Author

Review handoff for exact remote head 6e0000baad74162b6175031c44d811fd3cfde09b: addresses the three new P2 findings by validating original argument descriptors before cloning, isolating every composed accepted-result observer, and reserving all provider IDs before allocating synthetic IDs. Failing cases were reproduced first (4 failures). Local burn-in: 695 tests across 26 suites, TypeScript, changed-file zero-warning lint/formatting, package build, circular dependencies and CJS/ESM exports passed. CI and review on this exact commit are pending. The opt-in SDK API has not been merged, released, or switched on in LibreChat.

@danny-avila

Copy link
Copy Markdown
Collaborator

@codex review the latest head

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6e0000baad

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/openai/toolProjection.ts Outdated
Comment thread src/graphs/acceptedModelResponse.ts Outdated
@lia-by-librechat

Copy link
Copy Markdown
Contributor Author

Review handoff for exact remote head 9441e860cb3839fe297c3b6925441c5bec4beca6.

Invariant-based self-review

Reviewed the source-of-truth path, acceptance and publication ordering, input/result ownership, retries, cancellation, ID allocation, observer composition, subagent forwarding, usage/tracing, resource bounds and rollout compatibility. Codegraph and GitHub agreed on the starting head 6e0000ba; the graph provided locations and dependency leads, while source/tests established behavior.

Found and fixed:

  • Terminal chronology (also the new Codex P1): tests previously checked text after finish(), whereas the documented host finishes after the final answer. Real combined text/projector handlers now run through Run.processStream and the public finalizer. Terminal state comes from the last accepted response, so a completed tool/result/final-answer run stays stop rather than asking clients to execute historical tools again.
  • Invalid-result bound bypass (also the new Codex P2): invalid diagnostic payloads were cloned outside the valid-call limits. Reject a nonempty invalid-call array without reading/cloning its elements. Own data-descriptor checks reject accessors and non-array collection shapes at the snapshot boundary.
  • Stale/mixed result ownership: a nonempty map could leak calls from another response. Projectors now require an empty map and reject writes to it during collection; the constructor does not destroy previous response output.

Evidence: eight failing regressions for snapshot/map paths, then three failing real-composition/finalizer regressions before fixes. The final focused run passed 830 tests across 30 suites, including fallback, overflow, cancellation, prepared tools, nested subagent/replay paths and Langfuse callbacks/trace shaping. Real execution suite repeated 3/3 successfully. TypeScript, touched-file zero-warning ESLint, formatting, CJS/ESM build/exports and circular-dependency checks passed. CI and a fresh review must refer to the new commit, not an earlier green head.

Limits and next gate:

  • The graph event is not a security sandbox for arbitrary in-process callbacks. Payload isolation cannot prevent a trusted callback with a graph reference from mutating the graph directly.
  • The encoded-output limit is not a bound on provider buffers, transient input-key enumeration, or observer copies deliberately retained by host code. No production throughput/SLA claim.
  • Eager tool side effects cannot be rolled back. A resolved processStream() alone is not natural completion; halt/interrupt/cancel still requires abort.
  • Default OpenAI/Responses handlers remain unchanged. No merge, release, deployment, or LibreChat cutover. Actual HTTP/SSE, storage and real-client parity remain separate consumer-integration gates.

Local synthetic projector scaling check (10 repetitions after warmup): 64 calls/~250 KB averaged 0.78 ms; 256 calls/~1 MB averaged 2.59 ms; 1024 calls/~4 MB averaged 9.69 ms. Forty near-cap complete/abort cycles had no retained heap growth after explicit GC in this run. This excludes provider execution, graph snapshots, retained observers and HTTP, and is not a production performance guarantee.

@danny-avila

Copy link
Copy Markdown
Collaborator

@codex review the latest head

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9441e860cb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/openai/toolProjection.ts
Comment thread src/graphs/Graph.ts
@lia-by-librechat

Copy link
Copy Markdown
Contributor Author

Review handoff: cf031193c57bc7da3f1b3fd0ff9e695dcb17bdbe is documentation-only. README addition reduced from 687 to 213 words; ADR from 778 to 211 words; PR description condensed. Git confirms the source tree is identical to 9441e860cb3839fe297c3b6925441c5bec4beca6, the code head already submitted for Codex review. Formatting, diff and link-target checks passed. No code or test changes.

@lia-by-librechat

Copy link
Copy Markdown
Contributor Author

Exact pushed head: 1aee7ea2116716abdd045da3cfb836359612f7f8

Self-review follow-up: all three reproduced findings are addressed on the head below.

  • ToolNode now claims accepted batches through an awaited, registry-only event keyed by agent and message identity. The projector retires those calls before execution, including toolEnd runs, without discarding another agent's pending output.
  • Parsed calls, raw fragments, and malformed-call diagnostics are descriptor-checked before SDK accounting or dispatch. Per-emission snapshots preserve reusable provider records and canonical producer/consumer charge identity.
  • Execution uses iterative, descriptor-safe cloning instead of the projection serializer. Depth, count, and encoded-byte formatting caps remain confined to accepted-event/projection output; ordinary runs preserve deep/large JSON arguments and shared references.

Invariant review covered both graph-created ToolNode variants, observer failure/isolation, provider event spoofing, agent/message collisions, repeated-round buffer release, replay authorization and checkpoint restoration, cancellation, eager execution, fallback/preemption, and tracing. The replay regression caught and eliminated an early observer await before authority restoration. No checkpoint format or authorization rule changed. Hosts still finish only after natural completion and abort on interruption/error; there is no durable wire-delivery guarantee.

Verification:

  • Focused subsystem run: 25 suites passed; 569 tests passed. Existing skipped benchmark: benchmarks tool stream handling with deterministic arguments.
  • Final validation/real-graph rerun: 51 tests passed.
  • tsc --noEmit, ESLint with --max-warnings=0 on touched files, scoped import order, and git diff --check: passed.
  • npm run build (CJS/ESM/declarations) and npm run check:circular-deps: passed.
  • Not run locally: whole repository suite, live-provider calls, live-Langfuse-project verification, or the skipped benchmark.

No Codex review covers this new head yet. A maintainer can trigger that review from this exact-head handoff. Nothing has been merged or released.

CI at handoff: 12 checks passed, including all four unit shards; Anthropic summarization is still running. No unresolved inline review threads were present.

@danny-avila

Copy link
Copy Markdown
Collaborator

@codex review the latest head

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1aee7ea211

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/tools/ToolNode.ts
Comment thread src/graphs/acceptedModelResponse.ts
Comment thread src/graphs/acceptedModelResponse.ts
@lia-by-librechat

Copy link
Copy Markdown
Contributor Author

Exact pushed head: cb66b3f486c336cfc5717e8fba1789b5312734b3

This head replaces implicit "unclaimed means client" with trusted per-call SDK/provider/client disposition. A pure single-agent handoff requires clientDelegatedToolNames; mixed client/graph batches fail closed. Stream and invoke share a non-mutating SDK-owned snapshot, frozen results are accepted, and string-valued partial parsed calls cannot prestart or dispatch host work. The three prior Codex findings have focused regressions and replies on their inline threads.

Validation: 583 passing tests across 25 focused suites (one pre-existing benchmark skipped); tsc --noEmit, zero-warning touched-file ESLint, scoped import order, distributable build and circular dependency check passed. Real graph tests cover server-only, mixed provider/SDK, explicit client handoff, frozen invoke, fallback and SSE output. No live provider or live Langfuse project test was run. CI for this exact head is in progress. No release or merge.

@danny-avila

Copy link
Copy Markdown
Collaborator

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. You're on a roll.

Reviewed commit: cb66b3f486

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@danny-avila
danny-avila merged commit 06c7666 into main Sep 24, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants