Skip to content

fix: Parse canonical Codex rollouts with bounded reconciliation - #817

Draft
buildingjoshbetter wants to merge 1 commit into
fix/transcript-inline-publish-306from
fix/codex-rollout-publish-762
Draft

buildingjoshbetter wants to merge 1 commit into
fix/transcript-inline-publish-306from
fix/codex-rollout-publish-762

Conversation

@buildingjoshbetter

Copy link
Copy Markdown
Owner

Scope

Related to #762. Stacked on #815. Parse canonical native Codex user and assistant response items while preserving file order, repeated IDs and repeated text as separate occurrences.

The supported grammar follows official Codex commit 0b755b1945bf4a31560df2ce1469aeb044dccbc9, including ordinary Legacy/Paginated lifecycle and terminal projections.

  • Reconcile plain-text event echoes against canonical occurrences, including echoes written before responses. Events add no extracted turns.
  • Give terminal summaries separate evidence within their identified lifecycle, preserving independent echo counts and session boundaries.
  • Apply strict, positionally aligned provenance. Exclude inherited, compaction, harness, tool and retained-delivery content. Explicit unknown attribution cannot authorize text.
  • Preserve Claude and generic formats, including incidental payload fields on ordinary messages and known legacy exclusions.
  • Report partial coverage for unsupported markup projections, media, unknown variants and unexplained events. Diagnostics contain categories and counts without source text or identifiers.
  • Count meaningful parsed human occurrences after removing TrueMemory-injected blocks. The default admission threshold remains exactly five. Byte length and event echoes cannot inflate it.

Reconciliation cost

Each eligible canonical occurrence contributes eight index keys. Matching keeps a monotonic cursor per role and uses at most two binary searches per event. Canonical messages are never deduplicated or reordered.

At 400 unmatched or metadata-incompatible echoes, instrumented work fell from 160,000–480,000 operations in rejected v1 to 16,380–17,980 with the index. These are operation-count measurements. Independent comparison against a linear oracle covered 38,880 selector/cursor cases.

Validation

  • Reviewed author commit: 378b8f726b89ff2d0f808deade181c3f9e248d08.
  • Exact publication commit: b5dbde5037f583bac81a968042b9876e0ef0bae5.
  • Independent focused review: 54 tests and 93 subtests passed. All 53 prior test-method ASTs remain unchanged in the final legacy-exclusion correction.
  • Unprivileged author full suite: 2,986 tests and 2,293 subtests passed, with 17 skips and 3 deselections. Log SHA256: f6acbe9a3491fbea274c9fb19ac50e8abb666e7b4bd5d8c460bfa9628bb970a1.
  • Exact unprivileged publication full suite: 2,367 tests and 646 subtests passed, with 17 skips and 3 deselections. Log SHA256: 8bbf1b046d949d6f057c85a5c9899e40c8f19eaad32b0e6b1816c3c677720569.
  • Both full suites used isolated Linux containers, synthetic data, disabled networking and actual sqlite-vec. No GPU was exposed to these functional runs.
  • All four parser/source/test/documentation files match the reviewed author. The publication branch also includes the independently reviewed platform-config and M5: _load_config() Reads Disk on Every Search (2x JSON Parse per Query) #502 read-counter fixture corrections required for unprivileged testing. It has no feat: directives completion — land WIP safely (migration-skip crash, gate exclusion, injection cap, scoping, adapter parity, custom-tier forget) #589 fixture dependency.
  • Ruff, Python 3.10 grammar, whitespace, scoped privacy and public commit-identity checks passed. Hosted checks are pending.

Earlier failures remain recorded. V1 was rejected for quadratic matching and lossy generic-family dispatch. V2 was rejected for misclassifying known legacy exclusions with incidental payloads. The final exclusion regression fails 27 assertions against v2, with zero errors. The fixture's forced foreground config reread still fails its original assertion.

Remaining requirements

Capture still uses the list parser API and does not consume detailed coverage outcomes. C05 path admission, C07 native hook delivery and #768 durable completion policy remain open. Citation and proposed-plan transformations remain partial. Parser support alone does not establish complete automatic capture.

No model, reranker, dimensions or retrieval limits changed. No personal chats or memories were used. Draft only; no issue closure, main merge or release.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant