Skip to content

refactor(plugin): port candidate normalization to TypeScript - #837

Open
kmbroai wants to merge 9 commits into
dev/kyleb/python-free-candidate-file-operationsfrom
dev/kyleb/python-free-candidate-normalization
Open

kmbroai wants to merge 9 commits into
dev/kyleb/python-free-candidate-file-operationsfrom
dev/kyleb/python-free-candidate-normalization

Conversation

@kmbroai

@kmbroai kmbroai commented Sep 9, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Replace Python candidate normalization with the bundled Node helper and update its MCP caller. Preserve canonical candidate bytes and IDs, accepted command arguments, raw path handling, and atomic output replacement.

Changes

  • Remove the Python normalizer and its owned tests; update the launcher, package inventory, skill instructions, and runtime coverage.
  • Use standard JSON serialization with deterministic key ordering and group candidates as they are read.
  • Reuse the policy helper's path resolver and one incremental output generator across platforms.
  • Inspect Brotli assets after verified decompression so compressed bytes cannot cause false package-disclosure failures; retain scans of archive metadata, paths, and uncompressed files.
  • Store parsed options without value-type unions or casts while retaining existing argument behavior.
  • Synchronize with the updated Windows file operations in refactor(plugin): add wide Windows candidate file operations #836, including current main.

Testing

  • 54 focused normalizer and policy-helper tests passed; six Windows-only cases skipped on Linux.
  • 120 generated candidate sets matched Python's canonical output bytes and IDs, including Unicode, optional fields, and duplicate groups.
  • Lone-surrogate rejection and empty-ledger checks preserved atomic replacement and temporary-file cleanup.
  • Candidate-discovery and compact MCP integration checks passed against source and bundled runtimes in an isolated test environment.
  • Ruff source and format checks, SDK CI compilation, SDK/MCP types, formatting, plugin source compatibility, all nine source-checker tests, and bundled plugin build passed.
  • Three independent code review passes and a separate verifier completed with no actionable findings.
  • Full Linux SDK validation passed in both seeded and default order: 3,278 tests and 52 platform skips in each run.
  • 17 focused package checks, actual 523-entry tarball inspection, and installed-package smoke passed. The previous checker rejects the same tarball, confirming the compressed-byte false positive.
  • The full SDK and native platform matrix run in the PR checks.

Risk and rollout

This PR depends on #836. Canonical IDs, Unicode ordering, raw Unix paths, Windows native file handling, and atomic replacement remain compatibility requirements. The plugin helper invocation changes from normalize_candidates.py to launch_codex_security_mcp[.cmd] --helper normalize-candidates; accepted normalization arguments and defaults are preserved. The SDK CLI is unchanged. Later PRs in the migration stack will need to incorporate this updated parent.

Public disclosure review

  • No customer, partner, prospect, or user identities, data, or identifying details are included.
  • No credentials, personal data, private source, scan findings, or nonpublic links or tickets are included.
  • I reviewed the branch name, title, description, commits, changes, comments, logs, screenshots, attachments, and links for public disclosure.

@kmbroai
kmbroai added this pull request to stack #855 September 9, 2026 22:16
Comment thread plugins/codex-security/mcp-app/src/helpers/normalize-candidates.ts Outdated
Comment thread plugins/codex-security/mcp-app/src/helpers/normalize-candidates.ts Outdated
@mldangelo-oai

Copy link
Copy Markdown
Collaborator

Critical review with GPT-6 Astra at ultra

Reviewed cf0097647c79661cf3af770e9731f13c4e5432d3 against merge base 61999db83af2df76017580214fea330b2ce8612f, after the combined-stack QA.

The migration is worthwhile: candidate normalization and its MCP caller no longer need Python. No new correctness defect was found in the covered behavior. I left two nonblocking simplification comments: a 69-line reduction in numeric JSON parsing and a 12-line reduction in input ordering. #838 later shares the parser, so net savings across the full stack depend on that separate consumer review.

The disposable variants preserve the tested previously accepted artifact bytes and candidate IDs. Their deliberate differences are accepting integral spellings such as 1.0/1e0 and changing which malformed input is reported first when several are invalid. Canonical serialization, file-line bounds, path protections, input deduplication and atomic output remain part of the intended implementation.

Verification: 65 focused Bun tests passed with seven Windows-only skips; both affected MCP test files, nine portable-checker tests, and build/type/source checks passed. The 137-assertion baseline/head/variant harness passed under Node 22.22, then I independently replayed it under Node 22.13, 24.0, 26.0 and Bun 1.3.14: 137 checks passed in each run.

All new execution was on Linux. Exact-head hosted Windows/macOS CI and the separate synthetic Windows consumer checks were inspected; the combined-tree platform gap remains documented in the shared QA record.

kmbroai commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 10, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-29T11:01:47.576253Z e3e2afc New commits
🔒 Security Review ✅ Completed 2026-09-29T11:02:38.294031Z e3e2afc New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Chef's kiss.

Reviewed commit: cf0097647c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

kmbroai commented Sep 10, 2026 •

Copy link
Copy Markdown
Collaborator Author

Addressed both simplification comments: candidate parsing now uses JSON.parse and input paths use the direct deterministic sort. Canonical artifacts, IDs, line bounds, path checks, deduplication, and atomic replacement remain covered. Integral line spellings and malformed-input diagnostic differences are documented in the updated PR description.

Updated head: 280281d81678bb12f0668cd4f7244616ca5525d6. Hosted CI passes on this head, including Linux/macOS/Windows tests, native proofs, installed-package checks, plugin source contracts, types, and formatting. Codex Security Review also passes. The combined stack at 5bd1d1909ad99d2749fe8d3ec49a9a144ae2bac0 includes this head and has also passed platform CI.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2f3b519f7c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

signal,
);
const findings = await parseImportedFindings(
source.toString("utf8"),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reject malformed UTF-8 before parsing imports

When a JSON or CSV import contains invalid UTF-8 bytes, Buffer.toString("utf8") silently replaces them with U+FFFD before validation. Such an import can therefore succeed with mutated titles, paths, identities, or other finding data, while the retained source artifact still contains different raw bytes. Decode with a fatal UTF-8 decoder and reject malformed input instead of importing corrupted findings.

AGENTS.md reference: sdk/typescript/AGENTS.md:L23-L23

Useful? React with 👍 / 👎.

Comment on lines +180 to +181
if (directory !== undefined)
await rm(directory, { recursive: true, force: true });

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve successful feedback when log cleanup fails

When feedback with log attachments has already been accepted but removing the temporary attachment directory fails—for example because a Windows scanner briefly retains a file handle—this finally block rejects the completed operation and reports the upload as failed. A user retry can then submit duplicate feedback even though only optional log cleanup failed; handle or report the cleanup error without replacing the successful upload result.

AGENTS.md reference: AGENTS.md:L10-L10

Useful? React with 👍 / 👎.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants