Skip to content

🧯 fix: Fence Worker Pairing Revocation - #70

Open
danny-avila wants to merge 18 commits into
danny-avila/principal-code-workersfrom
danny-avila/code-worker-lifecycle
Open

🧯 fix: Fence Worker Pairing Revocation#70
danny-avila wants to merge 18 commits into
danny-avila/principal-code-workersfrom
danny-avila/code-worker-lifecycle

Conversation

@danny-avila

Copy link
Copy Markdown
Collaborator

Summary

I hardened bridge-worker lifecycle deletion so revocation and replacement invalidate pending pairing codes as well as active credentials. This is required before LibreChat can safely expose self-service environment deletion.

  • Add a per-worker pairing generation fence with the same bounded lifetime as the one-time code.
  • Redeem pairing codes atomically through Redis so concurrent revoke, replacement, and duplicate redemption fail closed.
  • Rotate the generation before revoking active credentials so a pending code cannot recreate a deleted worker.
  • Cover pending-code revocation and replacement invalidation with regression tests.

Change Type

  • Bug fix (non-breaking change which fixes an issue)

Testing

  • bun test src/bridge/pairing.test.ts — 10 passed
  • bun run build — passed with existing Rollup warnings
  • bun test — 599 ordinary tests passed; the unfiltered repository command also loads stress-tests/stress_test.js and reports its unavailable k6/http runtime

Test Configuration:

  • Bun 1.3.13
  • Redis behavior covered with ioredis-mock

Checklist

  • My code adheres to this project's style guidelines
  • I have performed a self-review of my own code
  • I have commented in complex areas of my code
  • My changes do not introduce new warnings
  • I have written tests demonstrating that my changes are effective
  • Local unit tests pass with my changes

@danny-avila

Copy link
Copy Markdown
Collaborator Author

@codex review exact head 6c255df27d341dfd81ed933256174543a7d2fefe

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Aug 30, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-08-31T02:32:34.618337Z 716fea5 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6c255dfb99

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread service/src/bridge/pairing.ts
@danny-avila

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review exact head d4db70e. This head adds legacy pairing cleanup plus a Recreate rollout fence, and the production Docker image was rebuilt and exercised against live Redis.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d4db70e918

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread service/src/bridge/pairing.ts
@danny-avila

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review exact head 33c16f3 after the legacy redemption compatibility fix.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 33c16f3aa8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread helm/codeapi/values.yaml
Comment thread service/src/bridge/pairing.ts Outdated
Comment thread helm/codeapi/templates/api-deployment.yaml
Comment thread service/src/bridge/pairing.ts Outdated
@danny-avila

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review exact head 33c16f3 and report only actionable findings introduced by this PR.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🎉

Reviewed commit: 33c16f3aa8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@danny-avila

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review exact head e333133 and report only actionable findings introduced by this PR.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e3331339f0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread service/src/bridge/pairing.ts Outdated
@danny-avila

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review the current PR head c5ab56b. Confirm that this exact commit is the reviewed commit and ignore findings that apply only to earlier heads. This head is rebased onto the authenticated principal-routing fixes from PR #69; focus on revocation linearizability, legacy rollout compatibility, and credential-bound lease behavior.

@danny-avila
danny-avila force-pushed the danny-avila/code-worker-lifecycle branch from c5ab56b to e085738 Compare August 30, 2026 19:54

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e085738917

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread service/src/bridge/pairing.ts Outdated
@danny-avila
danny-avila force-pushed the danny-avila/code-worker-lifecycle branch from e085738 to 0cc49ee Compare August 30, 2026 20:13
@danny-avila

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review the current PR head 0cc49ee. Confirm that this exact commit is the reviewed commit and ignore findings that apply only to earlier heads.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0cc49ee5ca

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread service/src/bridge/pairing.ts Outdated
@danny-avila
danny-avila force-pushed the danny-avila/code-worker-lifecycle branch from 0cc49ee to e2e51ac Compare August 30, 2026 20:32
@danny-avila

Copy link
Copy Markdown
Collaborator Author

@codex review\n\nPlease review exact head commit e2e51ac52aa56f6fa99eba19d5880e374403603a after rebasing onto #69 and making the legacy pairing migration window reopen safely after later rollbacks.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e2e51ac52a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread service/src/bridge/pairing.ts Outdated
@danny-avila
danny-avila force-pushed the danny-avila/code-worker-lifecycle branch from e2e51ac to b9f9723 Compare August 30, 2026 20:49
@danny-avila

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review exact head commit b9f9723 after rebasing onto the latest #69 head and bounding legacy pairing scans to one initial worker claim or concrete rollback evidence. Report only actionable findings on this exact head.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b9f97235b6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread service/src/bridge/pairing.ts
@danny-avila
danny-avila force-pushed the danny-avila/code-worker-lifecycle branch from b9f9723 to 2108f09 Compare August 30, 2026 21:19
@danny-avila

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2108f0963a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread service/src/bridge/pairing.ts Outdated
Comment thread service/src/bridge/pairing.ts Outdated
Comment thread service/src/bridge/pairing.ts Outdated
@danny-avila
danny-avila force-pushed the danny-avila/code-worker-lifecycle branch from 2108f09 to 6f2feec Compare August 30, 2026 22:00
@danny-avila

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6f2feecb7b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread service/src/bridge/pairing.ts Outdated
Comment thread service/src/bridge/pairing.ts Outdated
Comment thread service/src/bridge/pairing.ts
Comment thread helm/codeapi/templates/api-deployment.yaml
@danny-avila

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review the current restacked exact head b07f73b. Confirm this exact commit is reviewed; it includes the latest #69 settlement and rollout-compatibility fencing plus #70 lifecycle protections.

@danny-avila
danny-avila force-pushed the danny-avila/code-worker-lifecycle branch from 27970cd to b07f73b Compare August 31, 2026 01:46

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b07f73b2ad

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread helm/codeapi/scripts/safe-pairing-rollback.sh Outdated
Comment thread service/src/bridge/pairing.ts
Comment thread helm/codeapi/scripts/safe-pairing-rollback.sh Outdated
@danny-avila
danny-avila force-pushed the danny-avila/code-worker-lifecycle branch from b07f73b to e277ea7 Compare August 31, 2026 02:02
@danny-avila

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review exact head e277ea7 after restacking on #69 a8b3c17. Focus on last-moment rollback drain verification, fail-closed recovery after partial Helm failure, and the out-of-band rollback epoch that forces legacy pairing cleanup after re-forward. Confirm this exact commit is reviewed.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e277ea7d95

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread helm/codeapi/scripts/safe-pairing-rollback.sh Outdated
Comment thread helm/codeapi/scripts/safe-pairing-rollback.sh Outdated
Comment thread service/src/bridge/pairing.ts
Comment thread service/src/bridge/pairing.ts
@danny-avila

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review exact head ca53b09 after status-checking every pod relist, re-discovering and draining all release API Deployments after partial rollback failure, persisting durable epoch completion markers, and applying epoch cleanup before legacy redemption. Confirm this exact commit is reviewed.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ca53b09634

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread helm/codeapi/scripts/safe-pairing-rollback.sh Outdated
@danny-avila

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review exact head 175398a after rejecting forwarded Helm flags that could make rollback target a different namespace, cluster, kubeconfig, API server, or Kubernetes identity than the kubectl drain. Confirm this exact commit is reviewed.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 175398a11a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread helm/codeapi/scripts/safe-pairing-rollback.sh
Comment thread helm/codeapi/scripts/safe-pairing-rollback.sh Outdated
Comment thread service/src/bridge/pairing.ts Outdated
@danny-avila

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review exact head 716fea5 after rejecting Helm target overrides from both flags and environment, installing HUP/INT/TERM fail-closed recovery around the tracked Helm child, and validating pairing-code existence and ownership before any rollback-epoch scan. Confirm this exact commit is reviewed.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Bravo.

Reviewed commit: 716fea5599

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant