Skip to content

Add the knowledge publisher (kb publish) for the GPU box - #210

Merged
tzhouam merged 1 commit into
mainfrom
kb/publisher
Sep 28, 2026
Merged

tzhouam merged 1 commit into
mainfrom
kb/publisher

Conversation

@tzhouam

@tzhouam tzhouam commented Sep 28, 2026

Copy link
Copy Markdown
Collaborator

PR 7 of the knowledge-service redesign: the publisher (kb publish). It runs on the GPU box and is the only GitHub writer for knowledge PRs, using the owner's existing gh login.

  • Each round:
    • reads the signed control record over SSH (or a local dir); if it is stale, the round does nothing;
    • runs every item through check_item (signature, expiry, generation, pause, auto_merge in both the control record and the publisher's own adapter config);
    • performs the item, records it atomically, then writes a publisher-signed kb-ack.
  • Idempotent and crash-safe:
    • a performed item only has its ack re-sent;
    • an existing PR is adopted only if its head is exactly the deterministically rebuilt commit;
    • a corrupt completion record is set aside for a person, never guessed;
    • failed pause/close actions are retried every round and never acked as failed;
    • SSH failures are traced, never fatal.
  • Actions:
    • open_pr: rebuilt from the signed contents on base_sha; governed pages only; never overwrites a branch;
    • open_companion_pr: always a draft;
    • post_verdict: removes kb:hold and confirms it is gone first, so pause → resume → re-gate → enqueue completes;
    • enqueue: ready, then merge --auto --match-head-commit;
    • pause: procedure P;
    • update_branch, close.
  • Double gate: ALLOW_POST=1, plus ALLOW_PUSH=1 for branch pushes; otherwise it only records.

Commit hook approved; full suite and SPEC freshness pass locally.

🤖 Generated with Claude Code

The only GitHub writer for knowledge PRs. It runs as the repository owner
through the box's existing gh login, which never reaches the bot host or any
model environment.

- publisher: each round reads the service's signed control record over SSH
  (or a local directory) and does nothing when it is stale. Every outbox item
  goes through check_item (signature, expiry, generation, pause, and
  auto_merge in both the control record and the publisher's OWN adapter
  configuration). The publisher performs the item, records the result
  locally, then writes a kb-ack signed with the publisher key. Items it cannot
  verify or must not execute are neither performed nor answered. Every
  decision is traced to traces/publisher.jsonl.
- idempotent: an item already performed only has its ack re-sent; completion
  records are written atomically, and a corrupt one is set aside for people,
  never guessed. open_pr reuses an open PR on its branch only when its head
  is exactly the rebuilt commit (and a draft for companions). It rebuilds the commit deterministically from
  the signed contents on base_sha in a scratch index, and refuses to
  overwrite an existing branch with other content. It also refuses any path
  that is not a governed knowledge page.
- actions: open_pr and open_companion_pr (always a draft). post_verdict,
  enqueue and update_branch first check that the PR is open at the signed
  head; enqueue marks a paused draft ready, then runs gh pr merge --auto
  --match-head-commit. pause runs procedure P: dequeue, disable auto-merge,
  convert to draft (verified), then label kb:hold. close closes the PR.
  A post_verdict (issued only for an unpaused repository under its current
  generation) first removes kb:hold and confirms it is gone before
  commenting, so pause, then resume, then re-gate, then enqueue completes.
- a failed pause or close is never acknowledged: it stays in the outbox
  and is retried every round until it succeeds. Malformed control records
  and items are traced and skipped; they never stop the daemon.
- transport failures (ssh down while listing or delivering an ack) are
  traced, never fatal: the completion record stays and the ack is re-sent
  next round without repeating the action.
- double gate: without ALLOW_POST=1 a round only records what it would do;
  pushing a branch also needs ALLOW_PUSH=1.
- CLI: kb publish (--remote host:/state | --local dir) [--once]. It needs
  KB_SERVICE_PUBKEY, KB_PUBLISHER_KEY and KB_PUBLISHER_GIT_AUTHOR, and never
  opens the service ledger.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-28T13:20:01.268110Z 13ef822 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@tzhouam
tzhouam merged commit bd0047d into main Sep 28, 2026
5 checks passed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 13ef822719

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +245 to +246
item = check_item(envelope, control, self.service_public_key, now=self.clock(),
repo_publishes=publishes, repo_auto_merge=auto_merge)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Bind each signed item ID to its outbox filename

When a valid signed envelope is copied or renamed to another filename matching ITEM_ID, the publisher verifies the envelope but never checks that item.id == item_id. The completion lookup therefore uses the alias while _record_done uses the signed ID, leaving the alias perpetually uncompleted and re-executing it every round; a duplicated post_verdict item, for example, posts the same gate comment indefinitely. Reject filename/payload ID mismatches before performing the action.

Useful? React with 👍 / 👎.

Comment on lines +205 to +207
for item_id in item_ids:
done = self._done_path(item_id)
if done.is_file():

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Serialize publishers before checking completion state

If two publisher processes share the configured state directory—for example, the daemon overlaps a manual --once invocation—both can observe that done is absent and execute the same item concurrently. Besides duplicate verdict comments, concurrent open_pr handling can let one process create the PR while the other records and sends a failed acknowledgement after losing the branch-push race, potentially moving the changeset to failed. Hold an interprocess lease across the completion check, action, and record write.

Useful? React with 👍 / 👎.

Comment on lines +332 to +333
self.github.gh("pr", "comment", number, "--repo", self.github.repository,
"--body-file", "-", input=str(item.body["comment"]))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Make verdict posting recoverably idempotent

If the process is killed after gh pr comment succeeds but before _record_done runs, the next invocation finds no completion record and posts the verdict again. Thus the advertised crash-safe behavior only holds for open_pr, while this action can create duplicate comments and trigger the gate workflow repeatedly. Before posting, adopt an existing comment carrying the exact signed verdict or otherwise persist a recoverable operation identity.

Useful? React with 👍 / 👎.

Comment on lines +310 to +312
args = ["pr", "create", "--repo", self.github.repository, "--base", "main", "--head", branch,
"--title", str(body["title"]), "--body-file", "-"]
self.github.gh(*args, *(["--draft"] if draft else []), input=str(body.get("body") or ""))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reconcile PRs opened while a pause is racing

If kb pause runs after gh pr create succeeds but before the service collects this acknowledgement, the ledger still has the changeset in pr_requested, so pause_open_prs cannot see its PR number and issues no pause item. The later successful acknowledgement moves it to pr_open without checking the current pause state, leaving the newly created PR non-draft for the duration of the pause. The acknowledgement path needs to detect this state and immediately issue/apply procedure P.

Useful? React with 👍 / 👎.

Comment on lines +207 to +210
if done.is_file():
try:
record = json.loads(done.read_text(encoding="utf-8"))
except ValueError:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Quarantine structurally invalid completion records

A completion file containing valid JSON but not the expected acknowledgement object, such as {} or null, passes json.loads and is then handed to _send_ack, where indexing ack["item_id"] raises an uncaught KeyError or TypeError. Since serve does not catch failures from run_once, one such damaged or obsolete record terminates the publisher on every restart instead of being moved to .corrupt as intended. Validate the decoded record's type and required fields before resending it.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant