Skip to content

feat(github): decide whose PR feedback wakes agents with ordered rules - #1746

Draft
johnmatthewtennant wants to merge 8 commits into
jtennant/poppy/watching-sidebar-onboardingfrom
jtennant/poppy/github-author-allowlist
Draft

johnmatthewtennant wants to merge 8 commits into
jtennant/poppy/watching-sidebar-onboardingfrom
jtennant/poppy/github-author-allowlist

Conversation

@johnmatthewtennant

@johnmatthewtennant johnmatthewtennant commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Context

github_watch_pr quotes new pull request comments and reviews to the agent watching. Anyone who can comment on a public pull request can put text in front of that agent, and main's only filtering is dropping the signed-in account's own 🤖 comments.

Change

The GitHub plugin now decides whose feedback wakes agents with ordered allow and deny rules. All of it lives in src/bundled/github.

  • Settings → Plugins → GitHub comment authors holds the rules in one text box, one per line, saved on the device. Each is allow or deny, then any of these conditions, all of which must hold:
    • author:login, an exact login (a bot's ends in [bot]), or author:@me, the signed-in gh account.
    • author:/pattern/, a regular expression the login must match.
    • org:name, membership of an organization, named exactly.
    • body:/pattern/, a regular expression the comment must match.
    • prefix:text, what the comment starts with. It runs to the end of the line.
  • The first matching rule decides. Feedback no rule matches wakes nobody, so a list of allow rules is an allowlist.
  • Patterns are JavaScript regular expressions, with the i, m, s or u flags. Author patterns always ignore case, since GitHub logins do, and exact logins also match regardless of case. Org has no pattern form, because membership is only looked up for named organizations.
  • Saving compiles every rule and rejects the text with an error that names the line, such as Line 2: Invalid regular expression: /(/i: Unterminated group.
  • The defaults reproduce main and are shown in the box, where they can be edited. The 🤖 check moves out of watch.ts into the first rule:
    deny author:@me prefix:🤖
    allow author:@me
    allow
    
    While a bare allow remains, the card says anyone can wake agents.
  • Rules are saved under a v2 key. Lists saved by an earlier build of this PR aren't converted; they fall back to the defaults.
  • Organization membership is read through the existing pull gh command, only for organizations the rules name. It is batched to the 4096-byte input limit, trusted for an hour, and asked one login at a time after a failed batch. An answer older than an hour is not used, even when refreshing it fails. A login whose own read fails ten polls in a row counts as no member. Feedback waits for a later poll only when a rule needs a membership that hasn't been read.
  • Known limits: a membership hidden from the signed-in gh account, or one in a misspelled organization, counts as none. Bots and deleted accounts are in no organization.
  • Each poll rereads the rules, so a change applies to running watches, including feedback already waiting out the 90-second quiet window. CI and PR-state wakes are not affected.

Validation

src/bundled/github/watcher.test.ts drives the real watcher with the gh command answers faked, so these cover rule evaluation, membership reads and wakes, not live GitHub:

  • The default rules tell the viewer's comments and a stranger's, but not the viewer's 🤖 ones.
  • Pattern and per-organization rules tell only matching feedback; denied feedback is seen and never told.
  • Feedback that needs an unread membership waits, then is told once the membership can be read.
  • Tightening the rules while feedback waits in the quiet window keeps the newly denied author's comment from waking the agent. This test failed before the fix, telling the stranger's comment.
  • After an hour, a member whose membership refresh fails is not admitted until it can be read again. This test failed before the fix, waking the agent with the later comment.
  • Waiting feedback whose author's membership the new rules need, and which can't be read, waits until it can be read rather than being told. This test failed before the fix, telling the comment.

The settings card and the watcher share one stored policy: AuthorSettings.test.tsx saves through the card and reads the result with the watcher's loadPolicy, and the watcher tests above change rules mid-watch through that same storage. tests/browser/settings.spec.mjs checks the settings card in the real app in Chromium and WebKit.

Not tested: the native app against a real pull request and real organization membership.

Screenshots

Default rules, with the warning about a bare allow

Default GitHub comment author rules

A rule that does not compile

Line-numbered rule error

Saved organization rules

Saved organization rules

Base automatically changed from jtennant/simone/first-party-command-sources to main October 27, 2026 20:55
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/poppy/github-author-allowlist branch 2 times, most recently from d2a2c7d to a6933cb Compare October 28, 2026 13:55
@johnmatthewtennant johnmatthewtennant changed the title feat(github): limit whose PR feedback wakes agents feat(github): decide whose PR feedback wakes agents with ordered rules Oct 28, 2026
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/poppy/github-author-allowlist branch from 7ce88dc to 2649215 Compare October 28, 2026 18:36
@johnmatthewtennant
johnmatthewtennant changed the base branch from main to jtennant/poppy/watching-sidebar-onboarding October 28, 2026 18:36
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/poppy/github-author-allowlist branch from 2649215 to 13fe4ff Compare October 28, 2026 20:29
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/poppy/watching-sidebar-onboarding branch from 9c8472d to d66d510 Compare October 28, 2026 20:32
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/poppy/github-author-allowlist branch 3 times, most recently from 6e15c86 to 61ebd33 Compare October 29, 2026 01:43
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/poppy/watching-sidebar-onboarding branch from a3e3bd0 to c490af7 Compare October 29, 2026 02:57
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/poppy/github-author-allowlist branch from 61ebd33 to ee70d24 Compare October 29, 2026 02:57
johnmatthewtennant and others added 8 commits October 29, 2026 01:57
Agents act on the comments a PR watch quotes, so anyone able to comment
on a public pull request could steer them. Settings > Plugins > GitHub
comment authors now holds trusted organizations, trusted logins and
ignored logins. Once a trusted entry is saved, only trusted logins,
members of a trusted organization and the signed-in account wake
agents; ignored logins never do. Membership is read through gh, batched
and kept for an hour, and feedback whose author's membership cannot be
read waits for a later poll. Each poll rereads the settings.

Co-authored-by: Poppy <ace4dfe51b23bb4244ab9117dfd5e13b8239445e333225608c352e68cb3fe2b5@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: John Tennant <jtennant@block.xyz>
A login GitHub has no user for, such as a mannequin, failed its whole
batch, so every comment in it waited forever. A failed batch now asks
each login alone, and a login whose own read fails ten polls in a row
counts as no member. Batches are sized to the pull command's 4096-byte
input limit instead of a fixed ten logins. Logins may contain
underscores, as Enterprise Managed Users' do, and any case of [bot]. A
failed save shows its error, and the card and docs say that hidden
memberships and misspelled organizations count as none.

Co-authored-by: Poppy <ace4dfe51b23bb4244ab9117dfd5e13b8239445e333225608c352e68cb3fe2b5@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: John Tennant <jtennant@block.xyz>
The GitHub plugin now adds a settings card, so the keyboard order of
the Settings navigation includes it after Builderlab hosting. Turning
GitHub back on also adds that card to the stacked narrow navigation,
which pushes the toggle down, so the test scrolls it back into view
before checking it.

Co-authored-by: Poppy <ace4dfe51b23bb4244ab9117dfd5e13b8239445e333225608c352e68cb3fe2b5@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: John Tennant <jtennant@block.xyz>
Three lists and a hardcoded rule for agents' 🤖 comments could not say
things like "members of block, but not this bot" or "anyone at
squareup whose login starts with ada_". Settings > Plugins > GitHub
comment authors now holds one rule per line in a single text box. A
rule is allow or deny, then any of author:login, author:@me,
author:/pattern/, org:name, body:/pattern/ and prefix:text. The first
rule that matches decides, and feedback no rule matches wakes nobody.

Patterns are JavaScript regular expressions with the i, m, s or u
flags, and author patterns always ignore case, as logins do. Org stays
an exact name, since membership is only asked for named organizations.
Saving compiles every pattern and rejects the rules with an error that
names the line.

The defaults, deny author:@me prefix:🤖, allow author:@me and allow,
keep today's behavior and are shown and editable, so the 🤖 rule moves
out of watch.ts. Rules are saved under a v2 key, and lists saved under
v1 become the equivalent rules. Membership is kept per organization,
with the same batching, input limit and ten-poll give-up as before.

Co-authored-by: Poppy <ace4dfe51b23bb4244ab9117dfd5e13b8239445e333225608c352e68cb3fe2b5@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: John Tennant <jtennant@block.xyz>
Nobody uses the list settings yet, so lists saved by an earlier build
of this change fall back to the default rules.

Signed-off-by: John Tennant <jtennant@block.xyz>
Co-authored-by: Poppy <ace4dfe51b23bb4244ab9117dfd5e13b8239445e333225608c352e68cb3fe2b5@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ship

Feedback admitted into the quiet window was told even if the owner
tightened the rules before it was sent, and a membership answer older
than an hour still admitted an author when refreshing it failed.
Waiting feedback is now checked against the current rules on each
poll, and admission only uses membership read within the hour.

Co-authored-by: Poppy <ace4dfe51b23bb4244ab9117dfd5e13b8239445e333225608c352e68cb3fe2b5@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: John Tennant <jtennant@block.xyz>
Use the shared SettingsGroup form layout and its trailing action row,
as other explicit-save Settings forms do.

Co-authored-by: Poppy <ace4dfe51b23bb4244ab9117dfd5e13b8239445e333225608c352e68cb3fe2b5@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: John Tennant <jtennant@block.xyz>
Rechecking waiting feedback kept anything the rules could not yet
decide, so a comment that needed an unread organization membership was
still told once the quiet window passed. It is now unseen again and
read on a later poll, like new feedback that needs a membership.

Co-authored-by: Poppy <ace4dfe51b23bb4244ab9117dfd5e13b8239445e333225608c352e68cb3fe2b5@buzz.block.builderlab.xyz>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: John Tennant <jtennant@block.xyz>
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/poppy/github-author-allowlist branch from 310dece to 32fd8a8 Compare October 29, 2026 05:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant