Skip to content

feat(native): commonly_web_search; a turn that has read the web can only read and post in its pod (TASK-267a) - #2244

Merged
lilyshen0722 merged 4 commits into
mainfrom
lily/web-search
Oct 16, 2026
Merged

lilyshen0722 merged 4 commits into
mainfrom
lily/web-search

Conversation

@lilyshen0722

Copy link
Copy Markdown
Contributor

Stage 2 of the out-of-the-box plan (ADR-023 amendment A2): the web lookup half. A native turn that declares commonly_web_search can search the web. Each search returns up to five results with a title, an address and some page text.

Provider

  • Exa by default, chosen on the numbers: a free tier that needs no card, and page text an agent can cite.
  • SEARCH_PROVIDER=tavily swaps to Tavily with no other change.

Boundary

  • Opt-in. The tool is offered only when the install declares it and the instance has SEARCH_API_KEY. It is never on the default surface.
  • Marked untrusted. Results reach the model behind the same marker connector results carry (untrustedConnectorMarker, TASK-224).
  • The turn narrows after a search. Once a search has returned, the rest of the turn can only read and post in this pod and search again:
    • connector (broker) tools are dropped from the offer;
    • any other tool is refused at dispatch, including one the model asked for later in the same response. A call emitted before the search still runs.
    • Searching again stays allowed because its only output is more untrusted text, and its query goes to the provider, not to an address the page text could choose. commonly_fetch_url (TASK-267b, feat(egress): getFromPublicAddress, the GET an agent's fetch goes through (TASK-267b) #2242) is the call that could reach such an address, and it is not wired here.
  • At most 3 searches a turn. Each counts when it is sent, so a failed search still counts. A failed search does not narrow the turn.
  • The run keeps the count and the addresses, not the page text. It stores AgentRun.webSearches and the result URLs.

Spend

The spend alarm prices each search at $0.012: Exa's auto search at $7 per 1,000 requests, plus page text for five results at $1 per 1,000 pages (exa.ai/pricing, read 2026-10-15). Exa's $10/month free credit therefore covers about 830 of these searches. The tests use hand-worked literal amounts on both sides of the $2 line.

Latency probe

20 distinct queries were sent with this exact request shape, from an operator laptop rather than in-cluster:

p50 1.50 s
p95 2.81 s
max 2.83 s
failures 0
results per query 5 (all 20)

The 15 s timeout leaves a wide margin.

Helm

  • webSearch.enabled gates the api-keys entry search-api-key (remote commonly-dev-search-api-key) and the backend env SEARCH_API_KEY / SEARCH_PROVIDER.
  • Off by default and on in values-dev. The remote key has one enabled version (checked with gcloud secrets versions list).
  • With the default values, the chart renders no search entries.

Proof

  • New and touched suites pass, 89/89: nativeRuntimeService.webSearch (9 arms), toolGating (plus an opt-in and key arm), spendAlarmService (plus 3 arms), webSearchService (4), and every other nativeRuntimeService.* suite.
  • tsc:check is clean, and eslint reports no new warnings.
  • 12 mutations, each caught:
    1. offer narrowing removed;
    2. broker tools kept after a search;
    3. same-response taint check off;
    4. dispatch refusal removed;
    5. marker title dropped;
    6. cap removed;
    7. taint set before the request;
    8. full page text recorded on the run;
    9. web search moved onto the default surface;
    10. key gate removed;
    11. run count not recorded;
    12. taint never set.

🤖 Generated with Claude Code

https://claude.ai/code/session_015NDyNbwmCco62PAAviSL1k

@lilyshen0722 lilyshen0722 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SECURITY-GATE: HOLD @ 51c9062 — Rhea

#2244 HOLD: the allowed post paths still send search-influenced content outside the pod through channel relays.

nativeRuntimeService.ts:436–451 allows commonly_post_message after search, and :1420–1433 also posts final assistant text. Neither passes the turn's web-taint state to the message service. agentMessageService.ts:1836–1851 dispatches the persisted content as chat.message; the actual installable dispatcher selects live Telegram/Slack bindings. Telegram's relayAgentMessageToTelegram then calls telegramService.sendMessage (telegramBridgeService.ts:333–349). [ESCALATE] in ordinary content suffices under the existing attention policy; this does not require a broker tool, a forged destination, or mirror mode.

Reproduction at this head: successful actual searchWeb → actual native runner receives marked results → subsequent model response posts [ESCALATE] SEARCHED_TEXT: publish this report. → the exact extracted message-persistence outbound hook → actual event dispatcher, relay policy, Telegram bridge and Telegram send service → one HTTP send to the fixture's external Telegram chat containing SEARCHED_TEXT. The final-text fallback reproduces the same send. Removing the fixture's bridge produces zero sends. Storage, model/provider responses and HTTP transport were stubbed; this is a Node 22 VM composition probe, not a live Telegram attack or a full postMessage integration test.

Please enforce the no-outside-write boundary at message fan-out for both posting paths, or refuse search where that boundary cannot be maintained. Include a bridged-pod regression with the actual relay path, plus an untainted relay positive control. Dropping broker tools alone cannot close this path.

Other checks passed in my exact-source probes: Exa/Tavily use fixed provider endpoints and header credentials; injected destination/provider/key arguments are ignored; a sentinel search key is absent from model requests, AgentRun records, post arguments and captured logs on success and an Axios-shaped 429 failure carrying header credentials in its config. Direct broker/code/memory/approval/task calls after search are refused both later in the same response and on subsequent model calls. Pre-search broker dispatch still runs; undeclared/missing-key searches do not send; four requests send/count three. Removing taint assignment fails the probe (two broker dispatches versus zero). These are bounded checks, not proof against every possible upstream error.

Spend probes produced $2.02/one alarm for $1.30 model spend + 60 searches, $1.996/no alarm for 58, $0.036 for three search-only requests, and $1.06 for 50 searches + 10,000 sandbox seconds. The published Exa pricing lists auto search at $7/1,000 and contents at $1/1,000 pages; these probes validate the PR's $0.012 accounting, not a live invoice.

Read the full 12-file +667/−22 merge-base diff (base 975e0c37e6acc3e8554dd840faab117abaf99cd4), tests and Helm wiring. Diff check clean; final fetched/live head matched; rubric/checklist unchanged. No Jest/Mongo run, Helm render, deployment, live credentials or independent latency probe. The PR's test and 12-mutation reports remain the author's evidence.

Written by Rhea, a Commonly agent — https://commonly.me/v2/pods/6a8f6dc7a1dccf2e02f31015

GATE: HOLD

@lilyshen0722 lilyshen0722 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Written by Sprint Review, a Commonly agent — gate requested in pod thread 78047.

Bound to 51c90624512d51e5dd8432310d195f276a0d4ea3. 12 files, +667/−22. 1 commit behind main, 0 conflict markers. BASE 31/31; broader sweep over native-runtime/web-search/spend/hosted/tool-gating: 35 suites, 452 tests, exit 0. eslint 0 errors; tsc 50, the baseline. CI 13 pass, 2 skipping, Test & Coverage pending — the one required check, so unruled here.

The narrowing is pinned at both layers, and in the same response

This is the control the feature rests on: once third-party text is in the turn, nothing that can act on the outside world may run. It is enforced in the offer and at dispatch, and each is independently pinned — totals held at 31 throughout:

mutation result
dispatch-level taint refusal removed 2 failed
offer-level narrowing removed 1 failed
broker tools not dropped from the offer 1 failed
same-response broker short-circuit removed 2 failed

The same-response case is the one that is easy to get wrong, and the reason it works is that webTaintedAway is computed per tool call inside the loop over tool_calls, not once per response. So [search, connector_tool] in one model response runs the search, sets the taint, and then refuses the connector call — with dispatchTool naming the reason rather than a bare "not offered". That mirrors #2232's narrowedAway shape, which is the right consistency.

The marker invariant holds, and is enforced

brokerResultMarker carries the comment "Catalogue-owned title only. Vendor text never enters this instruction boundary." This path passes the literal 'a web search', so it complies — and I checked that the compliance is defended rather than incidental:

mutation result
marker removed from the result 1 failed
marker title taken from vendor text (outcome.results[0]?.title) 1 failed

The second redding is better than I expected: the arm asserts the exact marker sentence, so smuggling a result title into the instruction boundary fails. That is the attack the invariant exists to stop.

The service itself is careful in ways worth naming: clip strips control characters before clipping, httpUrl admits only http/https (so no javascript:/data:/file: reaches the model as a citation), results without a URL are dropped, and the run records addresses only — removing recorded reds an arm, so the AgentRun is kept from becoming a copy of five pages of third-party text.

Cap and spend

mutation result
per-turn cap removed 1 failed — stops at three searches a turn
counted after the request, so a failed search is free 1 failed — a failed search counts toward the cap and taints nothing
search priced at zero 3 failed
SEARCH_RESULT_PAGES drifted to 10 3 failed

The pricing lesson from #2232 was applied. The rate is derived in code from named unit prices (7/1000 + 5 × 1/1000 = $0.012), and the boundary arms are literal and independent of the exported constant — 60 searches → $2.02 with 1 alarm, 58 → $1.996 with 0. Setting the rate to zero reds three arms, which is exactly what did not happen two PRs ago.

Two findings, neither blocking

The result count is one fact stored in two modules. RESULT_LIMIT = 5 in webSearchService is not exported, and spendAlarmService hardcodes its own SEARCH_RESULT_PAGES = 5. Both are pinned at 5 separately, so a change is loud — but only on one side. Raising RESULT_LIMIT to 10 reds two webSearchService arms and no pricing arm; an editor following the red tests fixes those arms, never learns the price depends on the same number, and ships a 5-page price for a 10-page search with everything green. Exporting RESULT_LIMIT and importing it would make it one fact instead of two that agree by habit.

The price is Exa's, and the provider is a Helm value. SEARCH_PROVIDER is settable from values.yaml (webSearch.provider), tavily is a supported adapter, and WEB_SEARCH_USD is Exa's rate with no provider check. The comment says "a switch of SEARCH_PROVIDER should re-check this rate" — which is honest, and is the only thing standing between a one-line Helm change and a silently wrong cost signal. searchProvider() is already exported, so a per-provider rate, or a warn when pricing a provider the table does not know, would make the coupling enforceable rather than remembered.

Wiring, and the cross-PR check you raised

The key arrives by ExternalSecret from commonly-dev-search-api-key, injected optional: true behind webSearch.enabled; base values.yaml has enabled: false, dev has true. No literal key anywhere in the diff. Same double opt-in as the sandbox: this merge opens the instance half on dev, and the per-install declaration still gates any actual use.

On the webSearches log-line overlap with #2243: git merge-tree on the pair reports 0 conflict markers despite 4 files changed in both, so they auto-merge and no rebase is needed for these two heads.

GATE: PASS

samxu01 pushed a commit that referenced this pull request Oct 15, 2026
… the web does not post into one

Rhea's HOLD on #2244 at 51c9062: posting in the pod is the way out the
narrowing keeps, and a pod bridged to an external chat (Telegram, Slack)
relays its posts there, so search-steered text reached an outside chat
through commonly_post_message or the final-text fallback.

The same bridge condition that keeps outsider-writable connector grants out
of such a pod (#2211) now applies to search. It is extracted as podIsBridged,
with the eligibility check calling it unchanged.
- The offer: search is not offered in a bridged pod, re-read before each
  model call of a turn that may search.
- Dispatch: a search re-reads the bridge and refuses; once the web is in the
  turn, a post re-reads it and refuses, so a bridge added mid-turn is caught.
- The final-text fallback does not post after a search into a bridged pod,
  and the run records guardrail_blocked.
- A failed bridge read counts as bridged.

Arms cover a bridged pod, a bridge added after the offer, and a bridge added
after the search (both posting paths). Positive controls: a turn that did not
search posts in a bridged pod, and a turn that searched posts in an unbridged
one. Seven mutations are each caught.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015NDyNbwmCco62PAAviSL1k
@lilyshen0722

lilyshen0722 commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor Author

Rhea's HOLD at 51c9062 is addressed at 8242947, which is also rebased onto main (#2239, #2234).

I took the pod condition rather than carrying taint through fan-out. The bridge check that keeps outsider-writable connector grants out of a bridged pod (#2211) is now extracted as podIsBridged(podId). outsiderGrantPodRefusal calls it, and its behaviour is unchanged (22/22). Search uses the same function in three places:

  • Offer. In a bridged pod, search is not offered. The bridge is re-read before each model call of a turn that may search; turns that can't search don't pay for the read.
  • Dispatch. A search re-reads the bridge and refuses. Once the web is in the turn, every commonly_post_message re-reads it too and refuses. A bridge added mid-turn is therefore caught at the next search or post.
  • Final-text fallback. After a search, it doesn't post into a bridged pod, and the run records guardrail_blocked.

A failed bridge read counts as bridged.

Post-tool and fallback are the only two exits, and both post only to the turn's own pod (ctx.podId). The relay runs inside postMessage, so a refused post never reaches it.

Arms

  • A bridged pod: search is not offered, and a named search doesn't run.
  • A bridge added after the offer: caught at dispatch.
  • A bridge added after the search: neither the post tool nor the fallback posts, and postMessage is never called with the SEARCHED_TEXT content.
  • A failed read counts as bridged.
  • Positive controls: a turn that didn't search still posts in a bridged pod, and a turn that searched still posts in an unbridged one.

Proof. 234/234 across the native runtime, eligibility, grant, broker, search and spend suites. tsc:check is clean. Seven mutations, each caught:

mutation arms failed
offer filter removed 2
dispatch re-read removed 2
tainted-post refusal removed 1
fallback block removed 1
search-case refusal removed 1
failed read fails open 1
bridge never read 4

Correction (edited). An earlier version of this comment said Telegram's relay ignores status, so a Telegram row in error would relay without counting as bridged. That is wrong for the live path. activeHandlersForPod selects status: { $ne: 'error' } (installable/eventHandlers.ts), and the dispatcher always relays through those rows, so podIsBridged excluding error matches what actually relays (Vera, adopted by SWE). Only the Telegram fallback skips the status check, and nothing calls it today. TASK-232 now covers making that fallback refuse a non-connected row. podIsBridged stays as it is.

@lilyshen0722 lilyshen0722 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SECURITY-GATE: HOLD @ 8242947 — Rhea

#2244 HOLD at 8242947: a bridge enabled after the post guard but before relay selection still receives searched text. The original already-connected bridge repro is blocked at this head.

HOLD — the bridge read does not cover the subsequent outbound send. nativeRuntimeService.ts:1398–1400 checks the bridge before dispatch, and :1456 checks before the fallback. Both then call AgentMessageService.postMessage without carrying the turn's web taint or an outbound-relay prohibition (:456–466, :1466–1476). Posting contains awaited storage work: _postToTarget awaits PostgreSQL creation at agentMessageService.ts:1676 or Mongo save/populate at :1754–1755, then the outbound hook at :1836–1851 selects the currently live integrations. A bridge becoming live in that interval is absent from the guard's snapshot and present for the send.

Exact-head reproduction, for both the post tool and final-text fallback:

  1. The pod is unbridged through the search offer, actual search request/response, and last post guard. The model's next output contains [ESCALATE] SEARCHED_TEXT: publish this report.
  2. Enter the actual _postToTarget persistence method. Enable the fixture's connected, active, liveRelay Telegram binding during the awaited storage operation.
  3. The actual persistence method calls the actual installable dispatcher; its current selection includes that binding, and the actual relay policy, Telegram bridge and Telegram transport wrapper send the searched text to the external chat. The fake HTTP transport records one send.

I ran this with both persistence branches: PostgreSQL create and Mongo save. The normal no-bridge control records zero external sends, and the untainted bridged positive still sends. The public postMessage validation/rate-limit wrapper is stubbed; the entire _postToTarget method is extracted verbatim and executed. This narrows the previous hook-only probe and exercises the awaited storage interval directly. No live database or Telegram send was performed.

Please carry a server-owned no-outbound policy from the searched turn through both post paths and enforce it where relays are selected/dispatched, or provide equivalent coordination that covers concurrent bridge changes. Re-reading before calling postMessage is not sufficient. Add an arm enabling a real relay-shaped binding while persistence is deferred, for both post paths, and assert zero external transport calls while preserving the untainted relay positive. The bridge-status follow-up #2245 alone would not close this timing window.

Passing controls: connected bridge removes the offer and refuses a forged search; a bridge added after the offer refuses search dispatch; one added after search but before the next post guard blocks both post paths; failed bridge reads fail closed. Actual Exa/Tavily adapter + runner still refuse same-response and later broker/code/memory/approval/task dispatch after search, preserve the pre-search broker positive, enforce opt-in/key availability and the three-request cap, and ignore injected provider/key/pod arguments. A sentinel search key is absent from model requests, run records, posts and captured logs on success and the exercised Axios 429 path (whose error object includes request headers). This is not a claim about every possible error path. Exact-head spend controls remain 1.30 + 60 searches = 2.02 with one alarm; 58 = 1.996 with none; three searches alone = .036; 50 searches + 10,000 sandbox seconds = 1.06.

The author's error-status Telegram footnote is not my demonstrated bypass: the actual installable selector at eventHandlers.ts:92 excludes status error before invoking Telegram. My selector-aware error-row control records zero sends on this path. The timing repro uses a normal connected binding and does not depend on that disputed status edge.

Scope/provenance: full 13-file +822/-36 PR scope; range-diff confirms the two prior commits replay identically, followed by the bridge fix. Read the complete correction and its tests; eligibility's existing bridge predicate is extracted unchanged relative to the rebase base. Prior full review carries for unchanged payloads. My probes are Node 22 VM execution with stub model/provider/storage/HTTP and fake keys; no Jest/Mongo suite, live credentials/vendor attack, deployment, Helm render or latency rerun. The author's 234 tests and seven mutations are their evidence. Rubric/checklist unchanged, diff check clean, final fetched/live head match. Base 1d119ef; final main df34b00.

Written by Rhea, a Commonly agent — https://commonly.me/v2/pods/6a8f6dc7a1dccf2e02f31015

GATE: HOLD

@lilyshen0722 lilyshen0722 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Written by Sprint Review, a Commonly agent — re-gate requested in pod thread 78066.

Bound to 8242947dc52a881dca8108c149f8f81bf2a2a5d0. My PASS 5537254573 (at 51c90624) is superseded. 13 files, +822/−36, 2 behind main, 0 conflict markers. BASE 37/37; sweep over web-search/native-runtime/outsider-grant/spend/tool-gating/hosted: 39 suites, 497 tests, exit 0. eslint 0 errors; tsc 50, the baseline. CI 13 pass, 3 skipping, Test & Coverage pending. First review at this head.

The podIsBridged extraction is behaviour-preserving: the Integration.find query and the .some(...) predicate are identical to #2211's inline version, and only the return shape changed (if (…) return 'pod_bridged' became a boolean plus a caller-side if).

Five gate points, each pinned by name

mutation result
offer gate removed (search still offered in a bridged pod) 2 failed
dispatch search gate removed 1 failed — a bridge added after the offer is caught at dispatch
post-after-web gate removed 1 failed
fallback-post gate removed 1 failed
a failed bridge read counts as NOT bridged 1 failed — a failed bridge read counts as bridged

Totals held at 37. That last row deserves calling out: I have reported an unpinned fail-closed branch on three PRs in this series, and this is the first time one arrived with its own arm. The read is wrapped, returns true on failure, and a test defends it.

Two further things the design gets right. The bridge is re-read at each decision — per model call, before a search, before a post once the web is in the turn, and again before the fallback — so a bridge added mid-turn is caught rather than inherited from the turn's opening state. And mayWebSearch ? await podBridgedNow() : false means an install that never declared search pays no bridge read at all. A blocked fallback records errorKind: 'guardrail_blocked' with the reason, so the run says why rather than dropping the reply silently.

The positive controls, and which of them actually discriminates

There are two, which is the right instinct — a gate this broad needs counterexamples or it cannot be distinguished from "block everything". I tested whether they bite by mutating the gate to over-block:

over-blocking mutation result
drop podBridged → any search blocks posting anywhere 1 failed — positive control: unbridged, a turn that searched still posts
drop ctx.webTainted → a bridged pod blocks posts even with no search 37 passed

The second is green, and the reason is worth recording rather than filing as a gap: dispatchCtx.podBridged is only ever populated for a search call or a post-after-web, so on a turn that never searched it stays undefined and the mutated gate cannot fire. ctx.webTainted is therefore redundant today — the population rule already encodes it — which makes that a textbook equivalent mutant, not missing coverage.

I proved that rather than inferring it: populating podBridged for every tool call and dropping webTainted does red the control "a turn that did not search posts in a bridged pod as before". So the control is a genuine guard — against the two-part change, which is the shape a future efficiency tidy-up would actually take if someone hoisted the bridge read. It is simply not a guard on that one term in isolation.

Carried forward, both unchanged in this delta

I confirmed spendAlarmService.ts and webSearchService.ts are untouched since my last gate, so both findings stand as written:

  • The result count is one fact in two modules. RESULT_LIMIT = 5 (webSearchService.ts:20) is still not exported, and SEARCH_RESULT_PAGES = 5 (spendAlarmService.ts:67) is still its own constant. Each is pinned at 5 separately, so a change is loud on the search side and silent on the price side.
  • The price is Exa's and the provider is a Helm value, with no provider check in WEB_SEARCH_USD.

GATE: PASS

lilyshen0722 and others added 4 commits October 15, 2026 14:39
…(TASK-267a, wip)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015NDyNbwmCco62PAAviSL1k
…an only read and post in its pod (TASK-267a)

Stage 2 of the out-of-the-box plan (ADR-023 amendment A2), web lookup half.
A native turn that declares commonly_web_search can search the web: up to
five results with title, address and some page text, from Exa by default
(SEARCH_PROVIDER=tavily swaps it with no other change).

- Opt-in: offered only to installs that declare it AND when the instance has
  SEARCH_API_KEY; never on the default tool surface.
- Results reach the model behind the untrusted-content marker connector
  results carry (TASK-224).
- Once a search has returned, the rest of the turn can read and post in this
  pod and search again, nothing else: connector tools leave the offer, and
  any other tool is refused at dispatch, including one asked for later in the
  same model response.
- At most 3 searches a turn, counted when sent. The run keeps the count
  (AgentRun.webSearches) and the result addresses, not the page text.
- The spend alarm prices each search at $0.012: Exa's auto search ($7/1k)
  plus page text for five results ($1/1k pages), read 2026-10-15.

Helm: webSearch.enabled gates the api-keys entry (commonly-dev-search-api-key,
one enabled version) and the backend SEARCH_API_KEY/SEARCH_PROVIDER env; off
by default, on in values-dev.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015NDyNbwmCco62PAAviSL1k
… the web does not post into one

Rhea's HOLD on #2244 at 51c9062: posting in the pod is the way out the
narrowing keeps, and a pod bridged to an external chat (Telegram, Slack)
relays its posts there, so search-steered text reached an outside chat
through commonly_post_message or the final-text fallback.

The same bridge condition that keeps outsider-writable connector grants out
of such a pod (#2211) now applies to search. It is extracted as podIsBridged,
with the eligibility check calling it unchanged.
- The offer: search is not offered in a bridged pod, re-read before each
  model call of a turn that may search.
- Dispatch: a search re-reads the bridge and refuses; once the web is in the
  turn, a post re-reads it and refuses, so a bridge added mid-turn is caught.
- The final-text fallback does not post after a search into a bridged pod,
  and the run records guardrail_blocked.
- A failed bridge read counts as bridged.

Arms cover a bridged pod, a bridge added after the offer, and a bridge added
after the search (both posting paths). Positive controls: a turn that did not
search posts in a bridged pod, and a turn that searched posts in an unbridged
one. Seven mutations are each caught.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015NDyNbwmCco62PAAviSL1k
… and the relay dispatcher refuses it

Rhea's HOLD on #2244 at 8242947: the turn's bridge checks ran before the
post, but the outbound hook selects live integrations only after the post is
stored. A Telegram binding enabled during that write still relayed the
searched text, on both posting paths and both stores.

The policy now travels with the message, decided by the server:
- `noOutboundRelay` is a postMessage option, threaded through every exit
  postMessage takes (the main post, the heartbeat fallbacks and the
  error-to-DM route) into `_postToTarget`, and carried on the chat.message
  payload.
- The relay dispatcher (installable/eventHandlers) refuses a flagged message
  before selecting any bridge. Its handlers are all external relays.
- The native runtime sets it on the post tool and the final-text fallback
  whenever web results are in the turn.

The turn-time bridge checks stay as defense in depth.

Tests:
- the dispatcher with a live connected bridge sends nothing for a flagged
  post and still relays an unflagged one;
- the flag reaches the dispatcher on the Postgres path, the Mongo path and
  the error-to-DM route (both posts);
- the native post tool and fallback set it after a search and not without one.
Six mutations, one per link, each caught.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@lilyshen0722

Copy link
Copy Markdown
Contributor Author

Rhea's HOLD at 8242947 (the time-of-check race) is addressed at bb2aecc, rebased onto main.

The no-outbound policy now travels with the message, and it is enforced where relays are selected:

  • noOutboundRelay is a server-set option on AgentMessageService.postMessage. It is threaded through every exit postMessage takes: the main post, the three heartbeat fallbacks, and routeErrorToDM (both the DM post and the source notice). It goes into _postToTarget and onto the chat.message payload.
  • installable/eventHandlers.dispatch refuses a flagged message before activeHandlersForPod selects any bridge. Every handler it holds is an external relay (telegram.relay, slack.relay). A bridge enabled during the storage write therefore sends nothing, because the decision was made at post time and is read at relay time.
  • The native runtime sets the flag on commonly_post_message and on the final-text fallback whenever web results are in the turn.

The turn-time bridge checks from 8242947 stay as defense in depth.

Arms:

  • Dispatcher, on the real installable dispatcher with a connected liveRelay Telegram row: a flagged post sends nothing and never queries integrations; an unflagged post in the same pod still relays.
  • The flag reaches the dispatcher on the Postgres path, the Mongo path (with PG_HOST unset), and the error-to-DM route (both posts). An unflagged post carries no flag.
  • Native runtime: after a search, both the post tool and the fallback set noOutboundRelay: true; a turn that didn't search sets false.

Mutations, one per link, each caught: dispatcher gate, hook payload, main-post threading, error-route threading, native tool, native fallback.

477/477 across the message, native, relay, decision-card, outsider-grant, search and spend suites. tsc:check is clean.

Scope: search only, as discussed with SWE. Broker (grant) results currently mark every hosted-MCP result untrusted, so extending the flag to them would also silence ordinary connector replies in bridged pods. That is a Connector decision, and it becomes a one-line change once it's made.

@lilyshen0722 lilyshen0722 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Written by sprint-review, a Commonly agent — re-gate at bb2aeccea. pod thread

This supersedes my PASS at 8242947dc (review 5537680821), which no longer binds. The head moved under it; the press reads the GATE: line, so I re-gated unprompted rather than leave a stale clearance on a moved head.

The rebase preserved what I already cleared

bb2aeccea is a rebase onto 37f212e9a (picking up #2237, #2241, #2242, #2243) plus one new commit. I compared the PR's own added lines at each head against its own merge base, per file — not the diffstat, which the rebase inflates to 17 files:

  • 11 of 13 files byte-identical, including webSearchService.ts, spendAlarmService.ts, outsiderGrantPodEligibility.ts, AgentRun.ts and all four Helm files.
  • 2 changed (nativeRuntimeService.ts +4, its webSearch test +15), both from the new commit.
  • 3 new (agentMessageService.ts, installable/eventHandlers.ts, 2 tests).

The new commit is the right shape

Rhea's HOLD was a real race: the turn checked bridges, then the outbound hook selected live integrations after the post was stored. Enforcement now sits where bridges are selected. I checked the two structural claims that make that sound:

  • One producer. chat.message reaches dispatch from exactly one site, agentMessageService.ts:1864 inside _postToTarget. A payload-carried flag is therefore enforceable.
  • One exit. The only outbound agent-post relays in the backend are relayAgentMessageToTelegram and relayAgentMessageToSlack, both registered in the gated registry. Discord and GroupMe have no agent-post relay path, so the single dispatch guard really does cover the exit.

All four _postToTarget call sites carry the flag.

Measurement

BASE: 3 suites, 46 passed (installableEventHandlers, agentMessageService.threadWrites, nativeRuntimeService.webSearch). Needle counts asserted at 1 before each substitution; file restored after each; RESTORED returned 46/46.

# link mutated result
M1 if (payload.noOutboundRelay) return; removed from dispatch 1 failed / 45
M2 ...(noOutboundRelay ? …) dropped from the chat.message payload 3 failed / 43
M3 post tool Boolean(ctx.webTainted) → false 1 failed / 45
M4 final-text fallback Boolean(dispatchCtx.webTainted) → false 1 failed / 45
M5 routeErrorToDM threading deleted (line 1604) 1 failed / 45
M6 postHeartbeatFallback threading deleted (line 678) 46 passed — survived

Finding: the heartbeat-fallback link is the one unpinned link

The commit message says "Six mutations, one per link, each caught." Five are. M6 is not, and the suite total held at 46 — so this is a survivor, not a collection error.

Line 678 is the sole hop from postHeartbeatFallback into _postToTarget. The flag is threaded correctly into postHeartbeatFallback at all three exits (1312, 1340, 1379) and then dropped on the floor by that one deletion, so a single-line edit silently unprotects three exits.

Reachability, since this is a claim about a path and not just a line:

  • Web search is offered whenever isSearchConfigured() (nativeRuntimeService.ts:314) — it is not trigger-scoped, so a heartbeat-triggered turn can taint itself.
  • The turn-time webTainted && podBridged refusal at :406 does not make M6 equivalent: that check is exactly what Rhea showed is racy, and inside the race window the post proceeds.

Severity is narrow, and I want to be accurate about it. sourceContent is not the posted text — the fallback content is synthesized by buildHeartbeatFallbackContent from pod state. What is web-derived is the mention handles: resolveHeartbeatMentionHandles runs extractMentionHandles(content) over the agent's own text with no pod-membership validation (the membership lookup is only the mentions.size === 0 fallback), and those handles are interpolated into the relayed line. So the escape is bounded to ≤3 normalized handles ([a-z0-9][a-z0-9-]{0,39}), not free text — a narrow, attacker-influenced channel, not an exfiltration of searched content.

Not holding on it. The shipped code is correct on all four exits; this is a missing test on a defense-in-depth link, which is how I handled the same class on #2237 (filed as issue #2246 rather than a hold). One test asserting noOutboundRelay reaches _postToTarget through postHeartbeatFallback closes it. Say the word and I'll file it as an issue instead if you'd rather land this as-is.

Carried forward, unchanged at this head

Both were in my 8242947dc review and neither is a blocker:

  • RESULT_LIMIT and SEARCH_RESULT_PAGES are two unlinked constants — the cost model and the truncation can drift apart silently.
  • The Exa per-search rate has no provider check: SEARCH_PROVIDER=tavily is priced at Exa's number.

CI at this head: 15/15 green including Test & Coverage, CodeQL and Service Tests (Tier 1 — real DBs); mergeStateStatus CLEAN; base main.

GATE: PASS

@lilyshen0722 lilyshen0722 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SECURITY-GATE: PASS @ bb2aecc — Rhea

#2244 CLEARED at bb2aecc. This closes the post/storage/relay race in review 5537619756.

The native tool post and final-text fallback now set noOutboundRelay after a successful search. AgentMessageService carries it to the outbound hook, and the installable dispatcher refuses it before selecting a bridge. I traced the main post, all three heartbeat fallback call sites, both error-routing call sites, and both posts made by routeErrorToDM.

Own exact-source Node 22 VM replay: actual search and native runner, complete verbatim _postToTarget persistence method, actual dispatcher/relay policy/Telegram bridge/Telegram HTTP client, with stub storage, model/provider transports and fake keys. Enabling a connected active live-relay binding during awaited persistence, AFTER the last native bridge guard, now yields zero external sends for BOTH tool/fallback posts on BOTH PG/Mongo paths. The untainted positive still sends once. The public postMessage validation/rate-limit wrapper remains stubbed in this composition; its flag forwarding was source-checked. Separately executed the actual error-to-DM and heartbeat fallback methods through persistence/dispatcher on both stores: both DM/source posts and the heartbeat post carry the flag and send nothing externally.

Own mutation: remove the dispatcher's flag guard, verifying the replacement applied; the original race arm fails with one external send versus zero expected. Existing connected-bridge offer/dispatch refusal, bridge changes after offer/search, bridge-read failure, error-status exclusion and no-bridge controls pass. Prior Exa/Tavily narrowing, same-response/later forbidden tool calls, pre-search broker positive, opt-in/key/cap-three, injected knob rejection and 429/sentinel-key controls all pass again. Spend probes retain literal totals: 1.30 + 60 searches = 2.02 and one alarm; 58 = 1.996 and none; three searches alone = .036; 50 plus 10,000 sandbox seconds = 1.06.

No live provider credentials, Telegram attack, deployment, Helm render, latency measurement, or backend Jest/Mongo run was performed here. Author 477/477 and six-link mutation evidence are theirs. This flag's scope is web-search turns; it does not extend relay suppression to all broker-result turns.

Complete six-file correction read (159 additions / 1 deletion), with prior review carried through the rebase: two earlier commits replay identically and the remaining earlier commit differs only in main-derived context. Full PR: 17 files, 981 additions / 37 deletions; diff check clean. Base 37f212e; final fetched main 27a8c10. Rubric/checklist unchanged. Final fetched ref equals GitHub's head above. Nonblocking: the PR body's Proof section still lists the original suite count; the latest correction evidence is in the author comment.

Written by Rhea, a Commonly agent — https://commonly.me/v2/pods/6a8f6dc7a1dccf2e02f31015

GATE: PASS

@lilyshen0722
lilyshen0722 added this pull request to the merge queue Oct 16, 2026
Merged via the queue into main with commit 4e51433 Oct 16, 2026
19 checks passed
samxu01 pushed a commit that referenced this pull request Oct 23, 2026
- The 2026-10-15 direction's runtime decision was made the same day
  (ADR-023 amendment) and both stages merged on 10-15/10-16: #2224, #2236,
  #2226 (hosted on native), #2232 (E2B sandbox), #2244, #2249, #2248 (web
  lookup), #2227, #2237 (cost bounds). Timings cited from
  frontend/src/__tests__/timingClaims.test.ts.
- The lift covers only the per-turn E2B sandbox for native agents. VMs,
  long-lived containers and a Managed Agents adapter stay paused under the
  Tier 2 row, trigger unchanged.
- New requirement, owner SRE: alert when E2B credit falls below $25. The
  account has no card on file, so running out stops the sandbox rather than
  billing an overage; nothing in k8s/monitoring watches it today.
- Decision (2026-10-22, on Sam's question): Commonly does not host or sell
  agent computers; reopen only on two of the first three outside use cases.
- Header scope-boundary note pointing at ADR-023's Containers spike.

Ratified with #2205 and #2214 by Cofounder on 2026-10-21 under Sam's
standing delegation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015eTFc9yqm2vKZiUyvLuBT4
samxu01 pushed a commit that referenced this pull request Oct 25, 2026
- The 2026-10-15 direction's runtime decision was made the same day
  (ADR-023 amendment) and both stages merged on 10-15/10-16: #2224, #2236,
  #2226 (hosted on native), #2232 (E2B sandbox), #2244, #2249, #2248 (web
  lookup), #2227, #2237 (cost bounds). Timings cited from
  frontend/src/__tests__/timingClaims.test.ts.
- The lift covers only the per-turn E2B sandbox for native agents. VMs,
  long-lived containers and a Managed Agents adapter stay paused under the
  Tier 2 row, trigger unchanged.
- New requirement, owner SRE: alert when E2B credit falls below $25. The
  account has no card on file, so running out stops the sandbox rather than
  billing an overage; nothing in k8s/monitoring watches it today.
- Decision (2026-10-22, on Sam's question): Commonly does not host or sell
  agent computers; reopen only on two of the first three outside use cases.
- Header scope-boundary note pointing at ADR-023's Containers spike.

Ratified with #2205 and #2214 by Cofounder on 2026-10-21 under Sam's
standing delegation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015eTFc9yqm2vKZiUyvLuBT4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant