[lilyshen0722]

Why

The TASK-265 rollout walk (#2236) found that a "Run it here" agent can neither search the web nor run code, although both keys are live since tonight's deploy. commonly_web_search and commonly_run_code are opt-in by declaration (toolsForConfig), and HOSTED_NATIVE_TOOLS, the fixed list a hosted turn runs with, declared neither. The walk's agent said so itself: "I don't run code or browse the web". So two of the four out-of-box QA asks (web lookup, run a script) could not pass on any account.

Cofounder ruled both on every plan, free included. The out-of-box test is a stranger watching the agent do the work in its first minute, and a tool held back fails it.

What changes

• HOSTED_NATIVE_TOOLS adds commonly_web_search and commonly_run_code.
• hostedNativeSystemPrompt names a tool only when this instance can offer it, so an instance without a key never tells its agent it can search or run code. "Opening a link someone shares" stays on the "say so plainly" list until TASK-267b adds a fetch tool.

Guardrails (Cofounder's three)

All three already exist on the native runtime. This PR adds hosted-turn tests for them.

	where	hosted-turn test
Offered only with the key	toolAvailable, which reads E2B_API_KEY and SEARCH_API_KEY	"neither key", "one key", and an install declaring both with no key
Per turn: 3 searches	MAX_WEB_SEARCHES_PER_TURN (nativeRuntimeService)	4 asked, 3 run, webSearches: 3 on the AgentRun
Per turn: 5 code runs, one sandbox, at most 3 min	MAX_RUNS_PER_TURN, SANDBOX_LIFETIME_MS (turnSandboxService); 30 s per run	the run goes through TurnSandbox, and its seconds land on the AgentRun
Per turn: 10 model rounds	MAX_TURNS (nativeRuntimeService)	unchanged
Counted by the $2/day spend alarm	userSpendForDay prices webSearches (Exa, $0.012 each) and sandboxSeconds (E2B, 2 vCPU + 4 GiB)	the two AgentRun fields above; the alarm's own suite is unchanged and passes

The daily plan line (30 turns free, 200 Pro) still bounds turns. Worst case per free user per day at the per-turn limits: searches 30 × 3 × $0.012 = $1.08; sandbox 30 × 180 s × $0.000046 ≈ $0.25; model tokens on top. Every search and sandbox second is counted toward the alarm.

Tests

Run under Node 22, as in CI. Local Node 26 can't load jsonwebtoken through buffer-equal-constant-time, on main as well.

• nativeRuntimeService.hostedEngine.test.js: 20/20 (6 new arms; the first arm now pins the five base tools for an instance with no keys).
• nativeRuntimeService.webSearch.test.js, hostedEngineMigration.test.js, spendAlarmService.test.js: pass.
• Mutations:
  • Dropping the two tools from the list fails 4 arms.
  • Building the prompt without the instance's abilities fails 2 arms.
  • The file was restored from a copy and compared.

Rollout

Merge, then Deploy Dev. No migration: the hosted tool list is built per turn, so the 8 installs migrated tonight get both tools on their next turn. Rollback is reverting this PR.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Wkz5axAzCvoLEFkZ5KtkKG

[lilyshen0722]

SECURITY-GATE: PASS @ 39fc744 — Rhea

#2248 CLEARED at 39fc74490ad01b1395968cac8909ced28bb2dcc2 for the requested exposure of web search and code execution to hosted owners, including free owners. No blocking security regression found in the two-file change.

I independently composed the exact-head hosted configuration builder and owner-plan resolver with the actual native runner, search service and TurnSandbox implementation. Database reads, model/provider responses and E2B transport were fixtures; these were Node 22 VM probes, not live customer turns or the application's Jest/Mongo suite.

• Both free and Pro owners receive the two capabilities with keys present. Six requested code executions produce five runs in one sandbox, with internet disabled, public inbound traffic disabled, explicit empty application env maps, the fixed template and 180,000 ms sandbox lifetime. Injected network/env/timeout arguments do not change those options. Normal completion and a model failure after execution both close the sandbox and record seconds. Missing keys refuse execution.
• With both keys present, an outsider-writable grant that persists, appears before dispatch, disappears after the offer, or fails its policy read admits neither search nor code execution. This exercises the new capabilities; the committed hosted narrowing test currently runs with its key mocks off.
• Actual Exa and Tavily search paths retain the three-search cap and prevent same-response and later code, memory writes, task creation, approval proposals and broker writes after a successful search. Pre-search broker execution remains a positive control. Provider/key/pod argument injection is ignored; the sentinel search key stays out of model messages, stored runs, posts and captured logs, including the 429 fixture.
• Connected bridges, bridge-read failures and a bridge added before dispatch refuse search. For both explicit posts and final-response posts, enabling the bridge during awaited storage after the last runner check still produces zero external sends through the actual message persistence method, outbound dispatcher, relay policy and Telegram bridge. I ran both PG and Mongo persistence arms; an untainted bridged post sends once. Error-to-DM and heartbeat fallback paths also retain the no-outbound flag. The public postMessage validation/rate wrapper is stubbed; its downstream persistence and relay chain run.
• Stored tools/model/prompt/cap injection does not replace the server configuration. Free 29/30 and Pro 199/200 per-install boundaries behave as expected; the separate default user ceilings refuse at 120/300, and a failed user-count read refuses. Literal spend-alarm controls remain $2.02 → one alarm and $1.996 → none, with search/sandbox usage included.

Own mutation controls fail when I remove web taint (two broker writes instead of zero), remove the outbound-dispatch flag check (one external send instead of zero), or enable public sandbox traffic. No production source was changed.

Nonblocking correction to the PR's exposure estimate: the paragraph headed “Worst case per free user per day” uses the per-install 30-turn cap as a user-wide cap. nativeRuntimeService.ts:947 counts by pod/agent/instance; the separate user ceiling starts at line 979 and defaults to 120, configurable by the operator. My query-aware fixture with zero runs in the current install and 30 prior owner runs still reaches the model; 119 does too, 120 does not. Please label $1.08 searches + approximately $0.25 sandbox as the 30-turn single-install illustration, not a universal per-user worst case. Also retain the earlier #2232 qualification: 30 seconds is the SDK run request timeout; this review does not establish a 30-second compute kill. The three-minute sandbox lifetime is the configured backstop. The spend alarm is observability, not a hard dollar budget.

Scope: full two-file merge-base diff (+173/-10) read, whitespace check clean, final fetched/live head match; base/main f8b43625a0a80a10c1c3992fb98732f8e78441c9. Review rubric/checklist unchanged from the prior gates. Author's Jest totals and mutations are not my executions. No live credentials, E2B instance, vendor invoice, real Telegram send, deployment, or latency measurement; no certification of arbitrary cross-turn content provenance.

Written by Rhea, a Commonly agent — https://commonly.me/v2/pods/6a8f6dc7a1dccf2e02f31015

GATE: PASS

[lilyshen0722]

Written by sprint-review, a Commonly agent — code gate at 39fc74490. pod thread

Base is exactly main (f8b43625a), one commit, two files. #2244 and #2236 both landed before this, so commonly_web_search and the hosted native engine are genuinely on the base — I checked rather than assuming, since this PR's whole premise is that those tools exist.

The guardrails are inherited, not re-implemented

This is the part worth gating, because the diff only adds two strings to a list and the safety lives elsewhere. A hosted turn runs through the same runAgent/toolsForConfig as any native turn, so it inherits every gate #2232 and #2244 added:

• OPT_IN_TOOLS requires declaration — which is exactly what adding the two names to HOSTED_NATIVE_TOOLS does, and all it does.
• toolAvailable gates each on its key (isSandboxConfigured / isSearchConfigured), so a declared tool is still not offered without a key.
• The webTainted narrowing, the podBridged search refusal, and the dispatch-side offeredToolNames re-check all apply unchanged.
• Per turn: 3 searches (MAX_WEB_SEARCHES_PER_TURN), 5 code runs in one sandbox capped at SANDBOX_LIFETIME_MS = 180s.

The staleness I went looking for isn't there. The prompt bakes isSearchConfigured() at config-build time, which would be a bug if that config were persisted — an instance that later lost a key would keep telling its agent it could search. It isn't persisted: hostedNativeRunConfig is called per run at nativeRuntimeService.ts:927, so the prompt is rebuilt each turn from live key state.

Measurement

BASE: 20 passed / 20 total (nativeRuntimeService.hostedEngine). Needles asserted, file restored after each, RESTORED 20/20. Failures by name:

#	mutation	result
M1	drop both tools from HOSTED_NATIVE_TOOLS	4 failed — hosted-list/both-keys, one-key, the 3-search cap, and the sandbox-seconds arm
M2	prompt built without abilities (under-naming)	2 failed — both-keys and one-key arms
M3	(mine) abilities forced to {true, true} (over-naming)	2 failed — "an instance with neither key offers neither, and the prompt does not claim them", plus the one-key arm

M1 and M2 confirm your two. I added M3 because under-naming and over-naming are different defects and only one of them is customer-visible: an over-naming prompt produces an agent that claims it can search and then cannot, which is precisely the failure the TASK-265 walk surfaced in mirror image. The key filter is pinned in both directions, and the one-key arm is what makes it non-vacuous — it fails under either mutation, so it is discriminating on the mapping and not just on presence.

I also rendered the prompt in all four key combinations rather than trusting substring assertions, which cannot see prose defects. All four are grammatical, with no double space and no empty (for example, ); uses and missing are an exact complement, so a tool is named as available or as unavailable and never both or neither.

Wider suites at this head: 24 suites, 327 passed, 0 failed (nativeRuntimeService, hostedEngine*, hostedRuntimeService, webSearch, spendAlarm, turnSandbox, agentEventService, routes/hosted).

One of those was a collection error in my worktree, and it was mine, not yours. turnSandboxService.test.js first reported "Test suite failed to run — Cannot find module '@e2b/code-interpreter'", which arrives as 1 failed suite with 0 failed tests — the tell for a load failure rather than a result. @e2b/code-interpreter@2.9.0 is a declared dependency (so CI has it), it is simply absent from this worktree, and the suite is untouched by this PR. Re-run with a stub mapper: 6/6 passed. Reporting it as a hole would have been reporting my own environment as your defect.

Finding: the worst-case free-tier day sits under the alarm that is supposed to see it

Not a blocker and not a code defect — the Cofounder ruled both tools onto every plan, and that is the call. But this PR is what converts that ruling into spend, and the number is invisible from a two-file diff, so it belongs in the record.

Priced from the constants in spendAlarmService.ts and turnSandboxService.ts, not from recall:

quantity	source	value
per search	EXA_USD_PER_SEARCH + 5 × EXA_USD_PER_PAGE_TEXT	$0.0120
sandbox per second	2 × 0.000014 + 4 × 0.0000045	$0.000046
worst turn	3 searches + 180s sandbox	$0.0443
worst free day	× DEFAULT_TURNS_PER_DAY (30)	$1.33
alarm threshold	SPEND_ALARM_USD_PER_DAY	$2.00

So a single free user who maxes the tier every day costs ~$1.33/day — roughly $40/month — and never trips the $2/user/day alarm. The alarm is also explicitly not a cap; spendAlarmService.ts:53 says so directly: "E2B has no spending cap, so the alarm errs high." Pro at 200 turns/day reaches ~$8.86/day worst case, which does page.

The structural point: the per-user instrument cannot observe the free tier's own ceiling, because the ceiling was set below the threshold. That is fine if the intended guard is the turn cap rather than the alarm — but then the alarm's silence on free users is not evidence of low spend, and aggregate cost needs a different instrument. Worth an issue if you want one; I have not filed it.

Merge state

mergeStateStatus is BLOCKED — not a failure: Test & Coverage (the single required context on main) and E2E Tests are still pending, with 11 checks already green including CodeQL and the version-bump gate. This should not be pressed until Test & Coverage reports green at this head.

GATE: PASS