Skip to content

feat(hosted): the plan line — 30 hosted turns a day free, 200 on Pro (TASK-268) - #2237

Merged
lilyshen0722 merged 2 commits into
mainfrom
lily/hosted-plan-line
Oct 15, 2026
Merged

lilyshen0722 merged 2 commits into
mainfrom
lily/hosted-plan-line

Conversation

@lilyshen0722

@lilyshen0722 lilyshen0722 commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

What

The plan line for hosted agents (TASK-268, Stage 2), as Cofounder ruled on 2026-10-15: 30 hosted turns a day on free and individual accounts, 200 on Pro.

  • hostedCaps({ pro }) returns the line for a plan. Called without a plan it returns the free caps, so a limit never defaults to the paid line. Both numbers stay env-tunable (HOSTED_TURNS_PER_DAY, HOSTED_TURNS_PER_DAY_PRO).
  • hostedCapsForOwner(userId) reads the owner's entitlements.pro. Two authorized paths write it: the signed Stripe webhook, and an instance admin's entitlements PATCH (how pilots and design partners get Pro). No client field sets it. A missing user or a failed lookup reads as free.
  • Every hosted limit follows the owner:
  • The per-user ceiling for Pro owners rises to 300 (AGENT_USER_DAILY_RUN_CEILING_PRO). Free stays 120. Without this, a Pro owner's 200 hosted turns would be cut off at 120 by the shared per-user ceiling. This is a consequence of the ruling, not part of it; flagging it for Cofounder. At DeepSeek prices it bounds a Pro user at roughly $1.20–$2 of model spend a day, and the $2 spend alarm (feat(spend): page when one user's agent spend passes $2 in a UTC day (TASK-263) #2227) watches it.
  • The quickstart states both lines, and the docs fact test pins both numbers to the code.

Behaviour change

Free accounts go from 200 hosted turns a day to 30. Measured over 14 days of production, the heaviest user-day was 28 runs, so the line is above real use.

Tests

  • Caps: free by default, Pro on request, garbage env falls back.
  • Native engine: a free owner runs at 29 and stops at 30; a Pro owner runs at 30 and stops at 200; a Pro owner past 120 user runs still runs, while a free one gets user_ceiling.
  • Routes: availability reports the caller's own caps.
  • Docs: the fact test pins "30 turns a day" and "200 turns a day on Pro".
  • Mutations, each caught: the caps ignoring Pro; the ceiling ignoring Pro.
  • Wider run: 52 related suites, 676 tests (1 skipped). Lint and tsc 0.

🤖 Generated with Claude Code

https://claude.ai/code/session_015NDyNbwmCco62PAAviSL1k

@lilyshen0722 lilyshen0722 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SECURITY/DOCS-GATE: HOLD @ eafe96d — Rhea

One trust-contract correction is required: backend/services/hostedRuntimeService.ts:50 and the PR description say entitlements.pro is written only by the signed Stripe webhook. The existing PATCH /api/admin/users/:userId/entitlements explicitly accepts a boolean pro and saves it (backend/routes/admin/users.ts:180–217). It has not been retired now that billing exists.

At this exact head I executed the actual admin guard, entitlement handler and new cap resolver against stub persistence: a non-admin was refused with 403; an admin passed; {pro:true} returned 200; hostedCapsForOwner immediately returned 200 turns with no billing call. Thus the answer to the requested “anything other than the signed webhook?” check is yes: the trusted admin grant route. This is an existing authorized administrator capability, not an ordinary-user escalation or a new bypass introduced by this PR. Preserve it if intentional and correct the new source comment/PR claim to say Pro is a server-owned entitlement set by verified billing or authorized admins, with a regression arm for that boundary. If billing-only is actually required, close the admin Pro write instead. Do not ship the billing-only assertion while the admin writer remains.

The other requested security and limit checks passed in my exact-source Node 22 VM probes:

  • Invalid/missing/non-24-hex owner ids return free before querying. Missing users, query errors, absent/false/non-boolean pro return free; only literal true selects Pro. A Pro→free database change is reflected on the next call: this helper has no cache.
  • The actual profile update handler ignored nested, dotted and operator-shaped entitlement/billing input while accepting an ordinary bio edit. Registration/OAuth creation construct their own entitlement object; ordinary profile updates use an allowlist. The admin guard reads the caller's role from Mongo. The billing route verifies constructEvent before handleEvent, and server mounting preserves raw webhook bytes. Those billing-path checks were source inspection, not a real Stripe signature exercise.
  • Actual native runner: free 29 runs executes, 30 does not; Pro 30 executes, 200 does not. At 150 owner-wide runs Pro executes and free fails user_ceiling; Pro fails at 300. Failed plan lookup at 30 produces no model call. Actual DO meter distinguishes free/Pro at 30. Garbage environment values fall back to 30/200, and zero retains the hosted decline.
  • Availability uses the authenticated caller; provision/status resolve an installation owned by that caller; native and DO execution take installedBy. The quickstart's two literal lines match the defaults, and the changed fact test checks both.

Scope: all ten changed files (+124/−26) read, merge base a018f7b, diff check clean, final fetched/live head matches above; rubric/checklist unchanged. Stub DB/model transport only: no Jest/Mongo run, real billing event, deployed cap measurement, or docs rendering by me. The reported 52 suites/676 tests and mutations are author evidence. Reading a fresh entitlement does not independently certify the upstream billing event reconciliation logic. The 300 Pro ceiling is implemented as reported; this review is not a business-policy ruling on its amount.

Written by Rhea, a Commonly agent — https://commonly.me/v2/pods/6a8f6dc7a1dccf2e02f31015

GATE: HOLD

samxu01 pushed a commit that referenced this pull request Oct 15, 2026
Rhea's HOLD on #2237 (5536742280): the PR and its comments said only the
signed Stripe webhook makes an owner Pro, but an instance admin's
entitlements PATCH also grants it (how pilots and design partners get Pro),
and she confirmed it through the handler and the cap resolver. Admin-granted
Pro is meant to lift the hosted line, so the code stands and the wording is
corrected: hostedRuntimeService, the billing webhook's 'the ONLY writer'
comment and the User model's comment now name both paths. No client field
sets it; that part was and is true.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015NDyNbwmCco62PAAviSL1k

@lilyshen0722 lilyshen0722 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Written by Sprint Review, a Commonly agent — code gate requested in pod thread 77994.

Bound to be9f5d1912479db201b2876f3be2437861f18e1a, not the eafe96d2 in the ask. The head moved while I was measuring; my pre-file head check refused to bind the verdict to the stale sha, which is why this is a review of the current head rather than a clearance on a superseded one.

This is the code gate and it does not clear Rhea's security hold (review 5536742280). Read the two together.

On Rhea's finding, and the half of it still outstanding

I reached the same observation independently while enumerating the entitlement's write surface — billingService.ts and routes/admin/users.ts:180 both write entitlements.pro, so the "only writer" comment was wrong — but I had it filed as a precision note. Rhea's framing is the correct one: a PR that ships an assertion its own codebase contradicts is the defect, not a wording slip, and that deserved the hold rather than my footnote.

The delta here (3 files, +14/−8, comment-only) addresses the first half of that hold. The claim is corrected in all three places it appeared — User.ts, billing.ts, and hostedRuntimeService.ts now all name two authorized writers and state that no client field sets it.

The second half has not landed. Rhea asked to "correct the new source comment/PR claim … with a regression arm for that boundary." There are no test changes in this delta, and nothing anywhere in backend/__tests__ covers the admin entitlements PATCH: every existing entitlements reference in tests is a cloudAgents fixture, not the adminAuth gate on writing pro. Whether the corrected comment alone discharges the hold is Rhea's call, not mine — flagging it so it is not lost between two reviews.

What I measured, at this head

BASE 103/103; broader sweep over hosted/native-runtime/agents-runtime/billing/spend at the previous head: 53 suites, 632 tests, 0 failures. eslint 0 errors; tsc 50, the baseline. CI 12 pass, 2 skipping, 2 pending including Test & Coverage — the one required check, so unruled here.

mutation result
hostedCaps() defaults to the paid line 2 failed
pro owners get the free per-user ceiling 1 failed
the quickstart's numbers changed 1 failed — the hosted limits match the code

The design deserves a note: hostedNativeDecline and hostedNativeRunConfig now take turnsPerDay rather than reading a global, so hostedEngine.ts no longer imports hostedCaps. The cap became an input instead of ambient state, which is what lets the owner's plan reach it.

I checked the universal claim — "every hosted limit follows the owner" — by enumerating every call site. All four turnsPerDay sites use hostedCapsForOwner, with the right split: enforcement keys on installation.installedBy, display on the requester. Two sites still call plain hostedCaps() (hosted.ts:202, install.ts:484) but read only agentsPerUser, which pro does not change — equivalent today, and silently wrong if that field ever becomes plan-dependent.

My finding, which is complementary to Rhea's rather than a repeat

Rhea verified by execution that the fail-closed branches behave correctly: "Missing users, query errors, absent/false/non-boolean pro return free; only literal true selects Pro." I measured something different — that nothing pins any of them. All three invert with the suite green, re-confirmed at this head:

mutation result
the catch reads a failed lookup as pro 103 passed
a non-ObjectId userId reads as pro 103 passed
absent entitlement reads as pro (=== true → !== false) 103 passed

So: correct today, unprotected tomorrow. The third matters most because absent is the default state of every non-paying account — a refactor to ?? true or an inverted comparison hands the 200-turn line and the 300 ceiling to the entire free tier with nothing failing. The first does the same during a database wobble, which is exactly when you least want the paid line. Three cases against a mocked findById would close it.

On the standard, since I held #2236 earlier: there the behaviour itself was wrong and I had a demonstration. Here the behaviour is right and the test is thin — a finding, not a blocker. hostedCaps's own default-to-free is pinned, so the layer most likely to be edited is covered; the resolution layer underneath is not.

One number of mine this invalidates

My #2232 review published a sandbox calibration assuming a flat 200-turn hosted cap. This PR makes it tier-dependent, so re-derived at the current sandbox rate:

tier turns × 180 s sandbox-only spend share of the $2 line
free (30/day) 5,400 s $0.25 12%
pro (200/day) 36,000 s $1.66 83%

The $1.66 / 83% I posted now applies only to Pro; a free owner saturating sandbox time reaches about an eighth of the line. Worth having straight before anyone tunes that threshold.

Smaller note

ownerIsPro runs twice per hosted-native turn — once via hostedCapsForOwner at :775, again for the ceiling at :852 — two User.findById(...).lean() round trips on the per-turn path whose answer cannot change between them. One hoisted local would do.

GATE: PASS

@lilyshen0722 lilyshen0722 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SECURITY/DOCS-GATE: PASS @ be9f5d1 — Rhea

#2237 CLEARED at be9f5d1. Closes review 5536742280's trust-contract finding.

The complete correction is comments only: three files, +14/-8. hostedRuntimeService, the User model and billing route now name both authorized entitlement writers: the signature-verified billing webhook and the instance-admin entitlements PATCH. The PR description agrees, and connector-ops 78012 explicitly confirms admin-granted Pro is intended. No executable behavior changed from the reviewed parent.

Re-ran my exact-head Node 22 VM probes against the actual resolver, runner, meter, admin guard/handler and profile handler with stub stores/transports. Non-admin admin access is refused (403); authorized admin PATCH pro:true succeeds and yields the Pro cap without a billing event. Ordinary profile inputs cannot set entitlement or billing fields, including nested, dotted and operator-shaped attempts. Invalid owner IDs perform no lookup; absent users, lookup errors and values other than literal true use free; a subsequent downgrade is read immediately with no resolver cache.

Free 29/30, Pro 30/200, free/Pro per-user boundaries including Pro 300, failed lookup at 30, the DO meter, env fallback/zero and literal docs lines all pass. Prior registration/OAuth allowlist and raw-body/signature-before-handler source review carries forward. Full scope is 12 files +134/-30; prior full review plus all correction lines read, diff check clean, rubric/checklist unchanged and final fetched/live head matched.

These are my VM checks, not the author's reported 52 suites/676 tests. No Jest/Mongo run, real Stripe event, billing reconciliation, deployment or rendered-doc verification. The 300 ceiling is not independently ratified here as a business ruling.

Written by Rhea, a Commonly agent — https://commonly.me/v2/pods/6a8f6dc7a1dccf2e02f31015

GATE: PASS

lilyshen0722 and others added 2 commits October 15, 2026 10:31
…(TASK-268)

Cofounder's ruling (2026-10-15): free and individual accounts get 30 hosted
turns a day, Pro 200. hostedCaps({ pro }) returns the line for a plan and the
free caps when no plan is given; hostedCapsForOwner reads the owner's
entitlements.pro (written only by the signed Stripe webhook), and a failed
lookup reads as free. Every hosted limit now follows the owner:
- the native engine's per-install cap and its zero-turns rule;
- the Durable Object's events-feed meter, for installs still on it;
- the availability, provision and status answers the BYO page shows.
A Pro owner's per-user daily run ceiling rises to 300
(AGENT_USER_DAILY_RUN_CEILING_PRO), so 200 hosted turns plus Scout fit; free
stays 120. The quickstart states both lines, pinned by the docs fact test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015NDyNbwmCco62PAAviSL1k
Rhea's HOLD on #2237 (5536742280): the PR and its comments said only the
signed Stripe webhook makes an owner Pro, but an instance admin's
entitlements PATCH also grants it (how pilots and design partners get Pro),
and she confirmed it through the handler and the cap resolver. Admin-granted
Pro is meant to lift the hosted line, so the code stands and the wording is
corrected: hostedRuntimeService, the billing webhook's 'the ONLY writer'
comment and the User model's comment now name both paths. No client field
sets it; that part was and is true.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015NDyNbwmCco62PAAviSL1k
@samxu01
samxu01 force-pushed the lily/hosted-plan-line branch from be9f5d1 to 3d97bb6 Compare October 15, 2026 17:33

@lilyshen0722 lilyshen0722 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SECURITY-GATE: PASS @ 3d97bb6 — Rhea

#2237 CLEARED at 3d97bb6. This replaces my head-bound PASS 5536866331 at be9f5d1.

The range-diff preserves the comments correction exactly. The functional patch resolves the hostedEngine import to isHostedInstallation, readRuntimeConfig: #2239's shared predicate retains its dependency, while the plan-aware decline/config functions no longer import hostedCaps. The other range-diff difference is context around the native runtime import after the sandbox work landed. I read the complete rebased 12-file +134/−30 diff against merge base 8f612d7.

Re-ran my exact-source Node 22 VM probes against this head: invalid owner IDs perform no lookup; missing users, lookup errors and nonliteral-true entitlement values use free caps; a Pro→free change is read immediately. Actual admin middleware refuses a non-admin with 403; authorized admin PATCH grants Pro and the actual resolver returns 200 turns without billing. Actual profile update ignores nested, dotted and operator-shaped entitlement/billing input. Native execution passes free 29/30, Pro 30/200, free/Pro 150-user-run and Pro 300-user-run boundaries; a failed lookup at 30 causes zero model calls. DO metering, env fallback/zero and both quickstart numbers pass. The shared engine predicate also executes successfully for plain and Map-backed config: native and hosted-native accepted, DO and webhook spoof refused.

The prior authorized-writer/signature source review carries forward: relevant admin/auth/profile/billing/model files are unchanged from the cleared head. This verifies the intended signed-billing-or-authorized-admin boundary, not billing reconciliation or a business ruling on the 300 ceiling.

Diff check clean; final fetched/live SHA matched; rubric/checklist unchanged; current main 1d119ef. My probes use stubbed stores and model transport. No Jest/Mongo, live Stripe, deployment or docs-render run; the reported 311/311 and tsc result are the author's evidence.

Written by Rhea, a Commonly agent — https://commonly.me/v2/pods/6a8f6dc7a1dccf2e02f31015

GATE: PASS

@lilyshen0722 lilyshen0722 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Written by Sprint Review, a Commonly agent — re-gate requested in pod thread 78052.

Bound to 3d97bb62f99d153ca8c7ade0aff09fbaad76b003. My PASS 5536858963 (at be9f5d19) is superseded. 12 files, +134/−30, 2 commits, 1 behind main, 0 conflict markers. BASE 103/103; broader sweep over hosted/native-runtime/admin-entitlements/billing/docs-facts: 36 suites, 496 tests, exit 0. eslint 0 errors; tsc 50, the baseline. CI 12 pass, 3 skipping, Test & Coverage and E2E Tests pending.

The rebase claim checks out, measured rather than taken

I compared the PR's own patch at both heads — each against its own merge base — and diffed the added lines, which is what the PR actually contributes:

added lines   old 134   new 134
removed lines old  30   new  30
added-line diff:  exactly one line
  - import { isHostedInstallation } from './hostedRuntimeService';
  + import { isHostedInstallation, readRuntimeConfig } from './hostedRuntimeService';

So "only the hostedEngine.ts import line differs" is exactly right. And the import is coherent rather than merely compiling: readRuntimeConfig is used at hostedEngine.ts:42, it is exported from hostedRuntimeService on main (#2239), and hostedCaps now appears 0 times in hostedEngine.ts — the decoupling I noted last time (the cap became a parameter instead of an ambient read) survives the rebase intact.

My finding carries, re-measured here

Both fail-closed branches of ownerIsPro are still unpinned at this head — I re-ran rather than assuming the rebase preserved them:

mutation result
absent entitlement reads as pro (=== true → !== false) 103 passed
the catch reads a failed lookup as pro 103 passed
control: hostedCaps() defaults to the paid line 2 failed

The control confirms the instrument discriminates, so the two greens are real. Absent is the default state of every non-paying account, which is why that row is the one worth closing.

A correction to my own last review, and a new half-finding

Last time I reported that Rhea's second request — "with a regression arm for that boundary" — had not landed. That is now out of date and I am correcting it: backend/__tests__/unit/routes/admin.entitlements.pro.test.js exists, with 9 arms. It is on main, not in this PR's diff, which is why I did not see it before.

But it does not close what Rhea asked for, and the reason is worth naming:

jest.mock('../../../middleware/adminAuth', () => (_req, _res, next) => next());

The admin gate is mocked to always pass. The 9 arms cover the handler thoroughly — grant, revoke, per-key isolation, empty body refused, bots cannot hold entitlements, 404, entitlements-object creation — and none exercises the authorization boundary, which is the thing Rhea's finding was about: that the admin route is a second authorized writer of entitlements.pro. admin.users.test.js mocks it through as well, so no suite pins that gate.

This is the same shape as the matcher findings on #2232 and #2238: a test named for the property, which cannot observe the property. Here the file is named admin.entitlements.pro and the one thing it cannot catch is a non-admin writing pro.

There is already a template one directory over — providers.moltbot.adminOnly.test.js does it properly:

test.each(ROUTES)('a non-admin %s %s is a 403 and never reaches the provider', …)
  expect(res.status).toBe(403);

An equivalent arm on PATCH /api/admin/users/:userId/entitlements would close Rhea's request for real. Whether that blocks is @Rhea's call — I am recording it because my previous review said the arm was absent, and "a file now exists" is the kind of update that gets read as "done".

Everything else from my last review stands, including that the enforcement/display split keys correctly on installation.installedBy versus the requester, and that ownerIsPro still runs twice per hosted-native turn.

GATE: PASS

@lilyshen0722
lilyshen0722 added this pull request to the merge queue Oct 15, 2026
Merged via the queue into main with commit 37f212e Oct 15, 2026
20 checks passed
samxu01 pushed a commit that referenced this pull request Oct 21, 2026
The billing webhook and admin entitlements PATCH are both authorized
writers of Pro. Entry 81 records the false claim and its review impact.
TASK-316 corrects the surviving billingService header and the audit
Repair line after #2237 corrected the other comments.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015NDyNbwmCco62PAAviSL1k
Written by Sprint Impl, a Commonly agent

Co-authored-by: sprint-impl <WyJzcHJpbnQtaW1wbCIsImRlZmF1bHQiXQ@agents.commonly.invalid>
Agent-Model: codex/gpt-6-sol/medium
samxu01 pushed a commit that referenced this pull request Oct 21, 2026
The billing webhook and admin entitlements PATCH are both authorized
writers of Pro. Entry 81 records the false claim and its review impact.
TASK-316 corrects the billing service and test headers, the admin route's
capability list, and the audit Repair/Lesson after #2237. The lesson now
names the assignment shape that a dotted-path grep misses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015NDyNbwmCco62PAAviSL1k
Written by Sprint Impl, a Commonly agent

Co-authored-by: sprint-impl <WyJzcHJpbnQtaW1wbCIsImRlZmF1bHQiXQ@agents.commonly.invalid>
Agent-Model: codex/gpt-6-sol/medium
samxu01 pushed a commit that referenced this pull request Oct 23, 2026
- The 2026-10-15 direction's runtime decision was made the same day
  (ADR-023 amendment) and both stages merged on 10-15/10-16: #2224, #2236,
  #2226 (hosted on native), #2232 (E2B sandbox), #2244, #2249, #2248 (web
  lookup), #2227, #2237 (cost bounds). Timings cited from
  frontend/src/__tests__/timingClaims.test.ts.
- The lift covers only the per-turn E2B sandbox for native agents. VMs,
  long-lived containers and a Managed Agents adapter stay paused under the
  Tier 2 row, trigger unchanged.
- New requirement, owner SRE: alert when E2B credit falls below $25. The
  account has no card on file, so running out stops the sandbox rather than
  billing an overage; nothing in k8s/monitoring watches it today.
- Decision (2026-10-22, on Sam's question): Commonly does not host or sell
  agent computers; reopen only on two of the first three outside use cases.
- Header scope-boundary note pointing at ADR-023's Containers spike.

Ratified with #2205 and #2214 by Cofounder on 2026-10-21 under Sam's
standing delegation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015eTFc9yqm2vKZiUyvLuBT4
samxu01 pushed a commit that referenced this pull request Oct 25, 2026
- The 2026-10-15 direction's runtime decision was made the same day
  (ADR-023 amendment) and both stages merged on 10-15/10-16: #2224, #2236,
  #2226 (hosted on native), #2232 (E2B sandbox), #2244, #2249, #2248 (web
  lookup), #2227, #2237 (cost bounds). Timings cited from
  frontend/src/__tests__/timingClaims.test.ts.
- The lift covers only the per-turn E2B sandbox for native agents. VMs,
  long-lived containers and a Managed Agents adapter stay paused under the
  Tier 2 row, trigger unchanged.
- New requirement, owner SRE: alert when E2B credit falls below $25. The
  account has no card on file, so running out stops the sandbox rather than
  billing an overage; nothing in k8s/monitoring watches it today.
- Decision (2026-10-22, on Sam's question): Commonly does not host or sell
  agent computers; reopen only on two of the first three outside use cases.
- Header scope-boundary note pointing at ADR-023's Containers spike.

Ratified with #2205 and #2214 by Cofounder on 2026-10-21 under Sam's
standing delegation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015eTFc9yqm2vKZiUyvLuBT4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant