Skip to content

feat(first-session): the welcome leads with "Run an agent here, nothing to install", and a hosted agent shows live the moment it is - #2259

Merged
lilyshen0722 merged 1 commit into
mainfrom
growth/lead-with-hosted
Oct 16, 2026
Merged

lilyshen0722 merged 1 commit into
mainfrom
growth/lead-with-hosted

Conversation

@lilyshen0722

Copy link
Copy Markdown
Contributor

Follow-ups 1 and 2 from QA walk 3 (fresh stranger, production 47ba879b). The walk measured signup → live hosted agent at 18.8 s raw, first useful reply at 31.0 s, and 4 of 4 asks done.

1. Lead with hosted

That 18.8 s depended on the walk asking Scout for hosting. Scout's answer to "Connect an agent I already use" offers only the local path ("running on your machine… Claude Code or Codex?"). A stranger who doesn't ask never learns that nothing needs installing, yet that's the out-of-box path.

  • Where this instance can host (hostedEngineAvailable), the welcome's starts now open with "Run an agent here, nothing to install" (SCOUT_HOSTED_START).
  • Scout's prompt knows about the new start. Its first reply to "Connect an agent I already use" now adds one clause saying Commonly can run an agent here instead. The local path stays primary there, because that's what the person picked.
  • Layout: .v2-chat__starts is a one-column grid (minmax(0, max-content)), so the third start stacks; no CSS changes. I'll check it at 390 and 1440 on production with a fresh account after the deploy.

2. "live" and "Starting…" for the first 4 s

hostedState started as 'starting', and only a setInterval (every 4 s) flipped it. So the first check ran at +4 s, and until then the panel said "is live in My Workspace" and "Starting…" (walk 3 saw both at +0.2 s and +1.5 s).

  • Native provision now returns ready: true. Native turns run in-process, so the agent is live from provision. A Durable Object provision doesn't say ready: its worker isn't live until it polls. The rule stays in the backend.
  • V2AgentBYO shows running at once when provision says ready, with no status round trip. Otherwise it checks status immediately, then every 4 s.

Tests

  • authController.guideIdentity: when the instance can host, the welcome leads with the hosted start, aimed at this pod. The existing arm (no hosting in the test env) keeps the two-start welcome.
  • routes/hosted: native provision says ready: true; Durable Object provision doesn't.
  • V2AgentBYOHosted: a ready provision shows running with no status call. Without ready, the first check runs before any timer advances.
  • Mutations: dropping the hosted start, the ready field, the immediate check, or the ready shortcut each fail an arm.
  • Ran locally on Node 22: those suites plus scoutKnowledge, all green. tsc and eslint are clean on both sides.

After deploy

  • Existing Scout installs carry a copy of the prompt. Run refresh-native-agent-configs --app scout --apply, after a dry run.
  • New signups get the new welcome. Existing workspaces keep their welcome as posted.
  • Measure the share of new signups with a live hosted agent within 2 minutes of signup (Cofounder's measure for this lever). It goes in the daily pull.

Not in this PR

Walk 3's items 3 and 4:

🤖 Generated with Claude Code

https://claude.ai/code/session_0157AG2wo8mJYqKezXNs48Ya

…ng to install", and a hosted agent shows live the moment it is

QA walk 3 (fresh stranger, production 47ba879): signup to a live hosted
agent took 18.8s raw, but only because the walk asked Scout for hosting.
Scout's answer to "Connect an agent I already use" offers only the local
path, so a stranger who doesn't ask never learns that nothing needs
installing. And for its first 4 seconds the Running panel said "is live" and
"Starting…" at once.

Lead with hosted:
- Where this instance can host (hostedEngineAvailable), the welcome's starts
  open with "Run an agent here, nothing to install" (SCOUT_HOSTED_START). It
  navigates to Bring your own in this workspace with hosting chosen
  (?mode=hosted, #2250), so a live agent is two clicks from the first screen.
  The two sent starts follow unchanged. Where the instance can't host, the
  welcome is exactly as before.
- Scout's prompt knows the new start, and its first reply to "Connect an
  agent I already use" adds one clause saying Commonly can run one here
  instead.

Live when live:
- The native provision response says ready: true, since native turns run
  in-process. A Durable Object provision doesn't, because its worker isn't
  live until it polls.
- V2AgentBYO shows running at once when provision says ready, with no status
  round trip. Otherwise it checks status immediately, then every 4s, instead
  of waiting for the first interval.

Tests: the welcome leads with the hosted start aimed at this pod when the
instance can host (the existing arm keeps the two-start welcome when it
can't); native provision says ready and Durable Object provision doesn't; a
ready provision shows running with no status call; without ready, the first
check runs before any interval. Mutations: dropping the hosted start, the
ready field, the immediate check or the ready shortcut each fail an arm.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0157AG2wo8mJYqKezXNs48Ya

@lilyshen0722 lilyshen0722 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Written by sprint-review, a Commonly agent — code gate at fdb7444ee. pod thread

Base exactly main (0ceec68c1). Both parts check out, with one measured side effect worth knowing.

No rule drift, and the fail-safe points the right way

The welcome's hosting lead asks hostedEngineAvailable() — the same shared rule routes/hosted.ts and Scout's offer tool use, which #2250 deliberately de-duplicated. The href is server-built with encodeURIComponent(pod._id), matching the pattern already gated on #2250.

On the frontend, ready: provisioned?.ready === true is strict: an absent field (the Durable Object branch, or an older backend) is false and falls to the polling path. I confirmed ready: true is set on the native branch only (:270); the DO response at :312 does not carry it. And the immediate first check is safe because check's body is unchanged — a not-yet-ready response and a thrown request both keep polling, with no error state, so moving the first call to t=0 cannot surface a failure that previously never appeared.

Measurement

BASE: backend 103, frontend 16. Needles asserted, each file restored, RESTORED 103/16. Failures by name:

mutation result
hostedEngineAvailable() forced true (lead shown where it cannot host) 1 — "the welcome carries the starts in its payload, with the copy they sit under"
hosted lead dropped from starts 1 — "leads the starts with Run an agent here, aimed at this workspace…"
backend stops returning ready 1 — the provision arm
frontend treats ready loosely 2
immediate first check removed 2 — incl. "without ready from provision, status is checked at once, not after the first interval"

Both directions of the hosting gate are covered, which is the part I'd otherwise have asked for: an existing arm catches the lead appearing where the instance cannot host, and the new arm catches it missing where it can. So "a non-hosting instance sees no change" is verified rather than asserted. The new frontend arm is also stronger than it needed to be — it asserts no /api/hosted/status call at all on the ready path, pinning "no polling" rather than just "shows running", and the backend arm pins starts[0] and that starts.slice(1) still equals SCOUT_STARTS, so "the two sent starts are unchanged" is pinned too.

Finding: the "slow" message now arrives one interval earlier

Not blocking, and invisible from the diff's intent. ticks += 1 is the first statement in check, so the new immediate call consumes a tick:

HOSTED_STATUS_POLL_MS = 4000, HOSTED_STATUS_MAX_TICKS = 15
before: tick 15 fires at 60s
after:  tick 15 fires at 56s

So on the polling path a slow hosted start is declared "slow" 4s sooner than before. That is a small reduction in patience for exactly the case the message exists for. If 60s was chosen deliberately, HOSTED_STATUS_MAX_TICKS wants to be 16, or ticks should increment only on interval-driven calls.

Two notes

The ready contract is pinned on both sides independently — the backend arm asserts the literal field and the frontend arms read it — but no test spans the two, so each suite is green against its own mock. A rename would red the originating side, which is adequate; it is worth knowing there is no integration arm.

Also: on the ready path the frontend sets running and installs no timer, so nothing later corrects the panel if the in-process agent stops. That matches the stated semantics (native is live from provision) and I am not asking for a change.

CodeQL has not reported at this head — 7 checks pass and 6 are still pending, including CodeQL, the three Analyze jobs, E2E and Test & Coverage. This gate therefore says nothing about the security scan; it should not be pressed on my line alone until those report.

GATE: PASS

@lilyshen0722
lilyshen0722 added this pull request to the merge queue Oct 16, 2026
Merged via the queue into main with commit 53d7320 Oct 16, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant