Skip to content

feat(members): watch a Bot worker's browser in the Bot page side panel - #39074

Open
jingchaodev wants to merge 1 commit into
kirodotdev:mainfrom
jingchaodev:crew/kirocrew-developer/bot-worker-browser-view
Open

jingchaodev wants to merge 1 commit into
kirodotdev:mainfrom
jingchaodev:crew/kirocrew-developer/bot-worker-browser-view

Conversation

@jingchaodev

@jingchaodev jingchaodev commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: On a Bot's page, the person can watch a worker session's live browser in the side panel, without opening that worker session.

A Bot hands browsing to its worker sessions. The Bot's thread draws no row for a worker's work, and a worker's pane in the Bot page's Sessions tab draws no browser card either (a ChatPane names no session to TurnBlock). So a worker's live browser view shows only in that worker's own chat on the chat page, and the person has to leave the Bot's page to watch it, take over, or answer its sign-in ask. The Bot page's Preview tab shows only the Bot thread's own browser, and the page hosts no big view, so Expand on the Bot's own card opens a full-screen sheet over the thread.

Why it matters

The person talks to the Bot, but the browsing happens in a worker. Nothing on the Bot's page says that a worker is browsing now, and nothing there can show its page, so a sign-in the worker waits on is answered only by leaving for that worker's chat.

Not a goal

  • No new row in the Bot's thread: the recorded decisions on what a Bot's chat draws stand.
  • No automatic opening of the side panel, and no automatic switch to the Preview tab, when a worker starts browsing.
  • No browser card in a worker's pane in the Sessions tab.
  • No change to the chat page, the live card, the gateway, the browser routes or browse governance.
  • The Profile card's Sessions list stays as it is (no Live there).

What changed (motivation → approach → change)

A worker's live browser showed only in that worker's own chat on the chat page, so watching it meant leaving the Bot's page. The Bot page shows it in its own side panel instead, by reusing the chat page's big view (the stage store, PreviewTabBody, BrowserLiveStage) and pointing it at the Bot's sessions.

Bot page Before After
Sessions tab row Title, summary, open chevron A session that is browsing now also shows Live; a click opens its browser big in the Preview tab
Preview tab, Agent browser The Bot thread's own browser only The Bot thread and every open session stamped with the Bot's mate_root; Agent browser opens on the one browsing now, else on the turn that started last
Expand or Replay on the Bot's own live card (desktop) A full-screen sheet over the thread The side panel's Preview tab, as on the chat page; a phone keeps the sheet
  • useMembersPageSidePanel registers the page as the browser stage host (registerBrowserStageHost) while its thread is confirmed, and passes the Bot's open session keys to the panel (SidePanel.browserFollowKeys, then PreviewTabBody.followKeys). When the panel cannot show the Preview tab at that moment (the thread re-confirming, or a docked Profile keeping its schedule draft), the Bot's own card falls back to the full-screen sheet and any other stage is dropped, so no card says it is showing in the side panel.
  • components/PreviewTabBody.tsx shows the newest panel stage among its sessions (usePanelBrowserStage, hooks/useBrowserLive.stage.ts), one at a time, and opens Agent browser on useBrowserLiveNewest (hooks/useBrowserLive.ts). With no follow keys, as on the chat page, it works as before.
  • MateSessionsList takes an optional onWatch, and only the side panel's Sessions tab passes it, while the thread is confirmed. Live reuses the card's own strings ("Live", "Expand the live view"), so no catalog key is added.
  • The worker's stream, takeover, sign-in sheet and replay run on the worker's own session key, through the same routes the worker's own chat uses.
  • crew-mode.md, browser.md and the feature map describe the Bot page's Preview tab and the Live button.

A card in the Bot's thread was the other option. The recorded decision on what a Bot's chat draws rules it out.

Backwards compatibility

Compatible: the chat page passes no follow keys, so its Preview tab is unchanged, and both new props are optional.

Behavior change for existing users: on a Bot's page on desktop, Expand or Replay on the Bot's own live browser card opens the side panel's Preview tab, as on the chat page, instead of a full-screen sheet over the thread. A phone keeps the full-screen sheet, and so does the card while the panel cannot show the tab. No entry in docs/decisions/ sets the full-screen behavior on this page.

Removes nothing: no route, key, setting or row is removed, and the full-screen sheet stays for phones and for pages with no host.

Tests

  • BrowserLiveStage.test.tsx: a Preview tab with followKeys shows a followed session's browser (its stream visibility and takeover go to that session's key), opens on the session browsing now, else on the turn that started last, shows no stage of a session it does not follow, keeps one big view when another of its sessions opens one, and closes the stage when the tab is left.
  • MateSessionsList.test.tsx: Live shows only on a row whose session is browsing now, hands that worker to onWatch, and goes away when the turn ends; with no onWatch there is no Live.
  • MembersPage.sidePanel.test.tsx: a Sessions row's Live opens a panel stage for the worker and the Preview tab in Agent browser mode; while the thread re-confirms, Live is withheld, the Bot's own card falls back to the sheet, and a worker's stage is dropped.
  • MembersPage.schedules.test.tsx: the big view asks before a docked Profile's schedule draft goes; kept, the Bot's card opens the full-screen sheet; discarded, the Preview tab opens on it.

Each new test fails with its production change reverted. Determinism: the four touched test files ran 20 times together on vitest's parallel workers, each run in a shuffled order with its own seed (--sequence.shuffle --sequence.seed=<1000 + 7919k>, k = 1..20): 20 of 20 green.

Manual verification

The real Members page ran in headless Chromium on a Vite dev server, with the API answered from fixtures and one worker browse turn (an event and a frame) fed into the live-browser store. The harness is in a PR comment. The browsing worker's row shows Live, a click opens the Preview tab on the worker's live page, and the Bot's thread draws no browser card, in light and dark. An end-to-end run with a real worker browsing through the gateway was not done.

Real-machine verification

Machine: the author's Linux host, x86_64, 48 vCPU / 186 GiB, Node 24.22, Playwright Chromium (headless)
Before (main be1ce64): cd website && node scripts/capture-bot-worker-browser.mjs <out> with the harness from the PR comment -> locator.waitFor: Timeout 15000ms exceeded waiting for getByTestId('mate-session-watch'): the browsing worker's row offers no Live
After (head fc88f5c): the same command -> all frames verified (light and dark: one Live row, the stage live, no browser card in the Bot's thread)
Cleanup: own computer; the script stops its Vite server and Chromium, and the scratch worktree and its output are deleted

Screenshots / video

A fork without write access cannot use gh --attach, so the frames are committed under temp-screenshots/bot-worker-browser/. A maintainer removes that folder before merging. They come from the harness in Manual verification.

The Sessions tab: the worker that is browsing offers Live.

Sessions tab with Live on the browsing worker

After Live: the Preview tab shows the worker's live page, and the Bot's thread is unchanged.

Preview tab showing the worker's live browser

Dark

Sessions tab with Live, dark

Preview tab with the worker's browser, dark

Related Issues

Refs #33089

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license.

@jingchaodev
jingchaodev requested review from a team and KiroCrewCaptain as code owners November 18, 2026 10:26
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention labels Nov 18, 2026
@jingchaodev

Copy link
Copy Markdown
Contributor Author

The capture harness behind the screenshots and the Real-machine verification section. It is kept out of the diff: jscpd checks capture/ and scripts/, and this harness shares its gateway-free preamble with the existing Members page captures.

To run it, save the three files below into website/, then run node scripts/capture-bot-worker-browser.mjs <outDir> from website/. It mounts the real Members page on an in-process Vite dev server bound to 127.0.0.1, answers /api/** from fixtures, and feeds one worker browse turn (an event and a frame) into the live-browser store. Each frame checks its own state before it is written. On main the script stops at mate-session-watch (no Live); on this PR's head it prints all frames verified.

website/capture/bot-worker-browser.tsx
/**
 * Temporary capture entry: the REAL MembersPage with a Bot ("atlas") whose
 * worker session is browsing. The worker's `browser_live` event and one frame
 * are fed straight into the live-browser store, the way the websocket does.
 * API calls are answered by the capture script's route interception.
 */
import { createRoot } from 'react-dom/client'
import { Provider } from 'react-redux'
import { QueryClient, QueryClientProvider } from '@tanstack/react-query'
import { MemoryRouter } from 'react-router-dom'

import MembersPage from '../src/pages/members/MembersPage'
import { ThemeProvider } from '../src/hooks/useTheme'
import { initI18n } from '../src/i18n/all'
import { store } from '../src/store'
import { sseSlots } from '../src/store/dashboardSlice'
import { browserLiveStore } from '../src/hooks/useBrowserLive'
import type { DecodedBrowserFrame } from '../src/hooks/useBrowserLive.frame'
import { installScrollReveal } from '../src/lib/scrollReveal'
import '../src/index.css'

const params = new URLSearchParams(location.search)
const theme = params.get('theme') || 'dark'
const route = params.get('route') || '/members?member=atlas'
document.documentElement.setAttribute('data-theme', theme === 'light' ? 'kiro-light' : 'kiro-dark')
localStorage.setItem('mc-theme', theme)
localStorage.setItem('mc-color-theme', 'kiro')

const BOT = 'member-atlas'
const W = 'chat-atlas-w1'
const now = Date.now()
const iso = (minutesAgo: number) => new Date(now - minutesAgo * 60_000).toISOString()
store.dispatch(sseSlots([
  { key: BOT, title: 'atlas', messages: 2, running: false, mode: 'member', agent: 'kirocrew' },
  { key: W, title: 'Compare the three pricing pages', messages: 9, running: true, mate_root: BOT, created_by: BOT, created: iso(4), last_turn_ts: iso(0) },
  { key: 'chat-atlas-w2', title: 'Summarise the release notes', messages: 6, running: false, mate_root: BOT, created_by: BOT, created: iso(60), last_turn_ts: iso(40) },
] as never))

/** A 1280x800 "pricing page" with three plan cards. */
async function pricingFrame(seq: number): Promise<DecodedBrowserFrame> {
  const c = document.createElement('canvas')
  c.width = 1280
  c.height = 800
  const g = c.getContext('2d')!
  g.fillStyle = '#f6f7fb'
  g.fillRect(0, 0, 1280, 800)
  g.fillStyle = '#1f2a44'
  g.fillRect(0, 0, 1280, 64)
  g.fillStyle = '#ffffff'
  g.font = 'bold 26px sans-serif'
  g.fillText('Example Cloud', 40, 42)
  g.font = '18px sans-serif'
  g.fillText('Product     Pricing     Docs     Sign in', 820, 40)
  g.fillStyle = '#1f2a44'
  g.font = 'bold 40px sans-serif'
  g.fillText('Simple pricing for every team', 330, 150)
  const plans: [string, string, string][] = [['Starter', '$8', '#e7ecf7'], ['Pro', '$15', '#dce8ff'], ['Business', '$24', '#e7ecf7']]
  plans.forEach(([name, price, bg], i) => {
    const x = 130 + i * 350
    g.fillStyle = bg
    g.fillRect(x, 210, 310, 470)
    g.strokeStyle = i === 1 ? '#3b6fe0' : '#c9d2e6'
    g.lineWidth = i === 1 ? 4 : 2
    g.strokeRect(x, 210, 310, 470)
    g.fillStyle = '#1f2a44'
    g.font = 'bold 28px sans-serif'
    g.fillText(name, x + 30, 265)
    g.font = 'bold 54px sans-serif'
    g.fillText(price, x + 30, 345)
    g.font = '18px sans-serif'
    g.fillText('per seat / month', x + 30, 380)
    for (let r = 0; r < 4; r++) { g.fillStyle = '#9aa7c2'; g.fillRect(x + 30, 420 + r * 40, 220 - r * 25, 12) }
    g.fillStyle = i === 1 ? '#3b6fe0' : '#5d6b8a'
    g.fillRect(x + 30, 600, 250, 50)
    g.fillStyle = '#ffffff'
    g.font = 'bold 20px sans-serif'
    g.fillText('Choose plan', x + 95, 633)
  })
  const blob: Blob = await new Promise(r => c.toBlob(b => r(b!), 'image/jpeg', 0.8))
  return { header: { type: 'browser_frame', session_key: W, seq, w: 1280, h: 800, mask: [] }, jpeg: new Uint8Array(await blob.arrayBuffer()), receivedAt: now + seq * 100 }
}

async function main() {
  await initI18n()
  installScrollReveal()
  browserLiveStore.applyEvent({
    session_key: W, turn_id: 'w1-t1', state: 'live', url: 'https://example.com/pricing', title: 'Pricing - Example Cloud',
    browser: 'kiro_chromium', headless: true, rung: 0, fps: 30, started_at_ms: now - 40_000,
    steps: [
      { label: 'Open example.com/pricing', status: 'done' },
      { label: 'Read the Pro plan', status: 'active' },
      { label: 'Open the next vendor', status: 'pending' },
    ],
    step: { index: 1, label: 'Read the Pro plan' },
  })
  browserLiveStore.applyFrame(await pricingFrame(1))
  const queryClient = new QueryClient({ defaultOptions: { queries: { retry: false } } })
  createRoot(document.getElementById('root')!).render(
    <Provider store={store}>
      <QueryClientProvider client={queryClient}>
        <ThemeProvider>
          <MemoryRouter initialEntries={[route]}>
            <div className="h-screen flex flex-col bg-bg text-text" data-capture-root>
              <div className="flex-1 min-h-0"><MembersPage /></div>
            </div>
          </MemoryRouter>
        </ThemeProvider>
      </QueryClientProvider>
    </Provider>,
  )
  // More frames while the scene is open, as a live stream would send.
  let seq = 2
  const timer = setInterval(() => {
    if (seq > 60) { clearInterval(timer); return }
    void pricingFrame(seq++).then(f => browserLiveStore.applyFrame(f))
  }, 500)
}

main()
website/capture/bot-worker-browser.html
<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8" />
    <title>bot-worker-browser capture</title>
  </head>
  <body>
    <div id="root"></div>
    <script type="module" src="/capture/bot-worker-browser.tsx"></script>
  </body>
</html>
website/scripts/capture-bot-worker-browser.mjs
/**
 * Temporary screenshots for "watch a Bot's worker browse from the Bot page".
 * Mounts the REAL MembersPage through capture/bot-worker-browser.html,
 * API route-intercepted (gateway-free), on an in-process Vite dev server.
 *
 *   01-sessions-live   the side panel's Sessions tab: the browsing worker's row
 *                      offers Live; the Bot's thread draws no browser card
 *   02-preview-live    after Live: the Preview tab shows the worker's browser
 *
 * Usage (in website/): node scripts/capture-bot-worker-browser.mjs <outDir>
 */
import { chromium } from 'playwright'
import { createServer } from 'vite'
import { mkdirSync } from 'node:fs'
import { resolve } from 'node:path'
import { routeMembersApi } from './lib/members-fixtures.mjs'

const OUT = resolve(process.argv[2] || '../temp-screenshots/bot-worker-browser')
mkdirSync(OUT, { recursive: true })

const NOW = Math.floor(Date.now() / 1000)
const iso = (secAgo) => new Date((NOW - secAgo) * 1000).toISOString()
const MEMBERS = [{
  name: 'atlas', slug: 'atlas', bound: true, slot_key: 'member-atlas', running: false, kiro_agent: 'kirocrew',
  workspace: 'default', memory_store: 'default', model: '', last_active_ts: NOW - 60,
  last_message: 'A worker is comparing the three pricing pages now.',
}]
const SLOT = {
  key: 'member-atlas', title: 'atlas', running: false,
  messages: [
    { role: 'user', content: 'Which of the three vendors is cheapest for 20 seats?', ts: iso(90) },
    { role: 'assistant', content: 'I asked a worker to open the three pricing pages and compare them. I will report back when it is done.', ts: iso(60) },
  ],
  total: 2, has_more: false, next_before: 0,
}
const WORKERS = {
  root: 'member-atlas', root_kind: 'mate', workers: [
    { key: 'chat-atlas-w1', title: 'Compare the three pricing pages', agent: 'kirocrew', status: 'working', group: 'working', last_activity: iso(5), ledger_item: '', summary: 'Reading the Pro plan', created_by: 'member-atlas' },
    { key: 'chat-atlas-w2', title: 'Summarise the release notes', agent: 'kirocrew', status: 'idle', group: 'with_conductor', last_activity: iso(2400), ledger_item: '', summary: 'Sent the summary to atlas', created_by: 'member-atlas' },
  ],
}
const dashboardHtml = `<div style="padding:16px;font:14px system-ui;color:#888">
  <div style="font-size:18px;font-weight:600;margin-bottom:8px">atlas</div><div>Open items: 1</div>
</div><script>parent.postMessage({type:'kirocrew-dashboard:ready'},'*')</script>`

const server = await createServer({
  configFile: './vite.config.ts',
  server: { host: '127.0.0.1', port: 5197, strictPort: false },
  logLevel: 'warn',
})
await server.listen()
const BASE = server.resolvedUrls?.local?.[0]?.replace(/\/$/, '') || `http://127.0.0.1:${server.config.server.port}`
const { LD_LIBRARY_PATH: _mise, ...browserEnv } = process.env
const browser = await chromium.launch({ env: browserEnv })
const failures = []
const check = (name, ok, detail = '') => {
  console.log(`${name}: ${ok ? 'OK' : 'MISMATCH'} ${detail}`)
  if (!ok) failures.push(name)
}

for (const theme of ['light', 'dark']) {
  const ctx = await browser.newContext({ viewport: { width: 1440, height: 860 }, deviceScaleFactor: 1 })
  const page = await ctx.newPage()
  const errors = []
  page.on('pageerror', (e) => errors.push(String(e)))
  await routeMembersApi(page, SLOT, { members: MEMBERS })
  await page.route((u) => /^\/api\/(teams|sandbox-doc|members\/[^/]+\/(panel|dashboard)|chat\/slots\/[^/]+\/crew-workers|browser\/live\/.+\/recordings.*)$/.test(new URL(u).pathname), (r) => {
    const path = new URL(r.request().url()).pathname
    const json = (body, status = 200) => r.fulfill({ status, contentType: 'application/json', body: JSON.stringify(body) })
    if (path === '/api/teams') return json({ teams: [] })
    if (path === '/api/sandbox-doc') {
      const { html } = JSON.parse(r.request().postData() || '{}')
      return json({ url: 'data:text/html;base64,' + Buffer.from(String(html || '')).toString('base64') })
    }
    if (/\/panel$/.test(path)) return json({ panel: null, html: null })
    if (/\/dashboard$/.test(path)) {
      return json({ instance_version: 1, template: { id: 'bot-card', version: 1 }, html: dashboardHtml, rendered_html: dashboardHtml, manifest: { id: 'bot-card', version: 1, fields: {} }, state: 'live', scripts: true })
    }
    if (/\/crew-workers$/.test(path)) return json(WORKERS)
    return json({ error: 'no recording' }, 404)
  })
  await page.goto(`${BASE}/capture/bot-worker-browser.html?theme=${theme}`)
  await page.waitForSelector('[data-capture-root]')
  const panel = page.getByTestId('member-side-panel')
  await panel.waitFor()
  await page.getByRole('tab', { name: 'Sessions' }).click()
  const live = page.getByTestId('mate-session-watch')
  await live.waitFor({ timeout: 15000 })
  await page.mouse.move(700, 600)
  await page.waitForTimeout(800)
  check(`${theme} 01 one Live row`, (await live.count()) === 1, `count=${await live.count()}`)
  check(`${theme} 01 thread draws no browser card`, (await page.getByTestId('browser-live-card').count()) === 0)
  await page.screenshot({ path: resolve(OUT, `01-sessions-live-${theme}.png`) })

  await live.click()
  await page.locator('[data-testid="preview-tab-body"][data-mode="agent"]').waitFor({ timeout: 15000 })
  const stage = page.getByTestId('browser-live-stage')
  await stage.waitFor({ timeout: 20000 })
  await page.waitForFunction(() => {
    const c = document.querySelector('[data-testid="browser-live-stage"] canvas')
    return !!c && c.width === 1280
  }, undefined, { timeout: 20000 })
  await page.mouse.move(700, 600)
  await page.waitForTimeout(800)
  check(`${theme} 02 stage live`, (await stage.getAttribute('data-mode')) === 'live', `mode=${await stage.getAttribute('data-mode')}`)
  check(`${theme} 02 thread still draws no browser card`, (await page.getByTestId('browser-live-card').count()) === 0)
  await page.screenshot({ path: resolve(OUT, `02-preview-live-${theme}.png`) })
  check(`${theme} no page errors`, errors.length === 0, errors.join(' | '))
  await ctx.close()
}

await browser.close()
await server.close()
if (failures.length) {
  console.error(`CAPTURE FAILED: ${failures.join(', ')}`)
  process.exit(1)
}
console.log('all frames verified')

@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Nov 19, 2026
@jingchaodev
jingchaodev force-pushed the crew/kirocrew-developer/bot-worker-browser-view branch from b3f3a01 to a1020cb Compare November 19, 2026 11:07
@jingchaodev

Copy link
Copy Markdown
Contributor Author

Rebased: main conflicted with this PR in docs/feature-map/README.md, on the Crew Members row (#33148 added the New Bot Memory choice there). Base a822e07 -> 87430c2, head a1020cb. The row keeps main's Memory text and this PR's two Live / Preview tab clauses; nothing else changed. The related web tests, tsc, docs-lint and the feature-map gate pass, and the Real-machine verification was re-run on the new base and head.

@github-actions github-actions Bot added merge conflict Branch has merge conflicts with its base — author must resolve before merge and removed merge conflict Branch has merge conflicts with its base — author must resolve before merge labels Nov 19, 2026
A Bot's thread draws no row for a worker's browsing, and a worker's
pane in the Sessions tab draws no browser card, so a worker's live
browser view showed only in that worker's own chat on the chat page.

The Bot page registers as the browser stage host and hands its Preview
tab the Bot's own sessions. Agent browser opens on the session that is
browsing now, else on the turn that started last, and the tab keeps
one big view at a time. A Sessions row whose session is browsing
offers Live, which opens that browser big in the Preview tab. Expand
or Replay on the Bot's own live card opens the same tab instead of a
full-screen sheet; a phone keeps the sheet, and so does the card when
the panel cannot show the tab. The chat page passes no follow keys, so
its Preview tab is unchanged.

Refs kirodotdev#33089
@jingchaodev
jingchaodev force-pushed the crew/kirocrew-developer/bot-worker-browser-view branch from a1020cb to fc88f5c Compare November 19, 2026 20:20
@jingchaodev

Copy link
Copy Markdown
Contributor Author

Rebased: main conflicted with this PR in docs/feature-map/README.md, on the Crew Members row again (#39538 changed its Fresh start text). Base 87430c2 -> be1ce64, head fc88f5c. The row keeps main's Fresh start text and this PR's two Live / Preview tab clauses; nothing else changed. The related web tests, tsc, docs-lint and the feature-map gate pass, and the Real-machine verification was re-run on the new base and head.

@github-actions github-actions Bot added merge conflict Branch has merge conflicts with its base — author must resolve before merge and removed merge conflict Branch has merge conflicts with its base — author must resolve before merge labels Nov 19, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor) merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: action required A blocking check or review needs attention

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant