UX wave: T3 Code daily-driver fixes + generative UI everywhere (prioritized)

Part of the system RFC #66 (layer L5, the dining room). Layers L0–L1 there (update off 0.1.2, Hermes ACP fixes) come before most items below.

Priority order (updated 2026-11-02)

This lists every problem the owner hit in the T3 Code Android app, the desktop app and the CLI setup. Sources: his T3 threads (statev2 snapshot), Hermes/OMP session history, and data/operations/t3code-setup.md. Each one now has a tracking issue. Upstream refs are written as plain numbers to avoid cross-links, and upstream posts still go through the gate.

P0: breaks or misleads daily use

fix(acp): failed turns hide the real cause behind "Internal error" / "Failed to open … provider session" #58 Failed turns hide the real cause. 32 of 405 runs failed (27 on Hermes ACP), and the user only saw "Internal error" or "Failed to open … session". Joins upstream 5943/17072.
fix(acp): edit-approval payload exceeds the 8 MiB gateway frame and wedges the session hermes-agent#688 Edit-approval frame over 8 MiB wedges the Hermes session. This was the cause of the 8 "Internal error" runs. The fix exists but is uncommitted and not live.
Sessions started outside T3 (Tern, Hermes app) don't reach T3, and imported ones are empty #59 Sessions started in Tern or the Hermes app don't reach T3, and imported ones are empty. fix(acp): list and attach native gateway conversations hermes-agent#656 is live and a stopgap auto-import is running. History hydration (upstream 22015) and mobile import remain. Two Ideas drafts await owner approval.
Hermes threads show idle while its background work and subagents are still running #60 Hermes threads show idle while background work and subagents still run. T3's fix (upstream 24610) is Claude-only. Hermes upstream 137069/137380.
Agent-started thread stuck in Plan mode on Android while Runtime says Full access #61 Agent-started thread stuck in Plan mode on Android while Runtime says Full access. This is upstream 29250 + 27819: join, don't duplicate. Open upstream fixes: PRs 29267 and 29275.
No effort/thinking control for Hermes in T3. T3 already renders ACP configOptions, but Hermes ACP returns an empty list (acp_adapter/server.py:1090). Hermes-side, existing upstream venues: Hermes 136084, 78229, 18326. No new issue, so it is tracked here.

P1: Android speedups with code already written (one lab APK verifies all three)
7. #51 Hermes icon, #50 hold to steer, #49 context meter. Code is on feat/android-voice-context-steer, but no test has run (the MBP queue was held). Needs focused tests and a device pass.
8. Inline html_render for Hermes threads (the generative-UI track below). Upstream 29130; Julius's fix, PR 29141, is now verified on Android with stock Hermes (comment posted 2026-11-02). Our Hermes-side fix is parked.
9. #63 Worktree handoff on Hermes ACP kills the run and leaves the continuation and cwd behind. Joins upstream 22971.
10. #62 Android render errors and low-memory exits. Needs a logcat repro first. (The WebView-tap blank was a harness artifact.)

P2: next
11. #48 Continuous voice. Code exists, blocked on the transcription backend choice (on-device Android vs host).
12. #54 Nudge chips, #65 "Needs you" queue (web preview built, rendered flow unverified).
13. #64 Desktop/CLI setup friction: update path, claude-history-home error, Usage missing OMP/Hermes, no settings CLI. First step: update from 0.1.2 to current, then re-check before reporting anything upstream.

P3: later
14. OpenUI v2 chart boards and the A/B (generative-UI track below).
15. #55 Next-action button RFC (after #54 proves out).

Out of scope here: the Hermes Android app's own issues (live-mode regex error, stale-gateway model switch, refresh-token disconnects). They belong to kvnloo/hermes-android and Hermes.

Track: generative UI everywhere

Request

Agent-made visuals should show up where the conversation happens on every surface. Today a published html_render page doesn't appear in Hermes threads, and Android has no native generative-UI path. This wave covers both: get inline pages working, then add native OpenUI chart boards that are compared fairly with the HTML path.

Evidence

Inline pages missing for Hermes threads (all clients). Hermes registers the MCP server that T3 passes over ACP as editor_<name>_<8-hex tag>, and reports calls as mcp__editor_t3_code_<tag>__html_render. In packages/provider-acp, extractMcpToolCallIdentity and resolveT3McpToolId treat that as a foreign server, so the isHtmlRender recovery (takeHtmlRender → {htmlRender}) never runs. The stored item then has no htmlRender reference, and web, desktop and mobile all skip the inline row.
Verified on device. In a frozen replay copy, I changed only the tool identity to t3-code.html_render and cleared the Android test app's client cache. After that, the chart rendered inline in the feed of the com.t3tools.t3code.openui lab build. Force-stopping the app alone keeps the stale cached item.
The page works on Android when opened directly. Opened as an attachment, the WebView page passes every interaction: treemap drill-down and Back, tile readouts, stacked-area day values, and heatmap cells.
Reference cohort for the chart families: RFC: OpenUI-generated visual replies with Tern/Luau — verified loop, optional TanStack/Jev oh-my-pi#143 (Code/Churn explorer RFC), [RFC] Native Tern TSP surface for Hermes TUI hermes-agent#432 (RepoExplorer / HottestFiles), feat(chat): inline xyflow diagrams, images and declarative UI dash#87, and Theo's Oct 6 T3 Code visualization post (stacked area, activity heatmap, Code/Churn treemap).
Earlier browser A/B (exploratory, not causal). Median whole-test time for OpenUI vs json-render was 6.8 / 8.0 s, 6.5 / 7.4 s and 6.6 / 9.0 s across three cases. Provider cache and load were not controlled.

Work in this wave

Hermes fix: kvnloo/hermes-agent fix/acp-editor-mcp-identity. ACP tool titles map editor-supplied servers back to the editor's own name (mcp__t3_code__html_render), which T3's T3_MCP_TITLE_CALL already recognizes. Tool tests pass; verified in a live T3 turn on web and Android (see the 2026-11-02 results comment).
Lab base: experiment/openui-android-replay. openui-v1 reports as native Android views. T3_OPENUI_LAB_BUILD=1 builds a separate package with its own scheme and no OTA, so it never replaces the standard app.
Chart design, stacked on the lab base: experiment/openui-chart-catalog. An openui-v2 chart board with Treemap (drill-down and Back), BarChart, StackedArea and Heatmap panels. They render as native views on Android and as SVG/HTML on web.
- Layout, validation, readouts and catalog wording live in the dependency-free packages/client-runtime/src/openUiCharts.ts. Each client binds the schemas to its own zod and OpenUI copies, which avoids duplicate React or zod instances.
- There are reference answers for the three source-demo prompts. The isolated smoke run passes against stock @openuidev 1.0.2, including SSR of all three boards.

Open questions

Defensive T3 fix? Should T3 also accept editor_<alias>_<hex> server names, in case other ACP agents rename servers the same way? Or is fixing the agent side enough?
Feed blanking on Android. Tapping inside an inline html_render WebView blanks the Android feed until the user scrolls. So far this has been seen only on an agent-owned virtual display. Needs a repro on the main display.
Prompt delivery for v2. How should the v2 catalog prompt reach agents (MCP tool description, skill, or system prompt), so both libraries get identical prompt overhead in the A/B?

Acceptance

A live Hermes turn in T3 shows html_render inline on web, desktop and Android, with no replay edits.
Lab APK built from experiment/openui-chart-catalog; all three prompts render as native boards, and every interaction passes on device.
Same-prompt A/B run where only the UI library changes (stock html_render vs OpenUI v2), timed by whole-test traces, with prompt and token overhead recorded.
Inline WebView taps no longer blank the Android feed.
Docs in docs/user/openui-reports.md match the shipped behavior.

Fork-local first. Upstream venues to join, not duplicate: upstream 15968 (inline visualizations), plus upstream 29130 (filed 2026-11-02 by another user: the same editor_t3_code_<id> namespace bug), so join that instead of filing a new one.

Generative-UI track: results (2026-11-02)

Inline html_render for Hermes threads: verified live. I ran a sandbox T3 server (runtime 0.1.2) with Hermes ACP on gpt-6.1-sol. The same request was sent once with the stock Hermes and once with the Hermes fix loaded.

Stock Hermes: the tool is recorded as editor_t3_code_<tag>.html_render with a plain text result. No inline page appears on web (0 iframes) or on the Android lab build.
With the fix: it is recorded as t3-code.html_render with an {htmlRender} result, and the page renders inline on both web and Android. Taps inside the inline page work, and the feed stays intact.
Fix: fix/acp-editor-mcp-identity at 921a4a2adbc. It is rebased on current NousResearch main, which still has the bug; 26 ACP tool tests pass. No PR yet.
Upstream 29130 is the same bug. Julius's PR 29141 fixes it on the T3 side (runtimeModel.ts). Both fixes can coexist. I have not checked 29141 on Android yet; that is still the plan from RFC: T3 Code as the restaurant — one front of house for every kitchen (UX wave = dining room) #66.
Separate Hermes problem: html_render calls carrying a ~16 KB page failed twice as invalid JSON, and the model still said "published". A ~3 KB page worked.

OpenUI v2 chart boards (experiment/openui-chart-catalog @ eebe70d411). Fleet build c3 passed mobile and web tsc, 16 node tests, 6 OpenUI smoke tests and Gradle (APK sha256 688542b0…2add). It is installed as the lab package only.

Generation: 10 cases (repo treemap with churn, stacked area, activity heatmap, disk usage, CI durations, dense 7×24, deep paths, an unsupported pie, sparse zeros). The oracle checks validity and exact data fidelity.
- CLI: 10/10 for gpt-6.1-sol and 10/10 for the default model.
- Live inside T3: 9/9 turns passed, about 3–9 s per board. The pie request fell back to bars and said so.
On-device fixes found in this pass:
- Mobile used web color tokens, so muted text and the v1 report bars were invisible.
- v2 boards now auto-render once complete.
- Single-entry folders fold into their child (home/kvn/).
- Path labels in bars keep the file name.
- The readout no longer repeats the size when color is the same metric.
- Narrow treemap tiles get a name-only label.
- The stacked-area chart marks the selected category.
- Heatmaps over 12 columns use dense cells with sparse column labels.
Not done: no visual web pass (tests and typecheck only), no iOS. Scrolling far away can reset a chart selection; this is documented.

Upstream posts and corrections (2026-11-02, later)

Posted upstream at the owner's request:

Upstream PR 29141 (Julius's T3-side fix for the inline-page bug): a comment with an Android check on unmodified Hermes. PR head a5c3df8b21 was run from source and the prompt was sent from the phone. The item was stored as t3-code.html_render and the page rendered inline. Comment id 6590233564. The PR is linked to this T3 thread.
Upstream Ideas discussion 20049 ("Optional OpenUI input for html_render"): the chart-board prototype, the format A/B and screenshots. It also asks maintainers whether they prefer a fenced block or html_render input. Comment id discussioncomment-19361918.

Format A/B (gpt-6.1-sol, hermes chat, 5 cases × 2 reps, order alternated; data in v2/ab-1/):

	OpenUI board	HTML page
Median turn	20.9 s	52.8 s
Median after first text	1.8 s	32.4 s
Median output tokens	294	2,867
Data check	10/10 (parsed, exact)	10/10 (weak: numbers present)

Decisions:

Hermes fix branch fix/acp-editor-mcp-identity (921a4a2adbc) is parked. Upstream 29141 covers the seam in T3, and Hermes's editor-name prefix was intentional (its ACP-9 change). Opening a second fix would compete with the carrier. The branch stays on the fork in case another ACP client needs it.

Correction to my earlier comment:

The "~16 KB html_render failed as invalid JSON" claim was wrong.
Actual cause: Hermes read_file caps each line at 2,000 characters (tools/file_operations.py, ... [truncated]). The test page is a single 16 KB line. The model copied the truncated text into its call, the JSON broke at the cut, and the model still replied "published".
This is a test artifact plus a false success claim by the model, not a payload limit. Nothing is filed for it.

Added #69 (per-host icons: mobile icon picker + optional icon-only host labels) under P2. Per-host icons already exist on web; this tracks the mobile gap and the option to hide host-name text.

Generative-UI track, next step: draft PR #71 adds brand-colored activity rows ("Searched Gmail · invoices") and four read-only app cards for openui-v2 boards (Uber/Lyft rides, Gmail, Google Maps, GitHub PR), with a 40 s demo of a real Hermes turn. The phone-side "let T3 use my other apps" idea is split out as RFC #70.

Share/handoff RFC: #72

Hermes side of the generative-UI track: landed on fork branches (2026-11-03)

What	Branch @ SHA	Gates
kvnloo/hermes-agent#711, inline `html_render` + MCP `rawOutput` + editor MCP identity	`fix/acp-html-render-inline` @ `f8d77a0`	`tests/acp_adapter`: 458 passed, 0 failed; ruff clean
UX-wave identity branch (listed above)	`fix/acp-editor-mcp-identity`, fast-forwarded `921a4a2` → `f8d77a0`	same head as pingdotgg#711
kvnloo/hermes-agent#710, `/stats` + `trace_query` boards	`feat/stats-dashboard` @ `c9f96d8`	plugin tests + `tests/acp_adapter`: 483 passed, 0 failed; ruff clean
UX-wave union: fork `main` + pingdotgg#711 + pingdotgg#710	`integration/t3-ux-wave` @ `b382a19`	496 passed, 0 failed

One identity fix. The identity branch renamed the server in the tool title; Add file4.txt with initial content pingdotgg/t3code#711 named it in _meta.serverId with its own regex. They are now merged and share one parser, so title and _meta cannot disagree. T3 reads _meta first and falls back to the title, so both paths are covered.
Two bugs in Allow adding notes with user input in QnA tool pingdotgg/t3code#710 fixed while landing. exporter.py was hidden by the root .gitignore export* rule, so a clean checkout failed to import it. Fetching the plugin catalog before the built-in menu also broke four ACP ordering tests.
Not live. T3 launches the shared install (84496839, Oct 22 main plus fix(acp): list and attach native gateway conversations hermes-agent#656's commits and uncommitted work, including the fix: Pending user input displays with "Pending Approval" status pingdotgg/t3code#688 fix). That build has neither change. Making it live means moving the install to a branch that unions integration/t3-ux-wave with a current Add option to make commit messages follow Conventional style including scopes pingdotgg/t3code#656 and the fix: Pending user input displays with "Pending Approval" status pingdotgg/t3code#688 fix, then restarting the Hermes ACP processes. That is an owner step; nothing was installed or restarted.
Not merged into preview/tern-frontend. That branch is the Tern TS-frontend dogfood lane. It is 12,133 commits behind main and has no ACP gateway code. A real merge touches 11,450 files with 33 conflicts, which is the owner-gated sync in sync: Tern frontend preview onto current main (needs Kevin OK) hermes-agent#540.

Upstream (plain numbers, nothing posted): T3 29130 / 29141, 15968; Hermes 72433. The proposed upstream comments wait for owner approval.

UX bundle plan + verification wave 1 (2026-11-04)

Plan: we join the work maintainers are already landing instead of opening our own upstream PRs.

Redesign stack: maria's six-layer web redesign stack (upstream PRs 28839 to 28845) already merges small outside fixes against its maria/redesign/<layer> branches.
Verification: Julius's PRs labelled pr:needs-verification need independent checks on real environments.

Fork items are being split into small add-ons on existing upstream carriers:

50 (hold-to-steer): becomes evidence on upstream PR 15192.
73 (share + /handoff): the guest-share part goes onto upstream PR 29391; /handoff becomes evidence on Idea 27708.
48 (continuous voice): rebase onto the merged upstream Android dictation (PR 26734).
54, 65 and 71: rebuild on redesign layer 2/6.

Full plan, dependency graph and conflict analysis are in data/operations/t3code-ux-bundle/20261104/PLAN.md.

Wave 1 is running four lanes:

Lane	Upstream PRs
L1 desktop shell/PATH (zsh, real CLIs)	28719, 28904, 24632, 28493, 28256, 23571
L2 server vs real data and network	29183, 26794, 29720, 26601, 28219, 24608
L3 redesign layer 6 on Linux Electron	28845
L4 Android, preview APK built on Avrea, tested on the S22	15192, 29391

Each upstream PR gets at most one evidence comment, posted only after a parent review against the live head. The ledger is in .../wave1/STATUS.md.

Navigation Menu

Repository navigation

UX wave: T3 Code daily-driver fixes + generative UI everywhere (prioritized) #57

Priority order (updated 2026-11-02)

Track: generative UI everywhere

Request

Evidence

Work in this wave

Open questions

Acceptance

Generative-UI track: results (2026-11-02)

Upstream posts and corrections (2026-11-02, later)

Hermes side of the generative-UI track: landed on fork branches (2026-11-03)

UX bundle plan + verification wave 1 (2026-11-04)

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

UX wave: T3 Code daily-driver fixes + generative UI everywhere (prioritized) #57

Description

Priority order (updated 2026-11-02)

Track: generative UI everywhere

Request

Evidence

Work in this wave

Open questions

Acceptance

Activity

kvnloo commented on Nov 2, 2026

Generative-UI track: results (2026-11-02)

kvnloo commented on Nov 2, 2026

Upstream posts and corrections (2026-11-02, later)

kvnloo commented on Nov 3, 2026

kvnloo commented on Nov 3, 2026

kvnloo commented on Nov 3, 2026

kvnloo commented on Nov 4, 2026

Hermes side of the generative-UI track: landed on fork branches (2026-11-03)

kvnloo commented on Nov 4, 2026

UX bundle plan + verification wave 1 (2026-11-04)

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions