Skip to content

fix(chat): say in visible text that the tool-risk flag allowed and blocked nothing - #23869

Merged
iamwhatever merged 1 commit into
mainfrom
fix/12298-tool-risk-visible-claim
Oct 25, 2026
Merged

iamwhatever merged 1 commit into
mainfrom
fix/12298-tool-risk-visible-claim

Conversation

@iamwhatever

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A reader who cannot hover (touch or keyboard) can still see that Jev's risk badge on a tool card is only a flag, and did not allow or block the call.

The tool-card risk badge reads Jev: risky (0.88) Jev 👍👎. The sentence that says the flag changed nothing about whether the call was allowed lives only in a native title on the tier word. A title cannot be reached on touch or by keyboard, so those readers see a coloured "risky" word on a tool card and may read it as "this was blocked". This is item 3, first bullet, of #12298.

Why it matters

The badge's whole safety claim is that it is an annotation, not a decision. If only mouse users can read that claim, everyone else gets a badge that looks like a block. The UX review on #12253 called this the most worth doing of the four UX residuals.

Not a goal

What changed (motivation → approach → change)

Every reader now sees "flag only, did not allow or block" on the badge, after the score. Before, that claim lived only in a title tooltip, which touch and keyboard cannot reach. The text comes from a new key pages.chat.toolRiskBadge.annotation_note. The tooltip stays as the longer explanation. The phrase is worded so it never says the call ran or was not blocked, because a later permission check can still refuse a flagged call.

The new key has one line in each of the 12 translated catalogs, en.manual.json, the regenerated en-XA pseudolocale, and a translator note in en.context.json.

Backwards compatibility

Compatible: one new i18n key and one extra text span; nothing that works today stops working.
Removes nothing: the tooltip, the thumbs and every existing key stay as they are.

Tests

  • website/src/test/ToolRiskBadge.test.tsx: new case "says the flag allowed and blocked nothing in visible text, not only in a tooltip". It checks the note's visible text, that it is not carried by a title, and that the badge's text contains it.
  • Mutant: with the annotation_note call removed from the span, that case fails (1 failed, 22 passed); with it restored, all 23 pass.
  • i18n catalog suites (parity, QA, pseudolocale bundle, context sidecar, all 12 locale style gates, glossary, ratchets, dead keys, key reference): 23 files, 341 tests pass. npm run lint:i18n passes.

Proof: not provable -- frontend test (prove.py runs Python only)

Local GPT and Opus review lanes were skipped because the host was short on memory; the CI lanes review this PR instead.

Manual verification

Built the SPA and ran website/scripts/capture-tool-risk-badge.mjs against it. Both tiers in both themes show the new phrase on one line between the score and the thumbs.

Screenshots / video

Risky tier, light:

risky light

Caution tier, dark:

caution dark

Whole transcript, light and dark

transcript light

transcript dark

Related Issues

Part of #12298

Sibling sweep

Swept: git grep -n "annotation_title" -- website/src/
Siblings: none -- annotation_title is read only by ToolRiskBadge.tsx, the one place this claim is drawn

Pattern harvest

Not generalizable: one tooltip held a safety claim; other hover-only text in the app is advisory, not the meaning of the control

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@github-actions

github-actions Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 252719e5cd85b8686275365ab9311d637b8952af — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: it adds one visible span and one i18n key, takes nothing away, and the screenshots show this revision.

Notes

[DESIGN-REVIEWED] 252719e

Description read

[DESCRIPTION-READ] 11505f677d8f8b2c6b2558702bdc0320e39ef450c99bd184bfbb130a8927f0a5

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 252719e5cd85b8686275365ab9311d637b8952af — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks. The fix adds one visible span and one catalog key, and the defect it fixes is tracked in #12298, item 3.

What this change ships

Inventory (3 items) — 3 justified

Intent: let readers on touch or keyboard, who cannot reach the hover tooltip, see that the risk badge is a flag and not a block. This is a FIX.

  1. Every tool-card risk badge now shows "flag only, did not allow or block" between the score and the thumbs. — justified
  2. A new i18n key annotation_note, added to all 12 locales, the manual catalog, the pseudolocale and the translator-context sidecar. — justified
  3. A new test checks that the note is visible text and is not carried by a title. — justified

[FIRST-PRINCIPLES-REVIEWED] 252719e

Description read

[DESCRIPTION-READ] 08744bd692b13b8cb4c86c0f3d3387b4c96375c23322197540dac46e5a020b51

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 252719e5cd85b8686275365ab9311d637b8952af — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 252719e

Verdict parsed from the review's SHA-scoped output markers for commit 252719e5cd85b8686275365ab9311d637b8952af.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 252719e5cd85b8686275365ab9311d637b8952af false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ✅ PASS

UX-level review of 252719e5cd85b8686275365ab9311d637b8952af — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

UX-Verdict: PASS

Nothing blocks. The new note shows in all four screenshots, and the blind reader read it correctly: "the warning only marked the step and didn't stop it".

Notes

  • The blind reader said: "I'm left unsure whether the risky step actually happened." On a step like "Remove the tree…", the note doesn't say whether the step ran, so the user has to read the status icon for that. One option is to name who decided, without claiming the call ran: "flag only — your permission setting decided".
  • The reader saw "Jev" twice on each line ("Jev: risky … Jev 👍👎") and asked "why it appears twice". The new note now sits between the two, which makes the repeat more noticeable. Removing the repeated label is already tracked on Follow-up from #12253: tool.risk deferred findings (feature-map wording, late tool arguments, UX) #12298.

[UX-REVIEWED] 252719e

Description read

[DESCRIPTION-READ] bbab1e5bfbb0bb346497935de6c49363150021847ae6a4deca2100d27c9e0e65

That sha256 names everything this verdict read: the PR description, plus the 7 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 252719e5cd85b8686275365ab9311d637b8952af and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 252719e

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 252719e5cd85b8686275365ab9311d637b8952af false-positive: <reason, 20 to 500 characters>

@iamwhatever

Copy link
Copy Markdown
Contributor Author

Bundle Size Gate is red on this head because of a known red main, not this diff: the ModelEffortDropdown chunk is 505.1 KB against its 500 KB budget on pristine main too. Tracked in #23866, fix in #23867. This PR adds one short i18n string per locale. I will rebase once after #23867 merges.

🤖 AI-assisted comment 🤖

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 25, 2026
@iamwhatever
iamwhatever force-pushed the fix/12298-tool-risk-visible-claim branch from 9a9fff7 to 252719e Compare October 25, 2026 01:12
@iamwhatever

Copy link
Copy Markdown
Contributor Author

Rebased onto #23867 (the Bundle Size Gate fix for main): 0dab198 -> 3503760. New head 252719e. Badge and i18n catalog tests pass locally (19 files, 280 tests); CI re-running.

🤖 AI-assisted comment 🤖

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 25, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor Author

Review-ready at head 252719e: PR Readiness passed, 106 checks with 0 failing, the PR is mergeable and green on today's base (3503760). Design, First Principles and UX passed, and the GPT and Opus lanes have no blocking findings. This needs a maintainer's approval; I have not merged, approved or turned on auto-merge.

🤖 AI-assisted comment 🤖

@cruisercohen cruisercohen left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving 252719e5cd. This is a UX fix for #12298 item 3 and is not security scope.

The risk badge now states its key claim in visible text after the tier word and score ("flag only, did not allow or block"), so touch and keyboard readers no longer depend on a title they can't reach. The new string is annotation_note, added once in en.manual.json (where new English strings live), in all 12 translated locales and in en-XA, with a translator context entry. Nothing else in the locales changes. The test pins the visible text, checks the note carries no title, and checks the badge's own text contains it.

I could not run the website vitest suite locally, so this approval rests on the diff and the key check above. CI's readiness passed (91 pass, 15 skipping).

@buluoray buluoray left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving at 252719e5cd85b8686275365ab9311d637b8952af.

The badge's key claim, that Jev's flag neither allowed nor blocked the call, is now visible text instead of a hover-only title, so touch and keyboard readers get it too (ToolRiskBadge.tsx:66-70). The wording avoids saying the call ran, which matches the tooltip's "may still have been refused".

Checked:

  • The new annotation_note key is in all 12 translated catalogs, en.manual.json, the en-XA pseudolocale and en.context.json; I read each translation and all carry the same "flag only, neither allowed nor blocked" meaning.
  • ToolRiskBadge.test.tsx 23/23, src/i18n 51 files / 1035 tests, lint:i18n, and tsc -p tsconfig.app.json all pass locally on this head.
  • Removing the annotation_note call from the span turns "says the flag allowed and blocked nothing in visible text, not only in a tooltip" red.

Non-blocking follow-up: the translator note for tier_risky in en.context.json still says "the call ran"; worth aligning with the tooltip's softer claim in a later change.

@iamwhatever
iamwhatever enabled auto-merge (squash) October 25, 2026 06:35
@iamwhatever
iamwhatever merged commit 8ed35ca into main Oct 25, 2026
108 checks passed
@iamwhatever
iamwhatever deleted the fix/12298-tool-risk-visible-claim branch October 25, 2026 07:11
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants