Repository navigation
feat(chat): show which skills Jev picked, under the reply - #11994
Conversation
|
Intent: Give a reader of an assistant reply a receipt for the skill pick behind it — who picked what, how sure, what it saved — and a way to say whether that pick was right, for either side. Not a goal: the backend that stamps the record or serves the feedback route (a sibling branch); any surface for the aggregate |
Design Review (Fable 5) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS Spec fixture promises WatchThe contract is one-sided: Suggestions
[DESIGN-REVIEWED] 54cb701 |
First Principles Review (Fable 5) — 🟡 CONCERNSPremise-level review of First-Principles-Verdict: CONCERNS § 8 pins a top-level Not justified as shipped
What this change shipsInventory (10 items) — 8 justifiedIntent: let a reader of a reply see which skills Jev picked versus the word-matching rule, and rate the pick — an ADDITION.
[FIRST-PRINCIPLES-REVIEWED] 54cb701 |
Opus 4.8 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
UX Review (Fable 5) — 🟡 CONCERNSUX-level review of UX-Verdict: CONCERNS Two identical thumb pairs, each labelled with one faint word; the blind reader couldn't tell which pick each rates and "could easily press the wrong one." Watch
Suggestions
[UX-REVIEWED] 54cb701 |
GPT 5.6 Review — ✅ no blocking findings (all downgraded on adjudication)GPT 5.6 flagged blocking issues on This comment is updated in place on each push. Review detailsBLOCKING -- website/src/pages/chat/AssistantMessage.tsx:544 -- Variant switching misattributes the decision receipt Adjudication (Opus 4.8) — is blocking on each finding proportionate?I've confirmed the mechanism. Let me verify the recovery path — that the strip re-reads the prop on every render. The finding is clear from the code I've opened:
So switching to an older variant changes the reply prose but leaves the decision strip describing the primary (newest) turn. Weighing:
This is the LOW rung where False positive or not applicable? A repository writer can comment: |
7decd22 to
5d39db4
Compare
5d39db4 to
6d83486
Compare
|
span=7804e6a24f0e
|
|
span=e57ec16f1aff
|
|
span=9e54d236132b
|
|
span=4e811c121ed1
|
|
span=fd4154fb3ec8
|
|
span=320bff29ec2b
|
|
span=0547f77c2a45
|
|
span=59a99a6e1758
|
|
span=158c71389c34
|
buluoray
left a comment
There was a problem hiding this comment.
Approving at head 44a71339a3f2fafae45152d9181c5e193193637c. No blocking findings. Verified at this head; I did not take the disposition comments on trust.
The inertness the body discloses is true rather than understated: git grep decisions_strip -- src/kiro_crew is 0 and git grep "decisions/feedback" -- src/kiro_crew is 0 at this head. I am not re-litigating the split — the sibling gateway PR #11993 is the producer and I reviewed it in parallel.
The reader cannot draw a wrong statement
This is the property that matters most for a receipt, and it holds. asNames (website/src/pages/chat/decisionRecord.ts:101-105) returns null unless the value is an array whose every element is a non-empty string — it does not filter bad elements out. readDecisionStrip (:144-146) then refuses the whole record on baseline === null || jev === null before building anything, so sameSet (:108-114) is only ever reached with two lists that were read whole. Two differently-broken lists can no longer both collapse to [] and print "same pick · no skills" about a record the reader never understood. The GPT finding at span 11c7a563720a is genuinely closed, and closed in the stronger form (refuse the record) rather than by patching the comparison.
agree is recomputed from the two lists at :166 rather than read off the wire, so a hostile or stale flag cannot put a check mark over a real divergence. p outside 0–1 reads as null (:150).
Injection, theme, i18n
No HTML sink. No dangerouslySetInnerHTML or innerHTML anywhere in DecisionStrip.tsx; every server-supplied string — skill names included — renders as a JSX text child (:213,217,219,252-253,267 via names()/fmtList), which React escapes. The repo's prior markdown-injection class has no path here because there is no markdown or HTML rendering step.
Theme pairs are intact. No hex, rgb(, bg-white, text-black or literal Tailwind palette in the component. Colors are paired tokens only: bg-card text-muted ring-border on the container (:174), text-text on values (:120,195), hover:text-text (:187). forced-colors:border is present, so high-contrast mode keeps the strip's edge. No half-set foreground/background pair.
All 13 locales are key-symmetric. I extracted the added key set from each edited catalog: en.manual, en-XA, and bn de es fr hi it ja ko pt ru zh-CN carry an identical 25-line set (24 leaves under decisionStrip plus the parent). No locale carries a key en.manual lacks, and none is missing one it has. en-XA is genuinely machine-pseudolocalized — accented glyphs with bracket padding, e.g. "agreed_short": "[şàɱè þìçķ ···]" — with {{names}} placeholders preserved, so it was regenerated rather than hand-edited.
The spec contract really binds
decisionStripContract.test.ts:32-39 reads docs/system-specs/modules/decisions.md with readFileSync, locates ## 7. The decision strip's record and feedback, and JSON.parses the first fenced json block — it is not a hand-copied literal, and specFeedbackVocabulary() (:42-47) slices live spec text the same way. I confirmed the spec actually holds what the test looks for (§7 header at line 47, the json fence at 62, POST /api/decisions/feedback at 102, the retract prose at 105-106). A field renamed or respelled in the spec reds :59-73, :83, :125 and :133. So the reader is bound to a reviewed document rather than to itself, which is what the design lane asked for.
One non-blocking note on the ride-along gate
fileNameCasing.test.ts:89-101 walks all of website/src (skipping only node_modules and __snapshots__) and fails on any directory holding a case-only filename or importable-stem collision. It is a repo-wide gate landing inside a feature PR and it will run on every future PR.
The scope risk is real but bounded, and I do not think it should be split out. NOT_A_SPECIFIER_TARGET (:33) exempts .test / .spec / .d.ts pairs — which is why the two such pairs already on main stay green — and caseOnlyStems (:66) fires only when stems differ in case, so Foo.ts beside Foo.tsx is not flagged. The only way it reds an unrelated PR is if that PR introduces a genuine case-only module collision, which is a real macOS/Windows build break rather than a spurious failure. Worth knowing it scans the tree rather than the diff, so the failure it reports may name a file the PR did not touch.
Not merging; that is the maintainer's call. Whichever of this and #11993 lands second is the one that makes the feature visible, so the §7 fixture assertion on the producer side is the thing to confirm at that point.
e8480fe
44a7133 to
e8480fe
Compare
|
span=2f6082500a24
|
|
span=aa54965f8969
|
|
span=2f6082500a24
|
|
span=c37ae281ac99
|
|
span=3e40ecd07e9f
|
49e6f99 to
62a19cf
Compare
GPT 5.6 Review — 🔴 changes requested (blocking)GPT 5.6 found at least one blocking issue that must be resolved before merging This comment is updated in place on each push. BLOCKING -- docs/feature-map/README.md:87 -- The feature map contradicts itself
Adjudication (Opus 4.8) — is blocking on each finding proportionate?Line 87 (Decision strip row) lists the strip as shipped — "Rendered by every transcript host, the main chat and the panes alike" — with real component hosts ( The remedy is a one-clause deletion of the stale Harm rung: LOW (developer-doc contradiction, visible, self-correcting on code read). Conditions: contradiction confirmed at docs/feature-map/README.md:87 (shipped claim) and :285 ("NO shipped surface"). Real fix: remove the stale clause at :285 — near-zero cost, so no disproportion exists to justify downgrade. False positive or not applicable? A repository writer can comment: |
A one-line strip under an assistant reply says how that turn's skills were chosen: which the shipped word-matching rule would have loaded, which Jev answered, how sure it was, and the prompt tokens the narrower set saved. Agreement reads as one muted check over one list; a divergence names both. Opening it adds the question's own shape -- candidates offered, batches, characters of history carried, what was left out, which answers the gate refused and with what score -- and a second thumbs pair, so a reader can rate the word-matching rule and not only Jev. Every field is validated at the read, and `agree` is derived from the two name lists rather than trusted, so the strip cannot print a check over a real divergence. The strip appears only where the record exists AND the Decisions (Jev) switch is on, read through the Settings card's own consent query.
62a19cf to
54cb701
Compare
GPT 5.6 Review —
|
GPT 5.6 Review —
|
Problem / Motivation
The Decisions (Jev) preview can pick your skills for you. Nothing shows you what it picked.
A reader flips the switch in Settings › Developer › Feature Previews, and from then on some of their turns load skills Jev chose instead of the skills the shipped word-matching rule would have chosen. The reply looks the same either way. The only record is a line in a log file on the machine running Kiro Crew, so the one question a reader of that reply actually has — did the model get the right skills for this? — has no answer on screen, and no way to say yes or no.
Why it matters
A preview nobody can see is a preview nobody can judge. The switch asks a reader to send their message text off the machine, and gives them nothing back they can weigh against that. Without a per-turn receipt there is also no feedback: the people who can tell a good skill pick from a bad one are reading the reply, and they had no way to say so.
What changed (motivation → approach → change)
Under an assistant reply whose skills Jev chose, there is now one line saying what happened, with two thumbs to rate it.
Collapsed it reads like this, and truncates rather than wrapping:
skills · word matching: brazil, crux-code-reviews · Jev: brazil · (0.81) · saved 3.2K tokWhen the two sides picked the same skills there is nothing to compare, so the line prints that set once —
same pick · Jev and word match: brazil— naming both sides, because otherwise nothing on an agreed line says Jev was involved at all. When they differ it names each with its own list, because the divergence is the only thing a reader can act on.POSTed per side, each pair labelled with the side it rates (Jev,Word match). Jev's pair is on the line; the word-matching rule gets its own pair when open, so a reader can say the old rule was the better oneflowchart LR subgraph after["After"] direction LR R2["assistant reply"] --> S["skills · who picked what · 0.81 · saved 3.2K tok"] S --> T["👍 👎"] S --> E["open: candidates · batches · refused · error"] end subgraph before["Before"] direction LR R1["assistant reply"] --> L["a line in a log file"] end class R1 ctx class R2 ctx class L ctx class S added class T added class E added classDef added fill:#DCFCE7,stroke:#16A34A classDef changed fill:#FEF3C7,stroke:#D97706 classDef removed fill:#FEE2E2,stroke:#DC2626,stroke-dasharray:4 3 classDef ctx fill:#E0F2FE,stroke:#0284C7🟩 added · 🟨 changed · 🟥 removed · 🟦 unchanged
A reader of the reply now sees which skills were picked and by which side, and can say whether that was right.
Three choices keep it honest.
It cannot print a check over a real divergence.
pages/chat/decisionRecord.tsvalidates every field and draws nothing for a record it cannot check — including one whose two skill lists are not both readable whole, because two lists broken in different ways would each reduce to empty and then compare equal. Agreement is derived from the two name lists, never read from the wire, so a record claiming the sides agreed while naming different skills cannot hide that behind a check mark.The contract is written down.
docs/system-specs/modules/decisions.mdpins the record shape and the feedback wire, andsrc/test/decisionStripContract.test.tsparses that fixture out of the spec rather than restating it. A spec edit that the reader does not follow reds a test here instead of making the strip quietly never appear — which matters because drawing nothing is also what a release with no record looks like. The producer side is not bound to it: the base stampsmeta.decisions_stripfromchat_runner.pyandbuild_outcomedecides the keys, and that producer asserts no fixture. Three keys the § 8 fixture names --batches,droppedanderror-- have no stamping site at all, andagreeis stamped although the fixture drops it. Reconciling the two sides is the next change on this branch, and the reader's tolerance of an absent key is what keeps the gap cosmetic meanwhile.A receipt is history, so the switch does not hide it. The strip never reads consent. A stamped record already sits on this machine, so drawing it sends nothing, and the switch governs whether a future turn may ask Jev. Gating on it would cost an owner who tried the preview and turned it off the record of which past replies were Jev-picked.
The rest is plumbing.
pages/chat/AssistantMessage.tsxvalidates the record and draws the strip directly under the bubble, above the file chips;pages/ChatPage.tsxandapp-sdk/messageRenderers.tsxeach pass one property;types/index.tsdeclares where the record rides;api/client.tsgains the feedback call and its two wire vocabularies; every catalog underwebsite/src/i18n/localesgains the same 21 strings in its own language, andi18n/en.context.jsongains one translator note for the three-letterJevlabel;src/testholds the suite, the spec-contract test and one guard on file-name casing;scripts/capture-decision-strip.mjsand itspackage.jsonentry are the screenshot harness (no dependency added or removed);docs/feature-map/README.mdgains the row.Backwards compatibility
Compatible: every field is optional and a row without the record renders exactly as a row does without one -- which is every turn the seam did not decide, the ordinary case, since the preview is off by default. The strip is additive and adds no required input anywhere.
Tests
website/src/test/DecisionStrip.test.tsx, 27 cases, 98.6% statement and 96.3% branch coverage of the two new files.aria-expanded, and both thumbs carry a label.agreenorts— nothing reads either — and a key the reader does not know is ignored, so a producer may add fields.Jev,Word match) in visible text and in a tooltip, and the two labels differ, so a thumb cannot be read as counting for both.same pickand names both sides; the counts readSelection rounds,Context sentandSkills cut from the offerrather than mechanism words.website/src/test/fileNameCasing.test.tsguards a cross-platform trap this PR walked into. A module whose name differs from a sibling only in case resolves to ONE module on macOS and Windows, and TypeScript then refuses every import of both — while a Linux checkout compiles clean, so the first failure is the slowest job in CI. The reader module is nameddecisionRecord.tsfor that reason, and the gate now fails on the pair rather than the desktop matrix. It reaches only files a specifier can name: a test or spec file is collected by glob, never resolved, which is why two of those already pair this way on the default branch with the macOS leg green.Manual verification
Captured from the real built SPA through
scripts/capture-decision-strip.mjs(the shared transcript harness: realwebsite/dist, every/api/**call from fixtures, no gateway and no Jev), and the rendered text read back to confirm the copy resolves rather than printing key paths.Screenshots / video
Diverged and agreed, collapsed:
Open, with a refusal and a failed decision:
Dark
Related Issues
#11997 -- this is the frontend half of the Decisions (Jev) skills-select seam. The backend half is in the base: a sampled turn's outcome rides its reply as
meta.decisions_stripandPOST /api/decisions/feedbackrecords a verdict, both from #11993. So the strip draws for a decided turn as soon as this lands, and draws nothing on every turn the seam did not decide, which is the state every test above pins.Checklist
feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)