Repository navigation
fix(kas): show the telemetry-off notice once per chat - #35788
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: the per-chat mark targets the real cause, repeated session starts, and the optional metadata key is backward-compatible and easy to revert. Notes
[DESIGN-REVIEWED] a018133 Description read[DESCRIPTION-READ] a9a4d489205950c170ea7ec6452d4a644e0454f4fc07e6667ad8327513b55a5a That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks. Each item traces to the defect in #35781 and is covered by tests that fail on base. The per-chat mark sits at the right level: it records that the chat was already told. What this change shipsInventory (5 items) — 5 justifiedIntent: a user who opted out of kiro-cli telemetry sees the KAS telemetry notice once per chat, not on almost every message. This is a FIX.
[FIRST-PRINCIPLES-REVIEWED] a018133 Description read[DESCRIPTION-READ] 27ec7abd4524c6ab8c94bf2371dd410df2d22612e6d36f0542ca7a40837b9947 That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
bolichen97
left a comment
There was a problem hiding this comment.
Requesting changes on one point. The rest looks right to me.
New regex in product code. dashboard/slot_persistence/metadata_codec.py adds:
_TELEMETRY_NOTICE_MARK_RE = re.compile(r"\A[0-9a-f]{16}\Z")We keep new regex out of product code. A plain check does the same job:
_HEX = frozenset("0123456789abcdef")
def _read_telemetry_notice_shown(r: _Read) -> None:
mark = r.meta.get("telemetry_notice_shown")
if isinstance(mark, str) and len(mark) == 16 and set(mark) <= _HEX:
r.slot._telemetry_notice_shown = markThe existing "no mark" case in test_a_restart_does_not_repeat_the_telemetry_notice still covers the reject path.
The rest I read in full and have no concerns. _telemetry_notice_is_new dedupes only telemetry_unconfirmed_notice, stores a 16-char sha256 mark, and shows the notice again when its text changes. The metadata field is wired like human_seen, and the tests cover the once-per-chat case, a changed answer, a second chat and a restart.
Not checked: I ran no tests locally and relied on CI (96 green).
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Tier 1 auto-approve: fix (8 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: a dashboard chat shows the KAS telemetry notice once, keyed by a mark of its text saved on the chat's metadata line, with tests for repeat sessions and restart. Spec files changed as a ride-along (a minority of the diff on both file count and changed lines), not reviewed as a design decision: docs/system-specs/modules/acp-client.md.
Withdrawn: a maintainer requested changes on this head shortly before this approval was posted, so its no-requested-changes criterion no longer holds. Auto-merge is off; this PR now waits for a maintainer.
A KAS chat whose user turned kiro-cli telemetry off showed the "telemetry is off" notice on the first turn of every session the chat created or reloaded: after an idle timeout, a gateway restart, or a model, project or harness switch. In a long chat that was almost every message. The chat now keeps a mark of the last telemetry notice it showed and adds a row only when the notice text differs from it. The mark is saved on the chat's metadata line (telemetry_notice_shown), so a restart keeps it. Closes #35781
051fcd7 to
a018133
Compare
|
@bolichen97 I dropped |
Code change approver recommendation: ApproveWhy: the decider estimates a 12% chance that this change introduces a defect, within the 12% limit for approving on its own, and every check passes. The PR adds a per-chat mark (a 16-character hex digest of the last telemetry notice text) and adds the notice row only when the text changes. The mark is saved as an unowned, set-and-never-cleared metadata key, which the full save keeps from the disk copy (carry_unowned_metadata). The read accepts only exactly 16 hex characters, so a forged value just means the chat is told again. The only production caller passes a real slot that always has the field. Existing tests that pass object() never set the telemetry notice. The new tests cover several session starts, switching between the two notice texts, separate chats, a restart from disk, a forged value and the codec round trip. No regression on sibling notices. An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit |
bolichen97
left a comment
There was a problem hiding this comment.
Approving on a018133. My regex point is fixed: _TELEMETRY_NOTICE_MARK_RE is gone, and _read_telemetry_notice_shown now checks len(mark) == 16 and set(mark) <= _HEX. I compared this diff with the one I reviewed on 051fcd7. Apart from that swap, only hunk offsets from main moved. There is no re.compile or other regex in the diff.
Not checked: I ran no tests locally and relied on CI (97 green).
Problem / Motivation
Goal: A KAS chat tells a user who turned kiro-cli telemetry off about it once, not on almost every message.
A user turned kiro-cli telemetry off (
telemetry.enabled: false) and chatted on KAS. The chat kept showing the same notice: "ℹ Your kiro-cli telemetry is off, so Kiro Crew asked the KAS engine to turn its telemetry off too. KAS does not confirm this, so Kiro Crew cannot check that it took effect." In a long dashboard chat it showed on almost every message.Why it matters
Every user who opted out of kiro-cli telemetry and chats on KAS gets this row over and over. Nothing they do in the chat makes it stop. The rows bury the conversation, and they teach users to skip notices.
Not a goal
What changed (motivation → approach → change)
A chat now shows the telemetry notice once. It shows a telemetry notice again only when the text is a different one.
Symptom: the notice repeats. Cause: a chat goes through many KAS sessions. It gets one after it sat idle past
session.timeout_secs(60 minutes by default), after a gateway restart, and after a model, project or harness switch. Every session the chat creates or reloads carries the notice, and the dashboard added a row on the first turn of each one. Nothing remembered that the chat was already told.Change:
_notice_steer_upgradeinsrc/kiro_crew/dashboard/chat_turn/turn_spec_hooks.pykeeps a short mark of the last telemetry notice text on the chat's slot (_telemetry_notice_shown, instate_parts/chat_slot.pyandstate_parts/slot_session.py). It adds the row only when the text differs from that mark.telemetry_notice_shown(slot_persistence/metadata_codec.py), so a gateway restart keeps it. A value on that line that is not a mark is ignored, and the chat is told again.docs/system-specs/modules/acp-client.mddescribes the new rule.Two choices to note:
kiro-cli and the kiro harness show no such notice, so they set no rate to match. The change that added the notice (#33026) meant one row per chat.
Backwards compatibility
Compatible: the metadata line gains one optional key. A line without it reads as "not told yet", so an existing chat shows the notice one more time and then stays quiet.
Removes nothing: the diff adds a slot field, a metadata key and a check before one notice row. No path, key, record or function is taken away, and the notice still shows once per chat.
Tests
test/test_kas_cli_settings.py::test_a_chat_shows_the_telemetry_notice_once: three session starts of one chat give one row; the other notice shows once; a second chat is told on its own.test/test_kas_cli_settings.py::test_a_restart_does_not_repeat_the_telemetry_notice: a chat is saved with the real save, dropped, and restored from disk; its next session start adds no row. A line whose value is not a mark tells the chat again.test/test_slot_metadata_codec.py: the new key in the round trip for every read purpose, and in both key orders.test/test_state_parts_facade.py: the new slot field's default.Proof: PROVEN test/test_kas_cli_settings.py::test_a_chat_shows_the_telemetry_notice_once, test/test_kas_cli_settings.py::test_a_restart_does_not_repeat_the_telemetry_notice, test/test_state_parts_facade.py::test_a_fresh_slot_has_every_base_default (head a018133)
Determinism of the new and changed tests: 20 runs at
-n0, 10 runs at-n 4 --dist loadgroup --max-worker-restart=2withtest/test_kas_agent_prompt_floor.pyandtest/test_kas_withheld_credentials_notice.pyas co-tenants, and one shuffled order of the three test files. All green.Manual verification
Run on an isolated preview gateway built from this change, in a real KAS chat. The run was made before the last rebase onto main; the rebase changed no line of the diff. The chat's project folder had a
.kiro/settings/cli.jsonwith"telemetry.enabled": false.session/openedfor the chat. No new notice row.Result: five rows (user, notice, assistant, user, assistant), one telemetry notice. The chat's metadata line carried
telemetry_notice_shown.Related Issues
Closes #35781
Sibling sweep
Swept: git grep -n "if is_new" -- src/kiro_crew/dashboard/chat_turn/
Sibling: src/kiro_crew/dashboard/chat_turn/turn_spec_hooks.py -- not affected -- its other session-start rows either name a fix the user can apply (the six other rows of
_notice_steer_upgrade: an update command, a setting, the model picker, a file to remove) or state a fault of that session (Crew's tools are off; a field of the agent spec has no effect on this backend), so they end when the user actsSibling: src/kiro_crew/dashboard/chat_turn/mcp_session.py:236 -- not affected -- the row says this session lacks one of Crew's own servers, a fault of that session that its next session may not have
Pattern harvest
Not generalizable: the defect is one notice with no step the user can take, drawn by a loop built for notices that end when the user acts.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributions are accepted under the project's Apache-2.0 license.