Skip to content

fix: 0.13 backport, usage metrics on by default and the Kiro telemetry destination (#22397, #37183) - #37471

Merged
chenmingwei23 merged 1 commit into
release/0.13.0from
crew/backport-0.13.0-telemetry
Nov 15, 2026
Merged

chenmingwei23 merged 1 commit into
release/0.13.0from
crew/backport-0.13.0-telemetry

Conversation

@chenmingwei23

@chenmingwei23 chenmingwei23 commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Ship the default-on local usage metrics and the built-in Kiro telemetry destination in 0.13.0. Both merged on main after the 0.13 cut, so 0.13 does not have them today.

Backport onto release/0.13.0 (head b91bc8762644). That branch already carries #22394 (the Privacy chapter with Keep on / Turn off, 61a9a954f1); this PR adds the three main commits that the chapter's copy and the collection defaults depend on.

Each source commit was picked with git cherry-pick -x in main's merge order, then the picks and the settings help fix below were squashed into one commit, which lists the source PRs and their main commits. Two picks applied with no conflict; one carries hand-resolved hunks, listed below. No prerequisite commit was needed.

Pick Main commit What 0.13 users get
#22397 e58c25aa3293 Local usage recording on by default, with disk caps and a one-line CLI notice. Existing opt-outs are kept: heartbeat off with telemetry.enabled unset, telemetry.enabled: false (including quoted and non-bool values), KIROCREW_TELEMETRY=0, DO_NOT_TRACK=1.
#33422 ffd2339d95c0 RFC update: telemetry.enabled now defaults to true. Docs only.
#37183 c1222b94ef31 The public build sends the reviewed PRODUCT_METRICS list to the Kiro telemetry destination once privacy notice v3 has been shown; notice v3 names it; the OTLP exporter becomes a base dependency. Hand-resolved, see below.

Why it matters

The 0.13 release owner asked for these to ship in 0.13.0 final. Without #22397 and #37183 the 0.13 Privacy chapter (#22394) describes behaviour 0.13 does not have.

Not a goal

  • Any other main commit. Nothing outside the three picks is carried.
  • The All settings page entry for dashboard.kiro_telemetry_ack_version: that page is not on 0.13 (see below).

What changed (motivation -> approach -> change)

Hand-resolved hunks (#37183)

No code line was hand-resolved. Each hunk takes the pick's change and leaves out what belongs to a main commit that is not on 0.13. Original authorship is named in the commit message.

  • docs/guides/telemetry-otlp-export.md, the intro paragraph, the telemetry.enabled row of the Default posture table, and the paragraph on collection and egress: took the pick's text. 0.13's side still described recording as off by default, which is no longer true after feat(telemetry): record local usage metrics by default, with disk caps and a CLI notice #22397. Elsewhere in the file 0.13's own text is kept where main's differs because of commits not on 0.13 (the chat runtime sharing metrics table, the Ultra Mode wording, the traces troubleshooting row).
  • src/kiro_crew/dashboard/handlers/config_all_settings.py and test/test_config_all_settings.py: the pick adds one line to each, but neither file exists on 0.13 (the All settings page is a main-only feature). Left out.

Settings help text (not a pick)

The help for telemetry.enabled said default recording had no network egress, and the beacon_enabled help said recording egresses only to a configured destination after an explicit opt-in. With #37183 a default public install also sends the reviewed product metrics to Kiro telemetry once the notice that names it has been shown. This PR says so in both help strings and in the TelemetryConfig docstring, and regenerates config-baseline.json. Main has the same text; it needs the same fix there.

Backwards compatibility

Behaviour change by design: on upgrade, local recording turns on unless the install had opted out by one of the routes listed above. The Kiro telemetry destination sends nothing until notice v3 has been shown on the dashboard or printed on a terminal. Nothing is removed: no key, path or record.

Tests

Test changes come from the picks.

Release PRs get no backend test lanes in CI, so the backend tests ran locally with pytest -n 2. On the squashed head, all the files below together: 1,420 passed; the vitest files below: 159 passed; tsc clean.

  • The 16 test files the picks touch: 1,157 passed.

  • test_telemetry_consent_notice.py, metrics/test_kiro_export_destination.py, test_usage_notice.py, metrics/test_behavioral_metrics.py, test_collection_status_endpoint.py: 196 passed.

  • After the help-text fix: test_config_baseline.py, test_config_module_boundaries.py, test_file_line_cap.py, test_lts_telemetry_endpoint.py, test_telemetry_consent_notice.py, test_config_superseded_defaults.py, tracing/test_tracing_otlp.py: 494 passed.

Frontend on this head: tsc -p tsconfig.app.json clean, eslint on the changed files clean, scripts/check-i18n-keys.mjs OK. vitest: PrivacyChapter.test.tsx, PrivacyPanel.recordMetrics.test.tsx, PrivacyPanel.test.tsx, App.test.tsx, 159 passed.

flake8 on every changed Python file: clean. black: clean apart from src/kiro_crew/beacon.py and test/test_beacon.py, which are on .github/black-baseline.txt and unformatted on clean 0.13 and on main.

Manual verification

N/A. Unit coverage above; the same changes are on main.

Screenshots / video

Why no screenshot: every frontend change here is a backport of a main PR that carried its own screenshots.

Related Issues

Backport of #22397, #33422 and #37183.

Sibling sweep

Siblings: none -- the backport touches the same sites the picked PRs changed on main

Pattern harvest

Not generalizable: backport of already-reviewed changes

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...).
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license.

@chenmingwei23
chenmingwei23 requested review from CrysisDeu and removed request for a team November 15, 2026 02:46
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 15, 2026
@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ✅ PASS (a BLOCK with no Trigger and no Outcome does not block)

UX-level review of 9361645dcd144c3e9105130313fd166b8e50f157 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

UX-Verdict: PASS

cannot evaluate: missing screenshots of the Privacy chapter and Privacy settings copy this PR changes. The PR supplies no image, so the blind read was not performed.

Evidence gaps

  • Privacy chapter (PrivacyChapter.tsx), showing the Kiro telemetry state: the new privacy-kiro-telemetry line (whereBodyKiro), the whereBodyRecorded body and the turnOffScopeKiro Turn off scope next to Keep on. Closed by a screenshot of the chapter with the Kiro telemetry destination active.
  • Privacy chapter, showing the plain recording-on state (no Kiro telemetry), so the two wordings can be compared. Closed by a second chapter screenshot.
  • PrivacyDisclosureSections: the local-data line in its three forms (localDataBodyOn, localDataBodyOff and the default), the new doNotTrackCaption, and the shell commands now switched to DO_NOT_TRACK. Closed by a screenshot of Settings → Privacy with recording on and one with it off.
  • The description says the screenshots are on the main PRs (feat(telemetry): record local usage metrics by default, with disk caps and a CLI notice #22397 and feat(metrics): send product metrics to Kiro telemetry from the public build #37183). Those images show main's state and are not attached here, so they are not evidence for this revision. Attach them to this PR's description.

[UX-REVIEWED] 9361645

Description read

[DESCRIPTION-READ] 372aacfa4105db0ed1711c6953b68cf6a4bb08d495116fd8f34207d35717a931

That sha256 names everything this verdict read: the PR description, plus the 3 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS (a BLOCK with no Trigger and no Outcome does not block)

Design-level review of 9361645dcd144c3e9105130313fd166b8e50f157 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

This backport turns on network egress to a fixed Kiro endpoint in a release build, and that branch never runs in CI. The PR still says "Manual verification: N/A".

Blockers

Kiro telemetry egress has no manual verification on the 0.13 head
Trigger: the description says "Manual verification: N/A. Unit coverage above", but kiro_export.destinations() returns () exactly where tests run: "if _suppressed_for_tests or edition.is_lts() or beacon.is_ci(): return ()". It also returns () under any non-default KIROCREW_HOME. On top of that, "Release PRs get no backend test lanes in CI".
Outcome: the only path that posts to "https://telemetry.us-east-1.kiro.dev/v1/metrics" runs only on a real default-home install, and this is a hand-resolved pick onto 0.13 plus a new base dependency (opentelemetry-exporter-otlp-proto-http==1.44.0). If it goes wrong, 0.13.0 either exports before notice v3 is acknowledged or stamped, which can't be undone because the data has already left the machine, or it silently sends nothing. Unit tests with the destination suppressed can't tell these cases apart.
Fix: on a clean install of the release/0.13.0 head with the default home, observe the requests (proxy or debug log) in three states: before the notice is shown, after Keep on or the terminal stamp, and after DO_NOT_TRACK=1 or Turn off. Put the result in Manual verification.
Clears when: the description records that observed run: no request before notice v3, requests to the Kiro endpoint after it, no requests after an opt-out.

Notes

  • The PR is titled fix: but turns telemetry on by default for existing users at upgrade. The release notes should present it as a behaviour change, not a fix.
  • There is a visual delta: the Privacy chapter gains a Kiro telemetry line and a DO_NOT_TRACK caption, so <!-- no-visual-delta --> is inaccurate even though the main PRs carried screenshots.

[DESIGN-REVIEWED] 9361645

Description read

[DESCRIPTION-READ] c2e9e7ff5ec3b77847de28dc79419f2e9b2d8ebaa751d4629cf7b56abf9df193

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ human override accepted

@chenmingwei23 overrode this lane for 9361645dcd144c3e9105130313fd166b8e50f157 via /ai-review override (this commit only).

@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 9361645dcd144c3e9105130313fd166b8e50f157 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 9361645

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 9361645dcd144c3e9105130313fd166b8e50f157 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 9361645dcd144c3e9105130313fd166b8e50f157 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 9361645

Verdict parsed from the review's SHA-scoped output markers for commit 9361645dcd144c3e9105130313fd166b8e50f157.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 9361645dcd144c3e9105130313fd166b8e50f157 false-positive: <reason, 20 to 500 characters>

…y destination (#22397, #37183)

Backport to release/0.13.0 of three main commits, picked with
git cherry-pick -x in main's merge order and squashed into one commit:

- #22397 (e58c25a): record local usage
  metrics by default, with disk caps and a CLI notice. Existing opt-outs
  are kept: heartbeat off with telemetry.enabled unset, telemetry.enabled
  false (quoted and non-bool values included), KIROCREW_TELEMETRY=0,
  DO_NOT_TRACK=1.
- #33422 (ffd2339): RFC update,
  telemetry.enabled now defaults to true. Docs only.
- #37183 (c1222b9): the public build sends
  the reviewed PRODUCT_METRICS list to Kiro telemetry once privacy notice
  v3 has been shown; notice v3 names it; the OTLP exporter becomes a base
  dependency.

Hand-resolved for 0.13: in docs/guides/telemetry-otlp-export.md the intro,
the telemetry.enabled row and the collection/egress paragraph take the
pick's text, and 0.13's own text is kept elsewhere. The pick's one-line
additions to config_all_settings.py and its test are left out: the All
settings page is not on 0.13.

Also fixes the settings help for telemetry.enabled and beacon_enabled and
the TelemetryConfig docstring, which said default recording sends nothing
off the machine, and regenerates config-baseline.json.

Co-authored-by: Raymond Chen <chenmingwei23@users.noreply.github.com>
(cherry picked from commit e58c25a)
(cherry picked from commit ffd2339)
(cherry picked from commit c1222b9)
@chenmingwei23
chenmingwei23 force-pushed the crew/backport-0.13.0-telemetry branch from c7055a0 to 9361645 Compare November 15, 2026 03:34
@chenmingwei23

Copy link
Copy Markdown
Contributor Author

/ai-review override first-principles 9361645 owner-decision: backport of main's default-on local recording (#22397) and default Kiro telemetry export after the naming notice, LTS never (#37183), as the owner decided: #37183 (comment)

@github-actions

Copy link
Copy Markdown
Contributor

Human judgment recorded

@chenmingwei23 recorded a owner-decision release (class owner-decision) for the first-principles AI finding on 9361645dcd144c3e9105130313fd166b8e50f157.

backport of main's default-on local recording (#22397) and default Kiro telemetry export after the naming notice, LTS never (#37183), as the owner decided: #37183 (comment)

This decision applies only to this commit. A new push requires a new judgment.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 15, 2026
@chenmingwei23
chenmingwei23 merged commit de4155b into release/0.13.0 Nov 15, 2026
28 of 34 checks passed
@chenmingwei23
chenmingwei23 deleted the crew/backport-0.13.0-telemetry branch November 15, 2026 11:09
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 15, 2026
chenmingwei23 added a commit that referenced this pull request Nov 16, 2026
…f-form offer (#37460, #37463, #37545)

Backport to release/0.13.0 of three main commits, picked with
git cherry-pick -x in main's merge order and squashed into one commit.
They follow up #37183, which #37471 already put on 0.13:

- #37460 (2a679d6): the built-in
  kiro-telemetry destination is also returned under a non-default
  KIROCREW_HOME. kiro_export.destinations() drops only the
  is_default_home() check; the LTS, CI and test-suite gates and every
  consent gate stay. The heartbeat keeps its own default-home rule.
- #37463 (8de422d): installs that are off
  only because an older build wrote telemetry.enabled: false into their
  config are shown the Privacy chapter once, in an off form that names
  Kiro telemetry, with Turn on performance metrics / Keep off.
  kiro_export.off_offer_due decides that case; Keep off records only the
  notice version so it is not asked again.
- #37545 (09b0ddc): test only. The
  TestOffOffer tests fake the default data home, since the offer is asked
  only there and the test home is a private KIROCREW_HOME.

All three picks applied without conflicts; each backported patch is
identical to its main commit (git range-diff shows no change).

(cherry picked from commit 2a679d6)
(cherry picked from commit 8de422d)
(cherry picked from commit 09b0ddc)

Co-authored-by: Raymond Chen <chenmingwei23@users.noreply.github.com>
chenmingwei23 added a commit that referenced this pull request Nov 16, 2026
…f-form offer (#37460, #37463, #37545)

Backport to release/0.13.0 of three main commits, picked with
git cherry-pick -x in main's merge order and squashed into one commit.
They follow up #37183, which #37471 already put on 0.13:

- #37460 (2a679d6): the built-in
  kiro-telemetry destination is also returned under a non-default
  KIROCREW_HOME. kiro_export.destinations() drops only the
  is_default_home() check; the LTS, CI and test-suite gates and every
  consent gate stay. The heartbeat keeps its own default-home rule.
- #37463 (8de422d): installs that are off
  only because an older build wrote telemetry.enabled: false into their
  config are shown the Privacy chapter once, in an off form that names
  Kiro telemetry, with Turn on performance metrics / Keep off.
  kiro_export.off_offer_due decides that case; Keep off records only the
  notice version so it is not asked again.
- #37545 (09b0ddc): test only. The
  TestOffOffer tests fake the default data home, since the offer is asked
  only there and the test home is a private KIROCREW_HOME.

All three picks applied without conflicts; each backported patch is
identical to its main commit (git range-diff shows no change).

(cherry picked from commit 2a679d6)
(cherry picked from commit 8de422d)
(cherry picked from commit 09b0ddc)

Co-authored-by: Raymond Chen <chenmingwei23@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant