Skip to content

feat(metrics): send product metrics to Kiro telemetry from the public build - #37183

Merged
chenmingwei23 merged 1 commit into
mainfrom
feat/kiro-telemetry-default
Nov 15, 2026
Merged

chenmingwei23 merged 1 commit into
mainfrom
feat/kiro-telemetry-default

Conversation

@chenmingwei23

@chenmingwei23 chenmingwei23 commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: The public build sends the reviewed product metrics to Kiro telemetry while local recording is on, after the user has been shown a privacy notice that names it; LTS builds never send there.

Today the public DefaultTelemetryProvider.otlp_destinations returns only the operator's own telemetry.otlp_endpoint(s) entries, so the public build sends product metrics nowhere. The seam for a build-shipped destination (narrowing fields from #26480, label_sources, the PRODUCT_METRICS preset) is already on main; nothing in the public build uses it.

(Goal and Not a goal updated on the owner's instruction of 2026-11-14: default recording also exports to this destination, after a new notice; LTS never.)

Why it matters

Maintainers cannot see how the public build is used beyond the daily heartbeat count. The product metrics list was reviewed for export.

Not a goal

  • Changing the export rule for an operator's own collectors: they still need telemetry.enabled: true written or KIROCREW_TELEMETRY=1.
  • Sending any credential: no sign-in token is sent.
  • Any destination in an LTS build.

What changed (motivation -> approach -> change)

Owner decision (2026-11-14): in the public, non-LTS build, the built-in Kiro telemetry destination exports wherever local recording is on, including by default, but only after the user was shown a notice that names it (the dashboard's Privacy chapter once, with Keep on / Turn off, or the one-line CLI/headless notice). Before that nothing goes out. Every existing off switch stays off. LTS never sends there.

Approach: a destination built in code by the public provider, using the existing seam, plus one per-destination export rule in the recorder build and a notice version bump.

Want Change
One destination for product metrics New src/kiro_crew/metrics/kiro_export.py: kiro-telemetry, endpoint constant https://telemetry.us-east-1.kiro.dev/v1/metrics, allow=PRODUCT_METRICS, point_attributes={"ClientType": "kiro-crew"}, resource_allow without service.instance.id, process.pid, os.type (Settings says the operating system is never sent) or host.*, temporality="delta", process_env_credentials=False. DefaultTelemetryProvider.otlp_destinations returns config entries first, then it.
No credential leaves the machine The session adds only x-kiro-machineid (SHA-256 of the existing install id; never creates the id). No request without it, none to any other URL, no redirects, standard TLS verification, no client certificate.
Default recording exports to it, operator entries still need the opt-in _build_recorder keeps kiro_export.exported(destinations, cfg, outbound): this destination when kiro_export.notice_shown(), every other one only when outbound_consented.
Only after the notice that names it PRIVACY_NOTICE_VERSION 2 -> 3 (beacon.py, privacyNotice.ts), so the Privacy chapter shows once more to users who saw v2, and it gets a whereBodyKiro line naming Kiro telemetry (all 13 catalogs + pseudolocale). The CLI/headless line (metrics/notice.py) names it and now stamps the version it printed, so v2 stamps print once more. notice_shown = dashboard.privacy_ack_version >= 3 (Keep on, or kirocrew telemetry enable) or the CLI stamp at 3. A telemetry.enabled: true written under an older notice does not count.
The notice really reached the person notice_shown = dashboard.privacy_ack_version >= 3, or a separate kiro_telemetry_notice_shown stamp written only when a start with no dashboard (gateway --no-dashboard, run, chat) printed the Kiro line and the print succeeded. A headless start prints it even after a dashboard start stamped the generic line. The generic CLI stamp, the local-only line and a dashboard gateway's log line do not count.
Mixed collectors are disclosed apart Settings and kirocrew telemetry enable name the Kiro export on its own line beside a collector of the user's own that still waits for the opt-in. The chapter's first paragraph uses new whereBodyRecorded* strings (recorded here) when only Kiro telemetry gets metrics.
Acknowledged for this destination New dashboard.kiro_telemetry_ack_version (config field, theme-config PUT, config-baseline.json), written by Keep on only when the chapter showed the Kiro line, and by kirocrew telemetry enable only after its Kiro line printed to a terminal; read-only in the all-settings page. notice_shown reads it, not privacy_ack_version. The headless stamp is written only when the line went to a terminal. Turn off's scope line names the Kiro export; the Telemetry panel footer drops "local-only, no egress".
Disclosure while consent is unreadable live_recorder_state also reports a kept Kiro export (kiro_export.last_build_kept); export_permitted stays the operator opt-in, and the chapter's operator line no longer counts the built-in row.
A Keep on in a running gateway takes effect notice_shown() is part of _endpoints_key, so the next consent recheck rebuilds and attaches the reader.
Settings switch and telemetry enable keep working Because the notice names it, the enable refusals (core_config_patch.py 409, cli_commands.py) ask kiro_export.egress_needs_written_opt_in, which ignores it. The collection status adds enable_refused; PrivacyPanel.tsx pins the switch on that (falling back to otlp_configured), and has an off-state note for this destination. export_permitted is true when it is listed.
Not where it must not send () in an LTS build (with or without a policy collector), on a CI runner, under a non-default KIROCREW_HOME.
The exporter is present opentelemetry-exporter-otlp-proto-http==1.44.0 moves into install_requires (same pin as the otlp extra, which stays).
Disclosure Settings > Privacy lists it as included by the build; kirocrew telemetry status gets a Collectors: line; telemetry enable names it.
Tests never reach the network Autouse fixture in test/conftest.py turns it off; its own tests turn it back on.
Docs Export guide section and who-sends table, metrics.md, README privacy text, the crew-log telemetry RFC's export rule, the provider contract docstring.

Who sends after this change (public build, default data home, not CI):

Host Records locally Sends to Kiro telemetry
New install, notice v3 shown yes yes
Any host before notice v3 is shown yes no
Upgrade with heartbeat on, enabled never written, notice v3 shown yes yes
"enabled": true written under the v2 notice, v3 not shown yet yes no
Upgrade with heartbeat off, false / non-bool written, KIROCREW_TELEMETRY=0, DO_NOT_TRACK=1, telemetry disable no no
Policy otlp_hosts does not permit the host yes no
LTS build (with or without a policy collector) yes no
CI runner or non-default KIROCREW_HOME yes no
Operator otlp_endpoints entry, default recording yes yes (the operator entry does not export)

Backwards compatibility

No key, route or function is removed or renamed; the operator's entries keep their order, descriptors and opt-in rule. Behaviour changes, by the owner's decision: public-build users see the Privacy chapter (or the CLI line) once more, and after that their product metrics go to Kiro telemetry while recording is on. The collection status gains one field (enable_refused); a client that ignores it keeps today's behaviour. The exporter package becomes a base dependency.

Tests

  • test/metrics/test_kiro_export_destination.py: one test per row of the table above through real config files and the real get_recorder() (notice shown / not shown, CLI stamp v3 vs old stamp, written true under v2, heartbeat on/off, off switches, CI, operator entry with and without the opt-in), the rebuild fingerprint moving when the notice is shown, the enable refusals ignoring it and still refusing for an operator entry, the collection status fields, telemetry enable turning recording on and naming it, the CLI line text and stamp, provider order and narrowing, an operator entry with the same name is not it, LTS / CI / home / suite guard hide it, an LTS policy collector is the only destination, a policy host list refuses it, identity (machine id only, stored sign-in never sent), and one wire test through the real provider, build and OTLP exporter to a loopback receiver.
  • Updated for the version bump and the new field: test/test_telemetry_consent_notice.py (current version 3), test/test_telemetry_handlers_cov80.py (enable_refused in the payload).
  • Frontend: PrivacyChapter.test.tsx (names Kiro telemetry for the build's destination, not for an operator entry of the same name), PrivacyPanel.recordMetrics.test.tsx (switch enabled and writes when only this destination exists).
  • Run locally (-n 2 / vitest run): 1,057 + 996 backend tests across the metrics, telemetry, config-patch, LTS, beacon and CLI suites; 228 frontend tests including catalog parity; tsc --noEmit, eslint, the untranslated-values, DNT and pseudolocale checks.

Manual verification

Not done against the real endpoint from this machine: dev homes do not list the destination by design. The wire test drives the real provider, build and exporter against a loopback receiver. A real accepted export to the endpoint should be shown before merge.

Screenshots / video

Captured with scripted Playwright against the built SPA on a loopback static server, with /api stubbed (no gateway).

Privacy chapter re-shown for notice v3 (recording on, acknowledged v2, no collector of the user's own): metrics are recorded here, Kiro telemetry is named once on its own line, and the Turn off line says it stops that export too.

Privacy chapter at notice version 3

Settings > Privacy, recording off, only the build's destination: the switch stays usable and the note says turning it on also sends product usage metrics.

Settings Privacy, recording off

Settings > Privacy, recording on, only the build's destination.

Settings Privacy, recording on

Settings > Privacy, recording on, beside a collector of the user's own that still waits for the opt-in: "Your own collectors get nothing..." and the Kiro line are named apart.

Settings Privacy, recording on, mixed collectors

Related Issues

Related #21270. Supersedes the closed draft #22221.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 14, 2026
@github-actions

github-actions Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ✅ PASS

UX-level review of 9de526ddc2c32176b34f408a7dade9dabadd4ef3 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

UX-Verdict: PASS

No primary control was misread. The reader understood "Keep on", "Turn off" and the "Record metrics" switch, including that turning recording on sends data to Kiro telemetry.

Notes

  • whereBodyKiro gives the exported data a new name, "product usage metrics", but describes it with the same words as the local "performance metrics": "which features are used and how long things take". The blind reader listed "performance metrics" and "product usage metrics" among terms they "can't tell apart". Either say that the exported data is the same metrics, or name only what is different about it.
  • The phrase "labelled with a hashed form of the random install ID" lost the reader ("'Hashed' means nothing to me"). Replace it with "labelled with a scrambled code made from the install ID, not the ID itself".
  • recordMetricsEgressWaitingOwn puts a raw config key in front of the user. The reader had "no idea" what "telemetry.enabled: true" meant and said "it worries me". Point to the place where the user can act on it, for example "Your own collectors get nothing until you turn on export for them in config.json", with a link to the docs.

[UX-REVIEWED] 9de526d

Description read

[DESCRIPTION-READ] 74983a3fa491b3d40ddfa9a2807338526d9c417761b04fd3aecf31769a4cfd23

That sha256 names everything this verdict read: the PR description, plus the 7 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ human override accepted

@chenmingwei23 overrode this lane for 9de526ddc2c32176b34f408a7dade9dabadd4ef3 via /ai-review override (this commit only).

@github-actions

github-actions Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 9de526ddc2c32176b34f408a7dade9dabadd4ef3 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the owner made the default-export decision, the notice gate closes on failure, LTS, CI and dev-home builds return no destination, and the rendered evidence is attached.

Notes

  • Once a person keeps recording on after the v3 notice, every public install sends a stable x-kiro-machineid (a SHA-256 of its install id). Turning it off later stops new data but cannot recall what was sent. The Privacy chapter's whereBodyKiro line should say plainly that a per-install pseudonymous id is sent, not only "no credential".
  • exported() stores _last_build_kept in a module global that live_recorder_state reads. If two recorder builds run in one process (for example, a consent recheck racing a rebuild), the disclosure can report the earlier build's state. Returning it alongside the kept tuple would remove the shared state.

[DESIGN-REVIEWED] 9de526d

Description read

[DESCRIPTION-READ] 1c2ce2d0ea90bc5c9d7809a92f89df3be0c9e0ebc117c81892e4380c67631123

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 9de526ddc2c32176b34f408a7dade9dabadd4ef3 — this comment is updated in place on each push.

Review details

No findings.

Notes:

  • kirocrew telemetry status (src/kiro_crew/cli_commands.py:852) is misleading on a host whose only destination is Kiro telemetry and where the notice hasn't been shown yet. It says metrics stay on the machine "until telemetry.enabled: true is written", but export to Kiro actually starts once that notice is shown, and writing the flag does not start it.

[OPUS-REVIEWED] 9de526d

Verdict parsed from the review's SHA-scoped output markers for commit 9de526ddc2c32176b34f408a7dade9dabadd4ef3.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 9de526ddc2c32176b34f408a7dade9dabadd4ef3 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 9de526ddc2c32176b34f408a7dade9dabadd4ef3 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 9de526d

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 9de526ddc2c32176b34f408a7dade9dabadd4ef3 false-positive: <reason, 20 to 500 characters>

@KiroCrewCaptain

KiroCrewCaptain commented Nov 14, 2026 •

Copy link
Copy Markdown
Collaborator

Code change approver recommendation: A person should decide

Why: the decider raised 2 possible problems and estimates a 62% chance that this change introduces a defect; a person should check them.

I'm requesting changes. The PR makes the public build send product metrics and a hashed install id to a Kiro endpoint whenever recording is on, which it is by default, after a notice instead of a written opt-in. That reverses an explicit pin in the telemetry RFC: "the on default alone never exports, and neither Keep on nor the CLI notice counts as the opt-in". The only authority for the reversal is an "owner decision" claimed in the description and in the PR's own edit to a draft RFC. There is no accepted RFC on the base, no docs/decisions entry and no maintainer override, which the repository's process requires for this kind of default and trust-boundary change. The author also says a real accepted export to the endpoint should be shown before merge, and it hasn't been. The backend, frontend and E2E suites were still pending at decision time. The implementation itself looks careful: notice gating fails closed, and LTS builds, CI runners and non-default homes are excluded. Once a maintainer records the decision and real-endpoint export is shown, it may be mergeable.

Things to check (possible problems the decider raised; at this confidence many turn out to be fine):

  • This moves the privacy and trust boundary: product metrics plus a stable per-install identifier now leave the machine by default, gated only by a notice instead of a written opt-in. It reverses an explicit pin in the spec it touches. The only authority for the reversal is a claim in the PR descripti (docs/request-for-change/rfc-crew-log-telemetry-export.md, high)
  • The stated goal is that the public build actually sends product metrics to Kiro telemetry. The author says a real accepted export must be shown before merge, and it has not been. The only wire test uses a loopback receiver, so nothing yet shows the endpoint accepts this payload and these headers. By (src/kiro_crew/metrics/kiro_export.py, medium)

An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit 9de526ddc2 · 1791 changed lines. If this recommendation is wrong, reply here: replies are how the approver is evaluated.

@chenmingwei23
chenmingwei23 force-pushed the feat/kiro-telemetry-default branch 3 times, most recently from 3dc4fdd to 7e1d7dd Compare November 14, 2026 12:42
@chenmingwei23 chenmingwei23 changed the title feat(metrics): ship the Kiro telemetry destination in the public build feat(metrics): send product metrics to Kiro telemetry from the public build Nov 14, 2026
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 14, 2026
@chenmingwei23
chenmingwei23 force-pushed the feat/kiro-telemetry-default branch from 7e1d7dd to 7c1a451 Compare November 14, 2026 13:18
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Nov 14, 2026
@chenmingwei23
chenmingwei23 force-pushed the feat/kiro-telemetry-default branch 2 times, most recently from b41caf3 to 1294923 Compare November 14, 2026 14:54
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Nov 14, 2026
@chenmingwei23
chenmingwei23 force-pushed the feat/kiro-telemetry-default branch 3 times, most recently from 341f436 to decb1de Compare November 14, 2026 17:01
… build

The public telemetry provider now returns one built-in destination,
kiro-telemetry, after the operator's own entries. It sends the reviewed
PRODUCT_METRICS list to the Kiro telemetry endpoint with ClientType=kiro-crew
on every point, a reduced resource (no install id, pid or host.* keys), delta
temporality, no OS type, and none of the process-wide OTel header, credential or
certificate variables. Its session adds only x-kiro-machineid (a SHA-256 of
the existing install id, never created here); no sign-in token or other
credential is sent. It sends to no other URL and follows no redirect.

It is the one destination that exports under the recording default (owner
decision 2026-11-14): while local recording is on, by default or by choice,
once the privacy notice that names it was shown. The notice is version 3:
PRIVACY_NOTICE_VERSION goes from 2 to 3 in beacon.py and privacyNotice.ts, so
the first-run Privacy chapter shows once more and gains a line naming Kiro
telemetry, and the CLI one-line notice names it and stamps the version it
printed. Keep on or `kirocrew telemetry enable` (both record
a new dashboard.kiro_telemetry_ack_version, written only when the text shown
named Kiro telemetry, and by the CLI only after its line reached a terminal;
it is read-only in the all-settings page like privacy_ack_version) count, and so does the Kiro line printed on a terminal by
a start with no dashboard (gateway --no-dashboard, run, chat), which writes its
own stamp. A line that went to a log or a pipe, privacy_ack_version alone, and
a telemetry.enabled: true written under an older notice do not count. The shown
state is part of the rebuild fingerprint, read from the same config load the
build's export decision used, so a Keep on attaches the reader at the next
consent recheck.

Operator entries still need the written opt-in (outbound_consented). Every
off switch still stops it: heartbeat-off upgrades, a written false or non-bool,
DO_NOT_TRACK, KIROCREW_TELEMETRY=0, telemetry disable. A policy otlp_hosts
list can refuse its host. It is never returned in an LTS build, with or
without a policy collector, on a CI runner, or under a non-default
KIROCREW_HOME.

Because the notice names it, it no longer makes the Settings switch or
`kirocrew telemetry enable` refuse to turn recording on: both ask
kiro_export.egress_needs_written_opt_in, which counts only the other
destinations, and the collection status reports that as enable_refused.

The collection status keeps export_permitted for the operator opt-in, and a
recorder kept running while consent cannot be read still reports its Kiro
export. The Privacy chapter says metrics are recorded here and names Kiro
telemetry once, on its own line; Settings names the Kiro export apart from a
collector of the user's own that still waits for the opt-in (which it calls
"your own collectors"), and so does
`kirocrew telemetry enable`. The Kiro stamp is written only when the line
reached the terminal, and a headless start prints it even after a dashboard
start stamped the generic line or a redirected `telemetry enable` wrote the
generic acknowledgement; once printed to a log it is repeated only to a
terminal. While the Kiro line is still owed, the theme boot payload holds the
reported notice version back so the dashboard chapter shows once more. The Privacy chapter's Turn off line names the
Kiro export, and the Telemetry panel footer no longer says "no egress".

The OTLP/HTTP exporter becomes a base dependency (same pin as the otlp extra,
which stays). Docs: the export guide, metrics.md, README privacy text and the
crew-log telemetry RFC's export rule. The test suite turns the destination off
in test/conftest.py so no test reaches the network.
@chenmingwei23
chenmingwei23 force-pushed the feat/kiro-telemetry-default branch from decb1de to 9de526d Compare November 14, 2026 17:35
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 14, 2026
@chenmingwei23

Copy link
Copy Markdown
Contributor Author

Product decision (owner) for head 9de526d:

Public non-LTS builds export the reviewed PRODUCT_METRICS set to the built-in Kiro telemetry destination by default, but only after the user has been shown the notice that names it (the Privacy chapter with Keep on / Turn off, or the one-line CLI notice printed to a terminal). LTS builds never add this destination. Heartbeat off, a written telemetry.enabled: false or any non-bool value, DO_NOT_TRACK, KIROCREW_TELEMETRY=0, kirocrew telemetry disable, CI and a non-default KIROCREW_HOME stay off. Collectors a user adds still need a written opt-in.

The RFC text in this PR is updated to match; a maintainer review is still requested before merge.

@chenmingwei23

Copy link
Copy Markdown
Contributor Author

/ai-review override first-principles 9de526d owner-decision: owner decided default export of PRODUCT_METRICS to the built-in Kiro destination after the naming notice, LTS never, all opt-outs kept: #37183 (comment)

@github-actions

Copy link
Copy Markdown
Contributor

Human judgment recorded

@chenmingwei23 recorded a owner-decision release (class owner-decision) for the first-principles AI finding on 9de526ddc2c32176b34f408a7dade9dabadd4ef3.

owner decided default export of PRODUCT_METRICS to the built-in Kiro destination after the naming notice, LTS never, all opt-outs kept: #37183 (comment)

This decision applies only to this commit. A new push requires a new judgment.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention labels Nov 15, 2026
@KiroCrewCaptain

Copy link
Copy Markdown
Collaborator

This adds a new concept, so it needs a design sign-off from a maintainer before it can merge. A maintainer will post the decision here. To propose a design, follow the request-for-change process in CONTRIBUTING.md (docs/request-for-change/).

The First Principles finding was overridden by the PR author. That finding says the export consent gate now lets product metrics and a hashed install id leave a default public install, which main refuses. So a maintainer who is not the author has to decide this. Linked issue #21270 also asks for a maintainer sign-off on data scope before its pull requests merge. Neither this PR nor that issue has one yet.

What the review found on this head:

  • The gating fails closed on every path we traced. is_builtin matches the whole descriptor, so an operator collector named kiro-telemetry is not treated as the built-in one. The destination sends only PRODUCT_METRICS, strips host and process attributes, and refuses any other URL and any redirect. The loopback wire test runs through the real provider and exporter.
  • Two things to fix before merge, whatever the decision is:
    1. docs/system-specs/modules/metrics.md lines 13-14, 176-183, 188-195 and 2775-2780 still say the default never exports and that the 409 refuses the build's own destination. RFC section 1 item 3 still says there is no network egress. The code at this head does the opposite. Please change these paragraphs so they match the new kiro-telemetry paragraph, and keep the rule for operator collectors as it is.
    2. The body's Manual verification says a real accepted export to the endpoint should be shown before merge. Please attach one export from a default-home public build to the real endpoint, with the response status and the head SHA it ran on.

Optional:

  • cli_commands.py:852: before the notice is shown, kirocrew telemetry status says export waits for telemetry.enabled: true. For the Kiro destination it actually waits for the notice. A separate line for that case would be clearer.
  • kiro_export.py:275-277: no test pins the verify, cert and allow_redirects overrides. Only these lines stop an operator's OTel certificate settings from reaching the Kiro endpoint, so one unit test would protect them.
  • docs(rfc): crew usage metrics, wave 2 #22180 (same author) still says there is no default endpoint. Please make the two match.

@chenmingwei23
chenmingwei23 enabled auto-merge (squash) November 15, 2026 02:00

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Direction signed by maintainer (bolichen97): default on, per owner decision 11-15.

I read the whole diff (code, tests, frontend, all 13 locales). It does what the checklist for default-on export asks.

Export conditions

  • Export runs only when local recording is on and privacy notice v3 was seen. notice_shown() reads dashboard.kiro_telemetry_ack_version >= 3 or the notice stamp, and fails closed when neither is there.
  • destinations() returns nothing for LTS, CI, a non-default KIROCREW_HOME, and the test suite (autouse fixture in conftest.py).
  • Operator destinations still need outbound_consented. The enable refusals ignore only the built-in destination.

Off switches

  • _build_recorder returns a null recorder when consent is false, before any destination is built. So telemetry.enabled false or non-bool, KIROCREW_TELEMETRY=0, DO_NOT_TRACK, kirocrew telemetry disable, and an upgrade with the heartbeat off all stay off. Tests pin each one.

What leaves the machine

  • The endpoint is the constant https://telemetry.us-east-1.kiro.dev/v1/metrics. The session sends only to that URL, with redirects off and TLS verification on. Any other URL gets a local fake 401 and nothing is sent.
  • Only the metrics in PRODUCT_METRICS (counters and gauges with closed enum attributes) go out, plus ClientType and six resource keys: service name and version, runtime name and version, harness, process role. No install id, pid, host or OS fields.
  • The only header added is x-kiro-machineid, a SHA-256 of a versioned prefix plus the install id. install_id(create=False) never creates an id. No credentials, paths or prompt text are sent, and process_env_credentials=False.

Notice

  • PRIVACY_NOTICE_VERSION goes 2 to 3 in beacon.py and privacyNotice.ts. The CLI and headless line stamps the notice only when it printed to a terminal. The dashboard writes the ack only when the chapter showed the Kiro line, and shows the chapter again while the Kiro notice is owed.

Other

  • Export failures do not block startup.
  • No regex in product code. The new dependency opentelemetry-exporter-otlp-proto-http==1.44.0 is pinned in install_requires.

Follow-ups, not blocking

  • The PR says it was not run against the real endpoint. A real accepted export should be shown before or soon after merge.
  • The wire test checks metric names, resource keys, the header and ClientType. It does not check point attribute keys or values. That rests on the reviewed PRODUCT_METRICS definitions on main and test_product_metrics.py.

I did not run the tests locally.

@chenmingwei23
chenmingwei23 merged commit c1222b9 into main Nov 15, 2026
106 of 107 checks passed
@chenmingwei23
chenmingwei23 deleted the feat/kiro-telemetry-default branch November 15, 2026 02:08
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 15, 2026
chenmingwei23 added a commit that referenced this pull request Nov 15, 2026
… to Kiro telemetry

The help text for telemetry.enabled said default recording had no network
egress, and the beacon_enabled help said recording egresses only to a
configured destination after an explicit opt-in. With the Kiro telemetry
destination from #37183, a default public install sends the reviewed product
metrics to Kiro telemetry once the notice that names it has been shown. Say
so in both help strings and in the TelemetryConfig docstring, and regenerate
config-baseline.json.
chenmingwei23 added a commit that referenced this pull request Nov 15, 2026
…y destination (#22397, #37183)

Backport to release/0.13.0 of three main commits, picked with
git cherry-pick -x in main's merge order and squashed into one commit:

- #22397 (e58c25a): record local usage
  metrics by default, with disk caps and a CLI notice. Existing opt-outs
  are kept: heartbeat off with telemetry.enabled unset, telemetry.enabled
  false (quoted and non-bool values included), KIROCREW_TELEMETRY=0,
  DO_NOT_TRACK=1.
- #33422 (ffd2339): RFC update,
  telemetry.enabled now defaults to true. Docs only.
- #37183 (c1222b9): the public build sends
  the reviewed PRODUCT_METRICS list to Kiro telemetry once privacy notice
  v3 has been shown; notice v3 names it; the OTLP exporter becomes a base
  dependency.

Hand-resolved for 0.13: in docs/guides/telemetry-otlp-export.md the intro,
the telemetry.enabled row and the collection/egress paragraph take the
pick's text, and 0.13's own text is kept elsewhere. The pick's one-line
additions to config_all_settings.py and its test are left out: the All
settings page is not on 0.13.

Also fixes the settings help for telemetry.enabled and beacon_enabled and
the TelemetryConfig docstring, which said default recording sends nothing
off the machine, and regenerates config-baseline.json.

Co-authored-by: Raymond Chen <chenmingwei23@users.noreply.github.com>
(cherry picked from commit e58c25a)
(cherry picked from commit ffd2339)
(cherry picked from commit c1222b9)
chenmingwei23 added a commit that referenced this pull request Nov 15, 2026
…y destination (#22397, #37183) (#37471)

Backport to release/0.13.0 of three main commits, picked with
git cherry-pick -x in main's merge order and squashed into one commit:

- #22397 (e58c25a): record local usage
  metrics by default, with disk caps and a CLI notice. Existing opt-outs
  are kept: heartbeat off with telemetry.enabled unset, telemetry.enabled
  false (quoted and non-bool values included), KIROCREW_TELEMETRY=0,
  DO_NOT_TRACK=1.
- #33422 (ffd2339): RFC update,
  telemetry.enabled now defaults to true. Docs only.
- #37183 (c1222b9): the public build sends
  the reviewed PRODUCT_METRICS list to Kiro telemetry once privacy notice
  v3 has been shown; notice v3 names it; the OTLP exporter becomes a base
  dependency.

Hand-resolved for 0.13: in docs/guides/telemetry-otlp-export.md the intro,
the telemetry.enabled row and the collection/egress paragraph take the
pick's text, and 0.13's own text is kept elsewhere. The pick's one-line
additions to config_all_settings.py and its test are left out: the All
settings page is not on 0.13.

Also fixes the settings help for telemetry.enabled and beacon_enabled and
the TelemetryConfig docstring, which said default recording sends nothing
off the machine, and regenerates config-baseline.json.


(cherry picked from commit e58c25a)
(cherry picked from commit ffd2339)
(cherry picked from commit c1222b9)

Co-authored-by: Raymond Chen <chenmingwei23@users.noreply.github.com>
chenmingwei23 added a commit that referenced this pull request Nov 16, 2026
…f-form offer (#37460, #37463, #37545)

Backport to release/0.13.0 of three main commits, picked with
git cherry-pick -x in main's merge order and squashed into one commit.
They follow up #37183, which #37471 already put on 0.13:

- #37460 (2a679d6): the built-in
  kiro-telemetry destination is also returned under a non-default
  KIROCREW_HOME. kiro_export.destinations() drops only the
  is_default_home() check; the LTS, CI and test-suite gates and every
  consent gate stay. The heartbeat keeps its own default-home rule.
- #37463 (8de422d): installs that are off
  only because an older build wrote telemetry.enabled: false into their
  config are shown the Privacy chapter once, in an off form that names
  Kiro telemetry, with Turn on performance metrics / Keep off.
  kiro_export.off_offer_due decides that case; Keep off records only the
  notice version so it is not asked again.
- #37545 (09b0ddc): test only. The
  TestOffOffer tests fake the default data home, since the offer is asked
  only there and the test home is a private KIROCREW_HOME.

All three picks applied without conflicts; each backported patch is
identical to its main commit (git range-diff shows no change).

(cherry picked from commit 2a679d6)
(cherry picked from commit 8de422d)
(cherry picked from commit 09b0ddc)

Co-authored-by: Raymond Chen <chenmingwei23@users.noreply.github.com>
chenmingwei23 added a commit that referenced this pull request Nov 16, 2026
…f-form offer (#37460, #37463, #37545)

Backport to release/0.13.0 of three main commits, picked with
git cherry-pick -x in main's merge order and squashed into one commit.
They follow up #37183, which #37471 already put on 0.13:

- #37460 (2a679d6): the built-in
  kiro-telemetry destination is also returned under a non-default
  KIROCREW_HOME. kiro_export.destinations() drops only the
  is_default_home() check; the LTS, CI and test-suite gates and every
  consent gate stay. The heartbeat keeps its own default-home rule.
- #37463 (8de422d): installs that are off
  only because an older build wrote telemetry.enabled: false into their
  config are shown the Privacy chapter once, in an off form that names
  Kiro telemetry, with Turn on performance metrics / Keep off.
  kiro_export.off_offer_due decides that case; Keep off records only the
  notice version so it is not asked again.
- #37545 (09b0ddc): test only. The
  TestOffOffer tests fake the default data home, since the offer is asked
  only there and the test home is a private KIROCREW_HOME.

All three picks applied without conflicts; each backported patch is
identical to its main commit (git range-diff shows no change).

(cherry picked from commit 2a679d6)
(cherry picked from commit 8de422d)
(cherry picked from commit 09b0ddc)

Co-authored-by: Raymond Chen <chenmingwei23@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants