Repository navigation
fix(metrics): put the hashed install id on Kiro telemetry data points so they are not dropped - #37969
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: it reuses the existing hash on the existing endpoint. The only new piece is a stored per-point copy of an identity the privacy notice already says is sent. Notes
[DESIGN-REVIEWED] a90d0ce Description read[DESCRIPTION-READ] 73583784090e012e7856922f7b26b5a34277a1807d51fd049d366280b6e82149 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks. The fix is at mechanism level, and the receiving side, which is the real cause, is declared out of scope. The one rider is that operators can now set Not checked: whether the receiving side really reads Notes
What this change shipsInventory (4 items) — 3 justifiedIntent: get the product metrics from public-build installs into product analytics instead of having them dropped. FIX.
[FIRST-PRINCIPLES-REVIEWED] a90d0ce Description read[DESCRIPTION-READ] 3653f6eb863e61b657e1ed115c3b5a015ab6e694e5aa33fbc50e96d8e37e07fe That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
… so they are not dropped The built-in kiro-telemetry destination sent its identity only in the x-kiro-machineid request header. The endpoint accepts such a request but does not copy the header onto the points it stores, so every stored point had no identity and was dropped before product analytics. Each data point now carries machineId, the same hash as the header, when an install id exists. ALLOWED_POINT_ATTRIBUTE_KEYS admits machineId. is_builtin() compares descriptors without machineId, so one built before the install id existed still matches one built after. The recorder build mints the install id before it resolves destinations, so a fresh install's first export already carries it.
357a0d7 to
a90d0ce
Compare
Code change approver recommendation: ApproveWhy: the decider estimates a 12% chance that this change introduces a defect, within the 12% limit for approving on its own, and every check passes. Approve. The PR adds An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit |
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Approving. On head a90d0ce we checked four things. The built-in destination adds machineId only when an install id already exists, using the same machine_id_hash() as the x-kiro-machineid header, and it never mints an id. is_builtin strips machineId on both sides, so every caller still agrees before and after the id appears. The moved mint in _build_recorder still runs after the consent and OTel returns. Each code hunk has a test that fails when that hunk is reverted. The Proof head 357a0d7 is a patch-identical rebase of this head, and PR Readiness is green here.
Well done: the new precondition line assert beacon.install_id(create=False) == "" turns the existing loopback test into a guard for the ordering fix. With that line, every received point must carry the header hash from the very first export.
Optional:
- The point attribute is fixed when the recorder is built, but the header is read on every request. If the install id is minted again while the process runs, the two values can differ until the next rebuild. Adding
machine_id_hash()to_endpoints_keywould close that gap if it ever matters. - The Proof line still names head 357a0d7. You can refresh it the next time you edit the body.
…try data points so they are not dropped (#37969) (#37988) The built-in kiro-telemetry destination sent its identity only in the x-kiro-machineid request header. The endpoint accepts such a request but does not copy the header onto the points it stores, so every stored point had no identity and was dropped before product analytics. Each data point now carries machineId, the same hash as the header, when an install id exists. ALLOWED_POINT_ATTRIBUTE_KEYS admits machineId. is_builtin() compares descriptors without machineId, so one built before the install id existed still matches one built after. The recorder build mints the install id before it resolves destinations, so a fresh install's first export already carries it. (cherry picked from commit a90d0ce) Co-authored-by: Raymond Chen <chenmingwei23@users.noreply.github.com>
Problem / Motivation
Goal: Product usage metrics that the public build sends to Kiro telemetry reach product analytics instead of being dropped.
The built-in
kiro-telemetrydestination identifies an install only with thex-kiro-machineidrequest header, and sends no sign-in token. The endpoint accepts such a request (200 Accepted), but it does not copy that header onto the points it stores. So each stored point has no identity, and the analytics forwarder drops every point with no identity.Evidence from 2026-11-16: 66
kiro.crewrecords from public-build installs reached the analytics stage that day, and 0 of them carried any identity. A probe export from a fresh home got200 Accepted; its stored record had no identity either. Anonymous records from the Kiro IDE do get through, because the IDE putsmachineIdon each data point itself.Why it matters
Every public-build install that exports product metrics is invisible in product analytics today. The data leaves the machine, under the privacy notice, and is then thrown away, so the export costs the user bandwidth and gives the product team nothing.
Not a goal
What changed (motivation → approach → change)
Points are dropped because the identity sits only in a header the endpoint does not store. So each data point now carries the same hash as an attribute, under the key the receiving side already reads.
metrics/kiro_export.pydestinations()point_attributesaddsmachineId=machine_id_hash(), the exact value of the header, when an install id exists. With no install id nothing is sent anyway, so the key is left out.metrics/destination_filter.pyALLOWED_POINT_ATTRIBUTE_KEYSadmitsmachineId. The allowlist is shared, so an operator'sotlp_endpointsentry may name that key too.metrics/kiro_export.pyis_builtin()machineId. A descriptor built before the install id existed and one built after are the same destination; the shared session still tells an operator entry of the same name apart.metrics/provider.py_build_recordermachineIdon its points. The mint still runs only with consent.The raw install id still never leaves the machine; only its SHA-256 under a fixed prefix does, which the privacy notice already describes as "a hashed install ID". The notice text and the Privacy screen still match, so they are unchanged. The two docs that said
ClientTypewas the only point attribute namemachineIdnow.Backwards compatibility
Compatible: one key is added to an allowlist and to this destination's points; every config and descriptor that works today still works.
Removes nothing: no key, path, check or record is taken away.
Tests
test/metrics/test_destination_filter.py: the allowlist is{ClientType, machineId}and a wrapper acceptsmachineId.test/metrics/test_kiro_export_destination.py::TestIdentity::test_the_points_carry_the_header_hash: the point attribute equals the header value.test/metrics/test_kiro_export_destination.py::TestIdentity::test_it_is_still_builtin_when_the_install_id_appears_later: descriptors from before and after the install id exist are both built in; an operator entry namedkiro-telemetryandNoneare not.test/metrics/test_kiro_export_destination.py::test_the_real_provider_sends_product_metrics_to_the_endpoint: the real provider and exporter, from a home with no install id, post to a loopback receiver; every data point carriesmachineIdequal to the header.Proof: PROVEN test/metrics/test_destination_filter.py::TestRefusedPointAttributes::test_the_reviewed_key_is_accepted, test/metrics/test_kiro_export_destination.py::TestIdentity::test_the_points_carry_the_header_hash, test/metrics/test_kiro_export_destination.py::TestIdentity::test_it_is_still_builtin_when_the_install_id_appears_later, test/metrics/test_kiro_export_destination.py::test_the_real_provider_sends_product_metrics_to_the_endpoint (head 357a0d7)
Manual verification
N/A for the receiving side: whether stored points now reach product analytics can only be seen after this ships. Locally, with the
provider.pyhunk reverted, the loopback test sees 40 points with nomachineId; with it, all points carry the header's hash.Related Issues
Follows #37183, which added this destination. No separate issue.
Sibling sweep
Swept: git grep -nE "x-kiro-machineid|HEADER_MACHINE_ID|identity_headers" -- src/
Siblings: none -- kiro_export.py is the only destination that sends identity in a header; operator entries carry no built-in identity.
Pattern harvest
Not generalizable: the receiving side's choice not to store a request header is specific to this endpoint.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)