Repository navigation
docs(rfc): propose app usage metrics (installs, versions, sessions, run time) - #39107
chenmingwei23 wants to merge 1 commit into
Conversation
1c400d0 to
d89b269
Compare
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
First Principles Review (Opus 5.5) — ⏭️ skippedRevision |
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS This adds only a draft RFC and one index row; no code or contract changes, and nothing ships until a maintainer accepts it. [DESIGN-REVIEWED] 188631e Description read[DESCRIPTION-READ] bea360d3a36f78ffd2c6551da0439eee1436197968dcc437e6ca6fd345b7d1ec That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
Code change approver recommendation: A person should decideWhy: the decider estimates a 6% chance that this change introduces a defect, within the 12% limit for approving on its own, but it changes 500 lines, more than the 400 it may approve on its own. This PR adds one draft RFC and one index row. Neither changes code, config or schema. The code claims I checked hold on the head:
Trust and consent:
None of this moves a trust Tier: 2, held for a person: design-doc-gate: D. An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit |
|
This adds a new concept, so it needs a design sign-off from a maintainer before it can merge. A maintainer will post the decision here. To propose a design, follow the request-for-change process in CONTRIBUTING.md (docs/request-for-change/). Review result: this PR changes two docs files and no code. PR Readiness passed on this head, and the review found no blocking issue. The review checked the RFC's code references against its audited commit Optional notes for the design discussion:
|
|
Review of Input before acceptance
Smaller: §6.4's fleet bound rests on "35 public apps and 20 versions each (an assumption)". Fine for a draft, but please replace it with the store's real count before acceptance, since the "Not checked" list already names it. No blocking defect in the text. A Kiro Crew team member must sign off on this design before it merges. This review does not approve it. AI-assisted, sent by Bolin Review bot |
d89b269 to
90819ff
Compare
c115aca to
9ecbe96
Compare
…un time) Add rfc-app-usage-metrics.md (status draft) and its index row. It proposes per-app, per-version installs, updates and uninstalls, plus sessions, agent run time and foreground time under each app, as counters on closed labels behind a per-row notice floor, and retires kirocrew.app.task.completed. Code references were re-read on main at 5f20c7a and cite symbols, not line numbers. Credits #18855 as prior work on installs, active installs and version history.
9ecbe96 to
188631e
Compare
|
Closing this RFC. The first batch (kirocrew.app.installed and kirocrew.app.task.completed) is already on main via #38157 and #38347, so we will ship the rest as normal PRs instead of a separate design doc. The branch stays in place for reference, and the review points raised here (Space ids, Shared Crew undercount, the daily present row, and the collector key review) will be handled in those PRs. Thanks to everyone who reviewed. |
Implements RFC section 11 PR 5 (app usage metrics, closed #39107): emit_turn_duration gains owning_app, and both turn callers pass the live provider's owning_app. Each finished turn of an app's session adds its wall seconds, rounded up, with the bucket of that turn. A turn with no duration adds nothing; a raising recorder leaves the turn metric unchanged. Closes #39773
…p.present) Implements RFC section 11 PR 6 (app usage metrics, closed #39107). A gateway background task records one kirocrew.app.present row per installed app per UTC day, labelled entity_version, status (enabled/disabled/other) and backend (none/running/stopped/other). The first pass waits 10 minutes; the task then re-checks hourly and just after each UTC midnight, so a gateway that runs for days records every day. The last recorded UTC day is stamped next to the beacon's daily stamp, so restarts the same day record once. Nothing is recorded and no stamp is written while the recorder is off. Refs #39772
Implements RFC section 11 PR 5 (app usage metrics, closed #39107): emit_turn_duration gains owning_app, and both turn callers pass the live provider's owning_app. Each finished turn of an app's session adds its wall seconds, rounded up, with the bucket of that turn. A turn with no duration adds nothing; a raising recorder leaves the turn metric unchanged. Closes #39773
…p.present) Implements RFC section 11 PR 6 (app usage metrics, closed #39107). A gateway background task records one kirocrew.app.present row per installed app per UTC day, labelled entity_version, status (enabled/disabled/other) and backend (none/running/stopped/other). The first pass waits 10 minutes; the task then re-checks hourly and just after each UTC midnight, so a gateway that runs for days records every day. The last recorded UTC day is stamped next to the beacon's daily stamp, so restarts the same day record once. Nothing is recorded and no stamp is written while the recorder is off. Refs #39772
…pp pages Implements RFC section 11 PR 7 (app usage metrics, closed #39107): a new owner-only POST /api/telemetry/app-view, copied from ui-control (require_owner_dashboard_request, 256-byte bounded JSON body, 204), and two counters on kiro.crew: kirocrew.app.opened{surface} and kirocrew.app.foreground_time{surface,duration} (unit s). The server resolves the public id and version from the app name itself; an unknown name is private. The page sends one opened per mount and one span per visible stretch (keepalive); spans under 1 s are not sent and the server cuts a span to 24 h. Both count owner views only.
Implements RFC section 11 PR 5 (app usage metrics, closed #39107): emit_turn_duration gains owning_app, and both turn callers pass the live provider's owning_app. Each finished turn of an app's session adds its wall seconds, rounded up, with the bucket of that turn. A turn with no duration adds nothing; a raising recorder leaves the turn metric unchanged. Closes #39773
…pp pages Implements RFC section 11 PR 7 (app usage metrics, closed #39107): a new owner-only POST /api/telemetry/app-view, copied from ui-control (require_owner_dashboard_request, 256-byte bounded JSON body, 204), and two counters on kiro.crew: kirocrew.app.opened{surface} and kirocrew.app.foreground_time{surface,duration} (unit s). The server resolves the public id and version from the app name itself; an unknown name is private. The page sends one opened per mount and one span per visible stretch (keepalive); spans under 1 s are not sent and the server cuts a span to 24 h. Both count owner views only.
…#39777) Implements RFC section 11 PR 4 (app usage metrics, closed #39107): right after record_session_started counts kirocrew.session.started, a session whose provider carries a trusted owning_app also counts kirocrew.app.session_started with Entity = app_public_id, entity_version = app_public_version and kind = the session_kind (9 values). No owning_app, no count. A raising app recorder leaves session.started counted and the session start unchanged. Co-authored-by: Raymond Chen <chenmingwei23@users.noreply.github.com>
Implements RFC section 11 PR 5 (app usage metrics, closed #39107): emit_turn_duration gains owning_app, and both turn callers pass the live provider's owning_app. Each finished turn of an app's session adds its wall seconds, rounded up, with the bucket of that turn. A turn with no duration adds nothing; a raising recorder leaves the turn metric unchanged. Closes #39773
…p.present) Implements RFC section 11 PR 6 (app usage metrics, closed #39107). A gateway background task records one kirocrew.app.present row per installed app per UTC day, labelled entity_version, status (enabled/disabled/other) and backend (none/running/stopped/other). The first pass waits 10 minutes; the task then re-checks hourly and just after each UTC midnight, so a gateway that runs for days records every day. The last recorded UTC day is stamped next to the beacon's daily stamp, so restarts the same day record once. Nothing is recorded and no stamp is written while the recorder is off. Refs #39772
…pp pages Implements RFC section 11 PR 7 (app usage metrics, closed #39107): a new owner-only POST /api/telemetry/app-view, copied from ui-control (require_owner_dashboard_request, 256-byte bounded JSON body, 204), and two counters on kiro.crew: kirocrew.app.opened{surface} and kirocrew.app.foreground_time{surface,duration} (unit s). The server resolves the public id and version from the app name itself; an unknown name is private. The page sends one opened per mount and one span per visible stretch (keepalive); spans under 1 s are not sent and the server cuts a span to 24 h. Both count owner views only.
…pp pages Implements RFC section 11 PR 7 (app usage metrics, closed #39107): a new owner-only POST /api/telemetry/app-view, copied from ui-control (require_owner_dashboard_request, 256-byte bounded JSON body, 204), and two counters on kiro.crew: kirocrew.app.opened{surface} and kirocrew.app.foreground_time{surface,duration} (unit s). The server resolves the public id and version from the app name itself; an unknown name is private. The page sends one opened per mount and one span per visible stretch (keepalive); spans under 1 s are not sent and the server cuts a span to 24 h. Both count owner views only.
…pp pages Implements RFC section 11 PR 7 (app usage metrics, closed #39107): a new owner-only POST /api/telemetry/app-view, copied from ui-control (require_owner_dashboard_request, 256-byte bounded JSON body, 204), and two counters on kiro.crew: kirocrew.app.opened{surface} and kirocrew.app.foreground_time{surface,duration} (unit s). The server resolves the public id and version from the app name itself; an unknown name is private. The page sends one opened per mount and one span per visible stretch (keepalive); spans under 1 s are not sent and the server cuts a span to 24 h. Both count owner views only.
…39784) Implements RFC section 11 PR 5 (app usage metrics, closed #39107): emit_turn_duration gains owning_app, and both turn callers pass the live provider's owning_app. Each finished turn of an app's session adds its wall seconds, rounded up, with the bucket of that turn. A turn with no duration adds nothing; a raising recorder leaves the turn metric unchanged. Closes #39773 Co-authored-by: Raymond Chen <chenmingwei23@users.noreply.github.com>
…p.present) Implements RFC section 11 PR 6 (app usage metrics, closed #39107). A gateway background task records one kirocrew.app.present row per installed app per UTC day, labelled entity_version, status (enabled/disabled/other) and backend (none/running/stopped/other). The first pass waits 10 minutes; the task then re-checks hourly and just after each UTC midnight, so a gateway that runs for days records every day. The last recorded UTC day is stamped next to the beacon's daily stamp, so restarts the same day record once. Nothing is recorded and no stamp is written while the recorder is off. Refs #39772
…pp pages Implements RFC section 11 PR 7 (app usage metrics, closed #39107): a new owner-only POST /api/telemetry/app-view, copied from ui-control (require_owner_dashboard_request, 256-byte bounded JSON body, 204), and two counters on kiro.crew: kirocrew.app.opened{surface} and kirocrew.app.foreground_time{surface,duration} (unit s). The server resolves the public id and version from the app name itself; an unknown name is private. The page sends one opened per mount and one span per visible stretch (keepalive); spans under 1 s are not sent and the server cuts a span to 24 h. Both count owner views only.
…pp pages Implements RFC section 11 PR 7 (app usage metrics, closed #39107): a new owner-only POST /api/telemetry/app-view, copied from ui-control (require_owner_dashboard_request, 256-byte bounded JSON body, 204), and two counters on kiro.crew: kirocrew.app.opened{surface} and kirocrew.app.foreground_time{surface,duration} (unit s). The server resolves the public id and version from the app name itself; an unknown name is private. The page sends one opened per mount and one span per visible stretch (keepalive); spans under 1 s are not sent and the server cuts a span to 24 h. Both count owner views only.
…p.present) Implements RFC section 11 PR 6 (app usage metrics, closed #39107). A gateway background task records one kirocrew.app.present row per installed app per UTC day, labelled entity_version, status (enabled/disabled/other) and backend (none/running/stopped/other). The first pass waits 10 minutes; the task then re-checks hourly and just after each UTC midnight, so a gateway that runs for days records every day. The last recorded UTC day is stamped next to the beacon's daily stamp, so restarts the same day record once. Nothing is recorded and no stamp is written while the recorder is off. Refs #39772
…pp pages Implements RFC section 11 PR 7 (app usage metrics, closed #39107): a new owner-only POST /api/telemetry/app-view, copied from ui-control (require_owner_dashboard_request, 256-byte bounded JSON body, 204), and two counters on kiro.crew: kirocrew.app.opened{surface} and kirocrew.app.foreground_time{surface,duration} (unit s). The server resolves the public id and version from the app name itself; an unknown name is private. The page sends one opened per mount and one span per visible stretch (keepalive); spans under 1 s are not sent and the server cuts a span to 24 h. Both count owner views only.
…p.present) (#39783) Implements RFC section 11 PR 6 (app usage metrics, closed #39107). A gateway background task records one kirocrew.app.present row per installed app per UTC day, labelled entity_version, status (enabled/disabled/other) and backend (none/running/stopped/other). The first pass waits 10 minutes; the task then re-checks hourly and just after each UTC midnight, so a gateway that runs for days records every day. The last recorded UTC day is stamped next to the beacon's daily stamp, so restarts the same day record once. Nothing is recorded and no stamp is written while the recorder is off. Refs #39772 Co-authored-by: Raymond Chen <chenmingwei23@users.noreply.github.com>
…pp pages Implements RFC section 11 PR 7 (app usage metrics, closed #39107): a new owner-only POST /api/telemetry/app-view, copied from ui-control (require_owner_dashboard_request, 256-byte bounded JSON body, 204), and two counters on kiro.crew: kirocrew.app.opened{surface} and kirocrew.app.foreground_time{surface,duration} (unit s). The server resolves the public id and version from the app name itself; an unknown name is private. The page sends one opened per mount and one span per visible stretch (keepalive); spans under 1 s are not sent and the server cuts a span to 24 h. Both count owner views only.
…pp pages (#39793) Implements RFC section 11 PR 7 (app usage metrics, closed #39107): a new owner-only POST /api/telemetry/app-view, copied from ui-control (require_owner_dashboard_request, 256-byte bounded JSON body, 204), and two counters on kiro.crew: kirocrew.app.opened{surface} and kirocrew.app.foreground_time{surface,duration} (unit s). The server resolves the public id and version from the app name itself; an unknown name is private. The page sends one opened per mount and one span per visible stretch (keepalive); spans under 1 s are not sent and the server cuts a span to 24 h. Both count owner views only. Co-authored-by: Raymond Chen <chenmingwei23@users.noreply.github.com>
Problem / Motivation
Goal: Record a draft design for app usage metrics, so app authors can later see how many machines install, update, remove and use each version of their app.
Today Crew sends one app metric,
kirocrew.app.installed, counted on first install only and with no version (record_app_installedinsrc/kiro_crew/metrics/feature_usage.py). An app author cannot see which versions are in use, whether people update or uninstall, or whether anyone opens the app or runs its sessions.Why it matters
Apps built on Kiro Crew now have real users, and their authors have no numbers to decide what to fix or ship next. Any new metric also changes what leaves the machine, so the labels, the consent notice and the data volume need agreeing before code lands.
Not a goal
What changed (motivation -> approach -> change)
docs/request-for-change/rfc-app-usage-metrics.md, statusdraft. It proposes, per public app and version: installs, updates, uninstalls, a dailypresentcount, opens, sessions started under the app, agent run time and foreground time. All are counters onkiro.crewwith closed labels; durations use closed buckets, never histograms. New rows export only to people who acknowledged a new privacy notice (version 5). It also proposes retiringkirocrew.app.task.completed, and lists later waves (what an app carries, custom app counts) as designed but unscheduled.docs/request-for-change/README.md.5f20c7ae3aand cite symbols, not line numbers, as the docs lint requires.Backwards compatibility
Compatible: a new document and one index row; no code, config or schema changes.
Removes nothing: the diff only adds a document and an index row.
Tests
Proof: not provable -- docs or config only
python3 scripts/docs_lint.pyon the head: "All documentation checks passed".Manual verification
N/A -- docs only. Every cited symbol was checked to exist on main at
5f20c7ae3a.Real-machine verification
N/A -- the diff is documentation and runs nothing.
Related Issues
Related: #18855 (prior draft RFC on app usage stats). No tracking issue yet; acceptance is requested from a maintainer.
Checklist
Contribution License Agreement
Contributed under the project's Apache-2.0 license.