Skip to content

feat(metrics): count each installed app once per UTC day (kirocrew.app.present) - #39783

Merged
KiroCrewCaptain merged 1 commit into
mainfrom
feat/app-metrics-present-daily
Nov 20, 2026
Merged

KiroCrewCaptain merged 1 commit into
mainfrom
feat/app-metrics-present-daily

Conversation

@chenmingwei23

@chenmingwei23 chenmingwei23 commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Count how many machines have each app installed, per version, once per UTC day.

Today Crew counts installs, updates and uninstalls, but nothing says how many machines still have an app, which version they run, or whether it is switched off. This PR implements RFC section 11 PR 6 of the app usage metrics RFC (closed PR #39107; its text is the spec, the rest ships as normal PRs): a daily kirocrew.app.present row per installed app.

Why it matters

"Installed machines per version" and "version adoption" (RFC section 6.3) are read from this row. Without it an app author sees install events but not how many machines still have the app or how fast they move to a new version.

Not a goal

  • The sibling RFC rows session_started (PR 4), run_time (PR 5), opened / foreground_time (PR 7) and Space ids (PR 8).
  • The downstream collector key list. It must keep entity_version, status and backend before the analytics store keeps these keys; that is a privacy review by the collector owner, outside this repo. Until then the events arrive without those keys; the metrics store keeps every key.
  • Changing the beacon heartbeat itself or its switch telemetry.beacon_enabled (it does not carry app rows).

What changed (motivation -> approach -> change)

The row must be sent once per app per UTC day, survive restarts, and keep firing on a gateway that runs for days (the daily re-arm review point on #39107). So the task wakes on a timer and checks a stamp on disk, instead of firing once after start.

Part Change
metrics/app_present.py (new) Background task started at gateway boot. First pass 10 minutes after start; then a check at most every hour and one second past each UTC midnight. A pass reads the UTC day once and uses it for both the check and the stamp. It skips when that day is stamped or the recorder is off, else reads list_apps() and the tracked backend processes off the event loop, records one row per app, and writes the stamp. A failed listing writes no stamp and retries on the next wake.
beacon.py Stamp file app_present_last_sent beside the heartbeat's beacon_last_sent, written with the same atomic write (_write_stamp, shared with _mark_sent).
metrics/feature_usage.py record_app_present(app, status, backend, ids): Entity and entity_version as on installed; status enabled / disabled / other; backend none (no backend.entryPoint) / running / stopped / other.
slack/gateway_runtime/process_entry.py Starts the task next to the beacon thread; skipped in test_mode.
metrics/behavioral_metrics.py, metrics/product_metrics.py, metrics/schema_manifest.json One new row on scope kiro.crew, shape ("app", None, "app"); manifest regenerated with python test/metrics_schema_helpers.py --write (+1 row).
docs/guides/telemetry-otlp-export.md, docs/system-specs/modules/metrics.md Export table and metrics spec list the row and its keys.

status and backend are closed. Entity and entity_version use the exemption kirocrew.app.installed already has (RFC section 7). No request path runs the pass. Nothing is recorded, and no stamp is written, while telemetry is off.

Backwards compatibility

Compatible: one new metric row and one new state file; the schema gate allows a new row.

Removes nothing: _mark_sent keeps its stamp and its never-raise contract through the shared _write_stamp; no key, row or function is removed.

Tests

  • test/metrics/test_app_present.py: one row per app with the right version, status=disabled for a disabled app, and all three backend states (running, stopped, none); two restarts on one UTC day record one row per app (the stamp is on disk); the real run_daily loop crossing UTC midnight records one more row per app and nothing on a third wake the same day; a pass whose clock crosses UTC midnight stamps the day it recorded, so the new day is still due; the wait ends 1 s past midnight and never exceeds an hour; telemetry off records nothing and writes no stamp; a raising recorder leaves the pass complete and the stamp written; a failed listing writes no stamp and the next pass records.
  • test/metrics/test_behavioral_metrics.py: the new row gets the Feature / Activity / Entity shape.
  • Schema gates: scripts/check_metrics_schema_history.py --test (10 probes ok) and the gate against origin/main (+1 rows); test/metrics/test_metrics_schema.py, test/metrics/test_product_metrics.py, test/test_metrics_schema_history_gate.py pass.

Manual verification

N/A -- unit coverage sufficient: the tests drive the real run_daily loop and the real stamp file in a temp dir; the row is an in-memory add with no UI.

Real-machine verification

Machine: Linux cloud desktop, Amazon Linux 2023 (kernel 6.12), x86_64, 64 vCPU / 123 GB
After (head 157ee0d, rebased on main bdc6526): pytest -n 2 test/metrics/test_app_present.py test/metrics/test_behavioral_metrics.py test/metrics/test_feature_usage.py test/metrics/test_metrics_schema.py test/test_metrics_schema_history_gate.py test/test_beacon.py test/test_gateway_process_entry.py test/metrics/test_product_metrics.py test/test_flake_pattern_ratchet.py test/test_sweep_check.py test/test_review_repair_routing_skill.py test/test_packaged_docs_runtime_facts.py -> 586 passed
Cleanup: own computer

Related Issues

Closes #39772

Implements RFC section 11 PR 6 from closed #39107, and answers its review point that the task must re-arm every UTC day on a long-running gateway and send one row per app across restarts. Built on #39715 (RFC PR 3, merged), whose app_public_version() labels the row.

Checklist

  • A Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...); PR Hygiene does not count commits
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 20, 2026
@github-actions

github-actions Bot commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 157ee0d58ef7f34596c49434b6a4242267aa0aed touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 157ee0d58ef7f34596c49434b6a4242267aa0aed — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: this adds one row, takes nothing away, and the stamp on disk handles both restarts and a gateway that runs for days.

Notes

  • app_present.start() keeps _TASK but nothing cancels it when the gateway shuts down. If an in-process restart happens, a second task could run the pass. The stamp stops duplicate rows, but cancelling the task on teardown would keep the lifecycle tidy.

[DESIGN-REVIEWED] 157ee0d

Description read

[DESCRIPTION-READ] f437b0f244969b77083f2ab0ecb91e92bb017a17e0b8eeb1859e1657f93d8f17

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 157ee0d58ef7f34596c49434b6a4242267aa0aed — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks. The one note is the backend attribute, which the frozen Goal doesn't ask for.

Notes

  • Item 3 — oversized: the Goal is "Count how many machines have each app installed, per version", and ## Why it matters names only installed machines and version adoption. Neither needs backend (none / running / stopped / other); "whether it is switched off" is already covered by status. Its only support is closed PR docs(rfc): propose app usage metrics (installs, versions, sessions, run time) #39107, which is not a record on the base. Subtraction: delete APP_PRESENT_BACKEND_VALUES, _backend(), the list_app_processes() read and the backend key from record_app_present and the manifest row. Clears when: the mechanism is removed, or a human amends the Goal.

What this change ships

Inventory (8 items) — 6 justified

Intent: let an app author see how many machines still have their app installed, and on which version, day by day. This is an ADDITION.

  1. Each installed app is counted once per UTC day as kirocrew.app.present, labelled with its app id and version — justified
  2. The daily row says whether the app is enabled or disabled — justified
  3. The daily row says whether the app's backend is absent, running or stopped — oversized; not in the Goal or Why it matters
  4. A gateway background task runs its first pass 10 minutes after start, then wakes hourly and just after each UTC midnight; it is skipped in test mode — justified
  5. A new state file, app_present_last_sent, in the data home makes restarts on the same day record only once — justified
  6. The beacon stamp write is now shared, and it also swallows RuntimeError — undeclared; the wider catch is not mentioned
  7. The export guide and the metrics spec list the new row and its keys — justified
  8. The schema manifest gains one behavioral row on scope kiro.crew — justified

[FIRST-PRINCIPLES-REVIEWED] 157ee0d

Description read

[DESCRIPTION-READ] 7347679bfe72a28c36242754529ac2cdc3c2a483ba5fe2724f3684c38787bdc0

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 157ee0d58ef7f34596c49434b6a4242267aa0aed — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 157ee0d

Verdict parsed from the review's SHA-scoped output markers for commit 157ee0d58ef7f34596c49434b6a4242267aa0aed.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 157ee0d58ef7f34596c49434b6a4242267aa0aed false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 157ee0d58ef7f34596c49434b6a4242267aa0aed and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 157ee0d

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 157ee0d58ef7f34596c49434b6a4242267aa0aed false-positive: <reason, 20 to 500 characters>

@chenmingwei23
chenmingwei23 force-pushed the feat/app-metrics-present-daily branch 2 times, most recently from 9751049 to f7e5c67 Compare November 20, 2026 05:19
@chenmingwei23

Copy link
Copy Markdown
Contributor Author

Fixed in f7e5c67. record_today now reads the UTC day once through beacon.app_present_due_day() and stamps that same value with beacon.mark_app_present_recorded(day), so a pass whose clock crosses UTC midnight stamps the day it recorded and the new day stays due. Locked in by test_a_pass_that_crosses_utc_midnight_stamps_the_day_it_recorded in test/metrics/test_app_present.py.

@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Nov 20, 2026
@chenmingwei23
chenmingwei23 force-pushed the feat/app-metrics-present-daily branch from f7e5c67 to efe1056 Compare November 20, 2026 06:28
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: checking Automated validation is still running labels Nov 20, 2026
@KiroCrewCaptain

KiroCrewCaptain commented Nov 20, 2026 •

Copy link
Copy Markdown
Collaborator

Code change approver recommendation: A person should decide

Why: the decider estimates a 12% chance that this change introduces a defect, within the 12% limit for approving on its own, but it changes 404 lines, more than the 400 it may approve on its own.

Adds a daily kirocrew.app.present row for each installed app. A background task started in run_gateway, and skipped in test_mode, checks a stamp file on disk and records one row per app per UTC day.

Checked against the repo:

  • Data sources: list_apps() returns enabled and manifest on each app. list_app_processes() rows carry app_name and running.
  • Telemetry off: get_recorder().enabled is the shared recorder's switch (_meter is not None), so no row and no stamp are written while telemetry is off.
  • Privacy: app_public_id and app_public_version label non-store apps private. status and backend come from closed value sets. The attribute count (6) is far under MAX_ATTR_COUNT (32).
  • Stamp file: the new file follows the beacon's rule of not being listed for export or snapshot, so it never travels to another machine. It is written through the same atomic write, so a symlink planted at that path cannot redirect the write.
  • Midnight handling: a pass reads the day once and stamps that same value, so a pass that runs across midnight still leaves the new day due.
  • Retries: a failed listing writes no stamp and is retried on the next

Tier: 2, held for a person: security-sensitive path.

An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit 157ee0d58e · 404 changed lines. If this recommendation is wrong, reply here: replies are how the approver is evaluated.

@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Nov 20, 2026
@chenmingwei23
chenmingwei23 force-pushed the feat/app-metrics-present-daily branch from efe1056 to 165e689 Compare November 20, 2026 08:35
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: passed Eligible automated validation passed for the current revision merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: checking Automated validation is still running labels Nov 20, 2026
…p.present)

Implements RFC section 11 PR 6 (app usage metrics, closed #39107). A
gateway background task records one kirocrew.app.present row per
installed app per UTC day, labelled entity_version, status
(enabled/disabled/other) and backend (none/running/stopped/other).
The first pass waits 10 minutes; the task then re-checks hourly and
just after each UTC midnight, so a gateway that runs for days records
every day. The last recorded UTC day is stamped next to the beacon's
daily stamp, so restarts the same day record once. Nothing is recorded
and no stamp is written while the recorder is off.

Refs #39772
@chenmingwei23
chenmingwei23 force-pushed the feat/app-metrics-present-daily branch from 165e689 to 157ee0d Compare November 20, 2026 10:06
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention labels Nov 20, 2026
@KiroCrewCaptain
KiroCrewCaptain enabled auto-merge (squash) November 20, 2026 12:26

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. On head 157ee0d the review read the whole diff against linked issue #39772. It checked that record_today runs off the event loop through asyncio.to_thread, that the boot path gains only one deferred create_task, and that every app label goes through the existing app_public_id / app_public_version helpers and closed value sets. The review also ran 8 mutations against the new tests, and each one turned the test named for it red. The head merges cleanly onto current main.

Security: the only flagged path is src/kiro_crew/metrics/schema_manifest.json, which matched by name. It is the append-only metrics schema, not a scope or permission manifest. The diff touches no auth, sandbox or secret code. A private app's name and version never leave the machine. The new app_present_last_sent stamp only throttles telemetry, and it is read through the existing bounded, regular-file-only _read_state.

Done well: the pass reads the UTC day once and stamps that same day. So a pass that runs across midnight cannot mark the new day as done, and a test pins this.

  • Optional: in test/metrics/test_app_present.py:97, wrap the call as asyncio.run(asyncio.wait_for(app_present.run_daily(), timeout=10)). Then a loop that stops calling seconds_until_next_check fails by name, instead of hanging until the 120 s suite timeout.
  • Optional: in the _seams fixture in test/test_gateway_process_entry.py, stub app_present.start and assert it is called once outside test mode and never with test_mode=True. Right now no test fails if you delete the start block in process_entry.py:745-748.
  • Optional: add a Before line on main and an After line from a real gateway run (one row per installed app, stamp file written) to ## Real-machine verification.

@KiroCrewCaptain
KiroCrewCaptain merged commit 757660c into main Nov 20, 2026
156 of 160 checks passed
@KiroCrewCaptain
KiroCrewCaptain deleted the feat/app-metrics-present-daily branch November 20, 2026 12:26
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

metrics: add kirocrew.app.present, one row per installed app per UTC day

2 participants