Skip to content

feat(metrics): count kirocrew.app.session_started per app and version - #39777

Merged
KiroCrewCaptain merged 1 commit into
mainfrom
feat/app-session-started-metric
Nov 20, 2026
Merged

KiroCrewCaptain merged 1 commit into
mainfrom
feat/app-session-started-metric

Conversation

@chenmingwei23

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: An app author can see how many sessions run under their app, per app version and per kind of session.

Today Crew counts every session start (kirocrew.session.started) but cannot tell which app a session ran under. The app usage metrics RFC (closed doc PR #39107, section 11 PR 4) adds one counter for that.

Why it matters

Apps built on Crew have real users, and their authors cannot see whether anyone runs them. Sessions under an app are one of the wave 1 numbers the RFC promises (section 6.2), and the daily and weekly active machines in section 6.3 are derived from it.

Not a goal

  • kirocrew.app.run_time, kirocrew.app.present, kirocrew.app.opened and kirocrew.app.foreground_time (RFC PRs 5-7, separate PRs).
  • Labelling Kiro Space installs as space:<namespace>/<name> (RFC PR 8).
  • The downstream collector's key list: it must keep entity_version and kind for this row to arrive whole in the analytics store. That list lives outside this repository and is the collector owner's privacy review.

What changed (motivation -> approach -> change)

One new counter, kirocrew.app.session_started, on scope kiro.crew with the shape Feature = app, Activity = session_started, Entity = the app's public id.

Attribute Value
Entity app_public_id(owning_app): a built-in or store app's name, else private
entity_version app_public_version(owning_app): N.N.N, else other; private for a private app
kind the session's session_kind, the same 9 values kirocrew.session.started carries

It is counted in record_session_started (src/kiro_crew/metrics/sessions.py), right after kirocrew.session.started, when the provider's owning_app is set. Session allocation sets that field from the caller's slot app, never from transcript metadata. Putting it in the one shared recorder means every start path counts it the same way, and it can never count more than session.started. A session with no owning_app counts nothing. The recorder (record_app_session_started in src/kiro_crew/metrics/feature_usage.py) swallows its own failures, and a second guard in sessions.py keeps a raising recorder from touching the session start.

The four places every new name changes together are updated: _SHAPES in metrics/behavioral_metrics.py, PRODUCT_METRICS in metrics/product_metrics.py, the regenerated metrics/schema_manifest.json (+1 row), and the export table in docs/guides/telemetry-otlp-export.md. docs/system-specs/modules/metrics.md gains the row.

Backwards compatibility

Compatible: one new counter row; kirocrew.session.started keeps its attributes and is still counted first.

Removes nothing: the diff only adds a metric row, a recorder and docs.

Tests

  • test/metrics/test_session_started_attrs.py: a provider with owning_app counts one app.session_started with the right Entity, entity_version and kind (store app, pre-release version, private app, cron and sub-agent kinds); no owning_app (None, empty, not a string) counts none; a raising app recorder leaves session.started counted and the start recorded.
  • test/metrics/test_behavioral_metrics.py: the new row gets its Feature / Activity / Entity shape.
  • Schema gates: test/metrics/test_metrics_schema.py passes, and scripts/check_metrics_schema_history.py reports the schema only grows (+1 row).

Manual verification

N/A -- unit coverage sufficient: the change is one in-memory counter add behind the existing recorder, and the tests drive the real record_session_started.

Real-machine verification

N/A -- a metric row and its recorder; no user-visible behaviour changes on a machine.

Related Issues

Fixes #39774

Implements RFC section 11 PR 4 from closed #39107 (the app usage metrics RFC doc PR; the RFC is not on main).

Checklist

  • A Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...); PR Hygiene does not count commits
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Implements RFC section 11 PR 4 (app usage metrics, closed #39107): right
after record_session_started counts kirocrew.session.started, a session
whose provider carries a trusted owning_app also counts
kirocrew.app.session_started with Entity = app_public_id, entity_version =
app_public_version and kind = the session_kind (9 values). No owning_app,
no count. A raising app recorder leaves session.started counted and the
session start unchanged.
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 20, 2026
@github-actions

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 93e9b6f5b28c1042e7390efe07c3484b89e00752 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 93e9b6f5b28c1042e7390efe07c3484b89e00752 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the change only adds one metric row. It reads the trusted owning_app, which allocation sets before the start is recorded, and it can never count more than session.started.

Notes

  • The compaction recycle and background starts also call record_session_started, so they count an app session too. That matches session.started, but the dashboards built on this counter should state that it counts session starts, not distinct user sessions.

[DESIGN-REVIEWED] 93e9b6f

Description read

[DESCRIPTION-READ] 15ee5fca406e8224333809574e8062785466bb51527dec0cadfc11f8b7dc262b

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 93e9b6f5b28c1042e7390efe07c3484b89e00752 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks. This adds one counter for a harm the author names, linked to issue #39774. A grep found no existing metric that records which app a session ran under; session.started only records the trigger app.

What this change ships

Inventory (4 items) — 4 justified

Intent: an app author can see how many sessions run under their app, per version and per kind of session. This is an ADDITION.

  1. A new counter kirocrew.app.session_started (Entity, entity_version), counted beside session.started when a session is app-owned — justified
  2. A kind attribute (the 9 session_kind values) on that counter — justified
  3. A row for the counter in the behavioral and product metric registries and in the schema manifest (+1 row) — justified
  4. A row each in the export guide and the metrics spec — justified

[FIRST-PRINCIPLES-REVIEWED] 93e9b6f

Description read

[DESCRIPTION-READ] de3d3f9b68c08894c84f922eb4cb8a78d681e396332273f6dd57e1c49bf92651

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 93e9b6f5b28c1042e7390efe07c3484b89e00752 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 93e9b6f

Verdict parsed from the review's SHA-scoped output markers for commit 93e9b6f5b28c1042e7390efe07c3484b89e00752.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 93e9b6f5b28c1042e7390efe07c3484b89e00752 false-positive: <reason, 20 to 500 characters>

@github-actions

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 93e9b6f5b28c1042e7390efe07c3484b89e00752 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 93e9b6f

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 93e9b6f5b28c1042e7390efe07c3484b89e00752 false-positive: <reason, 20 to 500 characters>

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 20, 2026
@KiroCrewCaptain
KiroCrewCaptain enabled auto-merge (squash) November 20, 2026 06:01

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This review checked head 93e9b6f. provider.owning_app is set before every record_session_started caller that can carry an app. A private app is exported only as private/private through app_public_id/app_public_version, and kind is limited to the 9 SESSION_KIND_VALUES. The two touched test files pass (44 tests), and reverting the production code turns the new tests red. One thing done well: the counter runs inside the one shared start recorder, only after kirocrew.session.started is emitted, so every start path counts it the same way.

Security: the gate matched schema_manifest.json only because its name contains "manifest". The file grants no scopes, and this PR adds one telemetry row to it. The change touches no auth, sandbox, path or subprocess code, and no private app name or version leaves the machine.

Optional:

  • Add a test that makes the session.started emit raise and asserts no kirocrew.app.session_started is recorded. Today, deleting the return in sessions.py:373 turns no test red.
  • In the metrics.md row, say that the counter counts session starts, including compaction and background recycles, not distinct sessions.
  • After a failed compaction recycle, the kept session is counted again, the same way session.started already counts it on main.

@KiroCrewCaptain
KiroCrewCaptain merged commit 096f220 into main Nov 20, 2026
98 checks passed
@KiroCrewCaptain
KiroCrewCaptain deleted the feat/app-session-started-metric branch November 20, 2026 06:01
@KiroCrewCaptain

Copy link
Copy Markdown
Collaborator

Code change approver recommendation: A person should decide

Why: the decider estimates a 10% chance that this change introduces a defect, within the 12% limit for approving on its own, but it is held at tier 2: security-sensitive path.

The PR adds a new counter, kirocrew.app.session_started. record_session_started emits it, but only after kirocrew.session.started has been counted, and only when the provider's owning_app is a non-empty string. Session allocation sets that field from the caller's slot app on both paths: a warm-pool claim sets it at get_or_create.py:1104, and a cold start passes it to the factory. Both happen before record_session_started runs at get_or_create.py:1733.

The new count is a synchronous in-memory add that comes after the function's only cancellable point, so a start that gets cancelled and rolled back never counts an app session. The early return added after the session.started except block is harmless, since nothing followed it on main.

Labels reuse the existing app_public_id and app_public_version helpers: a private app shows as private, and a version that isn't N.N.N shows as other. kind is checked against the existing 9 SESSION_KIND_VALUES.

Every place that lists the app lifecycle metrics is updated: _SHAPES, PRODUCT_METRICS, the schema manifest (+1 row, and the 'Metric Schema Only Grows' check passed), the export guide and the system spec. The new t

Tier: 2, held for a person: security-sensitive path.

An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit 93e9b6f5b2 · 152 changed lines. If this recommendation is wrong, reply here: replies are how the approver is evaluated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

metrics: count kirocrew.app.session_started per app and version (app usage RFC PR 4)

2 participants