Repository navigation
feat: send WakaTime AI-coding heartbeats for agent coding turns - #9450
Conversation
Design Review (Fable 5, fork) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS One end-of-turn heartbeat per turn undercounts duration: a long agent turn collapses to a single timestamp, so billed coding time stays near zero. Watch
Suggestions
[DESIGN-REVIEWED] 8c7d5fd |
Opus 5 Review (fork) — ✅ no blocking findingsReviewed Review detailsTwo advisory findings; nothing blocks the merge. FINDING — src/kiro_crew/wakatime/heartbeats.py:114 — FINDING — src/kiro_crew/config/sections.py:7887 — [OPUS-REVIEWED] 8c7d5fd |
First Principles Review (Fable 5, fork) — 🟡 CONCERNSPremise-level review of First-Principles-Verdict: CONCERNS
Not justified as shipped
What this change shipsInventory (10 items) — 9 justifiedIntent: make agent coding time appear in the user's WakaTime AI-coding dashboard — an ADDITION.
[FIRST-PRINCIPLES-REVIEWED] 8c7d5fd |
be3df69 to
b034735
Compare
|
FINDING (chat_runner.py end-of-turn heartbeat block) — fixed in The finding holds: the end-of-turn block ran the
|
GPT 5.6 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings. |
b034735 to
f0e6bd3
Compare
|
BLOCKING (feature-map-correctness, loader omits send_heartbeats) — fixed in Correct and the most important one: the loader constructed
|
|
BLOCKING (no-blocking-call-on-event-loop, synchronous snapshot read) — rebutted (disproportionate) at The adjudication in this same review DOWNGRADED this finding, and the code now narrows it further.
|
|
BLOCKING (residual/crash-data-loss-corruption, buffered heartbeat can remain unsent) — fixed in Holds: a final coding turn inside the flush interval left its heartbeat buffered with no timer armed, so it was lost on exit.
|
|
BLOCKING (residual/crash-data-loss-corruption, repeated edits inflate line-change telemetry) — fixed in Holds: the block iterated
|
|
BLOCKING (residual/crash-data-loss-corruption, rejected tools counted as coding activity) — fixed in Holds:
|
|
FINDING (top-level-imports, in-function heartbeat import) — rebutted (disproportionate) at The in-function import is deliberate and the top-level alternative would break a stronger constraint.
|
|
BLOCKING (MCP
|
|
FINDING (sections.py Slack-scope wording) — fixed span=1b88ed28a3ad
|
|
NOT JUSTIFIED (buffer/timer/fingerprint apparatus) — fixed by subtraction span=d8e90a425f8b
|
|
NOT JUSTIFIED (
|
|
WATCH (turn-end-only cadence undercounts long turns) — accepted-and-deferred span=f890aead5299
|
|
SUGGESTIONS (fold the duplicated count blocks) — fixed span=ac45e24f7120
|
|
F1 (BLOCKING, The pending-coding set now routes every retained id through the turn's shared bounded-admission helper Covered by F2 (BLOCKING-FENCED,
FINDING ( |
|
All three findings from the GPT review of F1 (BLOCKING-FENCED, chat_runner.py:11683 — statusless calls counted before permission) — fixed. A coding tool_call now retains the turn's shared bounded key whether or not the frame carried F2 (BLOCKING-FENCED, chat_runner.py:14548 — truncated snapshots corrupt line-change telemetry) — fixed. F3 (chat_runner.py:11677 — overflow logged per dropped call; adjudication DOWNGRADED to advisory) — fixed anyway. The per-call Verified: |
|
self-added: yes
|
|
self-added: yes
|
|
self-added: yes
|
|
mechanism: end-of-turn promotion of undecided coding keys; dead tool_pending plumbing removed
|
|
|
|
|
|
|
|
GPT F1 (security-FENCED, The finding is accurate as a description of the seam, but the property it asks for is neither new to this change nor closable here in proportion:
This is a security-class fence, so clearance is a human override recorded by a repository writer, not an agent action. Recommendation to the maintainer: clear it as a pre-existing, out-of-scope seam — the honest scope of this PR is the heartbeat producer, and converting the config/vault resolution seam to a versioned commit protocol is separate work that should stand on its own if it is wanted. |
Kiro Crew could read a user's WakaTime stats but never sent anything back, so agent coding time was invisible in their WakaTime dashboard. This adds the send side: a completed agent turn that edited files or ran a command becomes a WakaTime heartbeat. A small in-process producer hooks at end-of-turn, where the usage telemetry already fires. A turn counts as coding activity only when it dispatched a filesystem-write or shell tool. Each qualifying turn enqueues one heartbeat on a bounded buffer, and a time-gated flush drains it, chunks to the API's 25-per-request cap, and sends. The send is fire-and-forget: it never awaits on the turn's hot path, and a failure drops the batch rather than retrying, since WakaTime dedupes by two-minute buckets. The heartbeat uses WakaTime's ai coding category and carries the turn's real signals: input and output token counts from the turn's usage, a total line-change count from the file-change snapshots the turn already keeps, and the project directory basename as the entity. No session identifier is sent. Editor and OS are resolved by WakaTime from the request User-Agent, so the client now sends a plugin-style User-Agent identifying Kiro Crew. Sending is a separate opt-in from reading. A new wakatime.send_heartbeats config flag defaults off, and the producer is gated on both it and wakatime.enabled, so a user who has not opted in sends nothing and pays nothing. The token and line counts come from the turn that just ran, so they are genuine since-the-last-heartbeat deltas; a line count is only sent when it can be computed. The project entity is the directory basename, never an absolute path. Part of #8094 Co-authored-by: Kiro Crew <noreply@kirodotdev.github.io> A tool awaiting permission retains only the turn's shared bounded key (the 16-char `_tcid_identity_key` digest, never the raw backend id), and admission is capped at `_MAX_TCID_SOURCES` with the overflow logged once, so the pending-coding set stays fixed-width regardless of how many distinct tool ids a turn produces. The permission-request drain uses the same digest key, so admission and drain agree by construction.
|
self-added: yes
|
|
self-added: yes
|
|
|
|
|
Problem / Motivation
Kiro Crew can already read a user's WakaTime coding stats (the productivity view and CSV/JSON export shipped earlier), but it never sends anything back. A developer's Kiro Crew coding time is invisible in their WakaTime dashboard: the agent edits files and runs commands for them, yet none of that activity shows up alongside the time logged from their editor. WakaTime now models AI-assisted coding as a first-class category with its own AI Percent, AI Prompts, and AI Sessions views, and Kiro Crew contributes nothing to them.
Why it matters
Developers who track their time in WakaTime lose the portion of it that happens through Kiro Crew. For anyone billing hours or measuring where their coding time goes, that is a real gap: agent-driven work is a growing share of a day and currently reads as zero. Feeding it into the AI-coding lane makes the picture complete and lets the existing AI-vs-human breakdown reflect reality.
What changed (motivation → approach → change)
Goal: turn a completed agent coding turn into a WakaTime heartbeat, using the client's existing bulk-send method (which had no producer).
Approach: a small in-process producer, hooked at end-of-turn where the usage telemetry already fires. A turn counts as coding activity only when it dispatched a shell command (as resolved by the whole-frame shell classifier — a raw
executetool kind alone does not count, since codex-acp labels read-only MCP calls with it) or a file-write tool; the tool-NAME fallback applies only to native tools, so an MCP server exposing a tool namedwriteis never misread as coding. Each coding tool call's id is held until its permission decision: a denial through Kiro Crew's permission gate drains it without counting, an approval counts it, and an id still held when the turn ends counts then — so a tool Kiro Crew's gate denied never logs time, while a harness that never sends permission events still has its executed tools counted at end of turn. The held set is bounded; ids beyond the cap are dropped and reported in one end-of-turn log line. Heartbeats are produced only from ordinary interactive sessions: a restricted (incognito/temporary) session and an app-owned slot never emit one, and a turn injected by a cron job, an automation loop, or a sub-agent does not count. The project a heartbeat is attributed to is captured at turn start, so a project switch landing mid-turn cannot relabel the work. Each qualifying turn schedules ONE fire-and-forget send task; the destination (base URL and API key) and the opt-in are resolved together inside that task at send time, so nothing is buffered across a credential rotation. The send never awaits on the turn's hot path and a failure drops the row rather than retrying — WakaTime already dedupes by two-minute buckets.The heartbeat is mapped to WakaTime's
ai codingcategory and carries the turn's real signals: input and output token counts read from the turn's usage, a total line-change count computed from the file-change snapshots the turn already keeps (a truncated snapshot is skipped, so an oversized file cannot inflate the count or stall the turn), and the project directory basename as the entity. No session identifier is sent. Editor and operating system are resolved by WakaTime from the request User-Agent rather than a body field, so the client sends a plugin-style User-Agent carrying Kiro Crew's real package version on every WakaTime API request, replacing the default that would log as an unknown OS.Sending is a separate opt-in from reading. A new
wakatime.send_heartbeatsconfig flag defaults off, and the producer is gated on both it and the existingwakatime.enabled, so a user who has not opted in pays nothing and sends nothing. The token and line counts come from the turn that just ran, so they are genuine since-the-last-heartbeat deltas; a line count is only sent when it can be computed, never guessed. The project entity is the directory basename only, never an absolute path.Where each piece lives:
src/kiro_crew/wakatime/heartbeats.py(new)src/kiro_crew/wakatime/client.pysrc/kiro_crew/dashboard/chat_runner.pysrc/kiro_crew/config/sections.py,src/kiro_crew/config/loader.pywakatime.send_heartbeatsopt-in flag (its help text names every exclusion, including incognito/temporary sessions) and its loader passthroughconfig-baseline.jsonsrc/kiro_crew/security_posture.pytest/test_wakatime_heartbeats.py(new)Tests
New
test/test_wakatime_heartbeats.pycovers the producer end to end with a stub client (no network):ai codingheartbeat carrying the token counts, line-change count, and project basename — and no session identifier.executekind with the shell signal false. An MCP-served tool namedwritedoes not count; a nativefs_writewith an empty tool kind still does; an MCP tool witheditkind still counts on the kind evidence.test/test_chat_runner_coverage.pylocks the accounting: a coding id drains uncounted on a Kiro Crew-mediated denial and a denied-only turn emits no heartbeat, an approval counts it, an id that never receives a permission decision still counts at end of turn, ids past the per-turn bound are dropped with exactly one end-of-turn summary line, a truncated file snapshot contributes nothing to the line count, injected (cron/automation) turns emit nothing, and the heartbeat project is the one captured at turn start.The existing WakaTime client tests continue to pass with the added User-Agent.
Manual verification
Verified against the live WakaTime dashboard: with the flag enabled, agent coding turns land in the account's AI-coding activity, and the AI Percent and AI Prompts panels populate for the project rather than showing the activity as unknown-OS human time. The token counts and line-change counts were confirmed to flow into the AI lane using the field shapes documented in the WakaTime heartbeats API. Portal screenshots are personal-account data and are not committed.
Screenshots / video
Why no screenshot: backend-only change — a heartbeat producer and config flag with no dashboard UI.
Related Issues
Part of #8094
Checklist
feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)