Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .github/coverage-baselines/backend.txt
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,20 @@ src/kiro_crew/apps/builtins/personal_shopper/backend/routes.py 71.5 # 186/260
src/kiro_crew/apps/builtins/workflows/server.py 61.2 # 82/134
src/kiro_crew/apps/hooks_integration.py 43.0 # 65/151
src/kiro_crew/apps/route_registry.py 51.9 # 67/129
# NOT a missing-tests entry, and NOT this PR's debt -- it exempts a CI-side
# coverage ATTRIBUTION defect in a file no PR here touches, tracked as #7597.
# The tests exist and pass: TestFleetProbe at test/test_pipeline_conductor_agent.py:179,
# 47 tests green, and the file measures 86% (201/229) when coverage is scoped by
# PATH. Under the lane's own `--cov=kiro_crew` (scoped by import name) the same
# passing tests attribute nothing to it -- locally it does not appear among the
# 1214 reported rows at all, and the 34/229 recorded below is exactly the
# import-only footprint of a bare exec_module. Adding tests cannot move this
# number, which is why the floor's usual remedy does not apply.
# REMOVE THIS ENTRY when #7597 is fixed. The gate enforces that itself: once
# attribution works the file reports >= 82.0% (floor + its 2.0pp band), which is
# the `graduated` verdict -- an ::error:: demanding this line be deleted. So the
# exemption cannot outlive the defect silently.
src/kiro_crew/builtin_skills/pipeline-conductor/scripts/fleet_probe.py 14.8 # 34/229
src/kiro_crew/channel.py 70.5 # 258/366
src/kiro_crew/cli_chat.py 29.5 # 46/156
src/kiro_crew/cli_cloud.py 61.2 # 178/291
Expand Down
59 changes: 54 additions & 5 deletions docs/system-specs/modules/metrics.md

Large diffs are not rendered by default.

37 changes: 37 additions & 0 deletions docs/system-specs/modules/session.md
Original file line number Diff line number Diff line change
Expand Up @@ -855,6 +855,43 @@ start_pool()
└── _ensure_background() → BACKGROUND_KEY session (persistent)
```

## Removing a session from the registry: record the end

**Every path that removes an entry from `_sessions` must report that removal to
`metrics/sessions.py`** — with `record_session_ended(key, end_reason=...)` for a
session that lived, or `discard_session_start(key)` for a registration being
rolled back before it ever became one. This is a correctness requirement on
lifecycle code, not a telemetry nicety, and it is documented here rather than only
in the metrics module because the people who can break it are editing this
subsystem.

The reason it matters more than a missing sample: a session start writes a
breadcrumb file that survives process death, which is what lets a session killed
with its gateway be counted at all. A removal that reports nothing leaves that
breadcrumb behind, and the next boot reads a surviving breadcrumb as a crash. So
an unrecorded removal does not lose a data point — it **fabricates a crash that
never happened**, inside the one population the instrument exists to expose.

Practical rules when you add or change a removal path:

- Report it in the SAME event-loop tick as the `pop` / `del` / `clear`, before the
path's first `await`. Later than that, a racing cold start can register a
successor under the same key and have its record consumed by the predecessor's
teardown.
- Give it its own `end_reason` if it is genuinely a different event. The enum is
closed and lives in `metrics/sessions.py`; reusing a label merges two
populations, and a metric is not a good reason to grow a lifecycle signature.
- A registration cancelled mid-flight is NOT an end. Use `discard_session_start`,
which consumes the breadcrumb without emitting a lifetime.

`test/metrics/test_session_duration.py::TestEveryRegistryRemovalRecordsAnEnd`
enforces this. It is fail-closed and discovers its own scope: an AST walk over
every module under `src/kiro_crew` that mentions `_sessions`, recognising the
`pop`, `del` and `clear` spellings, so a new removal path anywhere fails the gate
the day it lands rather than waiting to be added to a list. A container that
merely shares the attribute name can be exempted with a stated reason, and the
exemption self-voids if that module ever starts writing breadcrumbs.

## Security: PreToolUse Command Enforcement

Command denial is enforced by Kiro Crew's bundled `hooks.py` `PreToolUse` gate,
Expand Down
23 changes: 21 additions & 2 deletions src/kiro_crew/acp/_dispatch.py
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@
AcpEvent,
JsonRpcMessage,
)
from kiro_crew.metrics.tool_calls import note_tool_call_started, record_tool_call_finished
from kiro_crew.security import redact_credentials, redact_exfiltration_urls

logger = logging.getLogger(__name__)
Expand Down Expand Up @@ -879,6 +880,16 @@ def _build_tool_call_event(
_mcp_server_name = _kiro_mcp_server_name(update)
if tool_call_id and mcp_server_name_cache is not None:
mcp_server_name_cache[_ck] = _mcp_server_name
# Round-trip clock for kirocrew.tool.call.duration. Placed after the trusted
# MCP identity is resolved so an MCP call is classified by its transport
# rather than by the kind it reported; the terminal status is stamped in
# _build_tool_result_event. Keyed by the SAME origin scope as the caches
# above, because one runtime hosts many sessions and a backend-assigned
# toolCallId is unique only within one of them. Idempotent per scoped id, so
# the tool_call_update refinements that follow cannot restart the clock.
note_tool_call_started(
tool_call_id, kind=kind, mcp_server_name=_mcp_server_name, scope=cache_scope
)
# Same lifecycle for the trusted tool name (_meta.kiro.toolName) so the
# permission event can reconstruct the canonical mcp__<server>__<tool> for
# per-tool governance in the app-own-server auto-approve.
Expand Down Expand Up @@ -992,17 +1003,25 @@ def _mcp_content_text(payload: dict[str, Any]) -> str | None:
return "\n".join(parts)


def _build_tool_result_event(update: dict[str, Any]) -> AcpEvent | None:
def _build_tool_result_event(update: dict[str, Any], cache_scope: str = "") -> AcpEvent | None:
"""Build an ``EVENT_TOOL_RESULT`` from a ``tool_call_update`` carrying output.

Two output shapes: ``content[].content.text`` blocks (stream mid-turn), or
``rawOutput.items[]`` (``Text`` / ``Json.stdout``) on ``status=completed``.
Returns None when the update carries no output (refinement-only updates are
handled by :func:`_build_tool_refinement_event`).

``cache_scope`` is the emitting session's origin scope, forwarded only so the
duration histogram closes the same registry entry its start opened.
"""
tool_use_id = update.get("toolCallId", "")
if not tool_use_id:
return None
# Before the output parsing below, which returns None for an output-less
# update: a tool that completed with no output is still a completed
# round-trip. A non-terminal status is a no-op here, so a mid-stream update
# leaves the clock running for the real completion.
record_tool_call_finished(tool_use_id, status=update.get("status"), scope=cache_scope)
# Parts are collected RAW and redaction runs once over their JOIN, before
# the single 8000-char bound. Both orderings matter: bounding first can
# split a credential at a cut into fragments no redaction regex matches,
Expand Down Expand Up @@ -1340,7 +1359,7 @@ def parse_session_update(
)
return events
if kind == UPDATE_TOOL_CALL_UPDATE:
result = _build_tool_result_event(update)
result = _build_tool_result_event(update, cache_scope)
if result is not None:
events.append(result)
refine = _build_tool_refinement_event(
Expand Down
26 changes: 26 additions & 0 deletions src/kiro_crew/acp/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,7 @@
from kiro_crew.kiro_cli import known_kiro_cli_dirs, resolve_kiro_cli
from kiro_crew.mcp_gateway.claim import schedule_claim
from kiro_crew.mcp_gateway.session_servers import pooled_session_servers
from kiro_crew.metrics.tool_calls import note_tool_call_started, record_tool_call_finished
from kiro_crew.resource_status import inject_xdist_auto_cap
from kiro_crew.sandbox import (
RLIMIT_PROFILE_SESSION_HOST,
Expand Down Expand Up @@ -5760,6 +5761,21 @@ def _extract_tool_event(self, msg: JsonRpcMessage) -> AcpEvent | None:
)
# Build initial tool input string from raw params
tool_call_id = update.get("toolCallId", "")
# Start the round-trip clock here rather than at the yield: this is
# the first moment Kiro Crew sees the call, and the same id's terminal
# status is stamped in _extract_tool_call_update below. Both of this
# class's message loops reach this method, so instrumenting it covers
# them without a second call site in each.
note_tool_call_started(
tool_call_id,
kind=kind,
mcp_server_name=_kiro_mcp_server_name(update),
# getattr: a real client always carries _session_id, but this
# extractor is also driven directly with lightweight test
# doubles (test_acp_tool_identity), and telemetry must not be
# the reason such a double stops working.
scope=getattr(self, "_session_id", "") or "",
)
input_str = ""
if tool_call_id and raw_input:
input_str = (
Expand Down Expand Up @@ -5887,6 +5903,16 @@ def _extract_tool_call_update(self, msg: JsonRpcMessage) -> AcpEvent | None:
tool_use_id = update.get("toolCallId", "")
if not tool_use_id:
return None
# Stamped before the output parsing below, which returns None for an
# output-less update: a tool that completes with no output is still a
# completed round-trip and must not be dropped from the histogram. A
# non-terminal status is a no-op, so a mid-stream update leaves the clock
# running for the real completion.
record_tool_call_finished(
tool_use_id,
status=update.get("status"),
scope=getattr(self, "_session_id", "") or "",
)

output_parts: list[str] = []

Expand Down
9 changes: 9 additions & 0 deletions src/kiro_crew/artifacts.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@
WebAppTeardown,
webapp_metadata_from_dict,
)
from kiro_crew.metrics.events import ARTIFACTS_CREATED, emit_counter
from kiro_crew.publish_provider import DEFAULT_PROVIDER
from kiro_crew.security import is_sensitive_path

Expand Down Expand Up @@ -1174,6 +1175,14 @@ def create(
self._write_artifact(art, content)
logger.info("artifact created: slug=%s name=%s kind=%s", slug, name, kind)
self._fire_change("upsert", slug)
# After the write, so a failed create contributes nothing. ``kind`` and
# ``source`` are the values ``_validate_kind`` / ``_validate_source``
# already restrict to closed sets, and ``kind_auto`` says whether the
# kind was inferred rather than pinned by the caller.
emit_counter(
ARTIFACTS_CREATED,
{"kind": kind, "source": source, "kind_auto": bool(kind_auto)},
)
return art

def create_image(
Expand Down
12 changes: 12 additions & 0 deletions src/kiro_crew/cron.py
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@
from kiro_crew.constants import env_flag_enabled
from kiro_crew.cron_history import CronHistoryStore, CronRunRecord
from kiro_crew.executors import _CRON_QUEUE_WAIT_SECS, cron_gate_budget, subprocess_executor
from kiro_crew.metrics.events import CRON_FIRES, emit_counter
from kiro_crew.resource_status import admission_check
from kiro_crew.validation import CHANNEL_MAX_LEN, MAX_CRON_MESSAGE, MAX_SHORT_STRING

Expand Down Expand Up @@ -3302,6 +3303,17 @@ async def _run_job_isolated(self, job: CronJob) -> None:
started_at = meta[0] if meta else time.time()
trigger = meta[1] if meta else "scheduled"
self._job_start_times[job.id] = started_at
# One increment per execution, before the jitter sleep so a run cancelled
# during jitter still counts as fired. ``kind`` is the dispatch shape --
# ``script`` and ``command`` bypass the model entirely, so this is the
# split between jobs that cost tokens and jobs that cost none.
emit_counter(
CRON_FIRES,
{
"kind": "script" if job.script else "command" if job.command else "agent",
"trigger": trigger,
},
)
# Apply jitter to spread execution unless strict_schedule is set or manual
jitter = self._compute_jitter(job) if trigger != "manual" else 0
self._job_jitter[job.id] = jitter
Expand Down
37 changes: 37 additions & 0 deletions src/kiro_crew/hooks.py
Original file line number Diff line number Diff line change
Expand Up @@ -139,17 +139,53 @@ class ToolHookResult:
#: classify a sensitive-path or exfiltration deny as non-security.
security_deny: bool = True

@staticmethod
def _count(action: str, security_deny: bool) -> None:
"""Count one gate verdict. Best-effort; never changes the decision.

Called by the four factories below and nowhere else, which is what makes
it fire exactly once per gate consultation. A surface that OVERRIDES the
gate constructs a result directly (``chat_runner`` downgrades an
auto-approve to an interactive card this way, twice) and so is not
counted: the gate was consulted once, and counting every constructed
object would report one request as two decisions AND keep a count for a
verdict that was then discarded.

The two rejected alternatives were instrumenting the 23 exits of
``HookManager.on_tool_call`` (23 call sites on a security path) and
counting in ``__post_init__`` behind a ``from_gate`` flag -- the flag had
no reader other than the counter itself, so it was state carried purely to
signal, where calling this from the four factories says the same thing
with no field at all.

``action`` is one of three module constants and ``security_deny`` a bool,
so the series is bounded by construction -- no reason string, tool name or
command reaches the recorder.
"""
try:
from kiro_crew.metrics.events import APPROVAL_DECISIONS, emit_counter

emit_counter(
APPROVAL_DECISIONS,
{"decision": action, "security_deny": bool(security_deny)},
)
except Exception: # a tool decision must never fail on its telemetry
logger.debug("approval decision counter failed", exc_info=True)

@staticmethod
def allow() -> ToolHookResult:
ToolHookResult._count(TOOL_ALLOW, False)
return ToolHookResult(action=TOOL_ALLOW)

@staticmethod
def auto_approve() -> ToolHookResult:
ToolHookResult._count(TOOL_AUTO_APPROVE, False)
return ToolHookResult(action=TOOL_AUTO_APPROVE)

@staticmethod
def deny(reason: str) -> ToolHookResult:
"""Deny on a hard security check — the attempt is the problem."""
ToolHookResult._count(TOOL_DENY, True)
return ToolHookResult(action=TOOL_DENY, reason=reason, security_deny=True)

@staticmethod
Expand All @@ -160,6 +196,7 @@ def deny_policy(reason: str) -> ToolHookResult:
durable budget (cron auto-pause) does not treat a governance ceiling as
a defect in what it attempted.
"""
ToolHookResult._count(TOOL_DENY, False)
return ToolHookResult(action=TOOL_DENY, reason=reason, security_deny=False)


Expand Down
2 changes: 2 additions & 0 deletions src/kiro_crew/mcp_gateway/stub.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@
from kiro_crew.mcp_gateway import transport
from kiro_crew.mcp_gateway.hashing import hash_command, hash_effective_env
from kiro_crew.mcp_gateway.pool import READ_BUFFER_LIMIT_BYTES, PoolKey
from kiro_crew.metrics.events import MCP_RECONNECTS, emit_counter

logger = logging.getLogger(__name__)

Expand Down Expand Up @@ -1399,6 +1400,7 @@ async def _pause() -> bool:
return None
session.note_init_result(json.loads(forward))
session.reconnects += 1
emit_counter(MCP_RECONNECTS, {"pool": bool(pool_label)})
logger.info(
"stub reconnected to a restarted gateway (%s, reconnect #%d) pool=%s",
detail,
Expand Down
59 changes: 59 additions & 0 deletions src/kiro_crew/metrics/events.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,3 +58,62 @@ def emit_counter(name: str, attrs: dict[str, str | int | bool | float]) -> None:
#: Idle-sweep expiries — ``turn_active=True`` means the sweep killed a
#: runtime mid-turn, the teardown signature of the original incidents.
SESSION_IDLE_EXPIRED = "kirocrew.session.idle_expired"


# ---------------------------------------------------------------------------
# Business-event series — "did this subsystem do its job, and how often"
# ---------------------------------------------------------------------------
# Each of these is emitted at its own subsystem's call site rather than from a
# central observer, because there is no frame every one of them crosses. What
# they share is the attribute rule above: every value is a constant from a
# closed set the emitting site owns, never an id, a name, or a count that grows
# with input.

#: One per subagent whose spawn actually started. ``concurrency`` is the live
#: running count INCLUDING this one (the counter is emitted after the admission
#: increment), so it reads 1 for a lone subagent -- an integer bounded by the
#: spawn cap, which keeps the series small and makes the aggregator's MAX over it
#: the concurrency high-water mark. A separate high-water instrument would need
#: its own reset semantics and could not be read per-attribute like this.
SUBAGENTS_SPAWNED = "kirocrew.subagent.spawned"

#: One per cron job execution. ``kind`` separates the three dispatch shapes
#: (``agent`` runs an LLM turn, ``script`` and ``command`` bypass the model
#: entirely), which is the difference between a job that costs tokens and one
#: that costs none. ``trigger`` says whether the schedule or a human fired it.
CRON_FIRES = "kirocrew.cron.fires"

#: One per artifact created. ``kind`` is the artifact's validated type and
#: ``source`` the validated origin — both already closed sets in ``artifacts``.
ARTIFACTS_CREATED = "kirocrew.artifact.created"

#: One per dynamic-workflow run started, foreground or background (the
#: background entry point drives the same method). ``authored`` marks a run that
#: writes its own script from an intent; ``replay`` marks a restart-subtree that
#: reuses cached agent results instead of re-calling the model.
WORKFLOW_RUNS = "kirocrew.workflow.runs"

#: One per context compaction that reached a verdict, so the sum is compaction
#: ATTEMPTS. ``success`` reports whether the compaction itself completed, NOT how
#: much context it reclaimed -- no before/after reading is taken -- and it is
#: False when an in-place compaction failed and the session was recycled instead.
CONTEXT_COMPACTIONS = "kirocrew.context.compactions"

#: One per MCP stub reconnect to a restarted daemon. Each increment is a
#: bridge that died and was rebuilt under a live session, so a rising rate is
#: daemon instability that sessions are absorbing silently.
MCP_RECONNECTS = "kirocrew.mcp.reconnects"

#: One per tool-approval decision from the per-surface gate
#: (``hooks.HookManager.on_tool_call``), which every surface consults before a
#: tool runs. ``decision`` is the gate's own bounded action: ``auto_approve``
#: skipped the human, ``deny`` refused without asking, and ``allow`` is the
#: branch that falls through TO an interactive prompt — so the ``allow`` slice
#: is approvals shown and the ``deny`` slice is approvals denied.
#: ``security_deny`` separates a hard security refusal from a policy-state one.
#:
#: This generalises the backend-child pair above (:data:`CHILD_PERMISSION_DENIED`
#: / :data:`CHILD_PERMISSION_ROUTED`) to every approval prompt. Those two stay
#: exactly as they are: they measure a specific hang-resilience fix on the
#: child-permission path, and their population is not this one's.
APPROVAL_DECISIONS = "kirocrew.approval.decisions"
Loading
Loading