feat(conventions): add prompt_id correlation to the hook-telemetry envelope

[kyle-sexton]

Summary

Claude Code's hook input JSON carries a prompt_id field (v2.1.196+) that matches the prompt.id attribute on real OpenTelemetry events (per https://code.claude.com/docs/en/hooks, fetched 2026-07-22). This would let external tooling correlate a hook's local hook-telemetry envelope entry with the same turn's real OTel stream — currently the envelope (schema_version: "1.0", lib/hook-utils.sh:391 hook::emit_telemetry) has no such field.

Why deferred from #836

#836 (hook-observability: statusMessage/systemMessage/telemetry adoption) found this while researching fresh hooks docs, but adding it is a hook-telemetry schema change (schema_version 1.0 → 1.1), not a hook-observability concern, and touches every producer's data_json construction (~25 call sites) — a distinct, complete sweep of its own rather than a partial addition riding along #836's unrelated scope.

Scope (when picked up)

• Bump hook-telemetry's schema_version to 1.1, add prompt_id to the envelope (either a new optional parameter on hook::emit_telemetry, defaulting empty/omitted for backward compatibility, or captured inside each producer's data object).
• Update every producer hook (all wired plugins/*/hooks/*.sh — see docs/conventions/hook-telemetry/README.md's Implementers table) to extract prompt_id from its buffered stdin JSON and pass it through, so the field is uniformly populated across the fleet — a partial rollout would be indistinguishable from "prompt_id genuinely absent, pre-first-input" per the docs, which defeats the point.
• Document the correlation mechanism and the OTEL_* env-var-stripping rationale (why hooks can't emit real OTel directly) in docs/conventions/hook-telemetry/README.md.

Related

• Epic: lint/static-analysis gap closure #830 (epic: lint-static-analysis-gaps)
• feat(conventions): hook-observability fleet convention + fleet adoption #836 (hook-observability convention — where this was found and deferred)

[kyle-sexton]

This was generated by AI (triage-lane planning subagent).

Implementation plan — prompt_id correlation on the hook-telemetry envelope

What the code actually shows (verified against origin/main)

Central emitter. hook::emit_telemetry lives at lib/hook-utils.sh:391 (issue's :391 is correct). It is positional: <hook_id> <hook_event> <status> <start_epoch> <data_json> [repo_root] (args 1–6). The envelope is assembled by a single jq -n at lib/hook-utils.sh:424–431 with --arg schema_version "1.0" hardcoded at line 424. The top-level envelope keys are schema_version, timestamp, hook, hook_event, status, duration_ms, data — data is the only per-hook object; every other key is cross-cutting metadata.

Adopt-by-copy / sync mechanism (not in the issue — CI-gating). There is no shared import: lib/hook-utils.sh is the source and is copied into 12 carrying plugins at plugins/<name>/hooks/hook-utils.sh by scripts/sync-hook-utils.sh. The carriers are: actionlint, bash-format, biome-format, claude-ops, desktop-notification, eol-normalizer, go-format, guardrails, markdown-format, powershell-format, ruff-format, typos-format. That script has two gates:

• --check fails if any copy differs from the source.
• --check-bump <ref> fails if lib/hook-utils.sh changed vs <ref> but a carrier's .claude-plugin/plugin.json version did not bump (the plugin version is the consumer update-cache key; verified field: e.g. plugins/bash-format/.claude-plugin/plugin.json → "version": "0.5.2").

Consequence: editing the emitter forces sync-hook-utils.sh re-run + a version bump on all 12 carrier manifests. Never hand-edit the plugin copies.

Producer enumeration — real numbers, not the estimate. The issue estimates "~25 call sites." Reality: 26 producer files emit telemetry, with ~68 emit_tel/hook::emit_telemetry textual call sites — but only 26 actual hook::emit_telemetry invocation points (one per file), because most producers wrap the emitter. Every one of the 26 buffers stdin into INPUT via hook::buffer_stdin before emitting, so .prompt_id is uniformly extractable. Three call shapes:

• "$@"-forwarding wrappers (8): actionlint, bash-format, biome-format, eol-normalizer, go-format, powershell-format, ruff-format, typos-format — emit_tel() { …; hook::emit_telemetry "$@"; }, called 1–4×, each call passing exactly 6 args ending in "$REPO_ROOT".
• Fixed-arg wrappers (7): markdown-format, and guardrails block-dangerous-git, block-hook-bypass, block-no-verify, block-noncanonical-commit, hardcoded-path-check, secret-pattern-detection — one hook::emit_telemetry "<id>" … "${CLAUDE_PROJECT_DIR:-}" line, wrapper takes $1=status/$2=payload.
• Direct single callers (11): desktop-notification, all 8 claude-ops/*-audit, guardrails cli-flag-verify, flag-commit-pr-skill-bypass.

All 26 pass a 6th arg (repo_root/CLAUDE_PROJECT_DIR). Note the README Implementers table (docs/conventions/hook-telemetry/README.md:151–172) lists only 18 hook values and is a stale subset — e.g. actionlint, biome-format, eol-normalizer, powershell-format, block-dangerous-git, block-noncanonical-commit, flag-commit-pr-skill-bypass are missing, and skill-usage-expansion-audit.sh emits under hook: "skill-usage-audit". The authoritative producer set is the code (26 files), not the table. The table needs its own reconciliation (out of scope here; flagging).

Full doc/schema/test surface a schema bump touches (beyond the 26 producers):

• docs/conventions/hook-telemetry/README.md — envelope table (:46–54), the invariant line :44 ("these seven fields. All are required and always present"), correlation/OTel rationale.
• docs/conventions/hook-telemetry/envelope.schema.json — required[] + properties.
• docs/conventions/hook-telemetry/CHANGELOG.md — add a ## 1.1 entry.
• docs/conventions/hook-telemetry/examples/*.json — 14 golden fixtures, each "schema_version": "1.0".
• lib/hook-utils.test.sh — field-list loops (:114, :159), schema_version == "1.0" assertion (:129–134), emit-call fixtures.

The design fork — RECOMMENDED: envelope-level optional param

Decision defaulted: envelope param on hook::emit_telemetry (not inside each producer's data) — veto before merge

The issue frames the fork as (A) a new optional param on hook::emit_telemetry vs (B) captured inside each producer's data object. (The #836 deferral note assumed (B) — "touches every producer's data_json construction" — this plan deliberately reverses that.) Either way each producer must extract .prompt_id from its buffered $INPUT; the fork is only where the value lands in the envelope. Choose (A), envelope-level:

1. schema_version versions the envelope. A 1.0 → 1.1 bump is only coherent if the new field is an envelope field. Under (B) the envelope shape is unchanged and the version bump is misplaced. This is the decisive argument.
2. Semantic class. prompt_id is cross-cutting correlation identity — same class as hook, hook_event, timestamp — not hook-specific payload. data is explicitly "everything not universal to all hooks" (README :74, :80).
3. Uniform shape / meaningful absence. An envelope field appears identically for every producer at a fixed path. In data, shape is heterogeneous per hook, so a consumer can't reliably locate it, and "missing" conflates pre-first-input absence with this producer didn't add it.
4. Lower churn, less drift. (A) = one emit-point edit + one extraction line per file (26 files), plus a shared helper. (B) = editing each producer's bespoke data_json/jq template — strictly more work and more drift surface.

Sub-decision (maintainer may fold into the veto): field is always present, empty string when the run has no prompt yet (e.g. SessionStart/ConfigChange before first input), added to the schema required[]. Justified on invariant + stable shape — it keeps README :44's "all fields always present" true (now eight fields) and gives sinks one fixed shape — not on acceptance (see below). The param defaults empty for backward-compat: an unmigrated 6-arg caller still yields a valid 1.1 envelope with prompt_id: "".

Considered alternative (rejected, noted to avoid solution-shape bias): have the emitter read a global ${HOOK_PROMPT_ID:-} (set once per producer), matching its existing env-reading idiom (HOOK_TELEMETRY_SINK, CLAUDE_PROJECT_DIR). This sidesteps positional coupling but trades an explicit signature for implicit global state; prefer the explicit param unless the maintainer dislikes the positional precondition below.

Threading mechanism (why 26 files, not 68 edits)

Add a tolerate-absence helper to lib/hook-utils.sh (syncs to all 12 carriers):

# Extract prompt_id from buffered hook-input JSON; empty when absent
# (pre-first-input) or jq missing. NEVER returns nonzero — safe under `set -u`
# and safe to inline without a `|| true` guard.
hook::prompt_id() {
  command -v jq >/dev/null 2>&1 || { printf ''; return 0; }
  printf '%s' "$1" | jq -r '.prompt_id // empty' 2>/dev/null
}

Per producer, after INPUT=$(hook::buffer_stdin): PROMPT_ID="" then PROMPT_ID=$(hook::prompt_id "$INPUT") (init-first so an injected wrapper referencing $PROMPT_ID is set -u-safe even on the empty-stdin early paths). Then, by shape:

• "$@" wrappers (8): change the wrapper body to hook::emit_telemetry "$@" "$PROMPT_ID" — zero changes to the many emit_tel call sites.
• Fixed-arg wrappers (7) + direct callers (11): append "$PROMPT_ID" to the single emit line.

Precondition (state it in code review): wrapper injection lands prompt_id in slot 7 only because every forwarded call passes exactly 6 args through repo_root (verified true today). A future 5-arg call site would silently shove prompt_id into the repo_root slot — the shared hook::prompt_id helper and the conformance check below are what keep that honest.

Concrete sweep (ordered)

1. Emitter (lib/hook-utils.sh): add optional 7th param prompt_id="${7:-}"; bump --arg schema_version 1.0 → 1.1; add --arg prompt_id "$prompt_id" and prompt_id:$prompt_id to the jq object (:424–431); add the hook::prompt_id helper.
2. Sync + version bumps: run scripts/sync-hook-utils.sh; bump version in all 12 carrier .claude-plugin/plugin.json manifests (satisfies --check and --check-bump).
3. 26 producers: add the PROMPT_ID init+extract line and the per-shape emit edit above.
4. Contract docs: README — add prompt_id row to the envelope table, update :44 "seven"→"eight", and add a Correlation subsection: envelope prompt_id equals CC hook-input prompt_id which equals the OTel prompt.id attribute — the join key already extracted by plugins/claude-ops/skills/observability/otel/cc-otel.sql:96,190 (prompt.id AS prompt_id) and listed as an OTel-store join key in operator-setup-retention.md. Point (do not restate) to the existing OTel-stripping rationale at docs/conventions/hook-observability/README.md:88–92: Claude Code itself strips every OTEL_* exporter env var from hook subprocesses (cited there to code.claude.com/docs/en/monitoring-usage#administrator-configuration), so a hook cannot emit real OTel and the file-sink envelope + prompt_id is the only correlation path. (Verified: this stripping is CC platform behavior, not repo code — no repo grep hit, and none should.)
5. Schema: envelope.schema.json — add prompt_id to required[] and properties (type: string; describe empty-string-when-pre-first-input).
6. Fixtures: all 14 examples/*.json → schema_version: "1.1" + a prompt_id line; use a realistic value on the tool-event examples and "" on genuinely pre-first-input ones (e.g. config-change-audit, pre-compact-audit) to document the absence case.
7. CHANGELOG: add ## 1.1; note adding a field is additive/minor (consumers ignore unknown keys; additionalProperties: true) — 1.1 is not a breaking bump.
8. Tests (lib/hook-utils.test.sh): add prompt_id to both field-list loops; flip the schema_version assertion to 1.1; add a case asserting a passed 7th arg surfaces as envelope.prompt_id and that omitting it yields "".

Acceptance (and why "assert uniform" is not enough)

• Field populated across all 26 producers. With always-present-empty, a missed producer silently emits a well-formed prompt_id: "" — per-record indistinguishable from genuine pre-first-input. Output inspection cannot catch a miss. So acceptance requires a source-level conformance check, idiomatic here alongside --check/--check-bump: assert every plugins/*/hooks/*.sh (excluding the hook-utils.sh copies) that invokes hook::emit_telemetry also threads prompt_id/$PROMPT_ID. Recommend adding it as a sync-hook-utils.sh mode or a standalone test wired into CI. This check, plus uniform rollout — not the always-present encoding — is what makes an empty prompt_id reliably mean "genuinely pre-first-input."
• sync-hook-utils.sh --check and --check-bump pass; lib/hook-utils.test.sh passes; all 14 fixtures + envelope.schema.json consistent at 1.1.

Related / scope

• feat(conventions): hook-observability fleet convention + fleet adoption #836 (hook-observability) is where this was found and deferred (docs/conventions/hook-observability/README.md:95–101, which already links here). Standalone schema change — not an umbrella fold into Epic: lint/static-analysis gap closure #830 (epic: lint-static-analysis-gaps) or feat(conventions): hook-observability fleet convention + fleet adoption #836.
• Side-finding (separate issue candidate, not this scope): the README Implementers table under-lists producers (18 rows vs 26 emitting files) and should be reconciled.

[kyle-sexton]

This was generated by AI (triage-decide lane — intake routing).

Routed status: ready per T2 (ready-with-noted-decision). The planning subagent's plan (above) verified every path/count against origin/main (emitter lib/hook-utils.sh:391, 26 producer files — not the issue's ~25 estimate — and 12 carrier manifests needing schema_version bumps, CI-gated by sync-hook-utils.sh --check-bump, which the issue omits). High confidence; single maintainer-vetoable fork with a defaulted answer (see the bolded Decision defaulted line in the plan — envelope-param on hook::emit_telemetry).

No open prerequisite gates this. Standalone (not folded into #830/#836 — related only). Implementer: the 12-manifest bump is in-scope; a partial producer rollout is indistinguishable from genuine prompt_id absence, so the sweep must cover all 26 uniformly (acceptance criterion in the plan).

[kyle-sexton]

Claim (work lane, T3) — dispatching an implementation subagent this cycle. 2026-07-22T17:18:47Z

[kyle-sexton]