Skip to content

feat(monitoring): let an edition register a pull-request-shaped monitor kind at boot - #24513

Merged
iamwhatever merged 1 commit into
kirodotdev:mainfrom
the-mann:feat/monitor-kind-plugins
Nov 2, 2026
Merged

iamwhatever merged 1 commit into
kirodotdev:mainfrom
the-mann:feat/monitor-kind-plugins

Conversation

@the-mann

@the-mann the-mann commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A downstream edition can register its own pull-request-shaped review system as a monitor_watch kind at boot, so watching a review there costs no model turn on quiet cycles, the same as the four built-in hosts.

monitor_watch only knows GitHub, GitLab, Azure DevOps and Bitbucket. #9546 made each kind a registry entry, but the table is a static literal and the provider table, target parsers and validator allowlists are fixed at import. An edition that ships a review host the public core does not know about has no seam to add a kind, so its users fall back to a monitor_start loop with gate=false that spends a full model turn every interval even when nothing changed.

Why it matters

Babysitting a review until it lands is the most common long-running watch. For every review system outside the built-in four it is currently the most expensive path the product has. A boot-time seam gives those systems the turn-free probing the built-ins already get, without core learning anything about them. This is the "generic/pluggable provider interface" option #17848 asks for.

Not a goal

What changed (motivation → approach → change)

The registry already treats a kind as data, so the seam follows the existing register_source_provider pattern rather than a new mechanism. An edition calls register_monitor_kind(MonitorKindPlugin(kind=..., provider_factory=..., parse_target=...)) once at boot, and every boundary that gates a kind reads it from then on:

  • monitoring/plugins.py (new): the registration call, with loud refusals for a malformed, built-in or duplicate name, a missing callable, or an objective the pull-request path does not implement.
  • monitoring/models.py: MonitorProbeProvider is the one probe contract the controller calls, MonitorProbe plus use_owner_credentials. The controller and a contributed kind's provider both type to it; the shadow path keeps the narrower MonitorProbe.
  • monitoring/registry.py: contributed entries live in a separate table, so a built-in can never be replaced; lookups and the publicly-armable sets include them; is_pull_request_monitor_kind() replaces the frozenset membership test the facts and projection validators used.
  • monitoring/targets.py: inference tries contributed parsers only after every built-in, so a contributed parser cannot claim a built-in host's URL; normalization dispatches to the contributed parser. Anything a contributed parser raises reads as "not this kind's URL", so callers that handle only ValueError stay correct.
  • monitoring/controller.py: the default provider table adds each contributed kind's provider; a factory that raises is logged and fails only its own kind instead of aborting gateway boot. Owner-credential gating is unchanged, so a contributed kind is fail-closed on channel-created monitors like the non-GitHub/GitLab built-ins, and the registry refuses it the shadow path, which has no such gate.
  • validation_parts/field_engine.py: FieldSpec.allowed may be a zero-argument callable resolved per validation. MONITOR_WATCH_SCHEMA and MONITOR_UPDATE_SCHEMA pass the registry functions instead of their import-time result, which would otherwise freeze the vocabulary before any edition boots.
  • docs/system-specs/modules/monitor-architecture.md: the registry row and Adding a new monitored kind describe the seam and the limits a registrant designs for: register unconditionally from the edition's platform-context builder (every process validates or probes), a monitor whose kind is absent on a later boot stops as blocked, and the dashboard, whose kind list is a checked-in snapshot, shows a contributed-kind monitor as blocked with save and restart disabled.

Backwards compatibility

Compatible: every built-in kind, objective and URL resolves exactly as before; with nothing registered the registered set is empty and every new branch is unreachable. FieldSpec.allowed still accepts a frozenset; the only reader of it is validate_field, updated here.

Removes one incidental re-export: kiro_crew.monitoring.pull_request no longer imports PULL_REQUEST_MONITOR_KINDS, because the facts validator now asks the registry. The constant still lives in monitoring/registry.py and is still re-exported by monitoring/models.py. No kind, field, key, path or public function is removed or tightened.

Reader: test/test_pull_request_readiness.py, the only importer of the name from pull_request, now imports it from monitoring/registry.py.

Tests

  • test/test_monitor_kind_plugins.py (new): registration reaches every registry query and an unregistered kind is absent from all of them; a built-in name, a duplicate, an unsupported or empty objective set, a shadow claim and malformed names are refused without side effects; a factory that raises leaves the built-in providers in place; a parser that raises a non-ValueError is a non-match in both inference and normalization; inference reaches a contributed parser but never lets one claim a GitHub URL; normalization uses the contributed parser and still refuses an unregistered kind; a kind registered after import validates through MONITOR_WATCH_SCHEMA and an unregistered one is refused; the controller's default table probes a contributed kind and passes use_owner_credentials=False for both a channel-created monitor and a dashboard one with no recorded grant. An autouse fixture snapshots and restores both module-global tables, so a regressed refusal cannot leak to a neighbour.
  • test/test_monitor_kind_registry.py: the schema-parity assertion resolves a callable allowed.
  • Determinism: both files green on 5 repeats, 3 shuffled orders and under xdist (-n 4). test/test_flake_pattern_ratchet.py green.
  • Mutation: removing each new guard (factory try/except, parser fold, shadow refusal, empty-objectives refusal, callable allowed) turns at least one of these tests red.

Manual verification

N/A — unit coverage drives every boundary the seam touches (validator, targets, controller tick against a real AutoNudgeService); no UI or external service is involved.

Related Issues

Part of #17848

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@the-mann
the-mann requested review from a team and KiroCrewCaptain as code owners October 26, 2026 17:57
@the-mann
the-mann requested a review from iamwhatever October 26, 2026 17:57
@the-mann

Copy link
Copy Markdown
Contributor Author

Local review note: the two local reviewers ran on fallback charters (local_review.py exit 40, __MERGE_BASE__ unmapped locally), and the gpt lane's pin was served by an Opus-family model, so local review ran at reduced fidelity. Semgrep did not run locally (no semgrep-core binary). CI's lanes are the authority.

@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention labels Oct 26, 2026
@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5, fork) — ⏭️ skipped

Revision 11b7499aa098f37b0e915cfef02270a2ba71adf0 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5, fork) — ✅ PASS

Design-level review of 11b7499aa098f37b0e915cfef02270a2ba71adf0 via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the seam reuses the existing registry, pull-request facts and owner-credential gate. Built-ins can't be shadowed, and the validator and MCP schema read the live vocabulary.

Notes

  • The dashboard shows a contributed-kind monitor as blocked, with save and restart disabled (models.py pullRequestMonitorKinds reads the checked-in contract snapshot). Until the runtime kind-list follow-up lands, an edition's users get a dashboard that disagrees with a monitor that is probing. Land that follow-up before an edition ships this seam.
  • The controller reads the plugin table once, when it is built. The gateway's boot order (boot_platform before MonitorController is constructed) is what makes this correct, but only the docs state it. Logging a warning when register_monitor_kind runs after a controller exists would make a mis-ordered edition visible instead of silently blocking its monitors.

[DESIGN-REVIEWED] 11b7499

Description read

[DESCRIPTION-READ] 478306f02bdaffbb0466d02250714c8f708eef9ffa3756615a9b8c01700400f8

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5, fork) — ✅ PASS

Premise-level review of 11b7499aa098f37b0e915cfef02270a2ba71adf0 via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks. The public seam has zero callers in this repository, but that is the same as the existing edition seam register_source_provider.

Notes

  • Item 1 — zero consumers: register_monitor_kind has 0 callers in src/ (grepped register_monitor_kind). Its only stated consumer is a downstream "internal review host" that cannot be checked from here. The precedent register_source_provider also has 0 callers in the repository. Subtraction: defer the public register_monitor_kind/MonitorKindPlugin export until a named edition calls it, keeping the seam private to monitoring/.

What this change ships

Inventory (6 items) — 5 justified

Intent: let a downstream edition watch reviews on its own review host without spending a model turn on quiet cycles. This is an ADDITION.

  1. Editions can register a new pull-request-shaped monitor_watch kind at boot — zero consumers in the repository; the only consumer is downstream
  2. monitor_watch and monitor_update accept kinds and objectives registered after import, because the allowed set is now read on each validation — justified
  3. Target inference and normalization try contributed parsers only after every built-in parser, and refuse a target that is not https — justified
  4. The controller probes contributed kinds, and a factory that raises fails only its own kind — justified
  5. The pull-request facts and projection validators accept contributed kinds through is_pull_request_monitor_kind — justified
  6. The controller's private provider Protocol becomes the public MonitorProbeProvider in models.py — justified

[FIRST-PRINCIPLES-REVIEWED] 11b7499

Description read

[DESCRIPTION-READ] 76fe2306c3bbde218649dfcab4013d2b58b3587955d80741d21a2b658e2af1c3

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review (fork) — ✅ no blocking findings

Reviewed 11b7499aa098f37b0e915cfef02270a2ba71adf0 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 11b7499

@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review (fork) — ✅ no blocking findings

Reviewed 11b7499aa098f37b0e915cfef02270a2ba71adf0 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 11b7499

@the-mann
the-mann force-pushed the feat/monitor-kind-plugins branch from 7de7f79 to 61d08b6 Compare October 26, 2026 19:15
@the-mann

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: duplicate provider Protocol in plugins.py

  • Second spelling of the provider Protocol

fixed in 61d08b6: controller._Provider and plugins.MonitorProbeProvider are both deleted.
One MonitorProbeProvider(MonitorProbe, Protocol) now lives in monitoring/models.py; controller.py and plugins.py import it, and models.py imports neither.
MonitorProbe stays narrower on purpose: shadow.run_shadow_probe probes without the owner-credential gate, so it types to the plural contract without use_owner_credentials.
The Notes subtraction is taken too: unregister_monitor_kind and _drop_plugin_kind are deleted; the autouse fixture restores _PLUGINS and _PLUGIN_KINDS.
Verified: 193 monitor tests pass; the 52 plugin/registry tests pass in 3 shuffled orders and 5 repeats; mypy, isort, flake8 and docs-lint are clean.

@the-mann

Copy link
Copy Markdown
Contributor Author

@chenmingwei23 @kyleseaman could one of you approve the workflow runs for head 61d08b610? All 20 pull_request workflows are waiting at action_required, as GitHub requires for a fork push.

This push fixes the First Principles BLOCK on the previous head (one MonitorProbeProvider in models.py, and the test-only unregister helpers removed). The disposition is in the comment above.

One more thing: on the previous head, Detect changed surface was cancelled when its checkout hit the 15-minute job timeout. If it happens again I can't re-run it from a fork.

Thanks, Marcus

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 26, 2026
@the-mann
the-mann force-pushed the feat/monitor-kind-plugins branch from 61d08b6 to bb3e509 Compare October 26, 2026 21:02
@the-mann

Copy link
Copy Markdown
Contributor Author

Pushed bb3e509ea to fix the one red on 61d08b610: Backend Tests (3.12, 3) failed collecting test/test_pull_request_readiness.py. It imported PULL_REQUEST_MONITOR_KINDS from monitoring/pull_request.py, which this PR stopped importing there. The test now imports it from monitoring/registry.py, where the constant lives, and the body's Backwards compatibility section names that reader. The 96 test files that import kiro_crew.monitoring all pass locally (4456 tests).

On 61d08b610 Design and First Principles passed and Opus raised no findings. @chenmingwei23 @kyleseaman, could one of you approve the workflow runs for bb3e509ea? They need it again because this is a fork push.

Thanks, Marcus

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Oct 26, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Security Scope Review (fork) — 🔴 BLOCK (blocking)

Which legitimate operations does bb3e509eafa4ff243b77c7ba3139443e71669c17 newly refuse? Model proposes, scripts/deny_diff.py decides — updated in place on each push. A script-confirmed regression blocks PR readiness.

no leg reported, and this was not the nothing-to-adjudicate short circuit

No [SCOPE-REVIEWED] marker for this head was produced, so the model's text is withheld: it cannot be shown to describe this revision. See the job logs.

@the-mann

Copy link
Copy Markdown
Contributor Author

@chenmingwei23 @kyleseaman could one of you re-run three fork AI lanes on bb3e509ea? Each one failed in the pipeline itself, not on a finding, and a fork author can't re-run them:

  • GPT 6.1 Review: "verdict not published" (publication: not_published). The check says to re-run the lane to publish it.
  • Opus 5.5 Review: "review incomplete". Nothing was posted for this head; the comment still shows the 61d08b610 review, which had no findings.
  • Security Scope Review: [scope-floor:unsettled], "no leg reported".

The rest of the round so far: Design and First Principles passed on 61d08b610, and this push only changes one test import (see the comment above).

Thanks, Marcus

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 26, 2026
@the-mann

Copy link
Copy Markdown
Contributor Author

Why the three red lanes on bb3e509ea failed: all three hit API rate limit exceeded for installation between 21:56 and 22:01 UTC, so none of them reviewed the code.

  • Fork Opus 5.5 Review (run 40407276560): the review action failed at startup on the rate limit, so nothing was posted.
  • Fork Security Scope Review (run 40407276361): its generate step hit the same limit, so validate and publish had nothing to read.
  • Fork GPT 6.1 Review (run 40407276466): it reached a verdict, then the final check that the PR head is current hit the limit 6 times, so it withheld the comment (PUBLICATION: not_published).

As a stopgap I ran both line-level lanes locally against bb3e509ea, using the prompts extracted from codex-review.yml and claude-review.yml and the same models, GPT 6.1 Sol and Opus 5.5. Neither found anything. These results are only a pointer: PR Readiness reads the CI checks, so a re-run of the three fork lanes, or an override for this head, is still what clears it.

Thanks, Marcus

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 28, 2026
@github-actions github-actions Bot added the readiness: passed Eligible automated validation passed for the current revision label Oct 28, 2026
Comment thread src/kiro_crew/monitoring/targets.py Resolved
@the-mann
the-mann force-pushed the feat/monitor-kind-plugins branch from bb3e509 to 11b7499 Compare October 28, 2026 18:57
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: passed Eligible automated validation passed for the current revision readiness: action required A blocking check or review needs attention labels Oct 28, 2026
@the-mann

Copy link
Copy Markdown
Contributor Author

@tim-finnigan two things on 11b7499aa that need write access: the Code Review run (40749942726) was cancelled after its checkout step hung for 10 minutes, so Automated Rule Check and SAST (Semgrep) show red without having run, and Sensitive Change Review (40749952461) is waiting on approval. A re-run of the first and an approval of the second should be all this push needs.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Oct 28, 2026

@tim-finnigan tim-finnigan left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. I went through the seam and the target-parsing path, and the https check in _contributed_target closes the gap I raised, with host pinning now documented as the registrant's job.

@iamwhatever iamwhatever left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recommendation: REQUEST_CHANGES (issue required) at 11b7499aa098f37b0e915cfef02270a2ba71adf0.

The seam itself looks sound. Contributed kinds live in a separate table, so a built-in cannot be replaced, and they may not declare the shadow path or any objective other than review_ready (registry). Contributed parsers run only after every built-in, fold foreign-input errors into ValueError, and must return an https URL with a host (targets). The owner-credential gate is unchanged. The validator resolves the vocabulary per call, and validate_field is the only reader of FieldSpec.allowed.

Blocker: one deferred follow-up has no linked issue. The description says serving the kind list at runtime is a follow-up, and plugins.py documents the gap: the dashboard shows a probing contributed-kind monitor as blocked, with save and restart disabled. Design Review asks for that to land before an edition ships this seam. #17848 covers the provider seam, not the dashboard. Please open one issue for the runtime kind list and link it in the description.

AI findings judged non-blocking

  • Design Review: warn when a kind is registered after the controller exists. The ordering is documented, and a late registration fails closed as provider_unsupported (controller).
  • First Principles: register_monitor_kind has no caller in this repository. That matches the existing register_source_provider edition seam, and #17848 asks for a pluggable interface (plugins).

All checks are green on this head, tim-finnigan approved it, and there are no change requests. Once the issue is linked, this is ready from my side; a new push would need another review.

— crew-pr-reviewer (r134)

@the-mann

Copy link
Copy Markdown
Contributor Author

@iamwhatever opened #25689 for serving the monitor kind list to the dashboard at runtime, and linked it in the description under the out-of-scope list. No code change, head is still 11b7499aa.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: passed Eligible automated validation passed for the current revision labels Oct 29, 2026
@iamwhatever
iamwhatever merged commit d75dafd into kirodotdev:main Nov 2, 2026
117 of 120 checks passed
@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Nov 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants