[neekolas]

Spec Compliance should fail for material contract defects: an unrecorded requirement change, a wrong implementation of a new requirement, or a regression in an existing requirement. The current guidance also makes loss of test proof a blocker and lets the review duplicate general security findings outside the product specs.

This change defines those three blocker categories, requires a concrete before/after failure and affected party, and keeps evidence bookkeeping, missing tests, pending CI, and unchanged limitations advisory. It checks implementations of explicitly proposed requirements without treating every draft as approved. Approved base requirements remain the baseline; proposed amendments and new waivers still need owner review.

Audit findings

Inspected the 40 most recently updated PRs targeting self-hosted (October 7–13, 2026), their latest-head Spec Compliance results, and selected earlier revisions. The latest heads had 38 passes and 2 failures, so the sample does not support “most fail” at the current heads.

• #4468 failed for CI cache credentials, and #4480 failed for paths in CI cache reports. Both reports explicitly found no applicable product requirement; Security independently failed on the same issues. Those remain Security findings rather than duplicate Spec Compliance blockers.
• Earlier #4504 reviews caught rejection of stored remote attachments with scheme: "https". The repair restores legacy decoding and adds a wire fixture. This remains blocking under the new guidance even though CTYPE is draft.
• #4519 extends test waits without changing assertions; #4458 moves tests while retaining equivalent proofs. Both should continue to pass.
• #4499 proposes spec additions with gap waivers. Spec-only future work and pending owner review should not be mistaken for a runtime regression.

Macroscope configuration

Keep input: full_diff and conclusion: failure. Macroscope documents that this permits a failure for a finding and success for a clean review. Its configuration reference also says failure is unsupported with incremental input. The instruction guidance recommends explicit scope, evidence, output, and success rules.

Only the spec-review prompt changes. Security, Correctness, Code Quality, and Approvability protection retain their current settings, including human review for .macroscope/**.

Validation

• Markdownlint passed with the repository recipe's disabled rules (MD001, MD013), scoped to the changed file.
• Whitespace checked; YAML front matter is unchanged.
• Manually walked through the historical cases above and the three requested blocker categories. This is a policy review, not a replay of the model.
• The Nix recipe could not run because the local shell hook could not resolve Nix; a direct wrapper attempt was denied access to its daemon. No runtime code changed.

Note

Restrict Macroscope spec compliance review to material contract defects

Replaces the full-PR compliance audit in the spec-compliance review guidance with a focused policy in spec-compliance.md. The agent now blocks only for unrecorded material contract changes, incorrect implementations of new requirements, and regressions in existing requirements.

• Approved base-branch requirements are the binding baseline; proposed spec amendments are reviewed separately from implementation compliance
• Known limitations and waivers must be compared between base and head
• Documentation, tooling, CI, refactors, and test-only changes are excluded unless they affect a product contract
• Evidence rules cover reachable call paths, relevant assertions, removed or relaxed tests, and unavailable CI results
• Adds a structured blocker report format and an explicit successful-review message
• Behavioral Change: the agent no longer reports non-material findings, so reviews that previously flagged documentation or tooling issues now report success when no qualifying defect exists

^{Macroscope summarized 81a8392.}

[macroscopeapp]

Approvability

Verdict: Skipped

Macroscope did not run approvability analysis for this PR. This PR modifies 1 file matched by one or more of your neverApprove patterns: .macroscope/**, docs/self-hosted/agent-context.md, docs/specs/**, .github/CODEOWNERS, apps/backend/migrations/**, crates/xmtp_db/migrations/**. Changes to these paths are never approved automatically.

^{You can add or adjust custom eligibility rules. Learn more.}