Skip to content

feat(merge-retrospective): per-family gate proposals, ABSORBED-BY, escalation (#2097) - #2099

Merged
tvna merged 25 commits into
mainfrom
claude/gitapex-pr-2097-6f7ocd
Sep 27, 2026
Merged

tvna merged 25 commits into
mainfrom
claude/gitapex-pr-2097-6f7ocd

Conversation

@tvna

@tvna tvna commented Sep 24, 2026 •

Copy link
Copy Markdown
Owner

Summary

Executed from the approved Branch Plan in docs/gitapex/plans/2026-09-24-claude-gitapex-pr-2097-6f7ocd.md. merge-retrospective now files gate proposals per family instead of per repair, absorbs repairs a declared generic mechanism can express, and escalates a family that recurs 3 times. ADR 0006 records the decision.

  • A NEW repair, or a verified CLUSTER of NEW repairs, creates one family issue with one ACM row per member.
  • A DUPLICATE-OF #N repair creates a standalone gate-proposal record whose generator-made sweep line reads verdict DUPLICATE-OF #N, then closes it with state_reason: duplicate, duplicate_of: N.
  • ABSORBED-BY <gate-id> applies only to an active ssot gate declaring the new generic_mechanism: true; the repair lands on the single open gate-proposal: extend <gate-id> issue (created through the NEW flow the first time, otherwise recorded as its duplicate). Anything else falls back to NEW.
  • Occurrences = 1 + distinct titles of closed gate-proposal records marked as the family's duplicates. At 3 or more the family gets gate-proposal-escalated, which ranking-the-open-queue scores as Defect.

Facts

Assumptions

  • [inference] GitHub's marked_as_duplicate event is what sets GraphQL Issue.duplicateOf. This session cannot query GraphQL, so the field the CI scan reads was not read directly on test: live proof that issue_write duplicate_of sets duplicateOf (#2097) #2102; the first scheduled drift run is the direct check.
  • [inference] Only accounts with push access can add the gate-proposal label (GitHub drops label changes from others), per the REST issues documentation relied on in the design.
  • The procedure keeps posting as one account; records filed by a different account drop out of the skill's count (the CI scan still counts them).

Risk / blast radius

Rollback

git revert the merge commit.

Verification

Criterion Interpretation Planned ops Proof method Residual risk
Family-level filing (CLUSTER action; per-family issues, per-repair retro entries) A verified CLUSTER of NEW repairs creates one family issue with one ACM row per member. DUPLICATE-OF #N (verified open) creates a standalone record closed with state_reason: duplicate, duplicate_of: N; the entry records Filed as: that record SKILL.md, 4b reference, filing-mechanics, record-format, builder script (verdicts NEW and DUPLICATE-OF #N, duplicate_target), dedup-sweep hook Unit tests (family body, sweep line round-trip, hook/reader agreement) plus eval fixtures (cluster, umbrella match, concurrency): PASS locally CLUSTER verification stays a judgment call; create, close and the retro body are separate writes, covered by the exact-title search and the resume rule
ABSORBED-BY verdict Applies only to an active ssot gate declaring generic_mechanism: true; otherwise NEW. The repair lands on the open gate-proposal: extend <gate-id> issue; a closed one is finished work ssot schema/registry, ssot scan rule, builder, 4b text, record format Unit tests plus eval fixtures (absorbed; undeclared, deprecated and unknown ids fall back): PASS locally The mechanism may still be unable to express the repair; only two gates are declared generic
Escalate at 3 occurrences Count = 1 + distinct titles of closed gate-proposal duplicates of the family (skill: records this account opened; CI: GraphQL duplicateOf); at 3 or more add gate-proposal-escalated family_duplicate_titles, count_family_occurrences, scan escalation check, sync test, ranking rubric note Unit tests (threshold, double filing, forged and other-author records) plus eval fixtures: PASS locally First drift run expected red (see Risk); the label adds priority, not capacity
ADR Record the reversal of the per-repair standalone filing docs/adr/0006-file-gate-proposals-per-family.md Doc present, linked from the 4b reference None identified

Execution log

  • PlanApproved{run_id: 1a8e210f}
  • TaskCompleted{run_id: 1a8e210f, task_id: T1, commit_sha: 55537ac3}
  • TaskCompleted{run_id: 1a8e210f, task_id: T2, commit_sha: 55537ac3}
  • TaskCompleted{run_id: 1a8e210f, task_id: T3, commit_sha: b1bf79a0}
  • TaskCompleted{run_id: 1a8e210f, task_id: T4, commit_sha: d1f6d886}
  • TaskCompleted{run_id: 1a8e210f, task_id: T5, commit_sha: e778d458}
  • TaskCompleted{run_id: 1a8e210f, task_id: T6, commit_sha: 979f62cf}
  • StageDeviated{run_id: 1a8e210f, task_id: null, reason: battle-testing rounds 1-5 forged the recurrence-comment escalation count, action: escalate} -- the owner chose the closed-duplicate design (case E), implemented in 419f46b and hardened through 478bd9a.

Checklist

  • Tests pass locally
  • Docs updated if behavior changed
  • Issue number cited in every commit
  • If this PR adds/modifies a skills/*/SKILL.md, a docs/superpowers/specs/*.md design doc, a security-relevant skill, or a deterministic checker script (skills/*/scripts/*.py, evals/scripts/*.py, .github/scripts/*.py), a ## Skill audit evidence section discloses the required verdicts/waivers (see .github/scripts/gitapex_gate_skill_audit_disclosure.py)
  • If this PR adds a new Kept-edit-log entry to any evals/*/split.md, that entry discloses a Transfer check line (see .github/scripts/gitapex_gate_transfer_check_disclosure.py)
  • If this PR adds or increases a skills/*/SKILL.md's Stop-boundary bullets or named dispatch branches, evals/<skill>/tasks/*.yaml gained at least as many new fixtures (see .github/scripts/gitapex_gate_skill_branch_fixture_coverage.py)

Skill audit evidence

  • battle-testing-a-skill: PASS on round 7 (isolated dispatch, head a36dbbb), after FAIL on rounds 1-6; every round-6 and round-7 finding except the low dimension-11 one (see Risk) is fixed in 51fc8a9 and 478bd9a. Round 6's "no unit tests" finding was a snapshot-scope artifact: the tests live in tests/.
  • evaluating-skill-quality: WELL-FORMED-NOT-MATURE (isolated review, head 51fc8a9); every verdict-blocking gap (dimension 6 axes 6-8, dimension 7's orphaned script) predates this PR; shape check 62/62.
  • adversarial-coverage-mapping: NOT-RUN
  • checker-script-adversarial-review: RAN (Step 8 review-persona pass at 51fc8a9: 2 Blocking and 9 Advisory findings; both Blocking and A2, A3, A5-A8 fixed in 478bd9a)
  • deterministic-gate-quality: RAN (same pass read skills/evaluating-deterministic-gate-quality/references/dimensions.md; its dimension-15 finding, an unresolvable duplicate silently dropped, now fails loudly)
  • defeat-test-disclosure: RAN (test_defeat_* cases for forged sweep lines, free-text injection and text-based inflation, plus hook/reader agreement and an unresolvable-duplicate test)
  • drafting-a-skill-invocation-disclosure: NOT-RUN

Merge gate: independent review

This PR is also subject to the independent-review-pending required
status check (see .github/workflows/independent-review-pending.yml /
.github/scripts/gitapex_gate_independent_review_pending.py). It stays
pending/failing until a ## Independent review verdict section naming
this PR's current head commit is recorded in this body --
drafting-a-pr-to-merge's own Step 8 records it once its independent
review completes. There is nothing for you to do here now: do not
pre-fill this section yourself, and do not remove this note.

Independent review verdict

  • Verdict: CLEAN
  • Verified commit: 73ff510
  • Finding class: none
  • Round: 0
  • Outer layer: GitHub Copilot review requested via request_copilot_review on 2026-09-24 (around 23:29Z); no review_requested event or requested reviewer appeared on the PR, so this layer did not run. Copilot is Comment-only with no pass/fail signal even when it runs; the Claude Code Review App is not confirmed installed here.
  • Inner layer: fresh reviewing-an-artifact dispatch (low effort) against head 73ff510, with partial Step 0 deferral to evaluating-skill-quality (isolated dispatch), evaluating-deterministic-gate-quality and evaluating-context-channel-maturity (both condensed, in-context). The Execution log holds no TaskCompleted entry for this head, so no record was cited.
  • Confirmed, Advisory: the consolidation scan's escalation check will fail on the first scheduled retrospective-gate-drift run (for example gate-proposal-umbrella: skill-audit-disclosure post-push re-check gap #1800, about 7 occurrences, unlabelled); disclosed under Risk and accepted by the owner. A code comment in gitapex_file_gate_proposal.py calls the duplicate closure the relation the count is derived from, while the target is read from the sweep line (wording only).
  • Unconfirmed concerns (speculative, not verified above the confidence bar): a NEW filing with a non-exact visible sweep line plus a fenced exact DUPLICATE-OF line passes the hook and would be read back by duplicate_target (needs a hand-written body from the procedure's own account, later closed as a duplicate); two concurrent runs could both create one gate-proposal: extend <gate-id> issue in a narrow window (the Dedup-sweep count check denies the later create otherwise, and the next run fails closed on two matches); a same-cycle CLUSTER counts as one occurrence (design question under the owner's filing-count decision).
  • Earlier rounds on this PR (previous heads 478bd9a, 3ca7af7, 60129d8, 86b381b): each Blocking finding was fixed in the next commit -- hook/reader sweep-line agreement (3ca7af7, 60129d8, root-caused in 73ff510 by sharing the reader's pattern), the ABSORBED-BY summary in SKILL.md (3ca7af7), Covered by: in the record block (86b381b), and the 4b.3 exception list (73ff510).

Related Issue

Closes #2097

@tvna
tvna deployed to ruleset-verify September 24, 2026 14:44 — with GitHub Actions Active
@coderabbitai

coderabbitai Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Repository: tvna/gitapex/.coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 11980fda-ed52-4ffa-842c-2cd143e06b99


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

…creates (#2097)

The gate-proposal builder gains a multi-row family ACM body, an
append-only recurrence comment keyed on retro and repair index, an
occurrence counter with the escalation threshold and label, and the
per-mechanism absorption title. Only NEW may create an issue now, in
the builder and in the Dedup-sweep hook alike.

Its tests move to tests/, which CI collects; the old location under
skills/merge-retrospective/scripts was never in pytest testpaths.
@codecov

codecov Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.60%. Comparing base (b1a1fb4) to head (73ff510).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2099   +/-   ##
=======================================
  Coverage   99.60%   99.60%           
=======================================
  Files         180      180           
  Lines       31467    31521   +54     
  Branches     3877     3891   +14     
=======================================
+ Hits        31343    31397   +54     
  Misses        124      124           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@tvna
tvna deployed to ruleset-verify September 24, 2026 14:51 — with GitHub Actions Active
The consolidation scan now also checks the reverse direction: a closed
gate-proposal duplicate of an open umbrella must appear on its
Consolidates line (the 1800/2089 shape). It also flags an open family
whose occurrence count reached the escalation threshold without the
gate-proposal-escalated label. A sync test keeps the scan's copies of
the label, threshold and record regexes equal to the skill builder's.
Adds an optional generic_mechanism boolean to ssot gates (schema 1.7.0)
and sets it on defeat-test-mutation-coverage and
detection-logic-property-coverage. The registry scan rejects the
declaration on a gate that is not active, so ABSORBED-BY can only route
repairs onto a mechanism that is live.
@tvna
tvna deployed to ruleset-verify September 24, 2026 14:56 — with GitHub Actions Active
Step 4b gains the ABSORBED-BY verdict, verification rules for
DUPLICATE-OF, ABSORBED-BY and CLUSTER outside the dispatch, and an
action for every verdict: only NEW creates a family issue, while
duplicates and absorbed repairs post an append-only recurrence comment
and escalate the family at three occurrences. The retrospective body
still keeps one entry per repair. ADR 0006 records the decision, and
the ranking rubric scores an escalated family as a defect.
Seven merge-retrospective fixtures cover the new filing actions: a
verified cluster files one family issue; a duplicate posts a recurrence
comment; concurrent runs keep both records; an absorbed repair lands on
its mechanism issue; an undeclared ABSORBED-BY id falls back to NEW; and
escalation fires at the third occurrence but not the second.
@tvna
tvna deployed to ruleset-verify September 24, 2026 15:03 — with GitHub Actions Active
@tvna
tvna deployed to ruleset-verify September 24, 2026 15:12 — with GitHub Actions Active

tvna commented Sep 24, 2026

Copy link
Copy Markdown
Owner Author

eval-gate (Skill eval gate) fails on this PR, most recently at head 5e4ee14 (run 36018445858). This PR's diff is probably not the cause.

  • Fact: the job ends about 20 seconds after the claude CLI first contacts api.anthropic.com. A real run of the merge-retrospective suite is 38 fixtures x 3 trials and would take minutes, so the suite never actually ran. The run step reports this as FAIL (execution error), not as a score below threshold.
  • Fact: the same workflow failed the same way on fix(skills): remove design-history prose from three skills (#1790) #2093 (runs 704 and 706, both within about 30 to 80 seconds). That PR merged anyway. .github/workflows/skill-eval-gate.yml says this job "is not yet a required status check".
  • Speculation: the repository ANTHROPIC_API_KEY secret, or the account behind it, is rejected by the API (bad credential or no credit). I cannot read the secret or the model-CLI stderr from here to confirm this.

No fix exists that I can port: the failure is in CI credentials, not in code. I am not re-running the job, because the #2093 runs show it fails the same way on every head. I will keep watching this PR.


Generated by Claude Code

…2097)

- review-persona call site 6 now lists ABSORBED-BY, so the Step 4b
  reviewer can actually return it.
- Gate ids reject a trailing newline; recurrence comments reject
  retro and repair values their own parser cannot read back.
- Record parsing in the builder and the consolidation scan normalizes
  CRLF, and the scan fetches comments when the count field is missing.
- The Dedup-sweep hook recognizes any verdict text, so a second
  non-NEW sweep line is always ambiguous.
- ALREADY-SHIPPED records Covered by and is exempt from the close
  condition; a resumed run looks for an existing record before
  re-running Step 4b; stale per-repair title wording is updated.
- Two escalation fixtures now assert what they claim.
Battle-testing found that any comment shaped like a recurrence key was
counted toward escalation and could suppress a real post on resume.
Escalation now counts a prior key only when its retrospective,
re-fetched, records that repair as Filed as this family
(count_verified_family_occurrences). A resume skips the post only for
a key authored by the account the run posts as. Three adversarial
fixtures cover a closed DUPLICATE-OF target, a wrong CLUSTER, and a
forged key; CRLF normalization gains direct tests.
@tvna
tvna deployed to ruleset-verify September 24, 2026 15:24 — with GitHub Actions Active
@tvna
tvna deployed to ruleset-verify September 24, 2026 15:28 — with GitHub Actions Active
… by us (#2097)

The battle-test re-run confirmed an attacker could open their own issue
whose body imitates a Filed as line, comment the matching recurrence
key, and have it counted. A key now counts only when the issue it names
is labelled retrospective and was authored by a trusted account: the
one this run posts as, or the repository stub opener, since Step 0
fills a stub in place. A bare string of authors is rejected so
membership never degrades into substring matching. The forged-key
fixture now uses an attacker body that does record the filing.
@tvna
tvna deployed to ruleset-verify September 24, 2026 15:36 — with GitHub Actions Active
)

The third battle-test run confirmed that even a trusted retrospective
can carry a forged Filed as line inside multi-line quoted prose, so
verifying keys against retrospective bodies cannot be made sound.
count_verified_family_occurrences now counts a key only when the
comment carrying it was authored by an account that runs this
procedure (plus this run's own key). A comment's author is recorded by
GitHub and cannot be forged from text; an account missing from the
trusted set undercounts, which fails safe. Adds a fixture for an
ABSORBED-BY id missing from the registry.
@tvna
tvna deployed to ruleset-verify September 24, 2026 16:18 — with GitHub Actions Active
#2097)

The fourth battle-test round found two more ways to inflate the
escalation count: a Consolidates line in a family body its non-member
opener can edit, and the drift scan counting comments from anyone.
Every text-based check has now been forged in some round, so both the
skill counter and the scan copy trust the GitHub author_association
field instead: a recurrence key counts only from a comment by an OWNER,
MEMBER or COLLABORATOR (outside fenced blocks), and a Consolidates line
only when such an account opened the family issue. Such an account
could add the label directly, so this grants nothing new. The
author-blind counter is gone, comment listing is paged to exhaustion,
and a deprecated-gate ABSORBED-BY fixture is added.
@tvna
tvna deployed to ruleset-verify September 24, 2026 17:26 — with GitHub Actions Active
)

Five battle-test rounds forged every text-based recurrence record
(comments, the retrospective they name, quoted prose, an editable
Consolidates line). Per the owner's choice (case E), a recurrence is now
a standalone gate-proposal issue whose generator-made sweep line reads
`verdict DUPLICATE-OF #N`, closed with state_reason duplicate and
duplicate_of N. The escalation count is 1 plus the distinct titles of
closed gate-proposal duplicates of the family: label, closure and sweep
line are all push- or triage-gated, so no free text is read.

- builder: accept NEW or DUPLICATE-OF #N; add duplicate_target; count by
  distinct titles; drop the recurrence comment, association and fence
  helpers
- hook: accept DUPLICATE-OF #N again, keep denying ABSORBED-BY
- scan: count via GraphQL duplicateOf; drop the reverse-direction check
  and comment fetching, since the duplicate relation is now the record
- references: new recording procedure; extend-issue matches need the
  label and an ACM (N3); resume trusts only records this account opened
  (N5)
- ADR 0006, branch plan, six eval fixtures (two renamed) and eval status

Live proof: #2102 closed through MCP issue_write with duplicate_of 2097
reads state_reason duplicate, and #2097 gained a marked_as_duplicate
event.
#2097)

evaluate now accepts NEW and DUPLICATE-OF #N and denies ABSORBED-BY;
the detection-logic-property-coverage gate asks for a hypothesis
property over that regex.
@tvna
tvna deployed to ruleset-verify September 24, 2026 22:54 — with GitHub Actions Active
…iled (#2097)

Battle-test round 6: an issue author can rewrite their own body after a
maintainer labels it and closes it as a duplicate, re-pointing its
sweep line at any family. The skill now counts a closed duplicate only
when it was opened by the account the procedure posts as. The drift
scan reads GraphQL duplicateOf, so it was never exposed. The forged
record fixture gains that edited-body case.
@tvna
tvna deployed to ruleset-verify September 24, 2026 23:04 — with GitHub Actions Active
…#2097)

Battle-test round 7 (overall PASS) found the count filter lived only in
prose and that SKILL.md still described the dedup-sweep hook as
NEW-only. family_duplicate_titles now keeps only closed-as-duplicate,
labelled records opened by this account whose sweep line names the
family, with a unit test, and the hook description names both verdicts.
@tvna
tvna deployed to ruleset-verify September 24, 2026 23:12 — with GitHub Actions Active
…gn (#2097)

- an extend-issue title search counts open issues only, so a repair is
  never filed on a closed mechanism issue (B1)
- Filed as for an absorbed repair names the record unless it created the
  extend issue, matching the eval and the resume rule (B2)
- the hook accepts DUPLICATE-OF only in the shape duplicate_target reads,
  with an agreement test; duplicate_target also reads lone-CR bodies
- the scan fails loudly on an unresolvable or untitled closed duplicate
  instead of dropping it from the count
- plan ACM rows restated for the duplicate design; two eval fixtures
  pin escalation more tightly
@tvna
tvna deployed to ruleset-verify September 24, 2026 23:20 — with GitHub Actions Active
@tvna tvna removed the branch-plan-executing label Sep 24, 2026 — with Claude
@tvna
tvna marked this pull request as ready for review September 24, 2026 23:28
…rops (#2097)

Independent review round 1 (drafting-a-pr-to-merge Step 8):
- F1: the hook allowed indented, spaced, lowercase or fence-adjacent
  DUPLICATE-OF sweep lines that duplicate_target never reads back, so
  the skill would undercount escalation. A DUPLICATE-OF creation now
  needs exactly one line in the generator exact shape, with a test over
  each variant.
- F3: SKILL.md now says ABSORBED-BY creates the mechanism extend issue
  when none is open, matching its references.
@tvna
tvna deployed to ruleset-verify September 24, 2026 23:37 — with GitHub Actions Active
Independent review round 2, finding B: a fenced exact-shape line naming
one family next to a visible, non-exact line naming another passed the
hook, and the skill counted the record toward the fenced target. A
DUPLICATE-OF creation now needs its single exact line outside any code
block and naming the same target as the verified sweep line.
@tvna
tvna deployed to ruleset-verify September 24, 2026 23:44 — with GitHub Actions Active
…#2097)

Independent review round 3, finding F2: the fixed record structure in
SKILL.md gained Absorbed by but not the Covered by line its references
and close condition require for an ALREADY-SHIPPED repair.
@tvna
tvna deployed to ruleset-verify September 24, 2026 23:52 — with GitHub Actions Active
)

Independent review round 4, finding F1: a fenced exact NEW line beside a
DUPLICATE-OF line passed the hook, while duplicate_target saw two lines
and named no target. Rounds 1, 2 and 4 each found a shape where the hook
and the reader disagreed, so the hook now matches exact sweep lines of
any verdict with the reader own pattern, pinned equal by a test.

Finding F4: 4b.3 now names every repair that never reaches a family
issue (ALREADY-SHIPPED, review-worked-as-designed, reclassified).
@tvna
tvna deployed to ruleset-verify September 25, 2026 00:05 — with GitHub Actions Active
@tvna
tvna marked this pull request as draft September 25, 2026 00:26
@tvna
tvna marked this pull request as ready for review September 27, 2026 02:52
@tvna
tvna merged commit 2ec9d48 into main Sep 27, 2026
48 of 50 checks passed
@tvna
tvna deleted the claude/gitapex-pr-2097-6f7ocd branch September 27, 2026 02:57

This branch was successfully deployed

1 active deployment
ruleset-verify — 73ff5108 Deployed Sep 25, 2026 by tvna via ruleset-scan #1355
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(merge-retrospective): file gate proposals per family, add ABSORBED-BY verdict, escalate on recurrence

2 participants