Skip to content

A reopened issue re-enters intake as a new generation (US-47.10) - #1010

Merged
mkreyman merged 2 commits into
masterfrom
feature/us-47.10
Oct 19, 2026
Merged

mkreyman merged 2 commits into
masterfrom
feature/us-47.10

Conversation

@mkreyman

@mkreyman mkreyman commented Oct 19, 2026 •

Copy link
Copy Markdown
Owner

US-47.10, part of #986 (item 3). chg closes a chg-loop issue only after the failure stops recurring, and reopens it if the failure comes back. Until now loopctl ignored that reopen once the issue's story had advanced: the triage drain skips any record with a stage row, and only one story or closure may exist per record.

Generations

  • intake_records.generation is a new column, NOT NULL, default 1, with a CHECK that it is ≥ 1. The unique key becomes (tenant_id, source_id, issue_number, generation). Migration 20261019180000; down refuses while any issue has a generation 2.
  • Every delivery applies to the issue's newest generation.
  • A reopened delivery starts generation N+1 when the newest record's loop has ended: a closure row exists, or its story has reached merged, deployed, verified or done (Intake.generation_end_stages/0). The new record waits at pending_triage, and the drain picks it up like any new issue.
  • escalated and failed without a closure are not "ended": a human holds the first, and the second shipped nothing.
  • Only a reopen that would actually be applied bumps. It must be newer than the newest record, or the same second with different content; a same-second bump is flagged order_ambiguous.
  • An older delivery that arrives after the bump is dropped as older. A redelivery is stopped by the existing delivery dedup.
  • Two concurrent reopens both lock generation N; the new generation's insert is ON CONFLICT DO NOTHING, so only one bump happens. That race is tested in a sync module, since its subject is two committed transactions, and it is registered in the hygiene test.
  • The new generation inherits the old one's escalation. A reopen must not put a flagged issue back into triage.
  • Earlier generations, their stories and their closures are not touched.

Issue closer

  • New PullRequestSource.remove_label/3 sends DELETE …/labels/{name}, with the name encoded as one path segment. A 404 counts as success, and errors are classified the way every other write is.
  • Before adding its own label, the closer removes any other loopctl:resolution-* label, so a generation-2 close ends with exactly one. It removes first and adds second: a crash in between leaves no label, which the retry fixes, never two.
  • A closed issue that carries only our label counts as our close only if this closure recorded labelled_at. Otherwise it is abandoned: closed_by_other, which covers a generation-1 label left on an issue that someone else then closed.
  • A label already on the open issue is adopted and stamped labelled_at, so our own crash window still reads as ours.
  • The existing test "an issue already closed carrying our label is recorded closed…" now sets labelled_at in its fixture. That is the only state the real crash window can be in.

Deploy note: deliveries can be dropped during the rollout (CHANGELOG)

The migration swaps the unique index. Until each old machine is replaced, its intake insert names a conflict target that no longer exists, so it answers issues deliveries with 500, and GitHub does not retry them. loopctl rolls across 2 machines, so the window is about a minute. After the deploy, I redeliver every non-2xx issues delivery on each enrolled repo's hook (POST /repos/{o}/{r}/hooks/{id}/deliveries/{delivery_id}/attempts). The CHANGELOG names this step. An expand/contract split would avoid the gap, but generations would not work until the second release; I chose the single release.

Tests

  • intake_generation_test.exs covers:
    • every end stage bumps, and a closure alone bumps;
    • a story still in flight, an escalated or failed story, an issue with no story, and an edited delivery start nothing;
    • redelivery, an older delivery after the bump, and a stale reopen;
    • provenance;
    • escalation inheritance;
    • tenant isolation.
  • intake_generation_concurrency_test.exs covers two concurrent reopens.
  • issue_closer_test.exs, US-47.10 block:
    • different verdict, same verdict;
    • a transient removal failure;
    • a crash between the removal and the add;
    • a stale label closed by somebody else;
    • an adopted label.
  • github_pull_request_source_test.exs covers remove_label/3.
  • Mutation-checked:
    • removing the bump fails 11 tests;
    • losing the race onto the stale record fails the concurrency test;
    • no stale-reopen guard, no inheritance, no stale-label removal, a silent removal failure, an unmarked label counted as ours, and no adoption each fail their tests.

Defaults Mark may want to overrule

  1. The stages that end the loop are merged, deployed, verified, done, or a closure row. Set in Intake.generation_end_stages/0.
  2. A new generation inherits escalation. Change it in Intake's generation bump.
  3. Single release with a manual redelivery, rather than an expand/contract split.
  4. First-generation behaviour change: a maintainer who adds our exact label and closes the issue before the closer runs is now recorded as closed_by_other, not closed. Nothing is written to GitHub either way.

Review round (one, /lean-review high): all 6 findings fixed

  1. intake.ex newest_locked/2 chose the newest generation from its snapshot, before it waited on the lock. A delivery queued behind a bump therefore landed on generation N. It now re-reads after taking the lock and moves to the newer generation. With that in place the reopen-only :lost_race branch could never run, so it is deleted.
  2. Generation 1's closure, written after the bump, could close the reopened issue. The closer now abandons it as superseded_by_generation before any forge call (IssueClosures.superseded?/1).
  3. An inherited escalation wrote no audit entry. It now writes intake_escalated for the new record, with inherited_from_generation.
  4. The race test covered reopen-vs-reopen only. It now also covers an edit racing the bump. Mutation-checked: removing the re-read fails both races.
  5. The moduledoc arity is now loop_ended?/1.
  6. delete/2 duplicated write/4. It is now a one-line call to write/4, which takes an optional body.

Remaining window, by design: a reopen whose webhook commits after the closer's supersede check but before its close call. The reopen's own webhook latency already opens a window that no check here can close, so the closer does not re-check before close_issue.

chg closes a chg-loop issue only after its fingerprint stays silent, and
reopens it when the problem recurs. loopctl ignored that reopen once the
issue's story had advanced. An issue now has one intake record per
generation: a reopen whose newest record's loop has ended (a closure row,
or a story at merged, deployed, verified or done) starts generation N+1,
which triage picks up like a new issue. Earlier generations, their story
and closure stay untouched as the first fix's provenance.

The issue closer removes another generation's loopctl:resolution-* label
before adding its own (new PullRequestSource.remove_label/3), and counts a
closed issue carrying its label as its own close only when this closure
recorded labelled_at.

Part of #986 (item 3).
@mkreyman

Copy link
Copy Markdown
Owner Author

Review round 1 (code) counted at 40dc18b383b5 via Skill lean-review. The review-round counter's merge gate reads this marker on any machine.

…eded closures

- newest_locked/2 re-reads for a newer generation after taking the lock
  and moves to it, so a delivery queued behind a bump lands on the new
  generation; the reopen-only lost-race branch is gone with it.
- The issue closer abandons a closure whose record is no longer the
  issue's newest generation (superseded_by_generation) before any forge
  call, so generation 1's late closure never closes the recurrence.
- An inherited escalation writes its own intake_escalated audit entry.
- The race test covers an edit racing the bump; write/4 serves DELETE.
@mkreyman
mkreyman enabled auto-merge (squash) October 19, 2026 13:41
@mkreyman
mkreyman merged commit d6a6376 into master Oct 19, 2026
17 checks passed
@mkreyman
mkreyman deleted the feature/us-47.10 branch October 19, 2026 13:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant