Repository navigation
feat(issue-radar): a human removing the needs-human label resumes a skipped issue - #21541
Conversation
ebe266d to
0295576
Compare
|
Thanks for this fix. The branch conflicts with main now. Could you rebase it onto current main so the checks can run? |
a4e4759 to
649261a
Compare
|
Rebased onto current main (20fcbd5). The conflicts came from the recent splits, so the change now lands in the owner modules:
Locally: the issue-radar crew runtime/sweep, GitLab client coverage, entry-types facade and file-line-cap tests pass. |
bolichen97
left a comment
There was a problem hiding this comment.
Thanks for the rebase. The split into the owner modules reads well, and moving the tests into test_issue_radar_crew_runtime_sweep.py resolves the line cap point from my last pass.
Head reviewed: ca0f86f. The fork workflow runs on this head were held for approval; I approved them, so the AI review lanes have not reported yet. I found 0 must-fix and 3 should-fix items, listed inline:
- On GitHub the page-complete test counts rows after pull requests are filtered out, so a short page is judged complete when it is not. An answered issue that is off that page is then treated as closed.
- The brief now says the decision is a maintainer's, but the steps in "When a human answers" still do not say whose comment is the decision, and the wake carries no actor.
- Two docstrings and two spec rows still say only
needs-decisionandneeds-investigationare answerable; the code hands back six scopes.
The branch has 3 commits. PR Hygiene expects one, so please squash before the next push.
| # answer read from it alone waits for someone to look at the dashboard. | ||
| fresh = await asyncio.to_thread(partial(_read_updated_open_issues, key)) | ||
| if fresh is not None: | ||
| open_issues, complete = fresh, len(fresh) < ANSWER_LIST_LIMIT |
There was a problem hiding this comment.
🟡 On GitHub a page that held pull requests is judged complete, so an off-page answer is dropped.
list_updated_open_issues on GitHub fetches 30 rows from the issues endpoint, which also returns pull requests, and then removes the pull requests. On a repository where most recent updates are pull requests, fresh has fewer than 30 rows, so complete is True. A candidate that is not on that short page is then handled as a closed issue: its mark is dropped and no label read is made. If the label removal is not in the top 30 updates at the moment a sweep runs with a live crew, the answer is never seen until someone touches the issue again. No log line records it.
open_issues, complete = fresh, len(fresh) < ANSWER_LIST_LIMITWhy it matters: the goal is that an answer always brings the issue back. This path drops it silently, and it is most likely on busy repositories.
Suggestion: have the GitHub reader report whether the raw page was full before the pull request filter (for example return the raw row count, or (rows, page_full)), and set complete from that. For a candidate missing from a full page, read that one issue directly and count it against MAX_ANSWER_READS_PER_SWEEP. A test with a full raw page that contains pull requests, and an answered candidate past it, would pin this.
| 1. **Re-read the whole thread** — the description and every comment, not just the | ||
| newest one. The answer may be spread over several comments, or may change what | ||
| the issue asks for. | ||
| 2. **Act on what was decided:** |
There was a problem hiding this comment.
🟡 The hand-back steps still do not say whose comment is the decision.
Line 210 now says "The decision is a maintainer's". The steps the crew follows on the wake do not repeat it: step 1 says to re-read every comment, and step 2 says to act "on what was decided". On a public repository anyone can comment. A maintainer can remove the label after another user has posted text that reads like a decision, and the crew has no rule to tell the two apart. The sweep knows who removed the label (answer["actor"]), but the wake reason carries only #N human-answered.
2. **Act on what was decided:**Why it matters: this section turns thread text into the spec for a change. The source of that spec must be explicit where the crew reads it.
Suggestion: put the actor in the wake reason (for example #N human-answered by @login). In step 2, say that the decision is the one given by the person who removed the label or by a repository collaborator, and that other comments are context. If those people did not state a clear decision, treat it as "Still undecided".
| collected is gone from this index; that is the same bound every fold of the crew | ||
| log lives under, and the spec states it. | ||
|
|
||
| HUMAN ANSWERS. A pass scoped ``needs-decision`` or ``needs-investigation`` stops |
There was a problem hiding this comment.
🟡 Docstrings and specs name two answerable scopes; the code hands back six.
ANSWERABLE_SKIP_SCOPES in crew_store_records.py also holds architecture, new-feature, needs-design and breaking-change. This docstring and the comment block at crew_runtime.py:410 still name only needs-decision and needs-investigation. crew_ledger_spec.md:207-210 and the answered row in docs/reference/crew-log/session-types.md go further and say "every other scope stays skipped", which is false for the four scope-gate passes.
HUMAN ANSWERS. A pass scoped ``needs-decision`` or ``needs-investigation`` stopsWhy it matters: the spec is what a reader trusts. A reader who follows it will think a new-feature pass is permanent.
Suggestion: refer to ANSWERABLE_SKIP_SCOPES in the two code comments, and list all six scopes in the ledger spec and the session-types row.
ca0f86f to
c97b6c3
Compare
bolichen97
left a comment
There was a problem hiding this comment.
Thanks for the rebase. Head reviewed: c97b6c3.
I compared this head with ca0f86f: the change is a rebase onto main plus a commit squash, and the content of the PR's files is the same. So the three should-fix points from my previous review still apply. I left them inline on ca0f86f and am not repeating them line by line:
crew_runtime.pyaround line 614: on GitHub the page-complete test counts rows after pull requests are filtered out, so a short page is judged complete and an answered issue that is off that page is treated as closed.crew_brief.md"When a human answers", step 2: it still does not say whose comment is the decision, and the wake reason carries no actor.crew_store_ledger.pydocstring, the comment block increw_runtime.py,crew_ledger_spec.mdand theansweredrow insession-types.mdstill name onlyneeds-decisionandneeds-investigation, whileANSWERABLE_SKIP_SCOPESholds six.
The fork workflow runs on this head were held for approval; I approved them, so the review lanes have not reported yet. The branch has 2 commits and PR Hygiene expects one, so please squash on the next push.
c97b6c3 to
9bac0d8
Compare
|
bolichen97
left a comment
There was a problem hiding this comment.
Maintainer decision on the First Principles points at 527b9e0. Thanks for the rebase; the TTL fix for a queued answer is still in place.
Point 1, accepted. Resuming a skipped issue when a person removes the needs-human label is the natural reading of removing that label, and the maintainer decision posted here on 11-04 at 04:14 covers it. No knob is needed.
Point 2, upheld; requesting changes. Issue Radar is moving out of core (#26967), so new permanent crew-log schema in core is debt we would have to migrate later. This is the same call as #22711 and #28694. Please:
- keep the answered and skip state (
answered,skip.answered_at) in the app's owncrew-answer-checks.json; - leave core's crew-log schema and the radar
state_versionunchanged.
I will approve once that lands and CI is green.
|
/ai-review override first-principles 527b9e0 owner-decision: label-removal hand-back of needs-human passes is the maintainer's recorded decision #21541 (comment) (narrowed to needs-decision and needs-investigation). This head is the same commit rebased onto main with no behavior change; the identical content passed this lane at be5f42b. |
Human judgment recorded@bolichen97 recorded a owner-decision release (class
This decision applies only to this commit. A new push requires a new judgment. |
|
Thanks. Point 2 is done at 452b311 (rebased on main, still one commit):
Point 1 is unchanged, per your acceptance. Local GPT and Opus reviews are clean on this head; CI is running. |
bolichen97
left a comment
There was a problem hiding this comment.
Thanks: the point from my last change request is fixed at 452b311. The answered and skip state now lives in the app's own crew-answer-checks.json (backend/crew_answer_store.py), and core's crew-log schema is untouched. No regex is added.
Requesting changes for one bug that the GPT 6.1 and Opus 5.5 lanes both raise on this head, and that the code shows. _bound only counts rows without superseded_at:
if not (rows[k] or {}).get("superseded_at")
So once a crew re-skips an answered issue (commit_work_progress → crew_answer_store.supersede), that row is left out of ANSWERS_LIMIT and nothing ever removes it. Each issue that goes through answer and re-skip adds one permanent row, and that row can carry a pass.reason of up to RADAR_TEXT_LIMIT characters. read_skips reads the whole file on every sweep, pass write and crew page, and every update rewrites it. evicted_answers never counts any of this, and the spec's "500 in all, oldest evicted" is no longer true.
Please bound superseded rows too. Either count them against ANSWERS_LIMIT, or drop a superseded row once no ledger still holds a pass on its number; in both cases, count the overflow in evicted_answers. Please add a test that runs more than ANSWERS_LIMIT answer-and-re-skip cycles and asserts that the file stays bounded.
Separately, a few checks failed on this head from GitHub API rate limits (Sensitive Change Review, Dependency License Gate, Fork PR Description); the next push will run them again.
|
Fixed at acafd48: the GPT and Opus finding that superseded answers escaped the retention bound.
|
|
/ai-review override first-principles acafd48 owner-decision: label-removal hand-back of needs-human passes is the maintainer's recorded decision #21541 (comment) (narrowed to needs-decision and needs-investigation). The block on this head repeats that same point. The only change since 452b311 is the bound fix we asked for (superseded rows count toward the 500 cap, with evicted_answers and a test); it does not touch the hand-back. |
Human judgment recorded@bolichen97 recorded a owner-decision release (class
This decision applies only to this commit. A new push requires a new judgment. |
bolichen97
left a comment
There was a problem hiding this comment.
Approving at acafd48. The point from my last change request is fixed: _bound now counts superseded rows against ANSWERS_LIMIT, oldest first, raises answer_floor to each evicted time and counts every eviction in evicted_answers. test_superseded_answers_count_toward_the_bound runs answer-and-re-skip past the limit and checks that only the newest rows stay, that the evictions are counted, and that an evicted answer read again is refused. The state stays in the app's own crew-answer-checks.json, and no regex is added.
The First Principles lane passes on this head after the maintainer override. The remaining backend test shards are still running; auto-merge waits for them.
|
Rebased on main (one commit) at 4c7f4a2; the only conflict was the import line in One more fix: with every answer counted against |
bolichen97
left a comment
There was a problem hiding this comment.
The _bound change at 4c7f4a2 looks right to me: a superseded answer whose new pass has a scope a human cannot answer is pinned, pinned rows only come from rows already counted, so the total stays within ANSWERS_LIMIT, and test_a_reskip_to_an_unanswerable_scope_is_pinned_within_the_bound covers the case where every row is pinned.
Requesting changes for one point, which the First Principles lane also raises: the queued-answer path is unreachable in product code.
record_human_answerincrew_store_ledger.pyalways returns{"durable": True, "recorded": recorded}, and its docstring sayssession_idis "accepted for the caller's unchanged signature and not used".- The only reader of that result, in
crew_runtime.py, branches onwritten.get("durable") is False. - So that branch, the
pendingloop,_pending_fresh,pending_atand thewriter/unitresolution through_crew_live_unit(about 70 lines) never change the outcome. Only tests producedurable: False, by patchingrecord_human_answer.
Please either delete that path (the branch, _pending_fresh, pending_at, _crew_live_unit and the unused session_id parameter, plus the tests that only exercise it, and the spec text describing durable: false), or make product code really return durable: False in the case it describes, with a test that reaches it without patching record_human_answer.
|
Fixed at b4ecda1: the First Principles finding "Dead pending-answer branch". Since the answer moved to the app's own file, |
|
/ai-review override first-principles b4ecda1 owner-decision: label-removal hand-back of needs-human passes is the maintainer's recorded decision #21541 (comment) (narrowed to needs-decision and needs-investigation). The block on this head repeats that same point. Since acafd48 the only changes are a rebase on main, pinning re-skips to an unanswerable scope inside the 500 cap, and deleting the unreachable pending-answer branch; none touches the hand-back. |
Human judgment recorded@bolichen97 recorded a owner-decision release (class
This decision applies only to this commit. A new push requires a new judgment. |
|
Done at b4ecda1 (still one commit), following the first option:
Local GPT and Opus reviews are clean on this head, and the issue-radar suites plus ratchets pass. The First Principles block on this head is the same product-shape point, released again by owner decision. |
|
/ai-review override first-principles b4ecda1 owner-decision: label-removal hand-back of needs-human passes is the maintainer recorded decision, narrowed to needs-decision and needs-investigation: #21541 (comment) |
Human judgment recorded@bolichen97 recorded a owner-decision release (class
This decision applies only to this commit. A new push requires a new judgment. |
|
Fixed the GPT 6.1 finding at 9df62ef. @bolichen97 the dead queued-answer path you flagged at 4c7f4a2 was already removed in b4ecda1 (see the comment above). |
|
/ai-review override first-principles 9df62ef owner-decision: label-removal hand-back of needs-human passes is the maintainer recorded decision, narrowed to needs-decision and needs-investigation: #21541 (comment) |
Human judgment recorded@bolichen97 recorded a owner-decision release (class
This decision applies only to this commit. A new push requires a new judgment. |
|
Thanks: the The only red is Backend Tests shard 6 on Linux and Windows: |
…kipped issue A needs-human pass (needs-decision or needs-investigation) entered the repo-wide skip index forever, so a maintainer's answer never reached a crew. The sweep now reads label events for answerable passes; when someone other than the crew's own forge identity removes the needs-human label after the pass, it records the answer in the app's own crew-answer-checks.json (the pass is superseded, not deleted), drops the number from skipped_numbers and wakes the skipping crew with "#N human-answered by @Actor". The core crew-log schema and the radar fold are unchanged. A later pass marks the answer superseded and re-indexes the number. A comment alone still wakes nothing; terminal scopes and the scope gates (architecture, new-feature, needs-design, breaking-change) stay skipped whoever removes the label. The brief tells the crew to re-read the whole thread and to take the decision from the person who removed the label, and the crew read lists recent answers so a wake dropped mid-turn loses nothing. Which issues are open and unlabelled comes from the forge, not the dashboard's issues cache: the existing list_open_issues_first_page (newest update first, on every provider) once per sweep; when that read fails the sweep reads nothing and the next one retries. A candidate off that page is read directly, from the same per-sweep read budget, at most once per retry window, least recently read first. A read that finds no removal is marked against the pass and the issue's updated_at, so any later activity on the issue re-arms the read; a timeline that lags the list is retried after the window. Only the newest addition or removal of the label counts. A removal under the gateway's own login is logged once as a warning. A pass recorded after an answer is the writer's own observation, so it stands whatever the clocks, and the answer's time is the cutoff for later removals, so a consumed answer never wakes again. Part of kirodotdev#21540
An answer hands back the row that stood. Filtering answered rows before comparing precedence promoted a deferred pass that had lost to it, so the issue stayed skipped and no later sweep could clear it.
|
/ai-review override first-principles bc7f36475e2bc73d60f17eec89c4b007f0285084 owner-decision: label-removal hand-back of needs-human passes is the maintainer's recorded decision #21541 (comment) (narrowed to needs-decision and needs-investigation). This is the same product-shape point. This head is 9df62ef rebased onto main with both commits unchanged; the only conflict was a comment in radar_crew.py, kept both. |
|
AI-review override not recorded: |
|
/ai-review override first-principles a410dc5 owner-decision: agent: Maintainer accepted this design (label removal by a human resumes a skipped issue); head only updated onto main since the last override: #21541 (comment) |
Human judgment recorded@bolichen97 recorded a owner-decision release (class
This decision applies only to this commit. A new push requires a new judgment. |
bolichen97
left a comment
There was a problem hiding this comment.
Re-checked at a410dc5. My point from the last review is fixed: the unreachable queued-answer path is gone. No durable result or durable is False branch remains in the diff, so crew_runtime.py no longer carries a pending loop that product code cannot reach. The product-shape question from First Principles is recorded as a maintainer decision on this head. No regex is added in product code. All checks are green, and the branch is current with main. Approving.
Problem / Motivation
Goal: when a crew skips an issue to ask a maintainer, the maintainer's answer brings the issue back to the crew.
Today a crew that hits a question it cannot answer (
needs-decision,needs-investigation, or a scope gate such asnew-feature) posts the question, labels the issuecrew: needs human, and records a skip. The skip goes into the repository-wide skip index permanently, and the brief tells the crew "Do not wait, and do not come back to poll for a reply. Nothing will wake you". When the maintainer answers, nothing happens: every crew on the repo filters the number out before reading the issue, and no API or tool removes a number from the index. The only way forward is for a person to open a new issue that restates the decision, or to do the work by hand.Why it matters
Asking a human is the crew's correct move for exactly the issues that matter most: bugs where the spec is silent, behaviour two specs disagree on, and every new feature. Without a way back, each of these becomes a dead end, and the "a human decides, the crew implements" loop does not exist. On a GitLab test repo, of nine seeded issues, three needed a decision, and none could continue after the answer until we patched the crew.
Not a goal
duplicate,already-fixed,not-reproducibleand the other terminal scopes stay permanent, as designed.What changed (motivation → approach → change)
crew: needs humanissue is never picked up again.crew: needs humanremoved after the skip by a login other than the one crews act through (the gateway'sgh/glabuser, read fresh each sweep). A removal under the crew's login never counts, even when a person made it, because the forge cannot tell the two apart; an operator who runs the gateway under their own login answers from another account, or runs the gateway as a dedicated crew account. Separating the agent and operator identities inside the gateway is A gateway writes to the forge under one identity for agents and for the operator's dashboard actions #22527.crew_runtime.py: for issues skipped with an answerable scope, the sweep reads the label removals from the existing issue timeline (list_issue_timeline). On a qualifying removal it records the answer in the app's owncrew-answer-checks.json(crew_answer_store.py), drops the number fromskipped_numbers, and wakes the skipping crew with[crew wake: #N human-answered by @<actor>].crew_answer_store.py,crew_store_ledger.py: the answer state lives in the app's own file, not the crew log; core's crew-log schema and the radar fold (state_version) are unchanged. The original skip is kept, so the audit trail keeps both. A later skip marks the answer superseded and re-indexes the issue, so a non-answer cannot loop. Answers are capped at 500, keeping re-skips to an unanswerable scope; an evicted answer's removal is never recorded again.list_open_issues_first_page(newest update first, on GitHub, GitLab and Azure DevOps), once per sweep, kept to number, labels and update time. Removing a label movesupdated_at, so an answered issue is on that page. The dashboard cache, refreshed only when a person opens Issue Radar, is not used: when the forge read fails the sweep reads nothing and the next one retries. A candidate off the page is read directly, from the same budget, once an hour at most. A read that finds no removal is marked against the pass and the issue'supdated_at, so any later activity on the issue re-arms it. A timeline with no trace of the label may be a failed fetch, so its mark lapses after an hour. On Azure DevOps the login is read from the repo's own organization (azure_client_repo_reads.py), not the CLI default. A removal under the gateway's own login is logged once as a warning naming the login.crew_routes.py,mcp_core_parts/radar_crew.py) listsrecent_answers, so a crew whose wake was dropped mid-turn still knows who answered.crew_brief.md: a new section "When a human answers". Re-read the whole thread, act on the decision (implement it, close with an explanation, or re-ask and skip again if the thread still decides nothing).needs-decisionandneeds-investigationonly. Scope-gate passes (architecture,new-feature,needs-design,breaking-change) stay skipped whoever removes the label; handing those back needs an RFC first.crew_ledger_spec.md,docs/reference/crew-log/session-types.md.Backwards compatibility
Existing skips are unaffected until someone removes the label. The crew log and its folds are unchanged. No config, schema migration or UI change.
Tests
test/test_issue_radar_crew_runtime_sweep.py:duplicate): stays skipped;updated_atmoves, and the late removal is found;needs-investigationskip is handed back;test_issue_radar_crew_routes_work.py,test_issue_radar_crew_mcp_tools.py);test/test_issue_radar*.py,test/test_crew_log*.py, facade, projection and entry-type tests on currentmain: all pass excepttest_native_skill_projection.py::test_launch_record_resources_match_the_alias_file_kiro_cli_loads, which fails the same way on a cleanmaincheckout here (local environment). flake8 clean on the changed files.Manual verification
Backported to v0.9.0rc1 on a GitLab test stand and run end to end with two crews:
needs-decisionbug: the crew woke after the label was removed, implemented the chosen option, MR merged 11 minutes after the answer;new-featurerequest (CSV export): picked up 52 seconds after the answer. That run predates narrowing the answerable scopes; scope-gate passes now stay skipped.Screenshots / video
N/A: no UI change.
Related Issues
Part of #21540
This covers the
needs-decisionandneeds-investigationpasses. Handing scope-gate passes back to a crew is a product change that needs an RFC first, so #21540 stays open for that part.Pattern harvest
N/A
Checklist