Repository navigation
feat(source): POST /api/source/issues lists and searches GitHub and GitLab issues - #34742
Conversation
UX Review (Opus 5.5) — ✅ PASSUX-level review of UX-Verdict: PASS Nothing to check. The PR adds an API route and TypeScript types but no rendered control, so the missing blind read leaves no evidence gap. [UX-REVIEWED] 57e238e Description read[DESCRIPTION-READ] bf5f14c7ee0ee33bb268d800fb47f1f3f52a37712e3f415921506abfdf7fcce0 That sha256 names everything this verdict read: the PR description, plus the 3 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: the route only adds things, uses the same owner gate and redaction as the single-issue read, and checks results against the scope it was given. Notes
[DESIGN-REVIEWED] 57e238e Description read[DESCRIPTION-READ] f7e3ab151edf7575829c3c6d7691b37a34125bd058c62bb1dc915bc4c1e4c6bd That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks. Two items add surface that the Goal doesn't need: placeholder Jira/Linear values, and a hand-written markdown stripper used only for excerpts. Notes
What this change shipsInventory (10 items) — 8 justifiedIntent: apps and the dashboard can list and search one repo's or project's GitHub/GitLab issues through the gateway's owner gate and redaction. ADDITION.
[FIRST-PRINCIPLES-REVIEWED] 57e238e Description read[DESCRIPTION-READ] c5b993cb264f61f98824365aee4b98bd0e527ffd5e9193f881f0194dca57dea9 That sha256 names the PR description this verdict read. Recompute it with |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
ec21a1f to
f2efacf
Compare
Code change approver recommendation: A person should decideWhy: the decider thinks the change is probably fine but estimates a 20% chance that this change introduces a defect, above the 12% limit for approving on its own. Adds an owner-only POST /api/source/issues (GitHub/GitLab list and search). Auth matches api_issue_source. The scope is fixed by the gateway and checked again: GitHub refuses repo/org/user/owner and boolean terms in raw and keeps only rows whose repository_url ends with the scoped repo; GitLab reads only projects//issues and raw keys must be on an allowlist. Hosts must be on the existing allowlists, and rows are redacted before and after projection, then byte-capped, before caching. The GPT lane's PEM finding is real only for a key whose header was deliberately written with markdown emphasis AND whose base64 lines are under 40 characters (64-character lines are caught by the bare-secret check). It exposes at most about 280 characters to the owner, and the same body comes back whole and unredacted through the existing single-issue read on main, so this is not a regression. Non-blocking. An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit |
f2efacf to
39e7605
Compare
bolichen97
left a comment
There was a problem hiding this comment.
Requesting changes on 39e7605 for one point. The route itself looks right to me.
Auth is the same as the existing source routes. api_issue_search calls _authorize_owner_request(request, "source.issues.list") first, the same gate as api_issue_source on main: owner match once an owner is configured, the signed local identity otherwise, app tokens and unsigned callers refused. It is registered beside /api/source/issue in routes/chat.py, provider calls go through runner._run_json / _run_provider with the allowlisted host, and the rows are redacted before and after the projection and byte-capped. I found no unauthenticated path. The scope is written by the gateway (repo: first, results filtered by repository_url; GitLab is always the one project's endpoint), and the raw-query checks only keep the query inside that scope.
New regular expressions in product code. This repo keeps regex out of product code. issue_search.py adds import re and about 17 patterns: _CURSOR_RE, _HTML_COMMENT_RE, _FENCE_RE, _IMAGE_RE, _LINK_RE, _HTML_TAG_RE, _LINE_PREFIX_RE, _RULE_RE, _EMPHASIS_RE, _SPACE_RE, the inline re.search(r"\s\S*\Z", ...) in _drop_partial_word, _GITHUB_LOGIN_RE, _GITHUB_REPO_RE, _QUOTED_RE, _GITLAB_SEGMENT_RE, _GITLAB_LOGIN_RE and _HEADER_INT_RE. Some names match patterns elsewhere in the tree, but these are new copies in a new module, not moves. Plain string code covers each of them:
- Numbers (
_CURSOR_RE,_HEADER_INT_RE):s.isascii() and s.isdigit()plus a length check, ands[0] != "0"for the cursor. - Logins, repo names and path segments: a
frozensetof allowed characters,all(c in ALLOWED for c in s), a first-character check and a length bound. - Whitespace (
_SPACE_RE,_drop_partial_word):" ".join(text.split()), and cut at the last whitespace found by a reverse scan orrsplit(None, 1). _QUOTED_RE:_split_termsalready walks the quotes, so the same loop can drop quoted spans.- The markdown strip: one pass over
splitlines(keepends=True)withstartswithchecks for fences, rules, headings, list and quote markers, plusstr.findscans for<!--/-->,,[...](...)and<tag>, andstr.replacefor**,__,~~and backticks. That also removes the backtracking concern the_IMAGE_REcomment works around.
Regex in the tests is fine and out of scope. Once the patterns are gone I will re-read the module and the tests.
|
Thanks for this. The review found nothing that blocks on this head. The PR adds a row to docs/feature-map/README.md, and a change to that document needs a maintainer to sign off before it can merge. A maintainer will post the decision here. To propose a design, follow the request-for-change process in CONTRIBUTING.md (docs/request-for-change/). What the review checked on this head:
Done well: a refused query never echoes the submitted value, and the validation tests check that the provider fake was never called, not only that the status was 400. Optional:
|
… its cut, replace regexes with string code, say issue search refuses app tokens The portability gate reads a literal '/' split/join as filesystem path assembly; a GitLab project path is a URL path, so it uses a named separator as links.py does. The internal-content scan reads whole PEM markers in source lines; the test fixtures now assemble them from their dashes. The excerpt is stripped with its newlines kept, redacted, and only then flattened and cut. Per review, issue_search.py uses plain string code instead of regular expressions for its validators and markdown strip, with the same behaviour, checked against the old patterns kept as oracles in test_source_issue_search_string_parity.py. The feature map and app-kit reference say the endpoint answers the owner's dashboard session and refuses an app token, as /api/source/pull-request does.
39e7605 to
57e238e
Compare
bolichen97
left a comment
There was a problem hiding this comment.
Thanks, my point is fixed. I re-read the whole diff, tests included.
- Regex:
issue_search.pyno longer importsre. The validators use_charset_matchover ASCIIfrozensets, quoted spans are dropped withstr.find, and the markdown strip is plain string scans that stay linear.cache.pykeeps its existingimport re, which this PR does not touch. - Parity:
test_source_issue_search_string_parity.pykeeps the old patterns as test-only oracles and checks every replacement against them on edge cases, a seeded 4,000-item corpus and long adversarial runs. That is a good way to prove the rewrite changed nothing. - Auth and exposure (unchanged since my last read):
api_issue_searchcalls_authorize_owner_request(request, "source.issues.list")first, the same gate asapi_issue_source, and it is registered beside/api/source/issue. The test compares the two routes' denials case by case, and app-token grants stay path-segment bounded. The gateway writes the scope (repo:first, rows filtered byrepository_url; GitLab always reads the one project's endpoint), provider calls pin the allowlisted host, and rows are redacted before and after the strip and byte-capped before caching.
The red checks are cancelled duplicates or an earlier readiness run, each with a newer pass on this head.
Problem / Motivation
Goal: Let apps and the dashboard list and search one repository's or project's GitHub and GitLab issues through the gateway, with the same sign-in, owner gate and redaction as the single-issue read.
Today the gateway can read one issue by URL (
POST /api/source/issue) but has no way to list or search issues. The new app-sdk issue components (IssueList, IssueCard, useIssues) need a query endpoint with a fixed contract, so they can be built against it in parallel.Why it matters
Without a list endpoint each app that shows issues has to bring its own GitHub or GitLab client and credentials, outside the gateway's owner gate, host allowlist and redaction. One gateway route keeps credentials inside
gh/glaband gives every app the same validated, sanitized shape.Not a goal
provider_not_supportedhere and land later, one provider function each.What changed (motivation → approach → change)
Goal → one owner-only route that validates every field before any provider call and fixes the scope itself →
source_providers/issue_search.pyplus a handler and route.POST /api/source/issues(api_issue_searchinhandlers/source_providers.py, registered inroutes/chat.py) uses the same_authorize_owner_request, audit helper and_provider_error_responsemapping asapi_issue_source. Audit operation:source.issues.list. The app-token scope check admits the path only when a manifest declares/api/source/issues, then the owner gate refuses app tokens, as for/api/source/pull-request; the segment-based grant rule keeps/api/source/issueand/api/source/issuesapart (tested, unchanged).owner/repoor a GitLab project path.hostmust be github.com / gitlab.com or on the gateway's existinggithub_hosts/gitlab_hostsallowlist. Logins follow Issue Radar's login rules. Refusals never echo the submitted value.repo:<scope> is:issueitself, quotes free text word by word, and refuses raw terms that namerepo/org/user/owneror useOR/AND/NOT/parentheses; results are also dropped unless theirrepository_urlis the scoped repo, and pull requests are dropped. GitLab: the read is alwaysprojects/<scope>/issues; raw is parsed askey=valuepairs against an allowlist of narrowing filters, and the gateway'sscope=all, paging and ordering come first. Arguments go togh api/glab apias one path argument through the existing runner, so no field becomes a CLI flag.issue_search.pyvalidates, fetches and projects, and never imports the redactor.cache.search_issues(beside the single-issue, checks and contributor reads) runs_redact_provider_dataon the raw provider rows before projection (an excerpt joins lines, and a PEM key is only recognised across them) and again on the projected list (excerpts stripped, newlines kept); only then are excerpts flattened and cut. URLs are built from the validated host and scope, not from providerhtml_url/web_url; avatar URLs go throughprojection._safe_https_url. The excerpt is the body's first ~280 characters, stripped from the whole body, redacted, then cut at a word. Bodies are fetched whole (no jq cut), so both redaction passes see each secret entire. Only the endpoint's own refusals echo text; any other error answersprovider_error; GitLab paging headers must be ASCII digits._PROVIDERSmaps a provider id to onevalidate, onefetchand oneprojectfunction, so Jira and Linear are one entry each.nextCursoris the next page number. GitHub stops at its 1,000-result window (a last partial page reads the window's final 100); GitLab readsX-Next-Page/X-Totalfromglab api --include(total isnullwhen GitLab omits it). Results are cached for 30 seconds;refresh: truebypasses it.IssueSearchRequest,IssueSearchResponse,IssueSummary,IssuePersonandIssueSearchProviderinwebsite/src/types/index.sources.ts(re-exported fromindex.ts), matching the contract; an Issue Search section indocs/app-kit/api-reference.md; an API-only Issue search row indocs/feature-map/README.md.![,[or<is cheap.Backwards compatibility
Compatible: adds a new route and new exported types; no existing route, field or type changes.
Removes nothing: the diff only adds a route, a provider module, types and docs.
Known limits
Left as is in this PR (local Opus review advisories, not upheld as blocking):
limit) fails as a 503provider_errorinstead of returning results; markedshortcut:in_fetch_github_issues. Loweringlimitavoids it.cut_excerptreturns an empty excerpt when the first 281 characters of the stripped body hold no whitespace (for example a body that opens with a very long URL). Cosmetic."x"repo:other/r) skips the scope-qualifier refusal. Results stay scoped: rows outside the repository are dropped by therepository_urlcheck.Tests
test/test_source_issue_search.py(142 cases):/api/source/issue, the owner read is audited assource.issues.list, and the app-token grant is segment-bounded both ways (/api/source/issuevs/api/source/issues).repo:other/repo,-repo:,org:,user:,owner:,ORand parentheses are refused;--hostnameas scope, host or login is refused, and in text or raw it stays a word inside the encodedqpath argument, never a CLI argument; a non-allowlisted self-managed GitLab host is refused; GitLab raw cannot setscope,per_pageor a repeated filter.assignee: 'me'resolved to the GitLab username,assignee: 'me'with no signed-in user failing closed, signed-out GitHub mapping like the single-issue read.IssueSummary, a secret-shaped string in a GitHub or GitLab title or body comes back redacted in place throughPOST /api/source/issues, the cached copy is the redacted one, a PEM key in a GitHub or GitLab body is redacted before the excerpt joins its lines, an indented key whose END line lies past character 4,000 is redacted (red when the body arrives cut), a token split by**, backticks, a link or a multi-line comment in a link is redacted wherever it sits in the body, a token joined by the markdown strip is redacted (removing either redact pass turns at least one red), bodies are fetched without a jq cut, no excerpt keeps a cut token fragment, a non-ASCII-digit GitLab header is ignored and an unexpected providerValueErroris not echoed, excerpt stripping and cut, the link and image strip stopping at the next bracket (red on a pattern that crosses it), cache andrefresh.test/test_source_issue_search_string_parity.py(33 cases) keeps each replaced regex as an oracle and checks the string code agrees with it: every validator, every strip pass and the whole excerpt, on edge and adversarial inputs (Unicode digits, empty, long no-space runs, PEM headers with emphasis,(/)in urls, CRLF, Unicode spaces) plus a seeded generated corpus. Changing a bound, a digit test or the emphasis order turns it red.Edited existing test:
test_link_strip_tries_stop_at_the_next_bracketasserted_LINK_RE/_IMAGE_RE.match(text) is None, and those objects are gone. It now asserts the same property (the strip try at offset 0 fails, so the opener is kept) through_strip_bracket_linkson the same 4 inputs, which are also in the parity corpus. No other test changed.Determinism: 20 runs at
-n0in a shuffled order (seed 20261110) and 10 runs at-n 4 --dist loadgroupbesidetest/test_source_providers_issues.pyandtest/test_source_providers_endpoints.py, all green.Proof: not provable -- other: new feature, no fix to revert; the new tests exercise a route that does not exist without this diff
Manual verification
N/A — unit coverage sufficient: both provider reads are exercised through the real handler and runner with
gh/glaboutput faked at the subprocess boundary.Screenshots / video
Why no screenshot: the only frontend change is new exported TypeScript types; nothing renders differently.
Related Issues
None.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)