Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
251 changes: 232 additions & 19 deletions .github/workflows/triage.yml
Original file line number Diff line number Diff line change
@@ -1,61 +1,122 @@
name: Auto Triage Issues
name: Triage

# Two-stage triage for newly opened issues and pull requests.
#
# 1. Codex classifies the item and applies type/priority/component labels.
# 2. The maintainer on rotation this week is assigned as the human reviewer,
# and is responsible for confirming or correcting that classification.
#
# Stage 2 does not depend on stage 1 succeeding. If Codex is unavailable,
# misconfigured, or wrong, the item is still assigned to a human -- the AI pass
# is an accelerator for triage, never the thing that decides it.
#
# The rotation roster is every member with the `rust-sdk` role in
# modelcontextprotocol/access (src/config/users.ts), sorted for a stable order,
# with the ISO week number selecting whose turn it is. Reading it from that
# public repo instead of the org teams API means no `read:org` token is needed.
# Adding or removing a member there re-partitions future weeks. See docs/CONTRIBUTE.MD ("Triage")
# for the response-time expectations that go with being on rotation.
#
# Both stages also run on `reopened`, so labels get refreshed on an item coming
# back to life. Assignment is skipped when the item already has an assignee --
# an owner from the first triage pass keeps it instead of being swapped for
# whoever is on rotation the week it reopens.
#
# Secrets:
# CODEX_AUTH_JSON Codex credentials; refreshed and written back each run.
# CODEX_AUTH_STORE_TOKEN Fine-grained PAT with `secrets: write`. GITHUB_TOKEN
# has no `secrets` scope and cannot persist the refresh.
#
# Reviewer assignment needs no secret: the roster comes from a public repo and
# the default GITHUB_TOKEN can assign and comment.
#
# Any missing secret degrades classification to a skip, never a red X on the
# item.

on:
issues:
types: [opened, reopened]
pull_request_target:
types: [opened, reopened]

workflow_dispatch:
inputs:
issue_number:
description: "Issue number to triage"
number:
description: "Issue or PR number to triage"
required: true
type: string
dry_run:
description: "Dry-run mode (log the verdict and the pick, change nothing)"
required: false
type: boolean
default: true

permissions: {}

# auth.json carries a rotating refresh token; overlapping runs would clobber it.
concurrency:
group: codex-auth
group: triage-${{ github.event.issue.number || github.event.pull_request.number || inputs.number }}
cancel-in-progress: false

jobs:
preflight:
name: Check prerequisites
if: github.repository == 'modelcontextprotocol/rust-sdk'
if: >-
github.repository == 'modelcontextprotocol/rust-sdk'
&& (
github.event_name == 'workflow_dispatch'
|| (github.event.issue.user.type || github.event.pull_request.user.type) != 'Bot'
)
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
enabled: ${{ steps.check.outputs.enabled }}
issue: ${{ steps.check.outputs.issue }}
number: ${{ steps.check.outputs.number }}
kind: ${{ steps.check.outputs.kind }}
classify: ${{ steps.check.outputs.classify }}
dry_run: ${{ steps.check.outputs.dry_run }}
steps:
- id: check
env:
CODEX_AUTH_JSON: ${{ secrets.CODEX_AUTH_JSON }}
AUTH_STORE_TOKEN: ${{ secrets.CODEX_AUTH_STORE_TOKEN }}
ISSUE: ${{ inputs.issue_number || github.event.issue.number }}
NUMBER: ${{ inputs.number || github.event.issue.number || github.event.pull_request.number }}
# `issue` drives Codex: only issues get classified, since a PR's
# signal is its diff and that is `auto-label-pr.yml`'s job.
KIND: ${{ github.event.pull_request && 'pr' || 'issue' }}
DRY_RUN: ${{ inputs.dry_run || 'false' }}
run: |
set -euo pipefail
echo "issue=$ISSUE" >> "$GITHUB_OUTPUT"
{
echo "number=$NUMBER"
echo "kind=$KIND"
echo "dry_run=$DRY_RUN"
} >> "$GITHUB_OUTPUT"

if [ -n "$CODEX_AUTH_JSON" ] && [ -n "$AUTH_STORE_TOKEN" ]; then
echo "enabled=true" >> "$GITHUB_OUTPUT"
if [ "$KIND" = "issue" ] && [ -n "$CODEX_AUTH_JSON" ] && [ -n "$AUTH_STORE_TOKEN" ]; then
echo "classify=true" >> "$GITHUB_OUTPUT"
else
echo "enabled=false" >> "$GITHUB_OUTPUT"
echo "CODEX_AUTH_JSON or CODEX_AUTH_STORE_TOKEN is not configured; skipping triage." \
>> "$GITHUB_STEP_SUMMARY"
echo "classify=false" >> "$GITHUB_OUTPUT"
if [ "$KIND" = "issue" ]; then
echo "CODEX_AUTH_JSON or CODEX_AUTH_STORE_TOKEN is not configured; skipping AI classification." \
>> "$GITHUB_STEP_SUMMARY"
fi
fi

classify:
name: Classify issue
needs: preflight
if: needs.preflight.outputs.enabled == 'true'
if: needs.preflight.outputs.classify == 'true'
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: read
issues: read
outputs:
classification: ${{ steps.extract.outputs.classification }}
# auth.json carries a rotating refresh token; overlapping runs would clobber
# it, so every classify job across every issue serializes on one group.
concurrency:
group: codex-auth
cancel-in-progress: false
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
Expand Down Expand Up @@ -101,7 +162,7 @@ jobs:
env:
GH_TOKEN: ${{ github.token }}
GH_REPO: ${{ github.repository }}
ISSUE: ${{ needs.preflight.outputs.issue }}
ISSUE: ${{ needs.preflight.outputs.number }}
run: |
set -euo pipefail
gh issue view "$ISSUE" --json number,title,body,labels > triage-issue.json
Expand Down Expand Up @@ -277,7 +338,7 @@ jobs:
apply:
name: Apply labels
needs: [preflight, classify]
if: needs.classify.outputs.classification != ''
if: needs.classify.outputs.classification != '' && needs.preflight.outputs.dry_run != 'true'
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
Expand All @@ -287,7 +348,7 @@ jobs:
env:
GH_TOKEN: ${{ github.token }}
GH_REPO: ${{ github.repository }}
ISSUE: ${{ needs.preflight.outputs.issue }}
ISSUE: ${{ needs.preflight.outputs.number }}
CLASSIFICATION: ${{ needs.classify.outputs.classification }}
run: |
set -euo pipefail
Expand Down Expand Up @@ -358,3 +419,155 @@ jobs:
echo "Removed: ${stale[*]}"
fi
} >> "$GITHUB_STEP_SUMMARY"

assign:
name: Assign human reviewer
needs: [preflight, classify, apply]
# The human backstop is the point of this workflow, so it runs even when
# classification or labeling failed or was skipped -- `always()` plus an
# explicit gate on preflight, which is the only prerequisite it truly has.
if: always() && needs.preflight.result == 'success'
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
issues: write
pull-requests: write
env:
ROTATION_ROLE: rust-sdk
# Passed through env, never interpolated into the script body: the
# classification is model output and `${{ }}` there would be injection.
TARGET_NUMBER: ${{ needs.preflight.outputs.number }}
CLASSIFICATION: ${{ needs.classify.outputs.classification }}
DRY_RUN: ${{ needs.preflight.outputs.dry_run }}
steps:
# The org's membership source of truth is public, so read the roster from
# it rather than the teams API, which would need a `read:org` token.
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
repository: modelcontextprotocol/access
path: access
persist-credentials: false

- id: members
run: |
set -euo pipefail
cat > members.mts <<'EOF'
import { MEMBERS } from './access/src/config/users.ts';
const role = process.env.ROTATION_ROLE!;
console.log(MEMBERS.filter(m => m.memberOf.includes(role)).map(m => m.github).join(','));
EOF
echo "list=$(npx -y tsx members.mts)" >> "$GITHUB_OUTPUT"

- uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
env:
ROSTER: ${{ steps.members.outputs.list }}
with:
script: |
const role = process.env.ROTATION_ROLE;
const number = Number(process.env.TARGET_NUMBER);

// This workflow also runs on `reopened`, so an item may already
// have been triaged. An existing assignee means a human already
// owns it; re-assigning would hand it to whoever is on rotation
// this week and comment a second time. Leave it alone.
const { data: target } = await github.rest.issues.get({
...context.repo,
issue_number: number,
});
const existing = (target.assignees || []).map((a) => a.login);

if (existing.length > 0) {
core.notice(
`#${number} is already assigned to ${existing.join(', ')}; ` +
`leaving the existing owner in place.`,
);
return;
}

// Roster: `rust-sdk` role members from modelcontextprotocol/access,
// sorted so week-to-week order is stable.
const roster = (process.env.ROSTER || '')
.split(',')
.map((login) => login.trim())
.filter(Boolean)
.sort();

if (roster.length === 0) {
core.setFailed(`No members with role '${role}' in modelcontextprotocol/access; nothing to assign.`);
return;
}

// ISO week number since epoch. Thursday-anchored so the rotation
// turns over on Monday 00:00 UTC rather than mid-Sunday.
const weeks = Math.floor((Date.now() / 86400000 + 3) / 7);
const onCall = roster[weeks % roster.length];

const classification = process.env.CLASSIFICATION;

core.info(`Roster (${roster.length}): ${roster.join(', ')}`);
core.info(`Week ${weeks} -> on call: ${onCall}`);
core.info(`Target: #${number}`);

// What the AI pass concluded, so the reviewer knows what to check
// rather than starting from scratch. Absent when Codex was skipped
// or failed, which is exactly when the human matters most.
let verdict = null;
if (classification) {
try {
verdict = JSON.parse(classification);
} catch {
core.warning('Could not parse the classification; assigning without it.');
}
}

// The verdict reaches a comment body, so each field is checked
// against the allowed set rather than trusted from model output.
const TYPES = ['bug', 'enhancement', 'question'];
const PRIORITIES = ['P0', 'P1', 'P2', 'P3'];
const COMPONENT = /^T-[A-Za-z]+$/;

const type = TYPES.includes(verdict?.type) ? verdict.type : null;
const priority = PRIORITIES.includes(verdict?.priority) ? verdict.priority : null;
const components = (verdict?.components || []).filter(
(c) => typeof c === 'string' && COMPONENT.test(c),
);

const summary = type && priority
? `Codex labeled this **${type}** / **${priority}**` +
(components.length ? ` (${components.join(', ')})` : '') +
`. Please confirm or correct those labels.`
: `Automated classification did not run for this one, so it needs a full manual pass.`;

if (process.env.DRY_RUN === 'true') {
core.notice(`Dry run: would assign #${number} to ${onCall}. ${summary}`);
return;
}

// addAssignees silently ignores users without write access, so
// verify the result and fall back to an @-mention comment.
const { data: updated } = await github.rest.issues.addAssignees({
...context.repo,
issue_number: number,
assignees: [onCall],
});
Comment thread
alexhancock marked this conversation as resolved.

const assigned = (updated.assignees || []).some((a) => a.login === onCall);

if (assigned) {
core.info(`Assigned #${number} to ${onCall}`);
} else {
core.warning(
`Could not assign ${onCall} (likely lacks write access); commenting instead.`,
);
}

const mention = assigned ? `@${onCall}` : `@${onCall} (assignment failed)`;
await github.rest.issues.createComment({
...context.repo,
issue_number: number,
body:
`${mention} is on triage rotation this week and is the reviewer for this one.\n\n` +
`${summary}\n\n` +
`<sub>Automated triage. Labels are a first pass and the assigned maintainer has ` +
`the final say.</sub>`,
});
7 changes: 7 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1855,6 +1855,13 @@ See [Oauth_support](docs/OAUTH_SUPPORT.md) for details.

See [docs/CONTRIBUTE.MD](docs/CONTRIBUTE.MD) to get some tips for contributing.

### Triage

New issues and pull requests are labeled by an automated pass and then assigned to
the maintainer on weekly rotation, who owns the final call. See
[docs/CONTRIBUTE.MD](docs/CONTRIBUTE.MD#triage) for the rotation and the response-time
expectations.

### Using Dev Container

If you want to use dev container, see [docs/DEVCONTAINER.md](docs/DEVCONTAINER.md) for instructions on using Dev Container for development.
49 changes: 49 additions & 0 deletions docs/CONTRIBUTE.MD
Original file line number Diff line number Diff line change
Expand Up @@ -31,3 +31,52 @@ If you are using coverage gutters plugin, add these config to let it know lcov o
"coverage-gutters.coverageBaseDir": "target/llvm-cov-target",
}
```

# Triage

New issues and pull requests are triaged by `.github/workflows/triage.yml`, which
runs in two stages.

1. **Automated classification.** Codex reads the issue against the checkout and
applies a type (`bug`/`enhancement`/`question`), a priority (`P0`-`P3`),
component `T-*` labels, and a workflow label. Pull requests skip this stage;
their labels come from the diff via `auto-label-pr.yml`.
2. **Human review.** The maintainer on rotation that week is assigned and
@-mentioned with what the AI concluded. They are responsible for the final
call, and the labels from stage 1 are only a first pass.

Stage 2 runs even when stage 1 is skipped or fails, so every item reaches a
person regardless of whether the automation worked.

## Rotation

The roster is every member with the `rust-sdk` role in
[`modelcontextprotocol/access`](https://github.com/modelcontextprotocol/access)
(`src/config/users.ts`), sorted by login for a stable order, with the ISO week number selecting whose turn it is. The
rotation turns over Monday 00:00 UTC. Adding or removing a member there changes
the roster and re-partitions future weeks, so the person on rotation can shift
when membership changes.

While on rotation, the expectation is:

- New issues are triaged within two business days.
- Critical (`P0`/`P1`) pull requests are resolved within seven days.

Assignment is a starting point, not ownership: reassign or hand off freely.

## Configuration

Classification needs two secrets, and if either is missing that stage is
skipped rather than failing the run:

| Secret | Purpose |
| --- | --- |
| `CODEX_AUTH_JSON` | Codex credentials, refreshed and written back each run. |
| `CODEX_AUTH_STORE_TOKEN` | Fine-grained PAT with `secrets: write`, since `GITHUB_TOKEN` has no `secrets` scope and cannot persist the refresh. |

Reviewer assignment needs no secret. The roster is read from the public
`access` repo, and the default `GITHUB_TOKEN` handles assigning and commenting.

To test either stage without side effects, run the workflow from the Actions tab
with `dry_run` enabled: it logs the verdict and the rotation pick, and changes
nothing.
Loading