Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 16 additions & 1 deletion .cargo/mutants.toml
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,11 @@ exclude_re = [
# poll, so these differ only when another thread writes the route between that
# read and this check, a window no deterministic test can stage. There the
# original retries on the new copy and the arm prevents a lost wake.
'replace \|\| with && in Fetching::poll_moved$',
'replace \|\| with && in Fetching::(poll_moved|moved)$',
'replace Fetching::moved -> bool with false$',
# The same for `Recover::poll`'s route watch, whose generation it just read; once no
# front moves the copy, the read never continues on the watch.
':\d+:43: replace \|\| with && in Recover::poll$',
':\d+:6: delete match arm Some\(Some\(Err\(err\)\)\) in <impl kio::Task for Fetching>::poll$',
# The reader list's sweep only bounds memory: a dead entry fails every
# `Weak::upgrade`, and `subscribe` sweeps the list again.
Expand All @@ -67,6 +71,17 @@ exclude_re = [
# and nothing is missing.
'replace < with <= in Copy::retains$',
'replace < with <= in Retain<.*>::holds_window$',
# `Placed` only sees ranges `Reader::resolve` placed, with no latest group: each
# start is a time it contains or the oldest, so it never looks a start up, waits on
# one, or counts one back, and an `Oldest` end only reaches it from a copy that
# accepted one, which every copy refuses.
'replace <impl Holdings for Placed>::(back|next_start|complete|resolves) ',
'replace == with != in <impl Holdings for Placed>::back$',
# A copy leaves a non-empty range starting at the oldest group or time 0 unresolved
# only when a cap at 0 empties every range, so a waiting range stops the window too.
'replace && with \|\| in Reader::resolve$',
# `owes_below` is asked only of a key outside the window, where no run starts.
'replace < with <= in Reader::owes_below$',
# Walking again before anything the read parked on fires returns the same
# `Pending`, so the idle shortcut only saves work.
'replace Subscriber::parked -> bool with false$',
Expand Down
44 changes: 34 additions & 10 deletions .github/workflows/mutants.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,19 +12,38 @@ on:
workflow_call:

jobs:
mutants:
name: Mutants (${{ matrix.shard }}/6)
# Sizes the changed shards to the day's diff, so a heavy day adds jobs instead
# of outgrowing one.
plan:
name: Plan the mutants
runs-on: ubuntu-24.04-arm
# A day of heavy route churn adds about an hour of changed mutants to the rotation's half hour.
timeout-minutes: 150
timeout-minutes: 15
permissions:
contents: read
# The changed mutants are those since the previous scheduled Nightly's commit.
actions: read
outputs:
base: ${{ steps.plan.outputs.base }}
matrix: ${{ steps.plan.outputs.matrix }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: ./.github/actions/nix
- id: plan
run: nix develop --command just rs mutants plan
env:
GH_TOKEN: ${{ github.token }}

mutants:
name: Mutants (${{ matrix.mode }} ${{ matrix.shard }}/${{ matrix.shards }})
needs: plan
runs-on: ubuntu-24.04-arm
# A rotation shard takes about an hour, and a changed shard at most as long.
timeout-minutes: 150
strategy:
fail-fast: false
matrix:
shard: [0, 1, 2, 3, 4, 5]
matrix: ${{ fromJSON(needs.plan.outputs.matrix) }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
Expand All @@ -38,16 +57,21 @@ jobs:
secret-access-key: ${{ secrets.MBX_R2_SECRET_ACCESS_KEY }}
write: false
# The checkout is thrown away, so mutating it in place keeps the build
# incremental and cached instead of copying the tree per mutant.
# incremental and cached instead of copying the tree per mutant. The step
# times out before the job, so an overrun still keeps its outcomes.
- name: Mutate
run: nix develop --command just rs mutants nightly ${{ matrix.shard }} 6
timeout-minutes: 140
run: nix develop --command just rs mutants "$MODE" "$SHARD" "$SHARDS"
env:
GH_TOKEN: ${{ github.token }}
MODE: ${{ matrix.mode }}
SHARD: ${{ matrix.shard }}
SHARDS: ${{ matrix.shards }}
MUTANTS_BASE: ${{ needs.plan.outputs.base }}
- name: Keep the outcomes
if: ${{ !cancelled() }}
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7
with:
name: mutants-${{ matrix.shard }}
name: mutants-${{ matrix.mode }}-${{ matrix.shard }}
path: |
target/mutants/*/mutants.out
target/mutants/missed.txt
Expand Down
21 changes: 0 additions & 21 deletions quest/m0/nightly/resume-mutants.md

This file was deleted.

36 changes: 0 additions & 36 deletions quest/m0/nightly/resume-window-mutants.md

This file was deleted.

23 changes: 23 additions & 0 deletions quest/m1/resume-owed-wake.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# [S] A key-order read holding a copy's cursor wakes when the owed group lands

## Goal

A key-order read on a front's logical track that leaves a copy's cursor below a group
outside its window, because the window still owes a group under it
(`Reader::poll_lowest`, the `owes_below` break in `rs/moq-net/src/model/resume.rs`),
wakes when that owed group lands on the copy.

## Plan

Found by [moq-dev/moq#9144](https://github.com/moq-dev/moq/pull/9144) while killing the
`owes_below` mutants: in `a_key_order_read_waits_below_a_group_outside_its_window_only_while_one_is_owed`,
a read polled with a real waiter after the break stays unwoken when group 0 lands on copy
b. The copy's peek returned the reserved group, so nothing registered a wake for a lower
key landing, and the read's idle shortcut then answers `Pending` until something else
fires (another copy's group, a route change, a deadline). With the replaced copy dead it
stalls until the budget runs out.

Reproduce it with that setup and a `Wake` waiter, then register the copy's arrivals
(`track.poll_arrived`, as the drained arm does) before the break, and assert the wake.

Public API: none. Wire: none.
Loading