Repository navigation
perf(moq-mux): a program's PMT arriving costs no pass over every waiting program - #9021
Conversation
…ing program Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kixelated
left a comment
There was a problem hiding this comment.
Automated review by review (OpenAI)
Reviewed commit: 74f539f8546e9652c9ebfdd67c88f8cf51ec6cdc
No blocker found. The incremental count is a good fit: PAT changes and expiry rebuild it, and PMTs decrement only entries that were still Wait::Since. The clock never moves backward, so retaining an earlier deadline can trigger an extra scan but cannot miss an expiry. That avoids the per-PMT scan without a second deadline index.
One useful regression test would put two waiting programs on one PMT PID, repeat an empty PMT for the first, then verify that unclaimed-PID counting starts only after the second stops waiting. That would preserve the behavior the temporary recount checked. Static review; I did not rerun tests or benchmarks.
There was a problem hiding this comment.
✅ No new issues found.
Approach: sound. It turns joining into a count that describe keeps up to date as it goes, and it lets deadline stay early, so a PMT arriving costs O(programs in that section) and no longer O(all waiting programs). This fixes the N² cause, not just the #8828 hashing constant.
Reviewed changes I reviewed the single commit, which changes rs/moq-mux/src/container/ts/programs.rs.
joining: bool→usize.time_waitsrecounts it as the number ofWait::Sinceentries.describesubtracts everySincewait it turns intoDescribedor removes when it starts a program. I checked the call sites.self.waitingchanges in only three places:followandexpireboth end intime_waits, anddescribeupdates the count itself.startandretirenever touch it. So the count stays equal to the number ofSinceentries, andself.joining -= joinedcannot underflow.describeno longer re-times. Taking a wait out of the set, or getting a freshPmtPacket::Section, only moves the true deadline later. A deadline left early therefore just triggers oneexpire. That pass finds nothing due:Wait::Sinceis guarded bynow > since + PMT_LIMIT, andPmtPid::overduereturns 0 when not overdue. It then re-times throughtime_waits, so each early deadline costs at most one extra pass.time_waitsloop rewrite. It computes the same minimum as before, and it now also countsSinceentries per program, where before it only set a per-PID flag.
claude-opus-5-5 | 𝕏
|
No actionable comments were generated in the recent review. Walkthrough
Merge Risk:
|
|
quest land watch: the merge queue removed this pull request at 74f539f because its The land fixer works on it in the background, and the watcher re-enqueues it at the back of the queue once it is fixed or judged flaky, or hands it back here with what it needs.
…eo_config_replaces_its_own_rendition
PASS [ 0.031s] ( 727/4853) moq-c test::publish_media_rejects_a_null_config
PASS [ 0.025s] ( 728/4853) moq-c test::raw_track_datagram_publish_consume
PASS [ 0.021s] ( 729/4853) moq-c test::raw_track_skips_are_final
PASS [ 0.053s] ( 730/4853) moq-c test::publish_catalog_roundtrip
PASS [ 0.030s] ( 731/4853) moq-c test::raw_track_and_group_abort_consume_their_handles
PASS [ 0.031s] ( 732/4853) moq-c test::raw_loc_video_uses_the_declared_catalog_container
PASS [ 0.032s] ( 733/4853) moq-c test::raw_track_publish_consume
PASS [ 0.026s] ( 734/4853) moq-c test::raw_track_sparse_groups_and_known_end
PASS [ 0.027s] ( 735/4853) moq-c test::raw_track_update_refuses_a_scaled_time_anchor
PASS [ 0.991s] ( 736/4853) moq-audio encode::capture::tests::an_hls_viewer_starts_an_on_demand_capture
PASS [ 0.029s] ( 737/4853) moq-c test::raw_track_subscription_options_and_update
PASS [ 0.030s] ( 738/4853) moq-c test::raw_track_time_anchor
PASS [ 0.028s] ( 739/4853) moq-c test::route_broadcast_survives_id_reuse
PASS [ 0.023s] ( 740/4853) moq-c test::server_keylog_requires_sslkeylogfile
PASS [ 0.018s] ( 741/4853) moq-c test::session_snapshot_error_preserves_destination
PASS [ 0.023s] ( 742/4853) moq-c test::server_listen_refuses_bad_config
PASS [ 0.029s] ( 743/4853) moq-c test::route_options_carry_cost_and_instance
PASS [ 0.020s] ( 744/4853) moq-c test::session_connect_invalid_url
PASS [ 0.083s] ( 745/4853) moq-c test::parity::video_encoder_without_a_broadcast
PASS [ 0.022s] ( 746/4853) moq-c test::status_callback_refuses_null
PASS [ 0.022s] ( 747/4853) moq-c test::subscription_defaults_are_moq_net_defaults
PASS [ 0.020s] ( 748/4853) moq-c test::tokens_reach_the_dial
PASS [ 0.017s] ( 749/4853) moq-c test::track_demand_reports_an_abort
PASS [ 0.023s] ( 750/4853) moq-c test::timestamp_writes_refuse_out_of_range_and_mismatch
PASS [ 0.040s] ( 751/4853) moq-c test::session_connect_and_close
PASS [ 0.024s] ( 752/4853) moq-c test::track_demand_follows_subscribers
PASS [ 0.025s] ( 753/4853) moq-c test::track_abort_unwanted_keeps_a_late_reader
PASS [ 0.043s] ( 754/4853) moq-c test::server_rejects_a_session
PASS [ 0.023s] ( 755/4853) moq-c test::track_demand_reports_current_state_before_close
PASS [ 0.021s] ( 756/4853) moq-c test::track_info_max_age
PASS [ 0.022s] ( 757/4853) moq-c test::track_dynamic_serves_a_fetch_miss
PASS [ 0.033s] ( 758/4853) moq-c test::timestamps_round_trip_exactly
PASS [ 0.023s] ( 759/4853) moq-c test::track_request_dynamic_survives_accept
PASS [ 0.025s] ( 760/4853) moq-c test::track_request_accepted_after_the_broadcast_closes_is_refused
PASS [ 0.030s] ( 761/4853) moq-c test::track_demand_reports_the_wanted_group
PASS [ 1.152s] ( 762/4853) moq-audio decode::backend::libopus::tests::family_one_decodes_in_canonical_order
PASS [ 0.023s] ( 763/4853) moq-c test::unknown_format
PASS [ 0.022s] ( 764/4853) moq-c test::versions_lists_the_offered_set
PASS [ 0.022s] ( 765/4853) moq-c test::unpublishable_catalog_containers_are_rejected
PASS [ 0.020s] ( 766/4853) moq-c test::video_raw_decode_output_rejected
FAIL [ 0.072s] ( 767/4853) moq-c test::server_accepts_a_session
stdout ───
running 1 test
test test::server_accepts_a_session ... FAILED
failures:
failures:
test::server_accepts_a_session
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 216 filtered out; finished in 0.05s
stderr ───
thread 'test::server_accepts_a_session' (1652965) panicked at rs/moq-c/src/test.rs:8727:5:
assertion `left == right` failed
left: -21
right: 0
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Cancelling due to test failure: 23 tests still running
PASS [ 0.034s] ( 768/4853) moq-c test::track_request_media_carries_hints
PASS [ 0.030s] ( 769/4853) moq-c test::video_publish_consume
PASS [ 0.026s] ( 770/4853) moq-c test::video_raw_publish_rejects_invalid_config
PASS [ 0.026s] ( 771/4853) moq-c test::viewers_split_the_receive_estimate
PASS [ 0.023s] ( 772/4853) moq-c test::zero_with_a_flag_set_is_a_real_value
PASS [ 0.021s] ( 773/4853) moq-c video::follow_tests::a_lowered_ceiling_beats_a_due_raise
PASS [ 0.021s] ( 774/4853) moq-c video::follow_tests::a_moved_lower_retries_the_dropped_target
PASS [ 0.019s] ( 775/4853) moq-c video::follow_tests::a_quiet_grant_retries_a_failed_raise
PASS [ 0.037s] ( 776/4853) moq-c test::zeroed_audio_hint_measures
PASS [ 0.048s] ( 777/4853) moq-c test::video_raw_publish_rejects_frame_size_mismatch
PASS [ 0.999s] ( 778/4853) moq-audio encode::producer::tests::other_kinds_send_their_silence
PASS [ 1.018s] ( 779/4853) moq-audio encode::producer::tests::batch_size_does_not_change_the_published_track
PASS [ 0.075s] ( 780/4853) moq-c test::video_abort_unwanted_keeps_a_late_reader
PASS [ 0.077s] ( 781/4853) moq-c test::video_raw_decode
PASS [ 0.097s] ( 782/4853) moq-c test::video_raw_publish_from_many_threads
PASS [ 0.156s] ( 783/4853) moq-c test::video_raw_decode_rgba
PASS [ 0.158s] ( 784/4853) moq-c test::video_raw_publish_consume
PASS [ 0.172s] ( 785/4853) moq-c test::video_raw_decode_resize
PASS [ 0.683s] ( 786/4853) moq-c test::a_route_credential_rides_and_renews_the_moq_transport_announce
PASS [ 1.048s] ( 787/4853) moq-c test::a_credential_rides_every_reconnect
PASS [ 2.010s] ( 788/4853) moq-audio encode::producer::tests::voice_sends_nothing_while_its_talker_is_silent
PASS [ 3.491s] ( 789/4853) moq-audio decode::consumer::tests::the_decode_path_conforms_to_the_corpus
PASS [ 4.416s] ( 790/4853) moq-audio ring::tests::stretch_plays_onto_the_clock_through_drift_and_target_changes
────────────
Summary [ 5.124s] 790/4853 tests run: 789 passed, 1 failed, 65 skipped
FAIL [ 0.072s] ( 767/4853) moq-c test::server_accepts_a_session
stdout ───
running 1 test
test test::server_accepts_a_session ... FAILED
failures:
failures:
test::server_accepts_a_session
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 216 filtered out; finished in 0.05s
stderr ───
thread 'test::server_accepts_a_session' (1652965) panicked at rs/moq-c/src/test.rs:8727:5:
assertion `left == right` failed
left: -21
right: 0
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: 4063/4853 tests were not run due to test failure (run with --no-fail-fast to run all tests, or run with --max-fail)
error: test run failed
mbx[cache]: object cache: 4 hits, 59 misses, 20 bypassed; 0 B downloaded, 0 B uploaded, 2.4 GiB stored locally
mbx[cache]: 1m 38s session; 21.11s estimated compiler time avoided (summed across compilations)
mbx[cache]: bypass reasons: 20 unsupported-crate-type
mbx[cache]: Cargo artifact reuse and CI cache archive transfers are not included above
mbx[savings]: found 1.4 TiB logical of target/ whose checkouts left long ago. the estate has been settled.
error: recipe `test` failed on line 366 with exit code 100
error: recipe `test-changed` failed on line 104 with exit code 100
##[endgroup]
##[error]ci: test rs failed
ci: wall and CPU seconds per module
71 104 check rs
98 444 test rs (failed)
4 37 check biome
8 8 check markdown
19 25 check gh
33 37 check carla_bridge
1 0 check drill
16 16 check shell
0 0 check large_files
1 0 check text
1 1 check coverage_tables
0 0 check toml
3 2 check nix
0 0 check justfile
169 681 whole run
error: recipe `ci` failed on line 120 with exit code 100
##[error]Process completed with exit code 100.
##[group]Run actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9
with:
name: cargo-timings
path: target/cargo-timings/cargo-timing-*.html
if-no-files-found: ignore
retention-days: 14
compression-level: 6
overwrite: false
include-hidden-files: false
archive: true
env:
BASE: d99aada62b5d4cc8d59046baf63568649de0947c
CHECK_FAIL_FAST: 1
NEXTEST_TEST_THREADS: 24
MBX_CACHE_LINKS: 1
MBX_REMOTE_URL:
##[endgroup]
With the provided path, there will be 4 files uploaded
Artifact name is valid!
Root directory input is valid!
Uploading artifact: cargo-timings.zip
Beginning upload of artifact content to blob storage
Uploaded bytes 197652
Finished uploading artifact content to blob storage!
SHA256 digest of uploaded artifact is 00c248c358789b0ee4a7a05874884aa1214c3d80bd7e7fe50dbdd50af7dfd9de
Finalizing artifact upload
Artifact cargo-timings successfully finalized. Artifact ID 14366199637
Artifact cargo-timings has been successfully uploaded! Final size is 197652 bytes. Artifact ID is 14366199637
Artifact download URL: https://github.com/moq-dev/moq/actions/runs/44170288115/artifacts/14366199637
Post job cleanup.
##[start-action display=Run jdx/mr-boxington-action@88acf2bb50552c7c508cbac13e5f8c576b95b1de;id=__007ddaab-1a3d-4f8e-989a-5bbf8a7521c9.mbx]
##[end-action id=__007ddaab-1a3d-4f8e-989a-5bbf8a7521c9.mbx;outcome=skipped;conclusion=skipped;duration_ms=0]
Post job cleanup.
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git version
git version 2.56.0
Temporarily overriding HOME='/var/cache/github-runner-work/_temp/2f7edf01-1c03-4a41-a7aa-a0359d23f25a' before making global git config changes
Adding repository directory to the temporary git global config as a safe directory
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git config --global --add safe.directory /var/cache/github-runner-work/moq/moq
Removing SSH command configuration
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git config --local --name-only --get-regexp core\.sshCommand
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
Removing HTTP extra header
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
Removing includeIf entries pointing to credentials config files
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git config --local --name-only --get-regexp ^includeIf\.gitdir:
[command]/nix/store/i2m3w4ly2w6grrmk1j59rb7zp09bssia-git-2.56.0/bin/git submodule foreach --recursive git config --local --show-origin --name-only --get-regexp remote.origin.url
A job completed hook has been configured by the self-hosted runner administrator
##[group]Run '/nix/store/qrn6ds64bzvwbm5nwf245ifz2c6jb9xw-ci-cache-gc.sh/bin/ci-cache-gc.sh'
shell: /nix/store/lfm00fnq66xfgbn099vqckabyl5jllz6-bash-interactive-5.3p20/bin/bash --noprofile --norc -e -o pipefail {0}
##[endgroup]
evicted 215 objects and 35 action results (1.5 GiB logical); 49.9 GiB logical remain
dropped 7 stale checkout records
dropped 20 session event streams
evicted 0 objects and 0 action results (0 B logical); 49.9 GiB logical remain
dropped 1 stale checkout records
Cleaning up orphan processes
(written by quest land watch) |
|
codex, gpt-6.1-sol, medium effort land fixer note:
(written by codex, gpt-6.1-sol, medium effort) |
|
quest land watch: re-enqueued this pull request at 74f539f, at the back of the merge queue, unchanged, since the land fixer judged the failure flaky: (written by quest land watch) |

Problem
The 11-16 nightly (run 43909101837) flagged
ts_programs/{1,4,16}+1000/{1,4}mbpsas regressions of +58% to +197%. Each case is a multiplex whose PAT names 1000 programs that list no stream. Bisecting locally:ts_programs/1+1000/1mbpsts_programs/16+1000/1mbpsEvery program whose PMT arrives (described, or started) re-ran
time_waits, a pass over every waiting program. With N programs joining, that is N² work. moq-dev/moq#8828 made each pass look up every PMT PID's hash entry too, which roughly tripled it. In a profile,time_waitsand itsPidhashing took about 60% of the run.Approach
A PMT arriving no longer re-times every wait:
describeleaves the deadline as it is. If that deadline passes early,expirefinds nothing due and re-times.joiningbecomes a count of programs not yet late, not a flag.describedecrements it for each program that stops waiting before it is late. A PAT or an expiry still recounts it in full, as before.Impact
Validation
cargo bench -p moq-mux --bench ts_programson this machine, main against this branch:ts_programs/1+1000/1mbpsts_programs/16+1000/1mbpsts_programs/1/1mbpsts_programs/16/1mbpsThe cases without idle programs stay within run-to-run noise.
Second check: a temporary assertion after every PMT compared the incremental count with a full recount of
Wait::Sinceentries. It held through all 2097moq-muxlib tests and everyts_programsbench case (--test). It was removed before commit.just checkpasses.The other moq-mux cases that nightly flagged are not this PR:
ts_si_renewal_pcr/*(+24% to +341%): since moq-dev/moq#8829, the catalog goes out with the first media instead of waiting two seconds for a mux rate. That put the first SI advertise inside every timed iteration. moq-dev/moq#8970 already moved it into setup, and main measures 1.5 µs against 4.7 µs at 505a86f.ts_stalled/{1,4}/256packets(+13%): this also bisects to feat(moq-mux): only a padded TS export waits for the mux rate #8829 (139 µs before, 157 µs after). The profile shows the catalog's deflate, which is the first catalog now going out within the one-second input. That is the intended cost of that change.Caused by
time_waitspass hash every waiting PMT PID, about 3x the cost of a pass that every PMT arrival already ran (found by check: nightly paired bench). Running that pass on every arrival is older than fix(moq-mux): a late PMT ends only its own program #8828.Alternatives
time_waitsalone, for example by storing each PID's waits beside itsPmtPid. That restores the old constant but keeps the N² startup.BTreeSetof due times) kept exact on every change. It costs more code for a deadline that only needs to be no later than the truth.Follow-ups
None.
Models
Size: S
Implementer: Claude Code, claude-opus-5-5, unknown effort
(Written by Claude Opus 5.5)
🤖 Generated with Claude Code
Review
Rewritten by
quest reviewafter each round.Head: 74f539f
Base: origin/main at 2dcb5f8
Risk: medium,
the base has no RISK.md to name what is always high or customer-facing, so every change is mediumClassified: medium by default
Reviewer: Codex, gpt-6.1-sol, medium effort
Session:
01a1eada-460d-7852-9139-0d994f5f51e2, round 1Score: 9/10,
The incremental joining count preserves the waiting-state invariant, and retaining an early deadline safely defers recalculation to expiry; source review found no actionable defects, with builds and tests intentionally not run.Rounds: 1; first round 0; findings 0: fixed 0, refuted 0, open 0
Findings: none