Skip to content

ci(obs): check OBS sources concurrently - #6684

Merged
kixelated merged 4 commits into
mainfrom
quest/m0/ci/obs-parallel
Oct 25, 2026
Merged

kixelated merged 4 commits into
mainfrom
quest/m0/ci/obs-parallel

Conversation

@kixelated

@kixelated kixelated commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

just obs compile now syntax-checks OBS's 20 source and test translation units concurrently, up to MOQ_CHECK_JOBS (default 3) at once, after header preparation finishes. Completes the quest/m0/ci/obs-parallel quest.

Is OBS still on the critical path?

Yes. I checked this first, since moq-dev/moq#6617 landed module lanes.

  • Full run (the moq-dev/moq#6617 queue entry, run 40118341261): the serial ffi lane took 515s (carla 4, py 265, kt 85, go 2, cpp 123, obs_compile 36), then dart ran alone for 109s. Wall time was 624s. obs_compile is the last module in that lane, and dart waits for it, so all 36s were on the critical path. Header prep was a no-op there (Cargo "Finished in 0.36s", every install "Up-to-date"), so nearly all of the 36s was compiling.
  • Entries touching cpp/moq/ or rs/moq-ffi/ run cpp and obs_compile back to back in the ffi lane, so the same holds.
  • Entries that only touch cpp/obs/ or sh/obs/: before this change, locally obs_compile (about 21s) was the longest module. Next were gh test (about 25s in CI) and shell (12s). Now obs_compile takes 9s and gh test (26s) finishes last.

Only one queue entry has run obs_compile since module lanes landed, so the per-scenario numbers above combine it with local runs.

Measurements

Local runs, 24 cores with load average around 10 from other work, same inputs, warm build:

round 1 round 2
header prep (includes.sh, warm) 0.88s 0.81s
compile.sh, MOQ_CHECK_JOBS=1 (serial) 20.7s 19.9s
compile.sh, MOQ_CHECK_JOBS=3 (default) 8.1s 9.1s

Checking each translation unit serially takes 0.1s to 2.0s. The slowest are moq-dock.cpp, moq-advanced-dialog.cpp, and obs-moq.cpp, which pull in Qt or all of libobs. A cold prep (building moq-ffi and the C++ package) took 48s once and isn't affected by this change.

Verification

  • Two broken sources (src/moq-service.cpp, test/moq-rate-test.cpp): both errors were reported whole, all 20 sources were checked (counted through a logging CXX wrapper), and the command exited 1.
  • Cancellation, using a CXX stub that forks a long-lived child (like GCC's driver forking cc1plus). TERM to compile.sh alone, TERM to its process group (how sh/dispatch.sh cancels a lane), and INT to its group (Ctrl-C) exited 143/143/130, with 3 children before and 0 after in each case.
  • Second pass, at MOQ_CHECK_JOBS=8 with a stub that fails two sources and SIGKILLs a third (an OOM kill): all 20 checked once each, every failure named, exit 1. HUP and INT to compile.sh alone also exited 129/130 with no compilers left and the log directory removed; a success leaves no log directory either.
  • Killed compilers on bash 5.2 (from a Codex review): bash 5.2 deletes a job that a signal ends from its table once it notifies, during the next cat or wait -n, so wait -n reported no such job and the gate aborted on pid: unbound variable, losing that source's diagnostics. A stub that SIGKILLs every third source hit it in 20 of 25 runs on bash 5.2.15. With each compiler run as a child of a subshell job, it passed 25 of 25 on 5.2.15 and 10 of 10 on 5.3.20 (the dev shell's), every killed source named. Normal exits stay in the table in script mode (up to CHILD_MAX), so they were never affected.
  • MOQ_CHECK_JOBS=0 fails with the same error message sh/dispatch.sh gives.
  • nix develop --command just check passed (obs_compile, obs, markdown, gh, quest, shell, large_files, toml, nix, justfile).

Decisions made without asking

  • Bound: the TU fanout reads MOQ_CHECK_JOBS with the same default of 3 and the same validation as sh/dispatch.sh, rather than nproc or a new variable, so there's one knob. The "local admission share" the quest mentions doesn't exist yet. quest/m0/ci/local-gate.md owns it, and I noted this second consumer there.
  • Process groups: each check runs in its own process group (set -m only around the launch), and the EXIT trap sends TERM to each group, then waits. GCC's driver doesn't pass a signal on to cc1plus, so killing only the driver pid would leave the compiler running.
  • Subshell per check: each compiler runs inside a ( ...; exit ) job rather than as the job itself, so the job exits normally even when the compiler is killed, which keeps its status collectable by wait -n on bash 5.2. The trailing exit stops bash exec'ing the compiler in the subshell's place. The cost is one extra bash fork per source.
  • Failures don't stop the rest: a failing source doesn't cancel the others, the same as the serial loop. Each source's output is buffered and printed whole when it finishes, in completion order, so concurrent diagnostics never interleave. A failing source also gets an error: <source> failed with status N line, since a compiler killed outright (more likely with three Qt checks at once) prints nothing naming its source.
  • sh/dispatch.sh untouched: I didn't add a cross-reference comment there. Any change to that file makes the dispatcher run every module, which would turn this PR's checks into a full 10-minute run.

Impact

  • Public API: none.
  • Wire: none.
  • Developer-facing: just obs compile (and the obs_compile check module) now runs up to MOQ_CHECK_JOBS compilers at once, and rejects a value that isn't a positive integer. Diagnostics go to stderr and print per source in completion order, not alphabetically.

Suggested follow-ups

  • The gh test phase (about 25s) now ends cpp/obs-only entries and runs on every diff. It may be worth a look if small entries matter.
  • In full runs, the ffi lane (515s) plus dart running alone (109s) is the whole critical path. py (265s) and cpp (123s) dominate it, so splitting that shared-state lane would save far more than this change.

🤖 Generated with Claude Code

(Written by Claude Opus 5.5)

kixelated and others added 2 commits October 24, 2026 12:09
obs_compile still sits on the critical path after module lanes landed: it
is the last module of the serial ffi lane that dart waits on. Check its 20
translation units under the MOQ_CHECK_JOBS bound once headers are ready,
buffering each source's diagnostics and reaping every compiler's process
group on cancel.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title Parallelize OBS source checks ci(obs): check OBS sources concurrently Oct 24, 2026
@kixelated
kixelated marked this pull request as ready for review October 24, 2026 19:24
@coderabbitai

coderabbitai Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: e1c71581-b2fe-4a44-b00c-22726bf3cf8a
Commits

Reviewing files that changed from the base of the PR and between 6d746f8 and 6ba8d94.

Files selected for processing: 1
  • sh/obs/compile.sh

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 0 remain after this review.

Walkthrough

sh/obs/compile.sh now validates MOQ_CHECK_JOBS and uses it to limit concurrent syntax checks for source and test files. The script collects and prints each check’s output, continues after failures, and reports failed sources with their exit statuses. It also terminates running process groups and removes temporary logs during cleanup. The local-gate plan documents the setting’s expanded scope. The OBS parallel-check quest and its Required-list entry have been removed.

Priority: Normal

Merge Risk: Minimal · up to 6ba8d

No merge-blocking issue is established; complete the normal checks before merging.

Pre-merge checks: Passed 5
Check name Status Explanation
Title check Passed The title clearly and concisely describes the main change: concurrent checking of OBS sources in CI.
Description check Passed The description directly explains the concurrent OBS source checks, the MOQ_CHECK_JOBS limit, validation, measurements, verification, and impact.
Docstring Coverage Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 1…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 1 files.

Finishing Touches
✨ Simplify code
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-24T20:43:20.092347Z 6ba8d94 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a7898b5e18

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sh/obs/compile.sh Resolved
kixelated and others added 2 commits October 24, 2026 12:41
A compiler killed outright, such as by the OOM killer under concurrent
checks, prints nothing that says which source it was checking.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bash 5.2 drops a background job that a signal ends from its job table
once any other job is collected or a foreground command finishes, so
wait -n reports "no such job" and the gate aborts on an unset pid,
losing that source's diagnostics. Run each compiler as a child of a
subshell job, which exits normally with the compiler's status.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

(Written by Claude Opus 5.5)

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Nice work!

Reviewed commit: 6ba8d94e39

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@kixelated

Copy link
Copy Markdown
Collaborator Author

Merging 6ba8d94 with the maintainer's approval. The non-Grok review of this head is clean, every finding is fixed or answered, and the pre-land quest check passes against origin/main.

(Written by Claude Opus 5.5)

@kixelated
kixelated added this pull request to the merge queue Oct 25, 2026
Merged via the queue into main with commit fb60c8d Oct 25, 2026
10 checks passed
@kixelated
kixelated deleted the quest/m0/ci/obs-parallel branch October 25, 2026 01:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant