Skip to content

fix(smoke): drain the C client's registrations before main returns - #3369

Merged
kixelated merged 2 commits into
mainfrom
quest/m0/2676-libmoq-process-exit-can-abort-in-glibcs-pthread-tpp
Sep 4, 2026
Merged

kixelated merged 2 commits into
mainfrom
quest/m0/2676-libmoq-process-exit-can-abort-in-glibcs-pthread-tpp

Conversation

@kixelated

Copy link
Copy Markdown
Collaborator

Summary

  • The rust -> c smoke leg was flaky, and 17bf0d9 papered over it with _exit(0) on the success path, blaming a glibc priority-protected-mutex assertion on openh264's worker threads during atexit. That diagnosis was wrong on every count: the subscribe client never encodes, so no openh264 thread exists, and libmoq's runtime thread is harmless at exit.
  • Root cause: a use-after-return of ctx_t. It lives on main's stack and is handed to four separate registrations as user_data. moq_session_close ends the session's status registration alone; moq_origin_consume_announced, moq_consume_catalog and moq_consume_video each keep the pointer until their own terminal (<= 0) callback, exactly as moq.h documents. The old code closed the session and returned immediately, so exit()'s call frames landed on the addresses ctx had occupied while libmoq's runtime thread was still calling on_frame, which locks ctx->mu and signals ctx->cv right over the live exit path.
  • Whichever field the exit path clobbers first picks the symptom. A corrupted return address gives SIGSEGV/SIGBUS; a corrupted pthread_mutex_t.__kind routes an ordinary lock down glibc's priority-protect branch with a garbage priority and gives the __pthread_tpp_change_priority assertion this issue was filed for.
  • The success path now closes all four registrations, waits for all four terminals, then returns. The failure paths keep _exit for the user_data lifetime reason from fix(smoke): stop the C client returning from main while libmoq holds ctx #2675.
  • No libmoq change. The contract is already documented correctly and covered by rs/libmoq/src/test.rs; the client just did not follow it. cpp/obs was never affected: moq_source_destroy already refcounts every subscription and waits for the terminals (and deliberately leaks rather than free early on timeout).

Reproduction

Linux, Debian 12 / glibc 2.36, aarch64, in a container: a moq-relay from this tree, an ffmpeg testsrc H.264 publisher through moq import avc3, and a subscriber built against libmoq.a. Each variant differs only in how main ends.

variant fatal signals
pre-17bf0d950 shape: moq_session_close then return 0, ctx on the stack 4 / 100
same, but ctx heap-allocated and leaked 0 / 200
close + drain all four registrations, then return 0 (this PR's shape) 0 / 200
no registrations at all: moq_origin_create then return 0 0 / 200
the fixed subscribe.c 0 / 200

The heap variant is the control that rules out libmoq's teardown: same call sequence, same live runtime thread at exit(), only the storage moved off the dying stack.

Under gdb, a faulting run shows it directly:

Thread 2 "libmoq":
#3  ___pthread_cond_broadcast (cond=0xfffff83e3910) at ./nptl/pthread_cond_broadcast.c:72
#4  on_frame (ud=0xfffff83e38e0, frame=19) at repro.c:61
#5  moq::ffi::OnStatus::call at rs/libmoq/src/ffi.rs:81
#6  moq::consume::Consume::run_track::{{closure}} at rs/libmoq/src/consume.rs:468

Thread 1 "repro": SIGSEGV, pc 0xffffa20faa00 (garbage), sp 0xfffff83e3900

The condvar being broadcast sits 16 bytes above the main thread's live stack pointer.

Public API changes

None.

Test plan

  • just check and just test on this tree.
  • The counts above, run against a relay and publisher built from this checkout.
  • The drain is itself a standing assertion: if any registration stops delivering its terminal, subscribe.c now fails the leg deterministically with teardown never quiesced (frame=… catalog=… broadcast=… status=…) instead of leaving a ~4% flake. That is what a regression here would look like.

Cross-package sync: no row applies. rs/libmoq's C ABI is untouched, so cpp/obs/src and doc/bin/obs.md need nothing.

Closes #2676

🤖 Generated with Claude Code

(Written by Claude Opus 5)

@coderabbitai

coderabbitai Bot commented Sep 3, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: cc8fdf42-a677-4a7a-8b7f-d7f5ffe414aa

📥 Commits

Reviewing files that changed from the base of the PR and between 3660b17 and f35c94c.

📒 Files selected for processing (3)
  • quest/m0/2676-libmoq-process-exit-can-abort-in-glibcs-pthread-tpp.md
  • quest/m0/README.md
  • test/smoke/clients/c/subscribe.c
💤 Files with no reviewable changes (2)
  • quest/m0/2676-libmoq-process-exit-can-abort-in-glibcs-pthread-tpp.md
  • quest/m0/README.md

Included review availability: Your plan provides up to 4 included reviews per hour; 3 remain after this review.


Walkthrough

The C subscribe smoke client now stores all registration handles and completion state in ctx_t. Callbacks process terminal events, errors, and video frames. Successful execution drains registrations, closes resources, and returns 0. Failure paths retain _exit(1) while callbacks may reference the stack-backed context. The related quest document and README entry were removed.

Merge Risk: ⚪ Minimal · up to f35c9

The C smoke client now completes callback teardown before normal exit, preventing callbacks from referencing expired stack state. No concrete current-head merge-blocking risk remains.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 8 functions across 1 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the main change: draining the C client's registrations before returning from main.
Description check ✅ Passed The description explains the use-after-return cause, the registration-drain fix, retained failure handling, reproduction results, and testing plan. It directly matches the changeset.
Linked Issues check ✅ Passed The changes address issue #2676 by removing the successful-path _exit workaround, closing all registrations, waiting for terminal callbacks, and allowing the C client to return normally only after tea…
Out of Scope Changes check ✅ Passed The changes remain within scope. The C smoke client implements the teardown fix, and the quest document and README entry are removed as issue-tracking cleanup. No unrelated libmoq or public API change…
  • Fix all pre-merge checks with AI
✨ Finishing Touches
✨ Simplify code
  • Create PR with simplified code
  • Commit simplified code in branch quest/m0/2676-libmoq-process-exit-can-abort-in-glibcs-pthread-tpp

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 3, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-04T16:51:01.822321Z f35c94c New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c91d7f630b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread test/smoke/clients/c/subscribe.c Resolved
Comment thread test/smoke/clients/c/subscribe.c Resolved
kixelated and others added 2 commits September 4, 2026 09:22
Placeholder to claim the quest branch.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The `rust -> c` leg was flaky, and 17bf0d9 papered over it with `_exit(0)`
on the success path, blaming a glibc priority-protected-mutex assertion on
openh264's worker threads at atexit. That diagnosis was wrong: the subscribe
client never encodes, so no openh264 thread exists, and libmoq's runtime
thread is harmless at exit.

The real mechanism is a use-after-return of `ctx_t`. It lives on main's stack
and is handed to four separate registrations as `user_data`. `moq_session_close`
ends the session's status registration alone; `moq_origin_consume_announced`,
`moq_consume_catalog` and `moq_consume_video` each keep the pointer until their
own terminal (<= 0) callback. The old code closed the session and returned
immediately, so `exit()`'s call frames landed on the same addresses `ctx` had
occupied while the runtime thread was still calling `on_frame`, which locks
`ctx->mu` and broadcasts `ctx->cv` right over the live exit path.

Reproduced on Linux/glibc 2.36 (aarch64) with a relay, an ffmpeg H.264
publisher and the pre-17bf0d950 shape: 4 fatal signals in 100 runs. Under gdb
the faulting process shows the `libmoq` thread inside
`pthread_cond_broadcast(&ctx->cv)` from `on_frame`, on a condvar 16 bytes above
main thread's live `sp`, while the main thread's PC is garbage. Whichever field
the exit path clobbers first picks the symptom: a corrupted return address
gives SIGSEGV/SIGBUS, a corrupted `pthread_mutex_t.__kind` routes an ordinary
lock down glibc's TPP branch and gives the reported
`__pthread_tpp_change_priority` assertion.

So the success path now closes all four registrations, waits for all four
terminals, and returns. That drain is also a standing assertion: a terminal
callback that stops firing fails the leg deterministically instead of leaving
a 4% flake. The failure paths keep `_exit` for the `user_data` lifetime reason
from #2675.

Measured on the same host, 200 runs each: pre-fix stack `ctx` 4/100 fatal;
drained `ctx` 0/200; no registrations at all 0/200; the fixed client 0/200.
`cpp/obs` was never affected: `moq_source_destroy` already refcounts every
subscription and waits for the terminals.

Closes #2676

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@kixelated
kixelated force-pushed the quest/m0/2676-libmoq-process-exit-can-abort-in-glibcs-pthread-tpp branch from c91d7f6 to f35c94c Compare September 4, 2026 16:47
@kixelated
kixelated merged commit 053297c into main Sep 4, 2026
3 checks passed
@kixelated
kixelated deleted the quest/m0/2676-libmoq-process-exit-can-abort-in-glibcs-pthread-tpp branch September 4, 2026 17:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

libmoq: process exit can abort in glibc's __pthread_tpp_change_priority

1 participant