Skip to content

gfx: close a camera with its node, give its fallback open a budget, and pick Syphon's decoder from the RHI - #2331

Closed
jcelerier wants to merge 6 commits into
masterfrom
fix/camera-release-and-syphon-metal-decoder
Closed

jcelerier wants to merge 6 commits into
masterfrom
fix/camera-release-and-syphon-metal-decoder

Conversation

@jcelerier

@jcelerier jcelerier commented Oct 10, 2026 •

Copy link
Copy Markdown
Member

Four independent fixes around camera inputs, plus one CI fix. Each is described
by what was measured, and where an earlier claim in this PR turned out to be
wrong it is corrected below rather than quietly dropped.

1. A removed camera node left its capture running

video_texture_input_protocol::stop_execution() is the only thing that closed a
camera, and it is guarded on camera_node — which
~video_texture_input_parameter() sets to null as its first statement, so once
the node is gone the function is a no-op. Nothing else closed the device:
~video_texture_input_protocol() is empty, and ~CameraInput() only runs when
the last shared_ptr to the camera dies, which the render node holds and the
nursery releases from a 100 ms QTimer.

The destructor now arms the stop before unregistering the node, and
stop_execution() stops the camera directly when there is no render node, since
nothing can be rendering from it then.

Verified on Windows by logging every CameraInput::close_file() with its format
context:

start OK   this=CF65E870 fmt=6648B640 640x480
close_file this=CF65E870 fmt=6648B640 running=1   <- real context, still running
close_file this=CF65E870 fmt=       running=0     <- idempotent second call
deviceRemoved iface=6BA6B170 live=CF65A160        <- only now does the device go

That close is not ~CameraInput: the same object receives further close_file
calls afterwards, so it was still alive.

Correction to this PR's earlier description. It claimed the sequence's third
open never completes. It is the second, and the first version of this PR was
measuring the wrong thing — the scenario's opened N marker only means
NodeStore.create returned an id, which it does whether or not the capture
opened. Instrumenting start_execution gives ok=1, ok=0, ok=1.

The frame-safe must_stop path is also defensive rather than load-bearing:
avformat_close_input does have to come after the rendered frames are released,
and REMOVE_NODE walks exactly there, but in this removal sequence
stop_execution finds camera_node null every time — 4 calls, 0 deferrals — so
the direct stop is what runs.

And closing promptly does not make the device immediately re-openable. In the
same run the next open still failed with Could not run graph (… device already in use …) two seconds after score had closed its own capture. That is dshow
releasing on its own schedule, not a leak on this side.

2. The camera's fallback open was skipped exactly when it was needed

CameraInput::start() retries with default settings when the requested mode
fails, but the retry was guarded on !m_interrupt.aborted() via expired(),
which is also true once open_timeout has merely lapsed — precisely the case
the fallback exists for. So a slow open became a hard failure with no retry, and
the fallback shared the first attempt's leftover budget.

Observed directly: open failed aborted=0 expired=1. The retry now gets its own
LibavTimeout, and in the instrumented run round 3 only succeeds because of it.

3. Syphon picked its decoder from whether a server was found

Unrelated to the above; verified on Apple M2 Pro under Metal.

4. A previously pinned addon stayed on a detached HEAD

clone_addon's unpinned-update branch ran git pull --ff-only against a
detached checkout left by an earlier pin, which fails, so the addon silently
stayed at the old commit. Reproduced in isolation and fixed by moving the
checkout back onto the remote default branch first.

Merge order: this needs
ossia/score-addon-videoio#12 merged first. master
already tracks videoio master, which still lacks <stdlib.h> for ::free, and
existing checkouts are shielded only by the bug this commit removes.

5. A camera that could not start said nothing

ExternalInput::start() returns a bool and start_execution() discarded it, so
a failed open was silent: node present, no frames, nothing in the log. It now
warns, so the Messages panel shows it.

To be explicit about what this is not: it does not make a failed open
observable outside the process. In --ui mode score installs no Qt message
handler at all — SafeQApplication installs DebugOutput only under
SCORE_DEBUG, and the Messages panel that would forward to stderr is not loaded
without the GUI. Measured with the camera held by another process for a whole
run: three opens refused, this warning emitted three times, zero lines on
stderr.

What this PR no longer claims to fix

An earlier revision carried a commit that unregistered a removed device from the
execution state, presented as the fix for a Pure virtual function called abort
seen while cycling cameras. It has been dropped, because it does not fix that
crash and cannot.
Instrumenting the state's identity at every lookup showed the
abort happens on an abandoned execution_state:

findNode st=D8494F80 n=4: … CF65A010   <- current state, live camera
findNode st=6B9CBDC0 n=4: … CF659D70   <- abandoned state, camera removed earlier
libc++abi: Pure virtual function called!

ContextData is replaced wholesale, and its use_count at teardown is
sometimes 2 — the aliasing shared_ptrs handed out by
acquireExecutionQueue()/acquireGCQueue()/weakSelf() keep the whole
ContextData, and therefore its execution_state, alive with every device
pointer still registered in it. get_name() is non-virtual over the pure
virtual get_root_node(), so walking a destroyed device aborts; whether it
surfaces as the pure-virtual abort or a plain SIGSEGV depends only on what has
overwritten the vtable by then. Both were observed from this one cause.

The dropped commit operated on the current state, which the stale devices are
not in. That crash is a separate bug and is not addressed here.

Testing

Regression cover lives in sat-mtl/scenic-native
(sat-mtl/scenic-native#4): camcycle, camleak,
camregap, camstop. They judge an open on the /dev/video* descriptors
score's own process holds, having been validated in both directions against
instrumented ground truth — camera free → 3/3 PASS, camera held by another
process → 0/3 FAIL, matching start_execution's per-round ok= exactly.

All four pass on this branch on Linux. camleak and camstop were also checked
with and without fix 1 on Linux and are unaffected there, as expected: v4l2 is
not exclusive and drops the handle promptly, which is why this defect shows on
Windows.

🤖 Generated with Claude Code

https://claude.ai/code/session_019QYyvhRXvSMSRQPaq12Teq

@coveralls

coveralls commented Oct 10, 2026 •

Copy link
Copy Markdown

Coverage Status

Coverage is 48.078% — fix/camera-release-and-syphon-metal-decoder into master. No base build found for master.

@jcelerier
jcelerier force-pushed the fix/camera-release-and-syphon-metal-decoder branch from 59481a0 to 701b3b3 Compare October 11, 2026 01:12
@jcelerier jcelerier changed the title gfx: release a camera with its node, and pick Syphon's decoder from the RHI gfx: close a camera with its node, give its fallback open a budget, and pick Syphon's decoder from the RHI Oct 11, 2026
jcelerier and others added 2 commits October 11, 2026 20:48
Removing a camera node left the capture running. The only place that closed it
was video_texture_input_protocol::stop_execution(), which is guarded on
camera_node -- and video_texture_input_parameter's destructor sets that pointer
to null before anything else, so once the node is gone the function is a no-op.
Nothing else closed the device: ~video_texture_input_protocol() is empty, and
~CameraInput() only runs once the last shared_ptr to the camera dies, which the
render node holds and the nursery releases from a 100 ms QTimer. So the device
stayed open well past the removal.

The parameter's destructor now arms the stop before unregistering the node, and
stop_execution() stops the camera directly when camera_node is null, since with
no render node nothing can be rendering from it.

Verified on Windows by logging every CameraInput::close_file() with its format
context. Removing the node closes the capture there and then:

    start OK   this=CF65E870 fmt=6648B640 640x480
    close_file this=CF65E870 fmt=6648B640 running=1   <- real context, still running
    close_file this=CF65E870 fmt=       running=0     <- idempotent second call
    deviceRemoved iface=6BA6B170 live=CF65A160        <- only now does the device go

That close is not ~CameraInput: the same object receives further close_file
calls afterwards, so it was still alive. It is one of the two direct stops added
here, which is also the only thing that can have closed it -- stop_execution
provably does nothing without a camera_node, and it never has one here.

Two things this does *not* claim. The frame-safe must_stop path is defensive:
avformat_close_input does have to come after the rendered frames are released,
and REMOVE_NODE walks exactly there, but in Scenic's removal sequence
stop_execution finds camera_node null every time -- measured, 4 calls, 0
deferrals -- so the direct stop is what actually runs. And closing the capture
promptly does not make the device immediately re-openable: in the same run the
next open still failed with

    [dshow @ ...] Could not run graph (sometimes caused by a device already in
                  use by other application)

two seconds after score had closed its own capture. That is dshow releasing the
device on its own schedule, not a leak on this side.

No behaviour change on v4l2, where the device is not exclusive and the handle is
dropped promptly; test-camera-leak and test-camera-stopped pass identically with
and without this commit on Linux.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019QYyvhRXvSMSRQPaq12Teq
…s found

A Syphon input on Metal with no server advertised built the OpenGL decoder, whose
shader samples a sampler2DRect. Metal has no such texture type, so the pipeline
was never created and every draw was then dropped -- silently, bar one qDebug
line per draw.

openServer() sets m_usingMetal from rhi.backend(), but only after three early
returns: no servers in the directory, and no server matching the configured
name. Both leave the flag at its default, false. initState() then chooses the
decoder from it at a point that is *outside* the `if (enabled)` guard, so it runs
whether or not a server was found, and picks PackedRectDecoder.

What Metal is handed, from qsb -d on the two decoders' fragment shaders:

    rect:  fragment main0_out main0(..., constant material_t& mat [[buffer(0)]],
             unknown_texture_type<float> y_tex [[texture(0)]], ...)
    2D:    fragment main0_out main0(..., constant material_t& mat [[buffer(0)]],
             texture2d<float> y_tex [[texture(0)]], ...)

"unknown_texture_type" is what SPIRV-Cross emits for an image dimension its MSL
backend cannot map -- spirv_msl.cpp's default case, reached for DimRect. qsb
reports no error; Metal rejects the generated source at runtime as
"no template named 'unknown_texture_type'", which is the reported failure, in a
signature carrying the `constant material_t&` the report also named.

The flag now says what the RHI is, which is what the decoder choice needs, and is
set before any early return. The two later assignments become redundant and go.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019QYyvhRXvSMSRQPaq12Teq
@jcelerier
jcelerier force-pushed the fix/camera-release-and-syphon-metal-decoder branch from 56e14c6 to 27ff3f5 Compare October 12, 2026 00:48
jcelerier and others added 4 commits October 11, 2026 20:55
checkout_ref detaches, so an addon that ever had a pin stays detached.
"git pull --ff-only" on a detached HEAD fast-forwards nothing -- it
fetches, leaves the old commit in place and exits 0 -- so dropping a pin
silently kept the pinned revision on every machine that had already
built with it, and only ever took effect on a fresh clone.

Observed on this campaign's Windows builder: after 4b2f7f4 dropped
score-addon-videoio's pin, the checkout still reported "## HEAD (no
branch)" at the pinned 3cb1b7a3d5 while the pull fetched
cef46ac..83335aa for master. Four builds compiled the pinned revision
believing they tracked master.

Move the checkout onto the remote's default branch before pulling, and
say so, so a stale pin is visible rather than silent.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019QYyvhRXvSMSRQPaq12Teq
CameraInput::start() tries the requested mode, and on failure retries
with the device's default settings. Both attempts shared one
LibavTimeout, and the guard before the retry tested
LibavInterrupt::expired(), which is true both when close_file() aborted
from another thread and when the deadline has merely passed. So an
attempt that used up open_timeout skipped the fallback altogether:
start() returned false, and the camera node came up with no capture
behind it -- the reported "black input".

Test only for aborted(), which is the close path asking us to stop and
must not be retried past, and give the retry a budget of its own rather
than whatever the first attempt left. aborted() is not cleared, so a
concurrent close still wins.

Found by reading the code while instrumenting a Windows camera cycle,
where the second open of a BRIO failed outright and start_execution
reported 0. Whether that particular failure was a timeout or a refusal
from dshow is not yet established -- the instrumented run could not tell
the retry being skipped from the retry also failing, since both call
close_file() exactly once. This is a correctness fix by inspection: one
deadline cannot serve two attempts.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019QYyvhRXvSMSRQPaq12Teq
ExternalInput::start() returns a bool and start_execution() discarded it, so a
camera that could not be opened was completely silent: the node existed, no
frame ever arrived, and nothing in the log explained it. That is the "black
input" with no diagnosis. Warn instead, so it reaches the Messages panel.

This is worth having in the GUI and nothing more, so to be explicit about what
it is not: it does not make a failed open observable from outside the process.
In --ui mode score installs no Qt message handler at all -- SafeQApplication
installs DebugOutput only under SCORE_DEBUG, and the Messages panel that would
otherwise forward to stderr is not loaded without the GUI -- so no qWarning or
qDebug from any plugin is visible there. Measured with the camera held by
another process for a whole run: three opens refused, this warning emitted three
times, zero lines on stderr.

The camera suites in sat-mtl/scenic-native therefore judge an open on the
/dev/video* descriptors score's own process holds, not on this message.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019QYyvhRXvSMSRQPaq12Teq
Cycling a camera node aborted the process in Execution::findNode with
"Pure virtual function called!", or segfaulted, depending on what had
overwritten the vtable by then. Both came from one cause.

DocumentPlugin::clear() replaces m_ctxData, but the replaced ContextData
is frequently not destroyed: the audio tick function holds a strong
shared_ptr to it (AudioTickHelper::m_context, ExecutionTick.cpp), and the
engine only hands the old tick back once the audio thread has actually
run. audio_engine::sync() skips that wait entirely when running() is
false, and every backend reports that in reachable states -- no JACK
server, a PortAudio stream that failed to open, a corked PulseAudio
stream, an SDL or miniaudio device that is not started, Dummy inactive.
So the replaced context can outlive its replacement indefinitely.

While it does, it keeps working. SetupContext is a Nano::Observer, which
unsubscribes only in its destructor, so it stays connected to
model.cables; and bind_address() connects rebind() to every port it
binds, disconnecting only its own earlier connections, so each port
signal reaches every live SetupContext. Both routes end in
set_destination_impl -> findNode, which walks the context's own
execution_state and calls get_name() on each device. get_name() is
non-virtual over the pure virtual get_root_node(), so the first
destroyed device aborts.

Measured on Windows, 6-round sequence, 4 runs: 2 live SetupContexts,
one cable creation delivered to both, and the retired one resolving
against a state still listing the camera device freed two rounds
earlier:

    on_cableCreated setup=C1CB0608 ctx=C1CB0770 st=6AA1F1C0   <- current
    on_cableCreated setup=6AA37C48 ctx=6AA37DB0 st=6AA1F980   <- retired
    CALLER set_destination setup=6AA37DB0 st=6AA1F980
    findNode st=6AA1F980 n=4: ... CD474430
    probing dev=CD474430
    libc++abi: Pure virtual function called!

So mark the outgoing context retired and have everything that would act
on its behalf do nothing. Chasing the individual subscriptions does not
work: disconnecting model.cables alone was tried and measured, and it
removed those deliveries (11 -> 9) while leaving the crash untouched,
because the port signals reach rebind() just the same.

Giving AudioTickHelper a weak_ptr instead was considered and rejected:
make_tick() returns functors that store ossia::execution_state& and
ossia::graph_interface& (buffer_tick, precise_score_tick, ...), so the
strong reference is what keeps alive what the tick already points at.
Weakening it would dangle those references on the audio thread, and
~AudioTickHelper -- which dereferences the context throughout and
asserts the UI thread -- could then run the last release on the audio
thread, destroying QObjects there.

execution_state::clear_devices() on the outgoing state is the backstop
for the paths that take a state rather than a Context:
register_port()/unregister_port() also walk the device list calling
get_root_node(), so an emptied list turns that class of use-after-free
into a lookup that finds nothing.

Nothing in the audio path changes, and the guards fire only for a
retired context, so a healthy engine -- which destroys the old context
promptly -- behaves exactly as before.

Validated against the same 4-run repro: findNode calls on a non-current
state go 1-per-run to 0, the scenario reaches DONE for the first time
instead of dying in round 3, and 30 cable deliveries land on a retired
context and are ignored. The four camera suites in sat-mtl/scenic-native
pass on Linux before and after.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019QYyvhRXvSMSRQPaq12Teq
@jcelerier
jcelerier force-pushed the fix/camera-release-and-syphon-metal-decoder branch from 27ff3f5 to 51fa9a5 Compare October 12, 2026 00:55
jcelerier added a commit that referenced this pull request Oct 12, 2026
@jcelerier

Copy link
Copy Markdown
Member Author

Merged into #2326 (gfx/shading-tiers) at 7f22521. libossia resolved to master 3e7ee8462e (same tree as c5d5ed307b). Closing in favour of #2326.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants