Target repository
intel/llvm — the crashing code lives at
unified-runtime/source/adapters/level_zero/v2/command_list_manager.cpp inside this monorepo
(the oneapi-src/unified-runtime standalone mirror is largely inactive now; intel/llvm is where
unified-runtime development actually happens per its commit history).
graphcapture-nullcheck.patch
sycl-fix-snapshot.zip
Summary
libur_adapter_level_zero_v2.so segfaults on the first kernel submission to any SYCL queue on
an Intel Arc Pro B70 (Battlemage, PCI ID 8086:e223). Root-caused via gdb + objdump against the
actual shipped binary (which ships with full DWARF debug info, not stripped) down to a specific
instruction: ur_command_list_manager::isGraphCaptureActive() calls through
ZeGraphExt.zeCommandListIsGraphCaptureEnabledExp without checking whether the pointer actually
resolved — call *0x278(%r15) with a null value stored at that offset. ZeGraphExt.Supported
(the gate this function trusts) is apparently true even though this one entry point never got
populated for this device/driver combination.
This function runs on every single SYCL queue submission, not only when a caller explicitly
uses SYCL graph capture — queue_impl::isNativeRecording() calls it unconditionally as part of
handler::finalize(). So it's not an edge case: it's the very first thing that happens on the very
first kernel launch.
Root cause
Traced through intel/llvm history: PR #21145 — "Fix urQueueIsGraphCaptureEnabledExp when a fork
is created in a graph"
(merged 2026-02-02) replaced a safe, purely-internal state read —
*pResult = graphCapture.isActive();
— with a raw, unguarded driver call:
ze_result_t ZeResult =
ZE_CALL_NOCHECK(hContext.get()
->getPlatform()
->ZeGraphExt.zeCommandListIsGraphCaptureEnabledExp,
(getZeCommandList()));
*pResult = (ZeResult == ZE_RESULT_QUERY_TRUE);
ZE_CALL_NOCHECK does not null-check the function pointer before invoking it. This is exactly the
code our shipped 2026.1.1 adapter still runs (confirmed by symbol name — the shipped binary still
calls this function isGraphCaptureActive, matching this commit; a later commit renamed it to
queryGraphCaptureActive, so the packaged 2026.1.1 build is pinned to an internal snapshot from
around this point, despite its later build date).
checkGraphExtensionSupport() / ZeGraphExt.Supported is supposed to gate this — platform.cpp
computes it by ANDing (*funcAddr != nullptr) over every entry in the graph extension's
mandatory-function table, zeCommandListIsGraphCaptureEnabledExp included. On this device/driver
combination that computation evidently doesn't hold: Supported reads true at call time, but the
specific pointer is null in memory (confirmed directly via objdump on the installed .so, not
inferred from source).
Environment
- GPU: 2× Intel Arc Pro B70, 32 GB VRAM each (Battlemage, PCI ID
8086:e223, "BMG G31")
- Board/CPU: Gigabyte X299 AORUS Gaming 7, Intel Core i7-7800X (Skylake-X)
- OS/kernel: Ubuntu-based, kernel 7.0.x,
xe DRM driver
- oneAPI: DPC++/C++ Compiler 2026.1.1 (
2026.1.1.20260724), oneMKL 2026.1.0
- Level Zero loader: 1.28.2
libze-intel-gpu1 (compute-runtime): 26.27.39122.11
- llama.cpp: reproduced on both current
master (876a432, ggml 0.18.0) and tag b8935
(f454bd7, ggml 0.10.0) — i.e. across ~months of unrelated ggml-sycl changes, with identical
crash signature both times.
- GPU BARs: full 32 GiB resizable BAR on both cards, confirmed via
lspci (Region 2: Memory ... [size=32G]) and zeInit/zeDeviceGet succeeding cleanly outside of any compute call.
Repro
source /opt/intel/oneapi/setvars.sh --force
llama-bench -m <any-gguf> -ngl 99 -dev SYCL0 -p 32 -n 16 -r 1
Crashes identically regardless of model architecture, quantization, or prompt/generation length,
100% of the time, on the very first compute call.
Backtrace (gdb, full symbols — binary ships with DWARF debug info)
Thread 1 "llama-bench" received signal SIGSEGV, Segmentation fault.
0x0000000000000000 in ?? ()
#0 0x0000000000000000 in ?? ()
#1 0x00007fffe8d32414 in ur_command_list_manager::isGraphCaptureActive(bool*) ()
from libur_adapter_level_zero_v2.so.0
#2 0x00007fffe8d95314 in v2::ur_queue_immediate_in_order_t::queueIsGraphCapteEnabledExp(bool*) ()
from libur_adapter_level_zero_v2.so.0
#3 0x00007fffe8d6d331 in ur::level_zero::urQueueIsGraphCaptureEnabledExp(ur_queue_handle_t_*, bool*) ()
from libur_adapter_level_zero_v2.so.0
#4 0x00007fffe39704cb in urQueueIsGraphCaptureEnabledExp () from libur_loader.so.0
#5 0x00007fffefeb4085 in sycl::_V1::detail::queue_impl::isNativeRecording() const () from libsycl.so.9
#6 0x00007fffeff2fea5 in sycl::_V1::handler::finalize() () from libsycl.so.9
#7 ... queue_impl::finalizeHandlerInOrderNoEventsUnlocked<sycl::_V1::handler>(...) () from libsycl.so.9
#8 ... queue_impl::submit_impl(...) () from libsycl.so.9
#9 ... sycl::_V1::queue::submit_with_event_impl(...) () from libsycl.so.9
#10 oneapi::mkl::gpu::launch_kernel_3D(...) () from libmkl_sycl_blas.so.6
#11 oneapi::mkl::gpu::run_check(...) () from libmkl_sycl_blas.so.6
#12 oneapi::mkl::gpu::have_binary_kernels(int*, sycl::_V1::queue*) () from libmkl_sycl_blas.so.6
#13 oneapi::mkl::gpu::mkl_blas_gpu_sgemm_driver_sycl(...) () from libmkl_sycl_blas.so.6
#14 oneapi::mkl::gpu::sgemm_sycl_internal(...) () from libmkl_sycl_blas.so.6
#15 oneapi::mkl::gpu::sgemm_sycl(...) () from libmkl_sycl_blas.so.6
#16 oneapi::mkl::blas::sgemm(...) () from libmkl_sycl_blas.so.6
#17 oneapi::mkl::blas::column_major::gemm(...) () from libmkl_sycl_blas.so.6
#18 ggml_sycl_op_mul_mat_sycl(...) () from libggml-sycl.so.0 <- last llama.cpp frame
#19 ggml_sycl_op_mul_mat<no_quantize_q8_1>(...) () from libggml-sycl.so.0
#20 ggml_sycl_mul_mat(...) () from libggml-sycl.so.0
#21 ggml_backend_sycl_graph_compute_impl(...) () from libggml-sycl.so.0
...
Disassembly of the actual crashing function in the shipped .so (objdump -d,
ur_command_list_manager::isGraphCaptureActive(bool*) at file offset 0xed3a0):
ed3aa: sub $0x18,%rsp
...
ed3c2: mov (%rdi),%rdi
ed3c5: call f9a60 <ur_context_handle_t_::getPlatform() const>
ed3ca: cmpb $0x0,0x230(%rax) ; ZeGraphExt.Supported
ed3d1: je ed46f ; skip if !Supported (not taken here)
ed3d7: mov (%r14),%rdi
ed3da: call f9a60 <ur_context_handle_t_::getPlatform() const>
ed3df: mov %rax,%r15
...
ed409: mov 0x68(%r14),%rdi ; getZeCommandList()
ed40d: call *0x278(%r15) ; <-- ZeGraphExt.zeCommandListIsGraphCaptureEnabledExp, NULL
Supported (platform+0x230) reads true; the call target stored at platform+0x278
(ZeGraphExt.zeCommandListIsGraphCaptureEnabledExp) is 0. rip after the call is exactly 0x0,
consistent with a null value at that memory location, not a bad this or stack corruption.
Fix (patch attached, tested working)
Null-check the resolved pointer at each of the five call sites in command_list_manager.cpp that
trust ZeGraphExt.Supported to guarantee a specific pointer is non-null, and fall back to
UR_RESULT_ERROR_UNSUPPORTED_FEATURE (or, for the capture-active query specifically, to
*pResult = false, since that query runs unconditionally on the hot path and a hard error there
would be a behavior change SYCL callers don't expect from what has always been a cheap boolean
check). Full diff: graphcapture-nullcheck.patch (attached).
Built and tested locally: cloned intel/llvm at commit fc6ef33cb092 (the exact commit whose code
the shipped 2026.1.1 binary still runs, confirmed by matching symbol names), applied this patch,
built just the ur_adapter_level_zero_v2 CMake target against the system Level Zero loader
(1.28.2) and oneMKL 2026.1.0, and loaded the patched .so via UR_ADAPTERS_FORCE_LOAD in place of
the shipped one — no other changes.
Result: llama-bench (llama.cpp master, 876a432) runs to completion on both GPUs, repeatedly,
with correct output, where the unpatched adapter crashed 100% of the time on the first call.
Performance (patched v2 adapter vs. v1-adapter workaround vs. Vulkan)
Same model, same machine, same settings — deepseek2 30B.A3B Q4_K_M (GLM-4.7-Flash, MoE), single
Arc Pro B70:
| Backend |
pp512 (t/s) |
tg128 (t/s) |
| Vulkan |
246.8 |
70.1 |
SYCL, v1 adapter (UR_ADAPTERS_FORCE_LOAD workaround) |
810.0 |
37.2 |
| SYCL, v2 adapter with this patch |
877.2 |
46.9 |
The patched v2 adapter is faster than both alternatives on prompt processing, and meaningfully
faster than the v1-adapter workaround on token generation too (+26%), though Vulkan still leads on
tg128 for this model. Confirmed stable across repeated runs and on both installed GPUs (SYCL0 and
SYCL1 independently: pp512 877.2 / 877.5 t/s, tg128 46.9 / 48.8 t/s).
This is a separate finding from the crash itself — noted for context since it was measured while
validating the fix, and because it changes the shape of the general "SYCL slower than Vulkan on
Battlemage" observation (see #22413): at least for MoE prompt processing, the v2 adapter is
actually ahead of Vulkan once it isn't crashing.
Ask
Happy to open this as a PR directly if that's preferred over a patch-attached issue — the change is
small, contained to one file, and already validated against real hardware. Also happy to add a
regression test to test/conformance/exp_graph/ exercising a driver where
zeCommandListIsGraphCaptureEnabledExp resolves to null but Supported is otherwise true, if
that's a useful signal to add given Supported's computation didn't catch this case in practice.
Target repository
intel/llvm — the crashing code lives at
unified-runtime/source/adapters/level_zero/v2/command_list_manager.cppinside this monorepo(the
oneapi-src/unified-runtimestandalone mirror is largely inactive now;intel/llvmis whereunified-runtimedevelopment actually happens per its commit history).graphcapture-nullcheck.patch
sycl-fix-snapshot.zip
Summary
libur_adapter_level_zero_v2.sosegfaults on the first kernel submission to any SYCL queue onan Intel Arc Pro B70 (Battlemage, PCI ID
8086:e223). Root-caused viagdb+objdumpagainst theactual shipped binary (which ships with full DWARF debug info, not stripped) down to a specific
instruction:
ur_command_list_manager::isGraphCaptureActive()calls throughZeGraphExt.zeCommandListIsGraphCaptureEnabledExpwithout checking whether the pointer actuallyresolved —
call *0x278(%r15)with a null value stored at that offset.ZeGraphExt.Supported(the gate this function trusts) is apparently true even though this one entry point never got
populated for this device/driver combination.
This function runs on every single SYCL queue submission, not only when a caller explicitly
uses SYCL graph capture —
queue_impl::isNativeRecording()calls it unconditionally as part ofhandler::finalize(). So it's not an edge case: it's the very first thing that happens on the veryfirst kernel launch.
Root cause
Traced through
intel/llvmhistory: PR #21145 — "Fix urQueueIsGraphCaptureEnabledExp when a forkis created in a graph"
(merged 2026-02-02) replaced a safe, purely-internal state read —
— with a raw, unguarded driver call:
ZE_CALL_NOCHECKdoes not null-check the function pointer before invoking it. This is exactly thecode our shipped
2026.1.1adapter still runs (confirmed by symbol name — the shipped binary stillcalls this function
isGraphCaptureActive, matching this commit; a later commit renamed it toqueryGraphCaptureActive, so the packaged 2026.1.1 build is pinned to an internal snapshot fromaround this point, despite its later build date).
checkGraphExtensionSupport()/ZeGraphExt.Supportedis supposed to gate this —platform.cppcomputes it by ANDing
(*funcAddr != nullptr)over every entry in the graph extension'smandatory-function table,
zeCommandListIsGraphCaptureEnabledExpincluded. On this device/drivercombination that computation evidently doesn't hold:
Supportedreads true at call time, but thespecific pointer is null in memory (confirmed directly via
objdumpon the installed.so, notinferred from source).
Environment
8086:e223, "BMG G31")xeDRM driver2026.1.1.20260724), oneMKL 2026.1.0libze-intel-gpu1(compute-runtime): 26.27.39122.11master(876a432, ggml 0.18.0) and tagb8935(
f454bd7, ggml 0.10.0) — i.e. across ~months of unrelatedggml-syclchanges, with identicalcrash signature both times.
lspci(Region 2: Memory ... [size=32G]) andzeInit/zeDeviceGetsucceeding cleanly outside of any compute call.Repro
Crashes identically regardless of model architecture, quantization, or prompt/generation length,
100% of the time, on the very first compute call.
Backtrace (gdb, full symbols — binary ships with DWARF debug info)
Disassembly of the actual crashing function in the shipped
.so(objdump -d,ur_command_list_manager::isGraphCaptureActive(bool*)at file offset0xed3a0):Supported(platform+0x230) reads true; the call target stored atplatform+0x278(
ZeGraphExt.zeCommandListIsGraphCaptureEnabledExp) is 0.ripafter thecallis exactly0x0,consistent with a null value at that memory location, not a bad
thisor stack corruption.Fix (patch attached, tested working)
Null-check the resolved pointer at each of the five call sites in
command_list_manager.cppthattrust
ZeGraphExt.Supportedto guarantee a specific pointer is non-null, and fall back toUR_RESULT_ERROR_UNSUPPORTED_FEATURE(or, for the capture-active query specifically, to*pResult = false, since that query runs unconditionally on the hot path and a hard error therewould be a behavior change SYCL callers don't expect from what has always been a cheap boolean
check). Full diff:
graphcapture-nullcheck.patch(attached).Built and tested locally: cloned
intel/llvmat commitfc6ef33cb092(the exact commit whose codethe shipped 2026.1.1 binary still runs, confirmed by matching symbol names), applied this patch,
built just the
ur_adapter_level_zero_v2CMake target against the system Level Zero loader(1.28.2) and oneMKL 2026.1.0, and loaded the patched
.soviaUR_ADAPTERS_FORCE_LOADin place ofthe shipped one — no other changes.
Result: llama-bench (
llama.cppmaster,876a432) runs to completion on both GPUs, repeatedly,with correct output, where the unpatched adapter crashed 100% of the time on the first call.
Performance (patched v2 adapter vs. v1-adapter workaround vs. Vulkan)
Same model, same machine, same settings —
deepseek2 30B.A3B Q4_K_M(GLM-4.7-Flash, MoE), singleArc Pro B70:
UR_ADAPTERS_FORCE_LOADworkaround)The patched v2 adapter is faster than both alternatives on prompt processing, and meaningfully
faster than the v1-adapter workaround on token generation too (+26%), though Vulkan still leads on
tg128 for this model. Confirmed stable across repeated runs and on both installed GPUs (
SYCL0andSYCL1independently: pp512 877.2 / 877.5 t/s, tg128 46.9 / 48.8 t/s).This is a separate finding from the crash itself — noted for context since it was measured while
validating the fix, and because it changes the shape of the general "SYCL slower than Vulkan on
Battlemage" observation (see #22413): at least for MoE prompt processing, the v2 adapter is
actually ahead of Vulkan once it isn't crashing.
Ask
Happy to open this as a PR directly if that's preferred over a patch-attached issue — the change is
small, contained to one file, and already validated against real hardware. Also happy to add a
regression test to
test/conformance/exp_graph/exercising a driver wherezeCommandListIsGraphCaptureEnabledExpresolves to null butSupportedis otherwise true, ifthat's a useful signal to add given
Supported's computation didn't catch this case in practice.