Isolate unloadable LINML_EP_LIBRARIES plugins; move TensorRT-RTX to 0.5.0 (1.7.1)

CHANGELOG.md

@@ -4,6 +4,18 @@ All notable changes to LinML are documented here. This project follows [Semantic
 
 ## [Unreleased]
 
+### Changed
+- **TensorRT-RTX 1.7.** The stock `trt-rtx` pin is `onnxruntime-ep-nv-tensorrt-rtx-cu13` **0.5.0** (x86_64 `manylinux_2_28` and aarch64 `manylinux_2_35`, as before), which bundles TensorRT-RTX **1.7.1** instead of 1.6.1. Same ABI and minimum ORT (1.24). TensorRT-RTX engines are not forward-compatible, so an engine cache built under 1.6.1 (`trt_engine_cache_enable`) is rebuilt.
+- **A vendored 0.4.0 wheel is replaced.** The wheel records no version, so the installer and `scripts/fetch-ep-plugins.sh` now treat the versioned runtime it bundles (`libtensorrt_rtx.so.1.7.1`) as the marker of the pinned build: a 0.4.0 install in the same directory is replaced, verified before the old directory is touched. `PluginPackageSpec.VersionMarkerFile` is the internal opt-in.
+- The SONAME links (`libtensorrt_rtx.so.1`, `libtensorrt_onnxparser_rtx.so.1`) now follow whichever runtime version the directory holds instead of a hardcoded 1.6.1, which would have left the 0.5.0 EP unable to load.
+- The patched WSL2 build stays at `0.4.0-wsl2p2` and is not rebuilt: the fix is expected upstream ([NVIDIA/TensorRT-RTX-EP-ABI#12](https://github.com/NVIDIA/TensorRT-RTX-EP-ABI/pull/12)). It links the stock wheel's runtime by SONAME and is verified on TensorRT-RTX 1.7.1 in WSL2.
+
+### Fixed
+- **A plugin library in `LINML_EP_LIBRARIES` that fails to load no longer breaks every command.** On a host with the MIGraphX plugin vendored but no ROCm (`libmigraphx_c.so.3` missing), sourcing `scripts/env-gpu-deps.sh` mapped MIGraphX anyway, and `providers list`, `doctor`, `run`, `serve` and `bench` all exited with an unhandled `ArgumentException` (`Failed to register EP plugin library`). Such a library is now skipped with a warning on stderr and not passed on, so other providers work; `doctor` reports it under its own provider. A library given with `--ep-library`, or the one for the provider requested with `--provider`, still fails as before.
+- **`env-gpu-deps.sh` maps MIGraphX only when ROCm is present.** The vendored plugin is added to `LINML_EP_LIBRARIES` only when `libmigraphx_c.so.3` resolves (`LD_LIBRARY_PATH` or the `ldconfig` cache), and a vendored mapping left by an earlier source is removed when it does not. A MIGraphX mapping you set to another path is kept.
+- **`run --require-provider` matches `doctor` on hosts that cannot run a provider.** A required provider this host lacks the hardware or architecture for (MIGraphX without ROCm, QNN off aarch64) is reported as not applicable, with no `linml catalog install` hint, whether or not its plugin is vendored. The install hint remains for a plugin that could run here but is not installed. Provider selection failures use the same rule.
+- **`env-gpu-deps.sh` no longer drops mappings when sourced from zsh.** The cleanup of stale TensorRT-RTX and OpenVINO mappings split `LINML_EP_LIBRARIES` by `IFS`, which zsh does not do, so it could discard unrelated mappings. It now behaves the same in bash and zsh.
+
 ### Added
 - **QNN in the catalog.** `linml catalog install qnn` installs the Qualcomm QNN plugin (`onnxruntime-qnn` 2.6.0, aarch64) on Snapdragon aarch64 hosts. On other architectures the catalog reports it as not applicable, `catalog install` refuses without downloading it, and `setup --all` and `doctor` skip it with the reason. Not verified on hardware.
 

README.md

@@ -171,7 +171,7 @@ OpenVINO, TensorRT-RTX, MIGraphX and QNN are plugin execution providers. LinML d
 | Plugin | Provider name | Platforms | Pinned artifact | Installed to (under `~/.local/linml-deps`) |
 |--------|---------------|-----------|-----------------|--------------------------------------------|
 | OpenVINO | `OpenVINOExecutionProvider` | Linux x86_64 with an Intel CPU, GPU or NPU | Intel `onnxruntime-ep-openvino` 1.6.1 wheel from PyPI, with the OpenVINO 2026.2.1 runtime bundled | `ep-plugins/openvino/onnxruntime_ep_openvino/` |
-| TensorRT-RTX | `NvTensorRTRTXExecutionProvider` | Linux x86_64; Linux aarch64 (SBSA) | NVIDIA `onnxruntime-ep-nv-tensorrt-rtx-cu13` 0.4.0 wheel from PyPI, with TensorRT-RTX 1.6.1 bundled | `ep-plugins/trt-rtx/onnxruntime_ep_nv_tensorrt_rtx/` (x86_64); `ep-plugins/trt-rtx-arm64/onnxruntime_ep_nv_tensorrt_rtx/` (aarch64) |
+| TensorRT-RTX | `NvTensorRTRTXExecutionProvider` | Linux x86_64; Linux aarch64 (SBSA) | NVIDIA `onnxruntime-ep-nv-tensorrt-rtx-cu13` 0.5.0 wheel from PyPI, with TensorRT-RTX 1.7.1 bundled | `ep-plugins/trt-rtx/onnxruntime_ep_nv_tensorrt_rtx/` (x86_64); `ep-plugins/trt-rtx-arm64/onnxruntime_ep_nv_tensorrt_rtx/` (aarch64) |
 | TensorRT-RTX on WSL2 | same | WSL2 with GPU passthrough (`/dev/dxg`) | LinML patched build `0.4.0-wsl2p2` (GitHub release; glibc 2.35+, the same floor as the wheel), used alongside the wheel above | `ep-plugins/trt-rtx-wsl2/linml_trt_rtx_ep/` |
 | MIGraphX | `MIGraphXExecutionProvider` | Linux x86_64 with an AMD GPU and ROCm | LinML build `2611.1` (GitHub release), built from AMD's `onnxruntime-ep-amdgpu` | `ep-plugins/migraphx/linml_migraphx_ep/` |
 | QNN | `QNNExecutionProvider` | Linux aarch64 with a Qualcomm Snapdragon NPU | Qualcomm `onnxruntime-qnn` 2.6.0 tarball (GitHub release), with the QAIRT 2.50.40 runtime bundled | `ep-plugins/qnn/onnxruntime_qnn/` |
@@ -246,14 +246,14 @@ These commands use only `curl`, `sha256sum`, `unzip`, and `mktemp`. Each downloa
   TMP="$(mktemp -d)"
   case "$(uname -m)" in
     x86_64)
-      WHEEL=onnxruntime_ep_nv_tensorrt_rtx_cu13-0.4.0-py3-none-manylinux_2_28_x86_64.whl
-      URL=https://files.pythonhosted.org/packages/9c/68/093626a054300fa7f4f1a6d18b775d185034c6bb7fb056ac4f9d1e9b50d4/$WHEEL
-      SHA256=68b8f7306b50cf76daf997b8be24ec3813498c15d48e0ddd211db14f2a9daa19
+      WHEEL=onnxruntime_ep_nv_tensorrt_rtx_cu13-0.5.0-py3-none-manylinux_2_28_x86_64.whl
+      URL=https://files.pythonhosted.org/packages/6b/ea/033a6d73b74a443cbf67247e3700b2a2f5076b37af4b00e433fbbd46fa6d/$WHEEL
+      SHA256=32f575c633d171b4edf4b53d3bb22b72a6c9c0a850d2982ce9a788a5d40d3298
       DIR=trt-rtx ;;
     aarch64)
-      WHEEL=onnxruntime_ep_nv_tensorrt_rtx_cu13-0.4.0-py3-none-manylinux_2_35_aarch64.whl
-      URL=https://files.pythonhosted.org/packages/41/63/ef110d73730969957740d770b29c158ca79a9df32c65ee081b01f8b64124/$WHEEL
-      SHA256=0ccf6a636f9f49d74e3e7e3b1b90287aa1c818db770b47fad6e8aa344f4290a5
+      WHEEL=onnxruntime_ep_nv_tensorrt_rtx_cu13-0.5.0-py3-none-manylinux_2_35_aarch64.whl
+      URL=https://files.pythonhosted.org/packages/9c/5e/9b63610651a0f5d2c8712b84f64c8cee3b3b83564cc20be207d00272d982/$WHEEL
+      SHA256=1f0bb2b89bb3dfd43096fccb8ec1f581c266c5852c4e6ba672b7bf790c8c96c2
       DIR=trt-rtx-arm64
       # The aarch64 wheel is manylinux_2_35: refuse an older glibc before downloading it.
       GLIBC="$(getconf GNU_LIBC_VERSION 2>/dev/null | awk '{print $2}')" || GLIBC=""
@@ -263,15 +263,21 @@ These commands use only `curl`, `sha256sum`, `unzip`, and `mktemp`. Each downloa
     *) echo "No TensorRT-RTX build for $(uname -m)" >&2; false ;;
   esac
   OUT="$DEPS/ep-plugins/$DIR/onnxruntime_ep_nv_tensorrt_rtx"
-  mkdir -p "$OUT"
+  # Extract into a staging directory and swap it in, so an upgrade does not leave an
+  # earlier build's runtime (libtensorrt_rtx.so.1.6.1 from 0.4.0) beside the new one.
+  mkdir -p "$(dirname "$OUT")"
+  STAGE="$(mktemp -d "$OUT.partial.XXXXXX")"
   curl -fsSL -o "$TMP/$WHEEL" "$URL"
   echo "$SHA256  $TMP/$WHEEL" | sha256sum -c
-  unzip -q -j -o "$TMP/$WHEEL" 'onnxruntime_ep_nv_tensorrt_rtx/*' -d "$OUT"
+  unzip -q -j -o "$TMP/$WHEEL" 'onnxruntime_ep_nv_tensorrt_rtx/*' -d "$STAGE"
   # The plugin links SONAMEs, but the wheel ships only versioned files: add the links.
-  (cd "$OUT" &&
-    ln -sf libtensorrt_rtx.so.1.6.1 libtensorrt_rtx.so.1 &&
-    ln -sf libtensorrt_onnxparser_rtx.so.1.6.1 libtensorrt_onnxparser_rtx.so.1 &&
+  (cd "$STAGE" &&
+    ln -sf libtensorrt_rtx.so.1.7.1 libtensorrt_rtx.so.1 &&
+    ln -sf libtensorrt_onnxparser_rtx.so.1.7.1 libtensorrt_onnxparser_rtx.so.1 &&
     ln -sf "$(ls libcudart.so.13.* | head -n 1)" libcudart.so.13)
+  chmod 755 "$STAGE"
+  rm -rf "$OUT"
+  mv "$STAGE" "$OUT"
   rm -rf "$TMP"
   echo "TensorRT-RTX installed to $OUT"
 )
@@ -790,7 +796,7 @@ for warning in report.compatibility_warnings:
 
 It covers:
 - **Built-in EPs**: `CPUExecutionProvider`, `CUDAExecutionProvider`, and `TensorrtExecutionProvider`
-- **Plugin EPs**: `OpenVINOExecutionProvider` (1.6.1), `NvTensorRTRTXExecutionProvider` (0.4.0), and `MIGraphXExecutionProvider` (2611.1)
+- **Plugin EPs**: `OpenVINOExecutionProvider` (1.6.1), `NvTensorRTRTXExecutionProvider` (0.5.0), and `MIGraphXExecutionProvider` (2611.1)
 
 ```csharp
 using LinML.Catalog;

docs/compatibility.md

@@ -18,9 +18,9 @@
 
 **TensorRT-RTX on aarch64.** NVIDIA's TensorRT-RTX 1.6 supports Ubuntu 24.04 SBSA
 (Linux aarch64) with CUDA 13.4, gcc 11.4, and compute capabilities 7.5, 8.6, 8.9,
-12.0 and 12.13. LinML pins the same EP version (0.4.0) for `manylinux_2_35_aarch64`
+12.0 and 12.13. LinML pins the same EP version as on x64 (0.5.0, TensorRT-RTX 1.7.1) for `manylinux_2_35_aarch64`
 (glibc 2.35 or newer), selected automatically on aarch64 hosts. The aarch64 build is
-not yet verified on aarch64 hardware. The x64 pin is unchanged.
+not yet verified on aarch64 hardware.
 
 ## Execution Providers
 
@@ -135,9 +135,11 @@ not yet verified on aarch64 hardware. The x64 pin is unchanged.
   detects WSL2 and an RTX Ampere+ GPU. On WSL2 the CLI prefers the patched build
   for `trt-rtx` (as does `env-gpu-deps.sh`); `TRT_RTX_CUSTOM_EP` overrides it (also in the catalog since 0.4.2). Notes:
   - The archive contains only the Apache-2.0 EP plugin `.so` (~5 MB, RUNPATH
-    `$ORIGIN`); it dynamically links the TensorRT-RTX 1.6.1 runtime
+    `$ORIGIN`); it dynamically links the TensorRT-RTX runtime by SONAME
     (`libtensorrt_rtx.so.1`, `libtensorrt_onnxparser_rtx.so.1`) and CUDA 13,
-    which the stock `trt-rtx` wheel provides.
+    which the stock `trt-rtx` wheel provides. It is built from EP 0.4.0 against the
+    1.6.1 SDK; since the stock pin moved to 0.5.0 it runs on that wheel's TensorRT-RTX
+    1.7.1 runtime (verified in WSL2 on an RTX 5070: `TensorrtRtxWsl2EndToEndTests`).
   - Built by `scripts/build-trt-rtx-ep.sh`, which pins the PR head commit and
     produces a byte-stable archive. It is **not** built in CI: the TensorRT-RTX
     SDK (headers + import libs) is under NVIDIA's license and cannot be

scripts/env-gpu-deps.sh

@@ -82,17 +82,23 @@ if [ -f "$TRT_RTX_EP" ]; then
     # remains the explicit override.
     if [ -n "${LINML_EP_LIBRARIES:-}" ]; then
         _linml_rest=""
-        _linml_old_ifs="$IFS"
-        IFS=';'
-        for _linml_pair in $LINML_EP_LIBRARIES; do
+        # Peel entries off one at a time rather than word-splitting on IFS, so
+        # this also works when sourced from zsh (which does not split $VAR).
+        _linml_todo="$LINML_EP_LIBRARIES"
+        while [ -n "$_linml_todo" ]; do
+            _linml_pair="${_linml_todo%%;*}"
+            case "$_linml_todo" in
+                *";"*) _linml_todo="${_linml_todo#*;}" ;;
+                *) _linml_todo="" ;;
+            esac
+            [ -n "$_linml_pair" ] || continue
             case "$_linml_pair" in
                 NvTensorRTRTXExecutionProvider=*|nvtensorrtrtx=*) ;;  # stale, drop
                 *) _linml_rest="${_linml_rest:+$_linml_rest;}$_linml_pair" ;;
             esac
         done
-        IFS="$_linml_old_ifs"
         LINML_EP_LIBRARIES="$_linml_rest"
-        unset _linml_pair _linml_rest _linml_old_ifs
+        unset _linml_pair _linml_rest _linml_todo
     fi
     LINML_EP_LIBRARIES="NvTensorRTRTXExecutionProvider=$TRT_RTX_EP${LINML_EP_LIBRARIES:+;$LINML_EP_LIBRARIES}"
     export LINML_EP_LIBRARIES
@@ -111,9 +117,16 @@ if [ -f "$OPENVINO_EP" ]; then
     _linml_has_ov=""
     if [ -n "${LINML_EP_LIBRARIES:-}" ]; then
         _linml_rest=""
-        _linml_old_ifs="$IFS"
-        IFS=';'
-        for _linml_pair in $LINML_EP_LIBRARIES; do
+        # Peel entries off one at a time rather than word-splitting on IFS, so
+        # this also works when sourced from zsh (which does not split $VAR).
+        _linml_todo="$LINML_EP_LIBRARIES"
+        while [ -n "$_linml_todo" ]; do
+            _linml_pair="${_linml_todo%%;*}"
+            case "$_linml_todo" in
+                *";"*) _linml_todo="${_linml_todo#*;}" ;;
+                *) _linml_todo="" ;;
+            esac
+            [ -n "$_linml_pair" ] || continue
             case "$_linml_pair" in
                 openvino=*) ;;  # stale alias, drop
                 linml.openvino=*|OpenVINOExecutionProvider=*)
@@ -122,9 +135,8 @@ if [ -f "$OPENVINO_EP" ]; then
                 *) _linml_rest="${_linml_rest:+$_linml_rest;}$_linml_pair" ;;
             esac
         done
-        IFS="$_linml_old_ifs"
         LINML_EP_LIBRARIES="$_linml_rest"
-        unset _linml_pair _linml_rest _linml_old_ifs
+        unset _linml_pair _linml_rest _linml_todo
     fi
     if [ -z "$_linml_has_ov" ]; then
         LINML_EP_LIBRARIES="linml.openvino=$OPENVINO_EP${LINML_EP_LIBRARIES:+;$LINML_EP_LIBRARIES}"
@@ -134,15 +146,58 @@ if [ -f "$OPENVINO_EP" ]; then
 fi
 
 # Convenience: default plugin EP library registration for MIGraphX
-# (vendored by scripts/fetch-ep-plugins.sh or LinML.Catalog).
+# (vendored by scripts/fetch-ep-plugins.sh or LinML.Catalog). The plugin links
+# against ROCm's libmigraphx_c.so.3, so it is only mapped when that library
+# resolves (loader path or ldconfig cache); without ROCm, MIGraphX is not
+# applicable on this host (see `linml doctor`) and the plugin cannot load.
+_linml_has_migraphx_runtime() {
+    # Written to work when sourced from zsh too, which does not word-split $VAR.
+    if printf '%s\n' "${LD_LIBRARY_PATH:-}" | tr ':' '\n' | while IFS= read -r _linml_dir; do
+            if [ -n "$_linml_dir" ] && [ -e "$_linml_dir/libmigraphx_c.so.3" ]; then echo found; fi
+        done | grep -q found; then
+        return 0
+    fi
+    for _linml_ldconfig in ldconfig /sbin/ldconfig /usr/sbin/ldconfig; do
+        if command -v "$_linml_ldconfig" >/dev/null 2>&1; then
+            "$_linml_ldconfig" -p 2>/dev/null | grep -q 'libmigraphx_c\.so\.3 '
+            _linml_rc=$?
+            unset _linml_ldconfig
+            return $_linml_rc
+        fi
+    done
+    unset _linml_ldconfig
+    return 1
+}
 MIGRAPHX_EP="$DEPS/ep-plugins/migraphx/linml_migraphx_ep/libmigraphx-ep.so"
-if [ -f "$MIGRAPHX_EP" ]; then
+if [ -f "$MIGRAPHX_EP" ] && _linml_has_migraphx_runtime; then
     case ";${LINML_EP_LIBRARIES:-};" in
         *";MIGraphXExecutionProvider="*|*";migraphx="*) ;;
         *) LINML_EP_LIBRARIES="MIGraphXExecutionProvider=$MIGRAPHX_EP${LINML_EP_LIBRARIES:+;$LINML_EP_LIBRARIES}" ;;
     esac
     export LINML_EP_LIBRARIES
+elif [ -n "${LINML_EP_LIBRARIES:-}" ]; then
+    # Drop the vendored mapping a previous source (or an older version of this
+    # script) exported; any other MIGraphX mapping is the user's and is kept.
+    # Walked entry by entry like the TensorRT-RTX and OpenVINO cleanups above.
+    _linml_rest=""
+    _linml_todo="$LINML_EP_LIBRARIES"
+    while [ -n "$_linml_todo" ]; do
+        _linml_pair="${_linml_todo%%;*}"
+        case "$_linml_todo" in
+            *";"*) _linml_todo="${_linml_todo#*;}" ;;
+            *) _linml_todo="" ;;
+        esac
+        [ -n "$_linml_pair" ] || continue
+        case "$_linml_pair" in
+            "MIGraphXExecutionProvider=$MIGRAPHX_EP") ;;  # stale vendored mapping, drop
+            *) _linml_rest="${_linml_rest:+$_linml_rest;}$_linml_pair" ;;
+        esac
+    done
+    LINML_EP_LIBRARIES="$_linml_rest"
+    export LINML_EP_LIBRARIES
+    unset _linml_pair _linml_rest _linml_todo
 fi
+unset -f _linml_has_migraphx_runtime
 
 
 # Convenience: default plugin EP library registration for Qualcomm QNN