diff --git a/CHANGELOG.md b/CHANGELOG.md index a9abf12..265993f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,18 @@ All notable changes to LinML are documented here. This project follows [Semantic ## [Unreleased] +### Changed +- **TensorRT-RTX 1.7.** The stock `trt-rtx` pin is `onnxruntime-ep-nv-tensorrt-rtx-cu13` **0.5.0** (x86_64 `manylinux_2_28` and aarch64 `manylinux_2_35`, as before), which bundles TensorRT-RTX **1.7.1** instead of 1.6.1. Same ABI and minimum ORT (1.24). TensorRT-RTX engines are not forward-compatible, so an engine cache built under 1.6.1 (`trt_engine_cache_enable`) is rebuilt. +- **A vendored 0.4.0 wheel is replaced.** The wheel records no version, so the installer and `scripts/fetch-ep-plugins.sh` now treat the versioned runtime it bundles (`libtensorrt_rtx.so.1.7.1`) as the marker of the pinned build: a 0.4.0 install in the same directory is replaced, verified before the old directory is touched. `PluginPackageSpec.VersionMarkerFile` is the internal opt-in. +- The SONAME links (`libtensorrt_rtx.so.1`, `libtensorrt_onnxparser_rtx.so.1`) now follow whichever runtime version the directory holds instead of a hardcoded 1.6.1, which would have left the 0.5.0 EP unable to load. +- The patched WSL2 build stays at `0.4.0-wsl2p2` and is not rebuilt: the fix is expected upstream ([NVIDIA/TensorRT-RTX-EP-ABI#12](https://github.com/NVIDIA/TensorRT-RTX-EP-ABI/pull/12)). It links the stock wheel's runtime by SONAME and is verified on TensorRT-RTX 1.7.1 in WSL2. + +### Fixed +- **A plugin library in `LINML_EP_LIBRARIES` that fails to load no longer breaks every command.** On a host with the MIGraphX plugin vendored but no ROCm (`libmigraphx_c.so.3` missing), sourcing `scripts/env-gpu-deps.sh` mapped MIGraphX anyway, and `providers list`, `doctor`, `run`, `serve` and `bench` all exited with an unhandled `ArgumentException` (`Failed to register EP plugin library`). Such a library is now skipped with a warning on stderr and not passed on, so other providers work; `doctor` reports it under its own provider. A library given with `--ep-library`, or the one for the provider requested with `--provider`, still fails as before. +- **`env-gpu-deps.sh` maps MIGraphX only when ROCm is present.** The vendored plugin is added to `LINML_EP_LIBRARIES` only when `libmigraphx_c.so.3` resolves (`LD_LIBRARY_PATH` or the `ldconfig` cache), and a vendored mapping left by an earlier source is removed when it does not. A MIGraphX mapping you set to another path is kept. +- **`run --require-provider` matches `doctor` on hosts that cannot run a provider.** A required provider this host lacks the hardware or architecture for (MIGraphX without ROCm, QNN off aarch64) is reported as not applicable, with no `linml catalog install` hint, whether or not its plugin is vendored. The install hint remains for a plugin that could run here but is not installed. Provider selection failures use the same rule. +- **`env-gpu-deps.sh` no longer drops mappings when sourced from zsh.** The cleanup of stale TensorRT-RTX and OpenVINO mappings split `LINML_EP_LIBRARIES` by `IFS`, which zsh does not do, so it could discard unrelated mappings. It now behaves the same in bash and zsh. + ### Added - **QNN in the catalog.** `linml catalog install qnn` installs the Qualcomm QNN plugin (`onnxruntime-qnn` 2.6.0, aarch64) on Snapdragon aarch64 hosts. On other architectures the catalog reports it as not applicable, `catalog install` refuses without downloading it, and `setup --all` and `doctor` skip it with the reason. Not verified on hardware. diff --git a/README.md b/README.md index 07dbc27..f8578c1 100644 --- a/README.md +++ b/README.md @@ -171,7 +171,7 @@ OpenVINO, TensorRT-RTX, MIGraphX and QNN are plugin execution providers. LinML d | Plugin | Provider name | Platforms | Pinned artifact | Installed to (under `~/.local/linml-deps`) | |--------|---------------|-----------|-----------------|--------------------------------------------| | OpenVINO | `OpenVINOExecutionProvider` | Linux x86_64 with an Intel CPU, GPU or NPU | Intel `onnxruntime-ep-openvino` 1.6.1 wheel from PyPI, with the OpenVINO 2026.2.1 runtime bundled | `ep-plugins/openvino/onnxruntime_ep_openvino/` | -| TensorRT-RTX | `NvTensorRTRTXExecutionProvider` | Linux x86_64; Linux aarch64 (SBSA) | NVIDIA `onnxruntime-ep-nv-tensorrt-rtx-cu13` 0.4.0 wheel from PyPI, with TensorRT-RTX 1.6.1 bundled | `ep-plugins/trt-rtx/onnxruntime_ep_nv_tensorrt_rtx/` (x86_64); `ep-plugins/trt-rtx-arm64/onnxruntime_ep_nv_tensorrt_rtx/` (aarch64) | +| TensorRT-RTX | `NvTensorRTRTXExecutionProvider` | Linux x86_64; Linux aarch64 (SBSA) | NVIDIA `onnxruntime-ep-nv-tensorrt-rtx-cu13` 0.5.0 wheel from PyPI, with TensorRT-RTX 1.7.1 bundled | `ep-plugins/trt-rtx/onnxruntime_ep_nv_tensorrt_rtx/` (x86_64); `ep-plugins/trt-rtx-arm64/onnxruntime_ep_nv_tensorrt_rtx/` (aarch64) | | TensorRT-RTX on WSL2 | same | WSL2 with GPU passthrough (`/dev/dxg`) | LinML patched build `0.4.0-wsl2p2` (GitHub release; glibc 2.35+, the same floor as the wheel), used alongside the wheel above | `ep-plugins/trt-rtx-wsl2/linml_trt_rtx_ep/` | | MIGraphX | `MIGraphXExecutionProvider` | Linux x86_64 with an AMD GPU and ROCm | LinML build `2611.1` (GitHub release), built from AMD's `onnxruntime-ep-amdgpu` | `ep-plugins/migraphx/linml_migraphx_ep/` | | QNN | `QNNExecutionProvider` | Linux aarch64 with a Qualcomm Snapdragon NPU | Qualcomm `onnxruntime-qnn` 2.6.0 tarball (GitHub release), with the QAIRT 2.50.40 runtime bundled | `ep-plugins/qnn/onnxruntime_qnn/` | @@ -246,14 +246,14 @@ These commands use only `curl`, `sha256sum`, `unzip`, and `mktemp`. Each downloa TMP="$(mktemp -d)" case "$(uname -m)" in x86_64) - WHEEL=onnxruntime_ep_nv_tensorrt_rtx_cu13-0.4.0-py3-none-manylinux_2_28_x86_64.whl - URL=https://files.pythonhosted.org/packages/9c/68/093626a054300fa7f4f1a6d18b775d185034c6bb7fb056ac4f9d1e9b50d4/$WHEEL - SHA256=68b8f7306b50cf76daf997b8be24ec3813498c15d48e0ddd211db14f2a9daa19 + WHEEL=onnxruntime_ep_nv_tensorrt_rtx_cu13-0.5.0-py3-none-manylinux_2_28_x86_64.whl + URL=https://files.pythonhosted.org/packages/6b/ea/033a6d73b74a443cbf67247e3700b2a2f5076b37af4b00e433fbbd46fa6d/$WHEEL + SHA256=32f575c633d171b4edf4b53d3bb22b72a6c9c0a850d2982ce9a788a5d40d3298 DIR=trt-rtx ;; aarch64) - WHEEL=onnxruntime_ep_nv_tensorrt_rtx_cu13-0.4.0-py3-none-manylinux_2_35_aarch64.whl - URL=https://files.pythonhosted.org/packages/41/63/ef110d73730969957740d770b29c158ca79a9df32c65ee081b01f8b64124/$WHEEL - SHA256=0ccf6a636f9f49d74e3e7e3b1b90287aa1c818db770b47fad6e8aa344f4290a5 + WHEEL=onnxruntime_ep_nv_tensorrt_rtx_cu13-0.5.0-py3-none-manylinux_2_35_aarch64.whl + URL=https://files.pythonhosted.org/packages/9c/5e/9b63610651a0f5d2c8712b84f64c8cee3b3b83564cc20be207d00272d982/$WHEEL + SHA256=1f0bb2b89bb3dfd43096fccb8ec1f581c266c5852c4e6ba672b7bf790c8c96c2 DIR=trt-rtx-arm64 # The aarch64 wheel is manylinux_2_35: refuse an older glibc before downloading it. GLIBC="$(getconf GNU_LIBC_VERSION 2>/dev/null | awk '{print $2}')" || GLIBC="" @@ -263,15 +263,21 @@ These commands use only `curl`, `sha256sum`, `unzip`, and `mktemp`. Each downloa *) echo "No TensorRT-RTX build for $(uname -m)" >&2; false ;; esac OUT="$DEPS/ep-plugins/$DIR/onnxruntime_ep_nv_tensorrt_rtx" - mkdir -p "$OUT" + # Extract into a staging directory and swap it in, so an upgrade does not leave an + # earlier build's runtime (libtensorrt_rtx.so.1.6.1 from 0.4.0) beside the new one. + mkdir -p "$(dirname "$OUT")" + STAGE="$(mktemp -d "$OUT.partial.XXXXXX")" curl -fsSL -o "$TMP/$WHEEL" "$URL" echo "$SHA256 $TMP/$WHEEL" | sha256sum -c - unzip -q -j -o "$TMP/$WHEEL" 'onnxruntime_ep_nv_tensorrt_rtx/*' -d "$OUT" + unzip -q -j -o "$TMP/$WHEEL" 'onnxruntime_ep_nv_tensorrt_rtx/*' -d "$STAGE" # The plugin links SONAMEs, but the wheel ships only versioned files: add the links. - (cd "$OUT" && - ln -sf libtensorrt_rtx.so.1.6.1 libtensorrt_rtx.so.1 && - ln -sf libtensorrt_onnxparser_rtx.so.1.6.1 libtensorrt_onnxparser_rtx.so.1 && + (cd "$STAGE" && + ln -sf libtensorrt_rtx.so.1.7.1 libtensorrt_rtx.so.1 && + ln -sf libtensorrt_onnxparser_rtx.so.1.7.1 libtensorrt_onnxparser_rtx.so.1 && ln -sf "$(ls libcudart.so.13.* | head -n 1)" libcudart.so.13) + chmod 755 "$STAGE" + rm -rf "$OUT" + mv "$STAGE" "$OUT" rm -rf "$TMP" echo "TensorRT-RTX installed to $OUT" ) @@ -790,7 +796,7 @@ for warning in report.compatibility_warnings: It covers: - **Built-in EPs**: `CPUExecutionProvider`, `CUDAExecutionProvider`, and `TensorrtExecutionProvider` -- **Plugin EPs**: `OpenVINOExecutionProvider` (1.6.1), `NvTensorRTRTXExecutionProvider` (0.4.0), and `MIGraphXExecutionProvider` (2611.1) +- **Plugin EPs**: `OpenVINOExecutionProvider` (1.6.1), `NvTensorRTRTXExecutionProvider` (0.5.0), and `MIGraphXExecutionProvider` (2611.1) ```csharp using LinML.Catalog; diff --git a/docs/compatibility.md b/docs/compatibility.md index 48169dd..5faf2bd 100644 --- a/docs/compatibility.md +++ b/docs/compatibility.md @@ -18,9 +18,9 @@ **TensorRT-RTX on aarch64.** NVIDIA's TensorRT-RTX 1.6 supports Ubuntu 24.04 SBSA (Linux aarch64) with CUDA 13.4, gcc 11.4, and compute capabilities 7.5, 8.6, 8.9, -12.0 and 12.13. LinML pins the same EP version (0.4.0) for `manylinux_2_35_aarch64` +12.0 and 12.13. LinML pins the same EP version as on x64 (0.5.0, TensorRT-RTX 1.7.1) for `manylinux_2_35_aarch64` (glibc 2.35 or newer), selected automatically on aarch64 hosts. The aarch64 build is -not yet verified on aarch64 hardware. The x64 pin is unchanged. +not yet verified on aarch64 hardware. ## Execution Providers @@ -135,9 +135,11 @@ not yet verified on aarch64 hardware. The x64 pin is unchanged. detects WSL2 and an RTX Ampere+ GPU. On WSL2 the CLI prefers the patched build for `trt-rtx` (as does `env-gpu-deps.sh`); `TRT_RTX_CUSTOM_EP` overrides it (also in the catalog since 0.4.2). Notes: - The archive contains only the Apache-2.0 EP plugin `.so` (~5 MB, RUNPATH - `$ORIGIN`); it dynamically links the TensorRT-RTX 1.6.1 runtime + `$ORIGIN`); it dynamically links the TensorRT-RTX runtime by SONAME (`libtensorrt_rtx.so.1`, `libtensorrt_onnxparser_rtx.so.1`) and CUDA 13, - which the stock `trt-rtx` wheel provides. + which the stock `trt-rtx` wheel provides. It is built from EP 0.4.0 against the + 1.6.1 SDK; since the stock pin moved to 0.5.0 it runs on that wheel's TensorRT-RTX + 1.7.1 runtime (verified in WSL2 on an RTX 5070: `TensorrtRtxWsl2EndToEndTests`). - Built by `scripts/build-trt-rtx-ep.sh`, which pins the PR head commit and produces a byte-stable archive. It is **not** built in CI: the TensorRT-RTX SDK (headers + import libs) is under NVIDIA's license and cannot be diff --git a/scripts/env-gpu-deps.sh b/scripts/env-gpu-deps.sh index e453fb8..2b8b61e 100644 --- a/scripts/env-gpu-deps.sh +++ b/scripts/env-gpu-deps.sh @@ -82,17 +82,23 @@ if [ -f "$TRT_RTX_EP" ]; then # remains the explicit override. if [ -n "${LINML_EP_LIBRARIES:-}" ]; then _linml_rest="" - _linml_old_ifs="$IFS" - IFS=';' - for _linml_pair in $LINML_EP_LIBRARIES; do + # Peel entries off one at a time rather than word-splitting on IFS, so + # this also works when sourced from zsh (which does not split $VAR). + _linml_todo="$LINML_EP_LIBRARIES" + while [ -n "$_linml_todo" ]; do + _linml_pair="${_linml_todo%%;*}" + case "$_linml_todo" in + *";"*) _linml_todo="${_linml_todo#*;}" ;; + *) _linml_todo="" ;; + esac + [ -n "$_linml_pair" ] || continue case "$_linml_pair" in NvTensorRTRTXExecutionProvider=*|nvtensorrtrtx=*) ;; # stale, drop *) _linml_rest="${_linml_rest:+$_linml_rest;}$_linml_pair" ;; esac done - IFS="$_linml_old_ifs" LINML_EP_LIBRARIES="$_linml_rest" - unset _linml_pair _linml_rest _linml_old_ifs + unset _linml_pair _linml_rest _linml_todo fi LINML_EP_LIBRARIES="NvTensorRTRTXExecutionProvider=$TRT_RTX_EP${LINML_EP_LIBRARIES:+;$LINML_EP_LIBRARIES}" export LINML_EP_LIBRARIES @@ -111,9 +117,16 @@ if [ -f "$OPENVINO_EP" ]; then _linml_has_ov="" if [ -n "${LINML_EP_LIBRARIES:-}" ]; then _linml_rest="" - _linml_old_ifs="$IFS" - IFS=';' - for _linml_pair in $LINML_EP_LIBRARIES; do + # Peel entries off one at a time rather than word-splitting on IFS, so + # this also works when sourced from zsh (which does not split $VAR). + _linml_todo="$LINML_EP_LIBRARIES" + while [ -n "$_linml_todo" ]; do + _linml_pair="${_linml_todo%%;*}" + case "$_linml_todo" in + *";"*) _linml_todo="${_linml_todo#*;}" ;; + *) _linml_todo="" ;; + esac + [ -n "$_linml_pair" ] || continue case "$_linml_pair" in openvino=*) ;; # stale alias, drop linml.openvino=*|OpenVINOExecutionProvider=*) @@ -122,9 +135,8 @@ if [ -f "$OPENVINO_EP" ]; then *) _linml_rest="${_linml_rest:+$_linml_rest;}$_linml_pair" ;; esac done - IFS="$_linml_old_ifs" LINML_EP_LIBRARIES="$_linml_rest" - unset _linml_pair _linml_rest _linml_old_ifs + unset _linml_pair _linml_rest _linml_todo fi if [ -z "$_linml_has_ov" ]; then LINML_EP_LIBRARIES="linml.openvino=$OPENVINO_EP${LINML_EP_LIBRARIES:+;$LINML_EP_LIBRARIES}" @@ -134,15 +146,58 @@ if [ -f "$OPENVINO_EP" ]; then fi # Convenience: default plugin EP library registration for MIGraphX -# (vendored by scripts/fetch-ep-plugins.sh or LinML.Catalog). +# (vendored by scripts/fetch-ep-plugins.sh or LinML.Catalog). The plugin links +# against ROCm's libmigraphx_c.so.3, so it is only mapped when that library +# resolves (loader path or ldconfig cache); without ROCm, MIGraphX is not +# applicable on this host (see `linml doctor`) and the plugin cannot load. +_linml_has_migraphx_runtime() { + # Written to work when sourced from zsh too, which does not word-split $VAR. + if printf '%s\n' "${LD_LIBRARY_PATH:-}" | tr ':' '\n' | while IFS= read -r _linml_dir; do + if [ -n "$_linml_dir" ] && [ -e "$_linml_dir/libmigraphx_c.so.3" ]; then echo found; fi + done | grep -q found; then + return 0 + fi + for _linml_ldconfig in ldconfig /sbin/ldconfig /usr/sbin/ldconfig; do + if command -v "$_linml_ldconfig" >/dev/null 2>&1; then + "$_linml_ldconfig" -p 2>/dev/null | grep -q 'libmigraphx_c\.so\.3 ' + _linml_rc=$? + unset _linml_ldconfig + return $_linml_rc + fi + done + unset _linml_ldconfig + return 1 +} MIGRAPHX_EP="$DEPS/ep-plugins/migraphx/linml_migraphx_ep/libmigraphx-ep.so" -if [ -f "$MIGRAPHX_EP" ]; then +if [ -f "$MIGRAPHX_EP" ] && _linml_has_migraphx_runtime; then case ";${LINML_EP_LIBRARIES:-};" in *";MIGraphXExecutionProvider="*|*";migraphx="*) ;; *) LINML_EP_LIBRARIES="MIGraphXExecutionProvider=$MIGRAPHX_EP${LINML_EP_LIBRARIES:+;$LINML_EP_LIBRARIES}" ;; esac export LINML_EP_LIBRARIES +elif [ -n "${LINML_EP_LIBRARIES:-}" ]; then + # Drop the vendored mapping a previous source (or an older version of this + # script) exported; any other MIGraphX mapping is the user's and is kept. + # Walked entry by entry like the TensorRT-RTX and OpenVINO cleanups above. + _linml_rest="" + _linml_todo="$LINML_EP_LIBRARIES" + while [ -n "$_linml_todo" ]; do + _linml_pair="${_linml_todo%%;*}" + case "$_linml_todo" in + *";"*) _linml_todo="${_linml_todo#*;}" ;; + *) _linml_todo="" ;; + esac + [ -n "$_linml_pair" ] || continue + case "$_linml_pair" in + "MIGraphXExecutionProvider=$MIGRAPHX_EP") ;; # stale vendored mapping, drop + *) _linml_rest="${_linml_rest:+$_linml_rest;}$_linml_pair" ;; + esac + done + LINML_EP_LIBRARIES="$_linml_rest" + export LINML_EP_LIBRARIES + unset _linml_pair _linml_rest _linml_todo fi +unset -f _linml_has_migraphx_runtime # Convenience: default plugin EP library registration for Qualcomm QNN diff --git a/scripts/fetch-ep-plugins.sh b/scripts/fetch-ep-plugins.sh index bf21dc1..e30b645 100644 --- a/scripts/fetch-ep-plugins.sh +++ b/scripts/fetch-ep-plugins.sh @@ -4,7 +4,7 @@ # ~/.local/linml-deps/ep-plugins/: # # trt-rtx NvTensorRTRTXExecutionProvider (NVIDIA). The wheel bundles the EP -# plugin .so AND the TensorRT-RTX 1.6.1 runtime libraries, so no +# plugin .so AND the TensorRT-RTX 1.7.1 runtime libraries, so no # separate TensorRT-RTX SDK install is needed. x86_64 hosts get the # x86_64 wheel; Linux aarch64 (SBSA) hosts get the aarch64 wheel. # openvino OpenVINOExecutionProvider (Intel onnxruntime-ep-openvino). The @@ -42,18 +42,27 @@ DEPS="${HOME}/.local/linml-deps" TMP_ROOT="$(mktemp -d)" trap 'rm -rf "$TMP_ROOT"' EXIT +# The TensorRT-RTX runtime the trt-rtx wheels bundle (same on both architectures). +TRT_RTX_RUNTIME_VERSION="1.7.1" +TRT_RTX_RUNTIME="libtensorrt_rtx.so.${TRT_RTX_RUNTIME_VERSION}" + fetch_trt_rtx() { local OUT="$DEPS/ep-plugins/trt-rtx/onnxruntime_ep_nv_tensorrt_rtx" - local WHEEL_URL="https://files.pythonhosted.org/packages/9c/68/093626a054300fa7f4f1a6d18b775d185034c6bb7fb056ac4f9d1e9b50d4/onnxruntime_ep_nv_tensorrt_rtx_cu13-0.4.0-py3-none-manylinux_2_28_x86_64.whl" - local WHEEL_SHA256="68b8f7306b50cf76daf997b8be24ec3813498c15d48e0ddd211db14f2a9daa19" - local WHEEL_NAME="onnxruntime_ep_nv_tensorrt_rtx_cu13-0.4.0-py3-none-manylinux_2_28_x86_64.whl" + local WHEEL_URL="https://files.pythonhosted.org/packages/6b/ea/033a6d73b74a443cbf67247e3700b2a2f5076b37af4b00e433fbbd46fa6d/onnxruntime_ep_nv_tensorrt_rtx_cu13-0.5.0-py3-none-manylinux_2_28_x86_64.whl" + local WHEEL_SHA256="32f575c633d171b4edf4b53d3bb22b72a6c9c0a850d2982ce9a788a5d40d3298" + local WHEEL_NAME="onnxruntime_ep_nv_tensorrt_rtx_cu13-0.5.0-py3-none-manylinux_2_28_x86_64.whl" local TMP="$TMP_ROOT/trt-rtx" echo "=== LinML EP plugin fetch (TensorRT-RTX) ===" + # The wheel records no version; the versioned TensorRT-RTX runtime it bundles + # marks the build, so an earlier wheel in the same directory is replaced. if [ -f "$OUT/libonnxruntime_providers_nv_tensorrt_rtx.so" ]; then - echo "Already vendored: $OUT" - return 0 + if [ -f "$OUT/$TRT_RTX_RUNTIME" ]; then + echo "Already vendored: $OUT" + return 0 + fi + echo "Vendored build at $OUT does not bundle $TRT_RTX_RUNTIME; replacing it." fi # Staged like the OpenVINO fetch: a failed extract never leaves a @@ -89,12 +98,18 @@ PYEOF # versioned files only. Create the SONAME symlinks next to the libraries. local lib for lib in libtensorrt_rtx libtensorrt_onnxparser_rtx; do - if [ -f "$STAGE/${lib}.so.1.6.1" ] && [ ! -e "$STAGE/${lib}.so.1" ]; then - ln -s "${lib}.so.1.6.1" "$STAGE/${lib}.so.1" + if [ -f "$STAGE/${lib}.so.${TRT_RTX_RUNTIME_VERSION}" ] && [ ! -e "$STAGE/${lib}.so.1" ]; then + ln -s "${lib}.so.${TRT_RTX_RUNTIME_VERSION}" "$STAGE/${lib}.so.1" fi done - if [ -f "$STAGE/libcudart.so.13.4.36" ] && [ ! -e "$STAGE/libcudart.so.13" ]; then - ln -s "libcudart.so.13.4.36" "$STAGE/libcudart.so.13" + # The CUDA runtime is patch-versioned (libcudart.so.13.4.36 in 0.5.0); link + # whatever patch version the wheel ships, as fetch_trt_rtx_arm64 does. + local cudart + if [ ! -e "$STAGE/libcudart.so.13" ]; then + cudart="$(compgen -G "$STAGE/libcudart.so.13.*" | head -n 1 || true)" + if [ -n "$cudart" ]; then + ln -s "$(basename "$cudart")" "$STAGE/libcudart.so.13" + fi fi rm -rf "$OUT" @@ -115,9 +130,9 @@ PYEOF fetch_trt_rtx_arm64() { local OUT="$DEPS/ep-plugins/trt-rtx-arm64/onnxruntime_ep_nv_tensorrt_rtx" - local WHEEL_URL="https://files.pythonhosted.org/packages/41/63/ef110d73730969957740d770b29c158ca79a9df32c65ee081b01f8b64124/onnxruntime_ep_nv_tensorrt_rtx_cu13-0.4.0-py3-none-manylinux_2_35_aarch64.whl" - local WHEEL_SHA256="0ccf6a636f9f49d74e3e7e3b1b90287aa1c818db770b47fad6e8aa344f4290a5" - local WHEEL_NAME="onnxruntime_ep_nv_tensorrt_rtx_cu13-0.4.0-py3-none-manylinux_2_35_aarch64.whl" + local WHEEL_URL="https://files.pythonhosted.org/packages/9c/5e/9b63610651a0f5d2c8712b84f64c8cee3b3b83564cc20be207d00272d982/onnxruntime_ep_nv_tensorrt_rtx_cu13-0.5.0-py3-none-manylinux_2_35_aarch64.whl" + local WHEEL_SHA256="1f0bb2b89bb3dfd43096fccb8ec1f581c266c5852c4e6ba672b7bf790c8c96c2" + local WHEEL_NAME="onnxruntime_ep_nv_tensorrt_rtx_cu13-0.5.0-py3-none-manylinux_2_35_aarch64.whl" local TMP="$TMP_ROOT/trt-rtx" echo "=== LinML EP plugin fetch (TensorRT-RTX) ===" @@ -130,9 +145,14 @@ fetch_trt_rtx_arm64() { return 1 fi + # The wheel records no version; the versioned TensorRT-RTX runtime it bundles + # marks the build, so an earlier wheel in the same directory is replaced. if [ -f "$OUT/libonnxruntime_providers_nv_tensorrt_rtx.so" ]; then - echo "Already vendored: $OUT" - return 0 + if [ -f "$OUT/$TRT_RTX_RUNTIME" ]; then + echo "Already vendored: $OUT" + return 0 + fi + echo "Vendored build at $OUT does not bundle $TRT_RTX_RUNTIME; replacing it." fi # Staged like the OpenVINO fetch: a failed extract never leaves a @@ -168,8 +188,8 @@ PYEOF # versioned files only. Create the SONAME symlinks next to the libraries. local lib for lib in libtensorrt_rtx libtensorrt_onnxparser_rtx; do - if [ -f "$STAGE/${lib}.so.1.6.1" ] && [ ! -e "$STAGE/${lib}.so.1" ]; then - ln -s "${lib}.so.1.6.1" "$STAGE/${lib}.so.1" + if [ -f "$STAGE/${lib}.so.${TRT_RTX_RUNTIME_VERSION}" ] && [ ! -e "$STAGE/${lib}.so.1" ]; then + ln -s "${lib}.so.${TRT_RTX_RUNTIME_VERSION}" "$STAGE/${lib}.so.1" fi done # The aarch64 wheel ships libcudart.so.13.4.46; link whatever patch version it has. @@ -368,8 +388,9 @@ fetch_trt_rtx_wsl2() { echo "=== LinML EP plugin fetch (TensorRT-RTX, WSL2 patch) ===" - # The patched EP dynamically links the TensorRT-RTX 1.6.1 runtime libraries, - # which only the stock wheel bundles; vendor the wheel first. + # The patched EP dynamically links the TensorRT-RTX runtime libraries (by + # SONAME, libtensorrt_rtx.so.1), which only the stock wheel bundles; vendor + # the wheel first. fetch_trt_rtx # The library alone does not say which build it is: an earlier one sits in the diff --git a/src/LinML.Catalog/PluginExecutionProvider.cs b/src/LinML.Catalog/PluginExecutionProvider.cs index c694172..1fd2345 100644 --- a/src/LinML.Catalog/PluginExecutionProvider.cs +++ b/src/LinML.Catalog/PluginExecutionProvider.cs @@ -81,6 +81,9 @@ private Variant SelectVariant() internal static bool Wsl2OptedOut() => EnvReader(Wsl2OptOutEnvVar)?.Trim().ToLowerInvariant() is "0" or "false" or "off" or "no"; + // A stock library is vendored but is not the pinned build (its version marker or manifest says so). + private bool StaleStockLibrary() => StockLibraryPath() is { } stock && !_spec.IsCurrent(stock); + private string? StockLibraryPath() => File.Exists(_spec.LibraryPath(_installRoot)) ? _spec.LibraryPath(_installRoot) @@ -272,9 +275,10 @@ public async Task EnsureReadyAsync(EnsureOptions options, bool downloaded = false; // Why a vendored build of another version could not be replaced, when it stays in use instead. string? updateFailure = null; + string? updateVersion = null; string WithUpdateNote(string detail) => updateFailure is null ? detail - : $"{detail} Could not replace the vendored build with {_wsl2Spec.Version}: {updateFailure}"; + : $"{detail} Could not replace the vendored build with {updateVersion}: {updateFailure}"; Variant variant = SelectVariant(); if (variant != Variant.Custom) { @@ -287,6 +291,10 @@ string WithUpdateNote(string detail) => updateFailure is null { needed.Add((_spec, false)); } + else if (options.AllowDownloads && StaleStockLibrary()) + { + needed.Add((_spec, true)); + } // A vendored build of another version (the pin moved on; 0.4.0-wsl2p1 needs a newer glibc than // most hosts have) is replaced when downloads are allowed. If it cannot be replaced, or @@ -304,6 +312,12 @@ string WithUpdateNote(string detail) => updateFailure is null { needed.Add((_spec, false)); } + else if (options.AllowDownloads && StaleStockLibrary()) + { + // An earlier stock build (0.4.0 before the TensorRT-RTX 0.5.0 pin) is replaced the same way; + // if that fails it stays in use. + needed.Add((_spec, true)); + } foreach ((PluginPackageSpec spec, bool hasFallback) in needed) { @@ -323,6 +337,7 @@ string WithUpdateNote(string detail) => updateFailure is null if (hasFallback) { updateFailure = ex.Message; + updateVersion = spec.Version; continue; } @@ -334,6 +349,7 @@ string WithUpdateNote(string detail) => updateFailure is null if (hasFallback) { updateFailure = $"timed out: {ex.Message}"; + updateVersion = spec.Version; continue; } diff --git a/src/LinML.Catalog/PluginInstaller.cs b/src/LinML.Catalog/PluginInstaller.cs index 97e156f..98807db 100644 --- a/src/LinML.Catalog/PluginInstaller.cs +++ b/src/LinML.Catalog/PluginInstaller.cs @@ -268,22 +268,22 @@ internal static void CreateRequiredSymlinks(string directory) return; } - string[] sonamePairs = - [ - "libtensorrt_rtx.so.1.6.1", "libtensorrt_rtx.so.1", - "libtensorrt_onnxparser_rtx.so.1.6.1", "libtensorrt_onnxparser_rtx.so.1" - ]; - - for (int i = 0; i < sonamePairs.Length; i += 2) + // The TensorRT-RTX runtime is versioned (libtensorrt_rtx.so.1.7.1 in the 0.5.0 wheel, 1.6.1 in 0.4.0), + // so link whichever version the directory holds to the SONAME the EP needs. + foreach (string library in (string[])["libtensorrt_rtx", "libtensorrt_onnxparser_rtx"]) { - string targetName = sonamePairs[i]; - string linkName = sonamePairs[i + 1]; - string targetPath = Path.Combine(directory, targetName); - string linkPath = Path.Combine(directory, linkName); + string linkPath = Path.Combine(directory, library + ".so.1"); + if (File.Exists(linkPath)) + { + continue; + } - if (File.Exists(targetPath) && !File.Exists(linkPath)) + string? target = Directory.EnumerateFiles(directory, library + ".so.1.*") + .OrderBy(path => path, StringComparer.Ordinal) + .LastOrDefault(); + if (target is not null) { - File.CreateSymbolicLink(linkPath, targetName); + File.CreateSymbolicLink(linkPath, Path.GetFileName(target)); } } diff --git a/src/LinML.Catalog/PluginPackageSpec.cs b/src/LinML.Catalog/PluginPackageSpec.cs index 3b91dc4..6a32fcb 100644 --- a/src/LinML.Catalog/PluginPackageSpec.cs +++ b/src/LinML.Catalog/PluginPackageSpec.cs @@ -11,6 +11,8 @@ namespace LinML.Catalog; /// MinimumGlibc, when set, is the oldest glibc the wheel loads on. /// VersionManifestFile, when set, names a JSON file in the package whose epVersion records the /// version it was built as: an install only counts as vendored when that equals Version. +/// VersionMarkerFile, when set, names a file in the package that only the pinned build ships (for a +/// wheel with no manifest, its versioned runtime library): an install without it is an earlier build. /// internal sealed record PluginPackageSpec( ProviderId Id, @@ -24,7 +26,8 @@ internal sealed record PluginPackageSpec( Version MinimumOrtVersion, string Architecture = "x64", Version? MinimumGlibc = null, - string? VersionManifestFile = null) + string? VersionManifestFile = null, + string? VersionMarkerFile = null) { public string InstallDirectory(string installRoot) => Path.Combine(installRoot, InstallSubdirectory); @@ -59,13 +62,16 @@ internal sealed record PluginPackageSpec( /// /// Whether the library at exists and is the pinned build. A package without - /// a has nothing to compare, so any library counts. With one, the vendored - /// version must equal : successive builds share one install directory, and a library - /// that is merely present may be an earlier build the pin replaced. + /// a or has nothing to compare, so any + /// library counts. With a manifest, the vendored version must equal ; with a marker, the + /// marker must sit beside the library: successive builds share one install directory, and a library that + /// is merely present may be an earlier build the pin replaced. /// public bool IsCurrent(string libraryPath) => File.Exists(libraryPath) - && (VersionManifestFile is null || string.Equals(VendoredVersion(libraryPath), Version, StringComparison.Ordinal)); + && (VersionManifestFile is null || string.Equals(VendoredVersion(libraryPath), Version, StringComparison.Ordinal)) + && (VersionMarkerFile is null + || File.Exists(Path.Combine(Path.GetDirectoryName(libraryPath) ?? string.Empty, VersionMarkerFile))); /// Whether the pinned build is vendored under . public bool IsVendored(string installRoot) => IsCurrent(LibraryPath(installRoot)); @@ -152,37 +158,44 @@ internal static class KnownPlugins // NVIDIA TensorRT-RTX standalone plugin EP. Works on RTX Ampere+ GPUs. The // wheel is per architecture: x86_64 for desktop and workstation hosts, and - // aarch64 for Linux SBSA hosts (TensorRT-RTX 1.6 supports Ubuntu 24.04 SBSA). + // aarch64 for Linux SBSA hosts (TensorRT-RTX supports Ubuntu 24.04 SBSA). // Under WSL2, stock PyPI wheels register but expose zero devices due to - // virtual PCI bus mapping; see docs/compatibility.md. + // virtual PCI bus mapping; see docs/compatibility.md. The wheel has no version + // manifest, so its versioned TensorRT-RTX runtime marks the build: a 0.4.0 + // install (TensorRT-RTX 1.6.1) in the same directory is replaced. + // The TensorRT-RTX runtime the 0.5.0 wheels bundle (both architectures). + private const string TensorrtRtxRuntimeMarker = "libtensorrt_rtx.so.1.7.1"; + internal static readonly PluginPackageSpec TensorrtRtxLinuxX64 = new( Id: ProviderId.TensorrtRtx, RegistrationName: "NvTensorRTRTXExecutionProvider", - Version: "0.4.0", - WheelUrl: new Uri("https://files.pythonhosted.org/packages/9c/68/093626a054300fa7f4f1a6d18b775d185034c6bb7fb056ac4f9d1e9b50d4/onnxruntime_ep_nv_tensorrt_rtx_cu13-0.4.0-py3-none-manylinux_2_28_x86_64.whl"), - WheelSha256: "68b8f7306b50cf76daf997b8be24ec3813498c15d48e0ddd211db14f2a9daa19", + Version: "0.5.0", + WheelUrl: new Uri("https://files.pythonhosted.org/packages/6b/ea/033a6d73b74a443cbf67247e3700b2a2f5076b37af4b00e433fbbd46fa6d/onnxruntime_ep_nv_tensorrt_rtx_cu13-0.5.0-py3-none-manylinux_2_28_x86_64.whl"), + WheelSha256: "32f575c633d171b4edf4b53d3bb22b72a6c9c0a850d2982ce9a788a5d40d3298", PackagePrefix: "onnxruntime_ep_nv_tensorrt_rtx/", InstallSubdirectory: Path.Combine("ep-plugins", "trt-rtx", "onnxruntime_ep_nv_tensorrt_rtx"), LibraryFileName: "libonnxruntime_providers_nv_tensorrt_rtx.so", MinimumOrtVersion: new Version(1, 24, 0), - Architecture: "x64"); + Architecture: "x64", + VersionMarkerFile: TensorrtRtxRuntimeMarker); // The same EP for Linux aarch64 (SBSA). Same version and ABI, and the same - // bundled TensorRT-RTX 1.6.1 runtime, built for aarch64 (manylinux_2_35, so + // bundled TensorRT-RTX 1.7.1 runtime, built for aarch64 (manylinux_2_35, so // glibc 2.35 or newer). Installs beside the x64 build, not over it. Not yet // verified on aarch64 hardware. internal static readonly PluginPackageSpec TensorrtRtxLinuxArm64 = new( Id: ProviderId.TensorrtRtx, RegistrationName: "NvTensorRTRTXExecutionProvider", - Version: "0.4.0", - WheelUrl: new Uri("https://files.pythonhosted.org/packages/41/63/ef110d73730969957740d770b29c158ca79a9df32c65ee081b01f8b64124/onnxruntime_ep_nv_tensorrt_rtx_cu13-0.4.0-py3-none-manylinux_2_35_aarch64.whl"), - WheelSha256: "0ccf6a636f9f49d74e3e7e3b1b90287aa1c818db770b47fad6e8aa344f4290a5", + Version: "0.5.0", + WheelUrl: new Uri("https://files.pythonhosted.org/packages/9c/5e/9b63610651a0f5d2c8712b84f64c8cee3b3b83564cc20be207d00272d982/onnxruntime_ep_nv_tensorrt_rtx_cu13-0.5.0-py3-none-manylinux_2_35_aarch64.whl"), + WheelSha256: "1f0bb2b89bb3dfd43096fccb8ec1f581c266c5852c4e6ba672b7bf790c8c96c2", PackagePrefix: "onnxruntime_ep_nv_tensorrt_rtx/", InstallSubdirectory: Path.Combine("ep-plugins", "trt-rtx-arm64", "onnxruntime_ep_nv_tensorrt_rtx"), LibraryFileName: "libonnxruntime_providers_nv_tensorrt_rtx.so", MinimumOrtVersion: new Version(1, 24, 0), Architecture: "arm64", - MinimumGlibc: new Version(2, 35)); + MinimumGlibc: new Version(2, 35), + VersionMarkerFile: TensorrtRtxRuntimeMarker); /// The TensorRT-RTX pin for this host: the wheel built for its architecture. public static PluginPackageSpec TensorrtRtx => diff --git a/src/LinML.Cli/Commands/BenchCommand.cs b/src/LinML.Cli/Commands/BenchCommand.cs index a5ff90c..b04d545 100644 --- a/src/LinML.Cli/Commands/BenchCommand.cs +++ b/src/LinML.Cli/Commands/BenchCommand.cs @@ -179,8 +179,11 @@ private static List DetermineTargetProviders( { try { - var libs = ProviderArgs.ResolveEpLibraries(epLibraryArgs, p.Id, root); - ProviderArgs.EnsureEpLibrariesRegistered(libs); + // An unrelated inherited mapping that fails to load must not keep this candidate + // from registering; its own mapping failing still drops it from the list. + ProviderArgs.RegisterEpLibraries( + ProviderArgs.ResolveEpLibraries(epLibraryArgs, p.Id, root), + epLibraryArgs, requiredProvider: p.Id); } catch { @@ -243,8 +246,9 @@ private static BenchmarkResultDto RunSingleBenchmark( { try { - var epLibraries = ProviderArgs.ResolveEpLibraries(epLibraryArgs, providerId, root); - ProviderArgs.EnsureEpLibrariesRegistered(epLibraries); + var epLibraries = ProviderArgs.RegisterEpLibraries( + ProviderArgs.ResolveEpLibraries(epLibraryArgs, providerId, root), + epLibraryArgs, requiredProvider: providerId); var scopedOptions = ProviderArgs.OptionsForProvider(epOptions, providerId); // Cold start measurement (session creation + first execution) diff --git a/src/LinML.Cli/Commands/DoctorCommand.cs b/src/LinML.Cli/Commands/DoctorCommand.cs index df454d3..33ce07b 100644 --- a/src/LinML.Cli/Commands/DoctorCommand.cs +++ b/src/LinML.Cli/Commands/DoctorCommand.cs @@ -65,8 +65,11 @@ public static Command Create(ServiceProvider provider, Option jsonOption) : ExecutionProviderCatalog.Create(root); // Register plugin EP libraries before discovery so they are listed. - var epLibraries = ProviderArgs.ParseEpLibraries(epLibraryArgs); - ProviderArgs.EnsureEpLibrariesRegistered(epLibraries); + // A plugin library inherited from LINML_EP_LIBRARIES that fails to load is not + // fatal here: it is named on stderr, and the catalog section below reports a + // catalog plugin's failure under its provider. + ProviderArgs.RegisterEpLibraries( + ProviderArgs.ParseEpLibraries(epLibraryArgs), epLibraryArgs, warnings: Console.Error); // Pre-register available catalog plugins before discovery so smoke tests and discovery cover them ProviderArgs.RegisterCatalogPlugins(epLibraryArgs, root); @@ -89,7 +92,8 @@ public static Command Create(ServiceProvider provider, Option jsonOption) { var scopedOptions = ProviderArgs.OptionsForProvider(epOptions, p); // Per provider, as `run` resolves them: the catalog's library for a plugin, not only the explicit flags. - var providerLibraries = ProviderArgs.ResolveEpLibraries(epLibraryArgs, p, root); + var providerLibraries = ProviderArgs.RegisterEpLibraries( + ProviderArgs.ResolveEpLibraries(epLibraryArgs, p, root), epLibraryArgs); readinessReports.Add(smokeRunner.RunSmoke( p, modelIdentity, tensorInputs, options: scopedOptions, epLibraries: providerLibraries)); diff --git a/src/LinML.Cli/Commands/ProvidersCommand.cs b/src/LinML.Cli/Commands/ProvidersCommand.cs index be5b29d..f355dec 100644 --- a/src/LinML.Cli/Commands/ProvidersCommand.cs +++ b/src/LinML.Cli/Commands/ProvidersCommand.cs @@ -32,8 +32,8 @@ public static Command Create(ServiceProvider provider, Option jsonOption) // Register plugin EP libraries before discovery so they are listed, the same // way `doctor` does, so the two commands report the same providers. - var epLibraries = ProviderArgs.ParseEpLibraries(epLibraryArgs); - ProviderArgs.EnsureEpLibrariesRegistered(epLibraries); + ProviderArgs.RegisterEpLibraries( + ProviderArgs.ParseEpLibraries(epLibraryArgs), epLibraryArgs, warnings: Console.Error); ProviderArgs.RegisterCatalogPlugins(epLibraryArgs, string.IsNullOrWhiteSpace(root) ? null : root); var discovery = provider.GetRequiredService(); diff --git a/src/LinML.Cli/Commands/RunCommand.cs b/src/LinML.Cli/Commands/RunCommand.cs index be5fb7c..0481a2f 100644 --- a/src/LinML.Cli/Commands/RunCommand.cs +++ b/src/LinML.Cli/Commands/RunCommand.cs @@ -59,19 +59,24 @@ public static Command Create(ServiceProvider provider, Option jsonOption) } var preferredProvider = ProviderArgs.ParseProvider(providerName); - var explicitLibraries = ProviderArgs.ParseEpLibraries(epLibraryArgs); - ProviderArgs.EnsureEpLibrariesRegistered(explicitLibraries); + // An inherited mapping (LINML_EP_LIBRARIES) for another provider that fails to + // load here is skipped with a warning; the requested provider's own is not. + var explicitLibraries = ProviderArgs.RegisterEpLibraries( + ProviderArgs.ParseEpLibraries(epLibraryArgs), epLibraryArgs, warnings: Console.Error); - var epLibraries = ProviderArgs.ResolveEpLibraries(epLibraryArgs, preferredProvider); + IReadOnlyDictionary epLibraries; try { - ProviderArgs.EnsureEpLibrariesRegistered(epLibraries); + epLibraries = ProviderArgs.RegisterEpLibraries( + ProviderArgs.ResolveEpLibraries(epLibraryArgs, preferredProvider), + epLibraryArgs, requiredProvider: preferredProvider); } catch (Exception ex) { if (requireProvider) { - Console.Error.WriteLine($"Error: {ex.Message}"); + var hint = ProviderArgs.ProviderUnavailableHint(preferredProvider); + Console.Error.WriteLine($"Error: {ex.Message}{(hint is null ? string.Empty : " " + hint)}"); return 3; } epLibraries = explicitLibraries; diff --git a/src/LinML.Cli/Commands/ServeCommand.cs b/src/LinML.Cli/Commands/ServeCommand.cs index 35d696f..b14ff23 100644 --- a/src/LinML.Cli/Commands/ServeCommand.cs +++ b/src/LinML.Cli/Commands/ServeCommand.cs @@ -77,19 +77,24 @@ public static Command Create(ServiceProvider provider) } var preferredProvider = ProviderArgs.ParseProvider(providerName); - var explicitLibraries = ProviderArgs.ParseEpLibraries(epLibraryArgs); - ProviderArgs.EnsureEpLibrariesRegistered(explicitLibraries); + // An inherited mapping (LINML_EP_LIBRARIES) for another provider that fails to + // load here is skipped with a warning; the requested provider's own is not. + var explicitLibraries = ProviderArgs.RegisterEpLibraries( + ProviderArgs.ParseEpLibraries(epLibraryArgs), epLibraryArgs, warnings: Console.Error); - var epLibraries = ProviderArgs.ResolveEpLibraries(epLibraryArgs, preferredProvider); + IReadOnlyDictionary epLibraries; try { - ProviderArgs.EnsureEpLibrariesRegistered(epLibraries); + epLibraries = ProviderArgs.RegisterEpLibraries( + ProviderArgs.ResolveEpLibraries(epLibraryArgs, preferredProvider), + epLibraryArgs, requiredProvider: preferredProvider); } catch (Exception ex) { if (requireProvider) { - Console.Error.WriteLine($"Error: {ex.Message}"); + var hint = ProviderArgs.ProviderUnavailableHint(preferredProvider); + Console.Error.WriteLine($"Error: {ex.Message}{(hint is null ? string.Empty : " " + hint)}"); return 3; } epLibraries = explicitLibraries; diff --git a/src/LinML.Cli/InferencePipeline.cs b/src/LinML.Cli/InferencePipeline.cs index 254fb94..09907bd 100644 --- a/src/LinML.Cli/InferencePipeline.cs +++ b/src/LinML.Cli/InferencePipeline.cs @@ -263,14 +263,9 @@ public static OpenSessionOutcome OpenSession( internal static string FormatSelectionFailure(string? failureReason, ProviderId? preferredProvider) { var errorMsg = $"Provider selection failed: {failureReason ?? "Unknown reason"}"; - if (preferredProvider is not null) + if (ProviderArgs.ProviderUnavailableHint(preferredProvider) is { } hint) { - var spec = KnownPlugins.All.FirstOrDefault(p => p.Id == preferredProvider); - if (spec is not null && spec.FindLibrary() is null) - { - var shortId = ProviderArgs.GetShortName(preferredProvider); - errorMsg += $" Run 'linml catalog install {shortId}' to vendor missing libraries."; - } + errorMsg += " " + hint; } return errorMsg; } diff --git a/src/LinML.Cli/ProviderArgs.cs b/src/LinML.Cli/ProviderArgs.cs index bc9d7a3..b6c5afd 100644 --- a/src/LinML.Cli/ProviderArgs.cs +++ b/src/LinML.Cli/ProviderArgs.cs @@ -62,7 +62,17 @@ private static readonly (string[] Aliases, ProviderId Id)[] KnownProviders = "(or a raw ORT EP name like CUDAExecutionProvider)."); } - private static readonly List BundledLibraryHandles = []; + // Handles of bundled runtime libraries loaded ahead of a plugin, kept for the life of the + // process. A set, so resolving the same plugin again does not record a handle twice. + private static readonly HashSet BundledLibraryHandles = []; + + private static void KeepBundledLibraryHandles(IEnumerable handles) + { + lock (BundledLibraryHandles) + { + BundledLibraryHandles.UnionWith(handles); + } + } /// /// Gets the canonical short alias for a known provider ID. @@ -183,7 +193,7 @@ bool MatchesProvider(string key) => var pluginDir = Path.GetDirectoryName(libraryPath); if (!string.IsNullOrEmpty(pluginDir)) { - BundledLibraryHandles.AddRange( + KeepBundledLibraryHandles( PluginInstaller.LoadBundledLibraries(pluginDir, spec.LibraryFileName)); } @@ -215,7 +225,7 @@ private static bool IsStockWheelPath(string path, PluginPackageSpec spec) => return null; } - BundledLibraryHandles.AddRange(PluginInstaller.LoadBundledLibraries(runtimeDir, spec.LibraryFileName)); + KeepBundledLibraryHandles(PluginInstaller.LoadBundledLibraries(runtimeDir, spec.LibraryFileName)); return variant; } @@ -289,6 +299,88 @@ public static void EnsureEpLibrariesRegistered(IReadOnlyDictionary + /// Register plugin EP libraries one at a time, isolating failures. A library named on + /// the command line () or mapped for + /// still fails fast, as + /// does; any other library that fails to + /// register (typically one inherited from LINML_EP_LIBRARIES whose runtime is missing + /// on this host) is reported to and left out of the result, + /// so it cannot break commands that do not use it. + /// + /// The libraries that registered. + public static IReadOnlyDictionary RegisterEpLibraries( + IReadOnlyDictionary epLibraries, + IEnumerable epLibraryArgs, + ProviderId? requiredProvider = null, + TextWriter? warnings = null) + { + var explicitKeys = new HashSet(StringComparer.OrdinalIgnoreCase); + foreach (var arg in epLibraryArgs) + { + var eq = arg.IndexOf('='); + if (eq > 0) + { + explicitKeys.Add(arg[..eq].Trim()); + } + } + + var requiredSpec = requiredProvider is null + ? null + : LinML.Catalog.KnownPlugins.All.FirstOrDefault(p => p.Id == requiredProvider); + + var registered = new Dictionary(StringComparer.Ordinal); + foreach (var (name, path) in epLibraries) + { + bool required = explicitKeys.Contains(name) + || (requiredSpec is not null && IsMappingFor(name, requiredSpec)) + || (requiredProvider is not null && string.Equals(name, requiredProvider.Value, StringComparison.OrdinalIgnoreCase)); + if (required) + { + EnsureEpLibrariesRegistered(new Dictionary { [name] = path }); + registered[name] = path; + continue; + } + + try + { + PluginEpRegistry.EnsureRegistered(new Dictionary { [name] = path }); + registered[name] = path; + } + catch (Exception ex) when (ex is not OperationCanceledException) + { + warnings?.WriteLine($"Warning: skipping EP plugin library '{name}' ({path}): {ex.Message}"); + } + } + return registered; + } + + /// + /// Why cannot run here and what to do about it, as a sentence + /// to append to an error: the host lacks its hardware or architecture (no install hint, as + /// installing it would not help), or its plugin library is not installed. Null otherwise. + /// + public static string? ProviderUnavailableHint(ProviderId? provider, string? root = null) + { + var spec = LinML.Catalog.KnownPlugins.All.FirstOrDefault(p => p.Id == provider); + if (spec is null) + { + return null; + } + + string? gap = Commands.SetupCommand.ArchitectureNote( + spec, System.Runtime.InteropServices.RuntimeInformation.ProcessArchitecture.ToString().ToLowerInvariant()) + ?? Commands.SetupCommand.MigraphXHardwareNote(spec.Id, Commands.SetupCommand.ProbeAmdRocm()); + if (gap is not null) + { + return $"{GetShortName(spec.Id)} is not applicable on this host: {gap}."; + } + + return spec.FindLibrary(root) is null + ? $"Run 'linml catalog install {GetShortName(spec.Id)}' to vendor missing libraries." + : null; + } + /// /// Registers, in this process, every catalog plugin that resolves on this host, so that /// discovery, readiness and catalog status all see the same EP devices. A plugin that diff --git a/tests/LinML.Catalog.Tests/PluginExecutionProviderTests.cs b/tests/LinML.Catalog.Tests/PluginExecutionProviderTests.cs index 04142d0..789e350 100644 --- a/tests/LinML.Catalog.Tests/PluginExecutionProviderTests.cs +++ b/tests/LinML.Catalog.Tests/PluginExecutionProviderTests.cs @@ -178,7 +178,7 @@ public void Catalog_ListsAllProviders_AndFindsThemById() IExecutionProvider? trtRtx = catalog.Find(Contracts.ProviderId.TensorrtRtx); Assert.NotNull(trtRtx); Assert.Equal(ProviderSource.Plugin, trtRtx.Source); - Assert.Equal("0.4.0", trtRtx.PinnedVersion); + Assert.Equal("0.5.0", trtRtx.PinnedVersion); IExecutionProvider? migraphx = catalog.Find(Contracts.ProviderId.MigraphX); Assert.NotNull(migraphx); @@ -310,4 +310,71 @@ public void HostGlibc_IsDetectedOnAGlibcLinuxHost() Assert.NotNull(PluginExecutionProvider.HostGlibc); Assert.True(PluginExecutionProvider.GlibcSatisfies(new Version(2, 0), PluginExecutionProvider.HostGlibc)); } + + // A stock wheel with no manifest (TensorRT-RTX) is told apart by its version marker. A 0.4.0 install + // predates the 0.5.0 pin: EnsureReadyAsync replaces it when downloads are allowed, as for the WSL2 build. + private static (PluginPackageSpec Spec, byte[] Wheel) MarkedStock() + { + byte[] wheel = FakeWheel.Build(); + return (FakeWheel.Spec(wheel) with { VersionMarkerFile = "libruntime.so.2621" }, wheel); + } + + private static void VendorEarlierStockBuild(string root, PluginPackageSpec spec) + { + Directory.CreateDirectory(spec.InstallDirectory(root)); + File.WriteAllText(spec.LibraryPath(root), "previous-build"); + File.WriteAllText(Path.Combine(spec.InstallDirectory(root), "libruntime.so.1999"), "previous-runtime"); + } + + [Fact] + public async Task EarlierStockBuildVendored_DownloadsAllowed_ReplacesItBeforeRegistering() + { + using var root = new TempDir(); + (PluginPackageSpec spec, byte[] wheel) = MarkedStock(); + VendorEarlierStockBuild(root.Path, spec); + var handler = new FakeWheelHandler(wheel); + var registrar = new FakeRegistrar(); + var provider = new PluginExecutionProvider(spec, root.Path, new PluginInstaller(handler), registrar, () => true); + + await provider.EnsureReadyAsync(new EnsureOptions(AllowDownloads: true)); + + Assert.Equal(1, handler.Requests); + Assert.Equal("plugin", File.ReadAllText(spec.LibraryPath(root.Path))); + Assert.False(File.Exists(Path.Combine(spec.InstallDirectory(root.Path), "libruntime.so.1999"))); + Assert.Equal(spec.LibraryPath(root.Path), Assert.Single(registrar.Registrations).Path); + } + + [Fact] + public async Task EarlierStockBuildVendored_DownloadsDisallowed_KeepsUsingIt() + { + using var root = new TempDir(); + (PluginPackageSpec spec, byte[] wheel) = MarkedStock(); + VendorEarlierStockBuild(root.Path, spec); + var handler = new FakeWheelHandler(wheel); + var registrar = new FakeRegistrar(); + var provider = new PluginExecutionProvider(spec, root.Path, new PluginInstaller(handler), registrar, () => true); + + await provider.EnsureReadyAsync(new EnsureOptions(AllowDownloads: false)); + + Assert.Equal(0, handler.Requests); + Assert.Equal("previous-build", File.ReadAllText(spec.LibraryPath(root.Path))); + Assert.Equal(spec.LibraryPath(root.Path), Assert.Single(registrar.Registrations).Path); + } + + [Fact] + public async Task EarlierStockBuild_WhenTheUpdateCannotBeDownloaded_KeepsUsingIt_AndSaysWhy() + { + using var root = new TempDir(); + (PluginPackageSpec spec, _) = MarkedStock(); + VendorEarlierStockBuild(root.Path, spec); + var registrar = new FakeRegistrar(); + var provider = new PluginExecutionProvider(spec, root.Path, new PluginInstaller(new TimingOutHandler()), registrar, () => true); + + var result = await provider.EnsureReadyAsync(new EnsureOptions(AllowDownloads: true)); + + Assert.Equal("previous-build", File.ReadAllText(spec.LibraryPath(root.Path))); + Assert.Equal(spec.LibraryPath(root.Path), Assert.Single(registrar.Registrations).Path); + Assert.NotEqual(ProviderEnsureStatus.Failed, result.Status); + Assert.Contains($"Could not replace the vendored build with {spec.Version}", result.Detail); + } } diff --git a/tests/LinML.Catalog.Tests/PluginInstallerTests.cs b/tests/LinML.Catalog.Tests/PluginInstallerTests.cs index 426e099..773d07b 100644 --- a/tests/LinML.Catalog.Tests/PluginInstallerTests.cs +++ b/tests/LinML.Catalog.Tests/PluginInstallerTests.cs @@ -317,8 +317,11 @@ public async Task Install_ReplacesLeftoverDirectoryWithoutLibrary() Assert.Empty(Directory.GetDirectories(dir)); } - [Fact] - public void CreateRequiredSymlinks_CreatesRelativeSonameLinksOnLinux() + // Whichever TensorRT-RTX runtime the wheel bundles (1.7.1 in 0.5.0, 1.6.1 in 0.4.0) gets its SONAME link. + [Theory] + [InlineData("1.7.1")] + [InlineData("1.6.1")] + public void CreateRequiredSymlinks_CreatesRelativeSonameLinksOnLinux(string runtimeVersion) { if (!OperatingSystem.IsLinux()) { @@ -326,16 +329,58 @@ public void CreateRequiredSymlinks_CreatesRelativeSonameLinksOnLinux() } using var root = new TempDir(); - string targetFile = Path.Combine(root.Path, "libtensorrt_rtx.so.1.6.1"); - File.WriteAllText(targetFile, "dummy"); + foreach (string library in (string[])["libtensorrt_rtx", "libtensorrt_onnxparser_rtx"]) + { + File.WriteAllText(Path.Combine(root.Path, $"{library}.so.{runtimeVersion}"), "dummy"); + } PluginInstaller.CreateRequiredSymlinks(root.Path); - string linkFile = Path.Combine(root.Path, "libtensorrt_rtx.so.1"); - Assert.True(File.Exists(linkFile)); - var linkTarget = File.ResolveLinkTarget(linkFile, returnFinalTarget: false); - Assert.NotNull(linkTarget); - Assert.Equal("libtensorrt_rtx.so.1.6.1", linkTarget.Name); + foreach (string library in (string[])["libtensorrt_rtx", "libtensorrt_onnxparser_rtx"]) + { + string linkFile = Path.Combine(root.Path, library + ".so.1"); + Assert.True(File.Exists(linkFile)); + var linkTarget = File.ResolveLinkTarget(linkFile, returnFinalTarget: false); + Assert.NotNull(linkTarget); + Assert.Equal($"{library}.so.{runtimeVersion}", linkTarget.Name); + } + } + + // A wheel with no manifest (the stock TensorRT-RTX wheel) is told apart by a file only the pinned build + // ships. 0.4.0 and 0.5.0 share one install directory, so a 0.4.0 install must be replaced, not kept. + private static PluginPackageSpec Marked(byte[] wheel) => + FakeWheel.Spec(wheel) with { VersionMarkerFile = "libruntime.so.2621" }; + + [Fact] + public async Task Install_MarkedPackage_ReplacesAnInstallWithoutTheMarker() + { + using var root = new TempDir(); + byte[] wheel = FakeWheel.Build(); + PluginPackageSpec pinned = Marked(wheel); + VendorByHand(pinned, root.Path, buildInfoVersion: null); + Assert.False(pinned.IsVendored(root.Path)); + + bool installed = await new PluginInstaller(new FakeWheelHandler(wheel)).InstallAsync(pinned, root.Path, CancellationToken.None); + + Assert.True(installed); + Assert.Equal("plugin", File.ReadAllText(pinned.LibraryPath(root.Path))); + Assert.True(pinned.IsVendored(root.Path)); + } + + [Fact] + public async Task Install_MarkedPackage_KeepsAnInstallWithTheMarker() + { + using var root = new TempDir(); + byte[] wheel = FakeWheel.Build(); + PluginPackageSpec pinned = Marked(wheel); + var handler = new FakeWheelHandler(wheel); + var installer = new PluginInstaller(handler); + await installer.InstallAsync(pinned, root.Path, CancellationToken.None); + + bool second = await installer.InstallAsync(pinned, root.Path, CancellationToken.None); + + Assert.False(second); + Assert.Equal(1, handler.Requests); } [Fact] @@ -351,7 +396,7 @@ public async Task Install_WhenAlreadyPresent_RepairsMissingSymlinks() string installDir = spec.InstallDirectory(root.Path); Directory.CreateDirectory(installDir); File.WriteAllText(spec.LibraryPath(root.Path), "dummy lib"); - File.WriteAllText(Path.Combine(installDir, "libtensorrt_rtx.so.1.6.1"), "dummy soname"); + File.WriteAllText(Path.Combine(installDir, spec.VersionMarkerFile!), "dummy soname"); var installer = new PluginInstaller(); bool installed = await installer.InstallAsync(spec, root.Path, CancellationToken.None); diff --git a/tests/LinML.Cli.Tests/ExitCodeTests.cs b/tests/LinML.Cli.Tests/ExitCodeTests.cs index eed741b..b61f86a 100644 --- a/tests/LinML.Cli.Tests/ExitCodeTests.cs +++ b/tests/LinML.Cli.Tests/ExitCodeTests.cs @@ -1,4 +1,5 @@ using System.Text.Json; +using LinML.Catalog; using LinML.Cli.Commands; using Xunit; @@ -224,8 +225,25 @@ public async Task Run_MissingRequiredPlugin_Returns3_WithInstallHint() "--provider", "migraphx", "--require-provider")); - Assert.Equal(3, exitCode); - Assert.Contains("Run 'linml catalog install migraphx' to vendor missing libraries.", stderr); + if (SetupCommand.ProbeAmdRocm()) + { + if (KnownPlugins.MigraphX.FindLibrary() is not null) + { + return; // vendored on a ROCm host: the provider may really run here + } + + Assert.Equal(3, exitCode); + Assert.Contains("Run 'linml catalog install migraphx' to vendor missing libraries.", stderr); + } + else + { + // Without ROCm, MIGraphX is not applicable whether or not its plugin is vendored + // (vendored, it fails to load for want of libmigraphx_c.so.3); installing it would + // not help, so there is no install hint, as in `doctor`. + Assert.Equal(3, exitCode); + Assert.Contains("migraphx is not applicable on this host", stderr); + Assert.DoesNotContain("linml catalog install migraphx", stderr); + } } [Fact] diff --git a/tests/LinML.Cli.Tests/ProviderArgsTests.cs b/tests/LinML.Cli.Tests/ProviderArgsTests.cs index 1231ece..f4e34f8 100644 --- a/tests/LinML.Cli.Tests/ProviderArgsTests.cs +++ b/tests/LinML.Cli.Tests/ProviderArgsTests.cs @@ -9,8 +9,16 @@ namespace LinML.Cli.Tests; /// Tests for ProviderArgs: provider aliases, EP library and EP option parsing. /// [Collection("WslProbe")] -public class ProviderArgsTests +public class ProviderArgsTests : IDisposable { + // A shell that sourced scripts/env-gpu-deps.sh inherits real plugin mappings; each test + // starts without them (tests that need one set it) so results do not depend on the host. + private readonly string? _inheritedEpLibraries = Environment.GetEnvironmentVariable(ProviderArgs.EpLibrariesEnvVar); + + public ProviderArgsTests() => Environment.SetEnvironmentVariable(ProviderArgs.EpLibrariesEnvVar, null); + + public void Dispose() => Environment.SetEnvironmentVariable(ProviderArgs.EpLibrariesEnvVar, _inheritedEpLibraries); + [Theory] [InlineData("cpu", "CPUExecutionProvider")] [InlineData("cuda", "CUDAExecutionProvider")] @@ -209,6 +217,39 @@ public void EnsureEpLibrariesRegistered_MissingFile_Throws() new Dictionary { ["bogus"] = "/nonexistent/libbogus.so" })); } + [Fact] + public void RegisterEpLibraries_InheritedFailure_IsSkippedWithWarning() + { + var warnings = new StringWriter(); + + var registered = ProviderArgs.RegisterEpLibraries( + new Dictionary { ["bogus"] = "/nonexistent/libbogus.so" }, + epLibraryArgs: [], + warnings: warnings); + + Assert.Empty(registered); + Assert.Contains("skipping EP plugin library 'bogus'", warnings.ToString()); + } + + [Fact] + public void RegisterEpLibraries_ExplicitFailure_Throws() + { + Assert.Throws(() => + ProviderArgs.RegisterEpLibraries( + new Dictionary { ["bogus"] = "/nonexistent/libbogus.so" }, + epLibraryArgs: ["bogus=/nonexistent/libbogus.so"])); + } + + [Fact] + public void RegisterEpLibraries_RequiredProviderFailure_Throws() + { + Assert.Throws(() => + ProviderArgs.RegisterEpLibraries( + new Dictionary { ["MIGraphXExecutionProvider"] = "/nonexistent/libmigraphx-ep.so" }, + epLibraryArgs: [], + requiredProvider: ProviderId.MigraphX)); + } + [Fact] public void GetShortName_ReturnsExpectedAliases() {