Skip to content

feat(sys): require the configured native library with ZVEC_STRICT_LIB - #20

Open
mrcs64 wants to merge 3 commits into
zvec-ai:mainfrom
mrcs64:feat/strict-external-native-lib
Open

mrcs64 wants to merge 3 commits into
zvec-ai:mainfrom
mrcs64:feat/strict-external-native-lib

Conversation

@mrcs64

@mrcs64 mrcs64 commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Problem

ZVEC_LIB_DIR is documented as the highest priority source for the native library, but a value that cannot be used is not an error. From zvec-sys/build.rs:

    // 1. Environment variable (highest priority — for advanced users)
    if let Some(custom) = env::var_os("ZVEC_LIB_DIR") {
        let path = PathBuf::from(&custom);
        if has_zvec_lib(&path) {
            return Some(path);
        }
        println!(
            "cargo:warning=ZVEC_LIB_DIR={} does not contain the required native library; \
             continuing library resolution.",
            path.display()
        );
    }

Resolution then continues with the sibling checkout, the submodule, vendor/lib, a cached or downloaded prebuilt SDK, or a source build. A typo, a stale path, or a directory that holds an incomplete SDK therefore links a different copy of the library than the one the environment selected, and the build succeeds. For a reproducible, audited or offline build that is the worst outcome: the mistake is silent.

The existing test invalid_explicit_library_directory_falls_back_to_vendor pins the current behavior deliberately, so this is an opt-in addition rather than a change.

Solution

ZVEC_STRICT_LIB=1 makes the explicit configuration the only accepted source. In resolve_lib_dir, the substitution chain is skipped entirely:

    let strict = strict_lib_dir_required();

    // 1. Environment variable (highest priority — for advanced users)
    if let Some(custom) = env::var_os("ZVEC_LIB_DIR") {
        ...
    }

    // Everything below substitutes another copy of the native library for the
    // configured one: a sibling checkout, the submodule, the vendor directory,
    // a cached or downloaded prebuilt SDK, or a source build. A strict build
    // forbids exactly that, so it stops here ...
    if strict {
        return None;
    }

and main reports the unresolved case as an error, whatever the acquisition settings say:

$ ZVEC_STRICT_LIB=1 ZVEC_LIB_DIR=/does-not-exist cargo build -p zvec-rust-sys
error: failed to run custom build command for `zvec-rust-sys v0.7.2`
  ZVEC_STRICT_LIB is set, but ZVEC_LIB_DIR=/does-not-exist does not contain the native
  library for x86_64-unknown-linux-gnu: expected libzvec_c_api.so. A strict build never
  falls back to a sibling zvec/ checkout, the submodule, the vendor directory, a cached
  or downloaded prebuilt SDK, or a source build. Point ZVEC_LIB_DIR at a directory that
  contains libzvec_c_api.so, or unset ZVEC_STRICT_LIB to use the normal resolution order.
  • The mode is opt-in and enabled only by 1 or the word true, the latter case-insensitively
    (TRUE, True); every other value -- including an empty one and the yes/on that other
    toolchains accept -- leaves the resolution order unchanged, so it cannot be switched on by
    accident.
  • Nothing can be acquired in strict mode: ZVEC_PREBUILT_URL, the prebuilt cache, ZVEC_REPO_URL and the source build all sit behind the early return.
  • Strict mode constrains the library, not the include path: ZVEC_INCLUDE_DIR resolution is
    deliberately unchanged -- a strict build can still take headers from a sibling checkout, which
    decides what downstream build scripts see rather than what the linker uses, so a build that must
    pin headers sets ZVEC_INCLUDE_DIR itself. DOCS_RS builds still return before resolution
    because they never link.
  • has_zvec_lib and the failure message now share one platform file-name table, so both name the same artifacts: libzvec_c_api.so, libzvec_c_api.dylib, zvec_c_api.lib (MSVC), or the MinGW .dll.a / .lib / .dll
    alternatives. Where more than one name is valid the failure message says "one of", so a MinGW
    message cannot read as if all three were required.
  • No new dependency and no manifest change: it is another ZVEC_* variable next to ZVEC_LIB_DIR and ZVEC_AUTO_BUILD, registered with cargo:rerun-if-env-changed.

Tests

zvec-sys/tests/build_script.rs gains eight cases, using the existing temporary-directory fixture:

Test Asserts
strict_mode_requires_an_explicit_library_directory without ZVEC_LIB_DIR the script fails and names the expected file; a valid vendor/lib substitute is not consulted and is not modified
strict_mode_rejects_an_incomplete_directory_and_never_falls_back a directory without the library fails the build, and the message names that directory
strict_mode_accepts_an_explicit_library_directory a valid directory resolves, and nothing is downloaded or built
strict_mode_stays_off_for_every_other_value "", 0, false, no keep the documented fallback to vendor/lib
strict_mode_never_reaches_acquisition with ZVEC_AUTO_BUILD=1, a working download path and a valid substitute present, the failure is still the strict one, no download tool ran and no source checkout was created
platform_library_names_are_honoured the file-name table for Linux, macOS, Windows MSVC and Windows GNU
strict_is_enabled_by_one_or_the_word_true_however_it_is_cased 1, true, TRUE, True enable the mode; yes, on, 2, a leading-space true and the empty string do not
the_expected_names_read_as_one_of_where_several_are_valid Linux and MSVC read as a plain name; Windows GNU reads as "one of ..." in the message too

The acquisition test keeps its clone URL pointed at a nonexistent local path, so a wrong implementation fails fast instead of fetching a repository. Disabling strict mode in the implementation makes exactly the three behavior-sensitive cases fail, so the suite discriminates rather than merely passing.

Validation

Linux x86_64, ZVEC_LIB_DIR pointing at the zvec-prebuilt-x86_64-unknown-linux-gnu SDK from the v0.7.2 release:

cargo fmt --all -- --check                              clean
cargo clippy --workspace --all-targets -- -D warnings   clean
cargo doc --workspace --no-deps                         no warnings
cargo test --workspace                                  232 passed, 0 failed, 1 ignored
cargo test -p zvec-rust-sys --test build_script          18 passed, 0 failed (10 before)
ZVEC_AUTO_BUILD=0 cargo check --workspace --lib          passes

Real build-script runs, not only in-process tests:

ZVEC_STRICT_LIB=1 ZVEC_LIB_DIR=<sdk> cargo build -p zvec-rust-sys
    -> Finished
ZVEC_STRICT_LIB=TRUE ZVEC_LIB_DIR=<sdk> cargo build -p zvec-rust-sys
    -> Finished (the documented casing)
ZVEC_STRICT_LIB=1 ZVEC_LIB_DIR=/does-not-exist cargo build -p zvec-rust-sys
    -> error: failed to run custom build command, with the strict message
ZVEC_STRICT_LIB=1 (no ZVEC_LIB_DIR) cargo build -p zvec-rust-sys
    -> error: failed to run custom build command, with the strict message
ZVEC_AUTO_BUILD=0 ZVEC_LIB_DIR=/does-not-exist cargo build -p zvec-rust-sys
    -> warning only; the existing fallback order is untouched

Compatibility and limitations

  • Default behavior is unchanged: the fallback order, the existing warning and the existing failure handling all stay as they are, and the existing tests for the download cache, extraction failures and the vendor fallback pass untouched.
  • Because the failure is raised when the build script runs, cargo check, cargo doc and cargo publish verification also fail while strict mode is on and the library is absent. That is the intended meaning of the mode; only DOCS_RS builds keep skipping resolution, as they do today.
  • Strict mode validates ZVEC_LIB_DIR by the same platform rule the linker uses; it does not verify ABI compatibility or the architecture of the library it finds.
  • The branch is merged with main after chore(vendor): track zvec c133f20; feat(fts)!: adapt to the embedded jieba dictionary #21: libzvec_c_api now embeds the cppjieba dictionary, so
    the build script no longer resolves or publishes one and strict mode has nothing to constrain
    there. The dictionary-specific parts this branch carried (the ZVEC_RUST_JIEBA_DICT_DIR
    registration, resolve_jieba_dict_dir and its test) are dropped in that merge.
  • Verified on Linux x86_64 originally; the merge itself was re-checked with cargo fmt --check,
    cargo clippy -p zvec-rust-sys --all-targets -- -D warnings, cargo test -p zvec-rust-sys --test build_script (18 passed) and cargo check --workspace --all-targets.
  • The platform file-name table is unit-tested, but no strict-mode build was run on macOS or
    Windows.

`ZVEC_LIB_DIR` is documented as the highest priority source for the native
library, but an unusable value only prints a warning and resolution continues
with the sibling checkout, the submodule, `vendor/lib`, a cached or downloaded
prebuilt SDK, or a source build. A typo, a stale path, or a directory holding an
incomplete SDK therefore links a different copy of the library than the one the
environment selected, without failing.

`ZVEC_STRICT_LIB=1` makes the explicit configuration the only accepted source:
`ZVEC_LIB_DIR` must be set and must contain the platform library, every
substitute is skipped, and a missing or invalid directory fails the build script
with the expected file name and target instead of falling back. The mode is off
by default, is enabled only by `1` or `true`, and never acquires anything.

`has_zvec_lib` now shares its platform file-name table with the failure message
so both report the same artifacts for the current target.

Tests: `zvec-sys/tests/build_script.rs` gains six cases — explicit directory
required, incomplete directory rejected, valid directory accepted, every other
value leaving the mode off, acquisition unreachable, and the platform file-name
table for Linux, macOS and Windows.
…ding

Review findings on the strict-artifact mode:

- `ZVEC_STRICT_LIB=true` (and `True`/`TRUE`) is now exercised, and the values
  other toolchains read as "yes" (`yes`, `on`) are pinned as off, so the accepted
  word is exactly `true`, case-insensitively, which the README now states.
- Strict mode no longer takes the cppjieba dictionary from a sibling checkout or
  the submodule: a dictionary from a different copy of the SDK is the same silent
  substitution the mode refuses. `resolve_jieba_dict_dir` stops after the
  configured directory in strict mode, and the test asserts the non-strict
  sibling pick and the strict refusal on one fixture.
- The failure message reads "one of a, b, c" where the platform accepts several
  artifact names, so the MinGW message no longer reads as if all three were
  required.
- README: the `true` casing, the dictionary rule, and the include-path
  consequence stated outright -- headers may still come from a sibling checkout,
  which decides what downstream build scripts see rather than what the linker
  uses.

Verified: build-script tests 19 passed (three new), workspace 232 passed and 0
failed, `clippy -D warnings` clean, and five real build runs -- strict with `1`
and with `TRUE`, missing directory, unset directory, and the non-strict fallback.
@mrcs64
mrcs64 marked this pull request as ready for review October 16, 2026 20:52
Upstream removed build-script cppjieba dictionary discovery in zvec-ai#21 (the
dictionary is now embedded in libzvec_c_api). Resolve by dropping the
remaining dictionary handling from this branch: the strict-mode dictionary
paragraphs in README, the ZVEC_RUST_JIEBA_DICT_DIR registration, and the
test that covered resolve_jieba_dict_dir. Strict mode now constrains only
the native library, as the upstream design implies.
@mrcs64

mrcs64 commented Oct 26, 2026

Copy link
Copy Markdown
Contributor Author

GitHub Actions on this repo are currently blocked by an account billing lock, so
no job started here or on main today. Could someone re-run once that's cleared?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant