fix(loader): confine $ref to the contract's directory tree

AGENTS.md

@@ -231,6 +231,7 @@ A 2026-04-16 security review (see `SECURITY_REVIEW.md` if committed, or the PR #
 - **Encrypted credential store fails loud.** `credentials/encrypted_store.py::_load_store` raises `CredentialError` on `InvalidToken` — never silently returns `{}` (which previously caused destructive overwrite on next write with the wrong key).
 - **Tool errors are typed, not text.** `dispatch_tool_call` returns `{"error": <ExcName>, "message": "Tool … failed — see server logs"}` on exception; the full `exc` goes to `LOG.warning(..., exc_info=True)` where the redactor can scrub it. Exception text is never round-tripped into the LLM context (prevents path / hostname / env-var leaks).
 - **Subprocess argv sanitisation.** `cli/auth.py::_sanitize_argv` redacts values of `--password`, `--token`, `--api-key`, `--key-file`, any flag ending in `-secret`/`-key`/`-token`/`-password`/`-passphrase`. Called from `AuthProvider._run_command` before DEBUG-logging the command.
+- **`$ref` resolution is confined.** Every external `$ref` goes through `util/ref_confinement.py::confine_ref`: URL schemes (`file://` included) and absolute paths are refused, and the `Path.resolve()`-d target must be `is_relative_to` the ref root — the root contract's directory unless the caller passes `ref_root=` / sets `FLUID_REF_ROOT` (process-wide, so an env root that is not a directory or does not contain the contract is ignored for that contract with a `ref_root_env_ignored` WARNING; an unusable `ref_root=` raises). Nested refs are held to the same root. Bundled OpenAPI fragments have no root, so only `#/...` refs reach openapi-spec-validator. A new `$ref` resolver must call `confine_ref`, not re-implement it.
 
 ---
 

SECURITY.md

@@ -58,6 +58,7 @@ FLUID Forge includes several built-in security measures:
 - **Credential redaction** — secrets are redacted from logs and plan output
 - **Provider auth isolation** — each provider manages its own authentication boundary
 - **Policy-as-code** — governance rules compile to native cloud IAM before deployment
+- **`$ref` confinement** — a contract's `$ref`s may only name files inside the contract's own directory tree (symlinks resolved); URLs and absolute paths are refused. Widening the root is an explicit opt-in (`FLUID_REF_ROOT` / `ref_root=`). See [docs/contract-refs.md](docs/contract-refs.md).
 
 ## Plugin Trust Model
 

docs/contract-refs.md

@@ -0,0 +1,227 @@
+# Composing a contract from fragments with `$ref`
+
+A contract can pull any object from another YAML or JSON file with `$ref`.
+`fluid validate`, `plan`, `apply` and `bundle` resolve every ref before they
+do anything else, so the rest of the pipeline sees one document.
+
+```text
+orders/
+├── contract.fluid.yaml
+├── owner.yaml
+└── fragments/
+    ├── builds/ingest.yaml
+    └── policy.yaml
+```
+
+```yaml
+# orders/contract.fluid.yaml
+id: sales.orders_v1
+metadata:
+  owner:
+    $ref: ./owner.yaml                     # the whole file
+builds:
+  - $ref: fragments/builds/ingest.yaml     # refs work inside lists
+exposes:
+  - exposeId: orders
+    policy:
+      $ref: fragments/policy.yaml#/gold    # file + JSON pointer
+```
+
+```bash
+fluid validate orders/contract.fluid.yaml   # refs resolved transparently
+fluid bundle   orders/contract.fluid.yaml   # print the single resolved document
+```
+
+A ref is resolved relative to the file that contains it, so
+`fragments/builds/ingest.yaml` may itself say `$ref: ../policy.yaml`.
+
+A `$ref` node is an object whose only key is `$ref`. Same-document refs
+(`$ref: "#/definitions/x"`) are left in place as written.
+
+---
+
+## Where a ref may point: the ref root
+
+Every ref must name a file inside the **ref root**. By default the ref root is
+the directory that holds the root contract file (`orders/` above), and it
+applies to every ref, including refs inside fragments. A fragment in
+`orders/fragments/` can reach anything under `orders/` and nothing outside it.
+
+The check runs after `..` segments and symlinks are resolved, so neither can
+be used to get out:
+
+| `$ref` written in `orders/contract.fluid.yaml` | Result |
+|---|---|
+| `./owner.yaml`, `fragments/policy.yaml#/gold` | resolved |
+| `../shared/policy.yaml` | refused: escapes the ref root |
+| `./link.yaml` where `link.yaml` is a symlink to `/home/me/x.yaml` | refused: escapes the ref root |
+| `/etc/hosts`, `C:\x.yaml`, `\\server\share\x.yaml` | refused: must be a relative path |
+| `file:///…`, `https://…`, `s3://…`, `//host/…` | refused: remote refs are not supported |
+
+A refused ref fails the command with a typed error that names the ref, the
+[JSON pointer](https://www.rfc-editor.org/rfc/rfc6901) of the `$ref` node, and
+the file it was written in:
+
+```text
+❌ Validation error: contract_load_failed
+   error: $ref '../shared/policy.yaml' at JSON pointer '/exposes/0/policy' in
+   /work/orders/contract.fluid.yaml escapes the ref root /work/orders (after
+   resolving '..' and symlinks); refs may only name files inside it. …
+```
+
+`fluid validate` exits 1 and `fluid bundle` exits 2. The check runs before the
+target is opened, so the error is the same whether or not the target exists.
+
+**Why.** Contracts are often untrusted input: a service or CI job may run
+`fluid validate` and `fluid bundle` on contracts its users upload. Without the root, a contract could compose any YAML or JSON file the
+process can read into itself, and `fluid bundle` would print it back.
+
+---
+
+## Widening the root (monorepos)
+
+To share fragments between products, set the ref root to a directory that
+contains both the contracts and the shared fragments:
+
+```text
+repo/
+├── shared/policy.yaml
+└── products/orders/contract.fluid.yaml   # $ref: ../../shared/policy.yaml
+```
+
+```bash
+FLUID_REF_ROOT=repo fluid validate repo/products/orders/contract.fluid.yaml
+```
+
+From Python, pass `ref_root=` to the loader:
+
+```python
+from fluid_build.loader import load_contract, RefConfinementError
+
+contract = load_contract("repo/products/orders/contract.fluid.yaml", ref_root="repo")
+```
+
+`load_contract`, `load_with_overlay` and `compile_contract` all accept
+`ref_root`. The argument wins over `FLUID_REF_ROOT`.
+
+Rules for the wider root:
+
+- It widens the root only for a contract inside it, and only if it is an
+  existing directory.
+- `FLUID_REF_ROOT` applies to every contract the process loads, so it is
+  ignored for a contract outside it, and when it is not a directory or cannot
+  be resolved at all (a `~user` that no longer exists, a symlink loop, a
+  parent the process cannot enter). That contract gets the default root, its
+  own directory, and a warning says so:
+
+  ```bash
+  FLUID_REF_ROOT=repo fluid validate /tmp/upload-1234/contract.fluid.yaml
+  ```
+
+  ```text
+  ref_root_env_ignored: contract /tmp/upload-1234/contract.fluid.yaml is outside
+  FLUID_REF_ROOT='repo' (…); the ref root must contain the contract. Ignoring it
+  for this contract: its $refs are confined to the contract's own directory
+  /tmp/upload-1234 (the default). Logged once per value in this process; a
+  later contract it is ignored for gets no warning, but its escape errors say
+  the variable was ignored and why.
+  ```
+
+  Refs that stay in that directory resolve as usual, and refs that leave it
+  fail as escapes. The warning is logged once per value in a process, not
+  once per contract, so a service that loads each upload from a fresh
+  directory logs it once. Every such escape error therefore also says that
+  `FLUID_REF_ROOT` was ignored for this contract, and why.
+
+  For a contract outside `FLUID_REF_ROOT` the variable is ignored, so it
+  cannot break or widen that contract. Every contract inside it is widened,
+  with no warning: a value left set in your shell widens every contract under
+  it, and a service whose upload directory sits under its `FLUID_REF_ROOT`
+  widens every uploaded contract. Scope the variable to one command instead
+  of exporting it.
+- `ref_root=` is set by the caller for one contract, so it is strict: a
+  `ref_root` that cannot be resolved, is not a directory, or does not contain
+  the contract raises `RefResolutionError`, naming `ref_root`.
+- A blank `FLUID_REF_ROOT` counts as unset: the default root applies.
+- It is only consulted when the contract has a ref to another file.
+- It widens the root and nothing else. URLs and absolute paths are still
+  refused, and refs that resolve into system directories (`/etc`, `/proc`,
+  `/private/etc` on macOS, …) are still blocked.
+
+Set it to the narrowest directory that works. Setting it to `/` turns the
+confinement off.
+
+### Upgrading: `../` refs to another product's fragments
+
+Before the ref root existed, a relative ref could climb out of the contract's
+directory, so monorepos shared fragments with `$ref: ../other-product/…`.
+Those refs now fail with `escapes the ref root` until the root is widened.
+Set `FLUID_REF_ROOT` (or `ref_root=`) to the repository root, or to the
+narrowest directory that holds the products and their shared fragments:
+
+```bash
+FLUID_REF_ROOT="$(git rev-parse --show-toplevel)" fluid validate products/orders/contract.fluid.yaml
+```
+
+Set it per command, as above, rather than with `export`: an exported value
+widens every contract under it that you load later in that shell.
+
+Contracts whose refs stay inside their own directory need no change.
+
+---
+
+## Catching the error in Python
+
+`RefConfinementError` is a subclass of `RefResolutionError`, so existing
+`except RefResolutionError` handlers catch it. It carries the details as
+attributes:
+
+```python
+from fluid_build.loader import RefConfinementError, load_contract
+
+try:
+    load_contract(path)
+except RefConfinementError as err:
+    print(err.ref)      # '../shared/policy.yaml'
+    print(err.pointer)  # '/exposes/0/policy'
+    print(err.source)   # file that contains the ref
+    print(err.root)     # the ref root it escaped
+    print(err.ignored_ref_root_env)  # FLUID_REF_ROOT value ignored for this
+                                     # contract, or None
+```
+
+`ignored_ref_root_env` is set when `FLUID_REF_ROOT` is set but did not apply
+to this contract (it cannot be resolved, is not a directory, or does not
+contain the contract).
+The root is then the contract's own directory, whatever the environment says.
+
+---
+
+## OpenAPI fragments inside a bundle
+
+When `fluid validate <bundle>.tgz` checks the OpenAPI documents extracted into
+`sources/openapi/`, the same check applies with **no** root, because a bundled
+fragment has no directory. Only same-document refs
+(`$ref: "#/components/schemas/Order"`) are allowed. Any other `$ref` is
+reported as an `OAS-REF-EXTERNAL` error, and openapi-spec-validator is not
+run on that fragment. If it were run, it would follow `file://` and
+`http(s)://` refs. Inline the referenced schemas under `components` instead.
+
+This applies to a `$ref` key anywhere in the fragment, including inside
+`example`, `examples.*.value` and `x-*` extension payloads, which the
+validator treats as data:
+
+```yaml
+components:
+  schemas:
+    Doc:
+      type: object
+      example:
+        $ref: https://json-schema.org/draft/2020-12/schema   # OAS-REF-EXTERNAL
+```
+
+Those keys are not skipped because the same names are also used for schemas.
+`properties: {example: {$ref: …}}` declares a property called `example`, and
+openapi-spec-validator does follow that `$ref`. If an example payload has to
+contain a `$ref`, rename the key in the payload (for example `ref`) or drop
+the example.

examples/0.7.1/bitcoin-multifile/README.md

@@ -48,6 +48,35 @@ fluid apply contract.fluid.yaml --yes
 |---------|-----|
 | **Team ownership** | Security team owns `access-policy.yaml`, compliance owns `sovereignty.yaml` |
 | **Independent versioning** | Add a new expose without touching governance configs |
-| **Reusable fragments** | `sovereignty.yaml` can be `$ref`'d by every EU data product |
+| **Reusable fragments** | `sovereignty.yaml` can be `$ref`'d by every EU data product, once `FLUID_REF_ROOT` names a directory that holds them all ([below](#sharing-a-fragment-with-another-product)) |
 | **Smaller diffs** | PRs touch only the fragment that changed |
 | **The engine stays simple** | validate/plan/apply always receive one resolved document |
+
+## Sharing a fragment with another product
+
+A `$ref` may only name a file inside the root contract's directory, so this
+contract can reach anything under `bitcoin-multifile/` and nothing outside it.
+A second product that wants the same `sovereignty.yaml` has to step outside
+its own directory, and that is refused by default:
+
+```yaml
+# examples/0.7.1/eu-orders/contract.fluid.yaml
+sovereignty:
+  $ref: ../bitcoin-multifile/fragments/sovereignty.yaml
+```
+
+```text
+❌ Validation error: contract_load_failed
+   error: $ref '../bitcoin-multifile/fragments/sovereignty.yaml' at JSON pointer
+   '/sovereignty' in …/eu-orders/contract.fluid.yaml escapes the ref root …/eu-orders …
+```
+
+Widen the root to a directory that contains both products:
+
+```bash
+FLUID_REF_ROOT=examples/0.7.1 fluid validate examples/0.7.1/eu-orders/contract.fluid.yaml
+```
+
+From Python, pass `ref_root="examples/0.7.1"` to `load_contract`. See
+[Widening the root](../../../docs/contract-refs.md#widening-the-root-monorepos)
+for the rules.

fluid_build/forge/core/validators.py

@@ -46,6 +46,11 @@
 import yaml
 
 from fluid_build.forge.core.bundle import SOURCE_SENTINEL, validate_manifest
+from fluid_build.util.ref_confinement import (
+    RefConfinementError,
+    confine_ref,
+    iter_external_refs,
+)
 from fluid_build.util.safe_yaml import load_yaml_safe
 
 LOG = logging.getLogger("fluid.forge.core.validators")
@@ -253,6 +258,25 @@ def validate_sql(
     return issues
 
 
+def _external_openapi_ref_issues(path: str, spec: Any) -> List[ValidationIssue]:
+    """One ``OAS-REF-EXTERNAL`` error per ``$ref`` that leaves the document."""
+    issues: List[ValidationIssue] = []
+    for pointer, ref in iter_external_refs(spec):
+        try:
+            confine_ref(ref, ref.split("#", 1)[0], base_dir=None, root=None, pointer=pointer)
+        except RefConfinementError as exc:
+            issues.append(
+                ValidationIssue(
+                    file=path,
+                    validator="openapi-spec-validator",
+                    severity="error",
+                    message=str(exc),
+                    code="OAS-REF-EXTERNAL",
+                )
+            )
+    return issues
+
+
 def validate_openapi(
     path: str,
     content: bytes,
@@ -294,7 +318,15 @@ def validate_openapi(
             )
         ]
 
-    issues: List[ValidationIssue] = []
+    # openapi-spec-validator follows external ``$ref``s through its default
+    # handlers — ``file://`` reads a host file (whose values then surface in
+    # the validation error) and ``http(s)://`` makes a request. A bundled
+    # fragment has no directory of its own, so the shared confinement check
+    # allows only same-document ``#/...`` refs; anything else is reported
+    # and the validator is not called on this fragment.
+    issues = _external_openapi_ref_issues(path, spec)
+    if issues:
+        return issues
     try:
         # validate_spec (legacy) and validate (current) both exist; prefer
         # whichever is available without importing an exact entry point.