feat(api): load a contract exactly as fluid plan sees it

[fas89]

One of three engine-side fixes requested by the maintainers (2026-10-02) for services that run contracts other people wrote: $ref confinement in the loader, a public contract-load API, and a DuckDB sandbox for contract SQL. The three (#687 $ref confinement, #688 public load API, #689 DuckDB sandbox) touch disjoint files and merge in any order. Each was built, adversarially reviewed (every finding judged by 3 independent skeptics), fixed and re-reviewed. Gates: ruff, black 24.10.0, and the full pytest suite on Python 3.12 and 3.10.

Summary

Adds fluid_build.api.load_contract (a contract file or a fluid bundle .tgz) and two in-memory forms, load_contract_from_text and load_contract_from_dict. Each returns a LoadedContract with:

• contract: the dict fluid plan plans. It goes through parse, $ref composition, the env overlay, alias rewrites and the legacy build: rewrite, in the engine's order.
• digest: the plan-digest canonicalisation of that dict.
• provenance: source, overlay, every file composed, and any $ref values left in place.

Every failure raises a single ContractLoadError carrying a stable event. The module only composes the engine's own loader; it adds no rewrite of its own. Docs: docs/CONTRACT_LOADING_API.md.

Hardening from review (second commit)

• env is a name, never a path. The engine builds overlay paths from env. Before this fix, an absolute or ../ env merged any .yaml/.yml/.json file into the returned contract (reproduced with a Docker config.json-shaped file). env must now match the fluid publish --env grammar (fluid_build._env_names), or the call raises contract_env_invalid before any file is read. "" is refused; None means no env.
• In-memory overlay follows the engine. When a $ref survives the merge (a $ref in the overlay, or a same-document #/ pointer), the engine's auto-bundle step drops the overlay. The in-memory forms now do the same: they return the base and log contract_overlay_not_applied, as load_contract does.
  • Without base_dir, if the document holds file $ref values, the engine's choice depends on the fragments. Passing an overlay then raises contract_overlay_needs_base_dir instead of guessing.
  • Both in-memory functions take a logger.
• The in-memory replay cannot fall behind the engine. The rewrites run from _ENGINE_REWRITES. A guard test parses load_contract_with_overlay and fails if the engine gains, loses or reorders a step. The docs' stability claim now says exactly what each form guarantees.
• Consistent events:
  • a YAML list root in a contract or overlay file is contract_not_a_mapping (it was contract_load_failed; the text form already said contract_not_a_mapping)
  • a file that is not UTF-8 is contract_parse_failed
  • bundle_not_found is now documented
  • .digest raises contract_not_serialisable instead of a bare TypeError (for example, an unquoted YAML date, which fluid plan also fails on)
• Docs truth:
  • .digest is the digest of the planned contract, not the fluid contract digest / upstreamDigest value.
  • contract equals plan.json["contract"] once keys are written as strings. A YAML on: key stays a bool, as it does inside the engine, so compare contracts by .digest.

Release notes (for the release CHANGELOG; not edited here)

• Added fluid_build.api.load_contract, load_contract_from_text, load_contract_from_dict, LoadedContract and ContractLoadError (fluid_build.api 1.1).

Tests

• tests/api/test_contract_load.py checks the following against the real fluid plan run():
  • the file form for every rewrite (aliases, legacy build:, $ref, overlay, bundle, a magic-word key) and both overlay-drop shapes
  • the in-memory forms against the file form
  • env-grammar rejections (absolute, ../, empty, separator)
  • the engine-step AST guard
  • the typed file-form events
  • the digest behaviours
• Negative controls:
  • With the pre-fix module restored, 14 of the new tests fail.
  • Adding a step to the engine loader turns the guard red.
  • Removing the UnicodeError mapping turns the non-UTF-8 test red.
  • Each file was restored byte-identical (sha256 checked).
• Local results:
  • pytest tests/ (-n auto, Python 3.12): 19747 passed, 637 skipped.
  • tests/api on Python 3.14: 78 passed.
  • ruff check fluid_build/ tests/ passes.
  • black==24.10.0 --check fluid_build/ tests/ passes.

Tested live

This is library-only: there is no CLI or UI surface. fluid plan was exercised for real inside the tests (plan_cmd.run). The probes for the date contract (planner_failed), the same-document pointer plus overlay (plan drops the overlay), a missing .tgz (bundle_not_found) and the magic-word key were run against the real plan command.

Decisions for the reviewer

• F6: I kept contract with the engine's bool and int keys and changed the docs (equality holds once keys are strings; compare by .digest), rather than returning coerce_keys_to_str(contract). One reviewer preferred coercing. I didn't, because coercion would be a rewrite of this module's own, would make the dict differ from what the engine's validator and planner see, and would merge a YAML on: key with a "True"/"true" key. Confirm or overrule.
• env="" is now refused with contract_env_invalid instead of being treated as no env, the way the engine treats it. Confirm that strictness is wanted for a downstream service.
• contract_overlay_needs_base_dir is a new raise for in-memory calls that pass an overlay, no base_dir, and a document with file $ref values. Before this change such a call silently merged; afterwards it raises. Callers following the docs (with base_dir) are unaffected.
• The engine itself (the CLI fluid plan --env ../x) still accepts path-shaped env values. This PR only closes the gap at the library boundary, as scoped. Whether the CLI should also enforce is_env_name is a separate decision, and it would touch _contract_loader.py/loader.py, which belong to the other PRs' file sets.
• Existing engine defect, documented here but not fixed because the loader files are off-limits for this PR. In fluid_build/_contract_loader.py, _auto_bundle_if_needed reloads the contract from the BASE file (compile_contract(path)) whenever the overlay-merged contract still holds any $ref. Reproduced with load_contract_with_overlay: (a) an overlay containing a $ref, or (b) a same-document '#/...' pointer in the contract, makes --env prod silently plan the base contract with no warning. In addition, _has_ref_pointers fires on any dict that has a '$ref' key, not only on pure ref nodes. A fix belongs with whoever owns _contract_loader.py (the $ref-confinement PR's file); the new API's provenance already reports this honestly, and its tests are written to stay green after such a fix.
• Existing engine defect: aliases are rewritten before a legacy build: is turned into builds:, and the alias table's paths start at builds. So an alias under a singular build: (for example source.kind: pg) is never rewritten, and fluid plan then rejects it at the schema gate (reproduced: local_plan_validation_failed). The new API mirrors the engine's order, and a test pins that. Swapping the order in _contract_loader.load_contract_with_overlay would fix it; the API would follow automatically, but the pinning test test_alias_under_a_legacy_build_matches_the_engine_loader would then need its last assertion updated.
• Coupling with the $ref-confinement PR: load_contract_from_dict/_text call loader._resolve_refs(contract, base_dir) and loader._parse_ref / _resolve_pointer. If that PR changes _resolve_refs's signature (for example a new required root argument), the text-form equality tests will go red when the PRs meet on main, which is the intended alarm. If it adds an optional argument, nothing changes.
• downstream service follow-up (separate repo): plan_binding._as_the_engine_loads_it can be replaced by fluid_build.api.load_contract_from_dict(parsed) once this release ships, gated on fluid_build.api.api_version >= '1.1'. Its existing contract_refs refusal can switch to checking LoadedContract.unresolved_refs.
• CONTRIBUTING requires a companion docs PR in Agenticstiger/forge_docs. The in-repo docs/CONTRACT_LOADING_API.md is written, but the forge_docs page is still to do (marked Docs TODO in pr_body).

Changes after review

These changes came from adversarial review rounds; each finding was upheld by at least 2 of 3 independent skeptics before it was fixed. A final re-review of the branch found nothing further.

R1 — 1c342d2

Review round 2: fixes in 1c342d2

• YAML aliases in the in-memory forms. Without base_dir, load_contract_from_dict copied the document with deepcopy, and deepcopy keeps anchor/alias sharing. The engine's $ref resolver rebuilds every dict and list, so in the file form an aliased node is already two objects when the overlay merge and the alias rewrites change nodes in place. In memory it was one object, so a change at one path reached every alias. The same input gave a different contract and a different .digest. The base document is now unshared before the merge. Sharing inside the overlay is kept, because the engine keeps it there too. A document that contains itself now fails with contract_load_failed, as the file form does.
• contract_not_a_mapping is raised only for the loader's root checks. It used to cover any plain ValueError, so a $ref holding a NUL byte was reported as "the root is not an object". That case is contract_load_failed again.
• env refuses only path-like values: empty, ., .., /, \, NUL, absolute or drive-qualified. Names fluid plan --env accepts and fluid publish --env does not (_staging, prod+eu, more than 64 characters) load again.
• A NUL byte in path or base_dir is now contract_not_found. It used to escape as a bare ValueError.
• The engine-step guard test also sees keyword calls and every other write to contract. It has a parametrised negative control over the engine's real source. The docs now say what the guard does not cover.

Tested: each fix is negative-controlled, both against the previous head and with each fix reverted on its own. black 24.10.0, ruff, lint-imports and mypy-strict pass. tests/api passes on 3.10 and 3.12. The full suite on 3.12, run as CI does, has 19740 passed and one setup timeout in the live MCP test under xdist. That test file passes when run alone.

R2 — 4cc39a4

Follow-up review fixes (4cc39a4)

• One event for a root that is not an object, overlay or not. The loader checks a YAML root but not a JSON one (nor one a root $ref composes), and it merges an overlay into dict(base). So with an env and an overlay, a JSON list root either failed with contract_load_failed or was quietly turned into a dict and loaded: [["k","v"]] became {"k":"v"}, and [] became the overlay alone. When an env is set, load_contract now reads the base before calling the engine and raises contract_not_a_mapping, the same event the same file gives without an env. If the base cannot be read, the engine's own load raises the error, so that event does not change either.
• The env rule is the same on every platform. The drive check now uses ntpath.splitdrive, so env="C:prod" is refused on Linux and macOS too, as the docs already said. Before, it was refused only on Windows.
• The engine-step guard fails closed. It now counts a call that is handed any part of contract as a step. It also reports any return of something other than contract, rebinding contract as a loop or tuple target, and any other read of contract (an alias, a sub-tree passed on, a loop over it, a container holding it). The docs describe exactly this.
• Tests. There are new negative-controlled tests for seven non-object JSON roots (each with no env, with an overlay, and without one), for the event a failing base keeps, for C:prod, and for 16 guard mutation shapes. 13 of those shapes pass the previous guard unnoticed.

Tested locally. black 24.10.0 and ruff are clean. tests/api passes on py3.10 and py3.12 (132 tests). The full suite passes on py3.10: 19667 passed, 771 skipped.

Documentation

Companion docs PR: Agenticstiger/forge_docs#139 (new pages for $ref confinement, the DuckDB sandbox and the contract-loading API, plus 0.18.0 release notes). It merges after 0.18.0 is on PyPI.

[github-actions]

📄 Documentation Reminder

This PR appears to be missing a documentation reference. Our docs live in a separate repo.

Please update the PR description with one of:

• Link a docs PR — check the "Docs PR linked" box and paste the URL
• Mark as no docs needed — check "No docs needed" with a justification
• Acknowledge docs TODO — check "Docs TODO" and create the docs PR before merge

See the Contributing Guide for details.