feat(adapters)!: prompt fallback for adapter functions without a trained adapter

[ajbozarth]

Pull Request

Issue

Part of #993

Description

Adds prompt-based fallbacks for Granite adapter functions on the LocalHF backend. When no trained aLoRA or LoRA adapter exists for a capability on the target model, resolution now falls back to a hand-authored prompt (packaged as an io.yaml instruction) run against the plain base model instead of failing — the only mechanism available for models a LoRA cannot be loaded into.

resolve_adapter walks the chain per capability + base model and registers the first rung that exists: model-specific aLoRA → model-specific LoRA → model-specific prompt → model-agnostic any prompt (with a warning on the any rung, since it is less accurate than a model-tuned one).

What's included:

• PromptBinding — a weightless WeightsBinding (no-op prepare/deactivate/release; activate() only logs), plus AdapterType.PROMPT and a widened Identity.adapter_type.
• Availability-aware resolve_adapter (_register_available_composed_adapter) that probes the chain via a tiny io.yaml fetch, replacing the previous hardcoded LoRA registration.
• A weightless HF generation path (_generate_composed_prompt) that runs the intrinsic's injected instruction against the base model with no adapter_scope.
• Generalized adapter-function invocation telemetry (_afire_invocation_complete / _await_generation_reporting_error) that now carries binding_type and revision; the OpenAI embedded path is migrated onto the shared helpers.
• Path-helper support for the prompt subdir and the any model slot (adapter_subpath / obtain_io_yaml now take an adapter_type string; new resolve_prompt_io_yaml).
• AdapterNotFoundError (a ValueError subclass) for the not-found signal obtain_lora raises, so the resolution walk and resolve_prompt_io_yaml advance only on a genuine miss and let other ValueErrors (e.g. a Hugging Face HFValidationError) propagate instead of being masked as "not found".
• Documentation for the prompt fallback (docs/docs/advanced/intrinsics.md) and the binding-shape references.

Behavior change worth a close look: because resolution is now availability-aware, it selects an aLoRA adapter where one exists on the LocalFile path — previously that path hardcoded LoRA, so an aLoRA was never auto-selected there. This closes a gap documented against Epic #929 "Phase 2", which has no discrete tracked issue of its own. The aLoRA-via-LocalFile path was not previously exercised in production, so it is worth confirming it loads and generates as expected.

Deferred to a follow-up PR: OpenAI and Ollama prompt fallback. Ollama landed as a third adapter backend on main while this work was in progress, so the follow-up will design the served-adapter prompt fallback uniformly across OpenAI and Ollama in the resolution layer (registering a PromptBinding so the high-level intrinsic helpers reach it) rather than per-backend. This PR is LocalHF-only and self-contained; the weights-binding support table reflects that (LocalHF ✅, OpenAI / Ollama —).

Note on the e2e tests: the end-to-end resolution tests target a temporary private repo (ajbozarth/granitelib-rag-r1.0, a weights-stripped copy of ibm-granite/granitelib-rag-r1.0 with prompt io.yaml files added). Because that repo is private, the tests xfail (rather than hard-fail) wherever it is not readable with the current token — including CI's public-only token and any other developer's token — and run normally for a developer with access. They should be retargeted to a public repo once one carries this layout (marked with a TODO in the test), at which point the access guard goes inert.

Testing

• Tests added to the respective file if code was changed
• New code has 100% coverage if code was added
• Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

New unit tests (no network): prompt-subpath / path-helper resolution, PromptBinding verbs + AdapterType.PROMPT, the availability walk across every rung (trained hit, LoRA fallthrough, model-specific prompt, any prompt + warning, hard-error abort, terminal not-found), and the weightless HF generate path. New e2e tests exercise the resolution chain — including an aLoRA-preference and a LoRA-only case — against the test repo. Ran the adapter / HF / OpenAI / Ollama / formatter unit + integration subsets locally after rebasing on main; the full suite runs in CI.

Attribution

• AI coding assistants used

---

Adding a new component, requirement, sampling strategy, or tool?

If your PR adds or modifies one of the types below, check the matching box. A checklist of type-specific review items will be posted as a comment.

• Component
• Requirement
• Sampling Strategy
• Tool

NOTE: Please ensure you have an issue that has been acknowledged by a core contributor and routed you to open a pull request against this repository. Otherwise, please open an issue before continuing with this pull request.

[planetf1]

Reviewed at 4643f67. Thanks for this, and particularly for the "behaviour change worth a close look" paragraph. It sent me to measure the aLoRA selection change on a real model, and that turned out to be the most consequential thing here.

The design holds up. All five decisions from #993's 2026-09-03 meeting are implemented as agreed, including the subtle one: AdapterNotFoundError makes the fallback a genuine not-found mechanism, so a Hub auth or validation error still propagates. The walk is 92% covered with every rung exercised, and PromptBinding is at 100%.

Four asks, each on the line it concerns:

1. huggingface.py:1244 an explicit adapter_types no longer has any effect on this backend.
2. adapter.py:539 preferring aLoRA regresses requirement checking today. The ask is small: keep today's rung order and link the issues. The gate belongs in #1678.
3. adapter.py:588 a prompt io.yaml with no instruction returns a fabricated score.
4. huggingface.py:3017 and :1408 the branches that make the feature work are not executed by any test. They do work, I ran them, so this is test debt.

Issues opened from this review: #1678 aLoRA activation is unverifiable, #1679 the published requirement-check aLoRA, #1680 result-level provenance, #1681 a coverage rule that hides these generation helpers from measurement.

Documentation points that have no line to sit on

• The title carries !, but the body never lists what breaks. As far as I can tell: the three path-helper signatures losing alora, the Identity.adapter_type widening for exhaustive consumers, and whatever validation-mechanism change survives ask 2.
• Sibling docs the new binding did not reach: metrics.py:1236, docs/docs/observability/metrics.md:317, the module list at _core.py:10-18, the stale Literal["lora", "alora"] at adapter.py:314, and the config= contract at adapter.py:674-681 and huggingface.py:2889-2896, which still says config is rejected for every shape other than Embedded while the code now accepts it for Prompt. The "composed Embedded adapter" wording in list_adapters and remove_adapter covers Prompt too now.
• Two sentences would help the next reader: why probing the pinned revision beats the catalogue for availability, given the epic's removed TODO said selection would come from the catalogue; and that prompt fallback is LocalHF-only today.
• The e2e test's TODO could cite #1655, so the stopgap is traceable from the code.

Things I checked that came back clean

Registry symmetry, so a prompt adapter can be listed and removed like the others. Lock ordering in the new registration branch, which keeps resolve then activation and does its fetch outside the activation lock. Enum widening, where every AdapterType consumer either asserts the two trained types or reads the catalogue. The Ollama signature port. Exports and __all__ in both packages. And the obvious worry from #1634, that RAG documents might be dropped on a new path: they cannot be, because the weightless path reuses the same rewriter output and the same generate_input.

Also #1655 already covers the private e2e fixture repository with acceptance criteria, which is more than most stopgaps get.

[planetf1]

An explicit adapter_types no longer has any effect on this backend.

Suggested change: pass the caller's types with "prompt" appended as the last rung, rather than None. None keeps meaning auto.

Today, with this branch: Intrinsic("my-check", adapter_types=(AdapterType.ALORA,)), as docs/docs/tutorials/07-custom-adapter-function.md:107 shows, with only a LoRA registered, runs the LoRA. On main it raised backend has no adapter for processing intrinsic.

Why it matters, four separate consequences:

• a caller who names a type is served a different one, with no warning
• the aLoRA-before-LoRA preference documented at adapter.py:1234-1238 is gone, so insertion order decides
• openai.py:1054 and ollama.py:732 still filter, so the three backends now differ in the same method
• the reroute gate at :723-729 still filters by type and then discards the adapter it found, so the adapter that authorises a reroute and the one that generates can differ

Restoring the old line is not the fix: a prompt winner has adapter_type == "prompt", which is never in action.adapter_types, and I checked that the old lookup makes the prompt rung unreachable.

Evidence, and one consequence to plan for

Three arms in a worktree at this SHA, with a guard that registers only a LoRA and asks for an ALORA-restricted Intrinsic:

Lookup argument	ALORA restriction honoured	Prompt rung reachable
None (this branch)	no, LoRA selected	yes
tuple(at.value for at in action.adapter_types)	yes	no, feature dead
tuple(...) + ("prompt",)	yes	yes

The third arm left the suite unchanged: 548 passed with identical failure sets across test/backends/test_adapters, test_huggingface_unit.py and test/formatters/granite.

The consequence: _util.py:226-230 passes (ALORA, LORA) explicitly, so under this rule every high-level helper counts as "the caller meant it" and loses the fallback. That call site needs to become an auto caller, but the explicit tuple is doing real work: the catalogue default is (LORA, ALORA), LoRA first, described as an availability set rather than a preference order. Drop it naively and helpers silently start preferring LoRA.

Which argues for naming "auto" in one place, perhaps a public selection_types on Intrinsic: the caller's tuple when set, otherwise the catalogue's set ordered aLoRA first with "prompt" appended. Backends then call one accessor, and the three getattr(..., "_adapter_types", None) reaches at huggingface.py:723, openai.py:919 and ollama.py:951 could retire.

[planetf1]

Preferring aLoRA regresses requirement checking today.

Revised, smaller ask: the walk this sits on does not exist on main, so the gate belongs in #1678, not here. This PR only needs to keep today's rung order, probing LoRA before aLoRA, with a comment pointing at #1678 and #1679. Flipping to aLoRA-first is then a small follow-up once the guard is in, and #1678's acceptance criteria already carry it.

Today: on main, requirement validation on a Granite 4.1 model is served the requirement-check LoRA, because the composed branch hardcoded LoRA. With this branch the walk prefers the aLoRA, which is published for 4.1-3b, 4.1-8b and 4.1-30b.

Measured on granite-4.1-3b with the catalogue's pinned adapters, ten requirements with obvious answers: LoRA 10 right, aLoRA 8. Both aLoRA misses fail open, above 0.99 for requirements the response plainly fails.

Why it matters: Requirement.validate() returns True, so a sampling or repair loop keeps a candidate it should have rejected, and users get this by upgrading without changing their own code.

The second defect underneath it, and the hypotheses I ruled out

That aLoRA never activates. It declares alora_invocation_tokens for <requirements>, its own io.yaml instruction writes <requirements>:, and the tokeniser merges > and : into one token, so PEFT finds no activation point and silently proceeds with none. Filed as #1679 for the artefacts and #1678 for Mellea being unable to see it.

Ruled out before concluding the weights are the issue: patching the prefix so activation fires left the French probe at 0.9988 against 0.9975, so activation is not the cause; the assembled prompt puts the adapter over the last 68 of 96 tokens from the marker onwards, which is what an aLoRA expects; and the requirement text is correctly interpolated in both arms. Two of the ten probes are requirements about the response rather than its content, so they may be out of distribution, which is why #1679 asks for real evaluation numbers rather than treating 8/10 as one. A larger three-arm run is under way.

[planetf1]

Update: the ask here is smaller than I first wrote, and the reason is worth recording.

_register_available_composed_adapter does not exist on main, so the selection gate I suggested has nowhere to live until this merges. That splits the work cleanly. #1678, which we have started, adds a generation-time guard that works on main today and reports when a loaded aLoRA's activation sequence is absent from the prompt. It touches nothing here.

So this PR only needs today's rung order preserved, plus a comment pointing at #1678 and #1679.

On the evidence, in fairness: the ten-item comparison is a smoke test, and two items are requirements about the response rather than its content, so it may overstate the gap. A three-arm evaluation is running now (LoRA, aLoRA as published, aLoRA with the activation prefix patched) across 4.1-3b and 4.1-8b, reported by requirement family. I will post the result on #1679 and update this thread either way, including if it shows parity on content requirements.

[planetf1]

A prompt io.yaml with no instruction returns a fabricated score.

Suggested change: require a non-empty instruction for a prompt rung, and fail here naming repository, capability, model slot and path.

Today: the prompt rung is implemented entirely by the rewriter injecting this config's instruction. That field is optional (constants.py:9-16), and this check only confirms the YAML parsed to a mapping. So a prompt io.yaml without an instruction resolves, registers and runs: the base model gets the bare conversation, llguidance forces the output into the declared schema, the caller gets something like {"certainty": 0.83}, it parses cleanly, and telemetry records outcome="success".

Why it matters: #993 puts prompt authoring with the library developers, so Mellea cannot review that repository's contents, and the one field the feature depends on is the one nobody validates. The failure mode is a confident number rather than an error.

In fairness, all six published io.yaml files I sampled do set instruction, and no prompt/ directory exists upstream yet, so this guards an authoring slip rather than something reproducible today. That also makes now the cheap moment to add it.

The alternative shape, and a smaller related suggestion

You could instead treat it as "not a usable prompt" and advance to the any rung, which fits the chain's spirit. I lean towards failing, because the artefact is wrong and somebody upstream needs to fix it, and reaching for a weaker prompt hides that. Note that resolve_prompt_io_yaml returns a path without parsing, so advancing on malformed input needs the validation inside the rung loop, making it the larger change.

Separately, the other malformed shapes fail loudly but late: an unexpected or missing top-level field raises from the rewriter at first generation. Running make_config_dict where the prompt config is cached would move those errors to resolve, where the repository path is still to hand.

[planetf1]

This branch is never executed by any test, nor is the dispatch branch at :1408.

Branch coverage over the full default suite (-m "not qualitative and not slow", all extras, 4546 passed) shows these arms never taken:

• [2936, 2939] the config= gate's PromptBinding arm
• [3017, 3018] this registration branch, with 3018 and 3023-3026 unexecuted
• [1408, 1409] the _generate_from_intrinsic prompt dispatch
• [1140, 1141] and [1140, 1145] _obtain_prompt_io_yaml_config

These are the two calls a real user makes: register what the walk resolved, then generate through it. The walk is well covered and _generate_composed_prompt has direct unit tests, so what is missing is the seam between them. test_composed_adapter_drives_generate_from_intrinsic at test_huggingface_unit.py:897-963 is the template for the dispatch half.

They do work, so this is test debt rather than breakage: I drove both on SmolLM2-135M and registration, dispatch, generation and parsing all behaved.

[planetf1]

This dispatch branch is the other half of the coverage gap in the thread on :3017: [1408, 1409] is never taken, so no test reaches _generate_composed_prompt through _generate_from_intrinsic, and the prompt arm of the emit at :1484-1501 has never fired.

One thing I checked while here, since this path had no test to lean on: on a clean base model with no PEFT adapter ever loaded, set_adapter([]) and active_adapters() both raise ValueError("No adapter loaded…") on transformers 5.10.2 and peft 0.19.1, which is exactly what deactivate_peft_adapter and _assert_correct_adapters catch. So the weightless path's assumption holds on a real model, not just against the MagicMock in the unit test.

[planetf1]

Wording: this says the fetch path is used "when a bare add_adapter(PromptBinding) self-fetches its io.yaml", but AdapterInput is Adapter | _AdapterCore | LocalFileBinding and LocalHFBackend.add_adapter's gate at huggingface.py:2907-2915 accepts only LocalHFAdapter, LocalFileBinding, EmbeddedIntrinsicAdapter or a composed Adapter. A bare PromptBinding is rejected with TypeError.

The reachable shape is a composed Adapter wrapping a PromptBinding, registered without config=. Same wording appears at huggingface.py:1118 and :2937. Worth fixing because these docstrings are published in the API reference.

[planetf1]

Nit: the Args: entry documents this as "Always AdapterType.PROMPT", but any AdapterType is accepted, and PromptBinding(adapter_type=AdapterType.LORA) would produce qualified_name == "x_lora" on a weightless binding.

The identity agreement check at huggingface.py:2999-3011 catches the mismatched case at registration, so the blast radius is small. A one-line guard or a narrowed annotation would make the docstring true.

[planetf1]

The comment and the test drifted from what the code now does, and the gap matters because this narrowing is one of the PR's better ideas.

The comment says a non-ValueError must propagate, and the stub raises RuntimeError. But the walk advances only on AdapterNotFoundError, so the interesting case is a plain ValueError (or an HFValidationError-shaped subclass) from the aLoRA or LoRA probe, which must propagate rather than being read as "not found". Nothing pins that today, although the equivalent case for resolve_prompt_io_yaml is covered at test_prompt_subpath_unit.py:126-142.

Two related wording points: the stub docstring at :48-52 describes "the rest raise ValueError, the not-found signal the walk advances on", which reintroduces exactly the conflation this PR removes; and test_adapter_resolution_e2e.py:91 asserts the broad ValueError where AdapterNotFoundError is the point.

[planetf1]

Two additions worth making here, both about what a user can tell from a result.

The section says the fallback is transparent, and flags reduced accuracy only for the any rung. A model-specific prompt also differs in quality from trained weights, and for a scoring helper that is the difference between a calibrated number and the model's self-report. One sentence would set expectations honestly.

There is also no documented way to find out which rung served a call. The value carries no provenance, and the telemetry route needs the hooks extra plus a plugin. backend.resolve_adapter(name).weights.binding_type already answers it, is public, and appears nowhere in the docs. Worth a line. Result-level provenance for the harder cases is #1680.

[planetf1]

While this paragraph is being edited, the model it describes has moved on.

It says routing follows from an adapter being loaded via backend.add_adapter(), which reads as opt-in by the user. After this PR, any adapter-function call resolves and registers an adapter implicitly, and on Granite 4.1 models that adapter can be an aLoRA, at which point the reroute gate matches and plain Requirement.validate() stops using LLM-as-a-judge. Same user code, different scoring mechanism, depending on the base model and on whether an adapter-function call happened earlier in the session.

Two things would help: say that resolution now registers adapters implicitly, and name default_to_constraint_checking_alora=False as the escape hatch for anyone who wants LLM-as-a-judge back. The PR body's "behaviour change worth a close look" paragraph covers the adapter-type switch, but not this consequence of it.

[markstur]

One more gap for this paragraph: on a model that only has a prompt fallback (no trained requirement-check adapter), Requirement.validate() still uses LLM-as-a-judge. The reroute gate only looks for "alora" adapters (huggingface.py:724-729), but the prompt adapter is registered as type "prompt" (adapter.py:597), so the gate never finds it.

Not a regression — those models had no adapter at all before this PR — but the docs should say the fallback doesn't cover the requirement path: the intrinsic helpers run the prompt while validate() runs LLMaJ on the same model. To get the prompt, ALoraRequirement(..., adapter_types=(AdapterType.PROMPT,)) works — the gate honours an explicit adapter_types (huggingface.py:723).

[planetf1]

Separate from the preference question in the thread above, and non-blocking.

Intersecting the probe order with metadata.adapter_types would stop a stray directory registering a type the catalogue denies, and saves a guaranteed-miss round trip for the three LoRA-only capabilities. I measured the probes at 116 to 191 ms each, so that part is tidiness rather than performance.

It also removes a small correctness gap: nothing currently cross-checks the probe against the catalogue, so an alora/ directory appearing for a capability the catalogue marks LoRA-only would be selected.

[planetf1]

Follow-up to my earlier review, from re-running the measurement behind "preferring aLoRA regresses requirement checking today". The premise has changed; see the updated #1679 and #1685.

• The aLoRA was never activated on this backend at all: adapters load through the bare model.load_adapter() path, so PEFT's aLoRA offset computation (which only runs inside the PeftModel wrapper) never executed. Zero hook calls, measured. That bug is fixed in #1685.
• The weights are not broken: the uncertainty aLoRA works as-published once activation is fixed, and the requirement-check items previously attributed to the weights answer correctly under activation.
• The requirement-check io.yaml tokenization mismatch is real; the durable fix is a granitelib republish, and #1685 carries a removable repair in the meantime.

What this means: aLoRA is broken on main today, and that is exactly why this PR's switch to preferring aLoRA was a problem when I reviewed it. If #1685 merges first, that switch becomes an improvement instead, and I'll take back that part of my review. #1678 (aLoRA silently never activating) is fixed by #1685, so it can close with it. My other points in the review are unchanged.

[jakelorocco]

I agree with Nigel's comments. I deleted my duplicate comments that raised the same issues.

[jakelorocco]

Is there a reason to not just pass the WeightsBinding object here instead of its sub-pieces? Similar to how the actual Identity object is getting passed in as well?