Repository navigation
bug(adapters): detect when an aLoRA cannot activate, and stop selecting it silently #1678
Description
What happens today
You call an adapter function on a backend that loads adapters locally, say core.requirement_check(...).
Mellea downloads an aLoRA, loads it, and reports it as active. You get a score back that looks
perfectly normal.
If the adapter's activation point never appears in the prompt Mellea built, the adapter never switches
on. It contributes nothing it was trained to contribute. Your score came from the base model instead.
Nothing in the returned value, the logs, or the telemetry says so. list_adapters() shows the adapter.
active_adapters() on the model shows the adapter. Every signal says it ran.
Why that is a problem
An aLoRA only applies from the point where a declared token sequence appears in the input. PEFT
implements this, and when the sequence is missing it gives up quietly:
# peft/tuners/lora/variants.py, calculate_alora_offsets()
else: # Invocation sequence not found in input
alora_offsets[i] = NoneThere is no warning and no error. So a user gets a number with no way to know it is worthless, and we
have no way to know either. This is worse than a failure, because the value is well formed and passes
every schema check we apply to it.
We have one confirmed case in the published adapters right now. The requirement-check aLoRA declares
an activation sequence its own instruction text cannot produce, so on Granite 4.1 models it never
activates. Details are in the linked upstream issue.
How it should behave
When Mellea loads an aLoRA and the declared activation sequence is absent from the prompt it assembled,
one of two things should happen:
- the situation is reported clearly, naming the capability, the base model, and the decoded sequence, or
- the adapter is treated as unavailable, so resolution falls through to the next rung and the user gets
a working adapter instead.
Either way it stops being invisible.
Where the check belongs
Mellea is the only component that holds both halves at once. It has the loaded adapter's config and the
prompt it just built. PEFT sees the mismatch and says nothing, the base model cannot know, and the user
has no access to either. The check itself is a subsequence search over token ids we already have.
Detail: the check has to run against the assembled prompt, not the instruction
Not every aLoRA activates on text from its own io.yaml. Of the five aLoRA-backed capabilities
published for granite-4.1-3b:
uncertaintydeclares<certainty>, which its instruction contains.answerability,query_rewriteandquery_clarificationdeclare
<|start_of_role|>assistant<|end_of_role|>, which the chat template adds.requirement-checkdeclares<requirements>, which its instruction defeats through tokenisation.
A test of the form "the sequence must appear in the instruction" would wrongly reject three of the five.
Detail: possible shapes
- Generation-time guard. After building
input_ids, if the active adapter declares
alora_invocation_tokensand the sequence is absent, log a warning. Cheapest option, covers
template-driven activation, and turns a silent wrong answer into something diagnosable. - Resolution-time gate. Treat an aLoRA that cannot activate as unavailable so the walk selects the
LoRA. This is what "prefer aLoRA where the infrastructure supports it" means in practice. Needs care,
because at resolve time the chat template has not been applied yet. - Both, with the gate deciding selection and the guard catching whatever the gate cannot predict.
Backends that load PEFT adapters all inherit the problem. For server-mediated and embedded deployments
activation happens outside Mellea, so those paths need answering separately rather than assuming.
Acceptance criteria
- An aLoRA whose declared activation sequence is absent from the assembled prompt is either reported
clearly or treated as unavailable, on every backend that loads adapters itself. - Tests cover both arms: a sequence that is present activates and stays quiet, a sequence that is
absent is reported or skipped. - A pin bump that promotes an aLoRA rung carries evidence that the aLoRA is no worse than the LoRA on
accuracy and on false-pass rate for that capability. The catalogue pins commit SHAs
(_CORE_R1_SHAand siblings inmellea/backends/adapters/catalog.py), so an upstream fix only
reaches users when somebody bumps a pin. That bump is the moment an adapter stops being skipped and
becomes the default path, which makes it the right place to ask for evidence. - A comment beside the pinned SHAs in
catalog.pynames that requirement, so whoever bumps them finds
it where they are working. - This issue owns the flip back to aLoRA-first. Following the review of feat(adapters)!: prompt fallback for adapter functions without a trained adapter #1654, that PR keeps today's
trained-rung order and probes LoRA before aLoRA, so promotion no longer happens through a pin bump: it
happens by reordering the probe in_register_available_composed_adapter. Once the guard exists and the
evidence question in bug(adapters): publishedrequirement-checkaLoRA never activates, and scores worse than the LoRA #1679 is settled, reorder it here. Criterion 3 stays as the trigger for any other
capability whose aLoRA arrives through a catalogue bump rather than through this reorder.
Related
Found while reviewing #1654, which makes the aLoRA rung selectable on the local Hugging Face path for
the first time. The published artefact problem that motivates criteria 3 and 4 is tracked in
#1679. Downstream visibility of a changed mechanism is #1680. The equivalent
question for server-mediated adapters is noted on #1651.
Activity
[-]bug(adapters): an aLoRA that cannot activate is used silently, with no signal to the user[/-][+]bug(adapters): detect when an aLoRA cannot activate, and stop selecting it silently[/+]planetf1 commentedon Sep 25, 2026
building a fix
docs(adapters): trim transient issue references from aLoRA guard docs
fix(adapters): raise or fall back when an aLoRA cannot activate (gene…
fix(adapters): raise or fall back when an aLoRA cannot activate (gene…
fix(adapters): raise or fall back when an aLoRA cannot activate (gene…
fix(adapters): raise or fall back when an aLoRA cannot activate (gene…