Skip to content

resolve_adapter() registers into _added_adapters outside _generation_lock #1562

Closed

Description

@planetf1

Epic: #929 | Blocked by: #1465 | Phase: 2

Problem

call_intrinsic (mellea/stdlib/components/intrinsic/_util.py) calls
backend.resolve_adapter(intrinsic_name) to lazily register an adapter on
first use. AdapterMixin.resolve_adapter mutates _added_adapters via
add_adapter() without holding LocalHFBackend._generation_lock.

Meanwhile, every verb that reads _added_adapters on the generation path
(load_peft_adapter, called from _IntrinsicPeftBinding.activate()/
.prepare(), and previously from _generate_with_adapter_lock directly)
does so while holding _generation_lock.

Two concurrent first-time calls for the same previously-unregistered
intrinsic_name can therefore race: both may see the name unregistered,
both call add_adapter(), and the outcome depends on unguarded interleaving
in add_adapter()'s own duplicate-registration check.

Impact

Low likelihood in practice (only triggers on truly concurrent first use of a
never-before-resolved adapter function on one backend instance), but a real
gap: the locking discipline that protects activation/generation doesn't
extend to registration.

resolve_adapter's own docstring already flags a related, narrower gap:

warnings.catch_warnings() modifies the process-global filter state and
is not async/thread-safe. Concurrent first-time resolves race on filter
restoration... Phase 2 (see epic #929) adds a lock.

That comment anticipated a lock being added for this method; #1465 added
reentrancy to _generation_lock for the activation/generation path but did
not extend it to resolve_adapter().

Suggested direction

Have resolve_adapter()'s registration path acquire the same
_generation_lock (now reentrant, so this composes safely with the rest of
the locking added in #1465) around the add_adapter() call, or introduce a
dedicated registration lock if serializing against generation is
undesirable.

Context

Found during #1465's code review (routing intrinsic generation through
adapter_scope). Verified this race's shape is unchanged by #1465 — the
same read-under-lock/write-without-lock pattern existed before that PR too
— so it's pre-existing, not a regression.


Update (2026-09-02) — concrete mechanism, widened race surface, acceptance criteria

The generic hook for this already exists:
AdapterMixin._adapter_activation_lock() (mellea/backends/adapters/adapter.py:588),
overridden by LocalHFBackend to return self._generation_lock
(mellea/backends/huggingface.py:2633) — the same lock LocalFileBinding's
activate/deactivate verbs already take (huggingface.py:2777,2781,2785).
The fix in resolve_adapter() (mellea/backends/adapters/adapter.py:601)
should wrap its add_adapter() call(s) in self._adapter_activation_lock()
rather than a LocalHFBackend-specific lock reference — that keeps the fix
backend-generic: a no-op contextlib.nullcontext() for backends that don't
need serialisation (e.g. OpenAIBackend), matching today's behaviour there.

Race surface is wider than originally scoped. #1018 (PR #1593, merged
2026-09-02) added an embedded-adapter branch to resolve_adapter()
(adapter.py, _uses_embedded_adapters branch): for
load_embedded_adapters=True backends, one resolve_adapter() call now
loops over every EmbeddedIntrinsicAdapter discovered from the checkpoint
and calls self.add_adapter(a) once per adapter — multiple unguarded writes
per resolve, not one. The fix must cover this loop too, not only the
original single-adapter LORA/IntrinsicAdapter path.

Acceptance criteria

Dependencies

Not blocked — #1465 merged 2026-08-25. Ready now.

Activity

  1. added
    area/backendsProvider-specific work: Ollama, HF, LiteLLM, OpenAI, Bedrock, vLLM
    area/adapter-functionsGranite adapter functions: framework and adaptiers including RAG, Guardian, Core
    p2Medium/low: minor bugs, niche features, polish, docs, tests, cleanup. Scoped, lower urgency.
    and removed
    area/backendsProvider-specific work: Ollama, HF, LiteLLM, OpenAI, Bedrock, vLLM
    on Aug 18, 2026
  2. self-assigned this
    on Sep 2, 2026
  3. added a commit that references this issue on Sep 3, 2026
    edfdf33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area/adapter-functionsGranite adapter functions: framework and adaptiers including RAG, Guardian, Corearea/backendsProvider-specific work: Ollama, HF, LiteLLM, OpenAI, Bedrock, vLLMp2Medium/low: minor bugs, niche features, polish, docs, tests, cleanup. Scoped, lower urgency.refactor

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions