Skip to content

worker: honour backend_factory when reporting the machine's backend - #12

Open
pradipta-lyzr wants to merge 1 commit into
open-gitagent:mainfrom
pradipta-lyzr:fix/worker-honors-backend-factory
Open

worker: honour backend_factory when reporting the machine's backend#12
pradipta-lyzr wants to merge 1 commit into
open-gitagent:mainfrom
pradipta-lyzr:fix/worker-honors-backend-factory

Conversation

@pradipta-lyzr

Copy link
Copy Markdown

What

run_worker resolved the machine's backend name via select_backend("auto") at
registration time, ignoring an explicitly supplied backend_factory.

-be_name = select_backend("auto").name
+be_name = (backend_factory() if backend_factory else select_backend("auto")).name

Why it matters

A worker introduces itself to the hub with its backend name before it starts
taking jobs. Doing that through select_backend("auto") means the machine must
have mlx or torch installed — so a worker started with an explicit
backend_factory raised RuntimeError: No training backend available and died
during registration.

That is the one case backend_factory exists to serve. The parameter was only
half-wired: _run_job already honours the override where the training actually
happens (worker.py:294); registration didn't.

Fallout this fixes

Two tests were failing on main:

- test_full_worker_loop_with_stub_backend
- test_worker_failure_is_reported_not_swallowed

Both hand in a stub backend to exercise the real _Link and the real websocket
without a training stack, and never got past registration. They each also burned
a 15s thread-join timeout waiting on a worker thread that had already died.

- CPU suite: 85 passed / 2 failed → 87 passed
- CPU suite runtime: ~25s → ~5s
- pytest no longer silently requires mlx or torch, which is what CLAUDE.md
already claims of it

Testing

pytest (excluding tests/gpu/, which needs CUDA) — 87 passed.
No new tests: the two that exist covered this exactly, and were failing for it.

run_worker read the backend name straight from select_backend("auto") before
the job loop, so a worker started with an explicit backend_factory still
insisted on resolving a real one — and raised "No training backend available"
on any box without mlx or torch installed. That is precisely the case
backend_factory exists to serve, and _run_job already honoured it; only
registration didn't.

This is why the two hub↔worker tests failed on main: they hand in a stub
backend to exercise the real _Link and the real socket without a training
stack, and never got past registration. They also cost 15 seconds each waiting
on a thread that had already died, so the CPU suite drops from ~25s to ~5s.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant