Use Modal async API to remove executor concurrency bottleneck

[msaroufim]

Summary

• invoke deployed Modal functions with the native asynchronous remote.aio() API
• avoid routing long-lived Modal calls through Python's CPU-sized default thread pool
• add a focused regression test covering the async call, CUDA include configuration, returned result, and progress messages

Why

ModalLauncher.run_submission() currently wraps blocking Function.remote() in loop.run_in_executor(None, ...). Python sizes that shared default executor from the KernelBot host's CPU count, so the API host can silently cap Modal concurrency far below BackgroundSubmissionManager.max_workers and the Modal workspace GPU quota.

For example, an API container exposing three CPUs receives a seven-thread default executor. Even with 24 submission workers and a 50-GPU Modal allowance, only seven blocking calls can reach Modal at once; the rest wait locally in the executor.

Modal provides remote.aio() specifically for asynchronous callers, so this change removes that accidental CPU-derived bottleneck.

Benefits

• lets KernelBot's existing 24-worker policy govern submission concurrency instead of API-host CPU count
• allows up to 24 ordinary Modal calls or 48 public/secret ranked calls to be in flight
• uses the Modal SDK's native non-blocking interface and avoids holding one local thread per remote GPU evaluation
• reduces hidden local queueing and makes Modal's backlog and GPU capacity the visible limiting layers
• preserves the existing remote function, configuration, return value, exception propagation, and progress reporting behavior

Validation

• uv run pytest tests/test_modal.py -m 'not integration' -q — 3 passed, 11 deselected
• uv run ruff check src/libkernelbot/launchers/modal.py tests/test_modal.py — passed
• local GPU Mode experiment using remote.aio() — 12 ranked submissions produced 24 T4 calls, all completed, with 20 peak remote containers
• broader backend + Modal unit invocation reached 7 passing tests; 3 database-backed tests could not start because the local Docker daemon was unavailable

[github-actions]

Coverage report

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
src/libkernelbot
utils.py
Project Total

This report was generated by python-coverage-comment-action