Repository navigation

feat: add scheduler backends (local/k8s/slurm) with unified CLI - #10

Merged

Terrence (TerrenceZhangX) merged 56 commits into

microsoft:mainfrom

TerrenceZhangX:zhangt/scheduler-support

Mar 20, 2026

Merged

feat: add scheduler backends (local/k8s/slurm) with unified CLI#10
Terrence (TerrenceZhangX) merged 56 commits into
microsoft:mainfrom
TerrenceZhangX:zhangt/scheduler-support

Commits on Mar 17, 2026
- feat: add K8s and Slurm scheduler backends for profiling jobs
  Terrence Zhang
  committed
  28301e0
- feat: switch to proper API clients for remote submission
  Terrence Zhang
  committed
  26c9f47
- chore: add proper pyproject.toml with dependency declarations
  Terrence Zhang
  committed
  52294a6
- fix: make pip install -e . and entry point actually work
  Terrence Zhang
  committed
  54b6152
- refactor: unified CLI as 'flowsim submit' instead of 'flowsim-submit'
  Terrence Zhang
  committed
  9e1c1f4
- fix: 'flowsim submit' submits by default, --dry-run to preview
  Terrence Zhang
  committed
  d37d8f3
- fix: validate cluster connection params before submit
  Terrence Zhang
  committed
  af48f0c
- feat: support env vars for cluster connection params
  Terrence Zhang
  committed
  87a2c33
- feat: config-first approach with flowsim init
  Terrence Zhang
  committed
  63ab491
- refactor: flowsim init takes CLI args instead of interactive prompts
  Terrence Zhang
  committed
  7116fef
- feat: PD disaggregation support + multi-node Docker test infra
  Terrence Zhang
  committed
  8987c38
- chore: add dev-setup.sh/dev-teardown.sh for one-click test clusters
  Terrence Zhang
  committed
  6f1bda2
- feat: add local scheduler backend — flowsim submit --scheduler local
  Terrence Zhang
  committed
  d2bb08e
- test: add 61 unit tests for scheduler CLI, backends, and config
  Terrence Zhang
  committed
  e5e303c
- feat: persistent logs under output_dir, flowsim status/logs, refuse K8s submit without PVC
  Terrence Zhang
  committed
  c60cd11
- fix: flowsim logs shows all log files (stdout + stderr) with listing
  Terrence Zhang
  committed
  ea3c27a
- fix: flowsim logs shows file locations + actionable commands instead of dumping content
  Terrence Zhang
  committed
  eb46c36
- test: integration tests for all 3 scheduler backends (local/k8s/slurm)
  Terrence Zhang
  committed
  0e59219
- feat: align CLI with standard job platform APIs
  Terrence Zhang
  committed
  2c36af0
- fix: CLI only shows scheduler-specific args based on --scheduler
  Terrence Zhang
  committed
  8cd62f8
- fix: use python3 instead of python in profile command
  Terrence Zhang
  committed
  84c8953
- fix: use YYYYMMDD_HHMMSS timestamp in log filenames
  Terrence Zhang
  committed
  fab6314
Commits on Mar 18, 2026
- scheduler: add Slurm CLI mode (sbatch/squeue/scancel) + integration test
  Terrence Zhang
  committed
  8f76052
- slurm: use YYYYMMDD_HHMMSS timestamp for output dirs (consistent with local/k8s)
  Terrence Zhang
  committed
  3edd5f4
- docs: add scheduler README with CLI usage and architecture overview
  Terrence Zhang
  committed
  9bc2d94
- profile: add --sweep for multi-point profiling in one job
  Terrence Zhang
  committed
  a92e32b
- test: add sweep integration tests (inline + file)
  Terrence Zhang
  committed
  0aeff89
- refactor: dedup shared utilities, deprecate Slurm REST mode
  Terrence Zhang
  committed
  f58d791
- review: fix remaining issues (stale docstring, unused vars, README defaults)
  Terrence Zhang
  committed
  3152a72
- remove Slurm REST dead code, rewrite README in English
  Terrence Zhang
  committed
  19973cb
- review: normalize Slurm states, implement _logs_cli, dedup image check, add env var docs
  Terrence Zhang
  committed
  892eeac
- fix: remove --slurm-submit-via from integration tests
  Terrence Zhang
  committed
  73cedbe
- refactor: move test infra to tests/integration/infra/, delete unused templates
  Terrence Zhang
  committed
  0a30f7f
- remove untested PD disaggregation code
  Terrence Zhang
  committed
  7831272
Commits on Mar 19, 2026
- simplify flowsim init: write annotated template instead of argparse
  Terrence Zhang
  committed
  b6dbbbb
- add --config flag to flowsim init
  Terrence Zhang
  committed
  95028db
- use template files for flowsim init instead of inline strings
  Terrence Zhang
  committed
  ac41690
- update README: reflect template-file init with --config option
  Terrence Zhang
  committed
  da8ab00
- docs: streamline READMEs, unify examples, remove legacy manual workflow
  Terrence Zhang
  committed
  059f3ea
- format: fix with black
  Terrence Zhang
  committed
  b0dfdd5
- docs: add --existing-ctx and --decode-tokens to all examples, default decode-tokens to 2
  Terrence Zhang
  committed
  9e2541a
- refactor: remove PyYAML fallback, make it a core dependency
  Terrence Zhang
  committed
  880fe05
- fix: reject k8s submit when no PVC or hostPath configured
  Terrence Zhang
  committed
  236548a
- docs: add missing parameters
  Terrence Zhang
  committed
  9daee82
- fix: unique job names, Slurm exclusive GPU, remove list_jobs prefix filter
  Terrence Zhang
  committed
  5e3d1bb
- refactor: restructure CLI into scripts/cli/ subpackage
  Terrence Zhang
  committed
  2a718a7
- fix: use runtime:nvidia for slurm compute node GPU access
  Terrence Zhang
  committed
  31dc15b
- refactor: rename test_scheduler_local.py → test_scheduler.py, rewrite header
  Terrence Zhang
  committed
  b7ec2cb
- docs: update output structure to include logs, merged, and shape dirs
  Terrence Zhang
  committed
  7d40889
Commits on Mar 20, 2026
- Update scripts/cli/submit.py
  
  Terrence (TerrenceZhangX)
  and
  Copilot
  authored
  86dd517
- Update schedulers/local.py
  
  Terrence (TerrenceZhangX)
  and
  Copilot
  authored
  2dbb896
- Update tests/integration/infra/kind-multi-node.yaml
  
  Terrence (TerrenceZhangX)
  and
  Copilot
  authored
  f30329a
- Update tests/integration/infra/slurm-compose.yaml
  
  Terrence (TerrenceZhangX)
  and
  Copilot
  authored
  4718764
- Update tests/integration/infra/slurm.conf
  
  Terrence (TerrenceZhangX)
  and
  Copilot
  authored
  8f79054
- fix: parameterize hardcoded host paths in slurm-compose
  Terrence Zhang
  committed
  7d64cfc
- fix: auto-mount output_dir in docker/enroot container modes
  Terrence Zhang
  committed
  76e75bd

Commits

Commits on Mar 17, 2026

Commits on Mar 18, 2026

Commits on Mar 19, 2026

Commits on Mar 20, 2026