Distributed training optimization for large-scale Mixture-of-Experts models.
- Adaptive All-to-All communication
- Dynamic expert clustering
- Topology-aware expert placement
- Up to 3.3× training speedup
Heterogeneity-aware LLM inference across heterogeneous GPU clusters.
- Topology-aware communication
- Memory-aware batch scheduling
- Throughput-aware workload allocation
- Up to 209% throughput improvement
Memory management and SSD-backed expert scheduling for resource-constrained MoE serving.
- GPU–CPU–SSD memory hierarchy
- Expert placement and migration
- Memory pressure simulation
- Serving-oriented scheduling