Repository navigation

(fsdp2 dev.) feat(moe): expert parallelism for MoE drafters on the FSDP2 backend (stacked on #915 + #931) - #932

Draft

yushengsu-thu wants to merge 12 commits into

sgl-project:mainfrom

yushengsu-thu:moe-ep-fsdp2

Commits on Oct 2, 2026

Add optional FSDP2 training backend
yushengsu-thu
committed
3acf1ae
Fix FSDP2 parameter retention and root resharding
yushengsu-thu
committed
11841f7

Commits on Oct 5, 2026

feat: MoE FFN skeleton for DFlash-family drafters (contracts, config, seams)

authored and
yushengsu-thu
committed
80fe135
moe: balance controllers observe the full RoutingResult

authored and
yushengsu-thu
committed
d83bc8f
feat: DeepSeek-V4 MoE FFN for DFlash-family drafters + DSV4-Flash ablation arm

authored and
yushengsu-thu
committed
cfc53a4
fix: make MoE drafter exports servable and reloadable

authored and
yushengsu-thu
committed
3b91aa2
feat: optional auxiliary balance loss for the noaux_tc controller

authored and
yushengsu-thu
committed
1c7ae9e
feat: warm-start MoE drafters from a DeepSeek-V4 target's experts

authored and
yushengsu-thu
committed
337960e
fix: memory-lean, rank-staggered warm start for large MoE drafters

authored and
yushengsu-thu
committed
174b599
feat: freeze warm-started MoE experts and keep them replicated under FSDP

authored and
yushengsu-thu
committed
fd0745e
Merge fork/codex/fsdp2-backend (#915) into the rebased MoE stack
yushengsu-thu
committed
7965b56
feat(moe): expert parallelism for DFlash-family MoE drafters on the FSDP2 backend
yushengsu-thu
committed
12c3f9d