Skip to content

From NVIDIA Megatron-LM for visibility - #18

Open
RaymondLi0 wants to merge 7491 commits into
bigcode-project:multi-query-attentionfrom
NVIDIA:main
Open

From NVIDIA Megatron-LM for visibility#18
RaymondLi0 wants to merge 7491 commits into
bigcode-project:multi-query-attentionfrom
NVIDIA:main

Conversation

@RaymondLi0

@RaymondLi0 RaymondLi0 commented Jan 24, 2023

Copy link
Copy Markdown
Collaborator

No description provided.

@RaymondLi0
RaymondLi0 changed the base branch from multi-query-attention to before-merge June 20, 2023 20:12
@RaymondLi0
RaymondLi0 changed the base branch from before-merge to multi-query-attention June 20, 2023 20:12
sraman-rgb and others added 27 commits June 25, 2026 23:30
@sraman-rgb
Fix fused MLA down projection with tensor parallelism (#5383)
e1b8454 
Signed-off-by: Siddhartha Raman <sraman@nvidia.com>
@adityasingh2400
Fix NameError in is_flashinfer_min_version when check_equality=False (#…
8bafe7c 
…4961)

Signed-off-by: Aditya Singh <adisin650@gmail.com>
@Phlip79
Add hybrid FSDP unit module support (#4329)
da42015 
Signed-off-by: Philip Petrakian <ppetrakian@nvidia.com>
@ko3n1g @claude
fix: set DATA_PATH for moe-dynamic-inference recipe (#5506)
476228d 
Signed-off-by: oliver könig <okoenig@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@yeyu-nvidia @claude
Add --qad-train-target {base|mtp|both} for QAD / MTP QAT (frozen-base…
c0d7848 
…, frozen-MTP, or co-train) (#4785)

Signed-off-by: Ye Yu <yeyu@nvidia.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
@zhongbozhu
[Main] Generalized fix for mxfp8 param gather (#5236)
0552f29 
Signed-off-by: Zhongbo Zhu <zhongboz@nvidia.com>
@wujingyue
Add CUDA graph training iteration test (#5417)
5949478 
Signed-off-by: Jingyue Wu <wujingyue@gmail.com>
@lauradang
test: restore G/G + lag=19 for gpt_grpo_tp4_pp1_dp2_8b throughput tes…
847de23 
…ts (#5514)

Signed-off-by: Laura Dang <laurad@nvidia.com>
@github-actions
Update copy-pr-bot.yaml [skip ci]
990ced9
@ko3n1g @claude
ci: cache-from a single coherent buildcache donor (#5509)
cbaa6eb 
Signed-off-by: oliver könig <okoenig@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@chtruong814
ci: Use GB300 for Github CI tests (#5520)
ed59a0a 
Signed-off-by: Charlie Truong <chtruong@nvidia.com>
@ko3n1g @claude
ci: pin HF_HUB_CACHE to bind-mounted cache for gpt-oss-20b inference …
0ff7226 
…test (#5512)

Signed-off-by: oliver könig <okoenig@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@deepakn94 @claude
Add inter-document attention masking to GPTDataset (#5298)
f88b85f 
Signed-off-by: Deepak Narayanan <dnarayanan@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
@github-actions
Update copy-pr-bot.yaml [skip ci]
25f6a09
@asolergi-nv
[CI] Fix gpt_dynamic_inference_tp2_pp2_ep2_gptoss_20b_swa tests (#5527
522a9dd 
)

Signed-off-by: Antoni-Joan Solergibert <asolergibert@nvidia.com>
@lmcafee-nvidia
Implement async scheduling for dynamic inference (#5453)
3c327f3 
Signed-off-by: Lawrence McAfee <lmcafee@nvidia.com>
@ko3n1g
build: bump transformer-engine to release_v2.16.post (#5517)
b9f7fb6 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@janEbert
Fix isort target Python version (#5567)
819912c 
Signed-off-by: janEbert <janpabloe@nvidia.com>
@janEbert
Fix PR template typo (#5566)
7b6eb02 
Signed-off-by: janEbert <janpabloe@nvidia.com>
@anlthms
Deduplicate tensor-splitting utility (#5545)
872442a 
Signed-off-by: Anil Thomas <anlthms@users.noreply.github.com>
@Phlip79 @ko3n1g
Add CI duties to oncall (#5510)
99b56a7 
Signed-off-by: Philip Petrakian <ppetrakian@nvidia.com>
Co-authored-by: oliver könig <okoenig@nvidia.com>
@yashaswikarnati
Thread dp_cp/expt_dp process groups through checkpoint load path (#5579)
817c1d5 
Signed-off-by: ykarnati <ykarnati@nvidia.com>
@rapatel
Fix TEGroupedMLP pre-backward unshard in fine-grained FSDP hooks for … (
f285ea5 
#4990)

Signed-off-by: Ritesh Patel <ripatel@nvidia.com>
@maanug-nv
[training migration] Finish ModelBuilder integration (#5516)
223e244 
Signed-off-by: Maanu Grover <maanug@nvidia.com>
@github-actions
Update copy-pr-bot.yaml [skip ci]
4849143
@Phlip79
Use NVIDIA inference credentials for Claude actions (#5589)
a3d761b 
Signed-off-by: Philip Petrakian <ppetrakian@nvidia.com>
@Phlip79
Update PR instructions (#5592)
36d11ce 
Signed-off-by: Philip Petrakian <ppetrakian@nvidia.com>
github-actions Bot and others added 30 commits July 30, 2026 00:25
@github-actions
Update copy-pr-bot.yaml [skip ci]
542ff53
@philipcmonk
Fix gradient counting for muon+expert biases (#6099)
6183f9d 
Signed-off-by: Philip Monk <pmonk@nvidia.com>
@Phlip79
Reorganize a few docs (#6109)
9436656 
Signed-off-by: Philip Petrakian <ppetrakian@nvidia.com>
@fanshiqing
Perf: skip per-param copy_ dispatch in the MXFP8 param copy-back (#6094)
8e57bb6 
Signed-off-by: Shiqing Fan <shiqingf@nvidia.com>
@lmcafee-nvidia @shanmugamr1992 @claude
test: add async scheduling inference coverage (#6104)
354b5d2 
Signed-off-by: shanmugamr1992 <shanmugamr1992@gmail.com>
Signed-off-by: Lawrence McAfee <lmcafee@nvidia.com>
Co-authored-by: shanmugamr1992 <shanmugamr1992@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
@svcnemo-autobot
test(ci): AUT-1173 set functional tests to ft_launcher (#6128)
ab16414 
Signed-off-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>
@AAnoosheh
Fix --freeze-all-layers for new model builder path + unit tests (#5926)
ff6b92a 
Signed-off-by: Asha Anoosheh <aanoosheh@nvidia.com>
@tdene @jalbericiola
Fix inference overflow into dummy blocks (#5950)
3fd8f33 
Signed-off-by: Teodor-Dumitru Ene <teodord.ene@gmail.com>
Co-authored-by: Jorge Albericio <jalbericiola@nvidia.com>
@ko3n1g
fix(tests): prefer staged release assets (#6149)
175b4cc 
Signed-off-by: oliver könig <okoenig@nvidia.com>
@wujingyue @svcnvidia-nemo-ci
Fix Megatron FSDP optimizer under-update (#5976)
ce8a8a7 
Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>
Signed-off-by: svcnvidia-nemo-ci <svcnvidia-nemo-ci@nvidia.com>
Co-authored-by: svcnvidia-nemo-ci <svcnvidia-nemo-ci@nvidia.com>
@wujingyue @claude
Rebind MFSDP v2 sharded grads once per gradient reduction (#6041)
9d416ee 
Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@cuichenx
Fix MLA dynamic inference decode flag (#4902)
0081c84 
Signed-off-by: Chen Cui <chcui@nvidia.com>
@wujingyue
Normalize MFSDP data-parallel axes at fully_shard (#6067)
dc3c9ab 
Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>
@ISEEKYAN
Add Agent Compose package and runtime interface skeleton (#5937)
ab20a60 
Signed-off-by: Yan Bai <bayan@nvidia.com>
@prajwal1210 @claude
Fix GTP full-iteration CUDA-graph capture regression (#6077)
ec53920 
Signed-off-by: Prajwal Singhania <psinghania@nvidia.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
@wdykas
Harden dynamic prefix cache allocation - mamba mostly (#6091)
d04e7a0 
Signed-off-by: William Dykas <wdykas@nvidia.com>
@github-actions
Update copy-pr-bot.yaml [skip ci]
6513e3e
@svcnemo-autobot
ci(cache): AUT-1188 harden container cache selection (#6160)
8543ae4 
Signed-off-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>
@svcnemo-autobot
ci(nvskills): AUT-1188 fix reusable workflow startup failure (#6162)
b19b1f4 
Signed-off-by: svcnemo-autobot <svcnemo-autobot@nvidia.com>
@wujingyue
Add MFSDP runtime schedule design (#6031)
d3519d5 
Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>
@utkarsh530 Utkarsh Utkarsh
@claude @shanmugamr1992 @guihong-nv
Feature/enable inference optimized qwen moe (#5700)
8339200 
Signed-off-by: Utkarsh Utkarsh <uutkarsh@nvidia.com>
Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com>
Signed-off-by: shanmugamr1992 <shanmugamr@nvidia.com>
Co-authored-by: Utkarsh Utkarsh <uutkarsh@nvidia.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com>
Co-authored-by: Guihong Li <guihongl@nvidia.com>
Co-authored-by: shanmugamr1992 <shanmugamr@nvidia.com>
@wdykas root
root root root root root root
Bs invar moe (#4871)
57deb43 
Signed-off-by: William Dykas <wdykas@nvidia.com>
Signed-off-by: root <root@nvl72103-T09.cm.cluster>
Signed-off-by: root <root@nvl72094-T13.cm.cluster>
Signed-off-by: root <root@nvl72098-T15.cm.cluster>
Signed-off-by: root <root@nvl72077-T17.cm.cluster>
Signed-off-by: root <root@nvl72005-T13.cm.cluster>
Signed-off-by: root <root@nvl72053-T14.cm.cluster>
Signed-off-by: root <root@nvl72151-T17.cm.cluster>
Co-authored-by: root <root@nvl72103-T09.cm.cluster>
Co-authored-by: root <root@nvl72094-T13.cm.cluster>
Co-authored-by: root <root@nvl72098-T15.cm.cluster>
Co-authored-by: root <root@nvl72077-T17.cm.cluster>
Co-authored-by: root <root@nvl72005-T13.cm.cluster>
Co-authored-by: root <root@nvl72053-T14.cm.cluster>
Co-authored-by: root <root@nvl72151-T17.cm.cluster>
@balasaajay
chore: update pytest version to 9.1.1 in pyproject.toml and uv.lock (#…
4ac5ad0 
…6170)

Signed-off-by: Ajay Balasa <abalasa@nvidia.com>
@lmcafee-nvidia @santhnm2
Fix LRU block accounting for resumed requests (#5995)
ee38649 
Signed-off-by: Lawrence McAfee <lmcafee@nvidia.com>
Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>
Co-authored-by: Keshav Santhanam <ksanthanam@nvidia.com>
@zhongbozhu
[Main] update moe single weight unit test (#6035)
d378bfb 
Signed-off-by: Zhongbo Zhu <zhongboz@nvidia.com>
@dimapihtar
fix ckpt conversion issue (#6151)
801d7e6 
Signed-off-by: dimapihtar <dpykhtar@nvidia.com>
Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com>
@Connor-XY
Add determinism developer documentation (#5718)
07fdc08 
Signed-off-by: Yan Xu <yxu1@nvidia.com>
@github-actions
Update copy-pr-bot.yaml [skip ci]
4d129d8
@wujingyue @svcnvidia-nemo-ci
Add MFSDP v2 MCore integration with FullyShardedOptimizer (#5865)
313f6e9 
Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>
Signed-off-by: svcnvidia-nemo-ci <svcnvidia-nemo-ci@nvidia.com>
Co-authored-by: svcnvidia-nemo-ci <svcnvidia-nemo-ci@nvidia.com>
@nvcsathe
Add streaming replies for dynamic engine (#5727)
1addfbd 
Signed-off-by: Chaitra Sathe <csathe@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.