Skip to content

Add ART Megatron support for GPT OSS#744

Merged
FurtherAI merged 566 commits into
mainfrom
austin/gpt_oss_model_support
Jul 8, 2026
Merged

Add ART Megatron support for GPT OSS#744
FurtherAI merged 566 commits into
mainfrom
austin/gpt_oss_model_support

Conversation

@FurtherAI

Copy link
Copy Markdown
Collaborator

Summary

Adds ART Megatron support for GPT OSS, including native vLLM LoRA serving, GPT OSS LoRA layout conversion/export, MXFP4 expert weight loading, sliding-window attention handling, and model-support workflow coverage.

Notable Non-Handler Changes

  • Add handler-driven HF weight-source planning so GPT OSS MXFP4 grouped expert weights can be materialized through Megatron Bridge while preserving the default direct safetensors load path.
  • Extend LoRA export/publish logic for GPT OSS packed expert layouts and explicit q/k/v target subsets.
  • Add handler-provided vLLM runtime defaults for GPT OSS MoE backend and OpenAI tool parser, while preserving explicit user config.
  • Implement Megatron softmax sink support in ART flex attention and CP attention/executor paths, plus the SM90 FlashAttention block-sparse dQ compatibility patch.
  • Add sparse MoE route-replay metadata lower-bound handling so inferred num_experts is never lower than topk.
  • Add model-support workflow ergonomics and diagnostics: single-stage runs, paired train-inf rerun guidance, and richer forward-trace hooks.
  • Update length trainability for reasoning models: unbounded scenario stream, small per-completion max-token variation, GPT OSS chat-template kwargs/system prompt, 20-step budget, and GPT OSS-specific pass thresholds.
  • Suppress Megatron-Core's expected DeepEP router-prob warning globally in ART provider setup; Megatron still casts bf16/fp16 router probs to fp32 before DeepEP dispatch.

Validation

  • Full GPT OSS model-support workflow passed earlier, including train_inf_mismatch, correctness_sensitivity, length_trainability, and native_vllm_lora.
  • Reviewed by a sub-agent; only low-severity test coverage nits were found and addressed.
  • uv run --no-sync prek run ty --all-files
  • uv run --no-sync prek run ruff --all-files
  • uv run --no-sync prek run ruff-format --all-files
  • uv run --no-sync prek run uv-lock-check --all-files
  • uv run --no-sync pytest --nbval --current-env --tb=short tests/unit/test_moe_routing_real_path.py tests/integration/megatron/model_support/test_workflow.py::test_build_validation_report_only_stage_skips_other_stages

FurtherAI added 27 commits June 29, 2026 00:18
…t_oss_model_support

# Conflicts:
#	src/art/megatron/context_parallel/core_attention.py
#	src/art/megatron/context_parallel/executor.py
#	src/art/megatron/flex_attn/attention.py
#	src/art/megatron/model_support/registry.py
#	tests/integration/megatron/model_support/hf_parity_worker.py
#	tests/integration/megatron/model_support/test_hf_parity_invariants.py
#	tests/integration/megatron/model_support/test_workflow.py
#	tests/integration/megatron/model_support/workflow.py
#	tests/integration/megatron/train_inf_mismatch/real_path.py
#	tests/integration/megatron/trainability/test_live_length_trainability.py
@FurtherAI FurtherAI marked this pull request as ready for review July 8, 2026 03:18
@FurtherAI FurtherAI merged commit 3492eb9 into main Jul 8, 2026
5 checks passed
@FurtherAI FurtherAI deleted the austin/gpt_oss_model_support branch July 8, 2026 03:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant