Issues

• Showcase / question: a board-proven offline language runtime on ESP32-C3, and whether some language capability may eventually reduce dependence on dense training paths

  foundation-model-stack/fms-fsdp#153 · Filter by author Juemian-Deng opened on Mar 18, 2026

• Inconsistency of filehandler which causes TypeError: ArrowHandler.is_legal() missing 1 required positional argument: 'filepath'

  foundation-model-stack/fms-fsdp#142 · Filter by author dangxuanhong opened on May 21, 2025

• Do not scan the same files multiple times in ScalableShardDataset

  foundation-model-stack/fms-fsdp#137 · Filter by author rualark opened on Apr 21, 2025
  2

  

• [Checkpoint] Rewrite Checkpointing

  foundation-model-stack/fms-fsdp#135 · Filter by author lchu6 opened on Apr 8, 2025
  1

  

• [PP + EP][Stage I] PP x Mamba

  foundation-model-stack/fms-fsdp#134 · Filter by author lchu6 opened on Mar 24, 2025
  1

  

• [PP + EP][Master Thread] Enable Pipeline Parallelism (PP) and Expert Parallelism (EP)

  foundation-model-stack/fms-fsdp#133 · Filter by author lchu6 opened on Mar 23, 2025
  11

  

• [Multimodal] Timeline for Creating Image-Text Dataloader

  foundation-model-stack/fms-fsdp#131 · Filter by author Nano1337 opened on Jan 26, 2025

• selective activation checkpoint with cpu offload

  foundation-model-stack/fms-fsdp#130 · Filter by author lllllinux opened on Jan 24, 2025

• add Mamba-MoE training support

  foundation-model-stack/fms-fsdp#128 · Filter by author lchu6 opened on Jan 23, 2025
  2

  

• Handling edge case of continued pretraining from finished run

  foundation-model-stack/fms-fsdp#126 · Filter by author daviswer opened on Jan 14, 2025
  1

  

• Dataset format restrictions

  foundation-model-stack/fms-fsdp#124 · Filter by author 401qingkong opened on Dec 19, 2024
  1

• Support for loading arbitrary huggingface models (llama, mixtral, gpt_bigcode) for speculative training

  enhancement
  foundation-model-stack/fms-fsdp#117 · Filter by author JRosenkranz opened on Sep 25, 2024

• Support for chat templates in speculative training

  enhancement
  foundation-model-stack/fms-fsdp#116 · Filter by author JRosenkranz opened on Sep 25, 2024

• out-of-memory when specu-training a llama2-13b model

  foundation-model-stack/fms-fsdp#113 · Filter by author YunChen1227 opened on Aug 28, 2024
  1

• tokenization on-the-fly for long documents

  foundation-model-stack/fms-fsdp#106 · Filter by author dangxuanhong opened on Jul 31, 2024
  2

  

• FMS-FSDP running on A100 8GPU machine failed with NCCL error messages

  foundation-model-stack/fms-fsdp#94 · Filter by author HenryTangMain opened on Jun 11, 2024

• The default model variant is 7b but it is not supported.

  foundation-model-stack/fms-fsdp#93 · Filter by author HenryTangMain opened on Jun 11, 2024
  2

• Repeatability of Small Model Training Script with fixed seed(s) and same dataset

  foundation-model-stack/fms-fsdp#92 · Filter by author pad9153 opened on Jun 6, 2024
  1

  

• A revisit on improving the performance of Data Loader

  foundation-model-stack/fms-fsdp#70 · Filter by author lchu6 opened on Apr 5, 2024
  2

• [speculator training] Support for loading different HF checkpoints for speculator training

  speculator training
  foundation-model-stack/fms-fsdp#66 · Filter by author pavi2707 opened on Mar 28, 2024
  1

• A write-up on Meta Device Init x Pretraining

  foundation-model-stack/fms-fsdp#64 · Filter by author lchu6 opened on Mar 26, 2024
  

• maximize mistral throughput

  foundation-model-stack/fms-fsdp#63 · Filter by author aldopareja opened on Mar 25, 2024
  2

• revert "raise Dynamo accumulated cache size limit"

  foundation-model-stack/fms-fsdp#53 · Filter by author lchu6 opened on Mar 19, 2024
  

• add FLOP counter

  enhancement
  foundation-model-stack/fms-fsdp#29 · Filter by author lchu6 opened on Feb 23, 2024