Issues

• Setup Finemath-4plus dataset for training

  LocalResearchGroup/llm-foundry#42 · Filter by author matdmiller opened on Feb 25, 2025
  1

  

• Add smoketests for our new features

  LocalResearchGroup/llm-foundry#27 · Filter by author vishalbakshi opened on Feb 8, 2025

• Port ModernBERT's Flash Attention Implementation for Training

  LocalResearchGroup/llm-foundry#26 · Filter by author vishalbakshi opened on Feb 8, 2025
  

• Remove YAMLs that we're not going to use

  LocalResearchGroup/llm-foundry#21 · Filter by author vishalbakshi opened on Feb 2, 2025
  1

  

• Port over unpadding from ModernBERT to Llama

  LocalResearchGroup/llm-foundry#20 · Filter by author vishalbakshi opened on Feb 2, 2025
  1

• Port ModernBERT sequence packing dataloader to our repo

  LocalResearchGroup/llm-foundry#19 · Filter by author vishalbakshi opened on Feb 2, 2025
  1

  

• Have our own Llama Implementation which doesn't depend on HuggingFace

  LocalResearchGroup/llm-foundry#18 · Filter by author vishalbakshi opened on Feb 2, 2025
  1

  

• Commit code to repositority for initial basic training runs

  high priority
  LocalResearchGroup/llm-foundry#13 · Filter by author vishalbakshi opened on Jan 27, 2025
  9

  

• Model and Dataset Size Selection

  LocalResearchGroup/llm-foundry#7 · Filter by author vishalbakshi opened on Jan 27, 2025