MARS
The official implementation of MARS: Unleashing the Power of Variance Reduction for Training Large Models
파일 탐색기
최종 버전 다운로드 (.zip)- cifar100_test_acc.png
- cifar100_test_loss.png
- cifar10_test_acc.png
- cifar10_test_loss.png
- fineweb_hella.png
- MARS-AdamW.png
- MARS-Lion.png
- MARS-Shampoo.png
- MARS.png
- ShampooH.png
- small_train.png
- small_val.png
- time_large.png
- time_medium.png
- time_small.png
- val_large.png
- val_medium.png
- val_small.jpg
- val_small.png
- xl_train.png
- xl_val.png
- train_gpt2_large_adamw.py
- train_gpt2_large_mars.py
- train_gpt2_large_muon.py
- train_gpt2_medium_adamw.py
- train_gpt2_medium_mars.py
- train_gpt2_medium_muon.py
- train_gpt2_small_adamw.py
- train_gpt2_small_mars.py
- train_gpt2_small_muon.py
- train_gpt2_xl_adamw.py
- train_gpt2_xl_mars.py
- prepare.py
- fused_mars_kernel.cuh
- multi_tensor_apply.cuh
- type_shim.h
- fused_mars_kernel.cu
- multi_tensor_mars_kernel.cu
- pybind_mars.cpp
- dependency_links.txt
- PKG-INFO
- requires.txt
- SOURCES.txt
- top_level.txt
- adamw.py
- adopt.py
- mars.py
- mars_fused.py
- muon.py
- setup_mars.py
- configurator.py
- cv_utils.py
- model_CNN.py
- model.py
- opt.py
- train_adamw.py
- train_adamw_fw.py
- train_CNN.py
- train_CV.py
- train_mars.py
- train_mars_fw.py
- train_muon.py
- small_train.png
- small_train_global.png
- small_val.png
- small_val_global.png
- val_large.png
- val_large_global.png
- val_medium.png
- val_medium_global.png
- val_small.png
- val_small_global.png
- xl_train.png
- xl_train_global.png
- xl_val.png
- xl_val_global.png
- train_gpt2_large_mars_m.py
- train_gpt2_large_moonlight.py
- train_gpt2_medium_mars_m.py
- train_gpt2_medium_moonlight.py
- train_gpt2_small_mars_m.py
- train_gpt2_small_moonlight.py
- train_gpt2_xl_mars_m.py
- train_gpt2_xl_moonlight.py
- prepare.py
- adamw.py
- mars.py
- mars_m.py
- mars_m_efficient.py
- moonlight.py
- run_CV.sh
- run_mars_m_large.sh
- run_mars_m_medium.sh
- run_mars_m_small.sh
- run_mars_m_small_fw.sh
- run_mars_m_xl_fw.sh
- run_moonlight_large.sh
- run_moonlight_medium.sh
- run_moonlight_small.sh
- run_moonlight_small_fw.sh
- run_moonlight_xl_fw.sh
- configurator.py
- cv_utils.py
- model_CNN.py
- model.py
- README.md
- train_CV.py
- train_mars_m.py
- train_mars_m_fw.py
- train_moonlight.py
- train_moonlight_fw.py
- run_adamw_large.sh
- run_adamw_medium.sh
- run_adamw_small.sh
- run_adamw_small_fw.sh
- run_adamw_xl_fw.sh
- run_CNN.sh
- run_CV.sh
- run_mars_large.sh
- run_mars_medium.sh
- run_mars_small.sh
- run_mars_small_fw.sh
- run_mars_xl_fw.sh
- run_muon_large.sh
- run_muon_medium.sh
- run_muon_small.sh
- LICENSE
- README.md
// repository documentation
Was this content helpful?
(0 ratings)