pytorch-memory-optim
This code repository contains the code used for my "Optimizing Memory Usage for Training LLMs and Vision Transformers in PyTorch" blog post.
파일 탐색기
최종 버전 다운로드 (.zip)- 08_baseline.py
- 08a_fsdp-defaults.py
- 08b_fsdp-custom.py
- 08c_fsdp-size-wrap.py
- 09_fsdp-act-checkp.py
- 10_fsdp-with-cpu-offload.py
- 10b_fsdp-with-cpu-offload-no-act-check.py
- local_utilities.py
- overview.png
- .gitignore
- 01-2_pytorch-fabric.py
- 01_pytorch-vit.py
- 02_mixed-precision.py
- 03_bfloat16.py
- 04_lower-batchsize.py
- 05_gradient-accum.py
- 06_sgd-with-scheduler.py
- 07_01_init-module.py
- 07_02_init-module.py
- 07_03_init-module.py
- 08_baseline.py
- 08a_fsdp-defaults.py
- 08b_fsdp-custom.py
- 08c_fsdp-size-wrap.py
- 09_fsdp-act-checkp.py
- 10_fsdp-with-cpu-offload.py
- 10b_fsdp-with-cpu-offload-no-act-check.py
- 11_delay-allocation.py
- 12_fsdp-overlap.py
- bonus_bigbird-after.py
- bonus_bigbird-before.py
- bonus_distilbert-after.py
- bonus_distilbert-before.py
- LICENSE.txt
- local_utilities.py
- logs.md
- README.md
- requirements.txt
// repository documentation
Was this content helpful?
(0 ratings)