OpenDiloco
OpenDiLoCo: An Open-Source Framework for Globally Distributed Low-Communication Training
File Explorer
Download Latest Version (.zip)- check-style.yml
- push-docker-image.yml
- config_14m.json
- config_150m.json
- config_1b.json
- config_2m.json
- config_60m.json
- .gitignore
- __init__.py
- ckpt_utils.py
- fixed_key.pem
- hivemind_diloco.py
- init_weights.py
- run_training.sh
- train_diloco_torch.py
- train_fsdp.py
- utils.py
- pull-c4.sh
- pull-model.py
- config.json
- generation_config.json
- model.safetensors
- test_train.py
- test_diloco_hivemind.py
- .dockerignore
- .gitignore
- .gitmodules
- .pre-commit-config.yaml
- CONTRIBUTING.md
- Dockerfile
- LICENSE
- pyproject.toml
- README.md
- requirements-dev.txt
- requirements.txt
π Installation Guide
git clone https://github.com/PrimeIntellect-ai/OpenDiloco
Downloads the entire project code from GitHub to your computer.
cd OpenDiloco
Moves into the project folder you just downloaded.
2. Official Install Script
Easy Recommended- Python 3 Python is required to use pip.
pip install .
Installs the package published on PyPI directly β no need to clone the source.
pip install --pre torchdata --index-url https://download.pytorch.org/whl/nightly/cpu
Installs the package published on PyPI directly β no need to clone the source.
pip install flash-attn>=2.5.8
Installs the package published on PyPI directly β no need to clone the source.
Pulled directly from this repo's README.
3. Docker
Easy- Git Needed to download the project code from GitHub.
- Docker Desktop Needed to build and run containers. Install it and keep it running in the background.
docker pull primeintellect/open_diloco:main
Type this command into your terminal and run it.
docker run -d --name open-diloco --ipc=host --network=host --gpus=all primeintellect/open_diloco:main
Runs the built image as an actual container.
docker exec -it open-diloco bash
Type this command into your terminal and run it.
Pulled directly from this repo's README.
4. Python
Easypython -m venv .venv
Runs the Python script (or module).
pip install .
Installs the package published on PyPI directly β no need to clone the source.
pip install --pre torchdata --index-url https://download.pytorch.org/whl/nightly/cpu
Installs the package published on PyPI directly β no need to clone the source.
pip install flash-attn>=2.5.8
Installs the package published on PyPI directly β no need to clone the source.
Pulled directly from this repo's README.