mistral
Mistral: A strong, northwesterly wind: Framework for transparent and accessible large-scale language model training, built with Hugging Face π€ Transformers.
νμΌ νμκΈ°
μ΅μ’ λ²μ λ€μ΄λ‘λ (.zip)- bug_report.md
- feature_request.md
- question.md
- pre-commit.yaml
- run-tests.yaml
- pull_request_template.md
- gpt2-benchmark-config.yaml
- gpt2-intensive-config.yaml
- gpt2-toy-config.yaml
- gpt2-mistral-medium-gcheck-config.yaml
- gpt2-debug-config.yaml
- gpt2-mistral-medium-config.yaml
- gpt2-mistral-medium-gcp-config.yaml
- gpt2-mistral-mini-config.yaml
- gpt2-mistral-small-gcp-config.yaml
- gpt2-scaling-config.yaml
- tutorial-gpt2-micro.yaml
- openwebtext.yaml
- shakespeare.yaml
- wikitext103.yaml
- wikitext2.yaml
- debug-conf.json
- hostfile
- z1-conf.json
- z1-offload-conf.json
- z2-debug-conf.json
- z2-medium-conf.json
- z2-offload-conf.json
- z2-small-conf.json
- z3-conf.json
- z3-offload-conf.json
- mistral-medium.yaml
- mistral-micro.json
- mistral-micro.yaml
- mistral-small.yaml
- benchmark.yaml
- gpt2-medium.yaml
- gpt2-small-short.yaml
- gpt2-small.yaml
- intensive.yaml
- mistral-medium.yaml
- mistral-micro.yaml
- mistral-small.yaml
- train_schema.py
- tutorial-shakespeare-gpt2-micro.yaml
- pydata-custom.css
- readthedocs-custom.css
- custom-class-template.rst
- custom-module-template.rst
- layout.html
- config.rst
- download.rst
- evaluate.rst
- install.rst
- train-output.txt
- train.rst
- wandb_example.png
- build_download_tables.py
- cluster_basics.png
- deepspeed.rst
- gcp_plus_kubernetes.rst
- generate.rst
- gke_standard.png
- kubernetes_menu.png
- multi-gpu.rst
- node_pool.png
- node_pool_gpu.png
- resume.rst
- tutorial_cluster.png
- api.rst
- conf.py
- contributing.rst
- fork.png
- getting_started.rst
- hugging_face_differences.rst
- index.rst
- LICENSE
- Makefile
- mistral_components.png
- README.md
- Dockerfile
- environment-cpu.yaml
- environment-gpu.yaml
- environment-m1.yaml
- export.py
- Dockerfile
- job-gpt2-micro.yaml
- pod-gpu.yaml
- pod.yaml
- run-demo-job.sh
- mistral-gpt2-medium.sh
- mistral-gpt2-small.sh
- ddp-multi.sh
- deepspeed-multi.sh
- fairscale-multi.sh
- ddp-multi.sh
- ddp-single.sh
- deepspeed-multi.sh
- deepspeed-single.sh
- ds-evaluation-bsz.sh
- fairscale-multi.sh
- fairscale-single.sh
- README.md
- vanilla.sh
- resume-single-node.sh
- mistral-sanity-gpt2-small.sh
- ddp.sh
- deepspeed.sh
- fairscale.sh
- multi-node.sh
- single-node.sh
- forget-me-not.sh
- mistral-gcp-gpt2-medium.sh
- mistral-gcp-gpt2-small.sh
- mistral-gpt2-medium.sh
- mistral-gpt2-small.sh
- README.md
- conda-requirements.txt
- pip-requirements.txt
- setup.sh
- test-requirements.txt
- __init__.py
- training_args.py
- __init__.py
- callbacks.py
- trainer.py
- __init__.py
- auto.py
- detokenization.py
- indexer.py
- tokenization_utils.py
- __init__.py
- auto_clm.py
- __init__.py
- overwatch.py
- __init__.py
- paths.py
- registry.py
- __init__.py
- wikitext103.yaml
- wikitext2-detokenized.yaml
- wikitext2.yaml
- z1-conf.json
- z2-small-conf.json
- gpt2-micro.json
- gpt2-micro.yaml
- gpt2-small.yaml
- gpt2-small-diff.yaml
- gpt2-small.yaml
- train-diff.yaml
- train.yaml
- pip-requirements.txt
- __init__.py
- README.md
- run_deepspeed_tests.py
- test_args.py
- test_checkpoint.py
- test_eval_loss_is_defined.py
- test_fp.py
- test_indexed_dataset.py
- test_online_benchmark_trainer.py
- test_seed.py
- test_valid_configs.py
- shakespeare.train.jsonl
- shakespeare.validation.jsonl
- README.md
- README.md
- .flake8
- .gitignore
- .pre-commit-config.yaml
- CONTRIBUTING.md
- generate_text.ipynb
- LICENSE
- Makefile
- mistral_models.json
- mypy.ini
- pyproject.toml
- README.md
- train.py
π μ€μΉ κ°μ΄λ
git clone https://github.com/stanford-crfm/mistral
κΉνλΈμμ νλ‘μ νΈ μ½λ μ 체λ₯Ό λ΄ μ»΄ν¨ν°λ‘ λ΄λ €λ°μ΅λλ€.
cd mistral
λ°©κΈ λ΄λ €λ°μ νλ‘μ νΈ ν΄λ μμΌλ‘ μ΄λν©λλ€.
2. Docker
μ¬μ μΆμ²- Git GitHubμμ νλ‘μ νΈ μ½λλ₯Ό λ΄λ €λ°μΌλ €λ©΄ νμν©λλ€.
- Docker Desktop 컨ν μ΄λλ₯Ό λΉλνκ³ μ€ννλ €λ©΄ νμν©λλ€. μ€μΉ ν μ€νν΄μ λ°±κ·ΈλΌμ΄λμ μΌλμΈμ.
docker build -f environments/Dockerfile -t mistral .
Dockerfileμ κΈ°λ°μΌλ‘ μ€ν κ°λ₯ν μ΄λ―Έμ§λ₯Ό λΉλν©λλ€.
docker run -p 8080:80 mistral
λΉλλ μ΄λ―Έμ§λ₯Ό μ€μ 컨ν μ΄λλ‘ μ€νν©λλ€.
3. Python
μ¬μpip install -r setup/pip-requirements.txt
requirements.txt λ±μ λͺ μλ νμ΄μ¬ λΌμ΄λΈλ¬λ¦¬λ₯Ό μ€μΉν©λλ€.
μ΄ λ ν¬μ READMEμ μ ν μ€μ λͺ λ Ήμ΄λ₯Ό κ·Έλλ‘ κ°μ Έμμ΅λλ€.
4. Make
보ν΅- Git GitHubμμ νλ‘μ νΈ μ½λλ₯Ό λ΄λ €λ°μΌλ €λ©΄ νμν©λλ€.
- Make Linux/macOSλ λ³΄ν΅ κΈ°λ³Έ μ€μΉλμ΄ μμ΅λλ€. Windowsλ λ³λ μ€μΉ(μ: MSYS2, WSL)κ° νμν©λλ€.
make
μμ±λ λΉλ μ€μ μ λ°νμΌλ‘ μ€μ μ»΄νμΌμ μ§νν΄ μ€ν νμΌμ λ§λλλ€.
