llama.cpp

(β˜… 124,999)

LLM inference in C/C++

File Explorer

Download Latest Version (.zip)

Showing a partial file list β€” download the zip above to see everything.

  • .clang-format
  • .clang-tidy
  • .dockerignore
  • .ecrc
  • .editorconfig
  • .flake8
  • .gitignore
  • .gitmodules
  • .pre-commit-config.yaml
  • AGENTS.md
  • AUTHORS
  • build-xcframework.sh
  • CLAUDE.md
  • CMakeLists.txt
  • CMakePresets.json
  • CODEOWNERS
  • CONTRIBUTING.md
  • convert_hf_to_gguf.py
  • convert_hf_to_gguf_update.py
  • convert_llama_ggml_to_gguf.py
  • convert_lora_to_gguf.py
  • flake.nix
  • LICENSE
  • Makefile
  • mypy.ini
  • pyproject.toml
  • pyrightconfig.json
  • README.md
  • requirements.txt
  • SECURITY.md

# Installation Guide

1. Get the code
git clone https://github.com/ggml-org/llama.cpp

Downloads the entire project code from GitHub to your computer.

cd llama.cpp

Moves into the project folder you just downloaded.

2. CMake

Medium Recommended
Prerequisites
  • Git Needed to download the project code from GitHub.
  • CMake The tool used to generate build configuration.
  • C/C++ 컴파일러 Windows needs Visual Studio (Community edition, free), macOS needs Xcode Command Line Tools, Linux needs the gcc/g++ package.
mkdir build && cd build

Creates a folder to hold the build output and moves into it.

cmake ..

Analyzes the source code and generates build configuration files (must be run inside the build folder).

make

Compiles the code based on the generated build configuration to produce an executable.

βœ… Check that an executable was created inside the build folder, then run it directly (e.g. ./build/app_name).

3. Node.js

Easy
Prerequisites
  • Git Needed to download the project code from GitHub.
  • Node.js Node.js must be installed to use npm. The LTS version is recommended.
⚠️ This is a large repository, so this method may point to an internal sub-package rather than the actual core product. Check the full README as well.
cd tools/ui

This project's files live in a subfolder, so move into it first.

npm install

Downloads and installs the libraries listed in package.json.

npm start

Starts the development/run server.

βœ… After running the command, open the address shown in the terminal (usually something like http://localhost:3000) in your browser.

4. Python

Easy
Prerequisites
  • Git Needed to download the project code from GitHub.
  • Python 3 On Windows, be sure to check 'Add Python to PATH' during installation.
pip install -r requirements.txt

Installs the Python libraries listed in requirements.txt (or similar).

python <μ‹€ν–‰ν•  파일λͺ…>.py # READMEμ—μ„œ μ •ν™•ν•œ μ‹€ν–‰ 파일λͺ…을 ν™•μΈν•˜μ„Έμš”

Runs the Python script (or module).

βœ… If it runs without errors and prints output in the terminal, it worked.

5. Make

Medium
Prerequisites
  • Git Needed to download the project code from GitHub.
  • Make Usually pre-installed on Linux/macOS. On Windows, install separately (e.g. via MSYS2 or WSL).
make

Compiles the code based on the generated build configuration to produce an executable.

βœ… If it finishes without errors, it worked. Try running the generated executable directly.

Today's picks

d3 β˜… 113,510

Bring data to life with SVG, Canvas and HTML. :bar_chart::chart_with_upwards_trend::tada:

Shell
awesome-design-md β˜… 109,537

A collection of DESIGN.md files analysis by popular brand design systems. Drop one into yo
awesome-react β˜… 74,352

A collection of awesome things regarding React ecosystem
puppeteer β˜… 95,481

JavaScript API for Chrome and Firefox

TypeScript
skills β˜… 170,812

Public repository for Agent Skills

Python
excalidraw β˜… 130,146

Virtual whiteboard for sketching hand-drawn like diagrams

TypeScript
// repository documentation