vllm

(β˜… 89,633)

A high-throughput and memory-efficient inference and serving engine for LLMs

File Explorer

Download Latest Version (.zip)

Showing a partial file list β€” download the zip above to see everything.

  • .clang-format
  • .coveragerc
  • .dockerignore
  • .git-blame-ignore-revs
  • .gitignore
  • .markdownlint.yaml
  • .pre-commit-config.yaml
  • .readthedocs.yaml
  • .shellcheckrc
  • AGENTS.md
  • build_rust.sh
  • build_vllm_ppc64le.sh
  • CLAUDE.md
  • CMakeLists.txt
  • CODE_OF_CONDUCT.md
  • codecov.yml
  • CONTRIBUTING.md
  • DCO
  • LICENSE
  • MANIFEST.in
  • mkdocs.yaml
  • pyproject.toml
  • README.md
  • RELEASE.md
  • rust-toolchain.toml
  • SECURITY.md
  • setup.py

# Installation Guide

1. Get the code
git clone https://github.com/vllm-project/vllm

Downloads the entire project code from GitHub to your computer.

cd vllm

Moves into the project folder you just downloaded.

2. Docker

Easy Recommended
Prerequisites
  • Git Needed to download the project code from GitHub.
  • Docker Desktop Needed to build and run containers. Install it and keep it running in the background.
docker build -f docker/Dockerfile -t vllm .

Builds a runnable image based on the Dockerfile.

docker run -p 8080:80 vllm

Runs the built image as an actual container.

βœ… Run docker compose ps to check the containers are Up. If the README mentions a port, open http://localhost:PORT in your browser.

3. CMake

Medium
Prerequisites
  • Git Needed to download the project code from GitHub.
  • CMake The tool used to generate build configuration.
  • C/C++ 컴파일러 Windows needs Visual Studio (Community edition, free), macOS needs Xcode Command Line Tools, Linux needs the gcc/g++ package.
mkdir build && cd build

Creates a folder to hold the build output and moves into it.

cmake ..

Analyzes the source code and generates build configuration files (must be run inside the build folder).

make

Compiles the code based on the generated build configuration to produce an executable.

βœ… Check that an executable was created inside the build folder, then run it directly (e.g. ./build/app_name).

4. Python

Easy
Prerequisites
  • Git Needed to download the project code from GitHub.
  • Python 3 On Windows, be sure to check 'Add Python to PATH' during installation.
uv pip install vllm

Installs the Python libraries listed in requirements.txt (or similar).

βœ… If it runs without errors and prints output in the terminal, it worked.

Pulled directly from this repo's README.

5. Rust

Medium
Prerequisites
  • Git Needed to download the project code from GitHub.
  • Rust (rustup) Installing via rustup also installs cargo.
cd rust

This project's files live in a subfolder, so move into it first.

cargo build --release

Compiles the Rust project.

cargo run

Builds and then immediately runs the program.

βœ… If cargo build finishes without errors, it worked. The executable is created under target/.

Today's picks

Awesome-Hacking β˜… 118,742

A collection of various awesome lists for hackers, pentesters and security researchers
public-apis β˜… 467,736

A collective list of free APIs

Python
TradingAgents β˜… 99,131

TradingAgents: Multi-Agents LLM Financial Trading Framework

Python
openai-cookbook β˜… 75,446

Examples and guides for using the OpenAI API

Jupyter Notebook
material-ui β˜… 98,885

Material UI: Comprehensive React component library that implements Google's Material Desig

JavaScript
browser-use β˜… 109,987

🌐 Make websites accessible for AI agents. Automate tasks online with ease.

Python
// repository documentation