lm-proxy
OpenAI-compatible HTTP LLM proxy / gateway for multi-provider inference (Google, Anthropic, OpenAI, PyTorch). Lightweight, extensible Python/FastAPI—use as library or standalone service.
File Explorer
Download Latest Version (.zip)- badge-publish.yml
- code-style.yml
- docker.yml
- gito-code-review.yml
- gito-react-to-comments.yml
- tests.reusable.yml
- tests.yml
- config.toml
- fallback.md
- http_headers.md
- https.md
- load_balancer_config.py
- vertex-ai.toml
- __init__.py
- allow_all.py
- in_config.py
- with_request.py
- __init__.py
- json.py
- python.py
- toml.py
- yaml.py
- __init__.py
- forward_http_headers.py
- rate_limiter.py
- __init__.py
- fallback.py
- __init__.py
- __main__.py
- app.py
- base_types.py
- bootstrap.py
- config.py
- core.py
- errors.py
- loggers.py
- models_endpoint.py
- utils.py
- lm-proxy_1_hacker_1600x672.png
- __init__.py
- config_fn.py
- extra_headers.yml
- no_api_key_check.yml
- rate_limiter.yml
- test_config.json
- test_config.toml
- test_config.yml
- test_disabled.py
- __init__.py
- conftest.py
- test_api_key_check_allow_all.py
- test_api_key_check_errors.py
- test_api_key_check_with_request.py
- test_config_loaders.py
- test_integration.py
- test_integration_headers.py
- test_loggers.py
- test_models_endpoint.py
- test_print_stream.py
- test_rate_limiter_integration.py
- test_resolve_connection_and_model.py
- test_utils.py
- .dockerignore
- .env.template
- .flake8
- .gitignore
- AGENTS.md
- CLAUDE.md
- config.toml
- coverage.svg
- Dockerfile
- LICENSE
- Makefile
- multi-build.py
- poetry.lock
- pyproject.toml
- README.md
# Installation Guide
git clone https://github.com/Nayjest/lm-proxy
Downloads the entire project code from GitHub to your computer.
cd lm-proxy
Moves into the project folder you just downloaded.
2. Official Install Script
Easy Recommended- Python 3 Python is required to use pip.
pip install lm-proxy
Installs the package published on PyPI directly — no need to clone the source.
pip install lm-proxy[anthropic,google]
Installs the package published on PyPI directly — no need to clone the source.
pip install lm-proxy[all]
Installs the package published on PyPI directly — no need to clone the source.
Pulled directly from this repo's README.
3. Docker
Easy- Git Needed to download the project code from GitHub.
- Docker Desktop Needed to build and run containers. Install it and keep it running in the background.
docker run -d --name lm-proxy \
Runs the built image as an actual container.
docker build --build-arg EXTRA_PIP_PACKAGES="lm-proxy-db-connector sqlalchemy psycopg2-binary" -t lm-proxy .
Builds a runnable image based on the Dockerfile.
Pulled directly from this repo's README.
4. Python
Easypip install lm-proxy
Installs the package published on PyPI directly — no need to clone the source.
pip install lm-proxy[anthropic,google]
Installs the package published on PyPI directly — no need to clone the source.
pip install lm-proxy[all]
Installs the package published on PyPI directly — no need to clone the source.
python -m lm_proxy
Runs the Python script (or module).
use_cache = true # requires installing cachetools if True: pip install cachetools
Installs the Python libraries listed in requirements.txt (or similar).
Pulled directly from this repo's README.
5. Make
Medium- Git Needed to download the project code from GitHub.
- Make Usually pre-installed on Linux/macOS. On Windows, install separately (e.g. via MSYS2 or WSL).
make
Compiles the code based on the generated build configuration to produce an executable.