mlx-openai-server

(★ 356)

A high-performance API server that provides OpenAI-compatible endpoints for MLX models. Developed using Python and powered by the FastAPI framework, it provides an efficient, scalable, and user-friendly solution for running MLX-based vision and language models locally with an OpenAI-compatible interface.

File Explorer

Download Latest Version (.zip)

- - linters.yml
  - publish.yml
- dependabot.yml
- - __init__.py
  - endpoints.py
- - __init__.py
  - audio_processor.py
  - base_processor.py
  - batch_scheduler.py
  - handler_process.py
  - image_processor.py
  - inference_worker.py
  - model_registry.py
  - video_processor.py
  - vlm_batch_scheduler.py
- - __init__.py
  - mflux.py
  - mlx_embeddings.py
  - mlx_lm.py
  - mlx_vlm.py
  - mlx_whisper.py
- - __init__.py
  - abstract_converter.py
  - glm4_moe.py
- - __init__.py
  - request_tracking.py
- - __init__.py
  - mflux.py
  - mlx_embeddings.py
  - mlx_lm.py
  - mlx_vlm.py
  - mlx_whisper.py
- - __init__.py
  - abstract_parser.py
  - function_parameter.py
  - functiongemma.py
  - gemma4.py
  - glm4_moe.py
  - harmony.py
  - hermes.py
  - kimi_k2.py
  - longcat_flash_lite.py
  - minimax_m2.py
  - mixed_think_tool_handoff.py
  - qwen3.py
  - qwen3_5.py
  - qwen3_moe.py
  - solar_open.py
  - step35.py
- - __init__.py
  - model.py
  - openai.py
- - __init__.py
  - debug_logging.py
  - dill.py
  - errors.py
  - outlines_transformer_tokenizer.py
  - prompt_cache.py
- __init__.py
- cli.py
- config.py
- main.py
- server.py
- version.py
- FUSION_PHASE0.md
- HANDOFFS.md
- - audio.wav
  - podcast.wav
- - gemma4.jinja
  - llama4.jinja
  - Longcat-Flash-Lite.jinja
- - attention.png
  - china.png
  - green_dog.jpeg
  - password.jpg
- - lab03.pdf
- - demo.mp4
- audio_examples.ipynb
- config.yaml
- embedding_examples.ipynb
- image_edit.ipynb
- image_generations.ipynb
- lm_embeddings_examples.ipynb
- responses_api.ipynb
- simple_rag_demo.ipynb
- structured_outputs_examples.ipynb
- transcription_examples.ipynb
- vision_examples.ipynb
- vlm_embeddings_examples.ipynb
- llm_health_dashboard.py
- - __init__.py
  - test_functiongemma.py
  - test_glm4_moe.py
  - test_harmony.py
  - test_hermes.py
  - test_minimax_m2.py
  - test_mixed_think_tool_handoff_log_harness.py
  - test_mixed_think_tool_handoff_pathological_streaming.py
  - test_mixed_think_tool_handoff_registry.py
  - test_qwen3_moe.py
  - test_streaming_tag_split_boundaries.py
- __init__.py
- test_batch_scheduler.py
- test_chat_completions_defaults.py
- test_chat_completions_prompt_history.py
- test_cli_sampling_defaults_parity.py
- test_handler_process_stream_cancellation.py
- test_inference_worker.py
- test_llm_contracts.py
- test_mixed_think_tool_handoff_stream_handler_integration.py
- test_multi_model_per_model_defaults.py
- test_multimodal_audio_inputs.py
- test_non_trimmable_checkpoint.py
- test_prompt_cache_cancellation.py
- test_prompt_cache_disk.py
- test_responses_request_conversion.py
- test_stream_tool_call_ids.py
- test_vlm_batch_scheduler.py
.gitignore
.pre-commit-config.yaml
AGENTS.md
configure_mlx.sh
LICENSE
Makefile
pyproject.toml
README.md

# Installation Guide

1. Get the code

git clone https://github.com/cubist38/mlx-openai-server

Downloads the entire project code from GitHub to your computer.

cd mlx-openai-server

Moves into the project folder you just downloaded.

2. Official Install Script

Easy Recommended

Prerequisites

Homebrew A package manager for macOS/Linux.

brew install ffmpeg

Installs the pre-built package via Homebrew — no source build required.

✅ After installing, open a new terminal and run the program's version command (e.g. --version) to confirm it worked.

Pulled directly from this repo's README.

3. Python

Easy

Prerequisites

Git Needed to download the project code from GitHub.
Python 3 On Windows, be sure to check 'Add Python to PATH' during installation.

uv pip install mlx-openai-server