KO
|
EN
gitlite — search
Search
#javascript
#python
#hacktoberfest
#react
#ai
#typescript
#llm
#go
#golang
#android
#machine-learning
#rust
#deep-learning
#linux
vllm-rbln
★ 57
Open GitHub ↗
vLLM plugin for RBLN NPU
Download README (.md)
Explore Similar Repositories
RBlog
:
基于springboot + vue 实现的博客网站
rblog
:
super fast blog engine written in rust. live demo @ https://blog.prabir.me
rblog
:
:bulb: A simple blog build with Koa and React.
rblx-godot
:
A Roblox-compatible environment inside Godot leveraging Rust.
rblwatch
:
RBL lookups with Python
// repository documentation
Was this content helpful?
★ 0
(0 ratings)
Select Rating:
★
★
★
★
★
Submit Feedback
Recent Feedback
×
Download README
Do you want to download the
README.md
file for
vllm-rbln
?
Download (.md)
# vLLM RBLN Plugin <div align="center"> <picture> <source srcset="assets/vllm-rbln-white.png" media="(prefers-color-scheme: dark)"> <source srcset="assets/vllm-rbln-black.png" media="(prefers-color-scheme: light)"> <img src="assets/vllm-rbln-black.png" alt="main-logo" width=90%> </picture> [](https://badge.fury.io/py/vllm-rbln) [](https://github.com/rbln-sw/vllm-rbln/blob/main/LICENSE) [](https://docs.rbln.ai/software/model_serving/vllm_support/vllm-rbln.html) [](https://raw.githubusercontent.com/RBLN-SW/vllm-rbln/main/CODE_OF_CONDUCT.md) </div> This repository provides the hardware plugin that enables vLLM on RBLN NPUs, including [ATOM](https://rebellions.ai/rebellions-product/rbln-ca25/) and [REBEL](https://rebellions.ai/rebellions-product/rebel-quad/). Built on top of [vLLM’s Plugin System](https://docs.vllm.ai/en/latest/design/plugin_system.html), it allows seamless integration with the vLLM ecosystem and provides high-throughput, low-latency LLM serving on RBLN hardware. Our plugin supports a wide range of popular LLMs and continues to expand to support all features enabled in vLLM, including advanced attention mechanisms. ## 🚀 Getting Started ### 📋 Prerequisites - `rebel-compiler` - `optimum-rbln` ### ⚙️ Installation You can install this project using `pip` or from source. #### Install via PyPI ##### Using uv ```bash uv pip install vllm-rbln --extra-index-url https://wheels.vllm.ai/0.22.0/cpu --torch-backend cpu ``` ##### Using pip ```bash pip install vllm-rbln --extra-index-url https://wheels.vllm.ai/0.22.0/cpu --extra-index-url https://download.pytorch.org/whl/cpu ``` #### Or from source ##### Using uv ```bash git clone https://github.com/rbln-sw/vllm-rbln.git cd vllm-rbln uv pip install -e . ``` ##### Using pip ```bash git clone https://github.com/rbln-sw/vllm-rbln.git cd vllm-rbln pip install -e . --extra-index-url https://wheels.vllm.ai/0.22.0/cpu --extra-index-url https://download.pytorch.org/whl/cpu ``` ### 🛠️ Development setup Building from source? See **[DEVELOPMENT.md](./DEVELOPMENT.md)** for the `uv` dev workflow. ### 📚 Documentation - [Overview & Supported Models](https://docs.rbln.ai/software/model_serving/vllm_support/vllm-rbln.html) - [API Tutorial](https://docs.rbln.ai/software/model_serving/vllm_support/tutorial/vllm_llama3-8b.html) ## 🤝 Contributing We welcome all contributions! Whether it's reporting issues, proposing enhancements, or improving docs—your input helps make the project better. See our [CONTRIBUTING.md](./CONTRIBUTING.md) for more information. ## 📄 License This project is licensed under the Apache License 2.0. See the [LICENSE](./LICENSE) file for more information. ## 📧 Contact - Join discussions and get answers in our [Developer Community](https://discuss.rebellions.ai/) - Contact maintainers at [support@rebellions.ai](mailto:support@rebellions.ai)
