memra

(β˜… 319)

Rust + CUDA inference engine for NVIDIA RTX PRO 6000 Blackwell and RTX 5090. Serves safetensors and GGUF over an OpenAI-compatible API, with per-device tuned defaults and speculative decode gated byte-identical to plain decode. Hosted instance: inference.tiyuvta.ai

Showing a partial file list β€” download the zip above to see everything.

  • .gitattributes
  • .gitignore
  • .ignore
  • AGENTS.md
  • ARCHITECTURE-H100.md
  • ARCHITECTURE.md
  • bench_vllm.py
  • Cargo.lock
  • Cargo.toml
  • CITATION.cff
  • CLAUDE.md
  • CONTRIBUTING.md
  • LICENSE
  • README.md
  • SECURITY.md

# Installation Guide

1. Get the code
git clone https://github.com/avifenesh/memra

Downloads the entire project code from GitHub to your computer.

cd memra

Moves into the project folder you just downloaded.

2. Official Install Script

Easy Recommended
curl -fsSL https://raw.githubusercontent.com/avifenesh/memra/main/tools/install.sh | sh

Downloads and runs the official install script in one line β€” this handles the full setup automatically.

βœ… After installing, open a new terminal and run the program's version command (e.g. --version) to confirm it worked.

Pulled directly from this repo's README.

3. Rust

Medium
Prerequisites
  • Git Needed to download the project code from GitHub.
  • Rust (rustup) Installing via rustup also installs cargo.
cargo build --release

Compiles the Rust project.

βœ… If cargo build finishes without errors, it worked. The executable is created under target/.

Pulled directly from this repo's README.

// repository documentation