llm_on_rtx_3090
Battle-tested guide for local LLM inference on Ubuntu 24.04 with NVIDIA GPUs. From fresh install to 32B models at 97% GPU utilization. Covers Docker GPU runtime fixes, NVMe storage setup, and Ollama deployment. Tested on Dell T5820 + RTX 3090 (24GB), adaptable to any NVIDIA setup.
// repository documentation
Was this content helpful?
(0 ratings)
