Best Local LLM for 8GB VRAM
Qwen3 8B is the strongest all-round pick for an 8 GB card: it needs about 5 GB at Q4, supports tool calling, and carries a fully commercial Apache 2.0 license. If you mainly write code, Phi-4 14B and Qwen 2.5 Coder 7B fit the same budget. Every model below runs on a GPU with 8 GB of VRAM using the default Q4 quantization in Ollama.
Model data last verified on June 27, 2026. All VRAM figures are for the Q4 quantization Ollama serves by default.
Every model that fits 8 GB of VRAM
10 of the 16 models in our verified catalog fit this budget at Q4, sorted with the largest fit first. Click any model for its full spec sheet, hardware cross-reference and run commands.
| Model | Params | Quant | Min VRAM | Context | Fit |
|---|---|---|---|---|---|
| Gemma 4 12BBest for fine-tuning | 12B | Q4 | 8 GB | 256K | 0 GB free |
| Phi-4 14BBest small coder | 14B | Q4 | 8 GB | 16K | 0 GB free |
| Mistral Nemo 12B | 12B | Q4 | 7 GB | 128K | 1 GB free |
| Llama 3.1 8B | 8B | Q4 | 5 GB | 128K | 3 GB free |
| Qwen3 8BMost popular | 8B | Q4 | 5 GB | 128K | 3 GB free |
| DeepSeek R1 Distill 8BMost private | 8B | Q4 | 5 GB | 128K | 3 GB free |
| Mistral 7B v0.3 | 7B | Q4 | 4 GB | 32K | 4 GB free |
| Qwen 2.5 Coder 7B | 7B | Q4 | 4 GB | 128K | 4 GB free |
| Gemma 3 4B | 4B | Q4 | 3 GB | 128K | 5 GB free |
| Llama 3.2 3BFastest | 3B | Q4 | 2 GB | 128K | 6 GB free |
Run the top picks with Ollama
One command each. Ollama pulls the Q4_K_M build by default and exposes an OpenAI-compatible endpoint at localhost:11434/v1.
ollama run gemma4:12bollama run phi4:14bollama run mistral-nemo:12bHardware that gives you 8 GB
Tracked cards and Macs with at least 8 GB of fast memory, cheapest first.
Frequently asked questions
What is the best local LLM for 8GB of VRAM?
Qwen3 8B is the best all-round choice. It needs about 5 GB of VRAM at Q4 quantization, outperforms the previous Qwen2.5 14B on most benchmarks, and ships under the Apache 2.0 license. For coding, Phi-4 14B and Qwen 2.5 Coder 7B are strong alternatives in the same budget.
Can I run a 13B or 14B model on 8GB of VRAM?
Yes, at Q4 quantization. Phi-4 14B needs about 8 GB and Mistral Nemo 12B needs about 7 GB, so both fit an 8 GB card. Full precision versions of those models do not fit; you need the quantized builds that Ollama serves by default.
How do I run these models?
Install Ollama, then pull any model with a single command, for example ollama run qwen3:8b. Ollama serves the Q4_K_M quantization by default, which is the version all VRAM figures on this page refer to.
Do these models run offline?
Yes. Once downloaded, every model on this page runs fully offline on your own machine. Nothing is sent to a cloud API and there is no subscription.
Different budget or use case?
The faceted model browser combines VRAM, family, license and task filters over the same verified catalog.