Best GGUF Models to Run by VRAM Tier (8GB, 12GB, 16GB, 24GB, 48GB)

The single most common local-AI question is also the most practical: "I have an X GB card, what is the best model I can actually run?" The honest answer changed a lot in 2026. Mixture-of-experts models like gpt-oss-20b and new Apache 2.0 releases like Qwen3 and Gemma 4 shifted what fits in each tier. This guide maps real GPU memory to the best GGUF model for it, gives you the exact ollama pull command, and explains the two numbers (quantization and context) that decide whether a model loads cleanly or dies with an out-of-memory error.

How VRAM maps to model size

A model's memory footprint is roughly its parameter count multiplied by the bits-per-weight, plus a KV cache that grows with context length. A 7B model at 16-bit (fp16) weighs about 14 GB. Quantize it to 4-bit and it drops to roughly 4 GB of weights, and with a small KV cache you are looking at about 5 GB total to load and run it. That is the whole reason quantized GGUF files exist: they let a 7B model fit on an 8 GB card with room to spare, and a 70B model fit on a pair of 24 GB cards instead of a datacenter.

Reading GGUF quant labels

GGUF files come in quantization levels, and the label tells you the size and quality tradeoff. Lower bits mean a smaller file and less VRAM, at the cost of some accuracy. For most people Q4_K_M is the sweet spot, which is exactly why Ollama defaults to it.

Practical advice: if a model fits at Q4_K_M with comfortable headroom, step up to Q5_K_M or Q6_K for better answers. If it barely fits at Q4_K_M, leave it there. Going below Q4 (Q3, Q2) is usually a mistake unless you have no other way to load the model, because quality degrades fast.

Quick reference: VRAM tier to model to command

8 GB tier: the dense 7B-8B sweet spot

An 8 GB card (RTX 3060 8 GB, RTX 4060, or an 8 GB Apple Silicon Mac) runs dense 7B and 8B models at Q4_K_M with everyday-fast speed. Qwen3 8B is the standout here: it is Apache 2.0 licensed, strong at reasoning and code, and pulls as qwen3:8b. Llama 3.1 8B is the reliable generalist, Mistral 7B is fast and lean, and Gemma 3 4B is the pick when you also need VRAM for a long context or other apps. Keep context at 4K to 8K on this tier so the KV cache does not push you over.

12 GB tier: MoE changes the math

On a 12 GB card (RTX 3060 12 GB, RTX 4070) the big unlock is mixture-of-experts. gpt-oss:20b has 20.9B total parameters but only 3.6B active per token, and OpenAI ships it pre-quantized to MXFP4, so the weights land around 13 GB. That is tight on exactly 12 GB and runs cleanly with a touch of CPU spillover or a slightly smaller context; on 16 GB it is comfortable. Dense options that fit this tier well are Phi-4 14B, Gemma 4 12B (Apache 2.0, 256K context), and Mistral Nemo 12B.

16 GB and 24 GB: into 27B-35B territory

16 GB (RTX 4060 Ti 16 GB, RTX 5070 Ti) is the first tier where a 27B model is realistic. Gemma 3 27B at Q4_K_M loads in about 15 GB and is a genuine quality jump over the 8 GB class. gpt-oss:20b also gets comfortable headroom here for longer context. Ornith-1.0 35B (MIT, a strong agentic coder) fits but is tight at Q4. Step up to 24 GB (RTX 3090, RTX 4090) and you get the good version of all of this: Gemma 3 27B at Q5_K_M or Q6_K for better answers, Mixtral 8x7B (47B total, MoE) with breathing room, and Ornith-1.0 35B running comfortably.

48 GB+ and dual-GPU: the 70B tier

Running a 70B model locally means roughly 40 to 48 GB of memory at Q4. That is one 48 GB card (RTX 6000-class), a 64 GB+ unified-memory Mac, or two 24 GB cards with the layers split across them. Llama 3.3 70B at Q4_K_M is the classic target here, around a 43 GB download. Ollama handles multi-GPU splitting automatically when it detects two cards, so a pair of 3090s or 4090s gives you 48 GB of pooled VRAM. The payoff over the 27B-35B tier is better long-form reasoning and fewer mistakes on hard, multi-step tasks; the cost is slower tokens per second and a much bigger electricity and hardware bill.

Context window also eats VRAM

Picking a model that fits is only half the equation. Every token of context you allow gets stored in the KV cache, which lives in VRAM alongside the weights. A modest 4K context adds a little; pushing a model to its full 128K or 256K window can add several gigabytes and tip a model that loaded fine into an out-of-memory crash mid-conversation. If a model loads but dies once the chat gets long, shrink the context window rather than the quant.

Pull and sanity-test any model

Whatever tier you land on, the test loop is the same. Pull the model, run a quick prompt with --verbose to see the real tokens-per-second, and check ollama ps to confirm it is fully on the GPU. If the first token is fast and the PROCESSOR column reads 100% GPU, you picked the right model for your card.

Hardware to hit each tier (2026 pricing reality)

If you are buying for a tier, plan around two things in 2026: the RTX 50-series is the current generation (5090 32 GB, 5080, 5070 Ti), and the 4090 and 4080 Super are discontinued, so used prices are high. The bigger surprise is system RAM. A DRAM shortage has pushed 64 GB DDR5 kits to around 1,150 dollars, which matters because CPU offload and unified-memory Macs both lean on RAM. For an 8 to 12 GB tier a current mid-range card is plenty; for 24 GB a used 3090 is the value play; for the 70B tier, dual 24 GB cards or a high-VRAM workstation card. See our GPU and RAM buying pages for current picks and prices at each tier.

Bottom line: match the model to the card, not the other way around. Pick your VRAM tier from the quick-reference table, pull the recommended GGUF model at Q4_K_M, and step up the quant only if it fits with room to spare. Keep an eye on context length, watch ollama ps to confirm 100% GPU, and you will get the best model your hardware can actually run fast.