Best Local AI Models for 24GB VRAM
A 24 GB card such as an RTX 3090 or 4090 runs nearly the whole catalog, including heavyweight coding specialists, with room to spare for longer context.
Llama 3.1 8B
Meta · Llama 3.1 Community
Meta's 8B workhorse with a massive 128K context window. Punches well above its weight class for everyday coding and chat tasks, runs comfortably on any consumer GPU with 6 GB VRAM, and delivers near-70B quality on many benchmarks.
Min VRAM
5 GB
Context
128K
Run with Ollama
ollama run llama3.1:8bLlama 3.2 3B
Meta · Llama 3.2 Community
The smallest production-quality Llama model, designed to run on laptops, Raspberry Pis, and other edge devices with minimal VRAM. Response latency is sub-second on modern CPUs, making it ideal for real-time assistants and local automation scripts.
Min VRAM
2 GB
Context
128K
Run with Ollama
ollama run llama3.2:3bQwen3 8B
Alibaba · Apache 2.0
One of the most-downloaded mid-size AI models on HuggingFace in 2026, with tens of millions of downloads. Qwen3 8B outperforms the previous Qwen2.5 14B on most benchmarks, supports hybrid thinking mode and tool calling, and runs on any GPU with 6 GB VRAM. Apache 2.0 license — fully commercial.
Min VRAM
5 GB
Context
128K
Run with Ollama
ollama run qwen3:8bMistral 7B v0.3
Mistral AI · Apache 2.0
The model that proved 7B parameters can match much larger models on reasoning tasks when trained carefully. Apache 2.0 licensed, so it is fully free to use commercially with no restrictions.
Min VRAM
4 GB
Context
32K
Run with Ollama
ollama run mistral:7bMistral Nemo 12B
Mistral AI · Apache 2.0
Built in partnership with NVIDIA and trained on a broad multilingual corpus, Nemo is Mistral's best model for non-English tasks across European and Asian languages. The 128K context window makes it practical for processing long documents locally.
Min VRAM
7 GB
Context
128K
Run with Ollama
ollama run mistral-nemo:12bGemma 4 12B
Google · Apache 2.0
Google's latest Gemma generation doubles the context window to 256K tokens and — a major shift from earlier Gemma releases — ships under the Apache 2.0 license. The 12B model is one of the most actively fine-tuned coding bases on HuggingFace in 2026, with hundreds of community GGUF variants. Runs on any GPU with 10 GB VRAM.
Min VRAM
8 GB
Context
256K
Run with Ollama
ollama run gemma4:12bGemma 3 4B
Google · Gemma Terms of Use
Google's compact Gemma 3 model brings the quality of a much larger system into a 4B package that runs on integrated graphics. A strong first choice for anyone wanting a capable, low-power model without needing a discrete GPU.
Min VRAM
3 GB
Context
128K
Run with Ollama
ollama run gemma3:4bGemma 3 27B
Google · Gemma Terms of Use
The largest Gemma 3 model and one of the best open-weight models in the 20–30B range. Fits in a single RTX 4090 (24 GB) with room to spare at Q4 quantization, and delivers instruction-following quality that rivals models twice its size.
Min VRAM
15 GB
Context
128K
Run with Ollama
ollama run gemma3:27bPhi-4 14B
Microsoft · MIT
Microsoft's research-driven Phi-4 focuses on synthetic high-quality training data, producing a 14B model that outperforms much larger models on coding and mathematical reasoning benchmarks. MIT licensed with zero usage restrictions for commercial projects.
Min VRAM
8 GB
Context
16K
Run with Ollama
ollama run phi4:14bDeepSeek R1 Distill 8B
DeepSeek · MIT
A distilled version of DeepSeek's R1 reasoning model that inherits chain-of-thought problem-solving capabilities in a compact 8B package. One of the few small models that can reliably work through multi-step math and logic problems without external tooling.
Min VRAM
5 GB
Context
128K
Run with Ollama
ollama run deepseek-r1:8bOpenAI gpt-oss-20b
OpenAI · Apache 2.0
OpenAI's first commercially-licensed open-weight model — a historic release under Apache 2.0. MoE architecture keeps only 3.6B parameters active at inference, so it generates tokens at nearly 7B speed despite 20B total params. Supports function calling and web browsing. Over 10M downloads across Ollama and HuggingFace. Pull it with ollama run gpt-oss:20b.
Min VRAM
12 GB
Context
128K
Run with Ollama
ollama run gpt-oss:20bOrnith 1.0 35B
DeepReinforce · MIT
Scores 75.6% on SWE-bench Verified — one of the highest results for any open-weight coding model and above most paid assistants. From the lab DeepReinforce, trained with RL self-improvement for terminal control and tool calling. MIT licensed, 262K context. There is no Ollama library tag yet — pull the GGUF directly with ollama run hf.co/deepreinforce-ai/Ornith-1.0-35B-GGUF.
Min VRAM
20 GB
Context
262K
Run with Ollama
ollama run hf.co/deepreinforce-ai/Ornith-1.0-35B-GGUFQwen 2.5 Coder 7B
Alibaba · Apache 2.0
A coding-specialist model fine-tuned on a massive corpus of source code across 40+ programming languages, delivering autocomplete and generation quality that rivals dedicated IDE tools. At 7B it is fast enough for real-time code suggestions on a single consumer GPU.
Min VRAM
4 GB
Context
128K
Run with Ollama
ollama run qwen2.5-coder:7bBreeze TTS 2
BreezeBlue · BreezeBlue Research Non-Commercial (code: Apache 2.0)
Released August 2026 and currently the #1 open-weight model on the Artificial Analysis speech arena (1,215 Elo). Supports voice cloning from a short reference clip, voice design from a text description, and inline direction tags like (sigh) for emotion. Needs a 12 GB GPU for eager inference or 24 GB for the faster path. Weights are BreezeBlue's Research Non-Commercial license — the code is Apache 2.0, but commercial use requires their paid API.
Min VRAM
12 GB GPU (7.7 GiB eager) / 24 GB GPU (14.4 GiB fast path)
Speed
Time to first audio <40ms · real-time factor 0.32
Install & run
git clone https://github.com/breezeblue-ai/breeze-tts.gitKokoro-82M
hexgrad · Apache 2.0
The most-downloaded text-to-speech model on HuggingFace by a wide margin, at 11.6M downloads a month. At just 82M parameters it is small enough to run on CPU alone — no GPU minimum is published, unlike most other speech models here. Apache 2.0 licensed, so it is fully free for commercial use with no restrictions. Ships with 8 languages and 54 voices out of the box.
Min VRAM
None published — small enough to run on CPU alone
Speed
—
Install & run
pip install -q kokoro>=0.9.2 soundfileFish Audio S2 Pro
Fish Audio · Fish Audio Research License (non-commercial)
Fish Audio's flagship speech model, trained on 10M+ hours of audio with fine-grained inline control over prosody and emotion across 80+ languages. Needs a 24 GB GPU for CUDA inference, though a CPU-only install path exists for testing. Free for research and non-commercial use under the Fish Audio Research License; commercial use requires a separate paid license.
Min VRAM
24 GB GPU (CUDA) — CPU-only install also available
Speed
Real-time factor 0.195 on a single H200 · time to first audio ~100ms
Install & run
git clone https://github.com/fishaudio/fish-speech.gitOmniVoice
k2-fsa · Apache 2.0 (code) / CC BY-NC (pretrained weights)
A zero-shot voice-cloning model from the k2-fsa (Next-gen Kaldi) team with the broadest language coverage of any open TTS model — 600+ languages from a single checkpoint. The code is Apache 2.0, but the pretrained weights are CC BY-NC because of constraints in the Emilia training data, so the released checkpoint is not free for commercial use without separate authorization.
Min VRAM
Not published — tested on server-class GPUs; also runs on Apple Silicon (MPS) and Intel Arc (XPU)
Speed
—
Install & run
pip install torch==2.8.0+cu128 torchaudio==2.8.0+cu128 --extra-index-url https://download.pytorch.org/whl/cu128Qwen3-TTS 1.7B CustomVoice
Alibaba · Apache 2.0
Alibaba's Qwen3-TTS with 9 premium preset voices and instruction-based control over tone, emotion and prosody across 10 major languages. Streaming generation gets end-to-end latency as low as 97ms. Apache 2.0 licensed — fully free for commercial use, unlike most other speech models on this page.
Min VRAM
Not published — FlashAttention recommended to reduce memory use
Speed
End-to-end synthesis latency as low as 97ms (streaming)
Install & run
pip install -U qwen-ttsWant to mix in more filters?
Open the faceted model browser to combine VRAM, family, license, developer and task filters, then sort the results your way.