Local AI How-To Guides
Run AI models on your own hardware. These guides cover Ollama and LM Studio setup, picking a model that fits your VRAM, quantization, and getting usable speed.
More guides
MiniCPM5-2B: Install and Run OpenBMB's 2B Model That Outscores 4B Rivals
MiniCPM5-2B is OpenBMB's 2.52 billion parameter model, released under Apache 2.0, and it averages 53.9 on OpenBMB's own benchmark set, ahead of Qwen3.5-4B's 51.1 despite having roughly half the parameters. Here is how to install it through vLLM, SGLang or Transformers, what the GGUF builds actually weigh, and what the benchmark table does and doesn't prove.
BERT Base Uncased: Specs, How to Run It, and Why It's Still Trending in 2026
google-bert/bert-base-uncased is a 110 million parameter masked-language model from 2018 that just re-entered Hugging Face's overall trending top 20. It has 48.8 million downloads, apache-2.0 licensing, and runs comfortably on a CPU. Here is how to run it, what it's actually still useful for, and why an eight-year-old model outranks this week's releases.
How to Run tencent/AuK Locally: Zero-Shot TTS, Voice Cloning and Speech Editing
tencent/AuK is a 1.5 billion parameter open source speech model that handles zero-shot voice cloning, instruction-guided TTS, pitch and speed editing, and audio source separation through one natural-language interface. It also depends on a separate 3 billion parameter Qwen2.5-Omni encoder at runtime, which changes the real hardware math. Here is what the specs actually are, how to install it, and how to run your first zero-shot clone.
Stable Diffusion 3.5 Medium: What's Different From SDXL and Flux
Stable Diffusion 3.5 Medium is Stability AI's 2B-parameter MMDiT-X text-to-image model. Here is how it differs from the SDXL and Flux coverage already on this site, how to install it with Diffusers, and the licensing catch that kicks in above $1M in revenue.
Qwen-Image-2512: The Text-to-Image Model That Renders Real Text
Qwen-Image-2512 is Alibaba's 20B-parameter text-to-image model, and its standout feature is generating readable English and Chinese text inside images, something SDXL and Flux still struggle with. Here is how to install it with Diffusers, what its Apache 2.0 license actually allows, and why the card does not publish a VRAM number.
How to Run Irodori-TTS Anime Locally: Japanese Voice Cloning and Voice Design
Irodori-TTS-v4.1-Anime is a free, MIT-licensed Japanese text-to-speech model fine-tuned on anime-style speech. Clone a voice from a short reference clip, or skip the clip entirely and describe the voice you want in words.
How to Run Breeze TTS 2 Locally: Voice Clone, Voice Design and Voice Direction
Breeze TTS 2 is a 3 billion parameter open-weight speech model that clones a voice from one reference clip and speaks English or Chinese in under 40 ms to first audio.
How to Run Kokoro-82M Locally: the Fastest Open-Weight TTS Model
Kokoro-82M is an 82 million parameter, fully Apache 2.0 text-to-speech model with 54 voices across 8 languages. At that size the weights are small enough to run on a CPU or almost any GPU, and installing it is one pip command.
Run Models
How to Install Ollama on macOS
Get Ollama running on a Mac in under 10 minutes: install via Homebrew or the native app, pull your first model, and confirm the local server is responding.
How to Install Ollama on Windows
Download the Windows installer, confirm PATH is set, verify the local API server is up, and understand when WSL2 gives you better GPU performance.
How to Run Llama 3 Locally with Ollama
Pull the Llama 3.3 model, run an interactive chat session, switch between the 8B and 70B variants, and understand the VRAM requirements for each size.
How to Pick the Right Local AI Model for Your Hardware
Match your GPU VRAM tier to concrete model recommendations, understand Q4/Q5/Q8 quantization, and use a simple rule of thumb to avoid out-of-memory crashes before you download.
Best GGUF Models to Run by VRAM Tier (8GB, 12GB, 16GB, 24GB, 48GB)
A current, no-nonsense map of which local models actually fit your GPU. Pick a VRAM tier, get the right GGUF model and the exact ollama pull command, plus the quant and context math that decides whether it loads or crashes.
Run LLMs in Your Browser With WebGPU: No Install, No Server (WebLLM)
Try local AI before you commit to Ollama or buy a GPU. WebLLM runs a real quantized model entirely inside a browser tab using WebGPU, with nothing sent to a server. This is the zero-install path, the model size limits, and the minimal code to embed it in your own page.
How to Use Ollama as a Drop-In OpenAI API
Point any OpenAI-compatible SDK or tool at your local Ollama server by changing one URL, and use OLLAMA_HOST to expose it on your network.
GGUF vs MLX vs NVFP4: Local AI Quantization Formats Explained
Three names keep showing up when you go to download a local model: GGUF, MLX, and the newer NVFP4. They are not interchangeable, and picking the wrong one wastes memory or leaves your hardware idle. Here is what each format is, which one your machine actually wants, and how to choose in ten seconds.
Local Apps & UIs
How to Set Up LM Studio for Local AI
Download LM Studio, grab a model from the built-in hub, start the local server, and have a working private AI chat running on your own hardware in about ten minutes.
Replace GitHub Copilot With a Local Coding Model (Qwen2.5-Coder + Continue.dev)
Drop the $10–19/month subscription and run your own coding assistant inside VS Code. This is the exact setup: Ollama plus Qwen2.5-Coder for chat and tab-autocomplete through the free Continue.dev extension, with honest notes on what you give up.
How to Set Up Open WebUI for Local AI
Run Open WebUI in Docker, connect it to Ollama, switch between models in one click, and create custom personas that save your best system prompts.
How to Run Mistral 7B Locally
Pull Mistral 7B with Ollama, understand when to use mistral-nemo instead, check your VRAM requirement, and tune a system prompt to get sharper answers.
Hardware & Builds
Best GPU for Running AI Locally in 2026
VRAM is the single spec that determines which models you can run. Here is how every major GPU stacks up for local LLMs and image generation in 2026.
How Much RAM Do You Need for Local AI?
RAM requirements for local AI depend on whether you are talking about GPU VRAM or system RAM, and which models you want to run. Here are the real numbers.
Mac vs PC for Local AI: Which Should You Choose?
Apple Silicon unified memory and PC discrete GPUs take completely different approaches to local AI. Here is what each does better and how to pick based on your workload.
How to Build a Local AI Workstation on Any Budget
Four budget tiers from $500 to $5,000+ with specific part recommendations and the real trade-offs at each level.
How to Set Up Local AI on 8GB of VRAM
An 8GB GPU runs the 7B to 8B model class comfortably at Q4 quantization, and Phi-4 14B just barely fits. Here is exactly what to install, which quantization format to pick, and real measured tokens-per-second so you know what to expect before you download anything.
How to Set Up Local AI on 12GB of VRAM
12GB of VRAM is the mixture-of-experts sweet spot: OpenAI's gpt-oss-20b needs about 12GB at Q4 and, because only 3.6B of its 20B parameters activate per token, generates at nearly 7B speed. Here is how to install it, which quantization it actually ships in, and cited real-world tokens-per-second.
Best Local LLM for 16 GB VRAM: Setup, Quantization and Real Speed
16GB of VRAM unlocks Gemma 3 27B, which needs about 15GB at Q4 and just fits, plus OpenAI's gpt-oss-20b with real headroom to spare. Here is how to install both, which quantization format to use, and measured tokens-per-second on the RTX 4060 Ti 16GB and RTX 5070 Ti.
How to Set Up Local AI on 24GB of VRAM
A 24GB card runs Ornith 1.0 35B, the top open-weight coding model on SWE-bench Verified at 75.6%, which needs about 20GB at Q4. Gemma 3 27B and everything smaller fit with headroom left for long context. Here is the install path, quantization choice, and cited RTX 3090 vs RTX 4090 speed.
Best Local LLM on Mac M4 16 GB: Setup, MLX vs GGUF and Real Speed
On a 16GB M4 Mac, the sweet spot is the 7B to 8B class: Qwen3 8B and Llama 3.1 8B both need about 5GB at Q4, leaving plenty of unified memory for macOS. Here is how to install Ollama or mlx-lm, which format to pick, and llama.cpp's own measured M4 speed.
Best Local LLM on Mac M4 Pro 48GB: Setup, Quantization and Real Speed
A 48GB M4 Pro Mac mini fits Llama 3.3 70B at Q4 quantization, about 48GB, right at the ceiling, and runs Ornith 1.0 35B or Gemma 3 27B with real headroom. Here is how to install both paths, which format to pick, and what's actually measured versus what isn't for this exact chip.
Model Spotlights
How to Run OpenAI's gpt-oss-20b Locally with Ollama
gpt-oss-20b is OpenAI's first open-weight model under Apache 2.0, and it runs on a 16 GB machine. Here is the exact, step-by-step setup with Ollama, from install to your first chat and API call.
Run DeepSeek V4 Locally: Which Size Actually Fits Your Hardware (Flash vs Pro)
DeepSeek V4 is a frontier open-weight release, but the headline size and the memory you actually need are not the same number. Here is what fits a single consumer GPU, what needs a workstation, and the realistic fallback for normal hardware.
Local OCR and Document AI: Self-Host baidu Unlimited-OCR for Private Document Parsing
Invoices, receipts, contracts and scanned PDFs are exactly the documents you do not want to ship to a per-page cloud API. baidu Unlimited-OCR is a brand-new open-weight model that parses whole PDFs in one pass, on your own hardware. Here is what it is, what it runs on, and how to go from a scanned page to structured JSON.
gpt-oss-20b vs gpt-oss-120b: Which Open Model Should You Run?
OpenAI shipped two open-weight models. One runs on a laptop, the other needs a serious GPU. Here is exactly how they differ and which one fits your hardware and use case.
Fine-Tuning
How to Fine-Tune an LLM with LoRA (Beginner Guide)
Understand what fine-tuning actually does, why LoRA makes it possible on consumer hardware, and what you need before you write a single line of training code.
How to Fine-Tune Llama 3 on Your Own Data with Unsloth
Install Unsloth, load Llama 3, format your JSONL dataset, run a LoRA training job, and export the result as a GGUF file ready for Ollama.
How to Prepare a Training Dataset for LLM Fine-Tuning
Build a clean JSONL dataset using the right conversation format, apply quality filters, hit the minimum example count, and use synthetic generation to fill gaps.
How to Run Your Fine-Tuned Model in Ollama
Take the GGUF file from your fine-tune, write a Modelfile, register it with Ollama, run it locally, and optionally push it to the Ollama registry.