In this categoryLocal AI · 24

Run Models

Local Apps & UIs

Hardware & Builds

Model Spotlights

Fine-Tuning

Local AIBeginner

How to Run Llama 3 Locally with Ollama

Pull the Llama 3.3 model, run an interactive chat session, switch between the 8B and 70B variants, and understand the VRAM requirements for each size.

10 minBeginner

Visit Llama

Llama 3.3 is Meta's most capable open-weight model family. The 8B variant runs on a laptop; the 70B variant needs a workstation-class GPU or a machine with large unified memory. Ollama handles all the download, quantization, and inference details so you can focus on using the model rather than compiling anything.

Model sizes and VRAM requirements

Choosing the right variant before you download saves time. The table below shows the quantized file sizes and the minimum VRAM or unified RAM needed to run each at reasonable speed.

llama3.3:8b — 4.7 GB download, needs 8 GB RAM/VRAM — fast on Apple Silicon and mid-range NVIDIA GPUs
llama3.3:70b — 43 GB download, needs 48 GB RAM/VRAM — requires a high-end GPU (RTX 4090, A100) or Apple M2 Ultra/M3 Max
llama3.3:8b-instruct-q4_K_M — slightly smaller 4-bit quantized variant, good balance of speed and quality
llama3.3:70b-instruct-q4_K_M — 4-bit quantized 70B for machines with 40-48 GB unified memory

What do q4_K_M and q8 mean?

These are quantization levels. q4_K_M stores model weights in 4-bit integers, which cuts file size roughly in half versus fp16 with a small quality loss. q8_0 is 8-bit and closer to full quality. Ollama defaults to q4_K_M for the untagged latest variant. You can specify a different quantization with ollama pull llama3.3:8b-q8_0.

Step 1: Pull the model

Pulling separately from running lets you download in the background while you keep working. Run pull and wait for it to finish. The weights are cached under ~/.ollama/models (macOS/Linux) or %USERPROFILE%\.ollama\models (Windows).

zsh - pull

$ollama pull llama3.3

pulling manifest

pulling 8b model... ████████████ 4.7 GB / 4.7 GB

verifying sha256 digest

writing manifest

success

Subsequent pulls of the same digest are instant (already cached)

Step 2: Start an interactive chat

ollama run opens a REPL where you type messages and the model responds. The session keeps conversation history so follow-up questions work. Exit with /bye or Ctrl+D.

zsh - interactive chat

$ollama run llama3.3

>>> Send a message (/? for help)

$Write a Python function that removes duplicate lines from a file.

def dedupe_lines(path):

seen = set()

with open(path) as f:

lines = [l for l in f if l not in seen and not seen.add(l)]

with open(path, 'w') as f:

f.writelines(lines)

$Now add a flag to preserve order.

The function above already preserves insertion order because...

$/bye

Step 3: Run a single prompt non-interactively

Pipe a prompt directly to ollama run for scripting. The model reads from stdin and writes the response to stdout, which makes it easy to compose with other Unix tools.

zsh - pipe a prompt

$echo "Explain VRAM in one sentence." | ollama run llama3.3

VRAM is dedicated high-bandwidth memory on a GPU used to store...

$cat error.log | ollama run llama3.3 'Summarize the errors:'

The log shows three recurring issues: a null pointer at line...

Step 4: Switch to the 70B variant

Tag the pull with :70b to get the larger model. Ollama stores both variants independently so you can switch by changing the name in ollama run.

zsh - 70B

$ollama pull llama3.3:70b

pulling 70b model... 43 GB (this takes a while on a home connection)

$ollama run llama3.3:70b

>>> Send a message (/? for help)

70B will be slower per token but noticeably more capable on hard tasks

Check memory pressure before pulling 70B

On macOS, open Activity Monitor and watch the Memory Pressure graph. On Linux, run free -h. If your system is already swapping, the 70B model will be painfully slow. Start with 8B and upgrade if quality is the bottleneck, not hardware.

Useful run-time flags

ollama run llama3.3 --verbose — show token generation speed (tokens/sec) and model load time
ollama run llama3.3 --ctx-size 8192 — increase context window beyond the default (if your RAM allows)
ollama ps — show currently loaded models and how much VRAM each is using
ollama stop llama3.3 — unload the model from GPU memory immediately to free VRAM for other tasks

You now have Llama 3.3 running entirely on your own hardware. No API keys, no rate limits, no data leaving your machine. The interactive REPL works for exploration, and the stdin pipe makes it scriptable. The next logical step is calling Ollama from code via its OpenAI-compatible API endpoint.

Local models run better with more VRAM. CompareRTX GPUs on Amazonbefore you upgrade.(affiliate link. We may earn a commission at no extra cost. Disclosure)

Related guides

Local AI

Beginner7 min

How to Pick the Right Local AI Model for Your Hardware

Match your GPU VRAM tier to concrete model recommendations, understand Q4/Q5/Q8 quantization, and use a simple rule of thumb to avoid out-of-memory crashes before you download.

Local AI

Beginner6 min

How Much RAM Do You Need for Local AI?

RAM requirements for local AI depend on whether you are talking about GPU VRAM or system RAM, and which models you want to run. Here are the real numbers.