How to Run OpenAI's gpt-oss-20b Locally with Ollama

OpenAI spent years as the company that made "open" a punchline. That changed with gpt-oss. The 20B variant is the first OpenAI model you can download, run on your own machine, and use commercially under the Apache 2.0 license — no API key, no per-token bill, no data leaving your computer. The best part: it fits on a 16 GB laptop or GPU. This guide walks through the entire setup with Ollama, the simplest way to run it, and gets you from a blank terminal to a working local chat in about ten minutes.

What gpt-oss-20b actually is

gpt-oss-20b is a mixture-of-experts (MoE) model with 20.9 billion total parameters but only 3.6 billion active per token. That design is why it punches far above its size class on speed: it generates tokens at roughly the pace of a 7B model while carrying the knowledge of a 20B one. It ships with a 128K context window, native function calling, web-browsing support, and a configurable reasoning effort (low, medium, high) so you can trade speed for depth on a per-request basis. OpenAI quantized it to MXFP4 out of the box, which is what keeps the memory footprint small enough for consumer hardware.

What you need before you start

• 16 GB of memory — VRAM on a discrete GPU (RTX 4060 Ti 16 GB, RTX 5070 Ti, or better), or 16 GB of unified memory on an Apple Silicon Mac (M1 through M4).
• About 13 GB of free disk space for the model download.
• Windows, macOS, or Linux. Ollama runs on all three.
• No GPU at all? It still runs on CPU with 16 GB of system RAM, just slower (a handful of tokens per second instead of dozens).

Step 1 — Install Ollama

Ollama is a free, open-source runner that handles the download, quantization, and serving for you. On macOS or Windows, download the installer from ollama.com and run it. On Linux, one command does it:

Confirm it installed by checking the version. If you see a version number, you are ready.

Step 2 — Pull and run gpt-oss-20b

This is the whole installation. One command downloads the model (about 13 GB, so give it a few minutes on a normal connection) and drops you straight into an interactive chat:

To exit the chat, type /bye or press Ctrl+D. The model stays on disk, so the next ollama run gpt-oss:20b launches instantly with no download.

Step 3 — Control the reasoning effort

One of gpt-oss's standout features is adjustable reasoning. For a quick factual answer you want speed; for a hard coding or math problem you want it to think. You set this in the system prompt. Inside the chat, use /set system to switch levels:

Use Reasoning: low for fast chat and autocomplete-style tasks, Reasoning: medium as a balanced default, and Reasoning: high when correctness matters more than latency. Higher effort uses more tokens and takes longer, but noticeably improves multi-step problems.

Step 4 — Use it from code (OpenAI-compatible API)

Ollama exposes an OpenAI-compatible endpoint at http://localhost:11434/v1, which means most code written for the OpenAI API works against your local model by changing two lines. Here is a raw curl call:

curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-oss:20b",
    "messages": [
      { "role": "system", "content": "Reasoning: medium" },
      { "role": "user", "content": "Explain MoE models in two sentences." }
    ]
  }'

And the same thing from Python using the official openai library — point it at your local server and use any string as the API key:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:11434/v1",
    api_key="ollama",  # required by the library, but ignored locally
)

resp = client.chat.completions.create(
    model="gpt-oss:20b",
    messages=[
        {"role": "system", "content": "Reasoning: high"},
        {"role": "user", "content": "Refactor this loop into a list comprehension: ..."},
    ],
)

print(resp.choices[0].message.content)

Step 5 — Add a chat UI with Open WebUI (optional)

The terminal is fine for testing, but for daily use you probably want a ChatGPT-style interface. Open WebUI is the most popular option and connects to Ollama automatically. If you have Docker installed, one command runs it:

Troubleshooting

The bottom line

gpt-oss-20b is the easiest way to run a genuinely capable OpenAI model entirely on your own hardware, for free, with a commercial-friendly license. Three commands — install Ollama, pull the model, run it — and you have a private assistant that handles coding, reasoning, and tool calls without an internet connection. For anyone who has wanted to drop their ChatGPT subscription for everyday tasks, this is the first OpenAI release that makes it realistic.