In this categoryLocal AI · 70

More guides

Run Models

Local Apps & UIs

Hardware & Builds

Model Spotlights

Fine-Tuning

Local AIIntermediate

How to Use Ollama as a Drop-In OpenAI API

Point any OpenAI-compatible SDK or tool at your local Ollama server by changing one URL, and use OLLAMA_HOST to expose it on your network.

12 minIntermediate

Ollama ships a REST API that mirrors the OpenAI chat completions spec. Any library or tool that accepts a custom base URL, including the official OpenAI Python and Node SDKs, LangChain, LlamaIndex, and Continue, can talk to your local Ollama server with zero code changes beyond the base URL and a dummy API key. This guide covers the raw curl calls, the Python SDK, and the environment variables that control how Ollama exposes itself.

What you need

Ollama installed and running (ollama serve or the background service)
At least one model already pulled (ollama pull llama3.3)
curl for the verification steps
Python 3.9+ or Node 18+ if you want to follow the SDK examples

Step 1: Understand the endpoint layout

Ollama exposes two API families on port 11434. The native Ollama API under /api is richer but non-standard. The OpenAI-compatible API under /v1 is the one you use for drop-in replacement.

GET http://localhost:11434 — health check, returns 'Ollama is running'
GET http://localhost:11434/v1/models — list available models in OpenAI format
POST http://localhost:11434/v1/chat/completions — chat completions (same body as OpenAI)
POST http://localhost:11434/v1/completions — legacy text completions
POST http://localhost:11434/v1/embeddings — embeddings (requires an embedding model like nomic-embed-text)

Step 2: Test with raw curl

Before touching any SDK, confirm the endpoint works with curl. The body is identical to what you would send to api.openai.com, but with a local URL and any non-empty string as the API key.

zsh - curl chat completions

$curl http://localhost:11434/v1/chat/completions \

-H 'Content-Type: application/json' \

-H 'Authorization: Bearer ollama' \

-d '{

"model": "llama3.3",

"messages": [{"role": "user", "content": "Why is the sky blue?"}],

"stream": false

{"id":"chatcmpl-...","object":"chat.completion","model":"llama3.3",

"choices":[{"message":{"role":"assistant","content":"The sky appears blue..."},

"finish_reason":"stop"}],"usage":{"prompt_tokens":14,"completion_tokens":82}}

The API key value does not matter

Ollama's /v1 endpoint accepts any non-empty Authorization header. Passing Bearer ollama or Bearer local-key both work. This is only for header compatibility — Ollama does no authentication by default.

Step 3: Use the OpenAI Python SDK

Install the official openai package and point it at your local server. Change only base_url and api_key. The rest of your code stays identical to what you wrote for the real OpenAI API.

zsh - install sdk

$pip install openai

Successfully installed openai-1.x.x

Python - openai sdk with ollama

local_chat.py

$python3 local_chat.py

The capital of France is Paris.

from openai import OpenAI
client = OpenAI(base_url='http://localhost:11434/v1', api_key='ollama')
response = client.chat.completions.create(
model='llama3.3',
messages=[{'role': 'user', 'content': 'What is the capital of France?'}]
)
print(response.choices[0].message.content)

Step 4: Use the OpenAI Node.js SDK

zsh - node sdk

$npm install openai

local_chat.mjs

$node local_chat.mjs

The speed of light is approximately 299,792,458 metres per second.

import OpenAI from 'openai';
const client = new OpenAI({ baseURL: 'http://localhost:11434/v1', apiKey: 'ollama' });
const res = await client.chat.completions.create({
model: 'llama3.3',
messages: [{ role: 'user', content: 'What is the speed of light?' }],
});
console.log(res.choices[0].message.content);

Step 5: Expose Ollama on your local network

By default Ollama listens only on 127.0.0.1, so only the local machine can reach it. Set the OLLAMA_HOST environment variable to bind to all interfaces and reach it from other devices on the same network, such as a phone or another dev machine.

zsh - expose on network (macOS/Linux)

For the current shell session:

$OLLAMA_HOST=0.0.0.0:11434 ollama serve

To make it permanent, add to your shell profile:

$echo 'export OLLAMA_HOST="0.0.0.0:11434"' >> ~/.zshrc

Ollama is running on 0.0.0.0:11434

From another machine: curl http://192.168.1.x:11434

PowerShell - expose on network (Windows)

$$env:OLLAMA_HOST="0.0.0.0:11434"

$ollama serve

Or set it permanently via System Properties > Environment Variables

Do not expose Ollama to the public internet without authentication

OLLAMA_HOST=0.0.0.0 is safe on a trusted home or office LAN, but never bind to a public IP without putting a reverse proxy with authentication (nginx + basic auth, or Tailscale) in front. Ollama has no built-in auth and anyone who can reach port 11434 can run unlimited model inference on your hardware.

Streaming responses

Ollama supports streaming the same way OpenAI does. Set stream: true in the request body and iterate over the server-sent events. Both the Python and Node SDKs abstract this into async iterators.

zsh - streaming curl

$curl http://localhost:11434/v1/chat/completions \

-H 'Content-Type: application/json' \

-H 'Authorization: Bearer ollama' \

-d '{"model":"llama3.3","stream":true,

"messages":[{"role":"user","content":"Count to 5"}]}

data: {"choices":[{"delta":{"content":"1"}}]}

data: {"choices":[{"delta":{"content":", 2"}}]}

data: {"choices":[{"delta":{"content":", 3, 4, 5."}}]}

data: [DONE]

Set OLLAMA_ORIGINS to allow browser requests

If you are calling Ollama from a web app running on localhost:3000, set the OLLAMA_ORIGINS environment variable to allow CORS: OLLAMA_ORIGINS=http://localhost:3000 ollama serve. You can also set it to * during development, but tighten it before deploying anything accessible to others.

At this point any tool or library built for the OpenAI API works against your local Ollama server. Switch model names to llama3.3, codellama, mistral, or any other pulled model. You can also run multiple models and switch between them per request, which the local server handles by loading and unloading from GPU memory automatically.

Local models run better with more VRAM. CompareRTX GPUs on Amazonbefore you upgrade.(affiliate link. We may earn a commission at no extra cost. Disclosure)