How to Use Ollama Local Models in Dify

Point a self-hosted Dify at a local Ollama server, register a model like Llama 3.1 or Qwen, and run your apps fully offline with no per-token cost.

12 minIntermediate

Visit Dify

Dify can use a local model served by Ollama instead of a hosted provider. This keeps your prompts and data on your own machine, removes per-token cost, and lets you build apps that run with no internet. The setup is short: run Ollama, pull a model, add the Ollama provider in Dify, and give it the right base URL so the containers can reach your host.

What you need

A self-hosted Dify instance (the Ollama provider needs a local model server, so this does not apply to hosted Dify Cloud)
Ollama installed and running from ollama.com
Enough RAM or VRAM for the model you plan to run
About 12 minutes

Step 1: Pull a model in Ollama

Install Ollama, then pull a model. Start with something small enough for your hardware so the first test is quick. Ollama serves an HTTP API on port 11434 once it is running, which is what Dify will talk to.

bash

download a local model

$ollama pull llama3.1

success

confirm Ollama is serving on 11434

$curl http://localhost:11434/api/tags

{"models":[{"name":"llama3.1:latest", ...}]}

Step 2: Add the Ollama provider in Dify

In Dify, open Settings and then Model Provider. Find Ollama in the provider list, install its plugin if it is not already there, and click Add Model. You fill in the model name exactly as Ollama knows it and the base URL where Ollama is listening.

Field	What to enter
Model name	The exact Ollama tag, for example llama3.1 or qwen2.5
Base URL	http://host.docker.internal:11434 when Dify runs in Docker
Model type	Chat, for a conversational model
Context size	Match the model's context window, for example 8192

Use host.docker.internal, not localhost

Dify's containers cannot reach Ollama at localhost, because inside a container localhost is the container itself. Use http://host.docker.internal:11434 so the container reaches Ollama on your host. On Linux, if host.docker.internal is not resolvable, use your host's LAN IP or add an extra_hosts entry mapping host.docker.internal to host-gateway.

Step 3: Select the local model in an app

Open any app, or create a new Chatbot from blank, and pick your Ollama model from the model selector in the prompt editor. Send a test message in the Debug and Preview pane. The response now comes from the model running on your machine, and nothing leaves your network.

Dify - local model selected

Model: llama3.1 (Ollama, local)

--------------------------------------------

System: You are a helpful assistant.

User: Say hello in one short sentence.

--------------------------------------------

Reply generated locally, no API cost

The Ollama model appears in the selector once the provider is added.

Caveats to plan around

Local models trade cost for resources and speed. A model that fits your RAM or VRAM will run; one that does not will be very slow or fail to load. Latency is higher than a hosted API on modest hardware, so keep prompts tight. Context windows on small local models are limited, which matters for RAG and long documents, so match the context size you enter in Dify to what the model actually supports.

Start small, then scale the model

Get the wiring working with a small model first. Once a test message returns cleanly, swap in a larger model by pulling it in Ollama and adding it as another model entry. You can keep several local models registered and switch per app.

Result: a Dify app running entirely on a local Ollama model, with no per-token billing and no prompt data leaving your machine.