Skip to Content
DocumentationGetting StartedInstall Ollama

Install Ollama

Time: about 20 minutes, most of it downloading · You’ll need: ~5 GB free disk, Docker installed

Ollama runs AI models directly on your machine — no API key, no per-token cost, nothing leaving your computer. This walkthrough installs it, gets you talking to a model, and connects it to Companion Hub.

Ollama is optional but strongly recommended. Without it, Hub apps that use AI need a cloud provider API key instead. You can also do both — see AI & Inference.

Install Ollama

Download the installer from ollama.com/download , open it, and drag Ollama to Applications. Launch it once — it runs in the background and starts automatically from then on.

Confirm it’s installed:

ollama --version

Expected:

ollama version is 0.5.4

Download and run your first model

Pull a model

llama3.2 is a good starting point — around 2 GB, and it runs acceptably on CPU alone:

ollama pull llama3.2

You’ll see progress as it downloads:

pulling manifest
pulling dde5aa3fc5ff... 100% ▕████████████████▏ 2.0 GB
verifying sha256 digest
writing manifest
success

Not sure what your hardware can handle? Hub’s Onboarding Wizard detects your CPU, RAM, and GPU and recommends models that fit — you can let it choose for you instead.

Actually talk to it

This is the part worth doing before anything else — it proves local inference works end to end:

ollama run llama3.2

You’ll get a prompt. Type a question and press Enter:

>>> Explain what a Docker volume is in two sentences.

The model responds locally, on your machine. Type /bye to exit.

The first response is slower than the rest — the model has to load into memory first. Subsequent replies in the same session are faster.

Confirm what’s installed

ollama list
NAME               ID              SIZE      MODIFIED
llama3.2:latest    a80c4f17acd5    2.0 GB    2 minutes ago

Let Hub’s containers reach Ollama

This is the step people miss, and it produces the single most common AI problem in Hub: apps report that no models are available even though ollama list shows several.

The reason: by default Ollama only listens on 127.0.0.1, which inside a Docker container means the container itself, not your host machine. Hub’s app containers reach your host at host.docker.internal — so Ollama has to be listening on all interfaces for them to connect.

Usually nothing to do — Docker Desktop routes host.docker.internal to the host automatically, and Ollama is reachable as-is.

Skip to Verify the connection. If that check fails, set OLLAMA_HOST=0.0.0.0:11434 in Ollama’s settings and restart it.

Binding to 0.0.0.0 makes Ollama reachable from any machine on your network, not just Docker containers. On a trusted home LAN that’s usually fine. On a shared or public network, firewall port 11434 to Docker’s bridge interface only.

Verify the connection

Two checks. The first proves Ollama is up; the second proves containers can reach it.

curl http://localhost:11434/api/tags

Expected — a JSON list including the model you pulled:

{"models":[{"name":"llama3.2:latest","size":2019393189,...}]}

Now the check that actually matters, from inside a container:

docker run --rm curlimages/curl -s http://host.docker.internal:11434/api/tags

Same JSON means you’re done. If you get Connection refused or a timeout, Ollama isn’t bound to all interfaces — repeat the step above.

On Linux, if the container check fails even after setting OLLAMA_HOST, your Docker version may not map host.docker.internal automatically. Confirm with docker run --rm --add-host=host.docker.internal:host-gateway curlimages/curl -s http://host.docker.internal:11434/api/tags — if that works, Hub will still connect, since it adds the host mapping itself.

Connect it to Companion Hub

Once Hub is installed, it detects Ollama automatically. Confirm:

cihub status

The Models section lists what Ollama has. You can also manage models through Hub:

cihub models list
cihub models install mistral

In the dashboard, Settings → AI shows the detected backend and lets you set which model apps use by default. Apps that declare AI support receive these settings automatically at install time — see AI & Inference.

Using a GPU

Ollama runs on CPU by default. If you have a discrete GPU, it will use it once the drivers are in place.

NVIDIA: install the NVIDIA Container Toolkit . Ollama detects CUDA automatically.

AMD: install ROCm drivers. Supported on Linux.

Check whether it’s actually working — load a model, then while it’s resident:

ollama ps
NAME               ID              SIZE      PROCESSOR    UNTIL
llama3.2:latest    a80c4f17acd5    3.5 GB    100% GPU     4 minutes from now

100% GPU means acceleration is working. 100% CPU means it fell back — usually the model is too large for your VRAM, or the drivers aren’t visible to Ollama.

Common problems

Hub shows no models

Run through Verify the connection. Almost always the OLLAMA_HOST bind step.

Model runs but is very slow

You’re on CPU, or the model is larger than your available RAM and is swapping. Try a smaller model:

ollama pull phi4-mini

Out of memory

The model needs more RAM or VRAM than you have. As a rough guide, a model needs about as much memory as its download size, plus overhead. Pick a smaller one, or let the Onboarding Wizard recommend based on your detected hardware.

Desktop app, Linux package, or headless server.

Next: Install Companion Hub

Backends, cloud providers, and how apps get AI config.

AI & Inference reference

See what Hub detected about your hardware.

System Inspector
Last updated on
Install DockerInstallation