💫 Local Bench

Validate your inference. Benchmark local LLM performance on your own hardware so you know exactly how fast your Ollama models run.

Install & availability

Available now. Install from the Companion Hub App Store. Verified on the test fleet: the published image pulls, starts, serves its web UI, and survives a container restart.


Hub App Store	Search Local Bench → Install
Web UI port	`3000`
Architecture	`linux/amd64` and `linux/arm64` (multi-arch)

Overview

Local Bench is a benchmarking tool for local language models served by Ollama. It runs prompts through your models, records throughput and timing alongside your hardware specs, and stores the results so you can compare runs over time. Use it before deploying a model to understand how it actually performs on your Hub.

Current capabilities

What Local Bench does today, as shipped:

Benchmarks models served by Ollama
Records results together with system specs for context
Persists runs to disk and exposes them via a REST API
Ships a curated model catalog as a fallback when Ollama isn’t reachable
Shows a friendly empty state on first run before any benchmarks exist

Connecting Ollama

Local Bench talks to Ollama via the OLLAMA_API_URL environment variable (default http://localhost:11434). Inside a container, localhost is the container itself, so point it at the host or a LAN address:


# Ollama running on the Docker host
docker run -p 3000:3000 \
  -e OLLAMA_API_URL=http://host.docker.internal:11434 \
  -v localbench-data:/app \
  ghcr.io/companionintelligence/ci-local-bench:latest
 
# Or a LAN address
-e OLLAMA_API_URL=http://192.168.1.50:11434

When Ollama is unreachable, GET /api/models returns HTTP 503 along with a curated catalog of models, so the UI degrades gracefully instead of breaking. Fix the connection by correcting OLLAMA_API_URL.

Persistence

Local Bench writes two files into /app:

benchmark_data.db — the SQLite results database
benchmark_results.csv — a CSV mirror of results

Mount a volume at /app (as in the example above) to keep your benchmark history across restarts.

API

Endpoint	Method	Purpose
`/api/results`	GET	Stored benchmark results
`/api/results-with-specs`	GET	Results joined with system specs (`?limit=` supported)
`/api/system-specs`	GET	Detected hardware specs
`/api/prompts`	GET	Benchmark prompt set
`/api/models`	GET	Available models (503 + curated catalog if Ollama is down)
`/api/run-benchmark`	POST	Run a benchmark (blocks while the run is in progress)

POST /api/run-benchmark runs synchronously and blocks until the benchmark completes. Expect the request to stay open for the duration of the run.

Use Cases

“Which model runs fastest on my hardware right now?”
Capture before/after numbers when you change GPUs or quantizations
Keep a CSV/SQLite history of how your Hub performs over time
Document your Hub’s specs alongside real benchmark figures

Setup

Install from Hub

Search for Local Bench in the Hub app store and install.

Open Local Bench

App hostnames use your configured local domain — .localhost is the default. If one doesn’t resolve on your setup, see Finding Your App’s URL.

Navigate to http://local-bench.localhost. On first run you’ll see a friendly empty state until you record a benchmark — an error banner up top, a Select Models to Benchmark list, and a Select Test Prompt picker with a curated set of prompt cards (AI Paragraph, Python Function, Math Problem, and more) to choose from.

Local Bench Run Benchmarks screen — a No benchmark results found error banner, a Select Models to Benchmark list showing deepseek-r1 variants as Not installed, a 0 installed / 42 supported summary, and a Select Test Prompt picker with prompt category cards like AI Paragraph and Python Function

Point it at Ollama

Set OLLAMA_API_URL to a reachable Ollama endpoint (e.g. http://host.docker.internal:11434 from a container, or a LAN address). With the default http://localhost:11434, the container only sees itself. When Ollama isn’t reachable, the model list falls back to the curated catalog shown above with every model marked Not installed.

Run a benchmark

Pick a model and a test prompt, then click Run Benchmark (or Select All to queue every model). Results are saved and appear via /api/results, alongside System Specifications, a Performance Chart, and a Detailed Results table you can refresh as new runs complete.

Local Bench Run Benchmark, Select All, and Deselect All buttons, followed by System Specifications, Statistics, Performance Chart, and Detailed Results sections, each empty until a benchmark has run

Troubleshooting

No models listed Ollama is likely unreachable, so /api/models returned 503 with the curated catalog. Check OLLAMA_API_URL points to a host/LAN address — not the container’s own localhost.

Results disappear after restart Mount a volume at /app so benchmark_data.db and benchmark_results.csv persist.

A benchmark request seems to hang POST /api/run-benchmark blocks for the whole run by design. Wait for it to finish.