👓 Omi (prototype)

Developer prototype for Omi capture hardware. This prototype is a backend that runs on a PC you control, connects to an Omi DevKit 2 over Bluetooth LE, transcribes audio locally with Whisper, and analyzes each transcript with a local Ollama model.

Overview

Omi is an open-source wearable line. The prototype targets two developer kits: the Omi DevKit 2, which streams audio over Bluetooth LE, and the omiGlass DevKit, an ESP32-S3 with a camera. The backend runs on a “companion PC” on your network. It scans for the device over Bluetooth, reads the device’s audio stream, transcribes it with the open-source Whisper model on that PC, asks a local Ollama model to label each transcript (topic, tone, intent, entities, urgency), and logs the result as a JSON event you can read over a REST API or a live WebSocket.

Transcription and analysis run on the companion PC. The first run downloads the Whisper model, and the Ollama models must already be pulled. The service also has an optional Omi API client (OMI_API_KEY, default https://api.omi.me) for Omi’s own cloud, and it accepts webhooks that Omi’s cloud can send to it. Those are separate from the local pipeline.

Key features

• Bluetooth LE audio — scans every 10 seconds for BLE devices whose name contains “omi”, “devkit”, or “glass”, connects, and starts an audio handler for DevKit devices.
• Local transcription — Whisper runs on the companion PC. The default model is base, set with WHISPER_MODEL.
• Local analysis — Ollama labels each transcript with a topic, tone, intent, entities, urgency, and short summary. The default chat model is llama3.2:3b.
• Event log and API — events are stored in a SQLite database on the PC. Read them with GET /api/events, watch them on the /ws/live WebSocket, or send events in through /api/webhooks/conversation, /api/webhooks/audio, and /api/webhooks/transcript.
• Required API key — every endpoint except GET /health needs an X-API-Key header.

Supported hardware

There is no phone relay and no Wi-Fi setup mode for the pendant in the repository. The companion PC talks to the device directly over Bluetooth, so it needs a Bluetooth adapter and must be within range of the device.

Companion devices compared

Omi is one of several Companion devices. Each one captures or shows something different, and they connect in different ways; this table records only what each device’s own page states.

Setup for each lives on its own page: Mentra Live, Even Realities, Pebble, Sipeed MaixCAM, and Home Assistant Voice PE.

Use cases

• Experiment with capturing conversations on your own hardware and transcribing them locally.
• Feed transcripts and events from a DevKit into your own tools through the event API.

Architecture

flowchart TD
    DK["Omi DevKit 2"] -->|"Bluetooth LE<br/>audio stream"| BLE["BLE manager"]
    BLE --> AP["Audio processor"]
    AP --> W["Whisper<br/>local"]
    W --> OL["Ollama<br/>topic · tone · intent"]
    OL --> EV["Event log<br/>on the PC"]
    OC["Omi cloud webhooks"] -->|"/api/webhooks/*"| EV
    EV --> API["REST API and /ws/live<br/>port 8000"]

Setup

You need a PC with Python 3.8 or newer, at least 8 GB of RAM (16 GB recommended), and 10 GB of free disk space. Linux is recommended. macOS and Windows are also listed.

chmod +x scripts/setup.sh
./scripts/setup.sh

ollama serve
# In a second terminal:
ollama pull llama3.2:3b
ollama pull moondream:1.8b-v2-fp16

./run.sh

Usage

Authenticate

Every endpoint except GET /health requires the key from .env:

export OMI_COMPANION_API_KEY=$(grep OMI_COMPANION_API_KEY .env | cut -d= -f2)
curl -H "X-API-Key: $OMI_COMPANION_API_KEY" http://localhost:8000/api/status

If no key is set, the service answers every guarded request with 503. It never falls back to open access.

Start capture

curl -X POST -H "X-API-Key: $OMI_COMPANION_API_KEY" http://localhost:8000/api/start

List connected devices with GET /api/devices and read logged events with GET /api/events.

Watch events live

The /ws/live WebSocket streams events as they happen. Browsers can’t set headers on a WebSocket handshake, so pass the key as an api_key query parameter. Other clients should send the X-API-Key header instead, because query parameters end up in proxy logs.

Limitations

• Not connected to Companion Memory or Hub. Nothing sends transcripts or events to Companion Memory. There is no Hub app listing, no Hub pairing, and no Omi source in Companion Memory.
• Audio decoding is a placeholder. The audio processor treats incoming bytes as raw 16-bit PCM. Decoding the device’s Opus stream isn’t implemented.
• No camera capture. The vision code exists, but nothing feeds it frames, and POST /api/capture/image returns a placeholder response.
• No web UI, speaker diarization, or retention settings. The service is API-only. The event store can delete events older than a set number of days, but no setting or API calls it.
• Most of the plan is unbuilt. The repository’s own implementation checklist has 6 items checked and 255 unchecked.

Troubleshooting

Ollama connection failed Check that Ollama is running with curl http://localhost:11434/api/tags. Start it with ollama serve.

No device found Make sure the DevKit is powered on and advertising, that Bluetooth is enabled on the PC, and that the device isn’t already connected to another host. The scan looks for names that contain “omi”, “devkit”, or “glass”.

Every request returns 503 or 401 The service refuses guarded requests until OMI_COMPANION_API_KEY is set, and rejects requests without a matching X-API-Key header. Run ./scripts/setup.sh or ./run.sh, which generate a key and write it to .env.

Port 8000 is already in use The service always listens on port 8000 on all network interfaces. The API_HOST and API_PORT settings in .env aren’t read. Find the process that holds the port with lsof -i :8000 and stop it.

Logs The service logs to the terminal where you started it.

Links

• Omi official site 
• Companion Memory
• Mentra Live