🥽 Spatial Scan

Digitize your reality. Walk through a room with your phone and turn it into a private, structured, explorable 4D map — geometry, detected objects, and the camera’s path through time — stored securely on your local node.

Overview

Spatial Scan is a browser-based AR room scanner and desktop world editor. On the phone it uses 8th Wall  SLAM (with a native WebXR fallback) to track the space and accumulate a live colourised point cloud; an on-device vision-language model labels the objects it sees. On the desktop it composes one or more scans into a single geo-anchored world you can inspect, label, reconstruct into Gaussian splats, and share.

Everything runs on your Hub (or a connected edge device) — scans never leave your local network. Results integrate with Companion Memory, making your physical spaces searchable, navigable, and usable as context for your Companion Agents, and can be explored over time in the Spatial Time Machine.

What a “4D map” is

A scan is not a flat mesh. Each world lives in a metric, geo-anchored frame (East / Up / North, in meters) and holds one or more scans, each separated into typed layers — point cloud, floor, detections, labels, camera path, camera frustums, RGB frames, depth, and Gaussian splat. The fourth dimension is time: every frame is timestamped and posed, and edits are an append-only history, so a single world can hold multiple scans of the same place captured at different moments and replay the camera’s trajectory through it.

Key Features

• Web AR scanning — 8th Wall SLAM tracks the room and builds a live point cloud; a native WebXR fallback runs on headsets and WebXR-capable browsers
• On-device object detection — an engine-agnostic vision-language layer labels objects as you scan; the default runs a local Ollama model (moondream), with a bundled ONNX model and a WebGPU in-browser path as alternates — detection can run entirely on-device with zero data exfiltration
• Live 3D object pins — detections are placed at their real world positions as labelled 3D pins that track the scene
• Walkthrough-video capture — no AR? Upload a phone video and Spatial Scan extracts posed keyframes into a scan
• Gaussian Splatting — train photorealistic 3D Gaussian splats from a scan’s keyframes (Brush, GPU-accelerated) and detect objects in the splat
• Dense reconstruction — densify streamed keyframes into a fused point cloud and camera poses (LingBot-Map)
• Desktop world editor — place many scans in one ENU world with layer toggles, semantic tagging, a metric measure tool, a geomap, transform gizmos, and progressive level-of-detail loading
• Companion Memory integration — scans are tagged with location, date, and semantic labels for recall
• Change over time — keep multiple scans of the same space and compare them
• Share by role — hand someone an owner / editor / viewer / scanner link
• Export — download models as .glb, .ply, or .splat
• Privacy — scans never leave your local network; the WebGPU detector never uploads a frame at all
• PWA + offline — installable on iOS/Android with an offline upload queue

The scanning experience

On your phone — capture a room

Open the link, tap through a three-step primer, grant camera + motion, and start sweeping the room. A live coverage meter tells you when you have enough.

Tap the capture button to identify objects — each detection appears as a card with a thumbnail, label, and confidence, and as a 3D pin in the scene. When you’re done, export or just let autosave sync it.

On the desktop — structure the 4D map

Open the world editor to see your scans placed in the shared ENU frame with detection labels, a compass, a scale bar, and a live coordinate readout. Toggle layers, tag objects, measure real distances, and re-arrange scans.

Queue reconstruction and detection jobs, then share the finished world with a role-scoped link.

Train a photorealistic Gaussian splat from the scan and detect objects within it, right from the Splat Toolbox.

How it works

Reconstruction Methods

Setup

Usage

Viewing a completed scan

Open the world editor and orbit / pan / zoom the scene. Switch render modes (solid, points, wireframe, semantic), toggle layers, and click an object to inspect its label, confidence, dimensions, and room/floor.

Measuring

Click Measure, then click two points — because the world is metric ENU, the readout is in real meters.

Tagging and editing

Select a scan or object in the Edit tab to move / rotate / scale it, assign a semantic tag, or add notes. Every edit is saved as an append-only patch.

Browsing in the timeline

Scans appear in the Companion Memory timeline under the Spatial source, tagged with location and date, and can be explored over time in the Spatial Time Machine.

Exporting

Export a scan or world as .glb (web / general 3D), .ply (point-cloud tools), or .splat (Gaussian-splat viewers).

Troubleshooting

Tracking never starts / “Move camera to initialise tracking…” forever Camera or motion permission was denied. Re-enable it in your browser/OS settings (iOS: Settings → Safari → Motion & Orientation Access, and camera access for the site) and reload over the HTTPS link.

“HTTPS required” on the start screen Camera and AR APIs only work on a secure origin. Open the Spatial Scan HTTPS link (not a plain-HTTP address).

Splat training or dense reconstruction fails immediately These jobs need streamed keyframes and a GPU worker. Start keyframe streaming (the ☁️ button) while scanning, and confirm the Hub has a GPU and the reconstruction workers configured.

Reconstructed model has holes Insufficient coverage. Capture additional angles of the problem areas and re-scan; the world composes multiple scans of the same space.

Scan not appearing in Companion Memory Ensure the Companion Memory connector is enabled in Settings → Integrations and check the connector log for API errors.

For developers

Spatial Scan is built on the open engineering repo companionintelligence/CI-Web-XR-Scan.

• System architecture  — data model, the 4D world model, the full API surface, and design decisions
• UX flow & FTUE spec  — the everyday-user journey, with a mobile + desktop screenshot gallery 
• Roadmap  — shipped vs. planned

Links

• Spatial Time Machine
• Photogram
• Earth
• Companion Memory