Skip to content

Camera passthrough for live view - #520

Draft
andrewleesteele wants to merge 2 commits into
mainfrom
hypeship/camera-passthrough-live-view
Draft

andrewleesteele wants to merge 2 commits into
mainfrom
hypeship/camera-passthrough-live-view

Conversation

@andrewleesteele

@andrewleesteele andrewleesteele commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Lets a live-view viewer share their real camera into the browser as a local camera device, on demand: when a page inside the browser calls getUserMedia(), the viewer's camera turns on; when the page releases it, it turns off. A manual toggle is also provided.

This turns the hand-built prototype from the spike (kernel-images#515) into reviewable, convention-matching code.

Scope / safety. This relays a real person's real camera so they can complete their own camera step (identity verification where the person is present, video, capture). It is not a way to feed synthetic or recorded video to a liveness check. Liveness vendors reject virtual cameras by design; a live test confirmed this (below). Nothing here attempts to disguise the device to defeat that detection.

How it works

viewer's camera ──WebRTC──▶ neko ──▶ /dev/video1 (ingest)
                                         │  camerad mixer (single writer)
                                         ▼
chromium getUserMedia ◀── /dev/video0 (camera pages see)
  • server/lib/camera + server/cmd/camerad — the manager owns the browser-facing loopback device as the single writer, so chromium never loses its source: it streams a placeholder frame until the viewer shares, then live ingest frames, then back to the placeholder. It detects when a page holds the device open (the on-demand signal) and publishes State both on a localhost endpoint and as a static file the live-view client can poll.
  • cmd/wrapper + supervisor/services/camerad.conf + Dockerfile — camerad starts and neko's webcam ingest is enabled only when a session sets KERNEL_ENABLE_CAMERA=true (which requires the headful WebRTC/live-view path).
  • client — kernel/camera.ts plus shareLocalCamera()/unshareLocalCamera() on the neko client: auto start/stop on the on-demand signal, a manual toggle, and re-request on orientationchange.

Test results (real browser, 2026-11-16)

  1. Passthrough works. A page's getUserMedia() auto-starts the viewer's camera; the browser gets live 1280×720; the camera stops ~3s after the page releases it. Verified on the WebRTC getUserMedia sample.
  2. A real liveness SDK rejects the virtual camera (expected). A widely used browser liveness demo refuses the session with a "3rd-party app is modifying the video / suspicious webcam configuration" error — it detects the loopback device. So passthrough alone won't carry a liveness step on a vendor that blocks virtual cameras; it's for flows whose vendor permits a relayed camera, and for non-liveness camera use.
  3. Orientation is a known limitation: capture is pinned to 1280×720 landscape, so rotating a phone doesn't change the aspect. The client re-requests on rotate; native per-orientation device caps are a follow-up.

bench/camera-passthrough-live-view/ has the findings and a no-image-build reproduction.

Verified here

  • go build / go vet / go test ./lib/camera/ — pass (manager, device helpers, state handler, ingest stall watchdog).
  • go build / go vet on cmd/camerad and the modified cmd/wrapper — pass.
  • Live, in a real browser with this PR's camerad installed by hand (image not rebuilt; the client controller injected as a script): a page's getUserMedia() auto-starts the viewer's camera, the page gets live video within ~1s, and the camera stops after the page releases it — for two sessions back to back. The second session exposed a bug (the ingest reader held the device after the first share ended), fixed in the second commit.

Not done yet (needs an owner)

  • Guest kernel / loopback module provisioning — the device needs V4L2 + v4l2loopback in the browser VM. See kernel-images#515; camerad tolerates the module being preloaded or built into the kernel.
  • Browser image build not run here; neko client build + UI wiring (camera.ts is written but not yet imported into app.vue/main.ts, and no vue-tsc run).
  • Companion kernel/API + SDK + docs change for a camera: true browser-create option and a media-share permission (read-only viewers must not be able to share).
  • Cleaner page-driven (CDP) on-demand signal instead of the /proc scan.

Draft — do not merge.

🤖 Generated with Claude Code

andrewleesteele and others added 2 commits November 16, 2026 15:42
Let a live-view viewer share their real camera into the browser as a local
camera device, on demand: when a page inside the browser calls getUserMedia()
the viewer's camera turns on, and it turns off when the page releases it.

- server/lib/camera + cmd/camerad: a manager that owns the browser-facing
  loopback device as the single writer, streaming a placeholder frame until the
  viewer shares, then live ingest frames, then back — so chromium never loses
  its source. Reports an on-demand state the live-view client polls.
- wrapper/supervisor/Dockerfile: start camerad and enable neko webcam ingest
  only when a session sets KERNEL_ENABLE_CAMERA=true (rides the live-view path).
- client: a camera controller (kernel/camera.ts) plus shareLocalCamera on the
  neko client; auto start/stop on the on-demand signal, manual toggle, and
  re-request on orientation change.

Scope: this relays a real person's real camera. It does not feed synthetic or
recorded video to a liveness check — liveness vendors reject virtual cameras by
design, and a live test confirmed that; see bench/camera-passthrough-live-view.

Server builds, vets, and tests clean. The browser image build and the neko
client build/UI integration still need to run; the kernel/API camera option is
a companion change. Draft.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A v4l2loopback read blocks forever once its producer goes away. camerad's
ingest reader then kept the device open with the previous session's format,
so the next neko webcam session never reached the browser. Kill and reopen
the reader when no frame arrives for 3s.

Found testing in a real browser: the first share worked, the next never went
live. With the fix, back-to-back shares both go live in about a second.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant