From 7a47749eb028b26375e60f9c0419bc11904df104 Mon Sep 17 00:00:00 2001 From: andrewleesteele <8799863+andrewleesteele@users.noreply.github.com> Date: Mon, 16 Nov 2026 15:42:09 +0000 Subject: [PATCH 1/2] Add camera passthrough for live view MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Let a live-view viewer share their real camera into the browser as a local camera device, on demand: when a page inside the browser calls getUserMedia() the viewer's camera turns on, and it turns off when the page releases it. - server/lib/camera + cmd/camerad: a manager that owns the browser-facing loopback device as the single writer, streaming a placeholder frame until the viewer shares, then live ingest frames, then back — so chromium never loses its source. Reports an on-demand state the live-view client polls. - wrapper/supervisor/Dockerfile: start camerad and enable neko webcam ingest only when a session sets KERNEL_ENABLE_CAMERA=true (rides the live-view path). - client: a camera controller (kernel/camera.ts) plus shareLocalCamera on the neko client; auto start/stop on the on-demand signal, manual toggle, and re-request on orientation change. Scope: this relays a real person's real camera. It does not feed synthetic or recorded video to a liveness check — liveness vendors reject virtual cameras by design, and a live test confirmed that; see bench/camera-passthrough-live-view. Server builds, vets, and tests clean. The browser image build and the neko client build/UI integration still need to run; the kernel/API camera option is a companion change. Draft. Co-Authored-By: Claude Opus 4.8 --- bench/camera-passthrough-live-view/README.md | 50 +++ .../camera-passthrough-live-view/camwatch.sh | 13 + .../kernel-camera.injected.js | 59 +++ images/chromium-headful/Dockerfile | 8 + .../chromium-headful/camera-placeholder.png | Bin 0 -> 13380 bytes .../client/src/kernel/camera.ts | 158 ++++++++ .../chromium-headful/client/src/neko/base.ts | 21 ++ .../supervisor/services/camerad.conf | 9 + server/cmd/camerad/main.go | 57 +++ server/cmd/wrapper/main.go | 18 + server/lib/camera/camera.go | 78 ++++ server/lib/camera/manager.go | 349 ++++++++++++++++++ server/lib/camera/manager_test.go | 68 ++++ 13 files changed, 888 insertions(+) create mode 100644 bench/camera-passthrough-live-view/README.md create mode 100644 bench/camera-passthrough-live-view/camwatch.sh create mode 100644 bench/camera-passthrough-live-view/kernel-camera.injected.js create mode 100644 images/chromium-headful/camera-placeholder.png create mode 100644 images/chromium-headful/client/src/kernel/camera.ts create mode 100644 images/chromium-headful/supervisor/services/camerad.conf create mode 100644 server/cmd/camerad/main.go create mode 100644 server/lib/camera/camera.go create mode 100644 server/lib/camera/manager.go create mode 100644 server/lib/camera/manager_test.go diff --git a/bench/camera-passthrough-live-view/README.md b/bench/camera-passthrough-live-view/README.md new file mode 100644 index 000000000..6cc2cd493 --- /dev/null +++ b/bench/camera-passthrough-live-view/README.md @@ -0,0 +1,50 @@ +# Camera passthrough for live view — reproduction & findings + +Harness and findings behind the production code in this PR (`server/lib/camera`, +`server/cmd/camerad`, the wrapper/neko wiring, and the client controller in +`images/chromium-headful/client/src/kernel/camera.ts`). Follows the +guest-kernel / loopback-module spike in kernel-images#515. + +## What this enables + +A live-view viewer shares their **real** camera into the browser so a page +inside it can use it. On demand: the viewer's camera turns on when a page calls +`getUserMedia()` and off when it stops. This is for a real person completing +their own camera step — **not** for feeding synthetic or recorded video to a +liveness check. + +## Findings (validated in a real browser, 2026-11-16) + +1. **Passthrough works.** A page's `getUserMedia()` auto-starts the viewer's + camera; the browser receives live 1280×720; the camera stops ~3s after the + page releases it. Verified on the WebRTC getUserMedia sample. + +2. **Liveness SDKs reject the virtual camera, by design.** A widely used + browser liveness demo refuses the session with an "a 3rd-party app is + modifying the video / suspicious webcam configuration" error — it detects the + v4l2loopback device. **Camera passthrough alone will not carry a liveness + step on a vendor that rejects virtual cameras.** We do not attempt to + disguise the device to defeat that detection: the check exists to confirm a + real person is physically present. Passthrough is for flows whose vendor + permits a relayed camera, and for non-liveness camera use (video, capture). + +3. **Orientation (known limitation).** Capture is pinned to 1280×720 landscape + end to end, so rotating a phone does not change the camera aspect. The client + controller re-requests `getUserMedia` on `orientationchange`; native + per-orientation device caps (reloading the loopback) remain a follow-up. + +## Reproduce against a running browser (no image build) + +Inside a browser VM with the camera kernel module available (see +kernel-images#515 for the guest-kernel bits): + +```sh +# load the loopback pair, run the manager, enable neko webcam ingest +camerad & # prebuilt binary from this PR +NEKO_CAPTURE_WEBCAM_ENABLED=true NEKO_CAPTURE_WEBCAM_DEVICE=/dev/video1 \ + supervisorctl restart neko +``` + +`kernel-camera.injected.js` is the exact client controller used in the live +test, as a drop-in `