Repository navigation
Camera passthrough for live view - #520
Draft
andrewleesteele wants to merge 2 commits into
Draft
andrewleesteele wants to merge 2 commits into
andrewleesteele wants to merge 2 commits into
Conversation
Let a live-view viewer share their real camera into the browser as a local camera device, on demand: when a page inside the browser calls getUserMedia() the viewer's camera turns on, and it turns off when the page releases it. - server/lib/camera + cmd/camerad: a manager that owns the browser-facing loopback device as the single writer, streaming a placeholder frame until the viewer shares, then live ingest frames, then back — so chromium never loses its source. Reports an on-demand state the live-view client polls. - wrapper/supervisor/Dockerfile: start camerad and enable neko webcam ingest only when a session sets KERNEL_ENABLE_CAMERA=true (rides the live-view path). - client: a camera controller (kernel/camera.ts) plus shareLocalCamera on the neko client; auto start/stop on the on-demand signal, manual toggle, and re-request on orientation change. Scope: this relays a real person's real camera. It does not feed synthetic or recorded video to a liveness check — liveness vendors reject virtual cameras by design, and a live test confirmed that; see bench/camera-passthrough-live-view. Server builds, vets, and tests clean. The browser image build and the neko client build/UI integration still need to run; the kernel/API camera option is a companion change. Draft. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A v4l2loopback read blocks forever once its producer goes away. camerad's ingest reader then kept the device open with the previous session's format, so the next neko webcam session never reached the browser. Kill and reopen the reader when no frame arrives for 3s. Found testing in a real browser: the first share worked, the next never went live. With the fix, back-to-back shares both go live in about a second. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Lets a live-view viewer share their real camera into the browser as a local camera device, on demand: when a page inside the browser calls
getUserMedia(), the viewer's camera turns on; when the page releases it, it turns off. A manual toggle is also provided.This turns the hand-built prototype from the spike (kernel-images#515) into reviewable, convention-matching code.
How it works
server/lib/camera+server/cmd/camerad— the manager owns the browser-facing loopback device as the single writer, so chromium never loses its source: it streams a placeholder frame until the viewer shares, then live ingest frames, then back to the placeholder. It detects when a page holds the device open (the on-demand signal) and publishesStateboth on a localhost endpoint and as a static file the live-view client can poll.cmd/wrapper+supervisor/services/camerad.conf+ Dockerfile —cameradstarts and neko's webcam ingest is enabled only when a session setsKERNEL_ENABLE_CAMERA=true(which requires the headful WebRTC/live-view path).kernel/camera.tsplusshareLocalCamera()/unshareLocalCamera()on the neko client: auto start/stop on the on-demand signal, a manual toggle, and re-request onorientationchange.Test results (real browser, 2026-11-16)
getUserMedia()auto-starts the viewer's camera; the browser gets live 1280×720; the camera stops ~3s after the page releases it. Verified on the WebRTC getUserMedia sample.bench/camera-passthrough-live-view/has the findings and a no-image-build reproduction.Verified here
go build/go vet/go test ./lib/camera/— pass (manager, device helpers, state handler, ingest stall watchdog).go build/go vetoncmd/cameradand the modifiedcmd/wrapper— pass.cameradinstalled by hand (image not rebuilt; the client controller injected as a script): a page'sgetUserMedia()auto-starts the viewer's camera, the page gets live video within ~1s, and the camera stops after the page releases it — for two sessions back to back. The second session exposed a bug (the ingest reader held the device after the first share ended), fixed in the second commit.Not done yet (needs an owner)
v4l2loopbackin the browser VM. See kernel-images#515;cameradtolerates the module being preloaded or built into the kernel.camera.tsis written but not yet imported intoapp.vue/main.ts, and novue-tscrun).camera: truebrowser-create option and a media-share permission (read-only viewers must not be able to share)./procscan.Draft — do not merge.
🤖 Generated with Claude Code