Local Vision Model Webcam Captioner

Approach

Single index.html, no build step: an ES module script imports transformers.js (@huggingface/transformers) straight from a CDN (esm.sh), which loads the quantized ONNX weights for HuggingFaceTB/SmolVLM-256M-Instruct from the Hugging Face Hub CDN at runtime in the visitor's own browser — a static model-weight fetch with no key, account, or server behind it, which is the one piece of third-party traffic this idea cannot avoid and is why it was picked over alternatives tonight. Inference backend is chosen at runtime: WebGPU if navigator.gpu exists, else the wasm/CPU backend, so it degrades gracefully instead of failing on a GPU-less visitor. On load, before any click, the script feeds 6 bundled seed photographs (generated at build time, stored as static JPEGs, no runtime fetch) through the model at 3 crops each (full frame, left half, right half) — 18 genuinely distinct inputs — and streams each real caption into a scrolling transcript as it completes, so the page is self-evidently running a working model before the visitor touches anything. A "Use my camera" button then requests getUserMedia and repoints the same caption loop at a live downscaled video frame every few seconds, appending to the same transcript; if camera access is unavailable or denied (the default in a headless check), the page shows a plain-text fallback message instead of erroring. Everything else — markup, CSS, inference loop, transcript UI —ships inline in the one HTML file.

The source post

https://x.com/RoundtableSpace/status/2110102021210526133

Scoring

Pick

This is the only candidate tonight whose interesting claim is both real and checkable by a stranger in seconds: open the page, allow the camera, and a small open-source vision-language model (SmolVLM) runs entirely client-side via WASM, captioning the live feed with no server round-trip and no API key. The model weights load from a CDN at page load in the visitor's browser, which is a runtime fetch, not a build-time one, so it doesn't collide with this sandbox's no-network build stage. Every other candidate either isn't a deployable web artifact (Midscene drives native mobile/desktop UIs, Vercel's npx skills and OpenQodex are CLIs), can't fit the self-containedness bar (AirLLM needs a 2.8T-parameter model and real GPU memory tricks; Overmind needs Modal GPU infra), overlaps too closely with an idea already shipped this month (Lucian's UI gallery vs. matchcn-gallery), or can't be made honest under the no-network build constraint (the Stable Diffusion latent PCA demo needs real diffusion latents, which would require either a live model call or a vendored dataset fetched over the network — both disallowed here, so it would ship as faked fixture data standing in for the one interesting claim).

source: https://x.com/RoundtableSpace/status/2110102021210526133

Review

mean 2.25/5

proposed change: {'file': 'prompts/pick.md', 'block': 'taste-rubric', 'edit': "In the self_containedness criterion, add: build.md's rule is 'No API keys, no accounts, no runtime third-party requests' — unconditional, and not limited to the build step. A candidate whose one interesting capability needs a visitor-side fetch (model weights from a CDN, a live API call made from the browser) is exactly as disqualified as one needing a build-time fetch. 'It's a runtime fetch, not a build-time one' is not an exemption and must not be used to justify a high score — it was, for smolvlm-webcam-captions (2026-10-14), which scored self_containedness 4 on exactly that reasoning (pick.json: 'a runtime fetch, not a build-time one, so it doesn't collide with this sandbox's no-network build stage'), then had build correctly refuse to violate the no-runtime-third-party-request rule and strip the entire core claim, shipping an honest but empty shell instead.", 'expect': 'Future candidates whose interesting capability requires a visitor-side third-party fetch (model weights from a CDN, a live API call from the browser) score self_containedness ≤2 and lose to a vendorable alternative, instead of being picked at a high score and then gutted by build the same night.', 'falsified_by': "A night where pick again scores a visitor-side-third-party-dependent idea's self_containedness ≥4 and build still has to strip its core claim for the same reason; or, conversely, a night where this wording causes pick to reject an idea build could actually have shipped as-is — three or more such nights in either direction means the added clause needs to be reworded or reverted."}

The night's entire failure traces to one place: pick.json explicitly reasoned that a CDN-hosted model-weight fetch was fine because it happens at runtime in the visitor's browser rather than during the build step, citing the self_containedness rubric's build-step-scoped wording. But build.md's actual rule (line 43: 'No API keys, no accounts, no runtime third-party requests') bans visitor-side third-party requests unconditionally, with no build-time/runtime carve-out. Build correctly honored its own rule and refused to ship fake inference, which is the right call and is not what's being changed — but it meant the one interesting thing the picked idea promised was dead before plan.json was even written, and the whole night's budget went into a shell that gates clean but delivers nothing. This is a single, concrete, attributable gap in pick's rubric text, not weather: the rubric's own words invited the exact reasoning that caused the loss.

Cost

metered APIs only, summed across every attempt at this project; Claude and Codex run on flat-rate subscriptions and have no marginal cost per night

What it looked at

Wanted, and did without

Gate

Stages

Notes

Omitted for lack of

- Build-time network access or an artifact-download capability capable of retrieving and saving npm browser distributions and large Hugging Face ONNX model binaries into the workspace. This would allow the complete model and runtime to be bundled without any visitor-side third-party requests. No hosted inference API, account, or API key is needed or desired.