Annotate2Spec
shippedDraw labeled boxes and arrows on a pasted screenshot or URL preview and get back deterministic JSON plus a ready-to-paste prompt describing exactly what to change, for a coding agent.
Approach
Single index.html, no build step: a fixed-size 'stage' div renders a fake dashboard screenshot using plain HTML/CSS (nav bar, cards, buttons, a chart placeholder) so no image generation or real screenshot is needed. An absolutely-positioned overlay layer sits on top; mousedown-drag-mouseup on the overlay draws a box, and a small inline popover lets the user type a label and pick an intent (move / remove / resize / restyle / unclear) or switch to arrow mode to connect two existing boxes. All annotations live in one JS array of plain objects (id, type, label, intent, bbox normalized 0-1, optional fromId/toId for arrows). Two pure functions render that array: toSpecJSON() does JSON.stringify into a <pre id="spec-json">, and toPrompt() walks the same array into a numbered, human-readable instruction block in a <textarea id="prompt-output">, both re-run on every add/delete/clear so the two panels are always a deterministic function of the annotation state. The page boots with ~20 pre-defined SEED_ANNOTATIONS already in that array (mixed boxes/arrows, mixed clear and deliberately ambiguous labels) so both output panels are already populated and scrollable before any click. A file input / paste handler can swap the stage's background for a user's own screenshot (as a data URL) without touching existing annotation coordinates, since bboxes are normalized fractions of the stage, not pixels.
The source post
https://x.com/awais_khawar/status/2109750960457490521
Scoring
Pick
| surprise | 3 |
|---|---|
| demonstrability | 5 |
| self_containedness | 5 |
| honesty | 5 |
The other three candidates are platform/pipeline concepts that can't be a single deployable web artifact or can't be honest about what they do here: Overmind and Axis are data/training pipelines, not something a stranger clicks through in 10 seconds, and Open Dots is a full local agent-rebuild whose one interesting capability (agents actually acting) needs a live model call this sandbox can't make. Annotate2Spec is the one idea that is purely a deterministic transform — draw a box, type a label, get JSON and a prompt back — with zero inference, zero third-party calls, and zero accounts. It's small, it's instantly legible in a browser, and nothing about its core claim is faked or mocked; the Chrome-extension packaging in the original post is dropped in favor of a standalone page that does the same drawing-to-structured-output trick.
Review
| shipped | 5 |
|---|---|
| honest | 5 |
| worth_it | 3 |
| efficient | 5 |
No change: nothing in tonight's run points at scout query noise, pick taste, a repeatable build lesson, or an underperforming executor. The only real finding tonight is a pipeline-code issue (publish.py's wait_for_https, written to tell a slow cert from a dead host, is never called from stage_publish, so a stuck Cloudflare domain validation would currently ship a published.json and a 'passed' night pointing at a URL that doesn't resolve yet) — that's a code fix outside every block I'm allowed to edit, so it goes to escalate instead of being forced into a prompt or routing change.
escalated: publish.py defines wait_for_https() specifically to distinguish 'certificate still issuing' from 'actually broken' (see its docstring), but stage_publish in build.py never calls it — it writes published.json right after attach_domain's two idempotent API calls succeed, without confirming the custom domain actually serves 200. Tonight's deploy resolved fine by the time of this review, and Cloudflare's own API showed the custom-domain validation still 'pending' at review time even though it worked — so this is currently a silent gap rather than tonight's failure. Worth wiring wait_for_https into stage_publish (or at least recording its result) so a night that ships to a domain stuck in pending validation is caught as a failure instead of recorded as shipped.
Cost
| total | $0.1293 |
|---|---|
| xai | $0.1293 |
What it looked at
Visual annotation Chrome extension: draw directly on any webpage, export deterministic JSON + AI-ready prompt for coding agents (no inference).
Overmind: open-source tool that logs AI agent runs, auto-detects failures, builds training datasets, and fine-tunes models via Modal (GitHub-linked).
Open Dots: fully local, self-contained rebuild of OpenAI’s Dots with folder-based chats, custom agents/avatars, file I/O, sandboxed browser control, and approval gates.
Axis: crowdsources real browser interaction demos to generate robot training data (concrete data-collection pipeline).
Gate
| pass | project directory exists — /Users/artax/code/builds/2026-10-13/project |
|---|---|
| pass | no build step needed — static project |
| pass | build output with index.html — /Users/artax/code/builds/2026-10-13/project |
| pass | index.html is a document — 25651 bytes |
| pass | local asset references resolve |
| pass | page loads without console errors |
Stages
| scout | grok · ok · 21.0s |
|---|---|
| pick | claude · ok · 47.5s |
| plan | claude · ok · 74.0s |
| build | codex · ok · 416.3s |
| gate | local · ok · 2.8s |
| publish | local · ok · 11.0s |
| review | claude · ok · 244.2s |
Notes
Omitted for lack of
None. All planned capabilities run locally in the browser.