Trace Autopsy
shippedA browser-based replay debugger that steps through recorded agent/LLM traces, showing exactly which failure check tripped, why, and how the response diverged from what was expected.
Approach
A single index.html with inline CSS and JS, no build step and no network calls. A JS array embeds 6 authored agent-trace fixtures (support-bot refund flow, SQL-generating data agent, code-review bot, multi-tool travel planner, RAG citation agent, content-moderation agent), each with 3-5 steps, 21 steps total, mixing role (user/assistant/tool), a named check (schema-validation, hallucination-detection, tool-call-correctness, policy-compliance, citation-grounding, latency-budget), an expected vs. actual string, and a verdict of pass/warn/fail (roughly half fail or warn so the failure-diff behavior is visible by default). On load the page renders two things at once: a flattened, filterable table of all 21 steps (trace name, step #, check, verdict badge) so the full dataset is visible without interaction, and a replay panel on the right already showing the first trace's first failing step — prompt, response, and a word-level diff between expected and actual computed client-side with a small LCS-based tokenizer, marking removed words with .diff-del and added/changed words with .diff-add. Clicking any table row or a trace in the sidebar loads that trace into the replay panel and resets the step scrubber; Prev/Step/Next buttons and a slider move through a trace's steps, each step re-rendering the prompt, response, check explanation, verdict badge, and diff. Verdict filter buttons (All/Pass/Warn/Fail) toggle which table rows are visible by toggling a CSS class rather than removing rows, keeping the interaction purely DOM-side.
The source post
https://x.com/scasso/status/2114167834624585994
Scoring
Pick
| surprise | 3 |
|---|---|
| demonstrability | 4 |
| self_containedness | 5 |
| honesty | 4 |
Setaccio Lab's pitch is a prompt/response/failure-check logger for replaying what went wrong in an AI demo — the right shape to rebuild as a single static web artifact: vendor a handful of authored agent-trace fixtures (prompt, response, the check that fired, expected vs. actual) into the repo, then let a stranger scrub through a timeline and watch each step's verdict and diff with no backend, account, or live model call. It's the only candidate of the six that clears every hard rule outright: llmdojo.dev and leonlin.de/demos are directories of other people's work rather than one buildable idea (and llmdojo's scope overlaps hard with prior nights' tokenizer/attention/training labs); whatsmyagentdoing is effectively last night's agent-office under a different name; the MIT photo-to-CAD model needs real model weights pulled at build time, which this sandbox's network-dead build stage cannot do; and the Harvey/claude-seo/Hallmark stack is a three-repo, live-API workflow, not a deployable artifact. Trace Autopsy ties with a reshaped llmdojo single-lab idea on total score, and wins the tie on surprise since replay/debug tooling for agent failures hasn't been the last three weeks' theme the way model-internals visualizers have.
Review
| shipped | 3 |
|---|---|
| honest | 5 |
| worth_it | 3 |
| efficient | 5 |
No change: NOTES.md reports nothing changed from plan, nothing could not be done, and nothing omitted, and gate.json passed on attempt 1 with no repairs — there is no repeatable scout, pick, build, or routing failure here to fix. The one real problem found (the custom domain returning 522 while the pages.dev deployment is healthy) is a publish/infra anomaly outside the editable set (scout query, pick rubric, build lessons, routing executor) and is a single data point against a 30-night history where no other night shows the same symptom, so it does not clear the three-nights-of-evidence bar this review is supposed to hold itself to.
Cost
| total | $0.1462 |
|---|---|
| xai | $0.1462 |
What it looked at
llmdojo.dev**: Fully interactive browser-based labs (50+) where you tweak tokenizers/attention/embeddings, train a neural net live, build RAG/agent workflows with tool calls/memory/guardrails, and run production setups like quantization/caching.
leonlin.de/demos**: Curated live collection of small interactive web demos built with the latest models (Opus 5.5, GPT-6 Astra, etc.)—perfect bite-sized examples of what a self-contained app can do.
whatsmyagentdoing** (GitHub): Tiny web/office visualization that renders Claude Code, Codex, or Gemini agents as animated characters moving around a desk—super fun, self-contained demo of agent behavior.
MIT photo-to-CAD model** (open-source): Upload a sketch or photo → instantly generates fully editable parametric 3D models exportable as STL; runs locally, no CAD software needed.
Setaccio Lab** (Java, open-source): Lightweight tool that logs every model prompt, response, and failure check so you can replay and debug exactly what went wrong in an AI demo.
Harvey + claude-seo + Hallmark stack** (three GitHub repos): End-to-end agent workflow—Harvey finds leads, claude-seo runs 19-agent audits, Hallmark redesigns sites—fully runnable on a Claude subscription with visible outputs.
Gate
| pass | project directory exists — /Users/artax/code/builds/2026-10-26/project |
|---|---|
| pass | no build step needed — static project |
| pass | build output with index.html — /Users/artax/code/builds/2026-10-26/project |
| pass | index.html is a document — 20008 bytes |
| pass | local asset references resolve |
| pass | page loads without console errors |
Stages
| scout | grok · ok · 40.2s |
|---|---|
| pick | claude · ok · 86.9s |
| plan | claude · ok · 50.3s |
| build | codex · ok · 250.8s |
| gate | local · ok · 2.8s |
| publish | local · ok · 10.6s |
| review | claude · ok · 141.0s |
Notes
Omitted for lack of
None. All planned capabilities operate locally with embedded data.