Skrlr AI Lab
I benchmark local AI on real rigs and post the receipts.
Benchmarks and agent experiments measured on my hardware — with units, conditions, and honest failures. Security evals are part of the lab, not the whole story.
One RTX 5090 · power capped at 403 W
Latest numbers
From the logs.
Every card links to method and caveats. The summary is on the grid; the write-up is the receipt.
38 s (median)
8-step distilled · 107 frames @ 768×448
vs full 20-step final: 320 s (median)
HUNYUANVIDEO 1.5 FAST41 s (median)
5 s clip
LOCAL LLM DECODE~199 tok/s
Qwen3.8-Flash-Next · IQ3_S · speculative decoding
SEEDVR2 UPSCALE146 s
121 frames → 1080p
SHARED 24H · WHOLE APP206 jobs · 0 failed
3 people · 0 s median wait · 24h
TEXT-TO-IMAGE2-2.5 s (median)
per image · shared workflow
What I test
Three lanes in the lab.
01 · Local inference
LLM, image, video
tok/s decode, text-to-image latency, video drafts vs finals, upscale passes — always with hardware and power cap noted.
02 · Agents
Overnight coding runs
Tool use, iteration counts, and whether the artifact actually shipped — including runs where tests passed and the product did not.
03 · Security
Agent red-teaming
Prompt injection, exfiltration, hijacking, and multi-turn social engineering in sandboxed evals. Program →
Failures I publish
Green tests, blank game.
Passing CI is not the product. This run is the tone I want for the lab.
52 tok/s (avg)
176/176 tests passed · game blank · 0/36 tasks
176/176 tests passed. The game came out blank. 0/36 tasks done.
Bonsai 27B on a 12 GB RTX 4070 SUPER: 52 tok/s avg, 27 iterations, 1,146 tool calls.
Follow along
Lab notes on X.
No embeds — just links. New benchmark write-ups hit RSS.
RSS: /rss.xml · Contact: Matt@skrlr.com