Sortie
A control plane for rented GPUs that leaves nothing behind.
Browser console over Kaggle, Modal and Verda spot. Every launch carries a deadline, idle and unreachable timeouts, and a delete that is verified by a re-read.
$80/mo → $0
I make LLM inference cheaper, and I can prove it.
Machine learning engineer. I take models from a notebook to a server with a budget and a deadline, and I write down the numbers.
- $80 → $0monthly stranded GPU storage
- 15×KV cache overestimate caught
- 193 msmedian decision-model latency, measured
01 — Projects
Opportunity Desk
An LLM filter over SEC filings with a measured judge model.
Pulls filings, extracts the facts that matter, and routes them through a schema-checked judge. The primary model was picked by an A/B on token cost and confidence disagreements.
−36% tokens
The Inference Guide
A 21,000-word field guide to serving LLMs, with the arithmetic shown.
KV cache sizing, roofline, quantisation, batching, and the hardware each one touches. Thirty-four figures. Checked on a Mac Studio and on rented A100 and B300 boxes.
15× smaller
02 — Writing
- KV cache arithmetic for a hybrid-attention model, checked against two real machines
The textbook KV formula overstates Gemma 4 31B's cache by about fifteen times at 32K context, because fifty of its sixty layers only ever look back 1,024 tokens. I work the real number for my Mac Studio, then compare the first-principles figure with what vLLM actually allocated on a rented B300, and explain the gap.
- Sortie: a control plane for rented GPUs that leaves nothing behind
A scheduler I was not watching stranded four volumes on a GPU cloud and billed $80 a month for storage I never used. The fix was not a reminder. It was a console where every GPU job is ephemeral by construction: pick a lane, launch with a deadline, work, download, and everything is deleted and verified.
- Decision models: what I measured before the wave hit
A decision model takes your text and your options and returns a probability over the options. It never writes a sentence. I live-tested TypeSafe's Jev in September, read the Needle 3 release against it, and wrote down what the schema guarantee does and does not buy. Then, in one week, OpenAI, Cloudflare and Perplexity each shipped one.
- Haiku vs Sonnet as a verdict writer: 8 cards, one gate, 36% fewer tokens
An A/B of two Claude models on the judgment layer of an SEC filing scanner. The cheaper model wrote more and was less calibrated. The config changed the same day.
03 — About
I run a small homelab for inference on Apple Silicon and rent GPUs when the job needs more. Everything on this site is built from logs, configs and commit history, not from memory. If a number is a single run, the post says so.