October 1, 2026 · 8 min read · inference · decision-models · evaluation · llm-ops

Decision models: what I measured before the wave hit

A decision model takes your text and your options and returns a probability over the options. It never writes a sentence. I live-tested TypeSafe's Jev in September, read the Needle 3 release against it, and wrote down what the schema guarantee does and does not buy. Then, in one week, OpenAI, Cloudflare and Perplexity each shipped one.

Dated October 1, 2026, the day Cloudflare and Perplexity both shipped a decision model and the category stopped being one vendor. Written October 12. The Jev test ran on September 17, the Needle 3 comparison on September 18. A postscript covers a Clef-omni run on October 9. Numbers are single runs, not benchmarks.

The contract

A decision model is a model that cannot write. You give it a state, which is text, and a set of questions. Each question comes with its allowed answers: a list of labels, a yes or no, or a rubric with a few levels. The model returns a probability distribution over the answers you supplied, for each question, and nothing else. There is no prose, no explanation, and no token that was not in your option list.

That contract is the whole point. A generative model asked to classify can return a label you did not offer, a sentence instead of a label, or JSON that does not parse. A decision model structurally cannot. The vendors call this zero hallucination. TypeSafe’s own launch post is honest about what that means: the number is not empirical, it is a definition. Schema validity is guaranteed. Semantic correctness is not. A perfectly valid approve on a request that should have been refused meets the guarantee.

For someone deploying models, the contract changes three things. Latency, because scoring a fixed option set is one forward pass, not a decode loop. Cost, because output tokens are free when there are none. And the shape of the surrounding code, because your application now owns abstention, thresholds, and every dependency between answers.

What I measured: Jev, September 17

TypeSafe’s Jev is a hosted API. The version I hit was jev-1.13.0. I wrote four small tasks with gold labels fixed by a separate agent before any request was sent, and ran them from my laptop.

The routing cases included paraphrases, negation, mixed requests, sarcasm and quoted instructions. The support cases included omitted facts, wrong dates, speculation and a passage that tried to instruct the model. The baseline was deliberately dumb: keyword, regex and substring rules.

Latency across the 48 accuracy cases: median 193 ms, observed p95 530 ms, range 123 to 774 ms, measured end to end from a MacBook Air over a shared HTTPS connection. Then a batching sweep, five repeats each:

Twenty questions in one request cost no more wall time than one. They cost more tokens, because each question’s labels and descriptions ride along. The whole run, 64 requests and 187 individual judgments, came to 31,368 input tokens, about 0.13 US cents at list price.

What this does not prove. Forty-eight short synthetic cases are a smoke test. Four ranking cases are barely a signal. Labels were agent-written. And the run says nothing about calibration: nearly every yes-or-no probability came back below 0.05 or above 0.92, and 28 of the 41 choice and score confidences were exactly 1.0. The middle band, which is where a threshold would live, was never exercised. The same file also shows identical repeated requests returning slightly different probabilities, 0.71 to 0.77 on one question. The model is not bit-deterministic, which matters if you plan to cache on input.

What I did not measure: Needle 3

Cactus shipped Needle 3 the same day I ran the Jev test. It is the other shape of the idea: a 121-million-parameter local model, a 35 MB file, CPU only, Apache-2.0 weights, that turns a command into a function call. I read the release and the docs against Jev. I did not run it, so everything here is the vendor’s contract, not my measurement.

The shallow framing is “Needle chooses functions, Jev makes decisions”. That is wrong in both directions. Needle also classifies and extracts, and returns a confidence. Jev also selects tools, with one choice question over the function descriptions and more choice questions for the bounded arguments. The real distinctions are four.

Bounded versus open-ended values. Needle generates argument spans under a byte-level grammar compiled from your schema, so it can produce a value you never listed: a search query, a name, a number copied from the request. Jev cannot. For a free value you write a regex that over-finds candidates and Jev picks one. That is safer, and it caps recall at whatever your regex found.

Sequential versus parallel. A generated tool call is a chain. The tool name conditions the arguments, and a wrong early token propagates. Jev answers each question in isolation. Nothing propagates, and nothing checks that the answers agree. TypeSafe’s own limitations page shows a refund probability of 0.72 and a not-refund probability of 0.47 from two questions on the same state. If answers depend on each other, that dependency is your code or a second round trip.

Where the guarantee lives. Needle’s grammar makes an invalid call unrepresentable, and its grounding check, confidence floor and repair pass are runtime code the vendor ships. Jev’s label validity is intrinsic, because it has no other output space, and everything else is yours: abstention through an explicit other option, thresholds, candidate generation, composition into an action.

Local versus cloud. Needle runs air-gapped on a device. Jev is a US-hosted API under an as-is agreement whose terms bar publishing benchmarks of the service.

The analogy I kept: a stencil and a scale. Grammar-constrained decoding and a fixed label set are stencils. Paint lands inside the letters, which says nothing about whether it is the right word. Confidence is a scale, and it reads correctly only if someone checked it against known weights. Neither vendor publishes a reliability curve. Independent testers have: one found a 0.99 selection from refund-related options for an unrelated weather report, and another measured accuracy on an arithmetic choice falling from 88% to 57% when the correct option moved from first to fourth position. Option order is a hyperparameter nobody documents.

Where it earns a place in my stack

The first draft of this comparison said Jev was the better first pilot and Needle was not ready as shipped. A second pass caught that the Needle failure reports I was leaning on were filed against Needle 2, before the release. The corrected position is about fit, not superiority. Neither has a measured accuracy advantage over the other on any matched task, because no matched task exists.

Jev’s question interface fits two jobs I actually have. First, a source-support check: code verifies a quoted passage exists, Jev answers whether the passage supports the sentence, low scores go to a larger model or to me. Second, reranking the top twenty hits from my own search index, where the fallback is simply the original order. Both have documented patterns and a trivial off switch. Needle’s command runtime fits a small local voice vocabulary: timer, playback, pause, open a known app. Each of these is a pilot against a baseline, not a deployment. The baselines are a cross-encoder reranker, an NLI model and, for commands, plain rules.

Where none of them belongs: arithmetic, dates, permissions, deduplication, exact matching. Both vendors say so and both are right. A cheap model does not earn a job that a comparison operator already does.

Then the wave

Between September 29 and October 1 the category went from one vendor to four. OpenAI previewed a Decisions API at its developer day on September 29, on the same endpoint shape, at ten cents per million input tokens. On October 1 Cloudflare released Clef, 27 billion parameters, and Clef-flash, 9 billion, with open weights under Apache-2.0. The same day Perplexity released a Decisions API and open-sourced a 27-billion-parameter model fine-tuned from Qwen3.8, at four cents per million input tokens with a 262K context, and published a panel where it edges Jev by 1.2 points overall while losing to it on several individual sets. Those numbers are Perplexity’s own.

Three thoughts, in order of how much I would bet on them.

The contract is now a commodity. Four vendors ship the same input and output shape within a fortnight. Whatever moat exists is in calibration, order robustness and price, none of which any of them publishes properly yet. Pin a version and tune your thresholds on your own held-out data, because the numbers will not transfer across vendors.

Open weights change the local lane. In September the choice was a tiny local model with a grammar or a hosted one with labels. Two 27-billion-parameter decision models with open weights mean the Jev contract can now run on a box I control, which also voids the clause about not publishing benchmarks. A 27B model in bf16 is about 54 GB, which fits a 96 GB Mac Studio with room for a cache. Whether the Metal path works on day one is a separate question, and the postscript answers it.

The guarantee did not get stronger, only cheaper. Every one of these models is a stencil. Each new vendor’s launch repeats the zero-hallucination line, and each one means schema validity. The measurement I still want from any of them, and have not seen, is a reliability diagram on a task with human labels.

Postscript, October 9: Clef-omni on my own hardware

Cloudflare released Clef-omni on October 9, the multimodal member of the family, a fine-tune of a 30-billion-parameter mixture of experts with 3 billion active, 70 GB in bf16, Apache-2.0. I tried it the same day.

On the Mac Studio, 96 GB M3 Ultra, the Metal path loads and returns near-uniform probabilities. Eager attention did not fix it, and a newer transformers release was worse than the one before it. The CPU path in bf16 is correct and takes 6 to 25 seconds per decision. So the open weights are real, and the local lane for this architecture is not, not yet, not on Apple silicon.

On a rented A100 80 GB spot instance at 94 cents an hour it worked fully. Load 15 seconds, 63.7 GB of VRAM resident. A text decision in 0.27 seconds, an image in 0.3, an audio clip in 0.3, a two-second video in 0.75. The whole session, including setup, cost 49 cents, and the box and its volume were deleted and verified gone afterwards, because that is what my GPU console is for.

One gotcha worth the cost of the run. The yes-or-no option order in the reference code is ("true", "false"), so index zero is the probability of true. My first media test read index one and looked inverted. That is exactly the order-sensitivity problem from September, now on my side of the API.