AI Decision Models: How They Work, Benefits & Use Cases

Most conversations about AI assume the goal is a chat model: something you talk to, that writes back. However, in a recent Passion Academy session, George Marmaras introduced a category of model built for something narrower and, for most businesses, more common: making a decision. No prose, no chat, just an answer and how sure the model is of it. He traced where this category came from, how it works under the hood and where it fits alongside the LLMs most people already use.

‍

Slides

‍

‍

‍

‍

Two Weeks That Created a Category

‍

Decision models aren't new in concept, but the category crystallised fast. In the space of about two weeks, a single product became an entire industry segment:

‍

• Sep 15: TypeSafe launched Jev, giving the category its name, "System One" models, via a closed API.
• Sep 15–22: A wave of open-weight alternatives followed: Laya, Nimble, Kev and OpenThai, all models you can run yourself.
• Sep 29: OpenAI released a Decisions API, built on GPT-6 Luna, in limited preview.
• Oct 1: Cloudflare shipped Clef and Clef-flash, open-weight models that also read images.
• Ongoing: Tooling caught up fast, with Ollama, OpenRouter, Vercel and LiteLLM all adding support.

‍

‍

The Problem

‍

The reason this category exists is that most AI work inside real products isn't writing, it's deciding. For years, the only tool available for that job has been a model built to write prose. That mismatch shows up clearly in practice. Ask an LLM "which team should handle this ticket? Reply in JSON," and you don't get a decision. You get a paragraph that happens to contain one:

‍

"Sure! Looking at this ticket, it seems to be about a failed payment, so the most appropriate team would probably be Billing. Here's the JSON you asked for: {"team": "Billing"}"

‍

To actually use that, you have to generate the paragraph, dig the answer out of it, check the format is valid and retry when it breaks. Every one of those steps adds time, cost, and another way for the pipeline to fail.

‍

‍

A Useful Lens

‍

Daniel Kahneman's distinction between System 1 and System 2 thinking maps unusually well onto this split.

• System 1: automatic, intuitive and effortless: recognising a face, reading a stop sign, sorting your inbox.
• System 2: deliberate, logical and effortful: planning a trip, solving a puzzle, writing an argument.

‍

In AI terms, decision models are the System 1 equivalent. LLMs, especially reasoning models are System 2. Real businesses run on both. Fast decisions happen thousands of times a day: which team should get this email, is this payment suspicious, is this invoice complete, does this CV meet the must-haves, is this review abusive. Slow decisions happen a few times a week: which market to enter next, how to answer a formal complaint, what goes into a contract. Each fast decision is small on its own but together they shape cost, risk and every customer's experience.

‍

‍

The Idea

‍

Give a decision model a situation and a set of questions. It returns answers and how sure it is of each one. Feed it "checkout has been failing for every customer since 9am" alongside the questions "which team?", "how urgent?" and "is a service down?".

‍

It comes back with: team → technical, 93%; urgency → today, 88%; down → yes, 97%.

‍

Most decision models work with three kinds of question. A choice picks one option from a list (which team should handle it: billing 6%, technical 93%, sales 1%). A score rates something on a scale (how urgent: can wait 2%, this week 10%, today 88%). A yes/no answers true or false (is a service down: yes 97%, no 3%).

‍

‍

Why It's Fast

‍

The speed difference comes down to how each type of model actually produces its output. An LLM is autoregressive: it writes its answer token by token, each one predicted from everything that came before. That's why a longer answer costs more time and more money and why you see it type out "Sure! Looking at this ticket..." before ever reaching the actual decision.

‍

A decision model is non-autoregressive. It reads the input once and every allowed answer is scored at the same time. There are no tokens to write, so there's no loop to run. It isn't a fundamentally different kind of model either, both are transformers, the same architecture behind LLMs. The difference is really just two changes: swap the next-token-prediction head for a scoring head that rates each option and remove the loop. One pass, then its done.

‍

‍

Consistency

‍

One underappreciated consequence of this design is repeatability. Send the same support ticket through an LLM five times with the question "which team should handle this?" and you can get billing, technical, billing, "it could be either...", technical. That's because an LLM samples: it picks each next token at random from its likely candidates, like rolling weighted dice, so two runs can land differently.

‍

A decision model, without a sampling step, gives the same scores for the same input every time: technical, 93%, five times in a row. That matters enormously if you ever want to test, audit, or debug the system, since a result you can't reproduce is a result you can't really investigate. This holds when scores are returned without sampling; it's worth checking how each specific provider implements this.

‍

‍

Can You Trust the Percentage?

‍

Every decision model answer comes with a confidence score but that number is only useful if it's calibrated (meaning stated confidence matches real accuracy). Say you set a rule: act automatically whenever the model is at least 90% sure. Look at 100 cases where a model said "90% sure." A calibrated model will be right about 90 times out of 100. An overconfident model might say "90% sure" and only be right 62 times out of 100, which makes that automation rule actively dangerous, since you'd be trusting a number that doesn't mean what it claims to mean.

‍

‍

How They're Trained

‍

The training method shapes what a model actually optimises for and this is where the three categories genuinely diverge.

‍

RLHF (reinforcement learning from human feedback) rewards answers people prefer, and produces the chat models most people use daily. The side effect: people tend to prefer answers that sound sure, so the model learns to sound confident whether or not it actually is. OpenAI's own GPT-4 technical report noted that calibration worsened after this kind of training.

‍

RLVR (reinforcement learning from verifiable rewards) rewards answers that can be automatically checked and produces reasoning models. Ask "what is 17 × 24?" and an automatic checker knows the answer is 408; the model gets rewarded for landing on it regardless of how confident it claimed to be along the way. The side effect: being right is rewarded, but stating overconfidence costs nothing, so the model learns to answer boldly. Research has reported that this specific training approach tends to make reasoning models overconfident.

‍

RL for calibration rewards confidence that actually matches reality, and is specifically what produces decision models. The model states a probability, the true outcome is later observed and it's penalised, lightly if its stated odds were honest, heavily if it was confident and wrong. Take a fraud-detection example: payments of a certain type turn out to be fraud 70 times in 100. A model that states 50% (too timid) pays a higher average penalty than one that states the true 70%, and one that states 90% or 99% (too sure) pays an even steeper penalty for being overconfident. The lowest average penalty comes from stating the actual odds. TypeSafe calls its specific version of this method RLCD and hasn't published it, but an independent open implementation exists, rewarding what's called a proper scoring rule.

‍

‍

Independent Benchmarks

‍

‍

‍

‍

A Teammate, Not a Replacement

‍

In practice, these don't compete with LLMs so much as pair with them. Three patterns came up repeatedly:

‍

• Decide, then write. The decision model routes the request, and an LLM drafts the reply. A support ticket goes to the right team, and then an LLM writes the response.
• ‍Write, then check. The LLM drafts, and the decision model checks it before it goes out. Does the draft promise a refund it shouldn't? Does it contain personal data?‍
• Escalate when unsure. The decision model handles a case when it's confident and hands it to an LLM or a human when it isn't. You choose the confidence threshold where that handoff happens.

‍

‍

‍

Potential optimal scenario: A request comes in, the decision model gives an answer and a confidence score. If it's sure, act automatically on the easy, repeated cases. If it's unsure, an LLM thinks it through and the decision model checks the result. If the stakes are high, a human reviews it and stays in charge where it matters. Research on this kind of cascade has found that keeping confident cases on the decision model and routing the rest to an LLM can perform on par with using the LLM for everything, at a fraction of the cost.

‍

‍

Where This Shows Up in Practice

‍

A few concrete examples from different domains:

‍

Finance. Thousands of news items and filings arrive every day, covering results, mergers, leadership changes, lawsuits and profit warnings. A decision model can read a headline like "Company X announces its CFO is resigning, effective immediately" and answer what happened (leadership change), whether it's good or bad (negative), and how important it is (high), then alert the portfolio managers who hold that stock.

‍

Recruitment. With 10,000 CVs in a database, a fast search can narrow that down to 200 loose matches, and a decision model then builds a careful shortlist of 15. For each candidate, it answers how well their skills match (a score), whether they have a must-have certification (yes/no), and whether their seniority fits the role (a score). The recruiter sets how much each answer counts, names, photos and ages are left out so the model can't use them, and a person makes the final call.

‍

Insurance. A claim form says "rear bumper damaged while parked on 3 October," alongside a photo. A decision model that reads both text and images can check whether the photo matches the claim (92% yes), and classify it as straightforward (81%), needing documents (15%), or suspicious (4%). Clean claims flow straight through; odd ones reach a human adjuster.

‍

‍

Choosing a Model

‍

The current landscape splits along two axes: open weights versus closed API, and speed. These are provider-reported figures on different hardware, so they're a starting point for comparison rather than a guarantee.

‍

‍

‍

Making It Yours

‍

Off-the-shelf is a starting point, not the destination. The real goal is a model tuned to your own data: the model decides on your real cases, you observe the real outcomes (was each decision actually right), you fine-tune using reinforcement learning on those outcomes and the result is a better model, tuned to your domain. Then you repeat. This is already available in practice: Cloudflare offers RL fine-tuning for its decision models and open models like Laya can be fine-tuned directly.

‍

‍

Honest Limits

‍

None of this should be taken as a universal fix and a few limits are worth naming plainly. A decision model can be confidently wrong: in one study, a decision model was highly confident on a task where it performed close to chance. It only picks from the options you give it: if the correct answer isn't on your list, it will still pick one of the ones that are. It doesn't explain itself, which is a real problem anywhere people are owed a reason for a decision, such as credit decisions. And vendor benchmarks are a starting point, not a guarantee: testing a model on your own data is the only way to know how it will actually perform on your use case.

‍

‍

The Broader Picture

‍

Decision models aren't a smaller or cheaper LLM. They're a different answer to a different question. An LLM is built to think something through and explain itself in words. A decision model is built to read a situation once and commit to an answer, with an honest number attached to how sure it is. Most AI work inside a real business is the second kind of task far more often than the first, routing, flagging, scoring, approving, and that's precisely the gap this new category of model is built to fill. The two aren't in competition; the more interesting systems being built right now use both, with the decision model doing the fast, repeated, bounded work, and the LLM stepping in for the cases that genuinely need thinking through.

‍

‍

References

‍

• Daniel Kahneman, Thinking, Fast and Slow (2011)
• GPT-4 Technical Report (OpenAI, 2023)
• Research reports on RLVR and reasoning-model overconfidence (arXiv 2609.38850)
• "Evaluating Decision Models for Text Annotation in Computational Social Science" (arXiv 2609.24574)
• OpenRouter Jev vs LLM benchmark; DMB benchmark (nibzard, GitHub); AY Automate benchmark

‍