Small LLM vs Frontier Model: Which One Does Your AI Feature Actually Need?

Most AI features inside a SaaS product do not need a frontier model. If the task is bounded (classify a ticket, extract fields from a document, draft an email from facts you already have), a small model with a strict prompt and an output validator is usually good enough and costs a fraction per call. Reserve the frontier model for open-ended reasoning, long multi-step tool use, and anywhere a wrong answer costs real money.

The question is not "which model is smartest"

Founders tend to ask us which model is best. That is the wrong question, because the best model on a leaderboard is rarely the best model for a feature that will run fifty thousand times a month inside your product. The right question is: what is the cheapest model that passes my eval set at the accuracy I need, with the guardrails I can afford to build?

Take the ticket-classification example from the top of this post and make it concrete. At fifty thousand tickets a month, the leaderboard winner and a distilled "flash" model will both tag the obvious tickets the same way. The difference shows up on the ambiguous ones, and a validator that checks the tag against your allowed list and retries on a miss catches a good share of what the small model gets wrong. The only way to know whether the remainder is acceptable is to run the fifty to a few hundred real tickets in your eval set through both models and compare, with the per-call price next to each result.

Framed that way, the decision stops being a bet on vendor marketing and becomes an engineering measurement. The small-model category has moved fast; the recent wave of "flash" and distilled releases from several labs means a model that would have been state of the art two years ago now costs very little per token and runs on a single GPU. Whether that is enough for you depends on four things.

Four questions that decide it

1. Is the task bounded?

A bounded task has a known input shape, a known output shape, and a small set of acceptable answers. "Pull the invoice number, due date and total from this PDF" is bounded. "Read this customer's last six emails and decide what to do" is not.

Small models are strong on bounded tasks because the prompt can carry most of the intelligence: the schema, the rules, three worked examples. They degrade on unbounded tasks because they need to improvise, and improvisation is exactly what gets worse as parameter count drops.

2. Can you check the output cheaply?

If a deterministic check can catch most bad outputs, you can run a smaller model and let the validator do the heavy lifting. JSON schema validation, regex on dates and amounts, a lookup against your own database, a length cap: these cost nothing per call and turn a model's occasional mistake into a retry instead of a bug a customer sees. We wrote about this pattern in deterministic guardrails for AI feature reliability.

If the only way to check the output is for a human to read it, the model has to be right the first time more often, and that pushes you up the price list.

3. What does a wrong answer cost?

A mis-tagged support ticket costs an agent ten seconds. A wrong dosage summary in a healthcare tool, or a wrong refund decision in an e-commerce agent, costs a lot more. Price the failure, not the call. Where the failure is cheap, optimise for cost per call. Where the failure is expensive, the frontier model's per-token premium is trivial next to the risk, and you should also add a human review step regardless of model.

4. Does the input fit, and does it need memory?

Small models typically have shorter usable context and get lost sooner in long documents. If your feature needs to hold a 200-page contract in view, or carry state across a twenty-step workflow, you are either chunking aggressively (more engineering) or paying for a model that can hold it. Multi-step agentic work with tools is the clearest case for frontier models, and it is worth reading agents versus workflows before you assume you need the agent at all.

What a small model is actually good at: our outreach engine

Our own outbound tool is a concrete example. It crawls each prospect's website with a self-hosted Firecrawl instance, hands the page text to a local model, and writes one tailored email per company. The task is bounded: the input is a few pages of text, the output is a short email following our template and tone, and a validator rejects anything that is too long, mentions a product we do not offer, or fails to cite a fact from the site.

What surprised us was that a single call doing "extract the facts and draft the email" beat a multi-step chain (extract, then plan, then write) on both cost and quality, even with the smaller model. Fewer hops meant fewer places for the model to drift from the source text. We covered the general case in single call versus agent chains. The point for a buyer: a well-constrained small model plus a validator was the right tool, and a frontier model would have added cost without changing the reply rate.

When you should pay for the frontier model

Be honest about the cases where cheap is expensive. The outreach engine above gets away with a small model for one reason: there is a single hop, page text in and email out, with a validator standing behind it. Remove any of those conditions and the picture changes.

• Open-ended reasoning over messy input. Triage that needs judgement, not rules. Anything where the instructions say "use your discretion". Our validator can reject an email that cites no fact from the site; it cannot judge whether the model chose the right fact, and when that judgement is the whole feature, you want the stronger model.
• Multi-step tool use. Agents that call several APIs, interpret results, and decide the next step. The moment our task became "crawl the site, decide which pages matter, check the CRM, then write", the small model would be carrying state across hops, which is exactly where it drops the thread.
• Low-volume, high-value calls. If the feature runs a few hundred times a month and each result is reviewed by a senior person, the cost difference is noise. Buy accuracy.
• Code generation and complex transformation. The quality gap is still wide here.
• The first version, before you have evals. It is often cheaper to launch on a strong model, collect real traffic, build an eval set of fifty to a few hundred real examples from it, and then downgrade where the evals say you can.

That last one matters. Starting on a frontier model and moving down is a measured decision backed by an eval set. Starting on a small model and discovering in production that it is not good enough is a rewrite of your prompts and guardrails under pressure, with no eval set to tell you when you are done.

How the choice changes your quote

Founders expect the small model to make the project cheaper to build. It mostly does not. The build cost is in the prompt design, the validator, the eval set, the retry and fallback logic, and the integration into your product. That work is the same whichever model sits behind it. If anything, a small model needs a thicker validation layer, which is slightly more engineering, not less.

What changes is the running cost. Per-call prices between a frontier model and a small hosted model differ by a large multiple, and over a year of real traffic that difference can exceed the original build budget. Ask for a projection at three volume levels before you sign; our guide to LLM cost optimisation lists the levers that matter beyond model choice, such as caching and prompt length.

Self-hosting the small model is a separate decision. Tools like vLLM and Ollama make it technically easy, but it only pays off at steady volume or when data residency forces your hand. At typical early-stage volumes, a cheap hosted API model wins once you count the engineering time.

The hedge: build the switch, not the bet

Whatever you decide, do not let it be baked into the code. Insist on three things in the brief:

• Model behind a single config value. Changing from a frontier model to a small one, or between providers, should be a deployment, not a sprint.
• An eval set shipped with the feature. Fifty to a few hundred real examples with expected outputs, run before every model change. Without it, "is the cheaper model good enough?" is a guess. We argue this belongs in the contract in evals in AI vendor contracts.
• A fallback path. When the small model fails validation twice, escalate that single call to the stronger model rather than failing the user. This gives you small-model economics with frontier-model worst-case quality.

Done this way, the model question becomes a dial you turn as traffic and evidence come in, instead of a decision you make once with no data.

If you are scoping an AI feature and want a straight answer on which model tier it needs and what it will cost to run, let's talk.