A client came to me with an AI bill that had quietly grown to several million rupiah a month, all running on the most capable model available, for tasks like tagging support tickets and extracting fields from invoices. That is the small language models vs large question in its most expensive form: using a frontier model for work that a much cheaper, smaller model handles just as well. The fix took an afternoon and cut the bill by more than half.
This is one of the clearest cost-optimization opportunities in AI right now, and most businesses miss it entirely because "just use the best model" feels like the safe default. It is safe for correctness. It is not safe for your budget, and at volume the difference compounds fast.
What Are Small Models Actually Good At?
Small models, the mini and lightweight tiers every major provider now offers, are genuinely strong at a specific category of work: narrow, well-defined tasks where the input is predictable and the output has one obviously correct shape. Five kinds of work fit that description reliably:
- Classification: is this email a complaint, a question, or spam.
- Extraction: pull the invoice number, date, and total from this document text.
- Short-form drafting: write a one-line acknowledgment reply.
- Simple summarization: condense a five-paragraph email into two sentences.
- Structured formatting: convert messy text into a clean JSON object matching a schema.
These tasks share a trait: the correct answer space is narrow, and the model does not need deep reasoning or broad world knowledge to get it right. A smaller model, trained on the same underlying techniques as its bigger sibling, handles these reliably, often at ten to twenty times lower cost per request.
Where Do Frontier Models Earn Their Price?
Frontier models earn their higher price when a task genuinely needs capability a small model cannot fake: reasoning that carries across several steps, judgment where tone and context both matter, or context long enough that missing one detail changes the outcome. Four categories of work call for that tier:
- Multi-step reasoning: working through a pricing calculation with several conditional rules.
- Nuanced judgment: drafting a sensitive customer response where tone and context both matter heavily.
- Long-context synthesis: reading a lengthy contract and flagging inconsistencies across sections.
- Ambiguous or open-ended tasks: where the input does not fit a predictable pattern.
If your task looks like any of these, do not cost-optimize your way into a cheaper model. The failure cost, a wrong contract clause missed, a tone-deaf reply to an angry customer, will exceed whatever you saved on the API bill.
The real architecture: tiered routing
The grown-up version of this is not "pick one model for everything." It is routing requests to the cheapest tier that can handle them reliably, and escalating only when needed. A simple version looks like this:
- Classify first, cheaply. A small model looks at the incoming request and tags its type and complexity.
- Route by tier. Routine, narrow tasks go to the small model. Anything flagged ambiguous, sensitive, or complex escalates to the frontier model.
- Monitor and adjust. Track how often escalation happens and whether the small model's outputs on routine tasks actually hold up, adjusting the routing rules over time.
This is the same architectural instinct behind the AI workflow patterns we cover in Workflow Automation: Wiring AI Into the Tools You Own: keep the expensive resource, in this case a costly model call, reserved for the cases that actually need it.
The grown-up version of this is not "pick one model for everything." It is routing requests to the cheapest tier that can handle them reliably, and escalating only when needed.
- Ervandra Halim, AI adoption consultant
A worked cost comparison
Here is a rough illustration using representative volume for a mid-size support operation handling roughly 3,000 AI-assisted requests a month, mixing ticket classification with occasional complex complaint drafting:
| Approach | Monthly requests | Approx. monthly cost |
|---|---|---|
| All requests on frontier model | 3,000 | Highest, often 5-10x the tiered approach |
| All requests on small model | 3,000 | Cheapest, but quality risk on the ~10% complex cases |
| Tiered: small model classifies, ~85% stays small, ~15% escalates | 3,000 | Middle ground, close to small-model cost with frontier-level quality where it matters |
The exact multiplier depends on your provider and task mix, but the pattern holds consistently across every client I have measured this for: tiered routing captures most of the savings of going fully cheap, while avoiding the quality risk of using a small model on tasks it cannot reliably handle.
How Do You Get Started Without Overbuilding?
Getting started without overbuilding means skipping the sophisticated routing system on day one. Start manually: look at your last month of AI usage, bucket the requests by actual complexity, and estimate how many could have run on a cheaper tier without a quality drop. If the number is meaningful, usually it is 60 to 80 percent of volume for support and back-office tasks, build a simple classifier step, even a rule-based one, before reaching for anything more elaborate. This is the same incremental instinct we cover in Automating Repetitive Back Office Tasks: Where to Start: start where volume is highest and judgment required is lowest.
The takeaway
The small language models vs large debate is not really about which is "better." It is about matching cost to the actual difficulty of the task in front of you. Most of what businesses run through AI today is routine enough for a cheap model to handle well. Reserve the expensive tier for the genuinely hard cases, build a simple routing layer between them, and revisit the split every few months as your volume and task mix change. The businesses winning on AI cost this year are not using cheaper models everywhere. They are using the right model for each job.
