Every AI vendor pitch I sit through eventually gets to the same slide: "we'll fine-tune a model on your data." It sounds impressive. It is usually the wrong answer. The fine-tuning vs rag vs prompting decision has a clear default order, and most business owners are being sold the most expensive rung on the ladder when the cheapest one would solve their problem in a week.
I have built AI features for a multifinance company and advised retail owners in Tangerang on the same question, and the pattern repeats: someone hears "fine-tuning" at a conference and assumes it is the serious, grown-up option. It is not. It is the last resort, not the starting ambition.
Here is the ladder I actually use when a client asks me to add AI to their product or workflow.
Start with prompting, always
Prompting means writing better instructions to an existing model like GPT-4 or Claude, with no extra infrastructure. This is rung one because it costs almost nothing and you can test it in an afternoon.
Most "the AI gives wrong answers" complaints I get called in to fix turn out to be prompting problems, not model problems. The instructions are vague, there's no example output, no constraints on tone or format, no explicit list of what to refuse. Fixing that is a text edit, not an engineering project.
Prompting solves:
- Drafting replies, summaries, and reports in a consistent format
- Classifying or triaging text (support tickets, leads, complaints)
- Rewriting or translating content
- General reasoning tasks that don't depend on your private data
Cost: effectively zero beyond API usage, usually under Rp 500,000/month for a small business's volume. If this solves your problem, stop here. Do not let anyone upsell you past it.
Add RAG when the AI needs to know your documents
Retrieval-Augmented Generation (RAG) means the AI searches through your own documents (SOPs, product catalogs, contracts, past tickets) and pulls relevant chunks into the prompt before answering. You reach for RAG the moment the honest failure mode is "the AI doesn't know that," not "the AI phrased that badly."
A pharmacy chain I worked with wanted a chatbot that could answer staff questions about drug interaction policies buried in a 40-page SOP document. Prompting alone was useless: the model had never seen that SOP. RAG solved it by indexing the document and retrieving the right paragraph for each question.
RAG is right when:
- Answers must be grounded in your specific policies, catalog, or history
- The underlying documents change often (so you can't just paste them into a prompt forever)
- You need to cite the source ("per SOP section 4.2") for trust or compliance
Cost: a vector database, an indexing pipeline, and ongoing maintenance when documents update. Budget roughly Rp 15-40 million for a proper build depending on document volume and how messy the source material is. This is the rung where most legitimate business AI projects should land. If you're deciding between this and building a custom pipeline from scratch, the comparison in Off-the-Shelf AI vs Custom AI Workflows is worth reading before you commit budget.
Fine-tune only for style, scale, or latency reasons
Fine-tuning means retraining a model's weights on your own examples so its behavior shifts permanently. It is expensive, slow to iterate on, and the wrong tool for "the AI doesn't know our facts" problems, because RAG solves that more cheaply and updates instantly when your data changes. Fine-tuning fixes a narrower set of problems:
- Consistent voice at scale. You need thousands of outputs a day in an extremely specific tone or format that prompting can't reliably hold.
- Cost or latency at high volume. A smaller fine-tuned model can replace a larger general model for one narrow task, cutting per-call cost once you're running millions of requests.
- Structured output the model keeps getting wrong. Rare, but it happens with unusual formats or domain-specific classification schemes.
If none of those three apply to you, fine-tuning is solving a problem you don't have. I've seen teams spend Rp 100+ million fine-tuning a model to "know" their product catalog, when a same-week RAG setup would have done it for a fraction of the cost and updated automatically every time the catalog changed.
A simple decision table
| Symptom | Right tool |
|---|---|
| Output is badly worded, wrong tone, wrong format | Prompting |
| Output is factually wrong because AI lacks your data | RAG |
| Output needs a very specific, hard-to-describe style at massive scale | Fine-tuning |
| You're running one model call per request and it's slow/expensive at huge volume | Fine-tuning (smaller model) |
Nine times out of ten, the actual business need sits in the first two rows.
The takeaway
Don't start a fine-tuning vs rag vs prompting conversation by asking "which is more advanced." Start by naming the actual failure: bad wording, missing knowledge, or scale economics. That single question routes you to the right rung almost every time, and it will save you from paying for infrastructure you don't need. If you want a second opinion on where your specific use case lands before you spend on either RAG or fine-tuning, that's a conversation worth having with a partner who has built both.