There are three ways to make a model do what you want, and they are routinely deployed against the wrong problems. The expensive version of this mistake is fine-tuning a model to teach it facts. It mostly does not work, and retrieval would usually have solved the problem in an afternoon.
One question sorts it. Is your problem knowledge, behaviour, or instruction?
Prompting: instruction
Changing what you say to the model. Free, instant, reversible, and where every project should start.
Prompting is unreasonably effective and chronically undersold, because it sounds too simple to be the answer. A carefully specified system prompt with clear constraints, a handful of worked examples, and explicit instructions on edge cases resolves a large share of problems teams attempt to solve with training.
Few-shot examples deserve specific mention, because they are the highest-leverage prompting technique and the most underused. Showing three correct input-output pairs conveys more about desired format and tone than several paragraphs of description. Vendor research on tool use found that supplying examples of correctly-formed calls moved accuracy on complex parameters from roughly seventy per cent to ninety.
The real limits: prompts consume context on every call, very long prompts become unwieldy to maintain, and prompting cannot supply information the model has no access to.
Retrieval: knowledge
When the model needs information it does not have — your internal documents, current data, anything that changes — the answer is to fetch it and include it at query time. This is retrieval-augmented generation.
Retrieval is the correct tool for every knowledge problem, and the reasons it beats training for this purpose are worth being explicit about:
- Updates are instant. Change the document; the next answer reflects it. No retraining.
- Answers are attributable. You can show which source produced a claim. Fine-tuned knowledge is diffused into weights and cannot be cited.
- Access control works. You can filter retrieval per user. You cannot unlearn a fact for one user of a fine-tuned model.
- Removal is possible. Deleting a document removes it. Deleting a fact from trained weights is an unsolved research problem.
Fine-tuning: behaviour
Continuing to train a model on your own examples, adjusting its weights. This genuinely changes the model — and what it changes is how the model behaves, not what it knows.
Fine-tuning earns its keep on a narrow but real set of problems: enforcing an output format or house style too intricate to specify in a prompt; a specialised classification task where you have thousands of labelled examples; domain conventions and terminology that resist description; and cost reduction, where a small fine-tuned model matches a large general one on a narrow task at a fraction of the price. That last case is the most commercially underrated.
Why fine-tuning fails at knowledge is worth understanding rather than taking on faith. Training adjusts weights across an enormous parameter space; a fact appearing in a few dozen examples produces a faint, diffuse signal, not a retrievable record. The model typically learns the form of your documents convincingly while remaining unreliable about their content — which is the worst possible outcome, because it sounds authoritative and is wrong. Teams discover this after the training bill.
Practical costs: you need a decent volume of consistent, high-quality examples, and data preparation is most of the work. Each new base model means redoing it. And fine-tuning can degrade general capability in ways your evaluation set will not catch.
The decision, compressed
- Model lacks information → retrieval
- Model has the information but handles it wrongly → prompting, then better retrieval
- Output format or style is consistently wrong after serious prompting effort → fine-tuning
- Quality is acceptable but cost or latency is not → fine-tune a smaller model
- Model needs to act, not just answer → tool use
These are complements, not rivals. The common production shape is a well-engineered prompt plus hybrid retrieval on a strong general model, with fine-tuning reserved for specific narrow components. The ordering matters though: exhaust prompting, then build retrieval, and only then consider training. Most teams that skip to step three end up back at step two having spent considerably more.
Last reviewed: September 2026.
