Ting Comparisons
RAG vs fine-tuning vs prompt engineering: which one fixes your AI's wrong answers?
The decision, in short
Find out why the answers were wrong before you choose. Missing or out-of-date information is fixed by giving the model the right source when the question is asked: retrieval for documents, a live tool for prices and stock. Wrong format, tone or reasoning is fixed with instructions and examples first, and with fine-tuning only after that.
Reviewed by Abdullah Al-RashedUpdated
A wrong answer. Why?
The facts were missing or out of date
Retrieval (RAG) for documents, a live tool for prices and stock
The format, tone or reasoning was wrong
Instructions and examples first
Still inconsistent after good examples?
Fine-tuning, tested on the same set
Trace ten real wrong answers before you choose.
How do the three compare, criterion by criterion?
Before reading the table, take ten real wrong answers and write next to each one the right answer and where it lives today. Most of the decision is in that list.
| Criterion | Prompt and examples | Retrieval (RAG) or a live tool | Fine-tuning |
|---|---|---|---|
| What it fixesHow to checkFor each wrong answer: was the right information missing, out of date, or there but used badly? | Instructions, tone, language and format the model was never told. | Facts the model doesn't have or has out of date: policies, product details, prices, stock. | Behaviour that stays inconsistent after good instructions and examples: a format, a tone, a fixed way of working. |
| Where the fix livesHow to checkAsk where you will change it next month, when the policy or the price changes. | In the instructions sent with every request. | In your documents and systems, read at the moment of the question. | Inside the model, fixed at the time it was trained. |
| Keeping it currentHow to checkChange one policy line and see how long the old answer survives. | EdgeEdit the instructions; the next request uses them. | EdgeUpdate the document or the system of record; the next answer can use it. | Retrain on new examples; until then the old behaviour stays. |
| What you need firstHow to checkAsk for the test set before anything is built. | EdgeA set of real questions with the expected answers. | Clean, current sources with access rules, and a log of what was retrieved for each answer. | Enough good examples of the exact output you want, and the same test set to prove the training helped. |
| How it fails[2]How to checkAsk for three failure cases from a vendor's own testing. | Long or conflicting instructions; it still invents facts it was never given. | It retrieves the wrong passage, an old version, or one the user shouldn't see. | It learns the mistakes in its examples, and it can still state false content with confidence. |
| How you test itHow to checkRe-run the same question set after every change. | Compare answers against the expected ones after each change to the instructions. | Check what was retrieved for each answer, not only the answer itself. | Compare against the prompt-and-retrieval result on the same test set. |
| Cost driversHow to checkEstimate the cost of one change, not only the first build. | Longer instructions add to every request. | Indexing, search infrastructure and keeping the sources clean. | Preparing examples, training runs, and retraining whenever the behaviour must change. |
| Best when | The model was never told the format, tone or rules | The facts are missing or out of date | Behaviour stays inconsistent after good instructions and examples |
An edge belongs to that row alone. It does not choose for you; the Best when row does.
Which should you choose, by situation?
Start with prompt and examples when
- You haven't yet written down what a correct answer looks like.
- The problem is tone, length, language or format.
- The facts the answer needs are already in the request.
Add retrieval or a live tool when
- The right answer is in a document, a policy or a system the model can't see.
- The information changes: prices, stock, policies, account status.
- People need to see where an answer came from.
Consider fine-tuning when
- Instructions and examples are tuned and tested, and the behaviour is still inconsistent.
- You have enough real examples of the exact output you want.
- The task is stable, so the training won't go out of date quickly.
OpenAI's guide to optimizing LLM accuracy treats these as levers for two different problems: give the model the right context when it lacks knowledge, has it out of date or needs proprietary information, and optimize the model itself when its format, tone or reasoning is inconsistent.[1]
What should you ask any vendor who proposes one of them?
- Which of our wrong answers did you trace, and what caused each one?
- Where does the price, policy or stock figure come from at the moment the question is asked?
- If you propose fine-tuning, what did instructions and retrieval achieve on the same test set first?
- How will the system keep up when our documents change next month?
- Can we see, in the logs, what was retrieved for each answer?
- What does it cost to retrain, and how often would we need to?
Questions people ask
Does fine-tuning teach a model our latest prices?
Is RAG the same as training the model on our documents?
Can good instructions alone be enough?
Where Ting fits, and when you don't need Ting
Where Ting fits
- Targeted AI Advisory answers this one decision in writing: we trace a sample of your wrong answers to their cause and recommend the smallest fix that works.
- If the fix needs building, it is specified in an AI Solution Blueprint, with the same test set as its acceptance criteria.
- See the Targeted AI Advisory
When you don't need Ting
- The wrong answers come from a document that is out of date. Update the document; no AI change is needed.
- Your team can trace ten real wrong answers to their cause, and fix the cause itself.
Sources
Primary sources, checked on the date shown.
- Optimizing LLM Accuracy (API documentation)OpenAI · checked
- NIST AI 600-1: Generative AI Profile (July 2024)NIST · checked
