RAG, fine-tuning, or prompting: which one do you actually need?

There are three ways to make an AI do what you want with your information: put it in the prompt, retrieve it at answer time (RAG), or bake it into the model (fine-tuning). They are not competing options, they are a ladder. You climb it only as far as you need.

The ladder, cheapest first

• Prompting. Give the model the instructions and context in the prompt itself. Fastest, cheapest, and enough for a surprising amount. Start here, always.
• RAG. When the knowledge is too large for a prompt or changes often, retrieve the right pieces at answer time. This is the right answer for ‘answer from our documents’, and it updates the moment the documents do.
• Fine-tuning. Train the model on your examples so it learns a style, format, or task it cannot be told. Powerful, but slow to build, expensive to maintain, and stale the day your data changes.

When fine-tuning is actually worth it

Fine-tuning earns its cost when you need consistent behaviour that a prompt cannot reliably produce: a specific tone at scale, a rigid output format, a narrow classification task with lots of examples. It is the wrong tool for teaching the model facts, that is what RAG is for, and facts change while a fine-tune is frozen at training time.

Why teams skip to the expensive rung

Fine-tuning sounds like the ‘real’ AI work, so teams jump to it and spend weeks training a model to do what a prompt plus retrieval would have done in an afternoon. We start at the bottom of the ladder on purpose: most problems are solved two rungs below where people expect, and the money is better spent on evals and retrieval quality than on a fine-tune you will have to redo.