RAG

Searching a corpus for relevant material and pasting it into the prompt so the model can answer from it.

Also known as retrieval-augmented generation

A model’s knowledge is frozen at its training cutoff and contains nothing private to your organisation. RAG closes both gaps by fetching relevant fragments at query time rather than relying on what the model memorised.

Large context windows were widely predicted to kill it. They did not: a million tokens is large against a document and trivial against a corpus, and stuffing the window is expensive and less reliable than retrieving well.