Pretraining

The first and largest training phase, where a model learns from raw text by predicting what comes next.

Almost all knowledge and capability is acquired here, and almost all of the compute is spent here. The result is a model that can continue text but has no particular inclination to be helpful or conversational.

Everything that makes a model feel like an assistant is added afterwards, in a much smaller and more deliberate phase.