Parameter

A single learned number inside a model. Collectively, what the model knows.

Also known as weight

Training adjusts parameters so the model predicts text better. Their values encode everything the model has learned, diffused across the network rather than stored in retrievable records.

Parameter counts were once the headline figure. They are far less useful now: sparse architectures mean only a fraction activate per token, and no major closed lab publishes counts at all.