Research

Storing and computing model weights at lower numerical precision to cut memory and bandwidth.

Whether a model’s stated confidence matches how often it is actually right.

A model stating something false with confidence: a predictable consequence of how models are trained and graded.

Working through intermediate steps before answering, which measurably improves accuracy on multi-step problems.

Spending more computation at the moment a question is asked, rather than only during training.

The mechanism letting each token weigh how much every other token matters to it.

An architecture where only a fraction of the network activates for any given token.

The neural network architecture underneath essentially every modern AI model, introduced in 2017.

Splitting text into tokens using a vocabulary learned from a training corpus.

A list of numbers representing a piece of text, positioned so that similar meanings land near each other.