Human raters compare model outputs; those preferences train a reward model, which then guides reinforcement learning on the base model. Anthropic’s published Constitutional AI is a variant where the model evaluates its own outputs against written principles to generate that preference data.
RLHF is what turns a text continuation engine into something that follows instructions and declines requests. It is also where much of a model’s characteristic tone comes from.