A well-calibrated model that says it is ninety per cent sure is right about nine times in ten. Standard cross-entropy pretraining produces roughly this property, which is generally desirable.
It also guarantees errors. A calibrated model assigning ninety per cent to the right answer will emit a wrong one roughly one time in ten — which is why hallucination cannot simply be trained away without breaking calibration.