Training a model and running it are different jobs. An inference chip is built for the second: serving answers quickly and cheaply, often with low-precision maths and lots of fast memory, rather than the huge all-purpose computing needed for training.
Big cloud companies design their own to cut cost per answer. Google described its first TPU, an inference chip in use in its data centres since 2015, in a 2017 paper; this week Microsoft said its Maia family will become the primary inference system for its products.