Voice cloning uses a text-to-speech model that has learned the sound of a particular person’s voice, its pitch, accent and rhythm, so it can read out any text in that voice. Some research systems need very little audio: Microsoft’s VALL-E paper (2023) reported results from a 3-second recording of a speaker the model had never heard.
It has real uses, such as giving a voice back to people who are losing theirs, or dubbing videos. It is also used in scams that impersonate relatives, bosses or public figures. In the US, the FCC ruled in February 2024 that robocall rules on artificial voices cover AI voice cloning.