Speech and Voice

Two kinds of speech technology matter. Speech to text, which listens, and text to speech, which talks. Both have become astonishingly good. In this lesson, you will learn how each works, what voice cloning is, and how to use voices responsibly.

• Listening: speech to text
• Talking: text to speech
• Voice cloning, and responsibility

Speech to text: how AI listens

To listen, the sound is first converted into a picture, called a spectrogram, which shows how the pitch and loudness change over time. A transformer then reads that picture, and predicts the words, much like predicting the next token. Models such as Whisper can transcribe many languages, handle accents, and even work through background noise. It powers subtitles, meeting notes, and voice assistants.

1. Sound wavea recording of the voice
2. Spectrograma picture of the sound
3. Transformerreads the picture
4. Textthe words, with timing

Text to speech: how AI talks

To talk, the process runs the other way. The text is turned into tokens, and a model predicts how the speech should sound, its rhythm, pitch, and emotion. A second model, called a vocoder, then turns that description into an actual audio wave. Modern voices breathe, pause, and change tone, so naturally that many listeners cannot tell them from a human.

1. Textwhat to say
2. Speech modelpredicts rhythm, pitch, emotion
3. Vocoderbuilds the audio wave
4. Natural voicewith pauses and tone

Voice cloning: power and risk

Some systems can clone a voice, from a short sample. Used well, it is wonderful. A creator can narrate in their own voice, without recording every word. A person who has lost their voice to illness, can speak again. But the same technology can be abused, to imitate someone without permission, for scams, or misinformation. The rule is simple. Only clone a voice with clear consent, and be honest that it is synthetic.

Summary