Speech to Text

Half of Bright Books' complaints arrive by phone, and each call disappears the moment that it ends. If we could turn those calls into text, every skill that Scribe has learned would work on them too.

• Half of complaints come by phone
• Calls disappear when they end
• Speech to text unlocks them

How Whisper works

Whisper, released by OpenAI in twenty twenty two, is a transformer for sound. It turns audio into a picture of frequencies, called a spectrogram. An encoder reads that picture, and a decoder writes the words, just like translation. Except that the source language is sound. It runs well on a laptop, and supports nearly a hundred languages.

Transcribing audio

Faster Whisper is an efficient version. We create a one second tone as a test signal, so that we don't need any file. We load the tiny model, and run transcribe. Run it. It tells us the length, and the language. A plain tone has no speech in it, so it finds little or nothing, which is a healthy result. With a real call recording, you'd get timed text. Every voiceover in this course was aligned with this very tool.

import numpy as np
from faster_whisper import WhisperModel

sr = 16000
t = np.arange(sr) / sr
audio = (0.2 * np.sin(2 * np.pi * 220 * t)).astype(np.float32)

model = WhisperModel("tiny.en", device="cpu", compute_type="int8")
segments, info = model.transcribe(audio, vad_filter=False)

print("Audio seconds:", round(info.duration, 1))
print("Language:", info.language)
print("Segments found:", len(list(segments)))

Audio seconds: 1.0
Language: en
Segments found: 0

Summary