Season 5: The AI Proposal · Episode 4 of 8
The Caption Heard Around the Office
You play: AI Engineering Intern
Included with Core programme and Certification prep
The situation
Jax has a livestream going out tonight: a creator interviewing a guest who moves between English and isiZulu, in a room with background music and occasional crosstalk. It needs live captions and an accessible summary. The first automated pass has already produced some captions that are wrong in ways that matter — a name mangled, an isiZulu greeting forced into English gibberish, a noisy segment transcribed as confident nonsense.
What you'll learn
- AI-901 language and speech implementation: speech-to-text (transcription) and its failure modes, language detection, translation, sentiment and entity analysis, confidence scores, and deciding when an automated caption is safe to publish versus needs a human.
- Choosing between key phrase extraction, entity and PII recognition, sentiment analysis, translation, speech-to-text and text-to-speech, and knowing that a multimodal model can accept audio directly.
Who you work with
Jax Rivera
Creator Partnerships Lead
Amara Okafor
ML and Data Engineer
Scenes
- 1.Tonight's stream
- 2.How captions fail
- 3.The caption flow
- 4.Which language or speech feature?
- 5.What ships tonight
- 6.Text and speech check
What you leave with
The caption flow transcribes each segment into a structured record with language, text and a needs_review flag driven by confidence. Tonight's policy auto-publishes the two safe segments (with an English translation alongside the isiZulu, original kept), holds the 0.51 crosstalk segment for a human, flags the guest's name for a check, and tells viewers the captions are auto-generated with a way to report errors.