Magpie TTS Zeroshot

Magpie TTS Zeroshot is a neural text-to-speech model developed by NVIDIA designed to synthesize expressive speech matching the voice characteristics of a short reference audio recording. Operating as part of a generative speech synthesis pipeline, it converts text tokens and reference audio codes into target acoustic tokens, enabling multi-language voice cloning.

Architecture and Capabilities

The model relies on an encoder-decoder transformer architecture (similar to T5-TTS) combined with a neural audio codec model operating at a low temporal rate. It employs alignment techniques and discrete token prediction to ensure stable, low-hallucination speech generation. The architecture supports zero-shot voice adaptation by ingesting brief reference speech prompts, rendering speech outputs in formats like English-US and European-Spanish with designated male and female persona options.

Usage and Prompting

For effective zero-shot voice cloning, users typically supply a clean 16-bit mono reference audio prompt ranging between 3 to 10 seconds sampled at 22.05 kHz or higher. The model supports specialized SSML markup tags such as <phoneme> with IPA specifications and <say-as> rules for fine-tuning pronunciation and text normalization during inference.