VibeVoice 7B is a text-to-speech (TTS) model developed by Microsoft Research, designed for generating expressive, long-form conversational audio. It is part of a family of models—including a 1.5B parameter variant—optimized for creating multi-speaker content such as podcasts and audiobooks. The model is engineered to synthesize continuous speech sequences for extended durations while maintaining speaker consistency and natural turn-taking dynamics between up to four distinct voices.
The model's architecture utilizes a next-token diffusion framework. It leverages a Qwen2.5 backbone to interpret textual context and dialogue structure, paired with a diffusion-based decoding head for high-fidelity acoustic generation. A key technical innovation is its use of specialized acoustic and semantic tokenizers that operate at an ultra-low frame rate of 7.5 Hz. This system achieves high audio compression (up to 3200x) while preserving vocal nuances, enabling the processing of long sequences within a large context window.
VibeVoice 7B features zero-shot voice cloning, which allows for the synthesis of personalized voices using reference audio samples as short as 10 seconds. It supports English and Chinese natively and demonstrates cross-lingual capabilities, ensuring speaker identity remains stable when switching between languages. The model is designed to capture natural prosody and emergent behaviors like spontaneous singing, addressing traditional TTS limitations in handling the rhythm and flow of real-world dialogue.