VibeVoice 1.5B is an open-source speech synthesis model developed by Microsoft Research, designed specifically for expressive, long-form, and multi-speaker conversational audio. It is optimized for generating continuous speech for durations up to 90 minutes, making it suitable for applications such as podcast production, audio dramas, and scripted dialogues where maintaining speaker consistency and natural turn-taking is critical.
Architecture and Innovation
The model utilizes a "next-token diffusion" framework that integrates a large language model (LLM) backbone with a lightweight diffusion head. Specifically, this version employs Qwen2.5-1.5B to process text, speaker information, and dialogue context. A key innovation of the framework is its use of dual continuous speech tokenizers—one acoustic and one semantic—operating at an ultra-low frame rate of 7.5 Hz. This approach enables the model to achieve high audio fidelity while maintaining the computational efficiency required for processing long sequences.
Key Capabilities
VibeVoice 1.5B supports up to four distinct speakers within a single generation session, providing natural prosody and transition between voices. The system is trained using a curriculum learning strategy that progressively increases context length up to 64K tokens, allowing it to handle complex, extended interactions. For safety and transparency, the model incorporates an audible disclaimer and an imperceptible watermark in its audio outputs to mitigate risks associated with impersonation or disinformation.