OpenAudio S1 Mini is an open-source multimodal speech model developed by Fish Audio. As a distilled version of the flagship S1 model, it is designed for efficient audio synthesis and low-latency interaction. The model utilizes a native multimodal transformer architecture—specifically a dual autoregressive (Dual-AR) design—that integrates semantic and acoustic information into a single framework. This approach allows the model to understand and generate speech directly, maintaining high prosodic consistency without the need for traditional intermediate text-to-phoneme pipelines.
The model was trained on a large-scale dataset comprising over 2 million hours of multilingual audio data, supporting more than 13 languages including English, Chinese, Japanese, German, and Spanish. Its core capabilities include zero-shot and few-shot voice cloning, enabling high-fidelity voice generation from audio samples as short as 10 to 30 seconds. It is further optimized with Reinforcement Learning from Human Feedback (RLHF) to capture subtle vocal nuances and intonations, providing realistic speech quality that rivals professional voice synthesis.
OpenAudio S1 Mini features advanced emotional and tone control, supporting a wide range of specific markers for expressions such as excitement, sadness, and whispering. It can also generate non-verbal vocalizations, including laughter, crying, and sighing. The model is optimized for performance on consumer-grade hardware, making it suitable for real-time conversational applications and edge deployment while preserving the core reasoning and instruction-following capabilities of the larger S1 series.