zephyr-orpo-141b-A35b-v0.1

Zephyr-orpo-141b-A35b-v0.1 is an open-source Mixture of Experts (MoE) language model released by Hugging Face’s H4 team in collaboration with Argilla and KAIST. Built upon the Mixtral-8x22B-v0.1 base architecture, the model consists of 141 billion total parameters. While the model's name references "A35b," the developers noted that the model actually activates approximately 39 billion parameters per token.\n\nThe model was aligned using Odds Ratio Preference Optimization (ORPO), a novel method that streamlines the alignment process by integrating supervised fine-tuning and preference optimization into a single step. This efficiency allowed the model to be trained in approximately 1.3 hours using only 32 H100 GPUs. The training data consisted of the distilabel-capybara-dpo-7k-binarized dataset, a collection of high-quality, multi-turn synthetic preferences.\n\nZephyr-141B-A35B is optimized for assistant-like behavior and strong instruction-following capabilities. It demonstrates high performance on specialized benchmarks such as IFEval, which measures adherence to strict formatting and constraint requirements. The model is released under an Apache 2.0 license.