openchat-3.5

OpenChat 3.5 is an open-source large language model fine-tuned from the Mistral-7B architecture. It is primarily distinguished by its use of Conditioned Reinforcement Learning from Fine-Tuning (C-RLFT), a training strategy inspired by offline reinforcement learning. This methodology enables the model to learn from mixed-quality datasets—treating different data sources as distinct reward signals—which removes the requirement for expensive human-annotated preference labels during alignment.\n\nDespite its compact size of 7 billion parameters, OpenChat 3.5 achieved performance parity with or outperformed larger models like ChatGPT (March 2023) on various benchmarks such as MT-Bench and MMLU at the time of its release. The model is optimized for high-throughput deployment and supports diverse use cases including mathematical reasoning, coding, and general conversation. It is released under the Apache 2.0 license, making it available for both research and commercial use.