Yi-34B-Chat is a bilingual large language model developed by 01.AI, the Beijing-based startup founded by Dr. Kai-Fu Lee. It is an instruction-tuned version of the Yi-34B base model, optimized specifically for conversational interactions and following complex human instructions in both English and Chinese.
The model is built on a decoder-only Transformer architecture and was trained from scratch on a massive dataset of 3 trillion tokens. While it shares architectural similarities with the Llama family, enabling compatibility with various existing tools and libraries, its weights and training data were independently developed. The chat variant underwent supervised fine-tuning (SFT) on curated alignment data to enhance its helpfulness, reasoning capabilities, and safety.
Capabilities and Performance
At its launch, Yi-34B-Chat demonstrated strong performance across several industry benchmarks, achieving high rankings on the AlpacaEval Leaderboard and performing competitively against models with much larger parameter counts. It is particularly noted for its emergent abilities in common-sense reasoning, logical deduction, and multilingual reading comprehension. The model supports a base context length of 4,000 tokens, which can be extended during inference to accommodate longer conversational histories.