Kimi K2 is a large-scale Mixture-of-Experts (MoE) language model developed by Moonshot AI. It features a total of 1 trillion parameters, with 32 billion active parameters per forward pass, a design that aims to balance high-capacity knowledge with computational efficiency during inference. The model is part of a series optimized for "agentic intelligence," focusing on autonomous tool use, complex software engineering, and multi-step reasoning.
Architecture and Training
The model was trained on approximately 15.5 trillion tokens using the Muon optimizer, which Moonshot AI implemented to maintain stability while scaling sparse MoE architectures. Kimi K2 supports a context window of 128,000 tokens and utilizes Multi-head Latent Attention (MLA) to manage long-context performance. It is designed to handle long-horizon tasks, such as executing hundreds of sequential tool calls without losing track of user objectives.
Capabilities and Variants
Kimi K2 is available in multiple variants, including Kimi-K2-Base for fine-tuning and Kimi-K2-Instruct, which is tailored for chat and agentic workflows. It has demonstrated strong performance in technical benchmarks, particularly in coding (LiveCodeBench and SWE-bench) and expert-level reasoning (GPQA). The model serves as the foundation for subsequent specialized versions, such as the Kimi K2 Thinking and Kimi K2.5 models.