K2 Horizon 3.7B

K2 Horizon 3.7B is a dense language model developed by the Institute of Foundation Models (IFM) at the Mohamed bin Zayed University of Artificial Intelligence (MBZUAI). Released in September 2026 as part of the K2 Horizon model family, it is designed to bridge the gap between compact on-device models and large-scale enterprise systems. The model is notable for being released as part of what the creators describe as the largest "fully open" model launch, providing not only the final weights but also training code, data recipes, and intermediate checkpoints.

The model features a decoder-only architecture with 3.7 billion core parameters (totaling approximately 5.06B including embeddings). It utilizes Grouped Query Attention (GQA) with 32 attention heads and 8 KV heads across 36 layers. A key technical highlight is its native 512K context window (specifically 524,288 tokens), which was integrated from the mid-training stages to support long-horizon reasoning and extensive document processing on consumer-grade hardware.

Training for K2 Horizon 3.7B involved approximately 20 trillion tokens, with a significant focus on reasoning capabilities. Nearly 17% of the pre-training corpus consisted of explicit reasoning trajectories, and approximately half of the total training data was high-quality synthetic data generated through IFM's internal pipelines. The post-training process included supervised fine-tuning (SFT) and reinforcement learning, with a specific emphasis on coding and agentic tool-calling tasks.

Performance benchmarks place K2 Horizon 3.7B as a competitive option for single-node fine-tuning and development. It is particularly optimized for coding assistance and complex problem-solving, achieving high scores on benchmarks such as SWE-bench and Terminal-Bench 2.1. The model is released under the Apache 2.0 license, facilitating both academic research and commercial adaptation without restrictive usage terms.