qwen3-next-80b-a3b-instruct

Qwen3-Next-80B-A3B-Instruct is a large language model developed by Alibaba's Qwen team, designed for high efficiency and long-context understanding. It utilizes a high-sparsity Mixture-of-Experts (MoE) architecture containing 80 billion total parameters, of which only 3 billion are activated during inference. This structure allows the model to match the performance of significantly larger dense models while maintaining the inference speed and computational footprint of a much smaller system.

A defining technical feature of the model is its hybrid attention mechanism, which combines standard Gated Attention with Gated DeltaNet (a form of linear attention). This architecture is specifically optimized for ultra-long context processing, natively supporting a 256,000-token window that can be extended up to 1 million tokens. The model also integrates Multi-Token Prediction (MTP) to enhance training stability and accelerate inference throughput.

Trained on a massive corpus of 15 trillion tokens, the model is refined through post-training to excel in instruction following and complex reasoning. According to Alibaba, the architecture achieves up to 10x higher throughput than equivalent dense models in long-context scenarios. The model is released under the Apache 2.0 license, supporting both research and commercial applications.