Gemma 4 26B A4B is a multimodal Mixture-of-Experts (MoE) language model developed by Google DeepMind. Released as part of the Gemma 4 family, it is designed to balance high-tier reasoning capabilities with inference efficiency. The model utilizes an architecture with 25.2 billion total parameters, of which approximately 3.8 billion are active during any single inference step, allowing it to provide performance comparable to much larger dense models while maintaining significantly lower computational overhead.
Architecture and Capabilities
The model features a hybrid attention mechanism that interleaves local sliding window attention with global full-context attention layers. It supports a context window of 256,000 tokens, enabling the processing of extensive codebases, long-form documents, and complex agentic workflows. Gemma 4 26B A4B is natively multimodal, capable of processing text, images, and video (up to 60 seconds at 1fps). It also introduces native support for system prompts, improving steerability and instruction following compared to previous generations.
Reasoning and Thinking Mode
A core feature of the model is its specialized Thinking mode, which allows the model to perform step-by-step internal reasoning before generating a final response. This reasoning process is exposed via a dedicated "thinking" block or reasoning tokens, which can be preserved across conversation turns to maintain logical consistency in complex problem-solving. The model excels in mathematical reasoning, coding, and logical deduction, frequently outperforming larger dense models on benchmarks like AIME and GPQA.
To utilize the reasoning capabilities effectively, users can enable the thinking parameter in supported environments. For structured tasks, the model supports native function calling and can be configured to produce outputs in specific formats like JSON without losing the internal reasoning chain. It is released under an Apache 2.0 license, making it suitable for both research and commercial applications.